Test Infrastructure

Table of Contents

Overview

The test stack combines BDD narrative, browser automation, an isolated in-memory database, and an in-memory filesystem.

Component Role
Storyteller BDD narrative — colored console output or file-based story reports
WebDriver ChromeDP browser automation + WebServer lifecycle management
WebDriverAsserter Browser-aware assertions (wait, then assert, screenshot on failure)
Asserter Low-level value assertions for non-browser checks
GrownStater Seeds the in-memory database with a realistic ecosystem
WebServer net/http + Gin router wrapper, managed by WebDriver
SupFilesystem Thin OS wrapper; backed by an in-memory mock during tests
Logger Structured contextual logging shared by all components

All components are injected via a single call to helpers.SetupTest.

Test Struct & Bootstrapping

type MyTest struct {
    Storyteller       *storyteller.Storyteller
    Asserter          *asserter.Asserter
    WebDriver         *web_driver.WebDriver
    WebServer         *web_server.WebServer
    GrownStater       *staters.GrownStater
    WebDriverAsserter *asserter.WebDriverAsserter
    SupFilesystem     *support.SupFilesystem
    Logger            *logger.Logger
}
helpers.SetupTest(this, testing)
ctx := helpers.SetupTest(this, testing)
  • Creates an isolation scene: in-memory SQLite, memory filesystem, mock exec/HTTP client, deterministic time.
  • Uses reflection to walk every exported field of this and resolves it from the DI Container/Canister by field name
  • Returns a context.Context that carries *testing.T
  • Rule: Every field in the test struct must be a pointer or interface. Struct-valued fields are skipped by the resolver.

Component Reference

Storyteller sup/lib/storyteller/storyteller.go

Storyteller`s sole purpose is to produce an easy-to-read test output.

If SUP_STORYTELLER_WRITER ENV variable is set to file, it writes the output in docs/stories/FILE_PATH.txt

Note that it does NOT perform any actual test action.

Method Description
SetAutoScenario(ctx context.Context) Derives the scenario name from the Go test function name. Call once at the top of every test.
SetScenario(scenario, description string) Manual override with an explicit description block.
As(actor string) Identifies the current actor, e.g. "User".
Given(step string) BDD Given step.
When(step string) BDD When step.
Then(step string) BDD Then step.
Also(step string) Continuation step.

Note: Consecutive calls of the same kind (Given, When, Then) are auto-prefixed in the output with “And”.

  • Rule: Call SetAutoScenario or SetScenario before any other narrative method.

Asserter sup/lib/asserter/asserter.go

Purpose: Low-level value assertions for non-browser tests.

All methods take ctx context.Context as their first argument.

On failure: writes tmp/last-test-fail.json, logs to the console with a caller link, and calls t.FailNow().

Method Description
IsTrue(ctx context.Context, bool) Asserts value is true.
IsFalse(ctx context.Context, bool) Asserts value is false.
WithoutError(ctx context.Context, err) Asserts err is nil.
WithError(ctx context.Context, err) Asserts err is non-nil.
EqualStrings(ctx context.Context, expected, actual) String equality.
NotEqualStrings(ctx context.Context, expected, actual string) String inequality.
EqualIntegers(ctx context.Context, expected, actual string) Integer equality.
StartsWithString(ctx context.Context, expected, actual string) Prefix check.
ContainsString(ctx context.Context, expected, actual string) Substring check.
NotContainsString(ctx context.Context, expected, actual string) Negative substring check.
CheckedCheckbox(ctx context.Context, label string, checked bool) Asserts checkbox state.
NotCheckedCheckbox(ctx context.Context, label string, checked bool) Asserts checkbox is not in state.
EqualStringReference(ctx context.Context, referenceName, actual string) Snapshot test: compares actual to tests/.../references/<name>.txt; creates the file on first run.
EqualStructureReference(ctx context.Context, referenceName, actual string) Same as above but marshals actual to JSON before comparing.
EqualReported(ctx context.Context, report*spy.Report) bool Compares spy call reports.
IsNotNil( context.Context, actual interface{}) Asserts value is not nil.

Port: read from the PORT environment variable; falls back to 8032.

WebDriver sup/lib/web_driver/web_driver.go

Purpose: Browser automation via ChromeDP. Also manages the WebServer lifecycle.

Note: Must variants don’t return errors but panic instead.

  • Session management
Method Description
StartSession(ctx context.Context) (ctx2 context.Context) Starts GinRouter + WebServer, waits up to 10 s for the server to be ready, then creates a ChromeDP context. Returns a new ctx that must replace the caller’s ctx.
StopSession(ctx context.Context) Shuts down the WebServer and ChromeDP context. Always call via defer.
  • Navigation & waiting
Method Description
SetWebDriveIntoContext(ctx context.Context) (context.Context, context.CancelFunc) Initializes and injects a ChromeDP browser instance into the context. Reads the CHROMEDP_HEADLESS env var to decide whether to run Chrome headless or with a
visible window.
WaitForServer(url string, timeout time.Duration) error Polls the given URL with HTTP GET requests every 50ms until the server responds successfully or the timeout expires.
Run(ctx context.Context, actions ...chromedp.Action) Thin wrapper around chromedp.Run. Guards against calling it before the driver is initialized (panics if not). All other browser interaction methods go through
this.
Navigate / MustNavigate (ctx, uri string) Navigates to http://localhost:<port><uri>.
WaitForText / MustWaitForText (ctx context.Context, text string) error Polls the page HTML for up to 5 s. Saves tmp/last_timeout_page.html on timeout.
WaitForTextOn / MustWaitForTextOn (ctx context.Context, text string, aria string) error Searches for text scoped to a specific ARIA role.
WaitForCurrentUrl / MustWaitForCurrentUrl (ctx context.Context, expectedUrl string) error Polls for an exact URL match for up to 5 s.
WaitForCurrentUrlStartsWith / MustWaitForCurrentUrlStartsWith (ctx context.Context, expectedUrl string) error Polls for URL prefix match for up to 5 s.
WaitForCurrentUrlContains / MustWaitForCurrentUrlContains (ctx context.Context, expectedUrl string) error Polls for URL substring match for up to 5 s.
WaitVisible(ctx context.Context, sel string) Waits up to 5 seconds for a DOM element matching selector to become visible.
WaitVisibleUntil(ctx context.Context, selector string) error Waits up to 5 seconds for a DOM element matching selector to become visible. Unlike WaitVisible (which panics), this
returns the error to the caller if the timeout is exceeded.
WaitReady(ctx context.Context, sel string) error Waits up to 5 seconds for a DOM element matching selector to become ready.
  • Interaction
Method Description
Click / MustClick (ctx context.Context, selector string) error
FillStringFormField / MustFillStringFormField (ctx context.Context, formFieldName string, value string) error Fills <input name="formFieldName"> using SendKeys.
ClickOnViaLabel / MustClickOnViaLabel (ctx context.Context, role string, label string) (err error) Generic ARIA role + label click.
ClickOnButtonViaLabel / MustClickOnButtonViaLabel ((ctx context.Context, label string) (err error) XPath finds an element with role=“button” and matching text, then clicks.
ClickOnCheckboxViaLabel(ctx context.Context, label string) (err error) Finds a checkbox; clicks only if not already checked.
CheckCheckboxViaLabel(ctx context.Context, label string) (err error) Marks the checkbox with matched label as checked.
SendKeys(ctx context.Context, selector string, v string) error Simulates keyboard input into a DOM element matched by selector. Delegates directly to chromedp.SendKeys.
Login(ctx context.Context) error Performs a full login flow.
AddCookie / MustAddCookie (ctx context.Context, name string, value string) error Sets a browser cookie with the given name/value scoped to localhost at path /.
  • Reading page state
Method Description
GetCurrentUrl(ctx context.Context) string Returns the current URL.
GetPageHTML(ctx context.Context) (html string) Returns the full page HTML.
GetOuterHTML(ctx context.Context, selector string) string Returns the outer HTML of the matched element.
GetInnerHTML(ctx context.Context, selector string) string Returns the inner HTML of the matched element.
IsCheckboxChecked(ctx context.Context, label string) (bool, error) Reads checkbox state via ARIA and returns a boolean.
  • Utilities
Method Description
TakeFullScreenshot(ctx context.Context) Saves a PNG to /tmp/test_screenshots/screenshot.png.
Login(ctx context.Context) Navigates to /auth/app-login?message=unauthorized and clicks the form button.
MakeUrl(uri string) string Prepends the base URL. Useful when building expected URL strings for assertions.
StayStill(ctx context.Context) Pauses for 10 minutes. Debugging only — do not use in CI.
TenMinutesPause(ctx context.Context) Alias for StayStill. Debugging only.
TenSecondPause(ctx context.Context) Pauses for 10 seconds. Debugging only.
OneSecondPause(ctx context.Context) Pauses for one second.

Rule: The session must be started before any other WebDriver call. The driver panics if Data.Initialized == false.

GrownStater sup/srv/staters/grown_stater.go

Purpose: Populates the in-memory database with a realistic application ecosystem before the browser session starts.

Method Description
GoToDefaultStage(ctx context.Context) Calls ReadyStater.GoToState (schema init), then creates: Atc, Devt, Gen-x, Agency, and Agency pre-prod apps, each with their Runtimes, Sources, Transponders, Servers, Resources, and Backups.
GoToStage(ctx context.Context, name string) Currently delegates to the same default logic; will allow stage variants in the future.

Rule: Call GrownStater.GoToDefaultStage before WebDriver.StartSession so that data exists when the HTTP server begins serving requests.

WebDriverAsserter sup/lib/asserter/web_driver_asserter.go

Purpose: Browser-aware assertions — waits first, then asserts; takes a screenshot on failure.

Method Description
AssertCurrentUrl(ctx context.Context, expectedUri string) Waits up to 5 s for an exact URL match; reports failure via Asserter.
AssertCurrentUrlStartsWith(ctx context.Context, prefix string) Waits up to 5 s for a URL prefix match.
AssertCurrentUrlContains(ctx context.Context, substr string) Waits up to 5 s for a URL substring match.
AssertText(ctx context.Context, text string) Waits up to 5 s for text to appear anywhere in the page HTML; takes a screenshot on failure.
AssertTextOn(ctx context.Context, text, ariaRole string) Waits up to 3 s for an XPath match on role=ariaRole and contains(text(), …); takes a screenshot on failure.
AssertCheckboxIsChecked(ctx context.Context, label string) Queries checkbox state; takes a screenshot on failure.

ARIA role constants are in the gitlab.com/blue-lila/kitcla/aria package (e.g. aria.AriaRoleButton, aria.AriaRoleHeading).

Failure reporting delegates to Asserter.MakeReport, so the same JSON report file and t.FailNow() flow applies as with the plain Asserter.

SupFilesystem sup/lib/support/sup_filesystem.go

Purpose: Thin wrapper around OperatingSystemIpl. During isolated tests this is backed by an in-memory mock, so filesystem operations never touch the real disk.

  • Path helpers
Method Description
TestDir() string Returns the directory of the calling _test.go file by walking the call stack.
  • File operations
Method Description
ReadFile / MustReadFile Read file contents.
WriteFile / MustWriteFile Write file contents.
MkdirAll / MustMkdirAll Create directory tree.
Remove / MustRemove Remove a file or empty directory.
RemoveAll / MustRemoveAll Remove a path and all its contents.
Stat / MustStat Get file info.
ReadDir / MustReadDir List directory contents.
Walk / MustWalk Walk a directory tree.
Rename / MustRename Rename or move a file.
Chmod / MustChmod Change file permissions.
IsNotExist(ctx, err) bool Tests whether an error represents “file not found”.
  • Environment operations
Method Description
Getenv(key string) string Read an environment variable.
Setenv / MustSetenv Set an environment variable.
LookupEnv(key string) (string, bool) Look up an environment variable.

Rule: Prefer Must* variants in test bodies to fail fast on unexpected OS errors.

Logger src/lib/logger/logger.go

Purpose: Structured, contextual logging shared by all components.

Levels: Debug, Info, Warn, Error.

Note: It is rare to call Logger directly in tests: components use it internally. It is useful when tracing unexpected behavior: this.Logger.Info(ctx, "debug marker")

WebServer src/lib/web_server/web_server.go

Purpose: Wraps net/http.Server together with the Gin router.

Note: Test authors do not call WebServer directly — its lifecycle is managed by WebDriver.StartSession and WebDriver.StopSession.

How They Work Together

Execution flow for a typical acceptance test

  • Key coupling points

StartSession returns a new context — always reassign: ctx = this.WebDriver.StartSession(ctx)

GrownStater must run before StartSession so the HTTP server serves real data on the very first request.

WebDriverAsserter delegates to Asserter — failure produces the same JSON report file and t.FailNow() call as a plain Asserter failure.

Annotated Skeleton Test

A minimal working acceptance test showing the correct call order:

package my_feature

import (
    "__PROJECT_NAME__/src/lib/logger"
    "__PROJECT_NAME__/src/lib/web_server"
    "__PROJECT_NAME__/sup/helpers"
    "__PROJECT_NAME__/sup/lib/asserter"
    "__PROJECT_NAME__/sup/lib/storyteller"
    "__PROJECT_NAME__/sup/lib/support"
    "__PROJECT_NAME__/sup/lib/web_driver"
    "__PROJECT_NAME__/sup/srv/staters"
    "testing"

    "gitlab.com/blue-lila/kitcla/aria"
)

type MyFeatureTest struct {
    Storyteller       *storyteller.Storyteller
    Asserter          *asserter.Asserter
    WebDriver         *web_driver.WebDriver
    WebServer         *web_server.WebServer
    GrownStater       *staters.GrownStater
    WebDriverAsserter *asserter.WebDriverAsserter
    SupFilesystem     *support.SupFilesystem
    Logger            *logger.Logger
}

func TestMyFeature(testing *testing.T) {
    this := &MyFeatureTest{}
    ctx := helpers.SetupTest(this, testing)
    this.Storyteller.SetAutoScenario(ctx)

    this.Storyteller.As("User")
    this.Storyteller.Given("a grown state")
    this.GrownStater.GoToDefaultStage(ctx)

    this.Storyteller.Given("a web session")
    ctx = this.WebDriver.StartSession(ctx)
    defer func() { this.WebDriver.StopSession(ctx) }()

    this.Storyteller.When("I navigate to the home page")
    this.WebDriver.MustNavigate(ctx, "/app")
    this.WebDriver.MustWaitForText(ctx, "Home")

    this.Storyteller.Then("I see the Home header")
    this.WebDriverAsserter.AssertText(ctx, "Home")
    this.WebDriverAsserter.AssertTextOn(ctx, "Home", aria.AriaRoleHeading)
}