Fetcher

Source: src/lib/fetcher/fetcher.go

Meet the Fetcher — the foundation for all data retrieval services across the application. It provides dynamic SQL SELECT query building with type-safe filtering, sorting, and pagination. Rather than hand-writing SQL strings in every service, callers build a FetcherMod by registering filters and sort orders, then pass it to BuildSelect to receive a complete, parameterised SQL statement.

Table of Contents

• Fetcher

  • Core Types

  • Filter Types

  • Methods

    • Filter Building (FetcherMod)
    • Raw Filter Conversion (FetcherMod)
    • Sorting (FetcherMod)
    • Debugging (FetcherMod)
    • Query Building (Fetcher)
    • Relation Helpers (Fetcher)
    • Database Access (Fetcher)
    • Assertion (Fetcher)

  • Data Flow

    • SQL Operations Reference

Core Types

These four types form the complete vocabulary for expressing a data retrieval request.

// Main executor that wraps a database connection.
type Fetcher struct {
    SqlDb *sql_db.SqlDb
}

// Query configuration object. Passed to builder methods and BuildSelect.
type FetcherMod struct {
    Paginated      bool              // enable LIMIT/OFFSET clause
    One            bool              // append LIMIT 1
    Count          bool              // SELECT COUNT(*) instead of columns
    Page           int               // current page (1-based)
    PerPage        int               // rows per page (default: 25)
    DefaultOrder   *FetcherOrderBy   // fallback order when OrderBys is empty
    Relations      []*FetcherRelation
    Filters        []interface{}     // active typed filters
    AllowedFilters filter.Filters    // filters eligible for ConvertRawFilters
    OrderBys       []*FetcherOrderBy
}

// Holds computed pagination metadata after a count query.
type FetcherPagination struct {
    Total      int
    Page       int
    PerPage    int
    LastPage   int
    Modulation *FetcherMod
}

// A single ORDER BY clause entry.
type FetcherOrderBy struct {
    Order  string  // "ASC" or "DESC"
    Column string
}

// Describes an optional relation/join that a caller can request.
type FetcherRelation struct {
    Relation  string    // relation name used with HasRelation
    Columns   []string  // columns to select from this relation
    AllColumn bool      // select all columns
}

// Raw string-based filter input, typically sourced from query parameters.
type RawFilter struct {
    Type      string  // must be "fields" to be processed
    Key       string  // maps to an AllowedFilter key
    Operation string  // "eq", "contains", etc.
    Value     string  // string representation of the value
}

Filter Types

Filters are typed structs appended to FetcherMod.Filters. Each filter type maps to a specific SQL fragment. The tables below group them by the kind of data they operate on.

String Filters

Numeric Filters

Boolean / Null / Time Filters

Methods

Filter Building (FetcherMod)

All methods in this group append a typed filter to mod.Filters. Call them on the FetcherMod before passing it to BuildSelect.

Raw Filter Conversion (FetcherMod)

These methods convert raw string-based filter input (typically from query parameters) into typed filters and append them to mod.Filters.

ConvertRawFilters(rawFilters []RawFilter)

Iterates over rawFilters, skips any entry whose Type is not "fields", resolves each entry against AllowedFilters via FilterByKey, then dispatches to the appropriate typed builder.

The following filter types are supported and the operations each one accepts are listed below.

📝 Note: ConvertRawFilters panics if it encounters a filter type in AllowedFilters that does not match any of the cases above. Ensure all entries in AllowedFilters are one of the supported types before calling this method.

FilterByKey(key string) filter.Filter

Searches AllowedFilters for the entry whose GetKey() equals key. Panics with an error if not found.

Sorting (FetcherMod)

These two methods append an order clause to mod.OrderBys. When OrderBys is non-empty, BuildSelect uses it; otherwise DefaultOrder is applied.

func (mod *FetcherMod) DescOrder(column string)
func (mod *FetcherMod) AscOrder(column string)

Both append a FetcherOrderBy to mod.OrderBys.

Debugging (FetcherMod)

func (mod *FetcherMod) Debug() string

JSON-serializes mod.Filters and returns the result as a string. Used internally by MustHave when panicking.

Query Building (Fetcher)

These methods assemble and return SQL strings from a configured FetcherMod. They do not execute queries — they produce the SQL and args that are then passed to the database access methods.

BuildSelect(mod *FetcherMod, table string, columns []string) (string, []interface{})

Assembles a full SELECT statement with the following structure:

SELECT   -- COUNT(*) when mod.Count is true, otherwise the quoted column list prefixed with o.
FROM <table> AS o
WHERE …  -- generated by Filters(mod, "o") if non-empty
ORDER BY … -- from mod.OrderBys, or mod.DefaultOrder as fallback
LIMIT … OFFSET … -- from Paginate(mod) when mod.Paginated && !mod.Count
LIMIT 1  -- appended when mod.One is true

Returns the SQL string and the corresponding args slice.

Filters(mod *FetcherMod, prefix string) (string, []interface{})

Iterates mod.Filters, generates one SQL fragment per filter, and joins them with AND. The prefix (e.g. "o") is prepended as prefix."column". Returns an empty string and nil args when there are no filters.

Fields(fields []string, prefix string) string

Formats a slice of column names into a quoted, comma-separated list with the given table alias prefix:

o."id", o."name", o."created_at"

Paginate(mod *FetcherMod) string

Returns a LIMIT n OFFSET m string. Defaults: PerPage = 25, Page = 1.

CalculatePagination(pagination *FetcherPagination, mod *FetcherMod)

Fills pagination.PerPage, pagination.Page, and pagination.LastPage from mod and pagination.Total:

LastPage = ((Total - 1) / PerPage) + 1

Relation Helpers (Fetcher)

HasRelation(mod *FetcherMod, name string) bool

Returns true if any entry in mod.Relations has Relation == name. Used by fetcher services to decide whether to JOIN or load additional data.

Database Access (Fetcher)

These are thin wrappers over sql_db.SqlDb that execute queries built by the methods above. Pass the SQL string and args returned by BuildSelect directly to these methods.

func (f *Fetcher) Query(ctx context.Context, query string, args ...interface{}) (*sql.Rows, error)
func (f *Fetcher) QueryRow(ctx context.Context, query string, args ...interface{}) *sql.Row
func (f *Fetcher) Exec(ctx context.Context, query string, args ...interface{}) (sql.Result, error)
func (f *Fetcher) IsNoRow(err error) bool

Assertion (Fetcher)

MustHave(has bool, mod *FetcherMod)

Panics with mod.Debug() output when has is false. Used to assert that a required record exists after a query.

⚠️ Warning: MustHave panics unconditionally when has is false. Use it only when a missing record represents an unrecoverable programming error or data integrity violation — not for validating user input or optional lookups.

Data Flow

SQL Operations Reference

The table below maps every filter type to the SQL fragment it generates, for quick reference.

Next Steps

• Hydrator — control which associations are loaded alongside the entities your Fetcher retrieves
• Handler — the write counterpart to the Fetcher; understand how entities are created and updated
• Appliers — see how appliers trigger fetcher calls after entities are created
• Maestros (Controllers) — see how Fetcher calls fit within the full MAE step flow