Chaining¶
laterite reads like a sentence. You read a file, then keep .validate()-ing
and .query()-ing and .at()-ing off the same handle — and nothing executes
until you ask for a frame. The handle stays the handle; the lazy parts stay lazy.
laterite.read(path|text=|data=) ──► Ags4File ─────────────────────────────────┐
│ │
.validate(warnings=) ──► self (Ags4File) …report on .report │
.at(code, ids) ──► AgsQuery (fan-out: one borehole's group set) │
.query(sql) ──► AgsQuery (lazy single-result builder) │
.sql(sql) ──► DuckDBPyRelation ◄── terminal │
.pipe(fn, *args) ──► whatever fn returns │
["LOCA"] ──► polars frame ◄── terminal │
.save / .certify / .text / .bytes ◄── terminals │
│
AgsQuery ── single-result branch ──────────────────────────────────────────────┤
.filter(expr) ──► AgsQuery (new) .select(*cols) ──► AgsQuery (new) │
└─ terminals: .frame() · .to_polars() · .to_pandas() · .relation() │
│
AgsQuery ── fan-out branch (from .at) ──────────────────────────────────────────┘
.groups ──► list[str] .frames() ──► dict[code, polars frame]
Every builder call returns a new AgsQuery — the handles are immutable, so
you can fork a chain without one branch mutating another.
The power ladder¶
Seven rungs, each runnable. They climb from a one-line self-chain to raw SQL, your own functions, and the certify fast-path.
1 · Minimal self-chain¶
.validate() returns the Ags4File itself, so it slots mid-chain; the
Report rides along on .report.
import laterite
# Validate a file. read(...).validate() returns the Ags4File (so it chains);
# the Report is on the .report property.
ags = laterite.read("examples/sample_site.ags").validate()
r = ags.report
print(
f"is_valid={r.is_valid} count={r.count} "
f"dict_version={r.dict_version!r} resolution={r.resolution!r}"
)
assert r.is_valid is True
assert r.count == 0
assert r.dict_version == "4.1.1" # auto-selected from TRAN_AGS
2 · Fan-out a branch¶
.at(code, ids) pivots the handle to one borehole's related group set and hands
back an AgsQuery. Ask it .groups to see what came along for the ride.
# what this shows: ags.at(code, ids) fans out to one borehole's related group set, exposed as q.groups.
import laterite
ags = laterite.read("examples/sample_site.ags")
q = ags.at("LOCA", ["BH01", "BH02"])
print(q.groups)
assert "LOCA" in q.groups and "SAMP" in q.groups
3 · Terminate the fan-out¶
.frames() materialises that record set as a dict of born-typed polars frames,
keyed by group code.
# what this shows: materialise a borehole's full record set as a dict of polars frames via .at(..).frames()
import laterite
ags = laterite.read("examples/sample_site.ags")
frames = ags.at("LOCA", ["BH01"]).frames()
# frames is a dict {group_code: polars frame}; pull one out by code (NOT q["SAMP"])
print(sorted(frames))
print(frames["SAMP"].height)
assert isinstance(frames, dict) and "SAMP" in frames and frames["SAMP"].height >= 1
4 · The lazy builder and its four terminals¶
.query(sql) opens a single-result AgsQuery. Stack .filter / .select —
nothing runs — then pick a terminal to materialise the plan.
# what this shows: the lazy single-result AgsQuery builder + its four terminals.
import laterite
import pandas as pd
import polars as pl
from duckdb import DuckDBPyRelation
# Build a query lazily: nothing runs until a terminal is called.
q = (
laterite.read("examples/sample_site.ags")
.query("SELECT * FROM LOCA")
.filter("LOCA_GL > 28")
.select("LOCA_ID", "LOCA_TYPE", "LOCA_GL")
)
# Four terminals materialise the same lazy plan:
frame = q.frame() # handle's default backend (polars)
pl_df = q.to_polars() # always polars
pd_df = q.to_pandas() # always pandas
rel = q.relation() # the lazy DuckDBPyRelation (not yet materialised)
print(pl_df)
# Projection is honoured exactly as declared.
assert frame.columns == ["LOCA_ID", "LOCA_TYPE", "LOCA_GL"]
assert pl_df.columns == ["LOCA_ID", "LOCA_TYPE", "LOCA_GL"]
# Each terminal returns its expected type.
assert isinstance(frame, pl.DataFrame)
assert isinstance(pl_df, pl.DataFrame)
assert isinstance(pd_df, pd.DataFrame)
assert isinstance(rel, DuckDBPyRelation)
shape: (7, 3)
┌─────────┬───────────┬─────────┐
│ LOCA_ID ┆ LOCA_TYPE ┆ LOCA_GL │
│ --- ┆ --- ┆ --- │
│ str ┆ str ┆ f64 │
╞═════════╪═══════════╪═════════╡
│ BH02 ┆ RC ┆ 32.49 │
│ BH03 ┆ RC ┆ 28.54 │
│ BH04 ┆ RC ┆ 29.04 │
│ BH05 ┆ RC ┆ 31.62 │
│ BH07 ┆ RC ┆ 31.33 │
│ BH08 ┆ CP ┆ 28.67 │
│ BH09 ┆ CP ┆ 30.98 │
└─────────┴───────────┴─────────┘
.frame() follows the handle's backend, .to_polars() / .to_pandas() force
one, and .relation() hands back the still-lazy DuckDBPyRelation.
5 · Raw SQL mid-chain¶
When the builder is too narrow, drop to .sql(sql) for a full join across
groups. It returns a DuckDBPyRelation — a terminal you materialise with .pl(),
.df(), or .arrow().
# what this shows: drop to raw SQL to join across groups, count samples per location.
import laterite
rel = laterite.read("examples/sample_site.ags").sql(
"SELECT l.LOCA_ID, count(*) n FROM SAMP s JOIN LOCA l USING (LOCA_ID) "
"GROUP BY 1 ORDER BY 1"
)
# rel is a DuckDBPyRelation (terminal); materialise to polars with .pl().
df = rel.pl()
print(df)
assert hasattr(rel, "pl")
assert df.height >= 1 and "n" in df.columns
shape: (14, 2)
┌─────────┬─────┐
│ LOCA_ID ┆ n │
│ --- ┆ --- │
│ str ┆ i64 │
╞═════════╪═════╡
│ BH01 ┆ 4 │
│ BH02 ┆ 2 │
│ BH03 ┆ 3 │
│ BH04 ┆ 4 │
│ BH05 ┆ 2 │
│ … ┆ … │
│ BH10 ┆ 3 │
│ BH11 ┆ 3 │
│ BH12 ┆ 3 │
│ BH13 ┆ 4 │
│ BH14 ┆ 4 │
└─────────┴─────┘
6 · Splice in your own step with .pipe¶
.pipe(fn, *args) passes the handle as fn's first argument and returns
whatever fn returns — so an escape hatch never breaks the chain. It works on
both Ags4File and AgsQuery.
# what this shows: .pipe(fn, *args) splices your own step into the chain — fn receives the handle as its first arg — on both Ags4File and AgsQuery.
import laterite
ags = laterite.read("examples/sample_site.ags")
# On an Ags4File: fn(self, *args) — the file handle is passed first, your extra args follow.
out = ags.pipe(lambda a, n: a.groups[:n], 3)
print("first 3 group codes:", out)
# On an AgsQuery: same contract — the query handle is passed in, you return whatever you like.
height = ags.query("SELECT * FROM LOCA").pipe(lambda q: q.frame().height)
print("LOCA row count via pipe:", height)
# .pipe returns your function's result verbatim, and passes the object as the first argument.
assert out == ["PROJ", "TRAN", "UNIT"]
assert ags.pipe(lambda a: a is ags) is True
assert height == 14
assert ags.query("SELECT * FROM LOCA").pipe(lambda q: q is not None) is True
7 · The certify fast-path¶
A clean .validate() can mint a .ags.idx certificate via .certify().
Re-read with that fresh cert and .validate() resolves without ever running the
rule engine — resolution reads certified, not exact.
# what this shows: the certify fast-path — a fresh .ags.idx cert lets validate() skip the rule engine.
import shutil
import tempfile
from pathlib import Path
import laterite
with tempfile.TemporaryDirectory() as tmp:
tmp_path = str(Path(tmp) / "site.ags")
shutil.copy("examples/sample_site.ags", tmp_path)
# certify() runs the validation itself and mints <path>.ags.idx for an error-clean file.
idx = laterite.read(tmp_path).certify()
# re-reading with the fresh cert lets validate() answer without running the rule engine.
ags = laterite.read(tmp_path, index=str(idx)).validate(warnings=False)
print(ags.report.certified, ags.report.resolution)
# `certified` says the ENGINE was skipped; `resolution` still says which dictionary judged it.
assert ags.report.certified
Single-result and fan-out don't mix on one AgsQuery
The single-result terminals (.filter / .select → .frame() /
.to_polars() / .to_pandas() / .relation()) and the fan-out terminals
(.frames() / .groups) belong to different AgsQuery shapes — a
.query(sql) builder versus an .at(...) fan-out — and are mutually
exclusive on a given handle. And because every builder call returns a new
immutable AgsQuery, reassign (q = q.filter(...)) rather than expecting
in-place mutation.
Where next¶
- The Cookbook chains these rungs into end-to-end recipes — join, filter, certify, and write back out.
- The Reference cheatsheet is the one-screen map of every method on the chain and what it returns.