Skip to content

Pack / encrypt for transport

Available in: Python · Node · CLI · Browser

Compress an AGS4 file for storage or transfer, then restore it byte-for-byte — no schema, no re-emit, just zstd (with optional age encryption on top).

# what this shows: zstd transport round-trip — pack a file to .zst, unpack it, prove byte-identical + smaller.
import shutil
import tempfile
from pathlib import Path

from laterite.transport import pack, unpack

with tempfile.TemporaryDirectory() as tmp:
    tmp = Path(tmp)
    src = tmp / "site.ags"
    shutil.copy("examples/sample_site.ags", src)

    packed = pack(src, dest=tmp / "site.ags.zst")
    restored = unpack(packed, dest=tmp / "restored.ags")

    original_bytes = src.read_bytes()
    restored_bytes = restored.read_bytes()
    original_size = len(original_bytes)
    compressed_size = packed.stat().st_size

    print(f"original:   {original_size} bytes")
    print(f"compressed: {compressed_size} bytes")
    print(f"round-trip byte-identical: {restored_bytes == original_bytes}")

    assert restored_bytes == original_bytes
    assert compressed_size < original_size

original:   7007 bytes
compressed: 1743 bytes
round-trip byte-identical: True

transport.pack zstd-compresses the file as-is and writes a .zst alongside it; transport.unpack reverses it. AGS4 is line-oriented, quote-heavy text, so it squeezes hard — roughly 4x here (7007 → 1743 bytes), and more on real deliveries with repeated headings. The round-trip is byte-identical: pack moves bytes, it doesn't parse or normalise them, so a packed-then-unpacked file matches the original down to its line endings and trailing whitespace. That makes it safe to pack a file you intend to validate later — nothing about it changes.

Both calls take a dest= to control where the output lands; omit it and the output sits next to the source with the .zst suffix added (pack) or removed (unpack).

Add encryption — lock / unlock are the same round-trip with a passphrase-encrypted age envelope on top of the zstd pack; without the passphrase the payload is opaque:

# what this shows: age-encrypted transport round-trip — lock a file with a passphrase, unlock it, prove byte-identical.
import shutil
import tempfile
from pathlib import Path

from laterite.transport import lock, unlock

with tempfile.TemporaryDirectory() as tmp:
    tmp = Path(tmp)
    src = tmp / "site.ags"
    shutil.copy("examples/sample_site.ags", src)

    # lock = zstd pack + age passphrase encrypt (scrypt KDF + ChaCha20-Poly1305).
    # Omit dest and it writes <src>.zst.age alongside the source.
    sealed = lock(
        src, password="correct horse battery staple", dest=tmp / "site.ags.zst.age"
    )
    restored = unlock(
        sealed, password="correct horse battery staple", dest=tmp / "restored.ags"
    )

    print(f"sealed: {sealed.name}")
    print(f"round-trip byte-identical: {restored.read_bytes() == src.read_bytes()}")

    # Encryption is transparent to the payload: unlock restores the exact bytes.
    assert restored.read_bytes() == src.read_bytes()
    assert sealed.suffixes[-2:] == [".zst", ".age"]

sealed: site.ags.zst.age
round-trip byte-identical: True

The KDF is scrypt at age's standard log_N 18 tier — deliberately expensive (~256 MiB), so a stolen envelope resists brute force. Pass log_n= to tune it, level= for the zstd ratio, and omit dest= to write <src>.zst.age alongside the source.

// what this shows: zstd transport round-trip — pack a file to .zst, unpack it, prove byte-identical + smaller.
import { transport } from "laterite";
import assert from "node:assert/strict";
import {
  copyFileSync,
  mkdtempSync,
  readFileSync,
  rmSync,
  statSync,
} from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";

const tmp = mkdtempSync(join(tmpdir(), "laterite-docs-"));
const src = join(tmp, "site.ags");
copyFileSync("examples/sample_site.ags", src);

transport.pack(src, join(tmp, "site.ags.zst"));
transport.unpack(join(tmp, "site.ags.zst"), join(tmp, "restored.ags"));

const original = readFileSync(src);
const restored = readFileSync(join(tmp, "restored.ags"));
console.log(`original:   ${original.length} bytes`);
console.log(`compressed: ${statSync(join(tmp, "site.ags.zst")).size} bytes`);
console.log(`round-trip byte-identical: ${original.equals(restored)}`);

assert.ok(original.equals(restored));
assert.ok(statSync(join(tmp, "site.ags.zst")).size < original.length);
rmSync(tmp, { recursive: true, force: true });

original:   7007 bytes
compressed: 1743 bytes
round-trip byte-identical: true

The transport namespace mirrors Python call-for-call — transport.pack(src, dest) / transport.unpack(src, dest) — and the outputs are the same standard zstd frames, so a file packed in Node unpacks in Python (or with plain zstd -d) and vice versa.

Add encryption — same envelope, passphrase-based:

// what this shows: age-encrypted transport round-trip — lock a file with a passphrase, unlock it, prove byte-identical.
import { transport } from "laterite";
import assert from "node:assert/strict";
import { copyFileSync, mkdtempSync, readFileSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";

const tmp = mkdtempSync(join(tmpdir(), "laterite-docs-"));
const src = join(tmp, "site.ags");
copyFileSync("examples/sample_site.ags", src);

// lock = zstd pack + age passphrase encrypt (scrypt KDF + ChaCha20-Poly1305).
transport.lock(
  src,
  join(tmp, "site.ags.zst.age"),
  "correct horse battery staple",
);
transport.unlock(
  join(tmp, "site.ags.zst.age"),
  join(tmp, "restored.ags"),
  "correct horse battery staple",
);

const original = readFileSync(src);
const restored = readFileSync(join(tmp, "restored.ags"));
console.log(`round-trip byte-identical: ${original.equals(restored)}`);

// Encryption is transparent to the payload: unlock restores the exact bytes.
assert.ok(original.equals(restored));
rmSync(tmp, { recursive: true, force: true });

round-trip byte-identical: true

transport.lock(src, dest, password) seals the same standard zstd+age envelope Python and the browser read; the positional password (with optional level / logN after it) is the only shape difference from Python's keyword args. A file sealed here opens with transport.unlock anywhere, or with stock age given the passphrase.

lat pack examples/sample_site.ags site.ags.zst   # → compressed (summary on stderr)
lat unpack site.ags.zst restored.ags             # → restored, byte-identical
lat read restored.ags | head -3                  # prove it: the first group codes

PROJ
TRAN
UNIT

lat pack / unpack are the shell face of the same zstd round-trip (the summary — ratio + timing — prints to stderr; here the round-trip is proved by reading a group back out of the restored file). lat lock / unlock add the age passphrase envelope. The passphrase is never a flag — argv leaks into ps and shell history; the precedence is --password-file <path> → $LAT_TRANSPORT_PASSWORD → an interactive prompt. --level tunes the zstd ratio and --log-n the scrypt tier (default 18). A file sealed here opens with transport.unlock in Python/Node — or stock age — anywhere.

The web app's Tools pane has the same lock/unlock round-trip running client-side: drop a file, enter a passphrase, download the sealed .zst.age (or unseal one you received). The output is byte-compatible with the Python/Node lock — a file sealed in the browser opens with transport.unlock anywhere, and vice versa — and nothing ever leaves your machine.

The sealed format is standard zstd inside a standard age envelope — not a laterite-only container — so recipients without laterite can still recover the file with stock age + zstd tooling given the passphrase.

See also: Cheatsheet · Certify a file