poka.sh

 Docs

Test a parser

how-to

This page covers testing one parser: unit tests, the corpus, coverage and scaling. For making a parser fast, see Make a parser fast and memory-bounded.

Run the tests

cargo test -p poka-<ext>
cargo build -p poka-<ext> --target wasm32-wasip1 --profile wasm-check

A --profile wasm-check build (Cargo.toml) proves the crate compiles for wasm in a fraction of the release build's time; build with --release whenever the wasm is installed, measured or shipped.

Run cargo from inside the repo, so that it reads the repo's .cargo/config.toml and its shared target directory.

Test on the corpus

Corpus tests read real files from poka-samples, expected as a sibling checkout (../poka-samples). Set POKA_SAMPLES if yours lives elsewhere. The tests fail loudly when the sample set shrinks below its floor, because a coverage claim depends on the samples still being there.

  1. Put a tests/samples.rs in the crate that parses every sample of the format.

  2. Run every crate whose samples are present:

    python3 scripts/corpus-tests.py

    It prints pass, fail or skip per crate. Pre-push runs it for the crates you changed, and for a shared crate those crates depend on: a corpus suite that lives in a shared crate (poka-sheet-common holds the ones for .xls, .xlsx and .xlsm) names the extensions it serves in that crate's Cargo.toml, under [package.metadata.poka.corpus] suffixes, because it has no plugin.json to route them from.

  3. Sweep every sample of the format, not the one in the bug report. A defect often lives only in the files nobody opened.

Never include_bytes! a poka-samples file unless it sits behind #[cfg(feature = "corpus-tests")]. CI clones this repo alone, and pre-commit rejects the bare form.

Prove the coverage

A parser that reads every corpus file without error can still drop a construct, and you do not notice, because what it drops never looks missing. Assert coverage with the strongest evidence the format offers:

  1. A formal schema, if the format publishes one. Check that every construct the corpus uses is one the schema declares.
  2. The file's own arithmetic. #Nodes then the nodes, a declared element count then the elements, a chunk length then the chunk. The file disagrees with a mis-parse, not your test.
  3. Exhaustiveness over the corpus. Account for every line, record or chunk, with nothing left over.
  4. An exhaustive tiling, when the format publishes nothing. Divide the file into contiguous, non-overlapping regions that cover it end to end, and name each one. A test can answer yes or no to that; a percentage drifts silently when a parser starts losing bytes.

A region can be named without being interpreted. "The numeric fields of the record before this one" is an attribution, not a reading. Report both numbers and keep them apart: how much is accounted for, and how much of that carries a meaning. Never average them into one figure.

An other or unknown bucket is good design, because it keeps what you do not model. Add a test that asserts it is empty over the corpus, or it becomes the place where coverage gaps hide.

Prove the asymptotics

Corpus benchmarks find constant factors. They do not find quadratics, because no corpus file is big enough. Give every lookup you make sub-quadratic a tests/scaling.rs, written as a ratio of two input sizes and not as a wall clock. The worked recipe is in Make a parser fast and memory-bounded.

Each parser also has a tests/bench.rs that parses the whole corpus in-process:

cargo test --release -p poka-<ext> --test bench -- --nocapture

Use that rather than the wall clock of cargo test, most of which is compilation and file I/O.

Run the repo checks

python3 scripts/validate-parser-plugin.py
python3 scripts/test-manifests.py
python3 scripts/check-parser-invariants.py
python3 scripts/scorecard.py --check

Pre-commit and pre-push run these for you once you set git config core.hooksPath .githooks. The process side (hooks, what CI runs and what stays local) is in CONTRIBUTING.md.

See also