Poka parsers scorecard
Generated by scripts/scorecard.py. Do not edit by hand: python3 scripts/scorecard.py rewrites it, and python3 scripts/scorecard.py --check fails when it is stale.
Generated file. Do not edit it by hand. Run
python3 scripts/scorecard.pyto rewrite it;--check(pre-push) fails when this copy is stale.
The README headline counts manifests. This page says what sits behind them. It reads only committed files in this repo, so it is the same on every machine: no corpus, no network, no clock. The tier rules are in section 7, with what each tier does and does not prove.
Every row has a basis. measured: counted from files in this tree. estimated: inferred from them; the row says what from, and it is not a claim about the parsers' behaviour.
1. Format census
Counted by scripts/format-stats.py, the single source for the format count, except where a row says otherwise.
| Figure | Count | Basis |
|---|---|---|
Directories under parsers/ |
892 | measured |
Of those, without a plugin.json (helper libraries, not parsers) |
13 | measured |
Manifests (crate plugin.json plus routes) |
1,045 | measured |
| crate manifests | 879 | measured |
routes in parsers/aliases.toml |
166 | measured |
stubs in parsers/unsupported.toml |
0 | measured |
| Distinct manifest ids | 1,045 | measured |
Unique file_extension values (disabled manifests included) |
1,016 | measured |
| Unique alias extension entries | 649 | measured |
| Extensions claimed by more than one manifest | 0 | measured |
| Disabled manifests | 32 | measured |
| Enabled manifests | 1,013 | measured |
| Crate manifests that are not valid JSON | 0 | measured |
2. Depth tiers
Every manifest is in exactly one tier. Counts are measured; what a tier means is the estimate, and section 7 says how far it can be trusted.
| Tier | Manifests | Share | Basis |
|---|---|---|---|
| T1 Parser with a shared viewer | 558 | 53.4% | measured |
| T2 Standalone parser | 20 | 1.9% | measured |
| T3 Metadata card | 272 | 26.0% | measured |
| T4 Alias-only route | 163 | 15.6% | measured |
| T5 Hex stub (backlog) | 0 | 0.0% | measured |
| T6 Disabled | 32 | 3.1% | measured |
| Total | 1,045 | 100.0% | measured |
Hex stubs split three ways: 0 in parsers/unsupported.toml, 0 other routes that run poka-hex, and 0 crates of their own whose viewer is poka-hex-viewer (measured).
Alias-only routes by the module they run (measured, top 10):
| Module | Routes |
|---|---|
poka-code |
80 |
poka-xml |
13 |
poka-pe |
12 |
poka-text |
11 |
poka-ini |
6 |
poka-cfb |
5 |
poka-compressed |
5 |
poka-pem |
3 |
poka-soe |
3 |
poka-doc |
2 |
3. What the headline counts
scripts/kpi-stats.py counts every enabled manifest as supported except the unsupported.toml stubs and routes to poka-hex. That includes crates that only hex-dump a file, because they have a source file of their own. The scorecard moves those to T5.
| Figure | Count | Basis |
|---|---|---|
| Counted as supported by kpi-stats (README headline) | 1,013 | estimated: the kpi-stats rule applied to the manifests (T1 to T4 plus own-crate hex shells) |
| of which own-crate hex shells | 0 | measured |
| Supported with hex shells removed (T1 to T4) | 1,013 | measured |
| Own crates that read something (T1 to T3, upper bound) | 850 | estimated: the viewer a manifest names, not the code behind it |
| Hex-dump backlog (T5) | 0 | measured |
4. Viewer usage
Enabled manifests by the viewer they name (measured). A standalone parser names none.
| Viewer | Manifests | Crates | Routes |
|---|---|---|---|
poka-tree-viewer |
318 | 281 | 37 |
poka-metadata-card-viewer |
273 | 272 | 1 |
poka-table-viewer |
113 | 99 | 14 |
poka-code-viewer |
81 | 1 | 80 |
poka-text-viewer |
71 | 50 | 21 |
poka-image-with-meta-viewer |
47 | 45 | 2 |
poka-3d-viewer |
41 | 41 | 0 |
| (none: standalone) | 26 | 20 | 6 |
poka-vector-viewer |
21 | 21 | 0 |
poka-workbook-viewer |
9 | 8 | 1 |
poka-slide-viewer |
4 | 3 | 1 |
poka-chart-viewer |
3 | 3 | 0 |
poka-ebook-viewer |
2 | 2 | 0 |
poka-diff-viewer |
1 | 1 | 0 |
poka-font-viewer |
1 | 1 | 0 |
poka-pdf-viewer |
1 | 1 | 0 |
poka-svg-viewer |
1 | 1 | 0 |
5. Test presence by tier
Crates under parsers/ that have each file or marker (measured). Presence is not a pass: tests/samples.rs needs the poka-samples corpus, so it runs locally (scripts/corpus-tests.py), never in CI. Alias-only routes have no crate, so they are not in this table.
| Tier | Crates | src/lib.rs |
Unit tests | tests/samples.rs |
tests/scaling.rs |
tests/bench.rs |
Quarantined |
|---|---|---|---|---|---|---|---|
| T1 Parser with a shared viewer | 558 | 95 | 231 | 92 | 9 | 35 | 0 |
| T2 Standalone parser | 20 | 1 | 7 | 1 | 0 | 0 | 0 |
| T3 Metadata card | 272 | 17 | 35 | 16 | 0 | 9 | 0 |
| T6 Disabled | 29 | 1 | 3 | 1 | 0 | 1 | 0 |
Helper crates (no plugin.json) |
13 | 11 | 11 | 2 | 0 | 0 | 0 |
| Total | 892 | 125 | 287 | 112 | 9 | 45 | 0 |
Unit tests means a #[cfg(test)] module under src/. Quarantined crates are listed in build/unit-test-quarantine.txt and skipped by the unit-test gate (0 listed).
6. Never-crash lint adoption
The root Cargo.toml has no [workspace.lints.clippy] table that denies panicking constructs, and no crate sets [lints] workspace = true, so no crate is held to the never-crash rule by a lint (measured). The rule is written in docs/how-to/make-a-parser-fast.md, "Never panic", and is exercised by the robustness sweep (scripts/quality-gates.py), not enforced by clippy. This section will report adoption by tier once there is a lint table and crates opt in.
7. How the tiers are derived
First match wins, in this order: disabled, hex stub, alias-only route, standalone, metadata card, shared viewer. Everything comes from plugin.json, parsers/aliases.toml, parsers/unsupported.toml and the module a route names; no source file is parsed for meaning.
T1 Parser with a shared viewer. A plugin.json crate, not disabled, whose viewer is a shared viewer other than poka-hex-viewer and poka-metadata-card-viewer.
- Shows: The crate has its own source and emits a typed AST (tree, table, text, image, 3D, vector, workbook and so on) for a viewer poka-viewers ships.
- Does not show: How much of the format it reads. A tree of a few top-level keys and a full decoder of a binary container both land here. Depth is shown only by a corpus test (section 5) and the module's own documentation.
T2 Standalone parser. A plugin.json crate, not disabled, with no viewer. SPEC.md section 2: it writes a complete PreviewResponse itself (3D scenes, PDF, media, archives).
- Shows: The crate has its own source and renders its own result.
- Does not show: That the result is complete, or that it is not a dump in another wrapper. Nothing in the manifest says what the response holds.
T3 Metadata card. A plugin.json crate, not disabled, whose viewer is poka-metadata-card-viewer. This is the tier of the generic-container and identification shells.
- Shows: The crate reads the file far enough to fill a titled list of key and value entries.
- Does not show: That there is more than identification. A card can carry a rich header or a single magic-number line; the manifest does not tell them apart.
T4 Alias-only route. An entry in parsers/aliases.toml that is not disabled and does not run poka-hex: no crate of its own, another module's wasm under a new id, name and extension.
- Shows: A host routes the extension to a module that is a real parser of something.
- Does not show: That the module understands this specific format. The parser tutorial asks for a corpus sample to be run first; the scorecard cannot see whether it was.
T5 Hex stub (backlog). Any manifest, not disabled, that is listed in parsers/unsupported.toml, runs the retired poka-hex module, or names the retired poka-hex-viewer as its viewer. Both are gone, so this tier is a guard against hex shells coming back.
- Shows: The extension is recognised and a file opens as a hex dump.
- Does not show: Any parsing. This is the pre-support state: kpi-stats.py reports the
unsupported.tomlstubs as a backlog, not coverage, but counts own-crate hex shells as full parsers; the scorecard does not.
T6 Disabled. disabled: true in a crate's plugin.json, or a disabled_reason on an aliases.toml route (SPEC.md section 2).
- Shows: Nothing: the host skips registration. Kept for the id and as a record.
- Does not show: Whatever depth the source has, it is not reachable.
What no depth tier can see: whether a parser's output is correct, how much of the format it decodes, and whether a poka-hex-viewer shell hides behind another viewer's name. For how much of the format is decoded, section 8 gives the breadth tier of the formats reviewed so far; for the rest, read the crate's module documentation and its tests/samples.rs.
8. Breadth tiers
A depth tier (section 2) says what kind of parser reads a format. A breadth tier says how much of the format Poka reads. It is not derived from the manifests: a reviewer assigns it from the parser source, its tests, the corpus and the format's own documentation, and writes it into the format's entry in docs/data/format-metadata/. Only formats that have a format page can be assessed, so this section covers those and says nothing about the rest of the registry. Counts are measured from those files; what a tier means is the reviewer's judgement, and docs/explanation/breadth-tiers.md says how far it can be trusted.
379 of 379 documented formats are assessed (100.0%).
| Breadth tier | Formats | Which | Basis |
|---|---|---|---|
| B1 Complete | 12 | BND, DIR, DWL, DWL2, GIF, MDD, MTD, PGM, QOI, RAS, RDD, SOE | measured |
| B2 Broad | 92 | 3MF, ASD, AWS-CREDENTIALS, BASH_HISTORY, BMP, CAPNP, CIN, CLASS, CNT, COOKIES, CRL, CRONTAB, CSR, CTB, D5MESH, DAE, DDY, DEM, DPX, DTB, DWG, EDN, EIO, ERR, EVT, FBS, FOLDED, FSTAB, GH, GHX, GLB, GLTF, GROUP, GZ, HDR, HOSTS, HTPASSWD, IDD, IDF, INF, INP, IPYNB, JKS, JPEG, JPG, JWKS, JWT, KNOWN_HOSTS, LNK, MAGNET, NETRC, OSW, P7B, P7S, PASSWD, PATCH, PCX, PDF, PEM, PGPASS, PNG, PPM, PRJ, REG, REGO, RS, SERVICES, SHA256SUM, SHADOW, SMK, SSHCONFIG, STL, STRACE, SUDOERS, SVELTE, SVG, SYSLOG, TAB, TBD, TGA, THRIFT, TLE, TTF, VCF, VUE, WEA, WEBP, WG, XCU, XER, XML, XPM | measured |
| B3 Core | 79 | 3DM, APNG, ASC, AUTOMATICDESTINATIONS-MS, B18, BDL, BEAM, CALLGRIND, CGM, CPUPROFILE, DCK, DDS, DER, DOC, DOCX, DOT, DRS, DSBXML, DXBC, DXF, DYF, DYN, ELF, EXE, FBX, GEOJSON, GML, GPKG, GPX, HEAPSNAPSHOT, IES, IFC, IFCZIP, JSTACK, JT, KML, KMZ, LDT, LUAC, MACHO, MFT, MID, MMD, NBT, NMAP, OBJ, ODB, ODP, ODS, ODT, OSM, OTT, OVPN, P12, P8, PCAP, PCAPNG, PF, PKPASS, PPROF, PPT, PPTX, PSD, PYC, RAD, SHP, STA, STEP, TCX, THEME, THMX, TSM, VSDX, WASM, WPG, XBE, XLSX, YANG, YAR | measured |
| B4 Summary | 184 | 7Z, ACE, ADA, ADF, AFM, AGDA, AHK, APK, APPIMAGE, APPLESCRIPT, APPX, AR, ARC, ARJ, BDF, BLO, BLP, BMFONT, BOCHS, BOM, BZ3, CAB, CAT, CBL, CFB, CLM, CORE, CPIO, CR, CRON, CYPHER, D.TS, D2, D64, DATA, DEB, DHALL, DLL, DMP, DSK, DYLIB, EBNF, EDB, EGG, ELM, EOT, EVTX, EXT4, F90, FEN, FNL, FNT, FON, FONTCONFIG, GB, GBA, GBS, GEN, GPT, GR, GRIB, HBS, HPROF, IMD, IMG-CRIU, IPTABLES, ISO, ISPROJ, JAR, JFR, JSONNET, KDBX, KK, KO, LEAN, LHA, LL, LUKS, LZ, LZ4, LZH, LZO, M2TS, MAP, MAR, MBR, MBTILES, MD-ROM, MDS, MIR, MLIR, MOJO, MP3, MP4, MPQ, MSI, MST, MSU, MTR, NES, NFT, NIM, NRG, NSF, NU, O, OCT, OSC, PAK, PAR2, PAS, PBF, PCF, PDB-DEBUG, PERFETTO-TRACE, PFA, PFB, PFM, PGN, PLI, PONY, PPK, PRG, PROLOG, PUML, PURS, PYI, RAR, RAR5, RBS, RES, RKT, ROC, RPM, SAV, SDG, SDW, SFC, SFD, SGF, SHX, SID, SKP, SMC, SNAPPY, SNOOP, SPC, SPV, SQLITE, SQLITE3, T64, TAR, TFM, THUMBCACHE, TLA, TM2, TMY3, TRACE.JSON, TSCN, TTC, TZX, UCI, UUE, V, VALA, VBS, VDI, VHD, VHDX, VLANG, VMDK, VPK, WAT, WAV, WIM, WOFF2, XAR, XEX, Z01, Z64, ZIG, ZIP, ZOO, ZST | measured |
| B5 Identification | 12 | BR, BSDIFF, DB, DGN, EKD, FLAC, LZMA, OGG, UMX, WEBM, Z, ZPAQ | measured |
| Not assessed | 0 | none | measured |
| Total | 379 | measured |
By group: one row per file in docs/data/format-metadata/, the formats it documents counted by breadth tier. A group with few formats has few format pages, not necessarily shallow parsers.
| Group | B1 | B2 | B3 | B4 | B5 | Not assessed | Formats |
|---|---|---|---|---|---|---|---|
| archives | 0 | 0 | 1 | 52 | 5 | 0 | 58 |
| audio-video | 0 | 0 | 1 | 1 | 3 | 0 | 5 |
| cad-3d | 0 | 6 | 7 | 1 | 1 | 0 | 15 |
| cad-bim | 2 | 6 | 5 | 0 | 0 | 0 | 13 |
| climate-light | 0 | 1 | 7 | 4 | 0 | 0 | 12 |
| documents | 0 | 2 | 2 | 0 | 0 | 0 | 4 |
| energyplus | 4 | 6 | 1 | 2 | 0 | 0 | 13 |
| fonts | 1 | 0 | 0 | 15 | 0 | 0 | 16 |
| games | 0 | 0 | 2 | 27 | 1 | 0 | 30 |
| gis | 0 | 4 | 5 | 4 | 0 | 0 | 13 |
| images | 4 | 13 | 4 | 0 | 0 | 0 | 21 |
| media-data | 0 | 2 | 2 | 5 | 1 | 0 | 10 |
| misc | 0 | 2 | 2 | 0 | 0 | 0 | 4 |
| office | 1 | 1 | 10 | 2 | 0 | 0 | 14 |
| source-code | 0 | 11 | 3 | 42 | 0 | 0 | 56 |
| system | 0 | 34 | 26 | 24 | 0 | 0 | 84 |
| system-pl | 0 | 4 | 1 | 5 | 1 | 0 | 11 |
| Total | 12 | 92 | 79 | 184 | 12 | 0 | 379 |