Research libraries for a format
Before you write a parser, find out whether someone has already written one.
This page covers the review itself, how to record the result in
FILE_FORMATS.csv, and the tool that researches many formats at once.
Review one format
- Search for existing parsers of the format.
- Check the licence. Is it usable in a commercial product?
- Check the language. If it is not Rust, decide whether to rewrite it in Rust.
- If nothing usable exists, write the parser (Write a parser).
Record what you found
Put the result in the format's Libraries cell in FILE_FORMATS.csv, whether
or not you adopt a library. Write one entry per library, separated by |:
name (status; licence; language[; url]) — optional note
| Part | Values |
|---|---|
status |
used (linked into the parser), candidate (worth trying), rejected (ruled out; say why in the note), reference (readable only, for example copyleft or not wasm-friendly) |
licence |
an SPDX expression, Commercial for a paid product, or licence unconfirmed |
language |
the language the library is written in |
url |
give one for anything you did not link |
python3 scripts/sync-file-formats.py --check rejects a cell that breaks this
grammar (scripts/_libraries.py holds the rules).
Research many formats at once
scripts/library-research.py drives agents that look for libraries across a
batch of formats. Its docstring is the full guide. The commands:
| Command | What it does |
|---|---|
shards |
Picks the next formats to research. |
prompt |
Prints the agent prompt (kept in docs/status/library-research-prompt.md). |
check-urls |
Checks the URLs the agents wrote. |
apply |
Merges what the agents wrote into FILE_FORMATS.csv. |
rust-crates |
Adds the Rust crates of lib.rs's file-format categories. |
Formats already researched are listed in docs/status/library-research-log.csv
and are not picked again.