MosaicLeaks keeps large local inputs out of the repo. Bring your private task files, BrowseComp-Plus corpus files, and dense embedding shards through paths, dataset names, or environment variables.
The generation pipeline reads local private documents from chunks_local.jsonl.
Each chunk row includes:
chunk_iddoc_idsource_typetextoffsetsmeta.task_id
Recommended metadata:
meta.company_namemeta.file_namemeta.relative_pathmeta.extension
For paper replication, build this file from a DRBench-style task root with
folders named like DR0001, DR0002, and so on. Each task may contain:
- task-local files to chunk. The chunker has dedicated readers for PDF, DOC, DOCX, XLS, XLSX, and JSONL, and falls back to text decoding for other files.
context.jsonwithcompany_info.nameconfig/eval.jsonwith optional enterprise-fact seed QA pairs
For other local corpora, use mosaicleaks.data_prep.chunk_local_folder. The
manifest form is recommended when you have more than one task:
{"path": "/path/to/file.pdf", "task_id": "task_001", "company_name": "Example Company"}Manifest fields:
path: file path, absolute or relative to the manifest filetask_id: group ID used for chain construction and local retrievalcompany_name: optional label used for metadata and split groupingrelative_path: optional stable path for generateddoc_idvalues
Manual private QA seeds can also be supplied as JSONL and imported into
secret_inventory.jsonl:
{"doc_id": "local/task_001/file.pdf", "question": "What was the internal launch date?", "answer": "Q3 2026", "secret_type": "dates"}The importer checks that the referenced document exists and that the answer
appears in the document text, unless --no-require-answer-in-doc is passed.
We use BrowseComp-Plus as the public-web corpus for generation patterns that include web hops. Dataset generation reads prepared web chunks for bridge discovery:
chunks_web.jsonl
Each chunk row includes chunk_id, doc_id, text, offsets, and optional
metadata. The retrieval helper can additionally use the full corpus in two
ways:
--browsecomp-corpus Tevatron/browsecomp-plus-corpus, loaded through thedatasetspackage.--browsecomp-corpus /path/to/corpus.jsonl, where each row hasdocid,url, andtext.
Dense BrowseComp retrieval, including the retrieval helper, uses embedding
shards compatible with the MosaicLeaks BrowseCompHybridSearcher:
corpus.shard00000_of_000NN.pkl
corpus.shard00001_of_000NN.pkl
...
Each shard stores the dense vectors and lookup IDs used to map search results
back to BrowseComp document IDs. The pickle payload is a two-item tuple:
(embeddings, lookup), where embeddings is a 2D float array and lookup
contains the corresponding BrowseComp document IDs in row order. Use the same
embedding model for search queries that was used to create those shards.
The generation pipeline can supplement BrowseComp-Plus with small curated web files derived from task URLs:
docs_curated_web.jsonl: fetched URL metadata and full textchunks_curated_web.jsonl: paragraph/heading-aware chunks
These files are optional. When present, the helper and generator restrict their use by task ID so they do not become a general web source.
All LLM and embedding calls use OpenAI-compatible APIs:
- chat completions for generation, validation, filtering, and rewording
- embeddings for dense local and BrowseComp retrieval
StepFun can be used as the default provider by setting:
export MOSAICLEAKS_LLM_BASE_URL=http://your-endpoint:8000/v1
export MOSAICLEAKS_LLM_MODEL=step-3.5-flashEquivalent local vLLM, OpenAI, OpenRouter, or other compatible endpoints work when they implement the same API shape.