What is inside a prediction-market series-day bundle: zstd JSON-lines tick files with L2 order books, nanosecond timestamps and a parser that reads every venue.
A series-day bundle is the unit our archive sells prediction-market data in: all contract files of one series — say, every Kalshi BTC 15m market — for one UTC day. This guide walks through exactly what is inside those files, byte by byte, so you can decide whether to write your own parser (it is easy) or use the free tooling (it is easier). Everything shown below comes verbatim from the free Kalshi sample day you can download right now.
Each file is named {instrument_id}_{date}.txt.zst and is zstd-compressed text. The first line is a JSON object with the instrument's masterdata; every following line is one event encoded as a JSON array. The first line of the Kalshi sample looks like this (truncated):
{"instrument":{"id":466528,"code":"KXBTCMINY-27JAN01-55000.00",
"native_code":{"default":"KXBTCMINY-27JAN01-55000.00"},
"type":"perpetual","state":"open",
"exchange_id":46,"exchange_code":"kalshi",
"ticksize":0.01,"lot_size":0.01,"min_order":0.01, …}}Raw files carry "type":"perpetual" even for event contracts — that is the recorder's internal vocabulary, kept stable for parser compatibility. The shop and analytics classify these venues as the prediction class.
Every event line shares one envelope, regardless of venue: [msgType, instrumentId, prevEventId, eventId, adapterTs, exchangeTs, data, …]. adapterTs is when our co-located recorder received the message; exchangeTs is the venue's own timestamp — both integer nanoseconds since epoch, UTC. The prevEventId/eventId chain lets you verify you have every message. A real trade from the sample:
[2,466528,"8454531473485012705","5466866425187754076",
1785283123014075654, ← adapterTs (ns, capture)
1785283123000000000, ← exchangeTs (ns, venue)
[[1,"0.6","2.88","e463494f-…",1785283123000000000]]]
↑side ↑price ↑qty ↑trade id ↑per-fill ts| msgType | Meaning | Notes |
|---|---|---|
| 0 | Order-book snapshot | Full L2 state; a new snapshot resets the book |
| 1 | Order-book update | Incremental L2 change: [side, price, quantity, count] — quantity 0 removes the level |
| 2 | Trades | One or more fills: side, price, quantity, trade id |
| 5 | Instrument state | Trading-status changes (different envelope shape) |
| 6 | Top-of-book (BBO) | Not present in the recorded Kalshi and Polymarket files — top-of-book comes from the L2 replay |
| 7 | Mark price | Derivatives; carries options greeks on options venues |
| 8 | Index price | Underlying reference |
| 9 | Funding | Perpetuals only |
| 17 | Liquidations | Derivatives only, files from 2026 on |
Prediction-market files consist almost entirely of types 0, 1, 2 and 5 — order-book life plus trades.
In the CryptoStruct prediction-market archive, Kalshi and Polymarket files do not carry a separate type-6 BBO stream — the top-of-book at any moment is reconstructed by replaying types 0/1, and the free reader does this for you with a Book class. Prices on event contracts live in 0..1: in the sample files shown here, the recorded tick size is 0.01 for the Kalshi contract (USD) and 0.001 for the Polymarket contract (USDC) — tick sizes are market-specific and may change over time. Turnover in our analytics is USD-normalized.
The free AI toolkit ships cryptostruct_reader.py — a streaming parser with a CLI. The reader itself uses only the Python standard library; decompression requires either the zstd CLI or the optional zstandard package, whichever is available:
# summary of a day file (counts per message type, time span)
python3 cryptostruct_reader.py info 466528_2026-07-29.txt.zst
# all trades as CSV / Parquet
python3 cryptostruct_reader.py trades 466528_2026-07-29.txt.zst --out trades.csv
# order book sampled every second, top 5 levels
python3 cryptostruct_reader.py book 466528_2026-07-29.txt.zst --every 1s --depth 5Writing your own parser instead is deliberately easy — iterate lines, json.loads, switch on msgType — but three properties of the format bite people who skim:
Day files are concatenations of multiple zstd frames. Some stream decoders stop silently after the first frame (Node's built-in zstd stream does). Use zstd -dc or a library that reads all frames — and sanity-check that your last event is near 24:00 UTC.
Prices and quantities are strings ("0.6", not 0.6) so no precision is lost in transit. Parse them with a decimal type if you aggregate money; casting to float is fine for features, wrong for accounting.
Timestamps like 1785283123014075654 exceed the 53-bit float mantissa — JavaScript's plain JSON.parse silently corrupts them. Use BigInt-aware parsing in JS; Python ints are fine.
Every sample and purchased day also serves flat exports via ?format= on its download link: trades.csv.gz, bbo.csv.gz, liquidations.csv.gz and Parquet variants — columns per our docs convention with microsecond UTC timestamps (the native files stay nanoseconds). Note the BBO export does not exist for files that carry no type-6 BBO stream (in this archive: Kalshi, Polymarket, BitMEX, Coinbase, Kraken spot) — there the L2 replay above is the way to top-of-book.
| Series | Venue | Days | Coverage | Size | |
|---|---|---|---|---|---|
| BTC Up/Down 15m | Kalshi | 180 | February 2026 – August 2026 | 43.1 GB | Browse days |
| Bitcoin price Above/below | Kalshi | 180 | February 2026 – August 2026 | 128 GB | Browse days |
| BTC Up/Down 5m | Polymarket | 194 | February 2026 – August 2026 | 204 GB | Browse days |
| BTC Up/Down 15m | Polymarket | 320 | October 2025 – August 2026 | 95.8 GB | Browse days |
| BTC Above (price strikes) | Polymarket | 466 | May 2025 – August 2026 | 54.8 GB | Browse days |
Live coverage of some flagship series (updates daily). Each day is €1 and downloads per file or as one ZIP.
This walkthrough covers the fields relevant for prediction-market work; the complete field-by-field specification (options greeks, funding, schema-version tails) lives in the shipped format reference and the official protocol docs. Sample lines are from July 2026 files; the schema is versioned and additive, so older files can lack newer trailing fields — parse positionally and tolerate unknown tails. Not investment advice; this is a file-format guide.
zstd-compressed JSON lines: line 1 is instrument masterdata, every other line one event (trade, L2 book update, snapshot) with nanosecond UTC timestamps. The schema is identical across all venues in the archive.
Both: every trade print and the complete L2 order-book evolution (snapshots + incremental updates). Top-of-book is derived by replaying the book — the free Python reader does that out of the box.
Yes — every sample and purchased day exports as gzipped CSV or Parquet via ?format= on its download link, no extra cost. Timestamps in the exports are microseconds; the native tick files keep nanoseconds.
Yes: complete free sample days — including one Kalshi and one Polymarket contract — are on the downloads page, in exactly the shop format, next to the free reader and format reference.