Querying S3 without Parquet
Can JSONL on S3 be queried without Parquet? Range search, a B-tree, and an LSM tree, timed on a real bucket. It works. Parquet still wins on upkeep.
parquet
3 posts
Dissecting Parquet
Inside a Parquet file: columns, row groups, pages, and the footer. Animations of min/max and Bloom filters pruning groups, then Range fetching the slices.
Running a Notification System with BullMQ, Part 4 - How to Read the History You Stored
Delivery-history JSON in S3 is hard to query. Four options on infrastructure, query cost, and freshness, and why nightly Parquet conversion won.