Parquet vs CSV vs JSON: Choosing the Right Delivery Format

The format decision almost never gets made in a design review. It gets made in a thread, when whoever is producing a dataset asks what format you want and somebody on the receiving side types “CSV is fine.” Nobody prices that answer. Six months later the dataset has grown by two orders of magnitude, the warehouse bill has a line item nobody can explain, and a ZIP code column has quietly lost its leading zeros across three years of history.

That is not a storage problem. It is a delivery problem. Most comparisons of Parquet, CSV, and JSON treat the three as places to put data, which assumes you control both ends. In analytics work that assumption breaks constantly. Someone else produces the data, you receive it, and everything that goes wrong between those two facts is invisible in a benchmark and expensive in production.

So this is the receiving end’s version. Six properties decide the format, five questions decide which one to ask for, and the failure modes are the ones that only appear at a handoff boundary.

What each format actually is, and what that costs you at read time

The useful separation is not text versus binary. It is what a reader has to do to get one column out.

CSV is row-oriented text. Values are laid out record by record, so pulling one column means reading every byte of every row and discarding most of it. Nothing describes types, because RFC 4180, the document that standardizes the format, defines no type system. Every value on the wire is text, and whether `007` is the integer 7 or the string “007” is decided by whatever reads it.

JSON is also row-oriented, but self-describing. Each record carries its own keys, so shape can vary between records and a reader handles that without a schema. Keys repeat on every record, and there are no statistics anywhere to help skip work. NDJSON, sometimes called JSON Lines, is worth naming separately, because for pipeline purposes it is not the same format: one record per line, no enclosing array, which makes the file splittable and streamable.

Parquet is column-oriented and binary. Rows are grouped into row groups, each row group splits into one column chunk per column, and every chunk is encoded and compressed independently. A footer carries the schema plus per-row-group statistics: min, max, and null count per column. Those statistics are the whole point. They let an engine read the footer, decide an entire row group cannot contain matching rows, and skip it without decompressing anything. That is predicate pushdown. Reading only the column chunks a query references is column pruning. Neither is available to a format with no metadata to consult.

The design has a cost the marketing rarely mentions. The Apache Arrow project states it plainly: Parquet data “cannot be directly operated on but must be decoded in large chunks.”

That explains most of Parquet’s weak spots. It is built for scanning many rows at once, not for reaching into a file for one of them. If your access pattern is a single-record lookup by key, you pay decode cost on a whole chunk to retrieve one row.

The six properties that decide the format

Those three descriptions explain the mechanics. They do not tell you which one to ask for, because that depends on what you are optimizing. Every format argument that goes in circles is one where the participants are optimizing different properties without saying so. Six matter for a delivery decision: size on the wire, scan cost, type fidelity, nesting, schema evolution, and inspectability. Two are worth a note before the table. Scan cost is not a performance metric on serverless engines, it is the invoice. Type fidelity fails silently, which is why it costs the most to find.

Property	CSV	JSON / NDJSON	Parquet
Size on the wire	Largest. Plain text.	Larger than CSV. Keys repeat per record.	Smallest. 5.1x smaller with Snappy, 6.8x with ZSTD, on the benchmark below.
Scan cost	Full file, every query.	Full file, plus parse cost.	Only referenced columns, and only row groups that pass the statistics check.
Type fidelity	None. Every value is text.	Partial. String, number, boolean, null. No decimal, timestamp, or integer width.	Strong. Types, precision, and nullability live in the file schema.
Nesting	None. Flat rows only.	Native, arbitrary depth.	Supported, but flattening is the practical route.
Schema evolution	Column position is the contract. Nothing enforces it.	Tolerant. A new key just appears.	Add and drop handled. Rename is not native.
Inspectability	Complete. Any text editor.	Complete for NDJSON, awkward for one large array.	None without a reader.

The pattern in that table is what makes the decision non-obvious. CSV and JSON are strong exactly where Parquet is weak, always on the same axis: human accessibility and shape tolerance on one side, machine efficiency and type guarantees on the other. No format wins both. Anyone telling you otherwise is describing a use case, not a format.

Why Parquet wins on analytics cost, and by how much

The size difference is large and easy to reproduce. A 2026 published benchmark ran 11,198,026 rows of public New York City taxi trip data, 20 columns, through three formats. CSV came out at 1.09 GB. The same data as Parquet with Snappy compression was 218 MB, which is 5.1 times smaller. With ZSTD it was 164 MB, 6.8 times smaller.

The query numbers from the same run, using DuckDB on a single machine with the operating system cache warm and timings taken as the median of three runs:

Query	CSV	Parquet with ZSTD	Speedup
Aggregate over one column	0.490 s	0.008 s	about 60x
GROUP BY aggregate	0.506 s	0.021 s	about 24x
Top 5 rows, all 20 columns	0.705 s	0.032 s	about 22x

The honest headline there is 22x, not 60x. That case reads all 20 columns, which is the fair comparison because CSV always reads all 20. The 60x case reads one column, and the gap is column pruning doing its job. The same benchmark shows `count(*)` running roughly 160 times faster, and that one should not be quoted at all, because it reads the row count out of the footer and never touches a data page.

The number that matters more is the one on the invoice. Amazon Athena bills on bytes scanned, rounded up to the nearest megabyte, with a 10 MB minimum per query. AWS documentation as of 2026 puts the saving at 30% to 90% per query from compressing, partitioning, and converting data to columnar formats. Their worked example: with 1 TB compressed 3:1, a query referencing one of three columns reads only that column and avoids two thirds of the file.

That range belongs to all three moves together, not the format alone. Converting to Parquet and leaving everything in one unpartitioned directory gets you part of the range, not the top of it. Sequence it: convert, then partition on the column you actually filter on, then tune file size.

There is a deeper reason the advantage is conditional rather than universal, and Daniel Abadi made the argument years before it became a cloud-billing question.

When CSV is still the right answer

The reflex answer to this article’s title is “Parquet, obviously,” and it is wrong often enough to state the exceptions precisely. Four conditions make CSV the correct engineering choice rather than a legacy compromise.

Every query reads every column, once. Column pruning has nothing to prune, so you pay Parquet’s encode and decode cost for a benefit you never collect.

The consumer cannot read Parquet. A finance team’s spreadsheet, a partner’s legacy loader, a regulator’s submission portal. A format the receiver cannot open has zero efficiency, whatever the benchmark says.

A human is in the review loop. In the first weeks of a new feed somebody needs to open the file and look at it, which is exactly when a delivery is least trustworthy.

The payload is small. Under a few megabytes, Parquet’s footer, schema, and per-chunk metadata stop being rounding errors. A 2 KB Parquet file can be larger than the same rows as CSV.

Now the cost, as a specification fact rather than a complaint. RFC 4180 defines no type system, no way to declare character encoding, no delimiter other than the comma, and no quote character other than the double quote. Everything else is convention, and the consequences are all silent. Aggressive type inference turns ZIP codes, phone numbers, SKUs, and account IDs into numbers and drops the leading zeros. An empty field and a quoted empty string are indistinguishable on the wire, so NULL and the empty string collapse into each other on the way into a database that distinguishes them. Values past 64-bit float precision lose digits without an error.

That is not a reason to refuse CSV. It is a reason to specify it. A CSV delivery contract should name the encoding, the quoting rules, and the null token, and ship a companion schema file with types. Ten minutes with the producer removes a whole class of silent corruption, the same discipline behind any working data quality framework.

Parquet is not always faster. Single-row lookups by key, streaming appends, and sub-megabyte payloads are all cases where Parquet loses to a text format. The footer must be read before anything else, and on tiny files that read dominates. Benchmark your access pattern, not the format.

When JSON is the only format that survives the handoff

CSV fails on types. Parquet fails on shape. JSON earns its place when the record, not the column, is the unit of meaning, which happens more than format-purist advice admits. Web extraction output where a product page carries three specification fields on one website and forty on another. Document processing output where each entity has its own confidence score and bounding box. Flattening to columns at the delivery boundary destroys information before you have decided what you need, which is one reason RAG pipelines fail in production for data reasons rather than model reasons.

The distinction that matters operationally is JSON versus NDJSON. A single top-level array cannot be split across workers or processed incrementally, because the parser does not know a record ended until it finds the closing bracket. Almost every JSON delivery problem in a pipeline is that problem. If you accept JSON, accept NDJSON.

The cost is real. Keys repeat on every record. There are no statistics to prune on, so every query parses everything. Type coverage is thin: string, number, boolean, null, with no decimal precision, timestamp, or integer width, so a currency amount arrives as a float and a timestamp as a string in whatever shape the producer chose.

The pattern most teams converge on is worth defending. Take the delivery in NDJSON, land it as-is, run one normalizing pass, write Parquet for everything downstream. The NDJSON stays as the record of what actually arrived, which is what you want when a downstream number looks wrong, and Parquet carries the query load. One extra step, and the right default for any feed whose shape is not yet stable, including most AI training data work.

The failures that show up at the delivery boundary, not in the benchmark

Everything to this point holds when you produce the file yourself. When somebody else produces it, four things you did not choose start deciding your outcomes, and none of them show up in a benchmark.

File layout you did not choose

Parquet’s efficiency depends on row group size, and the producer picks it. The Apache Parquet project’s current configuration guidance recommends large row groups, “512MB – 1GB”, and 8 KB data pages. Practitioner guidance in distributed engines runs lower, 128 MB to 512 MB. The tradeoff is direct: larger groups cut metadata overhead and give bigger sequential reads at the cost of memory during execution, smaller ones give better parallelism and finer skipping at the cost of more metadata.

The version that actually bites is the small-files problem. A producer that partitions by hour, or writes one file per API page, hands you thousands of small Parquet files. Every file carries its own footer and schema, so opening them dominates the work and the columnar advantage inverts. This is the most common way a Parquet delivery underperforms the CSV it replaced, and no benchmark shows it to you, because benchmarks use one well-formed file.

Schema evolution nobody versioned

Parquet handles added and dropped columns. It does not natively support renaming one. The standard workaround is to add a column with the new name and drop the old, which rewrites data. Type changes are readable across old and new files, but narrowing conversions lose precision quietly.

That limitation is why table formats exist. Apache Iceberg assigns every field a unique column ID and data files reference columns by ID rather than name, so a rename touches metadata only and no file is rewritten. Delta Lake reaches the same outcome, but column mapping has to be enabled on the table first.

The operational verdict: if the schema will move, a bare directory of Parquet files is a delivery artifact, not a table. Either put a table format over it, or make the producer version the schema and tell you when it changes. Detecting a rename after the fact is the same detection problem as any other silent break, which is why schema drift belongs in the same monitoring layer as freshness and volume. Teams already running data observability on third-party feeds have the alert. Teams receiving their first Parquet delivery usually do not.

Type coercion at the producer

Whatever library the producer writes with is deciding your types, and three break silently. Timestamp unit and timezone, invisible until an aggregation comes out wrong. Decimal versus float, where currency written as a float loses cents at scale. Integer width, where an ID that outgrows int32 wraps or errors deep in a backfill.

An unstated codec

ZSTD compressed the benchmark dataset to 164 MB against Snappy’s 218 MB, which is 25% less storage and transfer, but an older reader in the receiving stack may not support it. Name the codec in the contract rather than discovering it when a job fails.

All four are contract problems, not technology problems. Specify the delivery, in writing, before the first file lands.

{
  "format": "parquet",
  "compression": "zstd",
  "target_file_size_mb": 256,
  "row_group_size_mb": 256,
  "partition_by": ["event_date"],
  "timestamp_unit": "micros",
  "timestamp_timezone": "UTC",
  "decimal_columns": { "amount": "decimal(18,2)" },
  "null_encoding": "parquet_null",
  "schema_version": "2026-08-01.3",
  "schema_change_notice_days": 14
}

Absorbing this class of failure is most of what a managed extraction service is for. Forage AI handles selector drift, anti-bot evolution, and schema changes as part of the service, which moves the schema-change conversation out of your ingestion logs and onto the producer side. Delivery itself runs in CSV, JSON, XML, or any format specified for the engagement, over API access, direct download, or a cloud integration.

A decision path for delivery format

Comparison tables tell you what the formats are. They do not tell you what to ask for on Tuesday. Five questions, in order, and stop at the first that gives you an answer.

1. Does the record shape genuinely vary between records? If yes, NDJSON. Not JSON, NDJSON. Forcing variable-shape records into a fixed schema at the boundary destroys information you have not yet decided you can lose.

2. Can the consumer read Parquet? If no, CSV with a companion schema file, explicit encoding, and a documented null token. A format the receiver cannot open is not a format.

3. Will queries repeatedly touch a subset of columns? If yes, Parquet. This is what column pruning and predicate pushdown are built for.

4. Does the data change after it lands? Updates, deletes, backfills, corrections. If yes, Parquet under a table format such as Iceberg or Delta, not a bare directory.

5. Is a human reading it regularly? If yes, CSV, or Parquet plus a small CSV sample per delivery. The sample is cheap and it ends most incident calls faster than a query does.

Then six operating rules that make the answer enforceable. Specify the codec. Specify a target file size, because file count is a performance property and nobody defaults to a good one. Specify row group size in the same breath. Specify the partition column, and make it the one you filter on rather than the one that looks tidy in a bucket listing. Version the schema with a field in the data, not a note in a document. Agree a notice period before schema changes ship.

Verification is one command, and it should run on the first delivery rather than the first incident:

-- Convert with an explicit codec and row group size
COPY (SELECT * FROM read_csv_auto('raw/*.csv'))
  TO 'out/events.parquet'
  (FORMAT parquet, COMPRESSION zstd, ROW_GROUP_SIZE 1000000);

-- Verify what actually got written, per row group
SELECT row_group_id, compression, num_values,
       total_compressed_size, total_uncompressed_size
FROM parquet_metadata('out/events.parquet');

If the producer will not commit to those six lines, that is information about the delivery, and it is cheaper to have before the pipeline is built than after. It also decides whether an integration is a one-week job or a recurring one, which is why Forage AI works as an extension of your data team rather than handing over files and leaving the reconciliation to you. The same reasoning runs through our guide to data extraction automation.

Frequently asked questions

Is Parquet always better than CSV for analytics?

No. Parquet wins when queries read a subset of columns from a large dataset repeatedly, which is most analytics work. It loses on single-row lookups by key, on streaming appends, on payloads small enough that reading the footer dominates, and when nobody in the receiving stack can open it.

What is the difference between JSON and NDJSON for data delivery?

NDJSON puts one complete JSON record on each line with no enclosing array, which makes the file splittable, streamable, appendable, and readable with line-based tools. A single top-level JSON array has to be parsed as one unit. For any delivery larger than a few megabytes, NDJSON is the version you want.

Should I use Avro instead of Parquet?

Avro is row-oriented, so it fits write-heavy streaming and message payloads where whole records are read together. Parquet is column-oriented and better for analytical reads over many rows and few columns. Plenty of pipelines use Avro in transport and Parquet at rest. They are different positions in the same pipeline, not competitors.

How do I convert Parquet back to CSV, and when should I?

Any engine that reads Parquet exports CSV in one statement, DuckDB and Spark included. Do it when a downstream consumer has no Parquet reader, or when a human needs to inspect a delivery. Expect to lose type fidelity on the way out, so treat the CSV as a view of the data rather than a replacement.

Is Arrow a replacement for Parquet?

No, they are complementary. Parquet is a disk format optimized for space through compression and encoding. Arrow is an in-memory format optimized for direct computation without decompression. Store in Parquet, decode into Arrow, which is the position both the Arrow project and Wes McKinney have stated.

Where this leaves you

The format question looks like a technical preference and behaves like a contract term. The formats are well understood and the benchmarks are reproducible. What is not written down, on most teams, is which format arrives under which codec, at which file size, with which timestamp precision, and what happens when the producer renames a column.

Pick one active feed this week and write those six things down. If the producer confirms them, you have a delivery contract. If they cannot, you have found the thing that will break in the third month, while it is still cheap to fix. Then do the next feed. The decision path works the same whether the data comes from a vendor, another team, or a pipeline you inherited.

Sources

• MotherDuck (2026): CSV vs Parquet benchmark on 11,198,026 rows of public NYC TLC taxi data, DuckDB v1.5.5. motherduck.com
• Amazon Web Services (2026): Amazon Athena pricing and performance tuning, bytes-scanned billing and columnar conversion savings. aws.amazon.com
• Apache Parquet project (2026): File format configurations, row group and page size recommendations. parquet.apache.org
• Apache Arrow project (2026): Project FAQ on the relationship between Arrow and Parquet. arrow.apache.org
• Daniel Abadi (2017): Apache Arrow vs Parquet and ORC, on disk-versus-memory compression tradeoffs, DBMS Musings. dbmsmusings.blogspot.com
• Apache Iceberg project (2026): Schema evolution and column-ID based renames. iceberg.apache.org
• Databricks (2026): Update table schemas with schema evolution, column mapping in Delta Lake. docs.databricks.com

Related Articles

• Data Observability for Third-Party Datasets. How to catch schema drift, freshness gaps, and silent failures in data you did not produce.
• Data Quality Framework for External Sources. The validation layers and QA checklist to run on every incoming delivery.
• Automated Data Collection. Reference architecture for enterprise extraction pipelines, from orchestration through validation gates.
• What Is Data as a Service?. Delivery models, evaluation criteria, and how managed data delivery compares to in-house infrastructure.