I’ve been spending some time lately learning more about data analytics - partly out of curiosity, partly because I work as a developer at a private limited company and wanted to understand the pipeline better beyond just writing application code. Somewhere in that process, one term kept showing up again and again: Parquet.
At first, I skimmed past it. I already knew how to export data as CSV and figured that was good enough for most use cases. But the more I read about how real analytics systems handle large datasets, the more Parquet kept coming up as the default choice, not CSV. That was enough to make me stop and actually dig into why.
Why Not Just Use CSV?
CSV makes sense on the surface - it’s simple, readable, and every tool supports it. But once I understood how it’s structured, the limitations became obvious. CSV stores data row by row, so even if you only care about two columns out of twenty, there’s no way around reading the entire file top to bottom.
It also doesn’t compress well, since each row mixes together completely different types of data - text, numbers, dates - right next to each other. And there’s no schema stored in the file itself, so every tool that opens it has to guess, or be told separately, what type each column is supposed to be.
For small datasets, none of this really matters. But I learned that once you’re dealing with large tables and running the same analytical queries over and over, these small inefficiencies start to add up into real time and cost.
What Parquet Does Differently
Parquet takes the opposite approach - it stores data column by column instead of row by row. Once I understood that one idea, a lot of its advantages made sense on their own:
- It only reads what’s needed. A query touching two columns only reads those two columns, not the whole file.
- It compresses much better. Similar values sitting next to each other - all the timestamps together, all the status codes together - compress far more efficiently than a row of mixed data types.
- The schema is built in. Column names and types travel with the file, so there’s no ambiguity about what a column contains.
- It’s already the standard. Tools like Spark, Athena, BigQuery, and even pandas in Python all read Parquet natively.
To actually see this in practice rather than just reading about it, I tried it out with a small setup of my own - a PostgreSQL table and a Node.js script to export it, first as CSV and then as Parquet, just to compare.
CSV vs. Parquet, Side by Side
| CSV | Parquet | |
|---|---|---|
| Storage layout | Row by row | Column by column |
| Reading one column | Reads the entire file | Reads only that column |
| Compression | Minimal | Strong - similar values grouped together |
| Schema | Not included | Stored in the file |
| Human-readable | Yes | No (binary format) |
| Typical file size | Larger | Noticeably smaller for the same data |
| Best suited for | Small files, quick manual checks | Large datasets, repeated analytical queries |
CSV still has its place. It’s the easiest format to open and check quickly when you just need to glance at some data. But for anything meant to be queried repeatedly at scale, the columnar approach clearly wins.
What I Took Away From This
What started as a small side-topic while learning analytics turned out to explain something I’d taken for granted for a while - the file format you export data in isn’t just a minor technical detail. It directly affects how fast queries run and how much you end up paying for storage and compute later on.
If you’re getting into analytics or working with data exports from PostgreSQL (or any database), it’s worth taking the time to understand why Parquet is so widely used before defaulting to CSV out of habit.