What to do with a huge CSV file

A CSV that crashes Excel — a few gigabytes, tens of millions of rows — is awkward precisely because the obvious tools give up on it. You have three good moves: view it as-is in your browser, split it into smaller files, or load it into SQLite and query it with SQL. Huge CSV does all three; the last one is where most of the leverage is, so this guide walks the SQLite path end to end, with the exact commands for Windows, macOS and Linux.

huge csvsqlitesplitsqlhow-to

Three things you can do

Just view it (zero setup)

If all you need is to look at the file — read rows, search for a value, jump around — you don't have to convert anything. Drop the CSV onto hugecsv.com and it streams in your browser: the rows render with a virtual scroller, so a 20 GB file scrolls as smoothly as a small one, and nothing is uploaded — the file never leaves your machine. Use the filter box to narrow rows and the in-place find to jump to a value. When that's enough, you're done. When you need to compute over the data, keep reading.

Split it into smaller files

Sometimes the file isn't the problem — the next tool in line is. An importer caps uploads at 2 GB, a teammate's spreadsheet dies past a million rows, a script wants one file per worker. Splitting keeps the data as CSV but breaks it into manageable pieces.

In Huge CSV, open the file, then use the Export & split button (the download icon in the table toolbar). Under Split CSV, pick how many files you want — Into N files, anywhere from 2 to 10,000 — and press Split. You get N files of roughly equal size, each one a valid CSV that carries the header row, and every split lands on a clean record boundary, so a row is never cut in half (even rows with quoted commas or embedded newlines stay intact). On a Chromium browser it writes the parts straight into a folder you choose; elsewhere it downloads them. Like everything else here, the split streams in 1 MiB blocks, so it works on 20 GB files without loading them into memory.

Load it into SQLite for real queries

This is the big one. SQLite is a tiny, free, single-file SQL database that ships on every Mac and installs in seconds everywhere else. Once your CSV is inside it, "how many rows have status = 'active'?" or "sum revenue per country" is a one-line query instead of a script — and it stays fast over tens of millions of rows.

The example file

Say you have customers.csv — a 12 GB export with a header and ten columns:

id,first_name,last_name,email,country,city,signup_date,plan,last_login,lifetime_value
1,marcel,dupont,marcel.d@example.com,FR,Lyon,2021-03-04,pro,2026-06-18,1840.50
2,aisha,khan,aisha.k@example.com,PK,Karachi,2022-11-20,free,2026-06-20,0.00
3,marcel,boyer,m.boyer@example.com,CA,Montreal,2020-07-09,team,2026-05-30,9120.00
… tens of millions more rows …

Step 1 — export a .sql dump from Huge CSV

Open customers.csv in the viewer, click Export & split, and choose Export SQLite. You'll get a file named customers.sql.

One honest detail worth understanding: that .sql file is not a binary .sqlite database — it's a plain-text SQL dump. It contains one CREATE TABLE statement followed by batched INSERT statements, exactly like the dumps SQLite itself produces. Here's the shape of it:

CREATE TABLE "customers" (
  "id" TEXT,
  "first_name" TEXT,
  … one TEXT column per CSV header …
  "lifetime_value" TEXT
);
INSERT INTO "customers" ("id", "first_name", …) VALUES
  ('1', 'marcel', …),
  ('2', 'aisha', …),
  … up to 500 rows per INSERT, then the next batch … ;

Why a text dump instead of a ready-made database file? Because a binary .sqlite would mean holding the whole database in memory in the browser to build it — impossible for a multi-gigabyte file. A SQL dump streams: Huge CSV writes it row-batch by row-batch and never holds more than a small buffer, so the same export works on a 50 MB file and a 20 GB one. The one-time cost is that you run an import — which is the next step, and it's a single command.

Step 2 — import the dump into SQLite

Importing means running the statements in customers.sql against a fresh database file, customers.db. Pick your platform.

macOS

sqlite3 already ships with macOS — nothing to install. In Terminal, cd to the folder with the dump and run:

# reads customers.sql and builds customers.db next to it
sqlite3 customers.db < customers.sql

Linux

Install the CLI once, then the same import:

# Debian / Ubuntu
sudo apt install sqlite3
# Fedora / RHEL:  sudo dnf install sqlite
# Arch:           sudo pacman -S sqlite

sqlite3 customers.db < customers.sql

Windows

Install the CLI (via winget, or download the "sqlite-tools" bundle from sqlite.org and unzip it). PowerShell doesn't support the < input redirect that macOS and Linux use, so import with SQLite's own .read command instead:

# PowerShell
winget install SQLite.SQLite
sqlite3.exe customers.db ".read customers.sql"

# (Old cmd.exe does support the redirect:
#   sqlite3.exe customers.db < customers.sql )

After any of these you'll have a single file, customers.db, holding every row. That import is the slow part on a huge file — it's a one-time pass; querying afterward is fast.

Step 3 — query it with SQL

Open an interactive SQL shell on the database:

sqlite3 customers.db

You'll get a sqlite> prompt. Now the two questions from the top. Count the customers named "marcel":

SELECT COUNT(*) FROM customers WHERE first_name = 'marcel';

That returns a single number — the count of rows whose first_name is exactly marcel. String matching is case-sensitive by default, so to also catch Marcel and MARCEL, add COLLATE NOCASE:

SELECT COUNT(*) FROM customers WHERE first_name = 'marcel' COLLATE NOCASE;

Print the first ten rows, all columns:

.headers on
.mode column
SELECT * FROM customers LIMIT 10;

The two dot-commands aren't required, but .headers on prints the column names and .mode column lines the output up in neat columns instead of pipe-separated text — much easier to read with ten fields. When you're done, type .quit.

Two tips for big tables

First, if you'll filter on first_name a lot, build an index once and every later lookup is near-instant instead of scanning every row:

CREATE INDEX idx_customers_first_name ON customers(first_name);

Second, remember every column is TEXT. To compare the numeric lifetime_value — say, the high-value customers — cast it so SQLite compares numbers, not strings (otherwise '9' sorts after '1840.50'):

SELECT first_name, last_name, lifetime_value
FROM customers
WHERE CAST(lifetime_value AS REAL) > 1000
ORDER BY CAST(lifetime_value AS REAL) DESC
LIMIT 10;

From here it's ordinary SQL — GROUP BY country, SUM, joins to other tables, the lot. The CSV that wouldn't open in a spreadsheet is now a queryable database.

Which one should you pick?

If you only need to read the file — eyeball it, search it, scroll it — view it in Huge CSV and stop there; it's instant and uploads nothing. If a downstream tool chokes on the size, split it into pieces that still carry the header. And if you need to ask questions of the data — counts, filters, aggregates over millions of rows — export the .sql dump and load it into SQLite. The same viewer does all three, so you can start by just looking and reach for SQLite only when a question demands it.