Notion PostgreSQL Blocks Table Partitioning
Notion sharded its multi-billion-row blocks table by workspace across 32 separate Postgres databases (application-level sharding, not native partitioning) after autovacuum could no longer keep up with the growth rate of a single monolithic table.
Notion's entire content model is block-based: every paragraph, heading, database row, and embed is a row in a blocks table. By 2021, this table had grown to billions of rows, causing autovacuum to fall behind, query planner estimates to degrade, and table bloat to accumulate. The solution was application-level sharding by workspace across 32 separate Postgres databases (480 logical shards); Notion deliberately rejected native and third-party declarative partitioning to keep routing in application code. The migration itself: moving a live multi-billion-row table to the sharded schema: required months of careful dual-write and backfill execution.
Scale at Decision Point
Users
Tens of millions of users; exact count not disclosed at decision point (~2021)
Data Volume
Billions of rows in a single blocks table; table bloat from failed autovacuum cycles
Request Rate
Not disclosed; predominantly read-heavy with bursty writes on collaborative editing
PostgreSQL on AWS RDS; multiple read replicas for query distribution
Architecture Evolution
Initial Architecture
Single PostgreSQL blocks table containing all content for all users. Autovacuum scheduled for the entire table could not keep pace with write volume. Query planner row estimates degraded as table statistics became stale at billion-row scale.
- Autovacuum cannot keep up with write rate: dead tuple accumulation causes table bloat
- Query planner statistics become stale at billion-row scale: plan quality degrades
- Full table statistics collection takes too long to be useful
- Index size grows proportionally: eventual RAM exhaustion for hot index pages
Evolved Architecture
Application-level sharding of the blocks table by workspace across 32 Postgres databases (480 logical shards). Each shard covers a subset of workspaces, enabling autovacuum to run independently per shard at manageable pace. Redis caches hot block reads to reduce PostgreSQL load.
- Cross-partition queries require explicit routing logic: no transparent cross-partition joins
- Partition key selection is permanent: repartitioning a live table is operationally expensive
- Shard routing is maintained in application code: no native query router across shards
Key Transitions
Trigger
Autovacuum could not complete cycles on the monolithic blocks table fast enough to prevent dead tuple accumulation. Table bloat caused progressive query plan degradation and occasional full-table scan fallbacks.
Before
Single monolithic blocks table with billions of rows; autovacuum falling behind
After
Declarative partitioned blocks table by space_id; per-partition autovacuum
Outcome
Autovacuum became manageable per partition. Query planner statistics accurate per partition. Hot block reads served from Redis cache, reducing PostgreSQL I/O. Migration took months of careful dual-write, backfill, and validation.
Lessons
- Plan for partitioning before you need it: retrofitting a live billion-row table is extremely expensive
- Autovacuum behavior at table scale must be monitored proactively, not reactively
- Dual-write plus backfill is the only safe migration path for live high-write tables
Key Lessons
Design your partitioning key before your table reaches 100M rows
Notion's blocks table was designed without partitioning and grew to billions of rows. The migration required months of engineering effort on a system that could not afford downtime. Had partitioning been in place at table creation, autovacuum and statistics would have remained manageable throughout.
Applicable when: You have a high-write table that will grow beyond 500M rows over its lifetime
Autovacuum is not infinitely scalable: it is a resource-constrained background process
PostgreSQL's autovacuum runs continuously but is bounded by I/O and compute. A single table receiving billions of writes will eventually outpace autovacuum's capacity to reclaim dead tuples, causing progressive bloat and planner degradation.
Applicable when: Your PostgreSQL table write rate is high enough to challenge autovacuum completion cycles
Redis caching for hot entity reads is a prerequisite at collaborative product scale
Notion added Redis caching for block reads as part of this migration to reduce PostgreSQL read I/O pressure. Without it, the partitioned table would still have faced read amplification from collaborative workloads.
Applicable when: Your hot data fits in memory and read access patterns are predictable by entity key