Data Gravity Resists Migration
“The cost of migrating a dataset grows super-linearly with its size: not because the migration itself is hard, but because every other system that has coupled to it must be migrated simultaneously. ”
Once a database reaches 1TB with 50 downstream consumers, changing its schema, location, or technology requires coordinating 50 systems in addition to the database itself. Each new service that reads from a database adds a migration dependency. After three years, a central PostgreSQL instance may have 20 services reading from it, each with subtly different assumptions about the data model. The dataset has acquired gravitational pull: changing it requires moving everything in its orbit at the same time.
Why It Matters
Data gravity is the mechanism by which good early architectural decisions become catastrophically expensive to change. A single PostgreSQL database that starts as a clean, well-owned system becomes a shared dependency after the third team queries it directly. By year three, direct table access from application code has created an implicit contract between every query in every service and the exact column names, types, nullable constraints, and index behaviors of that table. This is not a schema contract that anyone designed: it accumulated one query at a time.
Migration cost has five components, each of which compounds with the number of consumers. Data transfer is the smallest component: I/O and network are expensive but predictable. Schema transformation is often non-trivial: the target system may not support all the data types or constraints of the source. The dual-write period, where both old and new systems must be kept in sync, requires maintaining parity while both systems accept writes. Validation must prove data parity across 1TB of data with zero tolerance for divergence. And cutover coordination requires every consumer to switch simultaneously, or to maintain dual read paths indefinitely during a rolling migration.
The GitHub MySQL-to-scale migration required gh-ost and dual-write sequenced across hundreds of services over years, not weeks. This is not an outlier. It is what data gravity looks like when you try to move a system that has become the gravitational center of a large service mesh. The lesson is not that migration is impossible: it is that the cost is set at schema inception, not at migration time.
Failure Modes
- ·Consumer coupling accumulation making schema changes require coordinated deployment across dozens of services
- ·Dual-write divergence during migration causing data inconsistency between old and new systems
- ·Migration validation gap leaving undiscovered data drift between source and target systems
- ·Partial cutover leaving some consumers on the old system and some on the new, creating split-brain data reads
- ·Rollback unavailability after cutover because the source system was decommissioned before parity was fully validated
Amplification Risks
- ⚡Each consumer added without enforcing API boundaries increases migration complexity by one coordinated deployment unit
- ⚡Schema drift during migration, where the source schema changes while migration is in progress, can invalidate transformation logic
- ⚡A failed cutover with partial consumer migration creates a permanent split-brain state that requires emergency rollback
Temporal Behavior
- ⟳Data gravity increases monotonically with time as more consumers are added and more code is written against the schema
- ⟳The dual-write period extends migration duration and creates a consistency risk window proportional to write volume
- ⟳Consumer migration can take months for large systems: the migration is a multi-month operational state, not a deployment event
Boundary Implications
- ◈Domain ownership boundaries prevent data gravity from forming across domain lines: the API is the coupling surface, not the database
- ◈The migration boundary must be explicit: define exactly which consumers have switched and which have not, with no ambiguity
- ◈Failure isolation boundaries break down when multiple domains share a database: a data corruption event affects all consumers simultaneously
Topology
- ·High-fan-out read topology creates high data gravity: every read edge is a migration dependency
- ·Topology paths that bypass domain APIs and read directly from the database create implicit schema contracts
- ·The migration blast radius is proportional to the number of service nodes connected to the data source in the topology graph
Scaling
- ·Migration complexity scales with both dataset size and number of downstream consumers: the interaction term is the dangerous one
- ·As datasets grow, dual-write validation costs grow proportionally: parity checking 1TB takes significantly longer than 100GB
- ·Horizontal scaling of consumers increases migration dependency count without increasing migration team capacity
Resilience
- ·Systems with high data gravity are fragile during infrastructure migrations: a rollback requires reversing all consumer cutover simultaneously
- ·API-mediated data access creates natural migration bulkheads: consumers can be migrated independently
- ·Validated dual-write with parity checking increases migration resilience by catching divergence before cutover
Governance Implications
- ·Direct table access from application code must be prohibited after the initial service reaches production: API contracts only
- ·New consumer onboarding to a shared database should trigger a data gravity assessment
- ·Domain ownership enforced at the API layer prevents consumer coupling from accumulating in the first place
Evolution Implications
- ·Every new consumer added to a shared database increases the future migration cost: this cost must be accounted for at consumer onboarding
- ·Strangler fig pattern applied to databases requires explicit consumer migration sequencing before source decommission
- ·Migrating from a monolithic database to domain-owned databases requires API-first refactoring before data migration
Mitigation Patterns
- →Enforce API contracts as the consumer interface: prohibit direct table access from services outside the owning domain
- →Use database-per-service or bounded context ownership to limit data gravity accumulation per service
- →Use strangler fig pattern for gradual migration: add new API, migrate consumers one at a time, decommission old system last
- →Implement dual-write with parity validation before any consumer cutover: never cut over on assumed parity
- →Maintain a consumer inventory for every shared database: know the full migration blast radius before beginning
Cross-References