Inside OpenAI’s PostgreSQL Strategy: Scaling a Single Primary to 800 Million Users

OpenAI is running ChatGPT and its API platform — serving around 800 million users — on a single primary PostgreSQL instance. Not a distributed SQL cluster, not a heavily sharded fleet, but one write primary with nearly 50 read replicas distributed across regions. That deployment handles millions of queries per second, with p99 latency in the low double-digit milliseconds and five-nines availability.

For data and infrastructure engineers, the significance isn’t that everyone should copy this architecture. It’s that OpenAI’s design choices are grounded in workload characteristics and operational constraints, not scale anxiety or infrastructure fashion. Their experience shows how far a mature relational database can be pushed through deliberate optimization and disciplined operations.

The surprising choice: single-primary PostgreSQL at massive scale

OpenAI disclosed details of its PostgreSQL deployment in a technical blog post, emphasizing that PostgreSQL has “for years” been one of the most critical systems behind ChatGPT and the company’s API. Over just the last year, their PostgreSQL load has grown by more than 10x and continues to rise quickly.

The headline architecture looks deceptively simple:

• One Azure PostgreSQL Flexible Server acting as the single write primary
• Almost 50 read replicas, spread across multiple regions, serving read traffic
• Millions of queries per second across this topology
• p99 latency in the low tens of milliseconds
• Approximately 99.999% availability (“five nines”)

Conventional wisdom suggests that once you reach this magnitude — hundreds of millions of users, huge query volumes — you should either:

• Shard PostgreSQL across multiple primaries to distribute writes, or
• Move to a distributed SQL database purpose-built for horizontal scale

Many organizations would have started down one of those paths long before hitting OpenAI’s scale. Both sharding and distributed SQL can remove the single-writer bottleneck, but at the cost of increased complexity: routing logic, distributed transactions, and heavier operational overhead.

OpenAI chose instead to stretch a single-primary PostgreSQL instance as far as possible through optimization, while selectively offloading workloads that don’t fit. That decision reframes a common scaling question: “When do we need to re-architect?” into “Which specific workloads actually require a different system?”

Why OpenAI still leans on PostgreSQL

PostgreSQL sits on the hot path for ChatGPT and OpenAI’s API, handling operational data for user interactions and platform behavior. The workloads it serves are heavily read-dominant. That matters because it plays to PostgreSQL’s strengths and mitigates some of its known write-scaling challenges.

PostgreSQL’s multiversion concurrency control (MVCC) preserves transactional isolation by keeping multiple versions of rows. Updates don’t overwrite in place; they create new row versions while older ones remain until vacuumed. At very high write rates, this leads to:

• Write amplification: Updates effectively behave like inserts plus additional cleanup work.
• Version bloat: Queries may need to scan through multiple row versions to find the current one.

Those trade-offs become critical at the scale OpenAI operates. Instead of fighting MVCC or expecting PostgreSQL to be something it is not, OpenAI allows the database’s characteristics to shape architectural decisions. Some workloads remain on PostgreSQL because they are predominantly read-heavy and operationally well understood. Others, especially write-heavy or naturally partitionable workloads, are moved elsewhere.

The key message for enterprises: PostgreSQL remains a strong fit for large, read-oriented operational workloads — including many AI-centric scenarios — as long as its write-path constraints are respected and managed.

Hybrid data strategy: PostgreSQL plus sharded systems

Rather than committing to a single database paradigm, OpenAI has adopted a hybrid approach. The guiding rule is blunt: no new tables in the central PostgreSQL primary for emerging workloads that are likely to be write-heavy or easily partitioned.

The strategy looks like this:

• Core, read-heavy operational data stays on the single-primary PostgreSQL instance, optimized aggressively.
• New workloads default to sharded or horizontally scalable systems such as Azure Cosmos DB, rather than being added to the central relational database.
• Existing write-heavy workloads that can be horizontally partitioned are gradually migrated out of PostgreSQL into those sharded systems.

This approach splits the difference between three extremes:

• Fully sharding PostgreSQL and taking on the complexity of multi-primary coordination
• Rewriting everything to a distributed SQL engine
• Letting a single monolithic database absorb every new feature until it collapses

For data and platform teams, this provides a pragmatic template: keep proven, reliable systems in place for the workloads they handle well, and route only specific pressure points — typically high-write or naturally sharded domains — onto specialized infrastructure. The emphasis is on selective migration rather than wholesale re-architecture.

Key optimizations behind OpenAI’s PostgreSQL deployment

OpenAI’s success at this scale is not the result of a single magic technique, but of many targeted optimizations stacked together.

1. Connection pooling
Connection establishment overhead can dominate latency and resource usage when traffic spikes. By introducing connection pooling, OpenAI reduced connection time from about 50 milliseconds to around 5 milliseconds. That 10x improvement not only reduces tail latency but also smooths load on the primary server.

For engineers, this underscores that connection management is a first-class scaling concern. The operational win here is not exotic: it’s using pooling effectively and tuning it to fit production traffic patterns.

2. Cache locking to prevent thundering herds
In high-traffic systems, cache misses for popular objects can trigger a “thundering herd”: hundreds or thousands of concurrent requests simultaneously falling back to the database. OpenAI addressed this via cache locking mechanisms that serialize or coordinate misses, so that only a limited number of requests hit PostgreSQL when a popular cache entry expires.

This keeps the primary from being overwhelmed during transient cache events and is particularly important with unpredictable traffic patterns like those common in AI applications.

3. Read replicas across regions
With nearly 50 read replicas deployed in multiple regions, OpenAI offloads a significant portion of traffic from the primary. The architecture exploits the read-heavy nature of the workload and allows lower-latency access for geographically distributed users, while preserving a single source of truth for writes.

Combined, these optimizations demonstrate how much capacity can be unlocked from a mature relational engine without changing the core consistency model or the overall database technology.

Operational discipline: limits, timeouts, and isolation

Equally important to OpenAI’s scaling story is operational discipline — strict constraints that protect the database from well-intentioned but risky changes and behaviors.

1. Layered defenses for reliability
OpenAI builds safeguards at multiple levels:

• Cache locking to contain thundering herds
• Connection pooling to control connection overhead
• Rate limiting at the application, proxy, and query layers
• Workload isolation so low-priority or experimental features do not share critical instances with core services

That last point matters operationally: newly deployed features that are not yet performance-tuned are prevented from degrading the reliability of mature, critical paths by being routed to separate instances.

2. Schema-change constraints
OpenAI permits only lightweight schema changes on the main PostgreSQL instance. Any change that would trigger a full table rewrite is disallowed. In addition:

• Schema-change operations have a strict 5-second timeout.
• Long-running queries are automatically terminated to avoid blocking maintenance and routine operations.

By placing time and impact limits on DDL and slow queries, the team reduces the chance that seemingly minor changes will unexpectedly impact availability or latency.

3. Throttled backfills
Data backfills are heavily rate-limited — sometimes to the point where an operation might take more than a week to complete. This reflects a deliberate decision: protect the live system even if it means slower background progress.

For data engineers and SREs, this is a clear statement of priorities. Backfills, schema evolutions, and heavy batch operations are treated as guests in the system, never allowed to compete aggressively with user-facing traffic.

ORMs and hidden scaling traps

Object-Relational Mapping frameworks like Django, SQLAlchemy, and Hibernate are widely used to speed up application development. They simplify data access but can obscure the actual SQL hitting the database. OpenAI’s experience highlights why this is dangerous at scale.

In production, the team discovered that one ORM-generated query — a complex join across 12 tables — was responsible for multiple high-severity incidents when traffic spiked. The query’s structure and cost were effectively hidden behind application abstractions until the system was under pressure.

The lesson for engineering teams is not that ORMs are unusable at scale, but that:

• ORM-generated SQL must be inspected, measured, and monitored in production.
• Expensive joins and implicit N+1 patterns can become critical failure points under load.
• Operational visibility into generated queries should be treated as part of the performance and reliability toolkit.

OpenAI’s incidents around ORM-generated queries underline the value of routinely auditing production SQL and setting guardrails (for example, query timeouts and cost-based checks) that catch pathological queries before they become outages.

What this means for your architecture decisions

OpenAI’s PostgreSQL story does not claim that a single primary is always the right answer. Instead, it reframes some commonly held assumptions about scaling:

• Read-heavy workloads can scale further on a single primary than many teams assume. User count alone is a poor proxy for when sharding or distributed SQL is required.
• Hybrid strategies are often more practical than full migrations. Moving only write-heavy or naturally sharded workloads off of PostgreSQL can avoid multi-year rewrites.
• Operational discipline is as important as architecture. Connection pooling, cache control, rate limits, strict schema rules, and long-query enforcement are as fundamental as the choice of database engine.
• AI workloads often fit the pattern. Many AI applications serve read-heavy, bursty traffic against a relatively stable set of operational data — a scenario where this style of PostgreSQL deployment can perform well.

For data engineers, backend developers, and infrastructure architects, the key takeaway is to base database decisions on measured workload patterns and clearly identified bottlenecks, not on generic thresholds or fear of future scale. Proven systems like PostgreSQL can go a long way when you optimize deliberately, set hard operational limits, and migrate selectively rather than reflexively.

Wholesale re-architecture is sometimes necessary, but OpenAI’s experience suggests it is not the default answer to every scaling challenge.