Postgres as the Everything Database: When Consolidation Works and When It Breaks

The strongest architectural current in the data layer is consolidation onto Postgres, and it is happening because the operational math changed, not because Postgres got fashionable. A team that would have run five systems in 2018, a relational database, a vector store, a message broker, a document store, and a search cluster, can now run one: pgvector for embeddings, SKIP LOCKED for job queues, JSONB for schemaless documents, native logical replication instead of change-data-capture middleware, and built-in full-text search. Each substitution is individually worse than the specialist it replaces on the dimension that specialist optimizes for. Collectively they are frequently better, because a system that is eighty percent as good on five axes and needs one backup story, one security model, one on-call rotation, and one set of transactional guarantees beats five best-in-class systems that must be kept consistent with each other by hand. The question is not whether consolidation works. It is where each substitution stops working, and whether an organization has named that threshold in advance or intends to discover it in production.

Key Takeaways

• The consolidation case is operational, not technical. Every Postgres substitution is weaker than the dedicated system on that system's home turf. The win comes from eliminating the seams: one backup and restore path, one security model, one set of transactional guarantees, and no distributed consistency problem between components that used to be separate.
• The strongest single argument is transactional consistency. Enqueuing a job in the same transaction as the row that caused it removes an entire class of dual-write bug that no amount of care fully eliminates when the queue lives in another system.
• Each substitution has a measurable ceiling, and they are not the same ceiling. Vector search degrades on recall and index build time at high dimensionality and volume. Queues hit autovacuum and bloat pressure long before they hit raw throughput limits. Full-text search loses on relevance sophistication well before it loses on corpus size.
• The binding constraint on Postgres consolidation is usually the single write primary, not any individual workload. Reads scale out through replicas; writes do not, and every consolidated workload competes for the same write path and the same autovacuum budget.
• The correct policy is consolidate by default below a stated scale, and split only on a measured bottleneck with the trigger written down in advance. Splitting on anticipation produces five systems and the distributed consistency problems that come with them, purchased before the problem they solve exists.

Why the Pull Is Real

Three things changed at once, and the combination is what makes this a genuine architectural shift rather than a preference.

Hardware outgrew the intuitions. Sizing instincts formed when a large server had modest memory and spinning disks. A single commodity node now carries hundreds of gigabytes of RAM and fast local storage, which means the working set for a very large number of real applications fits in memory on one machine. Architectures designed around the assumption that one database could not hold the load were solving a problem that has since moved.

The extensions got good. Postgres is an extensible engine rather than a fixed feature set, and the extension ecosystem closed the gaps that used to force a specialist purchase. Vector similarity search, time-series compression, geospatial indexing, and distributed sharding all arrive as extensions to the same engine, sharing its transactions, its backups, and its access control.

The seams got expensive. The hidden cost of five systems is not five licenses, it is the integration code between them and the failure modes that live in the gaps. When a row is written in the database and a job is enqueued in a broker, those two operations are not atomic, and every team that has run that architecture has written the reconciliation logic that copes with the moment when one succeeded and the other did not. Consolidation deletes that code and the bugs in it.

The transactional point deserves separate weight because it is the argument that does not depend on scale. A queue inside the database can be written in the same transaction as the state change that produced it. Either both commit or neither does. The outbox pattern exists precisely to simulate this property across a system boundary, and it is a workaround for a problem that consolidation removes rather than manages. Teams evaluating whether to introduce a broker at all should read the substitution question alongside the broader analysis of when event-driven architecture is worth it, because a great deal of what looks like a messaging requirement is a job queue with ambitions.

The Substitutions, Honestly

Vector search: pgvector instead of a vector database

pgvector adds a vector column type and approximate nearest-neighbor indexing to Postgres, with HNSW and IVFFlat index types covering the standard recall and speed trade-offs. For a corpus in the low millions of embeddings with moderate query concurrency, it is genuinely sufficient, and it carries one decisive advantage a dedicated store cannot match: the vectors sit in the same transaction and the same join as the relational data describing them. Filtering a similarity search by tenant, permission, date, and status is an ordinary SQL predicate rather than a metadata filter bolted onto a specialized index.

Where dedicated vector databases win is at the extremes: very large collections, high sustained query throughput, sophisticated quantization to fit memory budgets, and distributed sharding across nodes. Index build time is the ceiling most teams hit first, because building an HNSW index over a large collection is expensive and it competes with production traffic on the same machine. The trade-offs that justify a separate system are laid out in when you actually need a vector database, and the underlying mechanics in vector search and embeddings. The practical rule is that retrieval quality problems are almost never solved by changing vector stores, so a team struggling with relevance should not treat migration as the fix.

Queues: SKIP LOCKED instead of a broker

A job queue on Postgres is a table plus SELECT ... FOR UPDATE SKIP LOCKED, which lets concurrent workers claim distinct rows without blocking each other. It is straightforward, it is transactional with the rest of the application's writes, and it handles far more throughput than most teams assume, comfortably into the thousands of jobs per second on ordinary hardware.

The ceiling is rarely raw throughput. It is table churn. A queue table where every row is inserted, updated, and deleted within seconds generates dead tuples at a rate that pressures autovacuum, and a queue table that bloats will degrade the entire database, not just the queue. This is the failure that surprises teams, because it appears as general database slowness rather than as a queue problem. Aggressive autovacuum tuning on the queue table specifically, and partitioning by status or time, are the standard mitigations.

The genuine architectural limits are semantic rather than numeric. A Postgres queue is a work queue: claim, process, delete. It is not a durable log. If the requirement is replay from an arbitrary point, multiple independent consumer groups reading the same stream at different offsets, or retention of the event history as a system of record, that is log semantics and a broker is the right tool. Wanting those properties is a legitimate reason to add Kafka. Wanting higher throughput usually is not.

Documents: JSONB instead of a document store

JSONB stores structured documents with GIN indexing over their contents, which covers the schemaless-field use case that drove a great deal of document database adoption. The advantage over a separate document store is that flexible and relational data coexist in one query: a strict schema for the parts that need integrity, a JSONB column for the parts that vary, joined in a single statement with real foreign keys.

Two limits matter. Updates to a JSONB value rewrite the whole value rather than patching it in place, so a workload with large documents and frequent small updates generates far more write amplification than the equivalent in a document store built for it. And very large documents cross into out-of-line storage, adding indirection that shows up as latency. JSONB is excellent for documents that are read often, written occasionally, and measured in kilobytes. It degrades for documents that are large, hot, and partially updated many times per second.

Change data capture: logical replication instead of middleware

Postgres logical decoding exposes committed changes as a stream, and native publications and subscriptions move them between Postgres instances without additional infrastructure. For Postgres to Postgres replication, this removes an entire middleware tier.

It stops being sufficient when the target is not Postgres. Streaming changes into a warehouse, a search index, or an event bus, with schema evolution handling and transformation along the way, is what the dedicated capture tools do, and rebuilding that on raw logical decoding means owning the hard parts: slot management, handling of schema changes, and the operational rule that a replication slot which stops being consumed will retain write-ahead log segments until the disk fills. That last failure has taken down more Postgres instances than any query.

Full-text search: tsvector instead of a search cluster

Postgres full-text search covers stemming, ranking, and phrase queries with GIN indexes over generated tsvector columns. For product catalogs, document repositories, and internal search over moderate corpora, it works, and it composes with SQL filters in ways a separate index does not.

It loses on relevance engineering rather than on volume. Sophisticated ranking pipelines, typo tolerance and fuzzy matching, faceted navigation at scale, synonym and analyzer management, and per-field boosting are what dedicated search engines are for. A team that needs search to be a product feature with tuning owned by non-engineers will outgrow Postgres search on capability long before the corpus gets large. A team that needs a working search box will not.

The Ceilings That Actually Bind

Individual workload limits get the attention. The constraints that end consolidations are usually systemic.

Two of these deserve emphasis because they are frequently discovered rather than planned for.

Extension availability is a provider selection criterion. An architecture that depends on a specific extension is coupled to whichever managed providers ship it at a usable version. This is one of the more consequential ways the commercial dynamics around open-source databases reach into architecture, a pattern examined in the open-source database squeeze. Verify support before designing around an extension, not after.

Shared blast radius is the honest cost of consolidation. Five systems fail independently, and a broker outage degrades background processing while the application keeps serving. One system fails together. That is a real trade against the operational simplicity, and the answer is not to avoid consolidation but to invest the saved operational effort into tested restores and replica failover rather than pocketing all of it.

The Managed Landscape

Three groups, and the differences that matter are narrower than the marketing suggests.

Hyperscaler managed Postgres. The default choice, integrated with existing cloud accounts, networking, and billing. Mature operationally. The trade-offs are version lag behind upstream and a curated extension list that may not include what an architecture needs.

Postgres-native platforms. Vendors whose entire product is Postgres, competing on developer experience: instant provisioning, database branching for development and testing, separated storage and compute, faster access to new versions and extensions. Generally the better developer experience and the better extension coverage, with a smaller operational track record and a vendor concentration question.

Self-hosted. Full control over version and extensions, no restrictions, and the entire operational burden: patching, failover, backup verification, and the expertise to diagnose problems at three in the morning. Rational for organizations with genuine database engineering capability or requirements no provider satisfies.

The selection criterion most teams underweight is extension support at a current version, because it is the one that silently constrains architecture later. Price and region availability are easy to compare and easy to change. Discovering mid-project that the target platform does not offer a required extension is neither.

The Decision Framework

Consolidate by default. Start every workload on Postgres unless a specific, named requirement rules it out. The burden of proof belongs on the additional system, because the additional system carries permanent operational cost while the consolidated version carries a bounded migration cost if it fails.

Write the switch trigger down in advance. For each consolidated workload, state the measurement that would justify splitting it out: a recall target at a volume, a queue depth sustained over a window, a write latency percentile. A trigger written in advance is an engineering decision. A trigger invented during an incident is a purchase justified after the fact.

Instrument the trigger, not the anxiety. The reason splits happen prematurely is that nobody is measuring, so the loudest intuition wins. Once the specific metric is on a dashboard, the argument resolves against data.

Split one workload at a time, on evidence. When a trigger fires, move that workload and leave the rest. The failure mode is treating one workload exceeding its ceiling as proof that the whole consolidation was wrong, which restarts the five-system architecture wholesale.

Revisit on version upgrades. Postgres and its extensions improve materially between releases, and capability that forced a split two years ago may be native now. A split made on solid evidence in the past is not permanently correct.

Where Consolidation Goes Wrong

Queue tables that are never vacuumed properly. The most common concrete failure. It presents as database-wide slowness, so teams investigate everything except the queue table generating the dead tuples.

Analytical queries on the transactional primary. A scan-heavy report competing with transactional traffic for buffer cache and input-output. Straightforward to solve with a replica and easy to leave unsolved until it causes an incident.

Consolidating without the operational investment. The savings from running one system are supposed to fund better backups, tested restores, and replica failover. Teams that pocket the savings entirely have concentrated risk without buying down any of it.

Treating pgvector as a retrieval quality fix. Retrieval quality is a function of chunking, embedding choice, and reranking. Changing the vector store rarely changes it, in either direction.

Splitting on anticipated rather than measured scale. Building for a load that never arrives, and paying the distributed consistency cost the entire time. This remains the most expensive mistake in the category.

Frequently Asked Questions

Can Postgres really replace a dedicated vector database?

For most production workloads, yes. pgvector with HNSW indexing handles corpora in the low millions of embeddings at moderate query rates, and it wins decisively on filtered search, because permission and tenancy filters are ordinary SQL predicates joined against the source data. Dedicated stores win at very large scale, high sustained query throughput, and advanced quantization. Index build time competing with production traffic is usually the ceiling a team hits first.

Is a Postgres queue good enough, or is a broker needed?

Good enough for most job processing, and transactionally better, because a job can be enqueued in the same transaction as the state change that caused it, which removes the dual-write problem outright. The reasons to add a broker are semantic rather than volume-related: replay from an arbitrary offset, multiple independent consumer groups, or retaining the event log as the system of record. If the requirement is simply more jobs per second, tuning Postgres is usually the cheaper answer.

What breaks first when consolidating onto Postgres?

Usually the single write primary, since reads scale out to replicas but writes do not, and every consolidated workload competes for the same write path. In practice the first visible symptom is more often autovacuum pressure from a high-churn table, which presents as general database slowness rather than as a problem with the workload causing it.

Does consolidation increase outage risk?

It concentrates it. Five systems fail independently, so a broker outage degrades background work while the application keeps serving; one system fails together. That is a genuine trade rather than a hidden flaw, and the correct response is to spend part of the saved operational effort on tested restores, replica failover, and monitoring rather than treating the savings as pure gain.

When should a workload be split out of Postgres?

When a measurement crosses a threshold defined in advance: a recall or latency target missed at production volume, a queue depth sustained past a stated window, a write latency percentile breached. Split that one workload and leave the others. Splitting on anticipated scale, before any measurement supports it, buys distributed consistency problems in exchange for a bottleneck that may never arrive.

The Bottom Line

The Postgres consolidation argument is not that Postgres is the best system for any of these workloads. It is not the best vector store, the best queue, the best document database, or the best search engine, and any team choosing it should be honest that each substitution trades peak capability for integration.

The argument is that peak capability per component is the wrong optimization target for most organizations. The costs that actually accumulate are operational surface area, the integration code between systems, and the consistency problems that live in the gaps. One system with one backup story, one security model, and one transactional boundary eliminates all three, and for the overwhelming majority of applications the ceilings are far enough away to be irrelevant.

The discipline is in naming the ceilings before reaching them. Consolidate by default, write down the specific measurement that would justify each split, put that measurement on a dashboard, and split one workload at a time when the number says so. The teams that get this wrong are rarely the ones that consolidated too far. They are the ones that never defined what too far would look like.