Skip to content
fellowcoder
All articles

Postgres is probably enough

Before you add Redis, Elasticsearch, a vector database, and a queue: Postgres does all four adequately, and 'adequately' is the correct target until it isn't.

fellowcoder6 min read1,258 words

The reference architecture for a modern application has five data stores. Postgres for records. Redis for cache and sessions. Elasticsearch for search. A vector database for embeddings. SQS or RabbitMQ for jobs.

That is five things to provision, five to monitor, five sets of credentials, five failure modes, five upgrade paths, and — the part that actually hurts — five sources of truth that can disagree with each other.

For most applications, one Postgres does all five well enough that the second system is a net negative for years. Here is the case, mechanism by mechanism, including where it stops holding.

Queues#

The objection to queueing in your database is that polling hammers it. That objection predates SKIP LOCKED, which landed in Postgres 9.5 and turned this into a solved problem.

claim-a-job.sql
WITH claimed AS (
  SELECT id FROM jobs
  WHERE status = 'pending' AND run_at <= now()
  ORDER BY run_at
  FOR UPDATE SKIP LOCKED
  LIMIT 1
)
UPDATE jobs SET status = 'running', started_at = now()
FROM claimed WHERE jobs.id = claimed.id
RETURNING jobs.*;

FOR UPDATE SKIP LOCKED means concurrent workers each grab a different row instead of blocking on the same one. Twenty workers polling this contend for nothing. Add LISTEN/NOTIFY so workers wake on insert instead of polling on a timer, and you have a queue.

What you get for free, and would have to build against a dedicated broker:

  • Transactional enqueue. Insert the job in the same transaction as the row it's about. Either both happen or neither does. With an external broker you need the outbox pattern to get this, which is more moving parts than the queue you were trying to avoid.
  • Queryable state. "Why didn't this user's email send?" is a SELECT. On a broker it's a dead-letter queue you have to go inspect with different tooling.
  • Retries and scheduling are a run_at column and an attempts counter.

Where it breaks: sustained throughput in the tens of thousands of jobs per second, or fan-out to many independent consumer groups. Postgres will do low thousands per second comfortably on modest hardware. If you're past that, you've earned a real broker.

Cache and sessions#

UNLOGGED tables skip the write-ahead log. They're not crash-safe — which is exactly correct for a cache — and they're substantially faster to write.

cache.sql
CREATE UNLOGGED TABLE cache (
  key text PRIMARY KEY,
  value jsonb NOT NULL,
  expires_at timestamptz NOT NULL
);
CREATE INDEX ON cache (expires_at);

The honest comparison: Redis will be faster, and if your cache is genuinely hot you will feel it. But a great many caches exist to avoid a 400ms aggregation, and turning that into a 2ms indexed lookup captures nearly all the available win. The difference between 2ms and 0.3ms is not what your users are noticing.

Where it breaks: you need real data structures (sorted sets for leaderboards, HyperLogLog, streams), or your cache traffic starts competing with your OLTP traffic for buffer space. That second one is the actual reason to split — the cache evicting your working set from shared buffers is a real and unpleasant failure.

Postgres has had full-text search for over a decade and people keep not knowing it.

search.sql
ALTER TABLE articles ADD COLUMN search tsvector
  GENERATED ALWAYS AS (
    setweight(to_tsvector('english', coalesce(title, '')), 'A') ||
    setweight(to_tsvector('english', coalesce(body, '')), 'B')
  ) STORED;
 
CREATE INDEX articles_search_idx ON articles USING GIN (search);
 
SELECT id, title, ts_rank(search, query) AS rank
FROM articles, websearch_to_tsquery('english', $1) query
WHERE search @@ query
ORDER BY rank DESC LIMIT 20;

That gives you stemming, stop words, weighted fields, and Google-style query syntax via websearch_to_tsquery — quoted phrases, OR, and -exclusion all work. The generated column keeps the index current with no application code and no sync job.

The thing you avoid by keeping search in Postgres is worth more than the ranking sophistication you give up: there is no synchronization problem. No CDC pipeline, no reindex script, no window where the index is stale, no "search says it exists but the database says it's deleted."

Where it breaks: you need real relevance tuning, faceting, aggregations over result sets, fuzzy matching at scale, or multi-language analysis. Elasticsearch is genuinely better at search — the question is only whether you need it yet.

Vectors#

pgvector gives you embedding storage and approximate nearest-neighbor search inside the same database as everything else.

vectors.sql
CREATE EXTENSION IF NOT EXISTS vector;
 
ALTER TABLE chunks ADD COLUMN embedding vector(1024);
CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops);
 
SELECT id, content, embedding <=> $1 AS distance
FROM chunks
WHERE document_id = ANY($2)          -- ordinary SQL filter
  AND published_at > now() - '90 days'::interval
ORDER BY embedding <=> $1
LIMIT 20;

Look at that WHERE clause. Metadata filtering combined with vector search, in one query, with real SQL semantics — no pre-filter/post-filter tradeoff, no special filter DSL, no consistency question between your metadata store and your vector store.

This is the decisive advantage and it is underrated. In a dedicated vector database, "search only documents this user can access, published in the last quarter" is either a pre-filter that wrecks the index, a post-filter that returns too few results, or a metadata sync you have to maintain. In Postgres it's a WHERE.

Where it breaks: hundreds of millions of vectors, or index build times that stop fitting in a maintenance window. Below roughly ten million vectors, HNSW in Postgres is fine.

The real argument#

None of the above is "Postgres is the best tool for each of these jobs." It obviously isn't. Redis is a better cache, Elasticsearch is a better search engine, and a purpose-built vector store will beat pgvector at scale.

The argument is about when you should pay for the better tool.

Every additional data store adds a consistency boundary — a place where two systems can disagree and someone has to write reconciliation logic. It adds an operational surface: backups, upgrades, monitoring, capacity, an on-call runbook. It adds a failure mode where your app is up and one dependency isn't, and you have to decide what degraded means.

Those costs are constant and paid daily. The benefit is a performance ceiling you probably haven't hit.

Takeaway

The right time to add the specialized system is when you can name the specific Postgres limit you're hitting, with a number attached. "Search feels slow" is not that. "P99 on our search endpoint is 800ms, it's ts_rank over 40M rows, and we've already tuned the index" is.

Doing it well, if you do it#

Three things that make the single-Postgres approach hold up longer than people expect:

Separate connection pools per workload. Your job workers, your web requests, and your analytics queries should not share a pool. One runaway analytical query shouldn't starve request handling. PgBouncer with separate pools costs an afternoon.

Watch the buffer cache hit ratio. pg_statio_user_tables will tell you when one workload is evicting another's working set. That number crossing below 0.99 is the earliest honest signal that you're outgrowing a single instance — much earlier than a latency alarm.

Use read replicas before you use another system. Analytics, search, and anything else read-heavy can move to a replica with no application rearchitecture and no new consistency boundary. This buys most teams another order of magnitude.

Start with one database. Add the second system when you can point at the graph. That is not conservatism — it's just declining to pay for capacity you don't have a use for yet.