Reference: "Just Use Postgres for Everything" — claims and validity¶

Source: https://www.amazingcto.com/postgres-for-everything/ — Stephan Schmidt (Amazing CTO). Originally a 2022 essay (HN discussion, ~1100 points); the page now shows "Updated December 13, 2025" and has grown from ~9 to 25 points. Community companion list: https://github.com/Olshansk/postgres_for_everything.

Thesis: consolidate the stack onto Postgres to reduce moving parts, cognitive load, and operational overhead ("one connection pool, one monitoring dashboard, one backup strategy"); Postgres can carry an app "up to millions of users"; three 99.9 % systems compose to ~99.7 %. Add specialized tools only when you actually hit Postgres limits — not when you think you will.

Why this doc exists: this project adopted Postgres for stage decoupling (work queue + multi-process writes — see stage_contracts.md). The article is the broader "how far can we ride Postgres" map. Each point below is the article's claim, followed by a researched verdict: Solid / Solid-with-caveats / Contested.

The 25 points¶

Caching instead of Redis — UNLOGGED tables + TEXT/JSON with stored-proc expiry. Solid-with-caveats. UNLOGGED skips WAL, helping writes, not reads; head-to-head benchmarks put Redis at ~900k req/s vs Postgres ~15k TPS on the same box (dizzy.zone, De Lio, Cybertec). Fine at modest cache QPS; wrong for sub-ms latency or six-figure ops/sec. UNLOGGED tables are truncated on crash recovery (acceptable for a cache).
Message queue instead of Kafka — FOR UPDATE SKIP LOCKED "if you only need a message queue"; River (Go) as a job queue. Solid-with-caveats — and the load-bearing point for this project. The canonical Postgres queue pattern; a well-tuned cluster handles tens of thousands of jobs/sec (Netdata, vrajat). Failure modes: with thousands of workers each claim scans past locked rows (CPU burn — pgsql-hackers); high-churn job rows cause dead-tuple bloat needing aggressive autovacuum/partitioning (PlanetScale, richyen). Kafka remains right for log/stream semantics at millions of events/sec. Python equivalents of River: Procrastinate, PGQueuer.
Data warehouse — Postgres with TimescaleDB; DuckDB for S3. Contested. Timescale is excellent for time-series + point lookups, but ClickHouse is consistently 6–7× faster on large aggregations with far better compression (Tinybird, oneuptime); Timescale's TSL license also restricts managed-service use. Fine to ~hundreds of GB; not a serious OLAP warehouse.
Data lake — DuckDB/DuckLake with Postgres as the catalog. Solid-with-caveats (young tech). DuckLake (May 2025) really does use Postgres as a lakehouse catalog, and pg_duckdb hit 1.0 (MotherDuck, pg_ducklake) — but the space is <2 years old vs the Iceberg/Spark ecosystem. An accurate "possible," not a battle-tested default.
In-memory OLAP — pg_analytics with Apache DataFusion. Contested — the named tool is dead. ParadeDB's pg_analytics is archived; the living alternative is pg_duckdb (10–100× analytic speedups). The idea survives; the article's specific recommendation is stale.
Document store instead of MongoDB — JSONB; DocumentDB as drop-in. Solid-with-caveats. JSONB + GIN covers most "Mongo because JSON" cases (Ivon). The real limit is TOAST: documents over ~2 KB get compressed/chunked out-of-row, causing read amplification and degraded partial updates (Pachot, MongoDB benchmark — vendor, but the mechanism is real). The DocumentDB reference is current: Microsoft's MIT extension, donated to the Linux Foundation Aug 2025, powers FerretDB 2.x (Microsoft).
Cron daemon — pg_cron. Solid. Mature (Citus), offered on RDS/Aurora/Cloud SQL/Supabase (repo). Caveats: ≤32 concurrent jobs, each holds a connection; a job never overlaps itself; cron.job_run_details needs its own cleanup; 1-second granularity floor.
Geospatial — PostGIS. Solid — the strongest claim in the list. PostGIS is the industry standard (OpenStreetMap rendering, Carto); generally better than dedicated alternatives.
Full-text search instead of Elasticsearch — tsvector/GIN, websearch_to_tsquery. Solid-with-caveats. Handles "the first 80 %" to a few million rows / low-hundreds QPS (Xata, msezer.dev). Missing vs Elastic: BM25 relevance, faceting, typo tolerance, horizontal scale; degrades at tens of millions of rows. ParadeDB pg_search adds BM25 in-Postgres as a middle path.
JSON API generation — row_to_json/jsonb_agg, PostgREST. Solid-with-caveats. PostgREST is proven (backbone of Supabase — postgrest.org). The caveat is architectural taste: business logic migrates into SQL/views, which many teams find hard to test and version.
Auditing — pgaudit. Solid. pgaudit is the compliance standard on all major clouds. READ logging can explode log volume; configured conservatively the overhead is <~5 % (Neon). Trigger-based history tables complement it for row-level data audit.
GraphQL — Hasura / PostGraphile / pg_graphql. Solid. All production-grade; Hasura compiles GraphQL to single SQL queries (no N+1 — discussion). Same logic-in-DB tradeoff as #10.
Vector database — pgvector. Solid-with-caveats. With HNSW it matches or beats Qdrant/Pinecone at ~1M-vector scale on equal compute, and is the right default when vectors live next to relational data (HuggingFace benchmark, Tiger Data). Breaks down ~5–10M+ high-dim vectors (HNSW for 10M × 1536-dim needs 60–70 GB RAM; dedicated engines win at 100M+ — tensoria).
Session store — indexed session table; hstore as KV. Solid-with-caveats. Standard pattern (Django/Rails ship DB session backends); high-churn rows add vacuum pressure, and Redis-class QPS doesn't transfer (see #1). The hstore suggestion is dated — string-only, superseded by JSONB (docs).
Rate limiting — atomic updates + time windows instead of Redis counters. Solid-with-caveats. Works in production to roughly 10–15k req/s/node with sub-ms p50 (token-bucket writeup, Go+Postgres report). Past that, hot-bucket contention and pool pressure favor Redis.
Distributed locks — advisory locks. Solid-with-caveats. Great single-cluster coordination primitive. The big trap: session-level advisory locks break under PgBouncer transaction pooling (lock taken on one backend, "released" on another) — use pg_advisory_xact_lock variants or session-mode pooling (good_job issue, Crunchy Data). Locks vanish with the connection (usually a feature); not a cross-cluster lock.
Event sourcing — ordered, atomic event storage. Solid-with-caveats. Production-proven (Marten — https://martendb.io/events/; Commanded — https://github.com/commanded/eventstore); fits well under ~tens of thousands of events/sec with partitioning (reference impl). Gapless global ordering under concurrent writers is genuinely tricky (sequence gaps / commit-order races).
Testing database — transaction rollback per test; template databases. Solid. Both are standard, documented practice (template databases; pytest-django/Rails wrap tests in transactions). Rollback-per-test can't exercise code that itself commits or uses multiple connections — relevant here, which is why this repo uses per-test ephemeral schemas instead (see tests/conftest.py).
Metrics/monitoring — pg_stat_statements "for application performance metrics." Solid-with-caveats. The canonical low-overhead query-performance view (docs) — but it monitors the database, not the application; it is not an APM/tracing replacement.
Webhooks — HTTP extensions sending notifications on data changes. Contested. Synchronous pgsql-http blocks the transaction during the HTTP call — dangerous in triggers. Async pg_net fixes blocking but has documented dropped-request issues under load and no retry guarantees (Sequin benchmark). The robust pattern is an outbox table + worker — i.e. point #2, not in-DB HTTP.
File storage — metadata and large objects in-DB. Solid for metadata, Contested for the bytes. In-DB blobs read ~10× slower than the filesystem, bloat pg_dump (a 7.5 GB bytea table → 12 h dump reported), large objects need manual vacuumlo, bytea caps at 1 GB (Cybertec, EDB). Consensus: metadata in Postgres, bytes in S3/filesystem.
Cryptography — pgcrypto. Solid-with-caveats. Fine for column encryption / password hashing (docs). Skipped caveats: keys passed in SQL can leak into logs and pg_stat_statements; in-DB crypto means the DB sees plaintext and keys (serious threat models encrypt application-side); bcrypt-in-DB burns DB CPU.
Scaling — pg_partman. Solid-with-caveats. pg_partman automates time/serial partitioning — right for big append-heavy tables (and for pruning queue/event tables). But partitioning is single-node scaling; the honest bullet would also mention read replicas and Citus. "Millions of users on one big Postgres" is nonetheless well-attested.
Multi-tenancy — row-level security. Solid-with-caveats. RLS is real isolation, used in AWS's reference SaaS architectures (AWS). Pitfalls: missing (tenant_id, …) composite indexes (~100× slowdowns), per-row policy cost, tenant-context leakage through pools, superuser/BYPASSRLS holes (Bytebase).
Scheduled jobs beyond cron — pg_timetable. Solid. pg_timetable (Cybertec) supports chains, concurrency control, shell tasks. It's an external Go binary, so "one more process" — slightly against the article's own one-system thesis.

What the article omits: LISTEN/NOTIFY¶

The article (wisely) never recommends LISTEN/NOTIFY, but it's the most famous failure mode in this genre: NOTIFY serializes all committing transactions through a global lock under concurrent writers — see the recall.ai outage writeup. At this project's concurrency it would be fine, but polling with SKIP LOCKED is simpler and has no such edge. Don't add LISTEN/NOTIFY to wake workers.

Overall assessment¶

The thesis — default to Postgres, add specialized systems only at measured limits — is mainstream-validated for small teams. The 2025 list is uneven, though: rock-solid (PostGIS, pg_cron, SKIP LOCKED, pgaudit, JSONB, testing patterns, pgvector at moderate scale), stale (pg_analytics archived, hstore legacy), and genuinely contested (Timescale-as-warehouse, in-DB webhooks, blobs in the DB). The recurring HN criticism (2022, 2024) is that Postgres-for-X is often a worse tool but a better stack — which is exactly the article's bet, and a correct one below roughly tens-of-thousands-of-ops/sec per concern.

Community companion list (Olshansk/postgres_for_everything)¶

https://github.com/Olshansk/postgres_for_everything — ~2.3k stars, actively maintained (last push 2026-04), PR-driven. A flat link aggregation (~30 categories, no quality bar or benchmarks) inspired by the essay and the cpursley gist. Treat it as a discovery index, not an endorsement list. It extends rather than contradicts the essay — and its dedicated pooling/sharding sections (pgbouncer, supavisor, pgdog, spqr) implicitly concede the scale limits the essay glosses over.

Entries most relevant to this pipeline:

Queues: pgqueuer — the only Python job-queue lib in the list (asyncio, SKIP LOCKED + LISTEN/NOTIFY); the closest off-the-shelf replacement for our hand-rolled queue if it ever grows beyond us. pgmq — SQS semantics as a PG extension with a Python client (visibility timeouts, archival). pgque — one SQL file + pg_cron, philosophically closest to what we built. The list's 2ndQuadrant SKIP LOCKED explainer is our pattern's canonical reference, and Choose Postgres queue technology is the strongest single argument for plain-Postgres queues over Redis/RabbitMQ/SQS. (Notably absent from the list: Procrastinate, the best-known psycopg3-native Python queue.)
Worker wake-up: brandur's Notifier Pattern essay — the careful way to use NOTIFY if we ever want push instead of polling; see the LISTEN/NOTIFY caution above before considering it.
Observability: nothing queue-specific exists; pghero (perf dashboard), pgmonitor (Prometheus/Grafana), and pgcli/pgweb for ad-hoc inspection.
Audit/CDC (if stage-write audit is ever wanted): supa_audit or bemi lightweight; pgstream/Sequin to stream changes out.
FTS: the list adds pg_search (BM25) as the upgrade path, plus a VectorChord "FTS slow myth" rebuttal supporting native FTS done right.
Testing: pgtestdb (Go) — the fast per-test-DB pattern our pytest fixtures mirror with per-test schemas.
HTTP-from-Postgres (pgsql-http/pg_net): exists, deliberately not used here — fingerprint-heavy scraping (curl_cffi/pydoll) cannot run from inside Postgres.

What applies to this project¶

#2 (SKIP LOCKED queue) — adopted for the jobs table (see stage_contracts.md). Hygiene to keep: partial index on the live-status claim predicate, short claim transactions, prune/archive done rows (or pg_partman, #23) as scrape-menus' daily per-store job churn rises (the Scrapy menu stage that once gated this was superseded).
#16 (advisory locks) — viable alternative for "one process per state" guards; remember the PgBouncer transaction-pooling trap if a pooler ever appears. We chose queue claims for uniformity.
#6 (JSONB) — right home for raw scrape payloads (Stage-3 menus): extract hot query fields into real columns, keep the raw blob as JSONB archive; mind TOAST above ~2 KB on hot paths.
#9 (FTS) — at dispensary/product-catalog scale, Postgres FTS with websearch_to_tsquery + pg_trgm is entirely sufficient; Elasticsearch is not justified.
#18 (testing) — this repo uses per-test ephemeral schemas (not rollback-per-test) because the code under test commits.
#1/#14 (cache/session) — supports the "don't add Redis" instinct at scraper request rates.
Multi-process writes generally — Postgres's bread and butter (MVCC); just mind connection count (one per worker; pool if workers ≫ ~50).