This isn't just theory either, for example: Revolut is a bank that does all its event persistence and streaming on top of postgres. No traditional message queues/brokers in their stack.
If you start here, with the "Postgres will take you wherever you need to go" meme, without thinking extremely deeply about your schema and how you expect to evolve it in the future, you can easily paint yourself into a very difficult and expensive corner.
It's easy to use Postgres poorly in ways that result in painful centralized bottlenecks.
(Obviously this is largely true for anything, but I think that in 2026, where there's also a lot of more-specialized/less-fleible but much-easier-to-scale well-supported mature alternatives, you should be VERY wary of making everything have a single central SPOF. What are your users going to expect in terms of maintenance windows, etc.)
I'd be cautious with articles that say things like "All cloud providers allow you to run (and scale!) PostgreSQL by clicking a single button." with no mention of how long that will take and what options should be set to make it faster, or the costs of those things.
Having done that, e.g. used rabbitmq plus postgres, honestly I wish I had just used postgresql for both messages and data. It would have been easier to manage by an order of magnitude, especially at scale and needing to satisfy enterprise requirements. Also the flexibility of postgres would have solved problems that we ran into because of limitations of rabbitmq.
Without knowing any specifics of your uses, my usual starting point on that sort of design is that "messages AND data" is it's own special little way of ending up with a hard-to-debug-and-operate system. ;)
It's very hard to best-of-both worlds event-driven system + RDBMS-storage, it's very easy to end up with worst-of-both-worlds. Hello distributed transactions!
Again, you just should think about all the ways you want to use it and the maintenance/uptime requirements your users are going to have in advance.
It doesn't take very long (because compute and storage are separate in most of them) but good lord does it get expensive. Every time you click that upgrade button you are doubling your cost. It's really painful when you have a spiky workload that is performing fine like 95% of the time but you are watching the p99 and need to double the cost of a very expensive infra component, only to improve the experience of the heaviest 4% of your workload. This is to say nothing of the gambit you then have to play with reservations/prepays.
I haven't seen a way to get guarantees of upscaling operations under like 30 seconds (with Multi-AZ RDS) with well-supported RDS stuff (leaving out active-active setups with logical replication because that's a whole other can of worms).
If you know you're gonna be ok with that for a long time, go nuts. I'm just saying: think about it in advance!
The cost pain for spikes is also a thing - some of Aurora's billing models look potentially promising but I haven't used them in practice - though it's also somethings that's harder to avoid with alternatives. Distributed DBs aren't generally super friendly to dynamic scaling IME.
As SRE dealing with this at current company, a benefit of using well known software like Kafka is a lot of problems you will run into have solutions/guidance already available vs you having to explore solutions which a lot of time end with “Kafka could easily do this. “
There are two sizes of companies: those that can afford '1+ dedicated ____-person' and those that can't. Which should filter through to technology choices more than it does.
This problem doesn't go away with postgres. It's totally anecdotal but this is one thing that I've noticed different in mysql shops and postgres shops - with mysql there is usually at least one person on staff who knows MySQL DBA and scaling pretty well, with postgres it's rarely the case to have someone who knows the internals well - like you said, the person capable of maintaining it when it goes wrong.
You could argue it's because postgres requires less poking though I would say you don't need the DBA for when things go right.
Of course most people are just handing the management off to the cloud and that's potentially why, but it doesn't cover everything
I mean, with this custom thing, you have that question as well with downside is you cannot pick up the knowledge from off the street.
I became the Kafka guy at my current company, it took me about a week of reading and every time I had further question, I didn't have to bother anyone, I could Google and get data I needed.
When it's some NIH thing, you have to bother coworkers and knowledge is whatever is in YOUR company knowledge base with no ability to get knowledge from outside the company.
EDIT: You could also leverage contractors or outside support if not homegrown software.
Yup, and every time it breaks, you've got to pester someone whose job probably isn't maintaining that thing actively.
I worked at a startup with massive NIH syndrome, once. We even used our own in-house programming language, because it was "better than anything else out there on the market." It did have a lot of nifty features that others don't have: a pretty novel type system, programmatic macros, a built-in build system and other fun bells and whistles -- but also not-so-fun ones like having no syntax highlighter, LSP, or debugger, and having to constantly shuffle around your code to avoid ICEs in the compiler.
The compiler wasn't the product, but we found ourselves fighting that thing more actively than any of the real problems our custom programming language was supposed to solve. The CTO found himself spending all his nights and weekends mostly trying to get the compiler to not explode.
A few years later, after I had long left (for that reason, among many) I heard they switched to Python. Can't imagine how long it took them to get that all rewritten.
> e even used our own in-house programming language, because it was "better than anything else out there on the market." It did have a lot of nifty features that others don't have: a pretty novel type system, programmatic macros, a built-in build system and other fun bells and whistles
A DSL can work, but not for the features you list. Those features you already get from existing languages anyway!
If you need general programming language features like excellent type system, programmatic macros, a build system (doesn't need to be built into the language), etc... then use a general purpose programming language.
I have a DSL for backend/endpoints, and exactly none of those are in my feature list. What it has are things like easy way to specify access-control directives[1], the SQL query to execute, mapping request variables to SQL parameters, mapping SQL results-sets to response fields, etc.
I have another DSL for a test program. Both of those DSLs have specs that's literally 2x screens of bullet points and examples. LLMs can output those DSL programs because the spec for the DSL is so small.
For general purpose programming stuff (while loops, conditionals, etc) my DSLs break out to Python.
A good indicator that you shouldn't be creating a new language for production is when you find yourself implementing conditionals, loops, etc.
===========================
[1] Limit endpoint to specific roles, or members of the same team, or both, or even to the user itself - someone calling `/user/profile/update` should only be allowed if the profile they are updating is theirs, for example.
My general rule of thumb is "Use Postgres until you've discovered why you can't use Postgres."
Anything you introduce is another moving part you have to operate and maintain, and in the beginning, Postgres can probably handle it. Wait for load, see where its failing, and then you'll have a better idea if adding another tool is worth the cost.
In a lot of cases using SQLite means you write queries incompatible with RDBMS. No need to worry about race conditions or the amount of queries you make, when 100 selects are uber fast.
Yes, for very small or embedded, single-purpose systems sqlite is usually a good choice. It's very well tested, and there's nothing extra to run or manage. But be careful if the system starts growing beyond that, you'll want a real RDBMS before you abuse sqlite too much.
I would probably do an event-source architecture, where you record events on the client and then push them to the server when you're reconnected. It has a lot of benefits, for instance, trivial auditing and free serialization.
This kind of post (Postgres! It's all you need!) is getting pretty tiresome. Postgres does not even come close to a full replacement for Elastic, and that's just the first bullet.
Looking down the list it is pretty easy to go: Yes, postgres can be used instead of that for extremely basic use cases, but it all goes out the window you actually need any of the power of these other tools.
I think it would be helpful if some of these posts included scale. There are almost always two groups talking past each other
- I run my B2B application, Postgres only, and it is perfect for my 50k MAU. No complaints, sleeping soundly with the low complexity and a two man team.
- I work at FAANG, where we have 1 billion DAU, and this is a joke. Would fall over immediately. The dedicated ops teams for Kubernetes, Elastic, and Redis have never complained about scaling issues.
I broadly agree, but would tweak those numbers a little for small B2B apps: I ran a small B2B application on a cheap VPS using PostgreSQL as a primitive messaging interface, and even on a small VPS 50k DAU won't even cause the machine to break a sweat.
Agreed, but many of the criticisms I am reading here are assuming high scaling requirements and invalidating the approach entirely. When there are many business domains that will comfortably fit within a modestly specced database instance.
> Postgres does not even come close to a full replacement for Elastic
Size matters!
For most of the application out there elastic (or kafka or any other specialized tool) is just too much(and too costly). They can do fine with postgres or mysql. Actually, I'd argue that in a lot of cases even postgres is too much, probably sqlite is enough.
Sure, best not to overcomplicate early if you don't need it.
PG is great and I work with it daily, but it's also not a problem to think about scale early and at least have a notional plan for what to and how to know when scale is becoming an issue in your system as you're designing it. Even PG is overkill and sqlite is more than enough for some of my projects.
There are a lot of specialized tools available, but you definitely don't need to put every one in your toolbox. Experience and observation help you make those edits -- and of course there's almost always room for improvement, but "good enough" definitely exists (until it doesn't anymore :D).
This is the way. Notionally building a space/path to scale into architecture early, but delaying implementation of that scaling component until actually needed.
Then a system gets most of the benefits of not accidentally making it torturous to rearchitect for scale, without paying the headcount / complexity cost until it's needed.
The point is in general for people to just consider it, often people start out on their side projects or internal company projects and commission Elastic, Redis, Postgres, Kafka before even getting started. In reality they could fit it all into Postgres for a very long time.
Nobody is saying that a huge ecommerce store with complicated filtered search logic should throw away their Elasticsearch cluster and switch to Postgres.
If you actually start looking into these things, you often start looking at custom pg extensions, which means you just made the decision to "simplify" your stack by maintaining your own postgres cluster with custom extensions. This is just papering over the fact that you're increasing the complexity and saying "well it's still just postgres!" as you do it.
Installing & maintaining a Postgres extension is vastly, vastly simpler than running Elasticsearch and Kafka. Like, how could you even compare these things if you know what you are talking about?
Or maybe you are looking at it from "just swipe your credit card at AWS" perspective, in which case "just use Postgres" articles are for a different audience.
Not really, most popular extensions are available on GCP/AWS (https://docs.cloud.google.com/sql/docs/postgres/extensions) out of the box and there's many things that you can easily run in Postgres without extensions (Queue, KV store).
You're right that vanilla Postgres doesn't come close to replacing Elastic. There are efforts to resolve this, though, like ParadeDB: https://github.com/paradedb/paradedb (disclaimer: I work for ParadeDB)
I see Tantivy mentioned in your readme but AFAICT there is no PG-Tantivy sync. I also see "native vector support is coming to our search index soon". Could you clarify?
What do you suggest for a language like Malayalam which has no native support, preferably with low RAM requirements?
I'm partial to Typesense, especially for smaller data sets, since it runs primarily in memory, is easy to use and is hella fast. For bigger data sets, I hear good things about Meilisearch.
I have a lot of troubles with a small private instance of Rocket chat, all due to MongoDb stuff, versions, migrations and backups. I bet almost all private instances of Rocket chat would be perfectly served with Postgres.
Posts like this can be tiresome, yet the general consensus among developers seems to be "yeah, Postgre/SQLite is ok for 99% of the cases, but MY case is going to be in the 1%, because I am going to be the next Facebook".
Fwiw, I, as someone who has worked on Postgres for a long time, also find it quite tiresome. Like there's plenty stuff I wouldn't use Postgres for, and I can probably get get more out of it than most.
These types of articles needed to be written because we have gone way too much in the other direction. The issue is that people use too many tools prematurely when they are not needed at their stage. So yea, in most cases, you are probably better off just with Postgres. I m a culprit of this myself so I wouldn't say that I know better. It is just too tempting to setup too many tools to feel cooler or feeling that "we must use elasticsearch as no one does search in db".
> After some performance checks it became clear that PostgreSQL was even faster than reading from the file system for our use-case. PostgreSQL uses the file system very efficiently for its data - and it adds a lot of caching and efficient reading and writing strategies that can outperform writing and reading raw data on a file system.
This goes against conventional knowledge. I've always heard (and followed best practice) to avoid storing binary data in BYTEA columns that should otherwise be put on a filesystem or an object storage like S3.
I'd like to find out more about this, because in many cases it would be very convenient indeed to store it in the database itself.
The primary reason to avoid doing so is avoiding thrashing your buffers, along with increased size of backups, WAL bloat, etc.
Can you? Yes. Should you? Not at anything beyond a toy scale, unless you want to pay for more RAM to ensure that your normal OLTP queries don’t take a performance hit.
It's the other way around for me: as 99% of the stuff I develop is .NET (and I use EF Core for database stuff), I can get away with SQLite for local development, prototyping (and even staging), and then just "flip a switch" for it to run on production PostgreSQL.
I can't recommend this "switch". If you are not testing locally with the same relational database as in production, you can miss mistakes and bugs. This is not just theoretical. One example where I thought I will be fine using SQLite was with a small Django project. But time and time again I ran into limitations of either SQLite or Django's database adapter for SQLite, when it came to dealing with many to many relationships in the model and through tables, requiring me to work around the limitations. There is no guarantee, that these workarounds in turn will work the same in PostgreSQL in production.
Anyway, it is a basic practice of keeping test and dev environment as close as feasible to production, to avoid missing issues and wrong assumptions.
> Anyway, it is a basic practice of keeping test and dev environment as close as feasible to production, to avoid missing issues and wrong assumptions.
Containers are great for this during development.
Testcontainers are great for tests in particular when you don't want to use some mocked in-memory DB because those have the same issues as using a different DB during development: https://testcontainers.com/
Agreed, and it is easy to have a Postgres container for local tests or in CI as well. I don't really see much need to avoid Postgres in testing. Also if production runs in containers, testing can and probably should just run in containers as well. Also makes for a cleaner test setup usually.
That's kind of risky considering the radical differences between types in SQLite and Postgres. There are plenty of situations where EF will need to be configured differently in order to map your .Net types correctly to each DB. Or subtle differences in behavior due to storage differences (particularly SQLite's predilection for storing things as strings). It's so trivially easy to run Postgres in Docker that I don't really see the advantage of using SQLite for local dev.
Yes, especially for a web app where there’s realistically only a need for one VM/server. By the time you outgrow that approach, a very straightforward migration to Postgres is probably the least complex problem you face.
This is exactly how tightly coupled, unmaintainable software is constructed.
By picking the tools before understanding the model and building bespoke architecture.
You pick the tools that the business model requires. It might be a relational data store. It might not be. You might want an event store. You might want to reduce costs with lambdas and DynamoDB. You may need a pub/sub event broker.
The OP clearly loves Postgres. Cool. They also have limited experience with complex systems architectures because if they had that experience, they would have never written this article.
> PostgreSQL allowed us to use a fulltext search plugin to do everything in one system. No need to sync any data. No need to maintain and run two systems. It just worked and made us smile (after some tweaks of course). Simplicity.
I found MariaDB to be wonderfully simple to use for somewhat casual use cases: https://mariadb.com/docs/server/ha-and-performance/optimizat... and still reach for it in some personal projects, however the whole growing MySQL incompatibility is a big issue if the tech you use only officially supports MySQL and you can't (easily) get MariaDB specific DB drivers.
Personally, one of the best things about PostgreSQL is transactional DDL, every DB should support it. Also they handle JSON pretty nicely (though I'd prefer not to store data like that unless necessary) alongside excellent plugins like pgvector and PostGIS.
On the other hand, for things like queues, or even any sort of blob storage, I'd look at things like RabbitMQ or Garage (S3 compatible). Sometimes specialized software is nice for keeping things logically separated. I maintain that it's good to be able to divide your stack up by mechanisms/concerns (rather than business domain necessarily).
I tend to agree with quite a few points in the article, but some topics warrant some careful scrutiny.
* As a message queue:
Only if your required features are very basic, like if you need cluster communication and run your own coordination protocol on top.
* High Volume Time Series:
TimeScale works, but composes badly with other workloads on the same DB server ( from an operational perspective at scale )
* Vector Database:
The same issues as with TimeScale.. PgVector for example lives in its own seperate "world" and the query planner sees it as a very opaque thing. Forget about adding vector storage to an existing high volume db, that must server other complex queries.. PGVector will either trash your caches, or take over your cpu so that workloads that used to work fine stall. This is IMO not a pgvector problem itself ( Kudos to those guys ) but rather that postgresql extension apis are not very good at exposing custom costs and tradeoffs to the system as a whole.
* Raw Data:
Works for small files... why anyone would want to store large amounts of data in it would be a mystery, where it shines is accessing LOTS of small files where internal caching etc help a lot compared to raw filesystem access ( also a bit dependent on the filesystem and its tuning though )
* Microservice:
If your service is ONLY exposing json data from some database model, then it should not exist at all IMO. Create a view and be done with it.
Also to note: (Not a fault of PGVector again just a limit of our algorithmic knowledge)
PGVector does HSNW or IVFlat indices ... (there is nothing better persistent) however it breaks down with high latency at LARGE amounts of vectors ( 100MIO+ ) that seems like a high ceiling, but when designing production RAG systems, you tend to do per chunk embeddings, or even visual patch embeddings... e.g one page of a document becomes 1024 vectors in itself (for visual patch embeddings ) ... so you hit those limits at 100000 pages already.. something larger organizations definitly have.
> Microservice: If your service is ONLY exposing json data from some database model, then it should not exist at all IMO. Create a view and be done with it.
Yeah, this has nothing to do with Postgres. If the service is accessing a database that isn't internal to the service, then that database is already a standalone service in itself.
I like to consider Postgres the starting point for all of these things, that can be outgrown and replaced when appropriate. I do love just shoving everything in Postgres and seeing that I only end up needing a few additional dedicated services as the product groups. Redis is usually the next pickup for me.
Sure, thats a good way of working.. I just have the experience when handing over a project ( consulting ) anything i have put in place will never get replaced or kept for too long outgrowing its capacity by far, and offset with huge expenses in hardware or operations. Technically not my problem anymore ( except when it breaks on a maintenance contract ) but i still like to avoid it early if i can
Funny. I was just this joking this morning with a colleague about using Postgres for everything.
Considering adding mongo for unstructured data? Just use postgres jsonb.
Building a search index? Postgres is fine too.
Considering using redis for fragment caching? Just use an unlogged table in postgres with key value columns. Need pub/sub? Well just use postgres listen/notify.
Using postgres for everything has served me very well.
I've done that before and the code was a mess. It works at the beginning but APIs do much more than piping data from the database. When you start dealing with ACL, external calls, code reuse, etc. It's just nice to have all the tools available to you from something like Python or Go.
It's more "what one tool can do everything", not that its ideal. Like why people use Microsoft Teams even though its terrible.
The relational model and sql force us to simplify our data models too much by eliminating relationships or just not dealing with them.
Think about a nested json blob from some web service api and storing it in SQL in normalized tables. No one is going to do that. Everything just becomes a denormalized mess and everything is hacked around it.
Instead of modeling things in the proper way, most of the world's data is modeled in a way so that we don't have join explosions in sql queries because they look scary. Data pipelines become these scary batch transformations where data is dumped somewhere else without anyway to trace back where it came from.
I encounter so many end-user applications and systems where you wonder: "why couldn't they allow a list of items here instead of a single box" or "why can't this reference this other thing".
as someone who loves postgres, this take is getting pretty old. yes we can do quite a bit with extensions but extensions often need to interface with external systems and even then managed providers don't consistently support all extensions. some gaps: bm25 indexes, olap support, also extensions also run into licensing restrictions.
I use a SQL databases as needed. I've used Postgres, Sqlite, Duckdb, Json files with AWS Athena, Oracle enterprise for ERP systems (a multitude of schemas and objects with interoperability), and others.
I'm currently, deploying Duckdb with AWS S3 Tables (Iceberg) to see how it fits for a use case I have.
IT is great and always changing. Keep trying new things.
I love postgres and use it heavily, but I still don't fully understand how it overlook MySQL. Maybe because of Heroku adopting it.
MySQL was generally faster, and while MyISAM was a bit limited Innodb was pretty powerful, and you had the choice. It was also simpler (imo) and avoided a lot of the xid/vacuum issues.
That said, still love Postgres. But at the time it started eclipsing MySQL, MySQL felt better positioned.
I've not interacted with mysql a whole lot, but when i did I was regularly surprised that it didn't have stuff I was missing from Postgres. Off the top of my mind:
- Query planner is much worse (just yesterday I had to USE INDEX to sped up a query by 300x, I'm near-certain postgres would just have gotten it right)
- Indexes are much more limited: no GIST, no GIN
- No transactional lock (`pg_advisory_xact_lock` in postgres). This one was very surprising, it's a really useful thing and I had to implement it myself as a lock table
At least you have access to USE INDEX on MySQL. On Postgres it's not rare to have a query suddenly perform awful in production because some switch flipped in the planner and now it's picking some random index
Good news is that they just added a plan stability feature in pg19. It's actually full planner hints, so you can edit the plan to whatever you want if you have no fear, but the main motivation is exactly index stability.
Back in the day, the sentiment was the MySQL was more-performant but the criticized tradeoff of having "cut corners". I still remember when their transaction support InnoDB table engine came out. Anyway Postgres was viewed as slower but more standards compliant - so mature architects preferred that. MySQL, in my opinion, fell into default usage among LAMP stacks and PHP-using kiddies. Postgres took the crown over time.
MySQL had some problematic design decisions initially. They might be fixed now, but the impression remained. And later there was the added complication that they were bought by Oracle, so you didn't really know how this would turn out in the end.
PostgreSQL also had more features back then, e.g. the JSON support is very nice if you need to do anything that doesn't neatly fit into the relational model.
Not only where the problematic design decisions, but large numbers of people said for years "nobody will never need/want that anyway so stop talking about it". That is they didn't even attempt to talk about trade offs, you were just wrong if you suggested anything else.
Then people who knew something got involved (or likely were involved all along - but I never followed MySQL so I'm not sure) and fixed those because they matter and suddenly the crowd shut up.
Maybe things have changed, but my memory of MySQL ~15 years ago was that it was so... hacky. Basically, the (non-strict) JavaScript of the RDBMS world.
MySQL was a proven solution back in the day, i.e late 2000s. Github/Twitter/Heroku etc were using it. In the past 10-15 years, Postgres has come a long way.
MySQL was always behind in terms of features. In early 2000s it had very limited constraints. Most people were running in ISAM backend and did not even have transaction support. It was being used by people who did not understand how advanced relational DBs were being used. What changed is Postgres overtook its actual competition, which were Oracle, SQL Server and Sybase.
MySQL caught up as well as far as I know, but it still may have some poor defaults that are widely used.
In the early days, PostgreSQL was so much more awkward to deal with. Crash-prone at first, and then there was the whole business around having to drop the db during upgrades. It didn't really match MySQL operationally until around 2002.
During the dot com era it was common to develop and launch on MySQL with the intention of migrating to something else if they became successful (though your typical LAMP stack developer regarded Oracle and SQL Server as being deeply 'weird', so many were willing to stick with MySQL despite the well-known limitations of MyISAM).
From where I'm standing, it seems that PostgreSQL became clearly preferable for new projects from the mid 2000s onwards, but it was only the Oracle acquisition that began to push existing users off MySQL.
I would agree with that, although I personally found it preferable in 2001-2002, since it matched the feature set of SQL Server in all the important ways that MySQL didn't. I think it may have been late 2000s before it was clearly preferred for new projects and even later before it was the thing you had to explain why you weren't using it.
edit to add: MySQL did have operational advantages even later than mid 2000s since it had master/master replication from very early (I don't know how sound it was, I doubt it was perfect). That was a real reason to choose it. We still don't have it in Postgres without extensions and even there citus is not really the same.
Oh yeah, we were early adopters of MySQL's master/master replication at an online retailer. Certainly an "interesting" experience, and I remember becoming very well acquainted with the vagaries of the binlog!
(I remember it being heavily touted in the first edition of the O'Reilly "High Performance MySQL" book. The second edition was about twice the length, with most of the additional pagecount going into detailed explanations of why you should actually be very careful and do lots of testing before deciding to rely on master/master!)
Actually, I remember that one of the drivers away from MySQL before Oracle came along was the 4.x and 5.x period, when there were various performance regressions. And even when those were sorted out, the introduction of InnoDB made people realise that MySQL's apparent speed advantage was really just down to MyISAM lacking referential integrity and transactions.
Certainly, there were people clinging on to MySQL 3.23 for read-heavy data warehousing applications for a very long time.
> And even when those were sorted out, the introduction of InnoDB made people realise that MySQL's apparent speed advantage was really just down to MyISAM lacking referential integrity and transactions.
Yes we saw the same thing play out with MongoDB. Ack without fsync is indeed very fast.
In the times when everyone was installing Apache + PHP + "Some database" stack, the easy path was to use MySQL for a very simple reason: it had ready to use MS Windows installer.
Another thing: those were times when web applications were practically 99% reads, and not so great ACID was a non-issue.
Postgres is OK, but it has really a lot of quirks that are not that obvious.
> Timescale lately released the pgvector extension, that turns your PostgreSQL into a vector database.
I don't think this is accurate and smells like an LLM hallucination to me.
From the Timescale/Tiger Data _pgvectorscale_ project's README:
> pgvectorscale builds on pgvector with higher performance embedding search and cost-efficient storage for AI applications.
I think this is where the confusion originates. I believe pgvector is primarily Andrew Kane (@ankane) and a cadre of OSS contributors.
As an aside, I've used Timescale/Tiger Data products and was very happy with them and their support. Their team was very engaged and responsive to all of our questions. They also fixed a pretty gnarly indexing bug I uncovered in pgvectorscale in an impressively short amount of time.
Hey Raphael Bauer, if you're reading this, I suggest you change the color of hrefs on your website, all of them are purple and underlined, which usually is indicator for "Already visited URL". For me that's not that much of a problem, but it was something i kept noticing when reading the article. I wonder if anyone else also had this thought or am I alone as I didn't see anyone else mention this in the comments. But I wanted to signal that nevertheless :)
Postgres is great, but I certainly don't think it's great for everything. For instance, while you can in theory implement OLAP aggregation you're going to be hand-rolling a bunch of stuff that something like Clickhouse gives you for free declaratively.
I don't think the point is that PostgreSQL is great for everything.
But you may get by with a single piece of infrastructure instead of 7.
In most of the applications we build or maintain we use PostgreSQL + cloud storage. That's it. And it works very well, also for: storing JSON, full text search, as a queue, as a vector database. Other software may be better at providing those features, but I'm extremely happy we only need to understand & manage PostgreSQL.
The article says verbatim “PostgreSQL Replaces Clickhouse”.
Coming from storing billions of rows in Clickhouse and performing dozens of materialized operations I shudder to think about what that would look like in a DB that doesn’t even support declarative IVM.
Why write a fanboy text with unfair comparisons that hide the Postgres limitations?
For instance, for many simple needs MySQL is simpler than Postgres, with similar performance and consistency.
* No need for a connection pool, while many use cases with Postgres require PgBouncer and Co.
* Easy sort (and basic search) of multilingual text, because MySQL has case insensitive UTF8 collations.
* No need to VACUUM, which can be a hard problem (it was, the last time I used Postgres).
For full text search, I once worked on a project that considered several alternatives for this, including Postgres. Manticore Search was finally chosen because it was more performant, with better search results.
> * No need for a connection pool, while many use cases with Postgres require PgBouncer and Co.
is there a strong evidence you even need client side connection pool at all? What is the purpose?
The limitation is that you have many clients with connection pools, they hold internal PG connection without allowing it to be reused by other clients..
If you run a single application, or a few instances of the same application, you don't need an external pool and most frameworks have an internal connection pool anyway.
Not sure if I'm missing anything here, but if I want case-insensitive search I simply create an index on lower(column) and use that to query.
VACUUM is something you need to pay attention to at scale. And at that point you need to know your DB anyway and tune it. For smaller applications (and I don't mean only toy applications) it usually isn't an issue.
Ive built data warehouses and job queues on postgres. The DW got replaced with bigquery when we started doing more tracking. We still run postgres as an app-facing cache of the aggregated data from bigquery though.
The job queue runs on the cache db, scheduling jobs to move data from bigquery into postgres. It’s pretty neat.
Now we’ve run into near-real-time requirements so clickhouse is getting thrown into the mix.
It’s pretty funny the lengths we go to to implement user facing analytics that’s basically just “you are visitor number X” from 1995.
As usual, it depends on the scale, but it's a sane default for 99% of use cases.
Different use cases have different scalability limits in PG, when you get to them you need to deal with them.
It would be perfect if it had somewhat transparent sharding, I mean a way to add another instance and distribute load without having to stop everything.
There are solutions, but they tend to be involved and when you get to that point in many cases it makes sense to just move that workload to something else that scales better.
NVMe drives + Litestream + object storage(S3/R2..). sqlite simplifies things for the entire long tail of apps/services that aren't the Ubers and AirBNBs of the world.
It looks interesting, but deciding on where to put object storage is what keeps me from doing this. I don’t have an AWS or Cloudflare account and I’m not sure what to commit to.
Also, apparently Litestream could use a filesystem instead of an object store?
Yes, especially something that touches DB. Router and shell are stupidly simple and you get auto failover in some 30 minutes. That is something make me stick to mysql.
i love postgresql but once we added ai-generated dashboard to our homegrown analytics tool [0] some of the crazy (amazing) dashboards that the ops team was building began accumulating horrendously slow db queries. I considered dynamically adding indexes or alerting around postgres slow queries but also quickly prototyped mirroring the postgres data in clickhouse. At first could not believe how fast clickhouse was on arbitrary analytics queries - like 60s to 0.5s for some gnarly queries. truly amazing software that just works without any tuning for this kind of exploratory analytics workload.
so yes, i'm still a postgres maximalist (worker queues still in pg [1]) but (especially in the age of quick LLM prototypes) it's always worth measuring the more purpose-built approach.
Supporting more than 1 writer per process. Strict typing. Access controls. Replication at scale is more effecient than copy-pasting files (seems SQLite has improved on this one).
"MySQL was also potentially faster as it did not implement all features of the SQL standard. "
This is a not great start. I assume it refers to MyISAM which has not been relevant for over a decade at this point. InnoDB made different design than PG decisions and was (and perhaps still is) faster at point lookups.
Well, the poster explicitly talks about 2003 here: „ In 2003, MySQL was much more widely used than PostgreSQL. MySQL was also potentially faster as it did not implement all features of the SQL standard“
true to that - currently using psql (in a single monolithic codebase) as: sql db, json db, vector store, logs store, full-text search, queue, message bus.
Hardware is so far nowadays to make people with little systems knowledge confident to make such claims at least from their use cases. However it is neither generally reasonable nor efficient.
For the graph database idea in Postgres, PG 19 has native support for property graphs [0]. You can set up your tables and their relationships as nodes/edges, then query against them using Cypher-esque [1] syntax.
https://medium.com/revolut/recording-more-events-but-where-w...
It's easy to use Postgres poorly in ways that result in painful centralized bottlenecks.
(Obviously this is largely true for anything, but I think that in 2026, where there's also a lot of more-specialized/less-fleible but much-easier-to-scale well-supported mature alternatives, you should be VERY wary of making everything have a single central SPOF. What are your users going to expect in terms of maintenance windows, etc.)
I'd be cautious with articles that say things like "All cloud providers allow you to run (and scale!) PostgreSQL by clicking a single button." with no mention of how long that will take and what options should be set to make it faster, or the costs of those things.
It's very hard to best-of-both worlds event-driven system + RDBMS-storage, it's very easy to end up with worst-of-both-worlds. Hello distributed transactions!
Again, you just should think about all the ways you want to use it and the maintenance/uptime requirements your users are going to have in advance.
If you know you're gonna be ok with that for a long time, go nuts. I'm just saying: think about it in advance!
The cost pain for spikes is also a thing - some of Aurora's billing models look potentially promising but I haven't used them in practice - though it's also somethings that's harder to avoid with alternatives. Distributed DBs aren't generally super friendly to dynamic scaling IME.
That transition can be super super painful as you don't quite have enough work for the dedicated person.
You could argue it's because postgres requires less poking though I would say you don't need the DBA for when things go right.
Of course most people are just handing the management off to the cloud and that's potentially why, but it doesn't cover everything
I became the Kafka guy at my current company, it took me about a week of reading and every time I had further question, I didn't have to bother anyone, I could Google and get data I needed.
When it's some NIH thing, you have to bother coworkers and knowledge is whatever is in YOUR company knowledge base with no ability to get knowledge from outside the company.
EDIT: You could also leverage contractors or outside support if not homegrown software.
I worked at a startup with massive NIH syndrome, once. We even used our own in-house programming language, because it was "better than anything else out there on the market." It did have a lot of nifty features that others don't have: a pretty novel type system, programmatic macros, a built-in build system and other fun bells and whistles -- but also not-so-fun ones like having no syntax highlighter, LSP, or debugger, and having to constantly shuffle around your code to avoid ICEs in the compiler.
The compiler wasn't the product, but we found ourselves fighting that thing more actively than any of the real problems our custom programming language was supposed to solve. The CTO found himself spending all his nights and weekends mostly trying to get the compiler to not explode.
A few years later, after I had long left (for that reason, among many) I heard they switched to Python. Can't imagine how long it took them to get that all rewritten.
A DSL can work, but not for the features you list. Those features you already get from existing languages anyway!
If you need general programming language features like excellent type system, programmatic macros, a build system (doesn't need to be built into the language), etc... then use a general purpose programming language.
I have a DSL for backend/endpoints, and exactly none of those are in my feature list. What it has are things like easy way to specify access-control directives[1], the SQL query to execute, mapping request variables to SQL parameters, mapping SQL results-sets to response fields, etc.
I have another DSL for a test program. Both of those DSLs have specs that's literally 2x screens of bullet points and examples. LLMs can output those DSL programs because the spec for the DSL is so small.
For general purpose programming stuff (while loops, conditionals, etc) my DSLs break out to Python.
A good indicator that you shouldn't be creating a new language for production is when you find yourself implementing conditionals, loops, etc.
===========================
[1] Limit endpoint to specific roles, or members of the same team, or both, or even to the user itself - someone calling `/user/profile/update` should only be allowed if the profile they are updating is theirs, for example.
Anything you introduce is another moving part you have to operate and maintain, and in the beginning, Postgres can probably handle it. Wait for load, see where its failing, and then you'll have a better idea if adding another tool is worth the cost.
If you mean database size, SQLite can handle massive amounts of data. I've seen 281 TB quoted as theoretical max size.
SQLite is embedded for local applications with one writer mostly.
Postgres is for a client-server architecture with many writers.
When you start a project, you generally know which architecture you need.
Looking down the list it is pretty easy to go: Yes, postgres can be used instead of that for extremely basic use cases, but it all goes out the window you actually need any of the power of these other tools.
Size matters!
For most of the application out there elastic (or kafka or any other specialized tool) is just too much(and too costly). They can do fine with postgres or mysql. Actually, I'd argue that in a lot of cases even postgres is too much, probably sqlite is enough.
I tried to use rabbitmq for a small app, installed it, configured it and then it didn't work. Spent a day jumping through hoops getting it right.
Dumped it and used postgres, in half an hour. Worked like a charm.
PG is great and I work with it daily, but it's also not a problem to think about scale early and at least have a notional plan for what to and how to know when scale is becoming an issue in your system as you're designing it. Even PG is overkill and sqlite is more than enough for some of my projects.
There are a lot of specialized tools available, but you definitely don't need to put every one in your toolbox. Experience and observation help you make those edits -- and of course there's almost always room for improvement, but "good enough" definitely exists (until it doesn't anymore :D).
Then a system gets most of the benefits of not accidentally making it torturous to rearchitect for scale, without paying the headcount / complexity cost until it's needed.
Nobody is saying that a huge ecommerce store with complicated filtered search logic should throw away their Elasticsearch cluster and switch to Postgres.
Or maybe you are looking at it from "just swipe your credit card at AWS" perspective, in which case "just use Postgres" articles are for a different audience.
If you’re just starting out, keep things simple. Otherwise, you probably already know exactly why you need something more than Postgres.
What do you suggest for a language like Malayalam which has no native support, preferably with low RAM requirements?
Posts like this can be tiresome, yet the general consensus among developers seems to be "yeah, Postgre/SQLite is ok for 99% of the cases, but MY case is going to be in the 1%, because I am going to be the next Facebook".
My tip: store your company's source code on a samba file server. Only when that no longer performs well, switch to other systems like Git.
> After some performance checks it became clear that PostgreSQL was even faster than reading from the file system for our use-case. PostgreSQL uses the file system very efficiently for its data - and it adds a lot of caching and efficient reading and writing strategies that can outperform writing and reading raw data on a file system.
This goes against conventional knowledge. I've always heard (and followed best practice) to avoid storing binary data in BYTEA columns that should otherwise be put on a filesystem or an object storage like S3.
I'd like to find out more about this, because in many cases it would be very convenient indeed to store it in the database itself.
Can you? Yes. Should you? Not at anything beyond a toy scale, unless you want to pay for more RAM to ensure that your normal OLTP queries don’t take a performance hit.
I'm a huge PG fan, so I start everything with it, but SQLite is sane, and it generally has a happy upgrade path to PG If you need it.
Both are amazing technologies.
Anyway, it is a basic practice of keeping test and dev environment as close as feasible to production, to avoid missing issues and wrong assumptions.
Containers are great for this during development.
Testcontainers are great for tests in particular when you don't want to use some mocked in-memory DB because those have the same issues as using a different DB during development: https://testcontainers.com/
https://sqlite.org/stricttables.html
By picking the tools before understanding the model and building bespoke architecture.
You pick the tools that the business model requires. It might be a relational data store. It might not be. You might want an event store. You might want to reduce costs with lambdas and DynamoDB. You may need a pub/sub event broker.
The OP clearly loves Postgres. Cool. They also have limited experience with complex systems architectures because if they had that experience, they would have never written this article.
I found MariaDB to be wonderfully simple to use for somewhat casual use cases: https://mariadb.com/docs/server/ha-and-performance/optimizat... and still reach for it in some personal projects, however the whole growing MySQL incompatibility is a big issue if the tech you use only officially supports MySQL and you can't (easily) get MariaDB specific DB drivers.
Personally, one of the best things about PostgreSQL is transactional DDL, every DB should support it. Also they handle JSON pretty nicely (though I'd prefer not to store data like that unless necessary) alongside excellent plugins like pgvector and PostGIS.
On the other hand, for things like queues, or even any sort of blob storage, I'd look at things like RabbitMQ or Garage (S3 compatible). Sometimes specialized software is nice for keeping things logically separated. I maintain that it's good to be able to divide your stack up by mechanisms/concerns (rather than business domain necessarily).
* As a message queue: Only if your required features are very basic, like if you need cluster communication and run your own coordination protocol on top.
* High Volume Time Series: TimeScale works, but composes badly with other workloads on the same DB server ( from an operational perspective at scale )
* Vector Database: The same issues as with TimeScale.. PgVector for example lives in its own seperate "world" and the query planner sees it as a very opaque thing. Forget about adding vector storage to an existing high volume db, that must server other complex queries.. PGVector will either trash your caches, or take over your cpu so that workloads that used to work fine stall. This is IMO not a pgvector problem itself ( Kudos to those guys ) but rather that postgresql extension apis are not very good at exposing custom costs and tradeoffs to the system as a whole.
* Raw Data: Works for small files... why anyone would want to store large amounts of data in it would be a mystery, where it shines is accessing LOTS of small files where internal caching etc help a lot compared to raw filesystem access ( also a bit dependent on the filesystem and its tuning though )
* Microservice: If your service is ONLY exposing json data from some database model, then it should not exist at all IMO. Create a view and be done with it.
Yeah, this has nothing to do with Postgres. If the service is accessing a database that isn't internal to the service, then that database is already a standalone service in itself.
http://www.postgis.net
Considering adding mongo for unstructured data? Just use postgres jsonb.
Building a search index? Postgres is fine too.
Considering using redis for fragment caching? Just use an unlogged table in postgres with key value columns. Need pub/sub? Well just use postgres listen/notify.
Using postgres for everything has served me very well.
I've done that before and the code was a mess. It works at the beginning but APIs do much more than piping data from the database. When you start dealing with ACL, external calls, code reuse, etc. It's just nice to have all the tools available to you from something like Python or Go.
The relational model and sql force us to simplify our data models too much by eliminating relationships or just not dealing with them.
Think about a nested json blob from some web service api and storing it in SQL in normalized tables. No one is going to do that. Everything just becomes a denormalized mess and everything is hacked around it.
Instead of modeling things in the proper way, most of the world's data is modeled in a way so that we don't have join explosions in sql queries because they look scary. Data pipelines become these scary batch transformations where data is dumped somewhere else without anyway to trace back where it came from.
I encounter so many end-user applications and systems where you wonder: "why couldn't they allow a list of items here instead of a single box" or "why can't this reference this other thing".
Postgres has built in data types and functions that allows it to work with unstructured json documents, like you would use in MongoDB.
I use a SQL databases as needed. I've used Postgres, Sqlite, Duckdb, Json files with AWS Athena, Oracle enterprise for ERP systems (a multitude of schemas and objects with interoperability), and others.
I'm currently, deploying Duckdb with AWS S3 Tables (Iceberg) to see how it fits for a use case I have.
IT is great and always changing. Keep trying new things.
Cheers
MySQL was generally faster, and while MyISAM was a bit limited Innodb was pretty powerful, and you had the choice. It was also simpler (imo) and avoided a lot of the xid/vacuum issues.
That said, still love Postgres. But at the time it started eclipsing MySQL, MySQL felt better positioned.
- Query planner is much worse (just yesterday I had to USE INDEX to sped up a query by 300x, I'm near-certain postgres would just have gotten it right) - Indexes are much more limited: no GIST, no GIN - No transactional lock (`pg_advisory_xact_lock` in postgres). This one was very surprising, it's a really useful thing and I had to implement it myself as a lock table
PostgreSQL also had more features back then, e.g. the JSON support is very nice if you need to do anything that doesn't neatly fit into the relational model.
Then people who knew something got involved (or likely were involved all along - but I never followed MySQL so I'm not sure) and fixed those because they matter and suddenly the crowd shut up.
Also postgres is a "proper" db, so I'm glad it generally won out.
MySQL caught up as well as far as I know, but it still may have some poor defaults that are widely used.
During the dot com era it was common to develop and launch on MySQL with the intention of migrating to something else if they became successful (though your typical LAMP stack developer regarded Oracle and SQL Server as being deeply 'weird', so many were willing to stick with MySQL despite the well-known limitations of MyISAM).
From where I'm standing, it seems that PostgreSQL became clearly preferable for new projects from the mid 2000s onwards, but it was only the Oracle acquisition that began to push existing users off MySQL.
edit to add: MySQL did have operational advantages even later than mid 2000s since it had master/master replication from very early (I don't know how sound it was, I doubt it was perfect). That was a real reason to choose it. We still don't have it in Postgres without extensions and even there citus is not really the same.
(I remember it being heavily touted in the first edition of the O'Reilly "High Performance MySQL" book. The second edition was about twice the length, with most of the additional pagecount going into detailed explanations of why you should actually be very careful and do lots of testing before deciding to rely on master/master!)
Actually, I remember that one of the drivers away from MySQL before Oracle came along was the 4.x and 5.x period, when there were various performance regressions. And even when those were sorted out, the introduction of InnoDB made people realise that MySQL's apparent speed advantage was really just down to MyISAM lacking referential integrity and transactions.
Certainly, there were people clinging on to MySQL 3.23 for read-heavy data warehousing applications for a very long time.
Yes we saw the same thing play out with MongoDB. Ack without fsync is indeed very fast.
Another thing: those were times when web applications were practically 99% reads, and not so great ACID was a non-issue.
Postgres is OK, but it has really a lot of quirks that are not that obvious.
I don't think this is accurate and smells like an LLM hallucination to me.
From the Timescale/Tiger Data _pgvectorscale_ project's README:
> pgvectorscale builds on pgvector with higher performance embedding search and cost-efficient storage for AI applications.
I think this is where the confusion originates. I believe pgvector is primarily Andrew Kane (@ankane) and a cadre of OSS contributors.
As an aside, I've used Timescale/Tiger Data products and was very happy with them and their support. Their team was very engaged and responsive to all of our questions. They also fixed a pretty gnarly indexing bug I uncovered in pgvectorscale in an impressively short amount of time.
It is already a quite smooth experience, but there is work to make it even easier than that.
I work on Lakebase, opinions my own.
In most of the applications we build or maintain we use PostgreSQL + cloud storage. That's it. And it works very well, also for: storing JSON, full text search, as a queue, as a vector database. Other software may be better at providing those features, but I'm extremely happy we only need to understand & manage PostgreSQL.
Coming from storing billions of rows in Clickhouse and performing dozens of materialized operations I shudder to think about what that would look like in a DB that doesn’t even support declarative IVM.
42 is not the answer to everything.
42 is the Answer to the Ultimate Question about Life, the Universe, and Everything.
For instance, for many simple needs MySQL is simpler than Postgres, with similar performance and consistency.
* No need for a connection pool, while many use cases with Postgres require PgBouncer and Co.
* Easy sort (and basic search) of multilingual text, because MySQL has case insensitive UTF8 collations.
* No need to VACUUM, which can be a hard problem (it was, the last time I used Postgres).
For full text search, I once worked on a project that considered several alternatives for this, including Postgres. Manticore Search was finally chosen because it was more performant, with better search results.
is there a strong evidence you even need client side connection pool at all? What is the purpose?
The limitation is that you have many clients with connection pools, they hold internal PG connection without allowing it to be reused by other clients..
Not sure if I'm missing anything here, but if I want case-insensitive search I simply create an index on lower(column) and use that to query.
VACUUM is something you need to pay attention to at scale. And at that point you need to know your DB anyway and tune it. For smaller applications (and I don't mean only toy applications) it usually isn't an issue.
Or even pg_trgm trigram indexes, which are case-insensitive by default and support similarity search to accept typos and misspellings.
The job queue runs on the cache db, scheduling jobs to move data from bigquery into postgres. It’s pretty neat.
Now we’ve run into near-real-time requirements so clickhouse is getting thrown into the mix.
It’s pretty funny the lengths we go to to implement user facing analytics that’s basically just “you are visitor number X” from 1995.
Different use cases have different scalability limits in PG, when you get to them you need to deal with them.
It would be perfect if it had somewhat transparent sharding, I mean a way to add another instance and distribute load without having to stop everything.
There are solutions, but they tend to be involved and when you get to that point in many cases it makes sense to just move that workload to something else that scales better.
- https://www.reddit.com/r/PostgreSQL/comments/1vbo5j8/raw_xml...
- your post did not have a single word on XML hence my comment
NVMe drives + Litestream + object storage(S3/R2..). sqlite simplifies things for the entire long tail of apps/services that aren't the Ubers and AirBNBs of the world.
Also, apparently Litestream could use a filesystem instead of an object store?
But I sure wish it was 'core' and we didn't have to worry about it potentially going away, becoming de-supported..
https://github.com/seanwevans/pg_gpt2
https://github.com/seanwevans/pg_shell
https://github.com/seanwevans/pg_os
so yes, i'm still a postgres maximalist (worker queues still in pg [1]) but (especially in the age of quick LLM prototypes) it's always worth measuring the more purpose-built approach.
[0]: https://setoku.com
[1]: https://worker.graphile.org
Not 100% sure about using PG for file system at scale however. I'd love to hear more on the challenges (vacuum, toast, anything else?)
then Sqlite works as well too. can run the whole thing on Cloudflare.
running Postgres isn't difficult. but dealing with a VPS for low traffic is a headache that's not necessary.
No deamon. Single file per DB. Less configuration overhead.
Supporting more than 1 writer per process. Strict typing. Access controls. Replication at scale is more effecient than copy-pasting files (seems SQLite has improved on this one).
But for big, multi-writer work PG is the way to go.
I demonstrated this with ClickHouse: https://github.com/ClickHouse/pg_clickhouse/blob/main/doc/of...
We're working on a chdb based mechanism to copy to/from s3, maybe with fdw on top we can back table in s3
You can try similar things with pg_duckdb & pg_lake
I have decided to use clickhouse with that config because of missing S3 for logs and metrics for long term store.
This is a not great start. I assume it refers to MyISAM which has not been relevant for over a decade at this point. InnoDB made different design than PG decisions and was (and perhaps still is) faster at point lookups.
multiple processes connected to it.
But a lot of companies are trying to solve that, notably multigres, neki and even pgdog.
0: https://www.postgresql.org/docs/19/ddl-property-graphs.html
1: https://www.postgresql.org/docs/19/queries-graph.html