HN Reader

One of the more detrimental aspects of the Cloud Tax is that it constrains the types of solutions engineers even consider.

Picking an arbitrary price point of $200/mo, you can get 4(!) vCPUs and 16GB of RAM at AWS. Architectures are different etc., but this is roughly a mid-spec dev laptop of 5 or so years ago.

At Hetzner, you can rent a machine with 48 cores and 128GB of RAM for the same money. It's hard to overstate how far apart these machines are in raw computational capacity.

There are approaches to problems that make sense with 10x the capacity that don't make sense on the much smaller node. Critically, those approaches can sometimes save engineering time that would otherwise go into building a more complex system to manage around artificial constraints.

Yes, there are other factors like durability etc. that need to be designed for. But going the other way, dedicated boxes can deliver more consistent performance without worries of noisy neighbors.

A thoroughly good article. It's probably worth also considering adding a CDN if you take this approach at scale. You get to use their WAF and DNS failover.

A big pain point that I personally don't love is that this non-cloud approach normally means running my own database. It's worth considering a provider who also provides cloud databases.

If you go for an 'active/passive' setup, consider saving even more money by using a cloud VM with auto scaling for the 'passive' part.

In terms of pricing the deals available these days on servers are amazing you can get 4GB RAM VPSs with decent CPU and bandwidth for ~$6 or bare metal for ~$90 for 32GB RAM quad core worth using sites like serversearcher.com to compare.

Regardless of the cost and capacity analysis, it's just hard to fight the industry trends. The benefits of "just don't think about hardware" are real. I think there is a school of thought that capex should be avoided at all costs (and server hardware is expensive up front). And above all, if an AWS region goes down, it doesn't seem like your org's fault, but if your bespoke private hosting arrangement goes down, then that kinda does seem like your org's fault.

Just today I wasted some time due to an unexpected Tailscale key expiry and some other issues related to running a container cluster: https://blog.kronis.dev/blog/the-great-container-crashout

Right now, my plan is to move from a bunch of separate VPSes, to one dedicated server from Hetzner and run a few VMs inside of it with separate public IPs assigned to them alongside some resource limits. You can get them for pretty affordable prices, if you don't need the latest hardware: https://www.hetzner.com/sb/

That way I can limit the blast range if I mess things up inside of a VM, but at the same time benefit from an otherwise pretty simple setup for hosting personal stuff, a CPU with 8 threads and 64 GB of RAM ought to be enough for most stuff I might want to do.

This isn't even the end game for "one big server". AMD will give the most bang per rack, but there are other factors.

An IBM z17 is effectively one big server too, but provides levels of reliability that are simply not available in most IT environments. It won't outperform the AMD rack, but it will definitely keep up for most practical workloads.

If you sit down and really think honestly about the cost of engineering your systems to an equivalent level of reliability, you may find the cost of the IBM stack to be competitive in a surprising number of cases.

I've been doing hybrid colo+public cloud for over a decade and it's always been the most cost effective route at a certain scale. That specific break even point is lowering over time with the density and cost effectiveness of hardware.

Sure you need net/infra admins but the software and hardware these days are pretty management friendly and you'll find you still need (often more expensive "cloud") admins so you're not offsetting much management cost there.

Colocation is plentiful and providers often aggregate and resell bandwidth from their preferred carriers.

At one point we were up to 8 dell vrtx clusters and a few SANs, with 500+ VMs from huge msSQL servers to kube clusters the public cloud bill would have been well into the 6 figures even with preferred pricing and reserved instances. Our colocation bill was $2400/mo and that was mostly for power. The one thing that always surprised me was how much faster everything was - every time we had to scale-over into the cloud the public cloud node was noticably slower even for identical CPU generations and vcpu.

You need to be very keen about server deals, updates, support contracts and licenses - but it's really manageable and interconnecting with the cloud is trivial at this point - you can get a "cloud connect" fiber drop to your preferred cloud provider and connect your colo infra to your vpc.

HN uses two—one live and one backup, so we can fail over if there's a hardware issue or we need to upgrade something.

It's a nice pattern. Just don't make them clones of each other, or they might go BLAM at the same time!

https://news.ycombinator.com/item?id=32049205

https://news.ycombinator.com/item?id=32032235

https://news.ycombinator.com/item?id=32028511 (<-- this is where it got figured out)

---

Edit: both these points are mentioned in the OP.

A lot of the time businesses just aren't that important. The amount places I've seen that stress over uptime when nothing they run is at all critical. Hell you could drop the production environment in the middle of the day and yes it would suck and you'd get a few phone calls but life would go on.

These companies all ended up massively increasing their budgets switching to cloud workloads when a simple server in the office was easily enough for their 250 users. Cloud is amazing for some uses and pure marketing BS for others but it seems like a lot of engineers aim for a perfect scalable solution instead of one that is good enough.

I’ve found that it’s hard to even hire engineers who aren’t all in on cloud and who even know how to build without it.

Even the ones who do know have been conditioned to tremble with fear at the thought of administrating things like a database or storage. These are people who can code cryptography kernels and network protocols and kernel modules, but the thought of running a K8S cluster or Postgres fills them with terror.

“But what if we have downtime!” That would be a good argument if the cloud didn’t have downtime, but it does. Most of our downtime in previous years has been the cloud, not us.

“What if we have to scale!” If we are big enough to outgrow a 256 core database with terabytes of SSD, we can afford to hire a full time DBA or two and have them babysit a cluster. It’ll still be cheaper.

“What if we lose data?” Ever heard of backups? Streaming backups? Hot spares? Multiple concurrent backup systems? None of this is complex.

“But admin is hard!” So is administrating cloud. I’ve seen the horror of Terraform and Helm and all that shit. Cloud doesn’t make admin easy, just different. It promised simplicity and did not deliver.

… and so on.

So we pay about 1000X what we should pay for hosting.

Every time I look at the numbers I curse myself for letting the camel get its nose under the tent.

If I had it to do over again I’d forbid use of big cloud from day one, no exceptions, no argument, use it and you’re fired. Put it in the articles of incorporation and bylaws.

I helped bootstrap a company that made an enterprise automation engine. The team wanted to make the service available as SaaS for boosting sales.

They could have got the job done by hosting the service in a vps with a multi-tenant database schema. Instead, they went about learning kubernetes and drillingg deep into "cloud-native" stack. Spent a year trying to setup the perfect devops pipeline.

Not surprisingly the company went out of business within the next few years.

I run on VPSs as well. I ditched cloud a long time ago. Once my project starts making money, I will definitely buy my own hardware and collocate. Cloud is like dating apps. We had fun for a decade but it's time to get serious and get some things actually done and be productive again.

The complexity you introduce trying to achieve 100% uptime will often undermine that goal. Most businesses can tolerate an hour or two of downtime or data loss occasionally. If you set this expectation early on, you can engineer a much simpler system. Simpler systems are more reliable.

Related ongoing thread:

How many HTTP requests/second can a single machine handle? (2024) - https://news.ycombinator.com/item?id=45085446 - Aug 2025 (32 comments)

I often wonder if my home NAS/Server would be better off put onto a rented box or a cloud server somewhere, especially since I now have 1gbit/s internet. Even now the 20TB of drive space and 6 Cores with 32GB on Hetzner with a dedicated is about twice the price of buying the hardware over a 5 year period. I suspect the hardware will actually last longer than that and its the same level of redundancy (RAID) on a rented dedicated so the backup is the same cost between the two.

Using cloud and box storage on Hetzner is more expensive than the dedicated server, 4x owning the hardware and paying the power bill. AWS and Azure are just nuts, >100x the price because they charge so much for storage even with hard drives. Contabo nor Netcup can do this, its too much storage for them.

Every time I look at this I come to the same basic conclusion, the overhead of renting someone else’s machine is quite high compared to the hardware and power cost and it would be a worse solution than having that performance on the local network for bandwidth and latency. The problem isn't so much the compute performance, that is relatively fairly priced, its the storage costs and data transfer that bites.

Not really what the article was necessarily about but cloud is sort of meant to be good for low end hardware but its actually kind of not, the storage costs are just too high even a Hetzner Storage box.

These days we have more meta-software than software. Instead of Apache with virtualhosts, we have a VM running Docker instances, each with an nginx of its own, all connected by a separate Docker of nginx acting as a proxy.

How much waste is there from all this meta-software?

In reality, I host more on Raspberry Pis with USB SSDs than some people host on hundred-plus watt Dells.

At the same time, people constantly compare colo and hardware costs with the cost per month of cloud and say cloud is "cheaper". I don't even bother to point out the broken thinking that leads to that. In reality, we can ignore gatekeepers and run things out of our homes, using VPSes for public IPs when our home ISPs won't allow certain services, and we can still have excellent uptimes, often better than cloud uptimes.

Yes, we can consolidate many, many services in to one machine because most services aren't resource heavy constantly.

Two machines on two different home ISP networks backing each other up can offer greater aggregate uptime than a single "enterprise" (a misnomer, if you ask me, if you're talking about most x86 vendors) server in colo. A single five minute reboot of a Dell a year drops uptime from 100% to 99.999%.

Cloud is such bullshit that it's exhausting even just engaging with people who "but what if" everything, showing they've never even thought about it for more than a minute themselves.

I did this (well, a large-r VPS for $120/month) for my Rails-based sports streaming website. I had a significant amount of throughput too, especially at peak (6-10pm ET).

My biggest takeaway was to have my core database tables (user, subscription, etc) backed up every 10 minutes, and the rest every hour, and test their restoration. (When I shut down the site it was 1.2TB.) Having a script to quickly provision a new node—in case I ever needed it—would have something up within 8 minutes from hitting enter.

When I compare this to the startups I’ve consulted for, who choose k8s because it’s what Google uses yet they only push out 1000s of database queries per day with a handful of background jobs and still try to optimize burn, I shake my head.

I’d do it again. Like many of us I don’t have the need for higher-complexity setups. When I did need to scale, I just added more vCPUs and RAM.

Bare-metal servers sound super cheap when you look at the price tag, and yeah, you get a lot of raw power for the money. But once you’re in an enterprise setup, the real cost isn’t the hardware at all, it’s the people needed to keep everything running.

If you go this route, you’ve got to build out your own stack for security, global delivery, databases, storage, orchestration, networking ... the whole deal. That means juggling a bunch of different tools, patching stuff, fixing breakage at 3 a.m., and scaling it all when things grow. Pretty soon you need way more engineers, and the “cheap” servers don’t feel so cheap anymore.

Previously: https://news.ycombinator.com/item?id=32319147

Microservices vs not is (almost) orthogonal to N servers vs one. You can make 10 microservices and rent a huge server and run all 10 services. It's more an organizational thing than a deployment thing. You can't do the opposite though, make a monolith and spread it out on 10 servers.

I’ve been having those discussions with friends for the last 3 or 4 years. The downside of having local infra is pretty much having someone that has the experience to do it right. While this article covered the higher end, the math on the lower end tends to work out at 1 year of ownership depending on what you already have because you will probably already have a small rack and some networking gear.

My main concern is that at the current cloud premiums rates, I will be better off even if I need to hire someone specifically for managing the local infra.

This was written in 2022, but looks like it's most still relevant today. Would be interesting to see updated numbers on the expected costs of various hosting providers.

I work for a cloud provider and I'll tell you, one of the reasons for the cloud premium is that it is a total pain in the ass to run hardware. Last week I installed two servers and between them had four mysterious problems that had to be solved by reseating cards, messing with BIOS settings, etc. Last year we had to deal with a 7 site, 5 country RMA for 150 100gb copper cables with incorrect coding in their EEPROMs.

I tell my colleagues: it's a good thing that hardware sucks: the harder it is to run bare metal, the happier our customers are that they choose the cloud. :)

(But also: this is an excellent article, full of excellent facts. Luckily, my customers choose differently.)

The problem is sizing and consistency. When you're small, it's not cost effective to overprovision 2-3 big servers (for HA).

And when you need to move fast (or things break), you can't wait a day for a dedicated server to come up, or worse, have your provider run out of capacity (or have to pick a different specced server)

IME, having to go multi cloud/provider is a way worse problem to have.

and now consider 6th Gen EPYC will have 256 cores also you can have 32 hot-swap SSDs with like 10mil plus of random write IOPS and 60mil plus random read IOPS in a single 2U box

Yes, please. I mostly run a VM per project but will consolidate some stuff now that Kamal 2 supports it. I also posted before how this works here: https://nts.strzibny.name/multiple-apps-single-server-kamal-...

Don't forget the cost of managing your one big server and the risk of having such single point of failure.

Being a big server proponent myself. Usually for one reason or the other there is need to introduce some socket style communication to the frontend and that becomes impossible in a single machine after a certain threshold.

Is there something obvious that I'm missing?

I used a colo once a few years ago at a small datacenter in the midwest, I was shocked at how unprofessional everything was, machines laying in the hallway, a guy was sleeping in one of the offices. They let me setup my server and was left unattended several times, I could have just poked the power button on a nearby server or moved a cable or whatever. It was a 1.5 hour drive away, and I wasn't running anything serious so I just went with it but pulled my stuff out after my 1 year subscription was up.

>Unfortunately, since all of your services run on servers (whether you like it or not), someone in that supply chain is charging you based on their peak load.

This seems fundamentally incorrect to me? If I need 100 units of peak compute during 8 hours of work hours, I get that from Big Cloud, and they have two other clients needing same in offset timezones then in theory the aggregate cost of that is 1/3rd of everyone buying their own peak needs.

Whether big cloud passes on that saving is another matter, but it's there.

i.e. big cloud throws enough small customers together so that they don't have "peak" per se just a pretty noisy average load that is in aggregate mostly stable

The one big box assumes that you know how to configure everything for high performance. I suspect that skill has been lost, for the most part.

You really need to tweak the TCP/IP stack, buffer sizes, and various other things to get everything to work really well under heavy load. I'm not sure if the various sites that used to talk about this have been updated in the last decade or so, because I don't do that anymore.

I mean, you'll run out of file descriptors pretty quickly if you try to handle a few hundred simultaneous connections. Doesn't matter how big your box is at that point.

I'm in the process of breaking up a legacy deployment on "one big server" into something cloud native like Kubernetes.

The problem with one big server is that few customers have ONE (1) app that needs that much capacity. They have many small apps that add up to that much capacity, but that's a very different scenario with different problems and solutions.

For example, one of the big servers I'm in the process of teasing apart has about 100 distinct code bases deployed to it, written by dozens of developers over decades.

If any one of those apps gets hacked and this is escalated to a server takeover, the other 99 apps get hacked too. Some of those apps deal with PII or transfer money!

Because a single big server uses a single shared IP address for outbound comms[1] this means that the firewall rules for 100 apps end up looking like "ALLOW: ANY -> ANY" for two dozen protocols.

Because upgrading anything system-wide on the One Big Server is a massive Big Bang Change, nobody has had the bravery to put their hand up and volunteer for this task. Hence it has been kept alive running 13 year old platform components because 2 or 3 of the 100 apps might need some of those components... but nobody knows which two or three apps those are, because testing this is also big-bang and would need all 100 apps tested all at once.

It actually turned out that even Two Big (old) Servers in a HA pair aren't quite enough to run all of the apps so they're being migrated to newer and better Azure VMs.

During the interim migration phase instead of Two Big Server s there are Four Big Servers... in PRD. And then four more in TST, etc... Each time a SysOps person deploys a new server somewhere, they have to go tell each of the dozens of developers where they need to deploy their apps today.

Don't think DevOps automation will rescue you from this problem! For example in Azure DevOps those 100 apps have 100 projects. Each project has 3 environments (=300 total) and each of those would need a DevOps Agent VM link to the 2x VMs = 600 VM registrations to keep up to date. These also expire every 6 months!

Kubernetes, Azure App Service, AWS App Runner, and GCP App Engine serve a purpose: They solve these problems.

They provide developers with a single stable "place" to dump their code even if the underlying compute is scaled, rebuilt, or upgraded.

They isolate tiny little apps but also allow the compute to be shared for efficient hosting.

They provide per-app networking and firewall rules.

Etc...

[1] It's easy to bind distinct ingress IP addresses on even a single NIC (or multipe), but it's weirdly difficult to split the outbound path. Maybe this is easier on Linux, but on Windows and IIS it is essentially impossible.

A lot of these articles look at on-demand pricing for AWS. But you're rarely paying on-demand prices 24/7. If you have a stable workload, you probably buy reserved instances or a compute savings plan. At larger scales, you use third party services to get better deals with more flexibility.

A while back I looked into renting hardware, and found that we would save about 20% compared to what we actually paid AWS – in partially because location and RAM requirements made the rental more expensive than anticipated, and partially because we were paying a lot less than on-demand price for AWS.

20% is still significant, but it's a lot less than the ~80% that this and other articles suggest.

And then boom, all your services are gone due to a pesky capacitor on the motherboard. Also good luck trying to change even one software component of that monolith without disrupting and jeopardizing the whole operation.

While it is a useful advice to some people in certain conditions, it should be taken with a grain of salt.

> Part of the "cloud premium" for load balancers, serverless computing, and small VMs is based on how much extra capacity your cloud provider needs to build in order to handle their peak load. You're paying for someone's peak load anyway!

Eh, sort of. The difference is that the cloud can go find other workloads to fill the trough from off peak load. They won’t pay as much as peak load does, but it helps offset the cost of maintaining peak capacity. Your personal big server likely can’t find paying workloads for your troughs.

I also have recently come to the opposite conclusion for my personal home setup. I run a number of services on my home network (media streaming, email, a few personal websites and games I have written, my frigate NVR, etc). I had been thinking about building out a big server for expansion, but after looking into the costs I bought 3 mini pcs instead. They are remarkably powerful for their cost and size, and I am able to spread them around my house to minimize footprint and heat. I just added them all to my home Kubernetes cluster, and now I have capacity and the ability to take nodes down for maintenance and updates. I don’t have to worry about hardware failures as much. I don’t have a giant server heating up one part of my house.

It has been great.

Those servers are mainly designed for enterprise use cases. For hobby projects, I can understand why someone would choose Hetzner over AWS.

For enterprise environments, however, there is much more to consider. One of the biggest costs you face is your operations team. If you go with Hetzner, you essentially have to rebuild a wide range of infrastructure components yourself (WAF, globally distributed CDN, EFS, RDS, EKS, Transit Gateways, Direct Connect and more).

Of course, you can create your own solutions for all of these. At my company, a mid-size enterprise, we once tried to do exactly that.

WAF: https://github.com/TecharoHQ/anubis

CDN: Hetzner Nodes with Cache in Finnland, USA and GER

RDS: Self-hosted MySQL from Bitnami

EFS: https://github.com/rook/rook

EKS: https://github.com/vitobotta/hetzner-k3s

and 20+ more moving targets of infra software stack and support systems

The result was hiring more than 10 freelancers in addition to 5 of our DevOps engineers to build it all and handling the complexity of such a setup and the keep everything up-to-date, spending hundreds of thousands of dollars. Meanwhile, our AWS team, consisting of only three people working with Terraform, proved far more cost-effective. Not in terms of dollars per CPU core, but in terms of average per project spending dollars once staff costs and everything were included.

I think many of the HN posts that say things like "I saved 90% of my infra bill by moving from AWS to a single Hetzner server" are a bit misleading.

Ah, the folksy wisdom of the armchair. Sounds convincing, doesn't it? I mean, it includes math! And prices! The quoted prices are more expensive for the cloud. And he makes folksy claims that make sense, like the "fragile complexity" of having "more than one computer". It makes sense! Right??

But is he right? How do we know? Well for starters, look at his CV. He has never managed servers for a living. The closest he's come is working on FPGAs. So what's he basing all these opinions on? Musings? Thoughts? Feelings? Hope?

He makes a couple claims which it isn't obvious are bunk, so I'll address them here, in reverse order.

"microservice architectures in general add a lot of overhead to a system for dubious gain when you are running on one big server" - Microservices architectures are not about overhead or efficiency. They are an attempt to use good software design principles to address Conway's Law. If you design the microservice correctly, you can enable many different groups in an organization to develop software independently, and come up with a highly effective and flexible organization and stable products. Proof? Amazon. But the caveat is, you have to design them correctly. Almost everyone fails at this.

"It's impossible to get the benefits of a CDN, both in latency improvements and bandwidth savings, with one big server" - This is so dumb I'm not sure I have to refute it? But, uh, no, CDNs absolutely give a heap of benefits whether you have 1 server or 1,000. And CloudFlare Free Plan is Free.

"My Workload is Really Bursty - Cloud away." - Unless your workload involves massive amounts of storage or ingress/egress and your profit margin tiny, in which case you may save more by building out a small fleet of unreliable poorly-maintained colocated servers (emphasis on may).

"The "high availability" architectures you get from using cloudy constructs and microservices just about make up for the fragility they add due to complexity. ... Remember that we are trying to prevent correlated failures. Cloud datacenters have a lot of parts that can fail in correlated ways. Hosting providers have many fewer of these parts. Similarly, complex cloud services, like managed databases, have more failure modes than simple ones (VMs)." - Argument from laziness, or ignorance? He's trying to say that because something is complex it's also less reliable. Which completely ignores the reliability engineering aspect of that complexity. You mitigate higher numbers of failure modes by designing the system to fail over reliably. And you also have warm bodies running around replacing the failing parts, which fights entropy. You don't get that in a single server; once your power supply, disk, motherboard, network interface, RAM, etc fails, and assuming your server has a redundant pair, you have a ticking clock to repair it until the redundant pair fails. How lucky do you feel? (oh, and you'll need downtime to repair it.)

As usual, the cloud costs quoted is MSRP, and if you're paying retail, you're a fool. Almost all cloud costs can be brought down from 25%-75%, spot instances are a fraction of the on-demand server cost, and efficient use of cheaper cloud services reduces your need to buy compute at all.

"The big drawback of using a single big server is availability. Your server is going to need downtime, and it is going to break. Running a primary and a backup server is usually enough, keeping them in different datacenters. A 2x2 configuration should appease the truly paranoid: two servers in a primary datacenter (or cloud provider) and two servers in a backup datacenter will give you a lot of redundancy. If you want a third backup deployment, you can often make that smaller than your primary and secondary." - Wait... so One Big Server isn't enough? Huh. So this was a clickbait article? I'm shocked!

"One Server (Plus a Backup) is Usually Plenty" - Plenty for what? I mean we haven't even talked system architecture or application design. But let's assume it's a single microservice that gets 1RPS. Is your backup server a hot spare, cold spare, or live mirror? If it's live, it's experiencing the same wear, meaning it will fail at about the same time. If it's hot, there's less wear, but it's still experiencing some. If it's cold, you get less wear, but you're less sure it'll boot up again. And then there's system configuration. The author mentions the "complexity" of managing a cluster, but actually it's less complex than managing just two servers. With a fleet of servers, you know you have to use automation, so you spend the time to automate their setup and run updates frequently. With a backup, you probably won't do any maintenance on the backup, and you definitely won't perform the same operations on the backup as the server. So the system state will drift wildly, and the backup's software will be useless. It would be better to just have it as spare part.

The author never talks about the true failure modes of "one big server". When parts start to need replacing, it's never cheap. Smart hands cost, cost of the parts+shipping, cost of the downtime. And often you'll find there are delays - delays in getting smart hands to actually repair it correctly, delays in shipping, delays in part ordering/availability. Running out of power, running out of space, temperatures too high, "flaky" parts you can't diagnose, backups and restores, datacenter issues, routing issues, backbone issues. You'll tell yourself these are "probably rare" - but these are all failure modes, and as the author tells us, you should be wary of lots of failure modes. And anecdotes will tell you somebody has run a server for 10 years with no issue, while another person had a server with 3 faults in a month. To say nothing of the need to run "burn-in" on a new server to discover faults once it's racked.

Go ahead and do whatever you want. Cloud, colo, one server, multiple. There will be failures and complexity no matter what. You want to tell yourself a comforting story that there is "one piece of advice" to follow, some black and white world where only one piece of folksy wisdom applies. But here's my folksy wisdom: design your application, design your system to fit it, try not to pinch every penny, build something, and become educated enough to know what problems to expect and how to deal with them. Or if not, pay someone who can, and listen to them.