Up to a point, cloud can actually be very cost effective. If you are a small company, you don't actually need a lot. I ran our current service for almost one and a half year on Google Cloud Run for free. It stayed below the freemium threshold. Eventually I transitioned it over to two simple VMs. Main reason for this was that I abused these servers to also do background processing and Cloud Run is not very suitable for that as it can shut down servers in between requests and/or starve them of resources. But a small vm only costs something like 20$/month. Add another 25 or so for the load balancer and a Redis, and a few other bits and we're looking at ~100$ per month. That's very reasonable. That's not a cost worth optimizing without a very good reason. A good devops freelance person asks more per hour than that.
Where a lot of companies go wrong is premature scaling and overengineering. As you might guess from the above, I don't run kubernetes. Reason, the smallest cluster I can manage would probably be more expensive than the entirety of my setup. Before I even deploy anything. And it might not be big enough even.
I also don't run a managed postgresql. I'd like to. But I can't justify dropping north of 300$/month on that currently. A single node vm with backups is good enough. For now. I might go active master - passive master. Do I need a 3 or more node cluster? Not right now. I might one day. I always have the option to upgrade. But until then my DB is going to not break the bank. So, I run with a more modest/risky setup. It's a managed risk. I can buy it off when that becomes relevant.
The other day, our CEO vibe coded a cost calculator thingy. He did a nice job. It had to be deployed somewhere. So, I used Cloud Run. Running a small thing with essentially no traffic is as close to free as you can get. I slapped some basic authentication on it and used a git hub action that we run from the freemium allocated build minutes each month. I reviewed our Github bills a few days ago over the years: The most we payed was a few months where we were running out of build minutes in 2024. I payed 55cents for one of those bills. Our setup is cloud based but frugal. I could probably lower our bills by double digit percentages by moving everything into Hetzner. But it's literally not worth my time.
Cloud on the cheap is just really good value. You can host low traffic static websites on Cloudflare. If you have a load balancer in Google cloud (25$/month or so) you can use buckets and their CDN for next to nothing as an alternative. A small Redis costs next to nothing. We run a slightly expensive vm for batch jobs for a maybe ~2 hours per month in total. Each job only takes a few minutes. The machine is suspended in between jobs and we only pay for the hours it runs. Next to nothing.
So, yes, you can get some bare metal in Hetzner or wherever for next to nothing. But then you need to manage that setup and you need to put the time in to do that. Your time even if you don't value it personally is just about the most scarce and expensive resource a startup has if you are the CTO. The difference between you doing valuable work that moves the company forward or you procrastinating and penny pinching over stuff that does not matter can literally make or break the company.
If you spend 5K on infrastructure per month, that's a the same as a very low salary for a full time person. If you are spending that much, definitely do something because it's an obscene amount. Especially if you are early stage and have no traffic. But if you need to put somebody on that that costs closer to 10K/month for a few weeks or months, that's a project that has a pretty high price tag as well. So cutting 5K down to 1K is worth doing. But spending 10K to cut 1K to 500/month is probably going to take you years to earn back. And you lose time doing it.
Our startup is bootstrapped. We run on a bit of revenue now and our savings for most of the past five years. Anything I spend on stuff that isn't needed literally is money that otherwise would go to my bank account (via a salary raise or bonus). I understand frugality. A lot of this stuff is people getting spoiled with investor cash and spending like there is no tomorrow. If you have no accountability you optimize the infrastructure for how good it makes your resume look, not for cost.