(VP of Eng here) I filled out their sales form last year and never heard from them. We're currently spending $900k/year on AWS, and they didn't even acknowledge my request. crazy stuff.
Is this true? If so, how do you know? I have listened to almost of their podcasts. I don't recall them saying there are any type of customer they refuse to sell to. They told a funny story about a sales call with a US national laboratory. They went into the call assuming they would be asking for supercomputer. Instead, they learned they need a bunch of regular rack compute, not all supercomputers.
Also, the OP did not say they are a SaaS company. They only said they spend 900K USD per year with AWS.
I also think they work with government customers. I saw an open job position on their website requiring TS/SCI security clearance and full scope polygraph.
We had the opposite experience. We contacted them through their form this year and they were happy to discuss with us even when we communicated from the beginning we wouldn't be customer in the short to medium term and even for single rack systems.
The rack were not insanely expensive, expandable so you don't have to build fully stacked racks from the beginning. Purely based on a hardware, compares to other blade systems but pricier.
The interesting part was the software, management interface, Terraform provider and how everything just fit together. Having storage, network and compute all in a single managed rack package brings a lot of value and brings down TCO. Really appreciated the security group like approach to network policies.
Unfortunately I haven't worked with HPE or Dell recently so I'm not sure what they currently offer.
Historically, selling to a small niche of price-insensitive customers (e.g. mainframes, defense) breeds complacency and eventual failure. You can see VMware go down that death spiral over its history where every release was more expensive with less innovation.
> Historically, selling to a small niche of price-insensitive customers (e.g. mainframes, defense) breeds complacency and eventual failure.
VMware was a publicly traded company got more than a 20 year run where they dominated virtualization and captured over 80% of virtualized servers. That a publicly traded company sold out to another publicly traded company over and over after two decades where they got scraped for change in the couch cushions doesn't surprise me at all.
> (e.g. mainframes, defense)
And yet mainframes are still in use, still getting purchased, and Defense is by far the largest segment seeing movement in the US right now. TONS of dollars, much of it going into AI, drones, etc. Much of which needs digital and infrastructure development and deployment for R&D and operations.
Right now is probably an amazing time for them sales / interest wise. A LOT of money going into software and hardware attestation right now.
He asked the business for a sales rep through their sales portal a year ago. That is enough. Shouldnt have to complain a year later on a social media platform to get a response that says "hit me up".
Ignoring a sales request for a company that wants to spend ~1M only to say pls reach out to me again after we had our second funding round in 2026? The OP literally could go back to their sales lead db and respond like a real business. So yeah unprofessional and embarrassing.
You are 100% correct but you’d be shocked at the mortal terror that “self hosting” inflicts in the minds of even people who should really know better.
Also forget Dell. Check out DataPacket.com and other metal hosters. You don’t need to physically rack unless you are huge or have special hardware or security needs.
The cloud industry has done an incredible job at a kind of soft pervasive propaganda that running stuff is “hard.”
It is worth the exercise to honestly assess the maintenance, initial costs, ongoing costs, etc etc for owning your own. I did this at a company for a single server to let the ml group test fine tuning frameworks/throw crazy ideas at before spending real money on a training run and it paid for itself in like 3 months. Sometimes the math is overwhelmingly positive, sometimes it isn't. In general, the smaller/earlier you are the less it makes sense. Not because of the per hour costs, but because of the distraction away from implementing the core idea of the company. It is worth some, maybe a lot of, inefficiency to stay focused. When you get bigger and have strong fit and a clear direction it is a lot easier to focus on optimizing hosting costs and spending leadership time away from the core problem of the company.
Seems like people missed the end of what I wrote though. We run some bare metal but do not physically handle it. There’s many companies who will rent huge boxes with good connectivity in a data center. DataPacket is the best we have found for price and reliability.
Physically racking only makes sense if you are huge or have special requirements.
> Physically racking only makes sense if you are huge or have special requirements.
CISA has identified 16 US industries where they have noted that companies should be able to operate indefinitely without internet access. See the Water Plant hack stories coming through right now across more than 19 states, for example. Water, electricity, internet access / telecom, military, chemical, logistics, transportation, etc.
Oh man I cannot tell you how many times I've gotten into arguments on here and other technical forums years ago about self-hosting and how 'stupid' I was to not want to rent cloud servers
You’re expressing a very narrow engineer’s perspective that doesn’t consider the realities of managing bare metal hosting at any kind of scale.
These are business decisions, made in terms of core competencies, capex vs. opex, and the difficulties and cost of building out a reliable, sustainable hosting operation that handles all the compliance and security requirements, and the full range of “ilities” that real businesses have to deal with.
The fact that Bob in IT might be capable of doing some of this on his own doesn’t really enter into the picture. It’s not relevant.
There’s a reason that most companies don’t operate their own electricity generation systems. Much the same is true for computing systems.
Many people who have ever had to deal with the long tail of insanities in physical hosting, like "the ceiling burst and dumped water on a rack", "the RAID controller's capacitors exploded and now you need to figure out what still works", or "for some reason three of the servers won't talk to this switch but talk to anything else using the same cables, and the switch ports work for other devices", would happily pay a premium to only deal with SaaS logistics.
(Those were all firsthand examples; I'm not saying everyone needs cloud providers, but there are reasons beyond "really good salespeople" that people opt for offloading those logistics.)
Ok but add up a trickle of those examples, with staff to handle them, and you're still comfortably in the black.
You dont even have extra organizational overhead. Every cloud first company has a head of devops sitting in the chair where head of infra would be. They somehow wind up with like half the staffing anyways compared to running bare metal.
Our colo experience was pretty smooth, nothing insane like that, and saved us a bundle. Apparently ymmv. I hate working with AWS APIs by comparison, some of the worst UX I've ever seen.
I've dealt with both. I am continually amazed that people not only pay for AWS, but that it is so complicated, and that they use it in all sorts of absurd ways, not even just hosting 'vms' but using all sorts of amazon tools to do trivial tasks. I am sorry I just dont get it. Is it like learning salesforce and once you're sucked in you're just in? I am so glad I do not work at an AWS shop anymore and am very glad to be in a position where we do not use it.
You get AWS certifications, use as many AWS services as possible to pad your CV with the service experience and certifications, get your arguments and "well architected" dogma from AWS sales. Then you can get paid more at the next company (or at least high salary offers to extract a raise).
Also building high complexity systems that require a big expensive cloud engineer staff gets you status and visibility. In a tight spot you can still blame AWS for problems.
(Reality is not quite so bleak as people are not completely cynical)
I actually do get public cloud as well as running your own racks. There is room for both. Your startup is growing 100% every quarter but you're not sure if you have real PMF yet? Yeah probably stick with cloud. Zombiecorn growing 10-20% yoy or not growing and you're paying the cloud $10M/y for 6yo CPUs? - pretty foolish to continue doing that.
This sounds like either an extremely bad experience or an account from many years ago. Things are better now.
Also like I said: physically racking only makes sense if you are huge. Managed bare metal is a whole market category and it’s spectacularly cheaper than big cloud for most work loads. For bandwidth it’s like 1000X cheaper. That is not an exaggeration.
...or "the UPS batteries caught on fire", or "the first unit failed and the contractor who was supposed to replace it replaced the second one instead, resulting in an outage".
Or something as stupid as, "vendor requires $50 failed DIMM replaced under warranty to be returned by UPS instead of chucked into the e-waste bucket, but UPS cannot pick up from DC because the driver can't be bothered to ring the bell on the DC gate".
That one alone probably resulted in our longest ever ticket.
I probably wouldn't start with buying a big server if I was just starting out. I would consider renting one from Hetzner, OVH, or many others.
There are also many other alternatives. Example: VPS providers like Vultr which have expanded to offer more traditional "cloud" features like object storage, load balancers, managed DBs, etc. Their pricing is way more competitive than AWS, especially when you consider bandwidth.
Personally, I'd rent a small server first. About 20 years ago I had a small web hosting operation with my own racked servers. It's a pain to deal with when something goes wrong. Eventually we transitioned to renting dedicated servers.
There’s a sort of arguable sweet spot where someone motivated and knowledgeable can manage their own hardware, even if it’s just rented from Hetzner etc. But that just doesn’t scale.
Once you do try to scale that - not just the hardware but the staffing that’s needed to avoid a bus factor of 1, to ensure the required uptime especially if it needs to be 24x7, to handle disaster recovery, failover, security and compliance issues etc. etc., you quickly run into all the reasons that infrastructure management services and other computing services - i.e., cloud - have won out so definitively.
If you’re small enough that none of those things matter very much to you, and knowledgeable enough to run your whole system yourself, and have the time and inclination to deal with it, go for it! But such people are outliers who are really indulging a hobby more than anything else. It generally makes extremely little business sense.
the capex vs opex arugment would be persuasive if you couldn't lease servers. and obviously colo was always opex. basically the only time you'd get into serious capex was when you were building your own datacenters
You’re forgetting blame: if an on-prem system has an issue, that’s 100% on you. If AWS/GCP/Azure has an outage, that’s just bad luck, and everyone else suffers too.
Frankly. Most of that stuff is corporate bullshit. Compliance is 99% fucking theatre, cover your ass audits with a fancy check list.
The capex vs opex theatre is just stupid economics and bad generalization from wall street types.
Almost nothing of it is really real.
And your analogy may impress other glorified salesman, but it doesn't hold water for a second, electricity is fungible, computing is not, electricity is stateless, you computing infrastructure carries your data. Power consumption generally is not a competitive differentiator,computing often is. And of course, the economics IN THE FUCKING REAL WORLD is broken: It is very hard to compete with the prices of the grid, not so in the modern cloud world where hyperscalers captured market enough to feel free to start extracting monopolistic rent from their consumers. And even the premise is uninformed, heavy industry frequently resorts to co-generation, and now, ironically, even data centers projects are exploring it.
we spend 2-2.5M a year I wouldn’t consider us their product scale yet.
My understanding is the full rack is about $1.2M and smallest half rack is about $600k and this was before the RAM and other price hikes now.
You are still going to have some residual cloud workloads(so all AWS won’t migrate ) and HA and DR regions etc plus the maintenance and incidentals (power , connectivity ).
I don’t expect anyone less than 5M spends and high base loads is a good fit for their offering.
That is the starting point though , minimum you need to buy one more for FO/DR if you care about uptime .
It is not once and done you have some continuous costs in either direction.
1. you will not be able to still move all your workloads only your base loads .
2. There will be residual workloads not only if they are spot/flexible bursts but also they are too deeply integrated to Paas offerings .
3. you have to pay for power cooling and network (and its backup) for 5 years .
4. Finally you also have pay for infra team to maintain, this is increasingly difficult (and expensive) to hire and retain .
Don’t get me wrong I would love to be able buy something like this, but the cost economy is pretty steep and 1-2M spends is too early (it is not on oxide, serious hardware costs a ton these days )
I can't think of an outcome that would be more odious to Steve Tuck and Brian Cantrill. Brian in particular still talks about the soul-crushing experience of Oracle's hostile takeover of Sun Microsystems.
Which many keep forgetting is that there was no one else wanting to acquire Sun, IBM did an offer that was shortly thereafter withdrawn, and that was it.
Sun would have died, everything completely lost among creditors and that would be it, end of story.
Ironically, not only are you entirely wrong on the facts themselves, but the historical record in this case is incredibly (and unusually) crisp: the Merger Proxy Statement that Sun had to file with the SEC[0] has, in accordance with Delaware law, the full narrative of the acquisition.
In the "Background of the Merger" section of that document, there is an incredibly detailed (and interesting!) story of three companies: Party A, Party B, and Oracle. (As was well-known at the time, Party A is IBM and Party B is HP.) As that narrative makes clear (and contrary to your assertion), it was Sun that rejected IBM's definitive agreement, not the other way around. You can certainly argue that IBM's acquisition of Sun would have failed to complete for other reasons (regulatory and so on) -- but your assertion that "IBM did an offer that was shortly thereafter withdrawn" is simply (and demonstrably) wrong.
Actual shout out to Oracle inexplicably moving Java forward a ton while also not strangling it with fees and suing their own customers. Wonder if there's a story of some genius, unsung internal salesman who made that happen.
Joyent was likely a part of what convinced Brian Cantrill that a new cloud machine was needed. They had their own stack running on commodity OEM hardware in their own cloud - likely a painful experience, since Brian talks a lot about how much of a difference it makes to own the complete root of trust and everything in it.
To answer these questions (or accusations?): Yes, I was at Joyent for the acquisition by Samsung -- but I also was not a founder, did not have a board seat, etc., so the involvement that I had, while substantial at some level (working with the Samsung team when they were doing their significant due diligence, for example) was also ultimately limited. My job was to make the acquisition work, not to determine the fate of the company for which I was ultimately an employee.
It's also absurd to call the sale a "sellout" -- the company was not for sale when Samsung came calling. The deal that Samsung proposed was a good and fair one, and if I HAD been on the board, I would have absolutely voted for the acquisition. (It should be said that Samsung themselves had a very high threshold to close the deal -- 97% of shares IIRC?)
And all of THAT said: while I was supportive of the acquisition by Samsung of Joyent (and worked hard to make that acquisition work), when we started Oxide, Steve and I had (and have!) zero interest in building a company to be acquired. Oxide is our life's work (and I mean that "our" broadly, as many at Oxide feel that same calling), and our objective with Oxide is to build an independent, generational company. Indeed, this Series D is all about advancing that objective!
So, I know I am late to this discussion by a week (the coding rarely stops, and I often miss big developments), but, I was at Joyent when it was acquired by Samsung, and their interest was very similar to most Joyent customers: to find a way to shrink their tumorous, ever-growing AWS bill[0]. By acquiring Joyent, they did not have to build their own cloud-software from scratch, and they did not have to Ship-of-Theseus their internal legacy infrastructure. More importantly, they also got a team that knew how to maintain and improve that software stack.
While I practically never agree with the guy, Thiel did say that "every startup is a conspiracy". Which is just another way of saying that every business is a conspiracy. The Oxide "conspiracy"[1] is to divert a chunk of the enormous economic surplus that is being captured by Amazon (and Broadcom, and others), and in the process, put some meaningful amount of that surplus in the pocket of the customer (otherwise, there is no reason to switch).
This is an incredibly ambitious goal, that cannot be achieved without upfront capital. Part of it is certainly hardware (which is getting more expensive, rapidly, and so it might make sense to buy years-worth of it), but another part is the _switching cost_ (which the _challenger/conspirator_ usually has to cover for the customer). That cost can easily be measured in the millions of dollars (Dropbox, famously, migrated off of AWS, via a client-side reupload, because moving the data from AWS to their own DCs was prohibitively expensive[2]).
Also, WW3 is slowly unfolding. The only reason energy prices are not in "brownout territory" is because (IIUC) the world's largest oil consumer is importing half as much oil from the mid-east as it used to. A few of the things that I buy, have _not_ gotten more expensive in euros, but they have gotten more expensive in dollars (by around 5% last I checked). If you need to use dollars to stockpile input-goods, now is the best time to do that, if you anticipate that the dollar will lose value over the next year.
A similar logic applies to selling a company. Amazon, in 2016, was already on the path to _massively_ improving the performance of its VMs and cloud services (via using more SSDs, building custom hardware, etc), and bare-metal performance was one of the Joyent selling points. With the resources of a company like Samsung, Joyent could also (potentially) use faster hardware, etc.
However, even under the aegis of Samsung, some (let's call them) _political asymmetries_ could not be avoided. I cannot talk about _internal_ asymmetries, but _external_ ones are already public knowledge. In particular, in 2017 or 2018, spectre and meltdown CPU-exploits hit the industry. All the major cloud providers had advance knowledge of this (and were able to mitigate via KPTI), except for Joyent (who had to work with the OpenBSD community for a few months to fix this). In those few months, if customers wanted to be completely safe, they would have had to move their instances to a different cloud. It is unclear (to me, because I am an engineer and not an accountant or account manager) if Joyent could have survived that without being part of Samsung.
And by the way, this would not have been as urgent of a problem, if Joyent was selling physical machines (like Oxide is, right now), instead of renting them out to multiple tenants. Imagine if an adversary could just spin up a VM right next to yours on the same exact machine. Even without spectre and meltdown, they could probably impact the performance and latency of your VMs indirectly, if they were willing to spend enough money. I once did this by accident (because I, foolishly, overestimated Google) on GCP, via their lambda-equivalent, and found out when they told us that those workloads were moved to a different DC. So if this is a problem for _Google_, it's a problem for everyone.
For at least the last decade, HN has consistently (but, thankfully, not exclusively) been attacking Joyent (and now Oxide), for various perceived misbehaviors[3], while frequently letting much less ambitious projects off the hook. Engineering any meaningfully new or disruptive technology is a very challenging marathon, and doing so, in business circumstances (which can only be characterized as: circumstances where the other runners are armed and always out to get you, while sometimes, the universe itself decides to send a few lightning bolts and storms in your direction) is almost impossible, without either (1) a monopoly, like MSFT and GOOG and AMZN enjoy, or (2) massive amounts of investment-cash that can only come from a very smart and very keen sugar daddy[4].
[0]: I can't recall who said this, but someone at the time said, they were tired of buying Bezos a BMW every month (via their AWS bill). Sometimes, it wasn't even an issue with the size of the bill: Amazon competes with many, many companies out there.
[1]: Based on various public statements. So basically, the Joyent conspiracy, but this time on-prem (so maybe Joyent + Fishworks = Oxide), and with fewer faulty drives (IIRC, there was a batch of drives, worth a huge amount of money, that had bad firmware, which caused their throughput to drop sporadically -- the exact details escape me, but you can see why there is a distrust of firmware written by others (also worth noting, is that this HDD vendor did not even offer to replace the faulty drives, but instead offered a marginal discount on the next order)).
[2]: Not because of any real, physical cost, but because Amazon bills you for every byte that leaves their datacenter (but not for any byte that enters).
[3]: I think it started when Joyent did not honor the "lifetime storage" promise that it made to its customers from the 2000s.
[4]: If anyone knows any wealthy heiresses that are looking to get married (or for a concubine), in exchange for financing my ambition to build an invention that is simultaneously (1) the last invention humanity will ever need, and (2) the invention that humanity needs most urgently, please hit me up. I have a sense of humor and am hung AF.
Thank you for the response. I appreciate it. I ran SmartOS in production and enjoyed it thoroughly. I am sad to see it not doing as well as I had hoped. Thank for again for the long detail
I doubt that. Oxide was founded by a bunch of ex-Sun people who have already been burned by the Oracle acquisition. If you read through what they say, their company values, and how they act, it's pretty clear their intent is to grow a sustainable long-term business and they're not looking for an exit.
We are trying to build a sustainable long-term business! It's just that you need enormous amounts of money to get there when shipping this large a product.
The idea behind VC funding generally is that you need large infusions of capital to get to the point where the business becomes sustainable long-term. The first one is the most expensive, and so on. Hardware is capital-intensive compared to SaaS, and especially so in the current environment.
Read the blog post on their series C [0]. It's not long, but the most relevant excepts are:
> So if we didn’t need to raise, why seek the capital? Well, we weren’t seeking it, really. But our investors, seeing the business take off, were eager to support it. And we, in turn, were eager to have them: they were the ones, after all, who joined us in taking a real leap when it felt like there was a lot more risk on the table.
> ...
> Our intent in starting Oxide was not to be an acquisition target but rather build a generational company; this is our life’s work, not a means to an end. With our Series C, customers don’t have to merely take our word for it: we have the capital to assure our survival into the indefinite future.
Maybe you could read that and think its complete bullshit and they're lying their asses off. Considering the people behind Oxide and their history, that's vanishingly unlikely though.
The reasonable conclusion is that they would not have raised yet more money if it wasn't due to being offered very generous terms by investors who wouldn't threaten the long-term future of the business.
It says "they've entirely derisked capital" and now ~6 months later they raised twice as much. Lying is a strong word but that post clearly wasn't accurate at the time.
They've raised a lot of money and there will be pressure for an exit sooner rather than later.
You can be cashflow positive and still benefit from having a larger pool of cash to throw around, particularly in any situation involving hardware manufacturing.
If you tell your investors "our limiting factor is how fast we can spend to deliver on additional requirements for these new customers", then it can both be true that you're not going to miss payroll for 5 years no matter what happens tomorrow and more cash would be beneficial.
Look at how much the components cost 6 months ago and now. That may explain why some calculations shifted considerably. It may be an act of keeping up with the market and having some reserve.
Depends on the VC. Some VC's are happy to own great businesses, even long term. Most are definitely vultures after a quick turn around. Mostly it has to do with where the VC gets their funding. Most VC's get their funding from offering a fund with a 2-5 year time-frame. Some are 10 yr funds, and some are long-term funds or are funded by a family office or two, which can be happy with great businesses long term.
It makes sense for Broadcom to remove that avenue of escape, and it makes sense for Oxide's investors to charge a premium to Broadcom, and materialize their returns.
The customers will have to deal with it, of course. At least they bought physical systems instead of renting them, so they can use them until they're obsolete.
Is there anything they're doing that can't be replicated by hypervisor/management software on commodity x86 servers? Seems like their "secret sauce" is a software stack that "just works" more than any actual special feature of the hardware.
I feel like Broadcom with its VMWare acquisition could easily take these guys out if they wanted to (or for that matter, any OEM that has a line of servers + network & storage hardware). They don't, most likely because there isn't actually enough profit to be made there (Oxide having to raise money multiple times might be a hint).
Managing a single commodity device is pretty simple. Could even be a shell script with a few IPMI commands.
Managing 100s of devices from a mix of vendors is possible becomes itself a massive ball of crap to stick all the pieces together.
Then add firmware management because now there is a plethora of firmware updates to worry about.
And then dealing with networking discovery & orchestration for such setups is an added horror few even try to visualize.
Then abstract all these differences away so one can provision two different vendor’s servers connected to two different vendor’s managed switches in different racks to a new VLAN. And do it with SR-IOV while you’re at it…
Oh, I forgot shared storage… details, details…
Commodity computing is the modern day Tower of Babel.
Yes, it can be done much simpler if one treats all switches as unmanaged, all storage as local, and all networking as flat. That’s just not acceptable for a lot of use cases though…
Good question. Can’t eliminate the BIOS/UEFI on commodity systems. Can’t remove/cut down the BMC. Can’t plumb the hardware root of trust into the OS and VMs. Can’t do dynamic rack-level power capping (future roadmap). There’s a whole bunch of the software stack that only works because the hardware and software were designed together.
Oxide exists in part because commodity systems don’t work for building a cloud at scale (Joyent). Similarly, AWS, GCP, and Azure don’t use commodity systems, they use hardware that was designed to work together with their software, Nitro being a prime example.
> Is there anything they're doing that can't be replicated by hypervisor/management software on commodity x86 servers? Seems like their "secret sauce" is a software stack that "just works" more than any actual special feature of the hardware.
There's a reason Apple is consistently one of the top 5 most valuable companies. It isn't because their hardware does anything that can't be done on "commodity" hardware, its that they built a software stack that "just works" and part of that was tight coupling to the hardware.
Their machines are x86 but they've rearchitected basically everything else in the pursuit of a cohesive and integrated machine, they don't even have a BIOS. The secret sauce is having full control of the software stack, and as close to full control of the firmware as is possible on x86.
Yes, custom hardware is a significant part of Oxide. You have to build your own to do that stuff, and that’s why they did. I alluded to some of the things upthread.
I found out about this round from this thread, just like everybody else, but
> Oxide having to raise money multiple times might be a hint
That’s not the only reason to raise a round, by far, especially when you, you know, are building custom hardware. It isn’t a SaaS business.
> Is there anything they're doing that can't be replicated by hypervisor/management software on commodity x86 servers?
It was technically possible with (e.g.) OpenStack for years (decades?) before Oxide ever existing, and yet even with such a solution being around, some folks still went with Oxide. (Or, depending on the scale you want to talk about: Proxmox, XCP-ng.)
I wonder what's the selling point at that scale. If your ~monthly~ cloud spend ends in "M", you can easily justify hiring the talent needed to wrangle conventional bare-metal (in fact you can do so at much lower spends, but at these spends it becomes a rounding error).
Edit: my bad, read that as monthly instead of yearly. Still, a yearly spend of millions would still make sense to bring that in-house.
An in-house team is most likely competing with something like Dell or VMware, not really Oxide. A significant part of Oxide’s value proposition is that you’re buying hardware and software purpose built for each other. Unless you’re also going to go so far as to do all of that, which companies like Google do, of course, it’s not really the same thing.
This matters when your various vendors start pointing at each other when something goes wrong. Oxide is truly “one throat to choke” in a way others just aren’t, and stand by that quality.
(Not to mention other various efficiencies, like power, or removing things like the BIOS and BMC junk that’s in basically every other server you buy right now. And the ability to send attenuation from boot up through the host OS. Just tons of things they’re differentiated on that your in house team just isn’t going to do.)
VC or private equity fueled companies are weird. At my place we are spending 8 figures a year just in AWS, and it's not like they planning to move to bare metal but, they're in fact removing stuff from their old datacenter. And that bill doesn't include some of the other SaaS like Mongo or Elastic.
With bills of that magnitude, each time I do a little house cleaning and delete some old data, change storage classes, or discover some unused servers... the savings (that are barely a rounding error on their bill) could pay for a whole year of an engineer or a bunch of servers that could power a good chunk of their production traffic.
Its not about bare metal, its about owning the whole stack and giving visibility into every nook and corner, and then supporting that open sourced code. Every network switch, every interconnect, every piece or firmware or software they can attest to because they built it themselves, and that is worth its weight in gold to high criticality industries and governments. They can give you the provenance for every single hardware and software component. Every chip.
If you can roll around those possibilities in your head, you start to see how absolutely unique this is and why these extremely sensitive and important customers and industries want this.
