I probably wouldn't start with buying a big server if I was just starting out. I would consider renting one from Hetzner, OVH, or many others.
There are also many other alternatives. Example: VPS providers like Vultr which have expanded to offer more traditional "cloud" features like object storage, load balancers, managed DBs, etc. Their pricing is way more competitive than AWS, especially when you consider bandwidth.
Personally, I'd rent a small server first. About 20 years ago I had a small web hosting operation with my own racked servers. It's a pain to deal with when something goes wrong. Eventually we transitioned to renting dedicated servers.
There are also many other alternatives. Example: VPS providers like Vultr which have expanded to offer more traditional "cloud" features like object storage, load balancers, managed DBs, etc. Their pricing is way more competitive than AWS, especially when you consider bandwidth.