You buy the hardware (48 servers), rent part of a colocation rack with a 10 Gbps or 100 Gbps internet transit link, get a payment processor, make a webpage and GitHub demo with the API. Break down: $3000 labour, $20.5K hardware, $800 monthly rental fees, $376 car fees. When you shut down within a year, the $20.5K popular off the shelf hardware can easily be sold for $17K, a fact you can check from 25 years of data.
I would invest more than the initial $30K on optimization after the servers have found paying customers and thus have proven commercial viability. I would invest in software development, finetuning, retraining and above all reverse engineering GPU and neural engine instruction sets and adapting these open source models to the more than 2 quadrillion operations per second that these 48 servers can do.
48 servers at 60K also does not math to me at all. Even if I consider the second-hand ones, I could hardly find zen4 under ~10K. And this is without GPUs.
The numbers were really too good to be true but there's always a chance that there may be something I am not seeing correctly so I wanted to give the benefit of the doubt. Otherwise, the idea is not too crazy regardless of much higher input costs.
I broke down the first $30K investment cost for release of the online API product, that does not need further software development, sales or support.
You would be wise to do the software development I mentioned, do more sales and support than was covered under my initial $3000 labour fee. But that you can pay for with the revenues, it would not be the initial investment to see if it is viable as a business.
I would invest more than the initial $30K on optimization after the servers have found paying customers and thus have proven commercial viability. I would invest in software development, finetuning, retraining and above all reverse engineering GPU and neural engine instruction sets and adapting these open source models to the more than 2 quadrillion operations per second that these 48 servers can do.