how is the state of moving to a new hosting facility?
Right now, the set up system is being dockerized, and I think after that is ready we’ll move ASAP. So, depends on how long that takes (some of the smaller projects are ready, MusicBrainz is being worked on at the moment).
Thank you.
I hope there will be no delay.
This is not something that can be done in less than no time. When the dockerization is (believed to be) ready, the whole system will need testing; the idea currently is to set up a temporary test system on a few “cloud” servers. Then when everything is working in principle, MeB will need to order the various servers, rack space, etc., and setting up the network at the new data center will take some time, too. Finally, the actual move will probably need quite a bit of planning and some downtime, too.
6 Likes
If you create Docker files, what’s the point of ordering servers, rack space, etc.? Isn’t it time to move to the real cloud, e.g. Google Container Engine, and forever forget about bandwidth issues, load balancers, server failures, etc.? I am pretty sure that folks at Google will be happy to help.
It’s a money issue. “The cloud” is (for a permanent service) considerably more expensive than using actual hardware.
4 Likes
It’s a money issue. “The cloud” is (for a permanent service) considerably more expensive than using actual hardware.
I respectfully disagree based on 17 years of experience of running own hardware and 10 years of experience with the cloud. In my latest software venture I would have saved $1m if I used the cloud instead of stand-alone (co-location) setup with web/app servers and PostgreSQL.
Look at all the team notes for the past two years. How many hundreds (thousands?) of developer-hours have been spent fixing, patching, troubleshooting your own hardware? How many hundreds of hours have been spent planning the move to the new hosting company? How many more will be spent buying hardware, configuring it, testing it, moving files, etc.? Even at a reasonably low total cost of employee hour the total expenditure on system administration and maintenance will be very significant.
There are many reasons why so many companies are moving away from own hardware, and saving money is the key among them.
NB: Google automatically applies sustainable-use discounts, so if you run instances continuously, they are much cheaper. And their cluster management costs only $3.60 per day for up to 1,000 nodes (way more than you need). Take a look at this pricing calculator:
Let me clarify our intentions: We’re not suggesting to host our own hardware anymore, but instead renting servers from a hosting company. We’ve done the following planning to compare providers:
I can try and shove our estimated needs into the google calculator and see what comes up. If you feel strongly about this, please have a try. 
4 Likes
-
Your spreadsheet does not include a key expense - your team’s time. If your team spends an extra 100 hours per year in one option vs the other, how much does it cost? The same 100 hours can be spent on new features or improving the existing products. It may appear that it will be all smooth sailing once you move to the new hardware, but any self-managed setup will always require significant extra effort.
-
A key advantage of platforms like Google Compute/Container engines or AWS is automatic up/down scaling. You can have more resources during the peak demand offset by fewer instances at other times. This means that the same average capacity will result in a much better total throughput, i.e. a better user experience, at the same expenditure level. Plus you can provision even cheaper capacity for tasks that can be done opportunistically, e.g. rebuilding the search index.
-
Another advantage is budgeting. You can set the budget to $50 per day. Then move it to $60 per day or the other way at any time with just a couple of clicks. No commitments - you have both flexibility and total control over your spending.
-
Finally, Google drops costs at least once a year, often by 20-40%. And since there is no contract or term commitment, you benefit from lower prices right away. For that reason I would probably stay away from options that charge for initial setup or require term commitments.
Since this is an important one-time event, consider Google Container Platform as an option. I have no doubt, however, that whatever you end up doing will be a big step forward for the organization
Cheers and thank you for all your hard work!
3 Likes
Point 1: The reason why I didn’t include labor costs are because we’re moving to a docker setup, and the costs of moving to Hetzner/Google are roughly the same.
Point 2-4: These are nice, but the price-tag simply does not justify the exorbitant cost difference.
The plan to move Hetzner where we get less flexibility, but a lot of ram, CPU and bandwidth is about 1,500EUR. The rough equivalent at Google: about $6,000!
True cloud providers are utterly out of the question.
2 Likes
GCP, Azure and AWS are all growing at 60-100% yoy. Spotify is moving to Google Cloud as we speak, and Spotify does have high sustainable volumes (~12,000 servers)
I guess we just look at the numbers differently. I run my apps on App Engine, which is even more expensive than Compute Engine, but it saves me so much time that overall it comes out cheaper than even a bunch of free servers at my house.
Anyway, good luck with your move!