Server outage in July

Article about the 5 day downtime in July.

A short article about the biggest downtime I have ever experienced.

history

On July 26th, 2025 in the morning hours I have received an email from my server provider that they'll be doing the so-long promised migration from Luxembourg from Switzerland. I was told that around 24 hours of downtime is expected which was fine by me. Not great, but I can live with it.

start of the problems

On July 27th, in the late afternoon hours we got the info that the team that they hired have just even started going from LUX to CH(Luxembourg will be LUX, Switzerland will be CH from now on). No matter how you look at it, it is more than a day and yet there's no progress whatsoever.

continuation of the problems

On Monday the nodes were in the datacenter already, but it turned out later that only partially. Nodes containing the KVM's were dropped of in another DC in the Netherlands, where Cloudzy.ai, the new parent company of BuyVM colocates servers. The problem was that they were not meant to be there at all. At least the nodes containing the storage servers were in a good place.

tuesday, July 29

On Tuesday it unfolded that no one would be able to drive the servers to Switzerland, so they'll do a live migration instead which is obviously going to take longer than just racking the servers, plugging them in and doing little configuration so the new network would work with them. But the servers needed to be racked and the storage nodes needed cabling fix. Wednesday has passed with actual progress so the next day we would get our servers running.

final day of hell

Thursday morning the migration has finally started, perjel.hu came live around 11.00 AM and all services were operational by 4.00 PM that day since the storage nodes were fixed too.

positive part

We got new CPU's with the migration, retiring a Ryzen 9 5900X to a Ryzen 9 7950X3D which is a significant upgrade, including the DDR5 platform. I also personally upgraded to a bigger plan, which resulted in an even more significant performance increase, thanks to a twice as much RAM and CPU power. Now more people can access the perjel.hu services and sites simultaneously without any lag or slowdown to collaborate and work in a privacy-oriented, self-hosted environment.

conclusion

Although I never expected a downtime this large, I'm glad it is in the past. Also a huge thanks to Francisco, the previous owner and currently very valuable member of Frantech/BuyVM that he kept us updated as soon as he could in BuyVM's Discord server.