Currently, we peaked with over 7000 people in-game with another over 2000 in the queue.
Realm is online.
As you can see by the increased uptime, we have managed to resolve the main issue behind the crashes and stability should be much better from now on. As the Administrators, the responsibility of fixing the problem was on me and Emtec. It was an incredibly stressful period of time for us, as for 7 days we worked for over 16 hours a day on solving the issue. We barely slept, we barely ate and we pretty much didn't leave our PCs apart from getting some refreshments or at least some glimpses of rest, but in the end, we emerged victorious. This would not be possible without many people and we would like to thank all of them for their help, starting with the Netherwing Team first. Everyone came together and continued to perform their duties even if they weren't always fully informed about what's going on. We would also like to thank other developers who offered their help in this time of need, mainly the Tauri team (Higi, Lightguard, Chris) and Namreeb. They didn't have to, yet they spent days of their time helping us solve the problems. They are incredible people and words cannot express how thankful we are for their support. Collaborations in the private server scene happen very rarely, so we're really happy that it happened on Netherwing, especially in such a troubling time. We're certain that our paths will cross again in the future, hopefully in much better circumstances.
Additional thanks also go to Whitekidney and Bladez from Light's Hope, for a few tips about potential cause of this and other issues. Obviously, the server would not exist without you - the players. Despite the issues and crashes every half an hour, you kept logging in, you kept playing, you kept testing, which was incredibly important in helping us fix the problem. We received many messages of support and we knew that you kept believing in us. We hope that with those issues resolved you will finally be able to enjoy years of our work put into this project and help us make this server truly special. Thank you from the bottom of our hearts.
Last night, with the help of Tauri devs, we managed to make big progress stability wise and make core fixes and changes. We are now at an 8h uptime mark and it keeps rising. Due to these large core changes we have introduced some new bugs that haven't been seen before and Emtec is fixing some of them as we speak while Wolfenstein will take care of others. We are still monitoring the server performance for any new issue and crashes. We are not finished yet! Thank you!
At present we're working with the Tauri team in solving the crashes that have been bothering us and you. Until now, we've pushed fixes in our core that will help us determine any further potential causes. We, in all honesty, apologise for the current state of stability, we can't express that enough, this is not the intention we had in mind, nor wished for. We're not updating you on our every move since we don't wish to provide you only with all sorts of synonyms of "we're working on it". We care about the result as do our community. We hope that with this joint effort we'll deal with the current state of matters way faster, about which we'll surely inform. To address some concerns, unfortunately, this issue can't be dealt with a closed realm. In order to fix the issue, the whole server needs to be under load. We do care about your comfort, despite how the current stability does not reflect that and will continue to work nights and days to secure the stability in which everyone can enjoy the content we've created.
We would like to brief you on the format of the current situation and explain what we're up to. This instance we're battling with a single type of crash that keeps on appearing. While running the server with an enabled sanitizer we have gathered some data, which unfortunately wasn't enough, as the server was working far too slow, and thus, wasn't able to trigger the cause of our crashes. Currently, we've compiled the server in debug mode which will provide us with additional data, necessary to resolve the current stability issue. We're certain that they're not caused by any unusual or uncommon player's behaviour or exploits. This is not an easy type of crash to fix, nonetheless, we're positive that we'll surely deal with, by localizing exactly where in our code this situation occurs with the help of previously mentioned, gathered data. We're receiving warm words and waves of encouragement from our playerbase for which we're naturally thankful, never have we witnessed such warm feedback despite the issues. No issues of the sort will dampen our spirits and we'll continue to work day and night to finally find the ultimate solution. As for acknowledgements, it's due to appreciate and give credit to a few of our colleagues to whom we owe many thanks for their willingness and determination to help us.
We've noted a few good runs, with over 3 hours of uptime. However, the realm still suffers from crashes, developers are constantly working day and night on finding the ultimate solution.
The developers today have narrowed the crashes to be cooldown bound, however, they are not caused by every spell or item out there, but by a specific mechanic, they're experimenting since this morning with various solutions. Thankfully we've managed to somehow deal with crashes that occurred every few minutes, which were most annoying and frustrating for you and continue onwards with the rest of the issues that occur every half an hour or more.
To address some concerns - the population is not the main issue, as in server load. We've noted relatively moderate diff at our peak, which is the result of work done before we launched. It's an uncomfortable mess to experience the crashes as they ruin the overall joy of enjoying the content and frustrate each time they occur. All the hard work that went into scripting content in-game is now deranged by these issues. It's a priority and the developers continue to work on it, we have noted a few good uptime runs, however, it is not enough. We've prepared well in terms of optimization, however, we did not prepare enough in terms of potential crashes. We're constantly working, finding new solutions and trying them out, most crashes are located immediately, being worked on.
Join our Discord (below) to be up to date!
The main source of communication isour Discord, so make sure to join it and be up to date with any issues and information!