The World of Layonara

The NWN Persistent World => Layonara Server => Topic started by: Dorganath on August 12, 2016, 10:57:22 pm

Title: Important Server issue -- PLEASE READ!
Post by: Dorganath on August 12, 2016, 10:57:22 pm
Hello All,We've had an issue where the server got abruptly filled up by rogue events filling up event logs. We went from 20GB free to zero in under two (2) days due to logs filling up. This resulted in the server start-up problems we experienced tonight but it also caused some other unfortunate side-effects.To be specific, there's five characters which have fallen victim to this and may be unrecoverable by the means we have at our disposal.Pegasus55 - Michael
knightsfollytoo - Charlie Poetr
Erin Inman - Rebecca Poetr
Serissa1 - Katelyn Poetr
Borbre Barradok - Flint FireforgeWe recommend strongly that these characters do not try to log in until directed to do so by a GM who is assisting in the recovery/rebuild effort.There may be nothing we can do to directly restore these characters to their most recent state, as our backups have been overwritten by invalid files. However, we do have a record of their XP, completed static quests, etc., and we will work with the affected players to rebuild these characters to the closest approximation of their previous state as we can.As with the previous, similar incident, we encourage those affected to open a grievance request to start the process to get things set back to how they should be.We regret the inconvenience this may cause, and we'll work with you as much as we can. 
Title: I have not tried to log in
Post by: Alatriel on August 13, 2016, 08:47:36 am

I have not tried to log in since the crash.  If there is someone who can assist at some point today to check my character as listed in the previous post, please let me know when.  Thanks.

 

Title: Alatriel wrote:I have not
Post by: Dorganath on August 13, 2016, 11:15:51 am

Quote from: "Alatriel"&cid="2760749"

I have not tried to log in since the crash.  If there is someone who can assist at some point today to check my character as listed in the previous post, please let me know when.  Thanks.

 

 

I can pretty much guarantee your character (Rebecca) is gone. Your other characters should be fine and you're welcome to log in and check them out. For reasons of server stability, I cleared out the character files known to be corrupted last night. You'll have to take Rebecca through the grievance process to get her rebuilt and restored to some semblance of her original form, which you have already started.

Title: Alright everyone!First of
Post by: Rowana on August 13, 2016, 12:59:42 pm

Alright everyone!

First of all, thanks to everyone for your patience and understanding, both last night and over the next few days. I'm posting here for the visibilty and then I will be moving over later today to address grievances individually. Unfortunately, this issue overlapped some prior commitments by both DBGMs and they cannot see to anything regarding this issue (beyond what they already have) until probably Sunday, Aug 14th (U.S. Central for global reference). Please do not take any actions with your characters until you hear from Dorganath, Orth or a GM operating under their instruction. Just making the server stable and non-crashing for logins took at least 2 hours last night. Some of you do know how this goes, others don't so I'm trying to be very clear so everyone has an understanding of what's going on and why things will be taking a while. Last time we had something like this go down (Dec '15) there were unilateral and uninformed actions taken that complicated the process. Please don't do that. It will delay your recovery and return to normalcy.

Step 1 - for everyone seems to be have completed. We need a grievance thread for the afflicted characters. If that is done, there's nothing else you (the player) can do for the next couple of days. Don't worry about contacting them or asking about other things. All of the information will be funneled to you through those grievance threads as soon as it is available.

Step 2 - will entale the DBGMs going over your .bic and other files to see what's corrupted, what can be restored (if anything) and what unique and individual actions can be taken. Between Orth and Dorg, they've already put in a few collective hours of work on this, so there's some ideas but the picture is not yet complete. Please be patient until this step can be completed.

Step 3 - will be feedback from the team about where each character stands and what steps you will need to complete following that. I would look for that to happen over the next week but please understand they are balancing these tasks with a very busy time period (outside of Layo) and they can only do so much.

Step 4 - will be different for each character based on the feedback from Step three. (And now the steps are getting a little silly so I'll leave it at that for now.)

Again, thank you SO MUCH for your patience and understanding. We know this is frustrating. Both Orth and Dorg are frustrated as well, struggling with preventing future episodes and trying to understand what is going on. They really don't want this kind of stuff happening to you guys and are doing their very best to fix the cause. Especially, we're sorry for the folks who got hit right before their player event and may now have to rethink playtime plans since the original characters were afflicted and unplayable for the moment.

Later today I will be cleaing out the grievance forum and getting your threads all ready for feedback and then we should be all set until Dorg and/or Orth have time to complete the research needed.

Thank you, apoligies and all of that can not be said enough.

~row

Title: Yeah, I tried to log in with
Post by: cbnicholson on August 13, 2016, 09:21:05 pm

Yeah, I tried to log in with a different toon just now and NONE of my characters were listed at all.  I exited and tried again, can't connect. 8:15 p.m. cdt.

 

Title: This may mean that Charlie is
Post by: Dorganath on August 13, 2016, 09:25:00 pm

This may mean that Charlie is corrupted. I did attempt a restore from the backup, but the backup could also have been corrupted.

Removing him from the vault and bringing the server back up. Give me 10...

Title: Looks like everyone but
Post by: cbnicholson on August 13, 2016, 09:52:44 pm

Looks like everyone but Charlie is back. *phews*

Logged in and out with each, all okay, but Charlie.

 

Title: So is the server okay to be
Post by: Teo on August 13, 2016, 11:05:30 pm

So is the server okay to be logged into now or does it need some more time?

Title: For now everything is safe,
Post by: Rowana on August 13, 2016, 11:23:04 pm

For now everything is safe, yep.

~row