The World of Layonara  Forums

Author Topic: Important Server issue -- PLEASE READ!  (Read 1906 times)

Dorganath

Important Server issue -- PLEASE READ!
« on: August 12, 2016, 10:57:22 pm »
Hello All,We've had an issue where the server got abruptly filled up by rogue events filling up event logs. We went from 20GB free to zero in under two (2) days due to logs filling up. This resulted in the server start-up problems we experienced tonight but it also caused some other unfortunate side-effects.To be specific, there's five characters which have fallen victim to this and may be unrecoverable by the means we have at our disposal.Pegasus55 - Michael
knightsfollytoo - Charlie Poetr
Erin Inman - Rebecca Poetr
Serissa1 - Katelyn Poetr
Borbre Barradok - Flint FireforgeWe recommend strongly that these characters do not try to log in until directed to do so by a GM who is assisting in the recovery/rebuild effort.There may be nothing we can do to directly restore these characters to their most recent state, as our backups have been overwritten by invalid files. However, we do have a record of their XP, completed static quests, etc., and we will work with the affected players to rebuild these characters to the closest approximation of their previous state as we can.As with the previous, similar incident, we encourage those affected to open a grievance request to start the process to get things set back to how they should be.We regret the inconvenience this may cause, and we'll work with you as much as we can. 
 
The following users thanked this post: lonnarin, Acacea, Alatriel, davidhoff, willhoff

Alatriel

I have not tried to log in
« Reply #1 on: August 13, 2016, 08:47:36 am »

I have not tried to log in since the crash.  If there is someone who can assist at some point today to check my character as listed in the previous post, please let me know when.  Thanks.

 

 

Dorganath

Alatriel wrote:I have not
« Reply #2 on: August 13, 2016, 11:15:51 am »

Quote from: "Alatriel"&cid="2760749"

I have not tried to log in since the crash.  If there is someone who can assist at some point today to check my character as listed in the previous post, please let me know when.  Thanks.

 

 

I can pretty much guarantee your character (Rebecca) is gone. Your other characters should be fine and you're welcome to log in and check them out. For reasons of server stability, I cleared out the character files known to be corrupted last night. You'll have to take Rebecca through the grievance process to get her rebuilt and restored to some semblance of her original form, which you have already started.

 

Rowana

Alright everyone!First of
« Reply #3 on: August 13, 2016, 12:59:42 pm »

Alright everyone!

First of all, thanks to everyone for your patience and understanding, both last night and over the next few days. I'm posting here for the visibilty and then I will be moving over later today to address grievances individually. Unfortunately, this issue overlapped some prior commitments by both DBGMs and they cannot see to anything regarding this issue (beyond what they already have) until probably Sunday, Aug 14th (U.S. Central for global reference). Please do not take any actions with your characters until you hear from Dorganath, Orth or a GM operating under their instruction. Just making the server stable and non-crashing for logins took at least 2 hours last night. Some of you do know how this goes, others don't so I'm trying to be very clear so everyone has an understanding of what's going on and why things will be taking a while. Last time we had something like this go down (Dec '15) there were unilateral and uninformed actions taken that complicated the process. Please don't do that. It will delay your recovery and return to normalcy.

Step 1 - for everyone seems to be have completed. We need a grievance thread for the afflicted characters. If that is done, there's nothing else you (the player) can do for the next couple of days. Don't worry about contacting them or asking about other things. All of the information will be funneled to you through those grievance threads as soon as it is available.

Step 2 - will entale the DBGMs going over your .bic and other files to see what's corrupted, what can be restored (if anything) and what unique and individual actions can be taken. Between Orth and Dorg, they've already put in a few collective hours of work on this, so there's some ideas but the picture is not yet complete. Please be patient until this step can be completed.

Step 3 - will be feedback from the team about where each character stands and what steps you will need to complete following that. I would look for that to happen over the next week but please understand they are balancing these tasks with a very busy time period (outside of Layo) and they can only do so much.

Step 4 - will be different for each character based on the feedback from Step three. (And now the steps are getting a little silly so I'll leave it at that for now.)

Again, thank you SO MUCH for your patience and understanding. We know this is frustrating. Both Orth and Dorg are frustrated as well, struggling with preventing future episodes and trying to understand what is going on. They really don't want this kind of stuff happening to you guys and are doing their very best to fix the cause. Especially, we're sorry for the folks who got hit right before their player event and may now have to rethink playtime plans since the original characters were afflicted and unplayable for the moment.

Later today I will be cleaing out the grievance forum and getting your threads all ready for feedback and then we should be all set until Dorg and/or Orth have time to complete the research needed.

Thank you, apoligies and all of that can not be said enough.

~row

 

cbnicholson

Yeah, I tried to log in with
« Reply #4 on: August 13, 2016, 09:21:05 pm »

Yeah, I tried to log in with a different toon just now and NONE of my characters were listed at all.  I exited and tried again, can't connect. 8:15 p.m. cdt.

 

"Give a man a mask and he will show you his true face." 

Oscar Wilde
 

Dorganath

This may mean that Charlie is
« Reply #5 on: August 13, 2016, 09:25:00 pm »

This may mean that Charlie is corrupted. I did attempt a restore from the backup, but the backup could also have been corrupted.

Removing him from the vault and bringing the server back up. Give me 10...

 

cbnicholson

Looks like everyone but
« Reply #6 on: August 13, 2016, 09:52:44 pm »

Looks like everyone but Charlie is back. *phews*

Logged in and out with each, all okay, but Charlie.

 

"Give a man a mask and he will show you his true face." 

Oscar Wilde
 

Teo

So is the server okay to be
« Reply #7 on: August 13, 2016, 11:05:30 pm »

So is the server okay to be logged into now or does it need some more time?

 

Rowana

For now everything is safe,
« Reply #8 on: August 13, 2016, 11:23:04 pm »

For now everything is safe, yep.

~row