Author Topic: Forums and IRC - September 24, 2010 (Read 891 times)

Dorganath · « **on:** September 24, 2010, 09:29:26 am »

Dear Layonara Community,

We are aware that there continue to be issues with the forums, and efforts continue to bring them back to the proper level of functionality, including the visibility of posts made prior to last night (US/Canada time zones). It is pretty much a forum-wide problem that affects everyone equally, including those of us with higher-level access.

A side effect of the database problem affecting the forums is that the NickServ, ChanServ and aragen services on IRC are also down.

All I know about is that the forum database experienced a couple of crashes over the last couple of days, requiring a repair process. Last night, orth indicated to me that the database in question was quite large and he did not know how long it would take to run the repair completely.

At this time, I can offer no information as to when these services will be back in full operation or whether any data has been lost and/or corrupted due to the crash. I do not yet know the root cause of these problems either.

We are aware that this causes an inconvenience to people and a difficulty in conducting various RP and social functions for Layonara. We're also aware that this has likely suspended things like Character Applications, as the CA team cannot access the threads to comment upon or approve them.

So please bear with us as we work to sort this out. As more information becomes available, someone will update everyone. Until that happens, please assume that things are not working properly and assume that we are working toward a fix.

orth · « **Reply #1 on:** September 24, 2010, 03:37:56 pm »

A hard crash of the forum server caused the database to corrupt badly. Upon attempting to repair the database the mysql repair functionality was overzealous in making its corrections and we ended up losing 215 thousand or so posts! Yikes!

I've restored from backups but they are older than I wished they were. There may have been posts lost since the last backup and I'm not sure if I can recover them, but I'm attempting a few other things.

I feel awful for this, as a systems administrator I know the community trusts that these issues do not arise and their data is safe. I had recently moved our database for our forums from a separate server to the same server that the web site runs under and upon doing this our daily backups of our database were not functioning properly.

I'm sorry for any loss of creativity and organization, I'm still doing my best to see if I can track down more of the posts that may have been lost between the end of July and now.

-James

Filatus · « **Reply #2 on:** September 24, 2010, 03:49:46 pm »

Good thing I've been lax in updating my CDT.

gilshem ironstone · « **Reply #3 on:** September 24, 2010, 04:31:15 pm »

It's good to do some house-cleaning every now and then.

miltonyorkcastle · « **Reply #4 on:** September 24, 2010, 04:54:28 pm »

Thank you, Orth (and whoever else helps with this), for taking the time to maintain these forums for us. I definitely take these forums always being here for granted, because when they're not, I have one of those "lost" feelings. Sad, I know, but true, heh.

orth · « **Reply #5 on:** September 24, 2010, 05:03:00 pm »

Okay some good news.

I have the last 30 days of posts in a cached table. They aren't easy that easy to recover though, in fact I'm not even sure how best to handle it, but all the text is there.

If you have a CDT/Character Submission or such that you've posted in the last 30 days that you really want back, let me know and I'll be able to PM/email you the content for you to repost.

Tanman · « **Reply #6 on:** September 24, 2010, 05:04:52 pm »

Thats cool Orth and Dorg. These things happen and the main thing is that you keep us informed which you have been doing. Keep up the good work.

Frances · « **Reply #7 on:** September 24, 2010, 07:11:39 pm »

Something I thought of...

If you're trying to recover a lost thread, you could try googling it and using the cached copy. It might not be as up to date as you like, but you might get something usable.

Alazira · « **Reply #8 on:** September 24, 2010, 07:38:35 pm »

The Angel's Guild keeps our records of peoples sales and credits on the forums. I'm not sure we have anything in writing more recent than whenever Ferrit made her last update (assuming she saved it on her computer.. and I think that was a week or two back anyhow). We will REALLY have issues if that can't be found. Any help? They would have all been posted under "Guild Orders" In the Angels' Guild Forums.

Thanks!

Katie/Zira

orth · « **Reply #9 on:** September 24, 2010, 07:51:02 pm »

Quote from: Alazira

The Angel's Guild keeps our records of peoples sales and credits on the forums. I'm not sure we have anything in writing more recent than whenever Ferrit made her last update (assuming she saved it on her computer.. and I think that was a week or two back anyhow). We will REALLY have issues if that can't be found. Any help? They would have all been posted under "Guild Orders" In the Angels' Guild Forums.

Thanks!

Katie/Zira

I've PMed Serissa with details on obtaining that information.

orth · « **Reply #10 on:** September 24, 2010, 07:53:38 pm »

Quote from: Frances

Something I thought of...

If you're trying to recover a lost thread, you could try googling it and using the cached copy. It might not be as up to date as you like, but you might get something usable.

Unfortunately a lot of our boards are members only so google doesn't spider them, but it's still worth a shot if the thread was in one of the public forums.

darkstorme · « **Reply #11 on:** September 24, 2010, 09:08:50 pm »

There's also the character submissions - they're all gone.

Dorganath · « **Reply #12 on:** September 25, 2010, 12:46:23 am »

OK, everyone...Apparently we've had lingering problems with the forum database.

We think we have everything cleared up this time, but of course it's possible we'll still discover some other debris from the server's hard crash.

Again, we apologize for the inconvenience and difficulty. Hopefully this will put the crashy problems to rest and we can start to focus on recovering as many of the important missing posts as possible. We're working on an analysis of this cached data so we can formulate a plan to make the most of what we have.

Chazzler · « **Reply #13 on:** September 25, 2010, 09:09:42 am »

Cailomel's Goods & Wares could also need some recovering, meaning the stickied threads there.

Makashi · « **Reply #14 on:** September 25, 2010, 10:01:37 am »

Theres still quote a few threads I can't seem to open either, a few on the trade market, general discussion, and suggestions sections

Dorganath · « **Reply #15 on:** September 25, 2010, 11:26:33 am »

We know all about the missing posts, and they will span the entire forum. What we're wanting though is that information which is critical to you. Character applications, development threads and guild accounting are going to be the top priority, along with GM and plot information. Unfortunately, the cache we have is mostly just content, with no real identification about who posted it or what thread it goes in (just a post ID). Still, it's something, but it's also a huge amount of information.

So if there is anything you as a community can do to assist the process, we would be pretty grateful. For example, if you've written or backed up your CDT on your local PC, then please, re-post them. If your Trade and Market Hall thread is empty, just re-post. If you want to re-suggest a suggestion, please do so.

For guilds, getting a list of both character names and forum names would help greatly, since the contents of those posts sound very similar between the crafting guilds in a lot of cases, and they may be tough to separate.

For character apps, giving us the name of the submitted character to match up with your forum name would also be beneficial.

We're still in the early stages of trying to figure this all out, so the more help you can give, the better. Bear with us.

lonnarin · « **Reply #16 on:** September 25, 2010, 11:37:01 am »

The important thing is that we still maintained the thanks-counts. Otherwise Pseudonym would be an emotional wreck right now!

As for the rest, we shall rebuild! *dashes off to work on his posting statistics*

Xaltotun · « **Reply #17 on:** September 25, 2010, 12:21:19 pm »

Thanks to Ed for sending me this link as I still can't see it normally...made good reading and thanks to everyone working on it.

It's shame recent stuff got lost, but I guess you guys will honor my approved request for a +10 Short Sword with Immunity to losing Soul Strands?

Only kidding.

RollinsCat · « **Reply #18 on:** September 25, 2010, 03:16:11 pm »

Tricks for finding cached copies of pages:

Type into your google bar the exact name of the thread you are looking for with - Layonara after it ; Ex: Andrew Reid - Letters Home - Layonara.

If you see your thread there should be a small blue "cached" option to the right (for google). hit this, and it will show you the page with all the posts on it including (hopefully) the currently misplaced ones.

If you don't see your thread or see the wrong page, look for "more results from (xx) - ". In the case of the dev threads, it would say "More results from forums.layonara.com »" at the bottom of the other possible matches in blue - click this, and it will bring up all the pages it has cached. you can only view one page at a time, clicking on forward/back once in the cache will take you to the live page with the missing posts.

copy/paste away.

orth · « **Reply #19 on:** September 25, 2010, 05:39:56 pm »

For those of you PMing me asking for the restoration of certain things please help by providing me with ways to determine that a post is yours. For example, do you always sign off a certain way, or do you address your CDT to a certain person, maybe you know that you talked about a specific town or deity. Guilds maybe you follow the same structure for your ledger and we can pluck out posts that way.