The World of Layonara

The NWN Persistent World => Layonara Server => Topic started by: Dorganath on September 24, 2010, 09:29:26 AM

Title: Forums and IRC - September 24, 2010
Post by: Dorganath on September 24, 2010, 09:29:26 AM

Dear Layonara Community,

We are aware that there continue to be issues with the forums, and efforts continue to bring them back to the proper level of functionality, including the visibility of posts made prior to last night (US/Canada time zones). It is pretty much a forum-wide problem that affects everyone equally, including those of us with higher-level access.

A side effect of the database problem affecting the forums is that the NickServ, ChanServ and aragen services on IRC are also down.

All I know about is that the forum database experienced a couple of crashes over the last couple of days, requiring a repair process.  Last night, orth indicated to me that the database in question was quite large and he did not know how long it would take to run the repair completely.

At this time, I can offer no information as to when these services will be back in full operation or whether any data has been lost and/or corrupted due to the crash.  I do not yet know the root cause of these problems either.

We are aware that this causes an inconvenience to people and a difficulty in conducting various RP and social functions for Layonara.  We're also aware that this has likely suspended things like Character Applications, as the CA team cannot access the threads to comment upon or approve them.

So please bear with us as we work to sort this out.  As more information becomes available, someone will update everyone.  Until that happens, please assume that things are not working properly and assume that we are working toward a fix.

Title: Re: Forums and IRC - September 24, 2010
Post by: orth on September 24, 2010, 03:37:56 PM

A hard crash of the forum server caused the database to corrupt badly.  Upon attempting to repair the database the mysql repair functionality was overzealous in making its corrections and we ended up losing 215 thousand or so posts!  Yikes!

I've restored from backups but they are older than I wished they were.  There may have been posts lost since the last backup and I'm not sure if I can recover them, but I'm attempting a few other things.

I feel awful for this, as a systems administrator I know the community trusts that these issues do not arise and their data is safe.  I had recently moved our database for our forums from a separate server to the same server that the web site runs under and upon doing this our daily backups of our database were not functioning properly.

I'm sorry for any loss of creativity and organization, I'm still doing my best to see if I can track down more of the posts that may have been lost between the end of July and now.

-James

Title: Re: Forums and IRC - September 24, 2010
Post by: Filatus on September 24, 2010, 03:49:46 PM

Good thing I've been lax in updating my CDT. :D

Title: Re: Forums and IRC - September 24, 2010
Post by: gilshem ironstone on September 24, 2010, 04:31:15 PM

It's good to do some house-cleaning every now and then.

Title: Re: Forums and IRC - September 24, 2010
Post by: miltonyorkcastle on September 24, 2010, 04:54:28 PM

Thank you, Orth (and whoever else helps with this), for taking the time to maintain these forums for us. I definitely take these forums always being here for granted, because when they're not, I have one of those "lost" feelings. Sad, I know, but true, heh.

Title: Re: Forums and IRC - September 24, 2010
Post by: orth on September 24, 2010, 05:03:00 PM

Okay some good news.

I have the last 30 days of posts in a cached table.  They aren't easy that easy to recover though, in fact I'm not even sure how best to handle it, but all the text is there.

If you have a CDT/Character Submission or such that you've posted in the last 30 days that you really want back, let me know and I'll be able to PM/email you the content for you to repost.

Title: Re: Forums and IRC - September 24, 2010
Post by: Tanman on September 24, 2010, 05:04:52 PM

Thats cool Orth and Dorg. These things happen and the main thing is that you keep us informed which you have been doing. Keep up the good work.

Title: Re: Forums and IRC - September 24, 2010
Post by: Frances on September 24, 2010, 07:11:39 PM

Something I thought of...

If you're trying to recover a lost thread, you could try googling it and using the cached copy.  It might not be as up to date as you like, but you might get something usable.

Title: Re: Forums and IRC - September 24, 2010
Post by: Alazira on September 24, 2010, 07:38:35 PM

The Angel's Guild keeps our records of peoples sales and credits on the forums.  I'm not sure we have anything in writing more recent than whenever Ferrit made her last update (assuming she saved it on her computer.. and I think that was a week or two back anyhow).  We will REALLY have issues if that can't be found.  Any help?  They would have all been posted under "Guild Orders" In the Angels' Guild Forums.

Thanks!

Katie/Zira

Title: Re: Forums and IRC - September 24, 2010
Post by: orth on September 24, 2010, 07:51:02 PM

Quote from: Alazira

The Angel's Guild keeps our records of peoples sales and credits on the forums.  I'm not sure we have anything in writing more recent than whenever Ferrit made her last update (assuming she saved it on her computer.. and I think that was a week or two back anyhow).  We will REALLY have issues if that can't be found.  Any help?  They would have all been posted under "Guild Orders" In the Angels' Guild Forums.

Thanks!

Katie/Zira

I've PMed Serissa with details on obtaining that information.

Title: Re: Forums and IRC - September 24, 2010
Post by: orth on September 24, 2010, 07:53:38 PM

Quote from: Frances

Something I thought of...

If you're trying to recover a lost thread, you could try googling it and using the cached copy.  It might not be as up to date as you like, but you might get something usable.

Unfortunately a lot of our boards are members only so google doesn't spider them, but it's still worth a shot if the thread was in one of the public forums.

Title: Re: Forums and IRC - September 24, 2010
Post by: darkstorme on September 24, 2010, 09:08:50 PM

There's also the character submissions - they're all gone.

Title: Re: Forums and IRC - September 24, 2010
Post by: Dorganath on September 25, 2010, 12:46:23 AM

OK, everyone...Apparently we've had lingering problems with the forum database.  

We think we have everything cleared up this time, but of course it's possible we'll still discover some other debris from the server's hard crash.

Again, we apologize for the inconvenience and difficulty. Hopefully this will put the crashy problems to rest and we can start to focus on recovering as many of the important missing posts as possible.  We're working on an analysis of this cached data so we can formulate a plan to make the most of what we have.

Title: Re: Forums and IRC - September 24, 2010
Post by: Chazzler on September 25, 2010, 09:09:42 AM

Cailomel's Goods & Wares could also need some recovering, meaning the stickied threads there.

Title: Re: Forums and IRC - September 24, 2010
Post by: Makashi on September 25, 2010, 10:01:37 AM

Theres still quote a few threads I can't seem to open either, a few on the trade market,  general discussion, and suggestions sections

Title: Re: Forums and IRC - September 24, 2010
Post by: Dorganath on September 25, 2010, 11:26:33 AM

We know all about the missing posts, and they will span the entire forum.  What we're wanting though is that information which is critical to you. Character applications, development threads and guild accounting are going to be the top priority, along with GM and plot information.  Unfortunately, the cache we have is mostly just content, with no real identification about who posted it or what thread it goes in (just a post ID). Still, it's something, but it's also a huge amount of information.

So if there is anything you as a community can do to assist the process, we would be pretty grateful.  For example, if you've written or backed up your CDT on your local PC, then please, re-post them.  If your Trade and Market Hall thread is empty, just re-post. If you want to re-suggest a suggestion, please do so.

For guilds, getting a list of both character names and forum names would help greatly, since the contents of those posts sound very similar between the crafting guilds in a lot of cases, and they may be tough to separate.

For character apps, giving us the name of the submitted character to match up with your forum name would also be beneficial.

We're still in the early stages of trying to figure this all out, so the more help you can give, the better.   Bear with us.

Title: Re: Forums and IRC - September 24, 2010
Post by: lonnarin on September 25, 2010, 11:37:01 AM

The important thing is that we still maintained the thanks-counts.  Otherwise Pseudonym would be an emotional wreck right now!

As for the rest, we shall rebuild! *dashes off to work on his posting statistics*

Title: Re: Forums and IRC - September 24, 2010
Post by: Xaltotun on September 25, 2010, 12:21:19 PM

Thanks to Ed for sending me this link as I still can't see it normally...made good reading and thanks to everyone working on it.
 
 It's shame recent stuff got lost, but I guess you guys will honor my approved request for a +10 Short Sword with Immunity to losing Soul Strands?
 
 :D
 
 
 Only kidding.

Title: Re: Forums and IRC - September 24, 2010
Post by: RollinsCat on September 25, 2010, 03:16:11 PM

Tricks for finding cached copies of pages:

Type into your google bar the exact name of the thread you are looking for with  - Layonara after it ; Ex: Andrew Reid - Letters Home - Layonara.

If you see your thread there should be a small blue "cached" option to the right (for google).  hit this, and it will show you the page with all the posts on it including (hopefully) the currently misplaced ones.

If you don't see your thread or see the wrong page, look for "more results from (xx) - ".  In the case of the dev threads, it would say "More results from forums.layonara.com »" at the bottom of the other possible matches in blue - click this, and it will bring up all the pages it has cached.  you can only view one page at a time, clicking on forward/back once in the cache will take you to the live page with the missing posts.

copy/paste away.

Title: Re: Forums and IRC - September 24, 2010
Post by: orth on September 25, 2010, 05:39:56 PM

For those of you PMing me asking for the restoration of certain things please help by providing me with ways to determine that a post is yours.  For example, do you always sign off a certain way, or do you address your CDT to a certain person, maybe you know that you talked about a specific town or deity.  Guilds maybe you follow the same structure for your ledger and we can pluck out posts that way.

Title: Re: Forums and IRC - September 24, 2010
Post by: orth on September 25, 2010, 06:57:32 PM

We've been able to return 650ish posts to their proper threads.  If the post was the first post or the last post in the thread and it was posted in the last 31 days it will now be in its rightful thread.

A bit of an issue with this is we are unable to know exactly who posted the post, so we set the poster to be the person who started the thread.  If you see this as a problem in a thread, we can assign the post to the proper person.  Just tell us who originally wrote it.

Title: Re: Forums and IRC - September 24, 2010
Post by: Hellblazer on September 25, 2010, 07:10:07 PM

If possible to get the stickieds thread of the Cailomel goods and wares forum back to date please, that would be greatly appreciated

Title: Re: Forums and IRC - September 24, 2010
Post by: Dorganath on September 25, 2010, 07:47:35 PM

@Hellblazer and everyone else:

We're attempting to restore as many threads and posts as possible.  We're trying to attack this in a methodical way, and the intent is to get as much as we can restore and identify put back in its rightful place as quickly as possible.

As you can well guess, this cache of posts we have spans both the public and private sections of our forums, with of course varying degrees of privacy, so our efforts are taking this into account as well.

As there seems to be some misunderstandings, let me try to clear up a few things.

What we have is a cached list of posts that have been build up over the last month.  This means that we have a good chance of still having a month's worth of the past two months of missing posts.  Because of how the cache was built, there is a slight chance that we may have a bit more than the last month worth of posts, but that is by no means a guarantee, a promise or anything more than a hope at the present time.

Also, this cache is basically only a post ID number, a timestamp and pre-rendered HTML content.  That's it.

There is nothing to identify the thread to which the posts belonged.  There is nothing to indicate who made the post, outside of some clues contained in the post contents themselves.

To put this another way, there is no quick, easy or simple way, outside of a line-by-line review, to distinguish which were the stickied posts in a guild's forums from private CDT entries from Trade and Market Hall posts.  

While we understand your desires to have your important information restored, asking us to put one group of messages over another is neither helpful nor necessary.

Instead, as has been stated above, if you can provide any information that would help us identify some posts over others, for example if your guild uses a particular posting format for certain things, that would be very beneficial.  Also, providing us with a list of guild members by character and forum names may be helpful in reassigning the posts to the appropriate people.

We have been whittling away at the cached posts, and we have gone from something like 8300 down to about 2600 by eliminating posts that are already present, eliminating null entries in the cache and matching up the first post in each thread with what is available.  A handful of posts has been identified because of some consistency in formatting. Clearly that's an improvement, but the hard part is yet to come.  If we can find patterns, we can restore groups of them with a minimum of manual effort.  Otherwise, it's just going to take time.

To everyone who has supplied us with helpful information, we appreciate your contributions.

To anyone who wants to help, please provide us with any information that might be helpful, such as "All my CDT entries in the thread 'My CDT' start with 'Dear Diary'" or "I sign all my posts with '-me'".  Any patterns we can find...any commonalities that may exist...these things will help us pare down the list a bit more, making less of a manual task out of it.

EDIT: Also, an approximate date (and time if possible) when posts were made would be helpful as well in matching things up to the right places.

So bear with us.  We're getting there.

Title: Re: Forums and IRC - September 24, 2010
Post by: Hellblazer on September 25, 2010, 10:59:21 PM

Talia (Talia)
Gormunguard (Chazzler)
Breanna (Anamnesis)
Hanta deshaldi  (Mastiff of liv)
Fehriel (Hellblazer)
Eleandilethessa Quil'lyn (Xaltotun)
Draman (Grid Blader)
There was some post made by Elohanna.

As for helpful hint in the thread themselves I'm afraid there isn't much I can do about that.

Title: Re: Forums and IRC - September 24, 2010
Post by: Acacea on September 25, 2010, 11:42:46 PM

They can't see the authors of the posts they have. "It was posted by Talia" doesn't matter, they have no posts by Talia. Talia doesn't exist in those posts. They are just numbered. That's why it is hard for them and why is is going to take them a little while. Any phrase at all found within the post would help, but authors don't exist.

(edit for clarity) If you mean that they sign their names that way inside them, it would be more clear ;)

Title: Re: Forums and IRC - September 24, 2010
Post by: RollinsCat on September 26, 2010, 12:04:05 AM

Hellblazer:

here's how to send them a list of post IDs to speed up your restoration.

choose your browser of choice (I was using google until it got sick of me so this is using Yahoo)

Type in a search - I tried "site:forums.layonara.com Cailomel Goods - Layonara"

it brought up this:
http://74.6.239.185/search/srpcache?ei=UTF-8&p=site%3Aforums.layonara.com+Cailomel+Goods+-+Layonara&fr=yfp-t-976&u=http://cc.bingj.com/cache.aspx?q=site%3aforums.layonara.com+Cailomel+Goods+-+Layonara&d=4656663798089366&mkt=en-US&setlang=en-US&w=6f20ba14,43cf375f&icp=1&.intl=us&sig=q.3_3BoLeVk_Oa8RAFdrMg--

this has posts to 08-30-2010.  Using Google you can select "More results from forums.layonara.com »" if it's offered and get more cached pages.  This will require some hunting and playing with search parameters.

Right click the web page and select "View Source".

Start a spreadsheet.  three columns - post ID, thread/forum, and forum name of poster

from the srpcache file that pops up when you select "View Source", find the following: or

(as an example).  In this case 1609062 is the post ID and will be repeated several times inside each post.

List post IDs, thread, and forum name of poster.  send to team.

Should make things faster, hope this helps.

Title: Re: Forums and IRC - September 24, 2010
Post by: Hellblazer on September 26, 2010, 01:47:41 AM

Thanks mate I'll give this a try.

Title: Re: Forums and IRC - September 24, 2010
Post by: orth on September 29, 2010, 01:31:21 AM

Thanks to a lot of grunt work and knowledge of the community, Dorganath was able to restore and associate 812 more missing posts to their proper threads.  In a small percentage of the posts he was not able to ascertain the poster exactly so we used a dummy account "Unknown User".

In addition we found a way to restore 202 of the rightful posters to correlate to their posts in cases where we messed them up in the last restoration procedure.

I'm now going to cleanup the orphans a bit and see where we stand

Title: Re: Forums and IRC - September 24, 2010
Post by: miltonyorkcastle on September 29, 2010, 10:31:12 AM

Rock on.

Title: Re: Forums and IRC - September 24, 2010
Post by: gilshem ironstone on September 29, 2010, 10:59:31 AM

Can you use our profile signature as an identifier?

Title: Re: Forums and IRC - September 24, 2010
Post by: Script Wrecked on September 29, 2010, 11:02:14 AM

The signature is added each time the post is displayed, not when its posted; when you change your signature, its changed "everywhere".