Lagging the server through bad inventory management

[Filatus]

Some of you might wonder what I'm meaning with the title of this post. It is something that has come up before, but lately it seems to have been forgotten by most players. There are always complaints about lag, particulary on West server. After having spend quite some time there lately, I've found out that a lot of the big lag peaks coincide with players logging in and off. These same players generate the same amount of lag on Central, so it has nothing to do with West.

Bad inventory management is probably the reason behind this. If a character is carrying lots of gemdusts, the server will lag when they log on and off. I've seen players cause 12 seconds of lag logging on and off. Playing on West, where this happens the most, would be greatly improved if people would take greater care in making sure they don't have several boxes filled with dust.

I've had several moments in the past two weeks that I was in the middle of a fight and a player probably loaded with lots of gemdust, logged on and caused a 10 second lag peak. Everyone knows what such lag can lead to in tough fights. And the fact is it is so easy to avoid. All it takes is storing excess dusts in crates or selling it off.

I've often wondered before whether I should let players know whether they are lagging the server when logging. Until now I've never done this, since I didn't think it would be proper to start sending tells to people, because reall I'm just another player here.

So as you might have noticed by now, this post has two points. The first being a reminder to the players to refrain from carrying too many dusts around. Please keep a clean and tidy inventory, it really helps making the servers more fun.

The second being a question to the Team about whether it would be a good idea to let people know in-game they lag the server when logging in and off. I ask this question, because the player in question would probably not notice he's lagging the server. Thus the only way he could find out about is by letting him know. On the other hand, perhaps there are multiple reasons that can cause this lag apart from the gemdusts, causes that might be out of a player's hands. The funny thing is, for all I know I might even be lagging the server. If I am and removing some dusts would cure this, I would like to know it.

[Leanthar]

The reason this causes a problem (the large inventories) is because whenever a player logs in to the server NwN (hardcoded) looks through all items in the inventory, including bags and all of that. And every item (including stacks) are checked and verified. The main problem is that players have 6-20 bags and those bags are in turn filled or even 1/2 filled.

I have spoken to players quite a lot but for the post part players tend to forget about this issue. But it does indeed cause login spikes, sometimes up to 10-20 seconds if somebody has a lot of stuff in their inventory.

[Filatus]

That's why I was wondering whether it would be a good idea to be more direct about it. As a player you probably don't know whether you are creating such a lag spike until someone let's you know. As a crafter I can totally understand how quickly your inventory tends to clutter up.

But on the other hand there are also the, let's say 20, other online players to consider who are seeing the game freeze for 10 seconds or more.

And really, the biggest question for me at the moment, being a rather busy crafter, is "Am I causing lag spikes?".

[jrizz]

I would not be offended in the least if I was one of these players and you informed me. Any way I can help cut down lag I will. So this raises a question. If a palyers personal inventory can do this what about the houses filled with filled crates?

[Tobias]

I think those items are saved on the database that and they would only not cause alot of lag? I might be wrong though and I am sure one of the GM's could clarify it better.

[Jilseponie Wyndon]

If I wascausing a problem such as this I also would like to know . . . that and the answer about house inventory . .

~ Jil

[Leanthar]

House inventory causes small spikes to a degree but not really that much. Chest items in a house do not get loaded until the chest is opened. Housing items (the placeables) are loaded when the front door is opened, so it is staggered out pretty good.

But the inventory of players does indeed kill NwN in large ways. It is one of the reasons we changed crafting to the way it is now, one of the many. The more inventory one has the larger the spike and after you go over a couple of bags it gets quite noticeable.

[mikligardr]

a question, specifically to your first point:

to encourage reduction of character's inventories, has there been any consideration of having a merchant actually buy some of the most commonly produced CNR-type items at a reduced rate (reduced as it pertains to what the player-driven market is currently paying for such items) - - gem dusts, raw stones, skeleton bones, etc?


it is really great to be able to find and RP a sale with another player, but since you tend to carry around some of these things in the hopes of making a sale (which are often difficult especially for lower level items, and also due to players' schedules, time zones, and so forth), the inventory builds up rather quickly.     having an NPC merchant that buys some of the more common CNR-type products might at least encourage players to "dump" excess items that they have been lugging around, rather than stockpiling them for some eventual, hoped-for sale.

currently, the pawn shop vendor in Hlint isn't a viable option to reducing one's inventory, since he pays an insignificant amount for most things.  so, if a rough greenstone brings approximately 20-25 gold through a sale with to PC, for example, why not have a vendor (pawn shop guy, maybe the craft vendor in Hlint?) offer 10-13 gold for the item?   a nominal increase that provides an option to PC sales when a PC sale isn't possible would help get some things out of people's inventories.


i'm sure the concern here is an economic one - - the fear of too much gold in the system/inflation, etc... and that's understandable i suppose - - but again, it doesn't have to be an exhorbitant amount of gold paid for items, just a realistic and 'encouraging' increase over what is currently offered by NPCs.

[Leanthar]

It is indeed an economic issue. Everything is considered 'insignificant' when it comes to selling to vendors and I understand that. The problem is when we allowed things to be sold for more (for quite a long time) the economy pretty much disappeared and it was only a little higher than it is now. One of those no win situations. We put the temple stuff in to hope to ease the 'donation' issue some but that hasn't worked overly well--but it has helped that is for certain. We do plan to use that system as a basis for other systems that will likely help the problem some. But either way, I don't know if this problem will ever go away with NwN. NwN, while great mind you, is quite limiting when it comes to a lot of things and this is one of them.
  Sadly (I feel at least) Bioware has stated time and again they do not support PW's. And it is pretty clear that OE feels the same way, though they tend to talk about it in a slightly different way which I suppose is hopeful--but time will tell.
  So... for people that want to run TRUE PW's NwN is not a good solution and I don't think NwN2 will be either.

[Black Cat]

Filatus - 8/15/2006  2:44 AM
.
... And the fact is it is so easy to avoid. All it takes is storing excess dusts in crates or selling it off...

...So as you might have noticed by now, this post has two points. The first being a reminder to the players to refrain from carrying too many dusts around. Please keep a clean and tidy inventory, it really helps making the servers more fun. ...

 ...If I am and removing some dusts would cure this, I would like to know it.

Easy to avoid? Bank chest is too easily filled to be of any use and not everyone own a house or have a room to put crates in.
As for selling them? Perhaps if we could make a bit more of a profit than it actually is. But then there is the economic issue.

But yes, I probably would like to know if I'm lagging the server in that way,

[Honora]

I'm sure this has been brought up but what about a "storage unit"?  A building with crates that can be rented for a set RL amount of time perhaps, so players without housing can keep CNR and other saleable items in a safe location.  This would really benefit lower level characters who call the Wild Surge home for 10 or more levels.

[darkstorme]

Honora - 8/15/2006  8:21 AM

I'm sure this has been brought up but what about a "storage unit"?  A building with crates that can be rented for a set RL amount of time perhaps, so players without housing can keep CNR and other saleable items in a safe location.  This would really benefit lower level characters who call the Wild Surge home for 10 or more levels.

Agreed and seconded.

[Faldred]

Honora - 8/15/2006  8:21 AM

I'm sure this has been brought up but what about a "storage unit"?  A building with crates that can be rented for a set RL amount of time perhaps, so players without housing can keep CNR and other saleable items in a safe location.  This would really benefit lower level characters who call the Wild Surge home for 10 or more levels.

Agreed.  The problem is one of storage, especially for charatcers who participate in lots of crafts.  Now, if I'm causing a server problem, I can thin out Zug's inventory a fair bit (and will try to do so the next time I log on, but between an ox and the bank chest, that's less than 45 items I can keep off-charatcer, so the rest has to stay in the character's inventory.

Storage options need to be increased for journeymen crafters, or we're going to continue to see this problem -- Zug can't afford to throw away hard-to-come by CNR items, even if he can't use them yet because he doesn't have the rest of the recipe.  But they're just clogging up inventory in the meantime.

Another option to limit the number of slots used, and to make the existing storage structures more useful, is to increase the "stackability" of items.  Gem dusts -- why only 10 in a stack instead of 99?  Ditto for scrolls or sandpaper.  Shouldn't star dusts, berries, leaves, herbs, etc., be stackable?  Right now each takes a separate inventory slot, making them stack should make them easier to store in the limited spaces available.  (Or does stacking add to lag?)

As for temple donations, not everyone has easy access to their deity's temples (Vorax has a temple on Mistone, but not even Zug is dumb enough to make the trip there on his own) -- perhaps creating a generic donation center in major towns and cities (Hlint Town Hall, for example) where a character can donate to the deity or relief effort of his/her choice would increase the donations and remove the extra items from the game while still giving the crafter (or finder, in the case of dropped items) some kind of IC "good feeling" for doing so.

[Dorganath]

A couple things to point out....
  On storage facilities...we still need to limit the number of items that can be stored in a chest. The more items (especially if they're stackable) then the more lag when they're opened and closed.
  On stackability, I dare say that making things more stackable, whether increasing stack size or making an item stackable that is currently not, will not do much to discourage the practice of hoarding such items in one's inventory. It's not the number of stacks, it's the number of items that's the problem. by having most CNR things non-stackable (at least the raw CNR). we're effectively trying to limit how many of something peole can carry around.
  Let me throw some numbers at you. Take a large chest that can be bought for housing. You can store 30 items in that chest. If the item is stackable, like gem dusts, then that number increases to 300 right off. Increase the stack size to 20 and now that's 600 items. Item/object creation in NWN is one of the most "expensive" operations in terms of CPU usage, which of course contributes to lag.
  I might suggest a simpler solution, though I know it is going to be met with resistance....
  Get out of the practice of carrying around large amounts of CNR in hopes that you might use the stuff some day. If you can't use it or don't have any specific plans for it, get rid of it, or don't gather it to start. Maybe sell it to someone who can use it, sell it to the pawns or just toss it in the trash. I think a lot of crafters will get massive loads of CNR and then go and craft in a marathon session. For things that require non-stackable ingredients, this practice is generally self-limiting. For those who amass bags/boxes worth of gem dusts, I'm certain it can be done better/differently.
  To give you all a perspective, my main character was at least level 12 before he bought into a house with 3 other characters. He's physically not very strong at all, and could never carry much in terms of CNR. And so he would craft in smaller batches, utilize his ox as needed, and if he ran out of room (or weight) he either used it or dumped it. So it can be done.

[Eorendil]

Idea on gem dusts....
  I know that some gem dusts are used as a spell component and here's an idea to limit the number of item slots they take up while still allowing casters to keep large amounts on their person.  Pouches for gem dusts.  The pouch would have two different uses.  One would bring gem dusts into the pouch, effectively increasing the number of charges the pouch had.  The other would remove individual amounts of gem dust from the pouch for usage by the caster.  Now.. I suppose you could limit the max number of charges/individual servings it is capable of holding but that would be up to the person making the rules.  
  Now.. you'd have to have a pouch for each type of dust you needed to use and I'm not sure if you'd have to sell them pre-labeled for each type.  I've never tried to script something like that.
  That's my idea.   It would definately allow the users to still carry significant amounts on their person.. we all know gem dust doesn't take up that much room. And at the same time it would eliminate the vast number of gem dusts the server has to go through during the loggin in process.

[mikligardr]

so....after reading all the facts regarding storage, stacks of items, and lag, and the myriad of problems associated with them, i guess i'd like to ask again - - why not just make it more rewarding to sell off these lower level CNR items through an NPC?   with all due respect to Leanthar's points on the issue (and i appreciate his quick response) maybe the idea could be looked at again.  

if we are saying that storing, whether in character inventory, Bank chests or personal storage chests in homes causes lag issues regardless of whatever configuration is used, why not encourage PCs - especially lower level PCs who need to generate some coin as anyway - to dump the CNR at an NPC merchant who pays out a better amount?  

and Dorganath you have a valid point regarding changing the way we view CNR and how we all store it ("hoard" it perhaps is a better word    ), but, the Layo system places an undeniable importance and value on crafting here, both from an advancement and RP perspective, so much so that these seemingly little items of CNR become very important in the players' minds.   they aren't so easily gotten rid of i think.   and that's not to say that the system is incorrect or needs to be changed, but rather that the value of these items (due to their great potential through crafting, the risks involved in even finding some CNR and the dangers in trying to collect it) needs to be recognized and perhaps the "pay-out" for simply selling it off to an NPC be made somewhat equitable with the "pay-out" for selling it to a PC.

obviously it should never even be close to what the PC market would pay, but why not 33%? or 25%?  i guess it just seems a bit skewed when, from an RP perspective, my character hears that a crafter wants greenstones for 20-25 gold apiece, and, when unable to make a sale to that crafter for whatever reason, finds that the pawn shop vendor, a businessman, who is a living breathing person, probably with a lovely wife and family to feed, in the same town, just down the road from the craft hall, offers one gold coin for the same item.

i know why its done that way, certainly, but, its a bit paradoxical from my character's perspective.   it makes the interaction seem a little hollow and frustrating.

bottom line is:  you guys know about the economics and past experiences, so i only bring this up for discussion and to maybe suggest re-thinking the issue.    i gladly defer to your judgement on the whole thing, but if the lag is bad because of inventories and storage of CNR, maybe allow for a slightly more lucrative system of NPC sale of CNR, with a limit on the types and quantities that an NPC will buy.

thanks very much for the opportunity to discuss.  

[Leanthar]

This is something (another one of those many topics) that is brought up every 3-6 months, like clockwork. Also one of those things that we get tired of saying the same thing over and over.

"....why not just make it more rewarding to sell off these lower level CNR items through an NPC? with all due respect to Leanthar's points on the issue (and i appreciate his quick response) maybe the idea could be looked at again...."

Sure we can do that and rest assured we will have a character wipe within a few months because the economy is destroyed. Seen it, done it, too many times in 4+ years.

[jrizz]

I read on another server site that has a CNR system that they limit the amount of time you can carry raw CNR on your person. I am not sure how they enforce this. I have not used the scripting engine in NWN but I am an engineer can you tag a raw CNR with a datetime stamp when collected and then add a function to the clean up script that cleans up items older then X days? This would at least take care of the mass hording of raw CNR. As for processed CNR (ingots, dusts, cut gems, wood items) just increase the weight significantly. This would discourage the carrying around of large amounts. I would exclude any items that are used for spell components (topaz dust as an example).

[Stephen_Zuckerman]

Wow, that's a great idea for limiting the number of dusts. (The idea to make a chargable item for dusts.)

However, it's not just dusts that are the problem. Scrolls, licorice, ivy, ALL the stackable items that people hoard. In a box, I generally have probably thirty or forty items for the boxes I keep scrolls or gems in, on average. Add that to probably thirty wheat flours, and a variety of other CNR that I know exactly when I'll use, and you've got yourself a rather unattractive number of items to be created.

The problem, for me, isn't hoarding CNR that I may or may not use. Pyyran can't carry enough, weight-wise, for me to do that. The problem is the things I WILL use, and not having anywhere to store them. When I play Pyyran, he's a pretty active food crafter and alchaemist - those are CNR-heavy crafts, just in the variety of things you need. Myself, I'd be overjoyed to NOT carry the things around, but I don't have a place to store upwards of fifty items.

Why do most people get player housing? Storage.

What isn't available on Mistone any more (for purchase, at least)? Player housing.

The problem here isn't a lack of player housing. It's a lack of activity among players who bought housing back at the start of V2, but who don't play here anymore, and just haven't asked for the characters to be deleted. Is there a reasonable solution to this? Not really. Clearing ANY of the houses might end us up with a player who comes back after all this time, wondering where his house is, which isn't an attractive scenario.

So... A possible solution is rentable storage. By players, by the DM team's wonderful coding, by whatever. It's been suggested, earlier, that affordable rentable storage is the answer. The Arms is expensive enough that most characters couldn't afford a room there - it's a GREAT organization, but the room numbers are limited, even if one had the gold.

Is there a system that could be set up where every week, the system checks a given "slot" for a character to see if they've put more money in it, and if not, they go into a "warning period." After another week, everything in the chests gets given to some merchant. If a character puts money in during the "warning period," they have to pay double, or somesuch.

The Gnomish Contraption takes gold from a character and gives them a key. Couldn't it set an integer or two on the database, if it were configured for the system I mentioned?

[mikligardr]

Leanthar - 8/15/2006  1:38 PM

This is something (another one of those many topics) that is brought up every 3-6 months, like clockwork. Also one of those things that we get tired of saying the same thing over and over.

"....why not just make it more rewarding to sell off these lower level CNR items through an NPC? with all due respect to Leanthar's points on the issue (and i appreciate his quick response) maybe the idea could be looked at again...."

Sure we can do that and rest assured we will have a character wipe within a few months because the economy is destroyed. Seen it, done it, too many times in 4+ years.

er... okay.   sorry if i touched a nerve.  if it is brought up "like clockwork", doesn't that seem to indicate an ongoing problem?  i don't have 4+ years on the server, like many others playing here now, so if we bring up the topic, it's only because we don't know that the discussion has previously occurred and (apparently) been resolved from your point of view.

so if thats not a viable option, do you have any opinion regarding the inventory/storage - lag problem?

thanks.