Database cleanup

General discussion about Life is Feudal MMO and Life is Feudal: Your Own, The main section and backbone of the forums.
User avatar
HolyCrusader
Beta Tester
 
Posts: 251
Joined: 24 Nov 2014, 15:47

Database cleanup

Post by HolyCrusader » 02 Jan 2015, 20:34

A couple points and questions for other server admins out there...

Do you find that deleting large tables of unnecessary data helps server performance? For example, we deleted many rows in a table that was tracking skill-ups and character wounds. I also went around in GM mode and cleaned up a bunch of stumps and logs lying around. Before this we were starting to crash two times per day, and now its back to once per day.

If you've done some DB maintenance, can you please share what tables have data that can be removed without harming the server?


Rpgmoods
 
Posts: 7
Joined: 23 Oct 2014, 09:48

Re: Database cleanup

Post by Rpgmoods » 04 Jan 2015, 12:05

I would love to hear more about what i can prune out of the DB. My server's DB has blown up to around 170M now. My server does crash, but that seems to be happening during a feeding cycle when an animal dies of starvation.

My real concern is load times.
It now takes around 15 to 17 minutes to boot the server and some players are reporting load times into the server of 15+ minutes also.

My server hardware is solid and the game isn't taxing my resources at all. The server software and the DB both sit on an SSD and i hvae a 10Gbps connection so bandwidth should not be a problem.

The only other thing that could be holding the server up is the huge DB. The Devs have confirmed that long load times are a problem with how large DB's are processed by the game. They claim that partial fixes are coming in future patches but until then I would love for them to tell us what exactly we can "safely" remove from the DB.

User avatar
Itspreach
 
Posts: 6
Joined: 20 Feb 2015, 17:01

Re: Database cleanup

Post by Itspreach » 20 Feb 2015, 17:39

I would like to bump this post for info from other server hosts too please. I've read a lot of the forums recently and have heard that removing stumps will help - but also that a large number of animals (or any server setting over "1" for animals) may be a cause of the server taking upwards of 15+ minutes to start up.
When Religion Ruled the World, They Called it the Dark Ages.


Sebas555
 
Posts: 246
Joined: 10 Jan 2015, 02:01
Location: Quebec, Canada

Re: Database cleanup

Post by Sebas555 » 20 Feb 2015, 18:46

Ban tunnels of any kind on your sever, make a huge difference. Deleting very low q trees also helps... I play on a pretty active server and the game loads in less than 30 secs... no tunnels is the key I think because we have all trees and there is no auto uproot, even if they speed up the uproot timer to make it more easy to do.

User avatar
Itspreach
 
Posts: 6
Joined: 20 Feb 2015, 17:01

Re: Database cleanup

Post by Itspreach » 20 Feb 2015, 19:28

Sebas555 wrote:Ban tunnels of any kind on your sever, make a huge difference.


Doesn't that mean to ban mining too? How would we get ore and rocks?
When Religion Ruled the World, They Called it the Dark Ages.


Sebas555
 
Posts: 246
Joined: 10 Jan 2015, 02:01
Location: Quebec, Canada

Re: Database cleanup

Post by Sebas555 » 20 Feb 2015, 20:14

Itspreach wrote:
Sebas555 wrote:Ban tunnels of any kind on your sever, make a huge difference.


Doesn't that mean to ban mining too? How would we get ore and rocks?

Infinite plies... ;)

User avatar
Eslake
 
Posts: 73
Joined: 30 Dec 2014, 17:15

Re: Database cleanup

Post by Eslake » 23 Feb 2015, 08:47

Twiztedmike wrote:deleting large tables of unnecessary data helps server performance?


The reason most of them break the game if you delete is that the Version no longer matches up to what is kept in terrain_blocks.
Last edited by Eslake on 23 May 2015, 08:25, edited 1 time in total.

User avatar
Razoreqx
 
Posts: 91
Joined: 06 Oct 2014, 13:13

Re: Database cleanup

Post by Razoreqx » 23 Feb 2015, 18:45

Eslake wrote:
Twiztedmike wrote:deleting large tables of unnecessary data helps server performance?


Be very, very careful what you are deleting.

There are things that are handled sequentially and need to remain unless you 'shift' down the ID of all later entries to fill the gap.

The unfortunate reality is that the way they set things up is never going to allow lasting servers. Things just pile up.

As example.. geo_patch data.

A character logs on and walks over to 116978577 (a bump in the grass on the East coast)
There is no entry for this spot in geo_patch because nothing has ever happened here, so lets dig 1 time.

That one action adds the following.
Code: Select all
###+0   446   5313   1   0   1   116978577   5095   1   0   6000   1   14
###+1   446   5313   2   0   2   116978577   5096   NULL   NULL   NULL   NULL   NULL


Dropping the soil back makes it..
Code: Select all
###+0   446   5313   1   0   1   116978577   5095   1   0   6000   14
###+1   446   5313   2   0   2   116978577   5096   NULL   NULL   NULL   NULL
###+2   446   5313   2   0   2   116978577   5096   NULL   NULL   NULL   NULL
###+3   446   5314   1   0   1   116978577   5096   6   0   60000   14


To make the server viable long-term, picking that soil back up should revert it to the First entry..
Instead we get this.
Code: Select all
###+0   446   5313   1   0   1   116978577   5095   1   0   6000   14
###+1   446   5313   2   0   2   116978577   5096   NULL   NULL   NULL   NULL
###+2   446   5313   2   0   2   116978577   5096   NULL   NULL   NULL   NULL
###+3   446   5314   1   0   1   116978577   5096   6   0   60000   14   
###+4   446   5315   1   0   2   116978577   5096   NULL   NULL   NULL   NULL
###+5   446   5315   2   0   1   116978577   5096   1   0   60000   14


Every action that ever takes place on the landmap is stored, rather than the end-state. So the longer you play the larger it gets, and the more data has to be sent to Every user every time someone digs or drops anything.

And you cannot delete the now useless entries, because this is one of those instances I mentioned before.

I hope they eventually add a simple check for a previous state matching the current one, and removing all later entries to achieve the same result, but without the bloat.


Nice post and pretty much the same conclusion we found
Razors Edge
A Life is Feudal Persistent World
http://razors-edge.org
22 Custom MODS / Custom Graphics models Planned. Visit our Gallery.


RudgerWolvram
 
Posts: 37
Joined: 01 Jan 2015, 22:27

Re: Database cleanup

Post by RudgerWolvram » 26 Feb 2015, 05:42

I need to just sit down and try to work out a geo_patch patching system. Basically, when the server restarts, like maybe in the patch.sql, all soil in the patch gets crunched down.

The reason (at least the best i can figure) as to why the geo_patch keeps everything is because you place down some dirt, but you gotta place down 9 more to get it high enough for whatever reason. Now, you have 10 dirt layers, each one has it's own quality and type. e.g. soil, fertile soil, forest soil, etc.

So what I've been wanting to do is find all entries for each geoID that is soil, and then regress them to a single line as if you just put the soil down once. Of course, this would be for soil/sand/snow only and would completely ignore any ore piles. The danger with this though is put sand on ground, now, that square is forever sand when you dig, until you cover it with some soil and let the nightly reboot regress it, at least that's in theory, the game may see it as a removal of something that was placed and you end up with whatever the original map soil is.


Nachtschatten
 
Posts: 12
Joined: 19 Jan 2015, 08:57

Re: Database cleanup

Post by Nachtschatten » 26 Feb 2015, 10:26

i dunno why, but when im logging to server, hitting alt+tab to switch out of the loading screen, loading time goes down from 15+ minutes to 3 minutes... it takes a little while ingame until i can move but thats a matter of seconds


Remystemple
 
Posts: 80
Joined: 15 Jan 2015, 22:30

Re: Database cleanup

Post by Remystemple » 26 Feb 2015, 11:00

it has something to do with the gpu ( i think ). using windowed mode is helps too. and i have read that setting up a profile for LIF in your graphics card software helps. seems to have helped me. also running the tree remover script helps more than anything. my server is 3 weeks old now and load times are about 2 mins or less ( for me anyway ) it's not on my comp it's hosted. i owe that to removing low quality trees once / week.

User avatar
Razoreqx
 
Posts: 91
Joined: 06 Oct 2014, 13:13

Re: Database cleanup

Post by Razoreqx » 26 Feb 2015, 16:18

RudgerWolvram wrote:I need to just sit down and try to work out a geo_patch patching system. Basically, when the server restarts, like maybe in the patch.sql, all soil in the patch gets crunched down.

The reason (at least the best i can figure) as to why the geo_patch keeps everything is because you place down some dirt, but you gotta place down 9 more to get it high enough for whatever reason. Now, you have 10 dirt layers, each one has it's own quality and type. e.g. soil, fertile soil, forest soil, etc.

So what I've been wanting to do is find all entries for each geoID that is soil, and then regress them to a single line as if you just put the soil down once. Of course, this would be for soil/sand/snow only and would completely ignore any ore piles. The danger with this though is put sand on ground, now, that square is forever sand when you dig, until you cover it with some soil and let the nightly reboot regress it, at least that's in theory, the game may see it as a removal of something that was placed and you end up with whatever the original map soil is.



Let me know what you find. I've been working on the for a while now
Razors Edge
A Life is Feudal Persistent World
http://razors-edge.org
22 Custom MODS / Custom Graphics models Planned. Visit our Gallery.

Return to General Discussion