Ever increasing join lag due to a growing leaderboard. #8

New Issue

Not · 2022-09-15T19:47:55+02:00

Not commented

2022-09-15 19:47:55 +02:00

Servers with large ammount of records suffer from a lag spike when a new player joins. I susspect this is due to the size of the savegame being sent, not the plugin itself.
This issue is best fixed in the kart client, but some effort can be made to reduce sent data.
Also worth noting. Servers will crash trying to send 5000+ records. In other words 120kb+ something savegame.

For reference:
Server running no plugins will send 9.3kb savegame.
Server running 6953f343dc with 4550 records will send a 124kb savegame or ~26b per record.
Server running 2d4784f62e with 4550 records will send a 83kb savegame or ~17b per record.

2d4784f62e Compress stats into a single number.

~8 bytes improvement per record

Store playernames and skins once each in a separate table and track their index in the leaderboard table should in theory reduce bytes sent. In my testing this was not the case however, having a +/- 0 effect.

No improvement

Splits consume 8 bytes each (4 bytes per table index, 4 bytes per split). Removing them would save a significant ammount of bytes but thats less than ideal as they are a core function of the plugin. Seeing as they are on average 4 digits long, serializing them to strings and deserialize when needed could save some space.

~3.5 bytes improvement per record

Store mapid (MAPDE) instead of mapnum (145). May save 2 bytes per record.

No improvement

Remove map from score_t.

~0.6 bytes improvement

Don't store any records in netvars, query the server for data. There is no way to do this with the current api provided. (Technically it is possible by hacky means)

Reduce save counts per map/mode. 20 records is the default, give some recomendations for server operators based on how many maps they run.

Servers with large ammount of records suffer from a lag spike when a new player joins. I susspect this is due to the size of the savegame being sent, not the plugin itself. This issue is best fixed in the kart client, but some effort can be made to reduce sent data. Also worth noting. Servers will crash trying to send 5000+ records. In other words 120kb+ something savegame. For reference: Server running no plugins will send 9.3kb savegame. Server running 6953f343dc with 4550 records will send a 124kb savegame or ~26b per record. Server running 2d4784f62e with 4550 records will send a 83kb savegame or ~17b per record. 2d4784f62e Compress stats into a single number. - ~8 bytes improvement per record Store playernames and skins once each in a separate table and track their index in the leaderboard table should in theory reduce bytes sent. In my testing this was not the case however, having a +/- 0 effect. - No improvement Splits consume 8 bytes each (4 bytes per table index, 4 bytes per split). Removing them would save a significant ammount of bytes but thats less than ideal as they are a core function of the plugin. Seeing as they are on average 4 digits long, serializing them to strings and deserialize when needed could save some space. - ~3.5 bytes improvement per record Store mapid (MAP**DE**) instead of mapnum (145). May save 2 bytes per record. - No improvement Remove map from score_t. - ~0.6 bytes improvement Don't store any records in netvars, query the server for data. There is no way to do this with the current api provided. (Technically it is possible by hacky means) Reduce save counts per map/mode. 20 records is the default, give some recomendations for server operators based on how many maps they run.

Fl_GUI commented

2022-10-02 12:52:44 +02:00

I think the bigger problem is that it will always keep growing with the number of records. I think there should be an option to truncate the number of records. Perhaps by date or by position (only keep top 5).

Besides that, I've been thinking if a similar thing to mapranking can be done. That addons stores a file per map. But it doesn't have to show the results to the players.

But what if you had a script that takes the individual leaderboard files per map and hardcode it into a lua file for the next time the server boots.

I'm just brainstorming here.

I think the bigger problem is that it will always keep growing with the number of records. I think there should be an option to truncate the number of records. Perhaps by date or by position (only keep top 5). Besides that, I've been thinking if a similar thing to mapranking can be done. That addons stores a file per map. But it doesn't have to show the results to the players. But what if you had a script that takes the individual leaderboard files per map and hardcode it into a lua file for the next time the server boots. I'm just brainstorming here.

Fl_GUI commented

2022-10-03 23:42:20 +02:00

How evil would it be to create that hardcoded lua file from the main lua and load it on the main lua load with addfile.

Not commented

2022-10-04 19:18:44 +02:00

I think there should be an option to truncate the number of records

Is this not what lb_save_count does?

That addons stores a file per map.

Storing the records in separate files wouldn't solve this issue. The records would all have to be loaded at server start no matter how they are stored.

But what if you had a script that takes the individual leaderboard files per map and hardcode it into a lua file for the next time the server boots.

It is a possible solution and one that would work very well for this issue. Moving the data transmission to the addon downloader is genius. My only gripe with it is that it would require external scripts to work.

How evil would it be to create that hardcoded lua file from the main lua and load it on the main lua load with addfile.

This would have been delightfully devilish. But you can't write .lua files using lua.

> I think there should be an option to truncate the number of records Is this not what *lb_save_count* does? > That addons stores a file per map. Storing the records in separate files wouldn't solve this issue. The records would all have to be loaded at server start no matter how they are stored. > But what if you had a script that takes the individual leaderboard files per map and hardcode it into a lua file for the next time the server boots. It is a possible solution and one that would work very well for this issue. Moving the data transmission to the addon downloader is genius. My only gripe with it is that it would require external scripts to work. > How evil would it be to create that hardcoded lua file from the main lua and load it on the main lua load with addfile. This would have been delightfully devilish. But you can't write .lua files using lua.

Not referenced this issue

2022-10-07 01:42:54 +02:00

Lua loaded records #9

Not commented

2022-12-06 19:28:24 +01:00

Fixed in #9

Not closed this issue

2022-12-06 19:28:24 +01:00

Sign in to join this conversation.