Saving Rework

[McDiod]

Changes to Original System

• only save changes instead of everything (necessitates giving each tracked unit an ID)
• only save pos2D instead of pos3D
• save direction without decimals
• save color as an ID, then look up the actual color RGBA array on the client
• save timestamp as number, convert it to string on client
• adds sendingChunkSize as a config value

Performance Comparison

Comparison to old system was done on a dynamic mission and with a sample size of one, so take this with a grain of salt.

Recording duration: 30 minutes
Precision: 1 / second
Final database size: 1790

Old system load time: 183s
New system load time: 33s

However the load times are basically hard coded by GRAD_REPLAY_SENDING_DELAY. See fn_preparePlaybackServer (old) and fn_preparePlaybackServer (new).

The new system saves time by reducing the sending delay as well as sending multiple arrays at once (GRAD_REPLAY_SENDING_CHUNK_SIZE). It also loses some extra time while the clients assemble their database arrays for playback.

So the question is not how much faster the new system is, but how much extra speed it can get away with before running into synchronization issues. In a four player test there seemed to be no issues, but that might change with 30+ players.

https://youtu.be/qMKz6xqvOao

Example

Example array as stored and then broadcast by the server:

[
    [
        ["iconMan",0,[14701,16740.6],52,"Dylan Lee"," (Alpha 1-1)"],
        "12:00:20"
    ],
    [
        [nil,nil,[14716.6,16756.1],50,nil,nil],
        "12:00:25"
    ],
    [
        [nil,nil,[14733.7,16773.6],42,nil,nil],
        ["iconMan",0,[14679.1,16692.2],71,"Oscar Carter"," (Alpha 1-2)"],
        "12:00:30"
    ],
    [
        [nil,nil,[14751.5,16791.5],44,nil,nil],
        [nil,nil,[14700.3,16692.2],83,nil,nil],
        ["iconMan",0,[14660.1,16688.9],134,"Oliver White"," (Alpha 1-3)"],
        "12:00:35"
    ],
    [
        [nil,nil,[14769.4,16809.4],45,nil,nil],
        [nil,nil,[14722.8,16692.4],79,nil,nil],
        [nil,nil,[14677.8,16670.9],135,nil,nil],
        "12:00:40"
    ]
];

The same array, but assembled by client for playback:

[
    [
        ["iconMan",0,[14701,16740.6],52,"Dylan Lee"," (Alpha 1-1)"],
        "12:00:20"
    ],
    [
        ["iconMan",0,[14716.6,16756.1],50,"Dylan Lee"," (Alpha 1-1)"],
        "12:00:25"
    ],
    [
        ["iconMan",0,[14733.7,16773.6],42,"Dylan Lee"," (Alpha 1-1)"],
        ["iconMan",0,[14679.1,16692.2],71,"Oscar Carter"," (Alpha 1-2)"],
        "12:00:30"
    ],
    [
        ["iconMan",0,[14751.5,16791.5],44,"Dylan Lee"," (Alpha 1-1)"],
        ["iconMan",0,[14700.3,16692.2],83,"Oscar Carter"," (Alpha 1-2)"],
        ["iconMan",0,[14660.1,16688.9],134,"Oliver White"," (Alpha 1-3)"],
        "12:00:35"
    ],
    [
        ["iconMan",0,[14769.4,16809.4,],45,"Dylan Lee"," (Alpha 1-1)"],
        ["iconMan",0,[14722.8,16692.4,],79,"Oscar Carter"," (Alpha 1-2)"],
        ["iconMan",0,[14677.8,16670.9,],135,"Oliver White"," (Alpha 1-3)"],
        "12:00:40"
    ]
];

[McDiod]

Look like this is working. Load time comparison to original system yet to come. Check demo mission on server grad-replay_rework_testX.Altis.

[McDiod]

Tested with 4 people on dedicated without any major issues. One small issue is a freeze that happens here on the clients. Might be a problem on longer missions, because TFAR might time out or Windows could complain about Arma not responding. See video at 00:13.

Recording duration: 30 minutes
Precision: 1 / second
Final database size: 1790

https://youtu.be/ul1znYJUwQo

Edit: freeze fixed

[nomisum]

excited for the performance comparison

[dedmen]

Might be a problem on longer missions, because TFAR might time out or Windows could complain about Arma not responding.

If needed you could temporarily increase the TFAR timeout to something like 30 seconds. And then revert it back after you're done.

[McDiod]

Performance comparison added to OP.

[McDiod]

Okay so functionality-wise this should be fine. The only thing that I'm not 100% sure about is this new function:

/~https://github.com/gruppe-adler/grad-replay/blob/saving-rework/functions/server/fn_canTrackUnit.sqf

Which replaces this:

/~https://github.com/gruppe-adler/grad-replay/blob/master/functions/server/fn_startRecord.sqf#L97

I think the end result is the same, but I might have missed something.

---

I suggest we do a test build of Breaking Contact with this, test that again with as many people as we can get and then use it in an actual game on a monday.

[dedmen]

I would suggest passing _isMan into that function as parameter instead of calling the relatively expensive isKindOf again. As performance in that loop is kinda important.
I myself would even go so far as to not pass any arguments but instead re-use the variables from the higher scope.

Just saw you don't have _isMan before calling that function. But I think overall it would be better to move it above. The only case where _isMan is not needed is the empty vehicle case. And that returns true, thus enters the actual code which needs _isMan anyway.
So _isMan is always needed anyway.

[McDiod]

I myself would even go so far as to not pass any arguments but instead re-use the variables from the higher scope.

Does passing arguments and the params command have a big enough impact on performance to justify that? I mean I guess with the right comments we won't get confused about it in the future, but it's still kind of ugly.

[dedmen]

big enough impact on performance

is always relative. For the TFAR player position update loop it got a big enough impact that it matters for me. But only in the microsecond range.

[McDiod]

Vehicle icons are still showing up multiple times.

/~https://github.com/gruppe-adler/grad-replay/blob/saving-rework/functions/server/fn_canTrackUnit.sqf#L8
vielleicht ist es aber auch einmal der fahrer und einmal das fahrzeug als solches

[McDiod]

WIP again, newest changes still need testing

[McDiod]

Works. Can be merged if you don't want to do any additional testing.