Calculate confidence interval for benchmark measurements

[SiarheiFedartsou]

This PR:

• adds confidence intervals to some of benchmark results(i.e. instead of avg: 42ms we now show avg: 42 +- 2ms, but to achieve that each benchmark now runs 5 times on CI
• improves benchmark scripts, so it becomes easier to run them locally
• some of benchmark cases are removed to make benchmarks faster (e.g. there is no need to run map matching with 4 different radiuses, it is enough to have 2)
• added new ops/sec metric to each bencmark
• fix bug in bench utility when we used huge (up to 5k points) traces as input for map matching API which caused significant slow downs of benchmarks

It still doesn't make benchmarks very stable and I still quite often see huge "jumps" between master and PR, but it is a step to direction of making it more stable(in general IMO we should eventually have some separate machine for benchmarks to avoid "noisy neighbours" on GH actions).

Benchmark Results

Benchmark	Base	PR
alias	aliased u32: 1104.12 plain u32: 1094.34 aliased double: 956.011 plain double: 947.599	aliased u32: 1100.2 plain u32: 1112.57 aliased double: 954.001 plain double: 951.872
e2e_match_ch	Ops: 46.27 ± 0.12 ops/s. Best: 46.12 ops/s Total: 2831.65ms ± 7.31ms. Best: 2821.57ms Min time: 2.23ms ± 0.01ms Mean time: 21.61ms ± 0.06ms Median time: 16.20ms ± 0.07ms 95th percentile: 69.59ms ± 0.26ms 99th percentile: 84.18ms ± 0.15ms Max time: 97.50ms ± 0.50ms	Ops: 47.32 ± 0.14 ops/s. Best: 47.18 ops/s Total: 2768.90ms ± 8.19ms. Best: 2750.26ms Min time: 2.24ms ± 0.04ms Mean time: 21.13ms ± 0.06ms Median time: 15.53ms ± 0.24ms 95th percentile: 66.75ms ± 0.26ms 99th percentile: 78.91ms ± 0.31ms Max time: 93.24ms ± 0.43ms
e2e_match_mld	Ops: 62.03 ± 0.09 ops/s. Best: 61.88 ops/s Total: 2111.71ms ± 3.15ms. Best: 2105.63ms Min time: 1.86ms ± 0.04ms Mean time: 16.12ms ± 0.02ms Median time: 8.72ms ± 0.05ms 95th percentile: 53.51ms ± 0.15ms 99th percentile: 61.85ms ± 0.17ms Max time: 72.31ms ± 0.37ms	Ops: 66.83 ± 0.15 ops/s. Best: 66.49 ops/s Total: 1960.26ms ± 4.77ms. Best: 1954.97ms Min time: 1.83ms ± 0.03ms Mean time: 14.96ms ± 0.04ms Median time: 7.96ms ± 0.12ms 95th percentile: 49.21ms ± 0.20ms 99th percentile: 57.21ms ± 0.25ms Max time: 66.24ms ± 0.38ms
e2e_nearest_ch	Ops: 792.25 ± 4.10 ops/s. Best: 783.83 ops/s Total: 1262.22ms ± 6.92ms. Best: 1251.52ms Min time: 1.07ms ± 0.01ms Mean time: 1.26ms ± 0.01ms Median time: 1.17ms ± 0.01ms 95th percentile: 1.67ms ± 0.01ms 99th percentile: 1.73ms ± 0.01ms Max time: 4.35ms ± 2.61ms	Ops: 743.93 ± 9.35 ops/s. Best: 727.35 ops/s Total: 1344.51ms ± 18.36ms. Best: 1319.59ms Min time: 1.07ms ± 0.01ms Mean time: 1.34ms ± 0.02ms Median time: 1.20ms ± 0.01ms 95th percentile: 1.81ms ± 0.07ms 99th percentile: 2.97ms ± 0.20ms Max time: 6.29ms ± 1.35ms
e2e_nearest_mld	Ops: 802.40 ± 7.51 ops/s. Best: 783.80 ops/s Total: 1246.33ms ± 12.16ms. Best: 1234.80ms Min time: 1.06ms ± 0.01ms Mean time: 1.25ms ± 0.01ms Median time: 1.15ms ± 0.01ms 95th percentile: 1.65ms ± 0.01ms 99th percentile: 1.74ms ± 0.08ms Max time: 4.48ms ± 2.22ms	Ops: 744.44 ± 27.13 ops/s. Best: 699.31 ops/s Total: 1347.32ms ± 49.70ms. Best: 1289.08ms Min time: 1.06ms ± 0.00ms Mean time: 1.35ms ± 0.05ms Median time: 1.19ms ± 0.01ms 95th percentile: 1.93ms ± 0.25ms 99th percentile: 2.90ms ± 0.67ms Max time: 7.11ms ± 1.48ms
e2e_route_ch	Ops: 329.26 ± 3.47 ops/s. Best: 322.92 ops/s Total: 3035.96ms ± 31.12ms. Best: 2998.25ms Min time: 1.32ms ± 0.01ms Mean time: 3.04ms ± 0.03ms Median time: 3.07ms ± 0.02ms 95th percentile: 4.01ms ± 0.06ms 99th percentile: 4.42ms ± 0.08ms Max time: 7.05ms ± 2.25ms	Ops: 316.14 ± 11.74 ops/s. Best: 300.28 ops/s Total: 3173.61ms ± 118.26ms. Best: 3014.25ms Min time: 1.31ms ± 0.03ms Mean time: 3.17ms ± 0.12ms Median time: 3.13ms ± 0.07ms 95th percentile: 4.47ms ± 0.37ms 99th percentile: 6.12ms ± 0.98ms Max time: 9.07ms ± 1.11ms
e2e_route_mld	Ops: 287.98 ± 1.78 ops/s. Best: 284.37 ops/s Total: 3473.56ms ± 22.72ms. Best: 3446.35ms Min time: 1.29ms ± 0.01ms Mean time: 3.47ms ± 0.02ms Median time: 3.51ms ± 0.04ms 95th percentile: 4.72ms ± 0.03ms 99th percentile: 5.20ms ± 0.08ms Max time: 7.33ms ± 1.58ms	Ops: 269.30 ± 4.16 ops/s. Best: 263.78 ops/s Total: 3713.24ms ± 57.66ms. Best: 3600.81ms Min time: 1.31ms ± 0.01ms Mean time: 3.71ms ± 0.07ms Median time: 3.72ms ± 0.05ms 95th percentile: 5.16ms ± 0.15ms 99th percentile: 6.45ms ± 0.67ms Max time: 8.98ms ± 0.66ms
e2e_table_ch	Ops: 305.07 ± 1.96 ops/s. Best: 301.96 ops/s Total: 3277.56ms ± 21.07ms. Best: 3247.34ms Min time: 1.68ms ± 0.02ms Mean time: 3.28ms ± 0.02ms Median time: 3.27ms ± 0.02ms 95th percentile: 4.53ms ± 0.05ms 99th percentile: 4.88ms ± 0.10ms Max time: 8.12ms ± 2.40ms	Ops: 299.80 ± 2.08 ops/s. Best: 295.20 ops/s Total: 3334.58ms ± 23.24ms. Best: 3315.45ms Min time: 1.69ms ± 0.02ms Mean time: 3.34ms ± 0.02ms Median time: 3.32ms ± 0.02ms 95th percentile: 4.64ms ± 0.06ms 99th percentile: 5.20ms ± 0.22ms Max time: 8.90ms ± 1.54ms
e2e_table_mld	Ops: 108.28 ± 1.59 ops/s. Best: 104.95 ops/s Total: 9237.90ms ± 145.04ms. Best: 9080.94ms Min time: 3.70ms ± 0.07ms Mean time: 9.24ms ± 0.14ms Median time: 9.20ms ± 0.13ms 95th percentile: 14.10ms ± 0.25ms 99th percentile: 15.16ms ± 0.26ms Max time: 20.47ms ± 4.07ms	Ops: 106.82 ± 0.74 ops/s. Best: 105.67 ops/s Total: 9360.04ms ± 64.87ms. Best: 9266.90ms Min time: 3.76ms ± 0.04ms Mean time: 9.36ms ± 0.07ms Median time: 9.34ms ± 0.07ms 95th percentile: 14.24ms ± 0.11ms 99th percentile: 15.46ms ± 0.20ms Max time: 20.24ms ± 1.69ms
e2e_trip_ch	Ops: 96.16 ± 0.55 ops/s. Best: 95.04 ops/s Total: 10400.64ms ± 59.69ms. Best: 10305.98ms Min time: 1.56ms ± 0.13ms Mean time: 10.40ms ± 0.06ms Median time: 9.88ms ± 0.08ms 95th percentile: 18.34ms ± 0.10ms 99th percentile: 20.00ms ± 0.13ms Max time: 26.57ms ± 4.10ms	Ops: 92.23 ± 1.31 ops/s. Best: 90.05 ops/s Total: 10839.53ms ± 158.68ms. Best: 10669.79ms Min time: 1.67ms ± 0.19ms Mean time: 10.85ms ± 0.16ms Median time: 10.37ms ± 0.16ms 95th percentile: 18.89ms ± 0.18ms 99th percentile: 20.62ms ± 0.32ms Max time: 23.83ms ± 1.28ms
e2e_trip_mld	Ops: 56.88 ± 1.32 ops/s. Best: 54.81 ops/s Total: 17609.93ms ± 430.88ms. Best: 16922.75ms Min time: 1.82ms ± 0.18ms Mean time: 17.59ms ± 0.43ms Median time: 17.20ms ± 0.40ms 95th percentile: 28.72ms ± 0.72ms 99th percentile: 30.56ms ± 0.64ms Max time: 33.54ms ± 1.84ms	Ops: 55.40 ± 0.47 ops/s. Best: 54.57 ops/s Total: 18046.40ms ± 162.29ms. Best: 17749.95ms Min time: 1.72ms ± 0.22ms Mean time: 18.05ms ± 0.18ms Median time: 17.65ms ± 0.14ms 95th percentile: 29.29ms ± 0.22ms 99th percentile: 31.54ms ± 0.20ms Max time: 39.64ms ± 3.73ms
json-render	String: 7.45622ms Stringstream: 10.6755ms Vector: 6.85354ms	String: 6.64681ms Stringstream: 9.45432ms Vector: 6.97276ms
match_ch	Default radius: 4.64322ms/req at 82 coordinate 0.0566246ms/coordinate Radius 5m: 4.62759ms/req at 82 coordinate 0.056434ms/coordinate Radius 10m: 16.0754ms/req at 82 coordinate 0.196041ms/coordinate Radius 15m: 39.239ms/req at 82 coordinate 0.478525ms/coordinate Radius 30m: 336.981ms/req at 82 coordinate 4.10952ms/coordinate	Default radius: 4.47327ms/req at 82 coordinate 0.0545521ms/coordinate Radius 10m: 15.2426ms/req at 82 coordinate 0.185885ms/coordinate
match_mld	Default radius: 3.18365ms/req at 82 coordinate 0.038825ms/coordinate Radius 5m: 3.13708ms/req at 82 coordinate 0.0382571ms/coordinate Radius 10m: 11.7563ms/req at 82 coordinate 0.14337ms/coordinate Radius 15m: 29.9239ms/req at 82 coordinate 0.364925ms/coordinate Radius 30m: 346.859ms/req at 82 coordinate 4.22999ms/coordinate	Default radius: 2.79943ms/req at 82 coordinate 0.0341394ms/coordinate Radius 10m: 10.3753ms/req at 82 coordinate 0.126528ms/coordinate
osrm_contract	Time: 96.58s Peak RAM: 196.04MB	Time: 95.67s Peak RAM: 185.66MB
osrm_customize	Time: 1.37s Peak RAM: 116.67MB	Time: 1.31s Peak RAM: 115.26MB
osrm_extract	Time: 12.09s Peak RAM: 403.17MB	Time: 12.07s Peak RAM: 413.81MB
osrm_partition	Time: 2.02s Peak RAM: 135.80MB	Time: 2.16s Peak RAM: 148.92MB
packedvector	random write: std::vector 9881.38 ms util::packed_vector 81985.6 ms slowdown: 8.29698 random read: std::vector 8432.27 ms util::packed_vector 30989.4 ms slowdown: 3.6751	random write: std::vector 9850.49 ms util::packed_vector 81826.4 ms slowdown: 8.30684 random read: std::vector 8571.45 ms util::packed_vector 32923.7 ms slowdown: 3.84109
random_match_ch	1000 matches, default radius total: 7054.47ms avg: 7.05ms min: 0.00ms max: 501.08ms p99: 109.76ms 1000 matches, radius=10 total: 35260.86ms avg: 35.26ms min: 0.00ms max: 1910.36ms p99: 1869.08ms 1000 matches, radius=20 total: 67980.72ms avg: 67.98ms min: 0.00ms max: 9270.73ms p99: 1170.25ms	500 matches, default radius ops: 227.43 ± 0.93 ops/s. best: 225.15ops/s. total: 250.64 ± 1.03ms. best: 249.75ms. avg: 4.40 ± 0.02ms min: 0.15 ± 0.01ms max: 22.95 ± 0.06ms p99: 22.95 ± 0.06ms 500 matches, radius=10 ops: 67.70 ± 0.06 ops/s. best: 67.64ops/s. total: 945.40 ± 0.76ms. best: 943.67ms. avg: 14.77 ± 0.01ms min: 0.15 ± 0.00ms max: 214.26 ± 0.89ms p99: 214.26 ± 0.89ms 500 matches, radius=20 ops: 15.76 ± 0.07 ops/s. best: 15.67ops/s. total: 4124.58 ± 18.13ms. best: 4096.79ms. avg: 63.46 ± 0.28ms min: 0.30 ± 0.00ms max: 1104.51 ± 7.63ms p99: 1104.51 ± 7.63ms
random_match_mld	1000 matches, default radius total: 5873.04ms avg: 5.87ms min: 0.00ms max: 441.54ms p99: 82.59ms 1000 matches, radius=10 total: 30491.07ms avg: 30.49ms min: 0.00ms max: 1772.97ms p99: 1734.61ms 1000 matches, radius=20 total: 58897.53ms avg: 58.90ms min: 0.00ms max: 7754.15ms p99: 911.83ms	500 matches, default radius ops: 342.24 ± 2.08 ops/s. best: 337.21ops/s. total: 166.56 ± 1.02ms. best: 165.57ms. avg: 2.92 ± 0.02ms min: 0.12 ± 0.00ms max: 17.98 ± 0.07ms p99: 17.98 ± 0.07ms 500 matches, radius=10 ops: 120.45 ± 0.16 ops/s. best: 120.12ops/s. total: 531.36 ± 0.69ms. best: 530.67ms. avg: 8.30 ± 0.01ms min: 0.15 ± 0.00ms max: 101.85 ± 0.20ms p99: 101.85 ± 0.20ms 500 matches, radius=20 ops: 23.33 ± 0.02 ops/s. best: 23.31ops/s. total: 2785.64 ± 2.37ms. best: 2782.07ms. avg: 42.86 ± 0.04ms min: 0.20 ± 0.00ms max: 546.55 ± 1.13ms p99: 546.55 ± 1.13ms
random_nearest_ch	10000 nearest, number_of_results=1 total: 456.69ms avg: 0.05ms min: 0.01ms max: 0.24ms p99: 0.12ms 10000 nearest, number_of_results=5 total: 600.24ms avg: 0.06ms min: 0.02ms max: 0.17ms p99: 0.13ms 10000 nearest, number_of_results=10 total: 759.15ms avg: 0.08ms min: 0.03ms max: 0.22ms p99: 0.14ms	10000 nearest, number_of_results=1 ops: 21975.01 ± 190.44 ops/s. best: 21486.23ops/s. total: 455.12 ± 4.01ms. best: 452.06ms. avg: 0.05 ± 0.00ms min: 0.01 ± 0.00ms max: 0.20 ± 0.01ms p99: 0.12 ± 0.00ms 10000 nearest, number_of_results=5 ops: 16629.80 ± 27.18 ops/s. best: 16568.88ops/s. total: 601.33 ± 1.00ms. best: 600.40ms. avg: 0.06 ± 0.00ms min: 0.02 ± 0.00ms max: 0.18 ± 0.01ms p99: 0.13 ± 0.00ms 10000 nearest, number_of_results=10 ops: 13105.54 ± 10.10 ops/s. best: 13091.43ops/s. total: 763.04 ± 0.59ms. best: 761.93ms. avg: 0.08 ± 0.00ms min: 0.03 ± 0.00ms max: 0.18 ± 0.00ms p99: 0.14 ± 0.00ms
random_nearest_mld	10000 nearest, number_of_results=1 total: 459.13ms avg: 0.05ms min: 0.01ms max: 0.21ms p99: 0.12ms 10000 nearest, number_of_results=5 total: 600.80ms avg: 0.06ms min: 0.02ms max: 0.16ms p99: 0.13ms 10000 nearest, number_of_results=10 total: 753.47ms avg: 0.08ms min: 0.03ms max: 0.21ms p99: 0.14ms	10000 nearest, number_of_results=1 ops: 21859.96 ± 29.44 ops/s. best: 21810.65ops/s. total: 457.46 ± 0.62ms. best: 456.52ms. avg: 0.05 ± 0.00ms min: 0.01 ± 0.00ms max: 0.20 ± 0.01ms p99: 0.12 ± 0.00ms 10000 nearest, number_of_results=5 ops: 16385.43 ± 188.99 ops/s. best: 16078.06ops/s. total: 610.41 ± 7.08ms. best: 601.97ms. avg: 0.06 ± 0.00ms min: 0.02 ± 0.00ms max: 0.23 ± 0.07ms p99: 0.13 ± 0.01ms 10000 nearest, number_of_results=10 ops: 13000.73 ± 29.56 ops/s. best: 12946.38ops/s. total: 769.19 ± 1.75ms. best: 766.58ms. avg: 0.08 ± 0.00ms min: 0.03 ± 0.00ms max: 0.24 ± 0.05ms p99: 0.14 ± 0.00ms
random_route_ch	10000 routes, 3 coordinates, no alternatives, overview=full, steps=true total: 20080.28ms avg: 2.01ms min: 0.13ms max: 4.17ms p99: 2.97ms 10000 routes, 2 coordinates, no alternatives, overview=full, steps=true total: 9374.27ms avg: 0.94ms min: 0.06ms max: 3.18ms p99: 1.59ms 10000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true total: 17306.33ms avg: 1.73ms min: 0.07ms max: 5.23ms p99: 4.10ms 10000 routes, 3 coordinates, no alternatives, overview=false, steps=false total: 10431.25ms avg: 1.04ms min: 0.12ms max: 2.14ms p99: 1.50ms 10000 routes, 2 coordinates, no alternatives, overview=false, steps=false total: 4208.02ms avg: 0.42ms min: 0.05ms max: 0.77ms p99: 0.64ms 10000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false total: 9176.90ms avg: 0.92ms min: 0.06ms max: 4.91ms p99: 2.36ms 10000 routes, 3 coordinates, no alternatives, overview=false, steps=false, radius=750 total: 648.92ms avg: 0.06ms min: 0.01ms max: 1.48ms p99: 0.64ms 10000 routes, 2 coordinates, no alternatives, overview=false, steps=false, radius=750 total: 666.56ms avg: 0.07ms min: 0.01ms max: 0.48ms p99: 0.39ms 10000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false, radius=750 total: 901.19ms avg: 0.09ms min: 0.01ms max: 2.14ms p99: 1.16ms	1000 routes, 3 coordinates, no alternatives, overview=full, steps=true ops: 481.35 ± 0.43 ops/s. best: 480.53ops/s. total: 2044.26 ± 1.84ms. best: 2041.81ms. avg: 2.08 ± 0.00ms min: 0.36 ± 0.01ms max: 3.35 ± 0.13ms p99: 2.96 ± 0.01ms 1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true ops: 562.20 ± 1.17 ops/s. best: 560.58ops/s. total: 1778.73 ± 3.80ms. best: 1773.11ms. avg: 1.78 ± 0.00ms min: 0.06 ± 0.00ms max: 4.40 ± 0.11ms p99: 3.96 ± 0.05ms 1000 routes, 3 coordinates, no alternatives, overview=false, steps=false ops: 990.08 ± 6.09 ops/s. best: 979.22ops/s. total: 993.91 ± 6.13ms. best: 985.68ms. avg: 1.01 ± 0.01ms min: 0.26 ± 0.01ms max: 1.53 ± 0.06ms p99: 1.38 ± 0.02ms 1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false ops: 1093.41 ± 3.11 ops/s. best: 1086.93ops/s. total: 914.58 ± 2.61ms. best: 911.49ms. avg: 0.91 ± 0.00ms min: 0.05 ± 0.00ms max: 2.96 ± 0.02ms p99: 2.32 ± 0.01ms
random_route_mld	10000 routes, 3 coordinates, no alternatives, overview=full, steps=true total: 39584.30ms avg: 3.96ms min: 0.11ms max: 12.88ms p99: 6.67ms 10000 routes, 2 coordinates, no alternatives, overview=full, steps=true total: 14248.83ms avg: 1.42ms min: 0.06ms max: 3.74ms p99: 2.52ms 10000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true total: 41555.11ms avg: 4.16ms min: 0.06ms max: 13.81ms p99: 8.42ms 10000 routes, 3 coordinates, no alternatives, overview=false, steps=false total: 29677.57ms avg: 2.97ms min: 0.11ms max: 9.16ms p99: 5.19ms 10000 routes, 2 coordinates, no alternatives, overview=false, steps=false total: 9044.71ms avg: 0.90ms min: 0.05ms max: 2.80ms p99: 1.62ms 10000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false total: 33009.41ms avg: 3.30ms min: 0.05ms max: 15.40ms p99: 6.52ms 10000 routes, 3 coordinates, no alternatives, overview=false, steps=false, radius=750 total: 835.55ms avg: 0.08ms min: 0.01ms max: 4.40ms p99: 1.56ms 10000 routes, 2 coordinates, no alternatives, overview=false, steps=false, radius=750 total: 901.20ms avg: 0.09ms min: 0.01ms max: 1.20ms p99: 0.98ms 10000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false, radius=750 total: 1714.47ms avg: 0.17ms min: 0.01ms max: 5.08ms p99: 3.53ms	1000 routes, 3 coordinates, no alternatives, overview=full, steps=true ops: 234.67 ± 4.91 ops/s. best: 222.30ops/s. total: 4196.37 ± 91.99ms. best: 4126.09ms. avg: 4.26 ± 0.09ms min: 0.38 ± 0.02ms max: 9.56 ± 0.33ms p99: 7.10 ± 0.20ms 1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true ops: 236.11 ± 0.70 ops/s. best: 235.02ops/s. total: 4235.33 ± 12.57ms. best: 4217.97ms. avg: 4.24 ± 0.01ms min: 0.06 ± 0.00ms max: 10.16 ± 1.42ms p99: 8.38 ± 0.05ms 1000 routes, 3 coordinates, no alternatives, overview=false, steps=false ops: 318.69 ± 2.61 ops/s. best: 313.88ops/s. total: 3087.87 ± 25.55ms. best: 3049.77ms. avg: 3.14 ± 0.03ms min: 0.30 ± 0.00ms max: 8.79 ± 1.91ms p99: 5.54 ± 0.21ms 1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false ops: 297.43 ± 2.80 ops/s. best: 294.39ops/s. total: 3362.59 ± 31.75ms. best: 3292.65ms. avg: 3.36 ± 0.03ms min: 0.04 ± 0.00ms max: 7.76 ± 0.55ms p99: 6.66 ± 0.16ms
random_table_ch	250 tables, 3 coordinates total: 178.92ms avg: 0.72ms min: 0.48ms max: 1.82ms p99: 0.97ms 250 tables, 25 coordinates total: 1492.13ms avg: 5.97ms min: 5.35ms max: 7.97ms p99: 7.76ms 250 tables, 50 coordinates total: 3009.75ms avg: 12.04ms min: 11.07ms max: 20.12ms p99: 18.85ms 250 tables, 100 coordinates total: 6273.96ms avg: 25.10ms min: 23.61ms max: 27.72ms p99: 26.45ms	250 tables, 3 coordinates ops: 1446.54 ± 9.83 ops/s. best: 1420.64ops/s. total: 172.84 ± 1.19ms. best: 172.01ms. avg: 0.69 ± 0.00ms min: 0.48 ± 0.00ms max: 1.05 ± 0.30ms p99: 0.86 ± 0.04ms 250 tables, 25 coordinates ops: 170.86 ± 1.16 ops/s. best: 167.98ops/s. total: 1463.28 ± 10.13ms. best: 1454.20ms. avg: 5.85 ± 0.04ms min: 5.22 ± 0.01ms max: 8.02 ± 1.53ms p99: 7.00 ± 1.08ms 250 tables, 50 coordinates ops: 84.57 ± 0.34 ops/s. best: 83.74ops/s. total: 2956.24 ± 11.83ms. best: 2945.94ms. avg: 11.82 ± 0.05ms min: 11.00 ± 0.01ms max: 14.96 ± 2.92ms p99: 13.48 ± 1.36ms
random_table_mld	250 tables, 3 coordinates total: 727.72ms avg: 2.91ms min: 2.31ms max: 4.06ms p99: 3.88ms 250 tables, 25 coordinates total: 6540.88ms avg: 26.16ms min: 23.92ms max: 32.50ms p99: 29.02ms 250 tables, 50 coordinates total: 14126.00ms avg: 56.50ms min: 52.19ms max: 61.40ms p99: 60.16ms 250 tables, 100 coordinates total: 30989.03ms avg: 123.96ms min: 119.22ms max: 148.19ms p99: 131.11ms	250 tables, 3 coordinates ops: 341.38 ± 0.90 ops/s. best: 340.00ops/s. total: 732.33 ± 1.93ms. best: 729.26ms. avg: 2.93 ± 0.01ms min: 2.33 ± 0.01ms max: 4.02 ± 0.02ms p99: 3.85 ± 0.03ms 250 tables, 25 coordinates ops: 35.97 ± 0.23 ops/s. best: 35.50ops/s. total: 6950.53 ± 44.59ms. best: 6891.96ms. avg: 27.80 ± 0.18ms min: 24.98 ± 0.16ms max: 33.64 ± 2.53ms p99: 31.27 ± 0.62ms 250 tables, 50 coordinates ops: 16.86 ± 0.13 ops/s. best: 16.62ops/s. total: 14826.83 ± 115.74ms. best: 14685.85ms. avg: 59.31 ± 0.46ms min: 55.22 ± 0.29ms max: 70.57 ± 6.33ms p99: 64.27 ± 1.62ms
random_trip_ch	1000 trips, 3 coordinates total: 2122.92ms avg: 2.12ms min: 0.71ms max: 3.86ms p99: 2.73ms 1000 trips, 4 coordinates total: 2670.49ms avg: 2.67ms min: 1.15ms max: 3.47ms p99: 3.34ms 1000 trips, 5 coordinates total: 3195.15ms avg: 3.20ms min: 1.84ms max: 4.22ms p99: 3.90ms	250 trips, 3 coordinates ops: 477.24 ± 3.85 ops/s. best: 468.84ops/s. total: 523.89 ± 4.28ms. best: 518.10ms. avg: 2.10 ± 0.02ms min: 1.09 ± 0.01ms max: 3.01 ± 0.36ms p99: 2.70 ± 0.05ms 250 trips, 5 coordinates ops: 318.77 ± 0.41 ops/s. best: 317.89ops/s. total: 784.27 ± 1.01ms. best: 782.84ms. avg: 3.14 ± 0.00ms min: 2.12 ± 0.01ms max: 3.90 ± 0.03ms p99: 3.83 ± 0.01ms
random_trip_mld	1000 trips, 3 coordinates total: 5791.62ms avg: 5.79ms min: 2.72ms max: 14.08ms p99: 7.74ms 1000 trips, 4 coordinates total: 7349.33ms avg: 7.35ms min: 3.76ms max: 9.95ms p99: 9.12ms 1000 trips, 5 coordinates total: 8909.18ms avg: 8.91ms min: 5.58ms max: 11.70ms p99: 10.78ms	250 trips, 3 coordinates ops: 167.35 ± 3.79 ops/s. best: 161.16ops/s. total: 1494.82 ± 34.21ms. best: 1454.17ms. avg: 5.98 ± 0.14ms min: 3.87 ± 0.01ms max: 8.00 ± 0.34ms p99: 7.76 ± 0.30ms 250 trips, 5 coordinates ops: 109.19 ± 1.16 ops/s. best: 107.64ops/s. total: 2289.99 ± 23.10ms. best: 2248.04ms. avg: 9.16 ± 0.09ms min: 6.40 ± 0.04ms max: 11.75 ± 0.48ms p99: 11.18 ± 0.28ms
route_ch	1000 routes, 3 coordinates, no alternatives, overview=full, steps=true 461.351ms 0.461351ms/req 1000 routes, 2 coordinates, no alternatives, overview=full, steps=true 304.921ms 0.304921ms/req 1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true 559.358ms 0.559358ms/req 1000 routes, 3 coordinates, no alternatives, overview=false, steps=false 154.369ms 0.154369ms/req 1000 routes, 2 coordinates, no alternatives, overview=false, steps=false 98.5122ms 0.0985122ms/req 1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false 135.715ms 0.135715ms/req 1000 routes, 3 coordinates, no alternatives, overview=false, steps=false, radius=750 154.4ms 0.1544ms/req 1000 routes, 2 coordinates, no alternatives, overview=false, steps=false, radius=750 98.3972ms 0.0983972ms/req 1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false, radius=750 134.848ms 0.134848ms/req	1000 routes, 3 coordinates, no alternatives, overview=full, steps=true 495.373ms 0.495373ms/req 1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true 604.06ms 0.60406ms/req 1000 routes, 3 coordinates, no alternatives, overview=false, steps=false 163.442ms 0.163442ms/req 1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false 145.86ms 0.14586ms/req
route_mld	1000 routes, 3 coordinates, no alternatives, overview=full, steps=true 594.821ms 0.594821ms/req 1000 routes, 2 coordinates, no alternatives, overview=full, steps=true 398.752ms 0.398752ms/req 1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true 763.317ms 0.763317ms/req 1000 routes, 3 coordinates, no alternatives, overview=false, steps=false 286.142ms 0.286142ms/req 1000 routes, 2 coordinates, no alternatives, overview=false, steps=false 155.623ms 0.155623ms/req 1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false 308.771ms 0.308771ms/req 1000 routes, 3 coordinates, no alternatives, overview=false, steps=false, radius=750 283.26ms 0.28326ms/req 1000 routes, 2 coordinates, no alternatives, overview=false, steps=false, radius=750 155.769ms 0.155769ms/req 1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false, radius=750 309.304ms 0.309304ms/req	1000 routes, 3 coordinates, no alternatives, overview=full, steps=true 621.399ms 0.621399ms/req 1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true 811.974ms 0.811974ms/req 1000 routes, 3 coordinates, no alternatives, overview=false, steps=false 273.712ms 0.273712ms/req 1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false 298.655ms 0.298655ms/req
rtree	1 result: 209.515ms -> 0.0209515 ms/query 10 results: 246.859ms -> 0.0246859 ms/query	1 result: 207.243ms -> 0.0207243 ms/query 10 results: 242.812ms -> 0.0242812 ms/query