[Perf] Windows/x64: 100 Regressions on 5/19/2023 3:32:16 PM #17994

performanceautofiler · 2023-05-23T05:14:00Z

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
List - Duration of single invocation	11.80 μs	17.27 μs	1.46	0.07	True
ICollection - Duration of single invocation	12.69 μs	17.72 μs	1.40	0.03	True
Array - Duration of single invocation	12.95 μs	18.67 μs	1.44	0.04	True
Span - Duration of single invocation	11.05 μs	16.18 μs	1.46	0.00	True

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<Int32>.List(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.26676033591731 > 12.439909789830628.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -86.77500710983452 (T) = (0 -17268.48695932136) / Math.Sqrt((120978.29701821266 / (36)) + (1356.7108946769276 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.42166436278236646 = (12146.669362608818 - 17268.48695932136) / 12146.669362608818 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrue<Int32>.ICollection(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.717707158962796 > 13.316165011123905.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -105.69285471439198 (T) = (0 -17709.981708071697) / Math.Sqrt((53591.05154843198 / (36)) + (5657.877241076083 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.3644349782619606 = (12979.718337792072 - 17709.981708071697) / 12979.718337792072 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrue<Int32>.Array(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.66617084826762 > 13.988946821762648.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -113.75021344727318 (T) = (0 -18574.730433558838) / Math.Sqrt((64066.045827955204 / (34)) + (5101.177756196429 / (11))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (11) - 2, .025) and -0.421957845069172 = (13062.785579733736 - 18574.730433558838) / 13062.785579733736 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrue<Int32>.Span(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.18176326936376 > 11.661948094252176.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -139.94661903914326 (T) = (0 -16080.624636613893) / Math.Sqrt((22555.978292029104 / (36)) + (7878.588839935464 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.46822754955375556 = (10952.406281643021 - 16080.624636613893) / 10952.406281643021 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_HashCode

Benchmark	Baseline	Test	Test/Base	Edge Detector
Combine_5 - Duration of single invocation	96.69 μs	110.90 μs	1.15	True
Combine_3 - Duration of single invocation	86.30 μs	105.84 μs	1.23	True
Combine_4 - Duration of single invocation	76.17 μs	96.05 μs	1.26	True
Combine_1 - Duration of single invocation	58.45 μs	77.94 μs	1.33	True
Combine_2 - Duration of single invocation	72.39 μs	91.89 μs	1.27	True
Combine_6 - Duration of single invocation	108.59 μs	123.34 μs	1.14	True

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_HashCode*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_HashCode.Combine_5

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 110.89576051165147 > 101.45840801886793.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -424.42353802991835 (T) = (0 -110901.72390405103) / Math.Sqrt((35064.62038682523 / (35)) + (1159.49642294999 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.14592942564458566 = (96778.84293936232 - 110901.72390405103) / 96778.84293936232 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_HashCode.Combine_3

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 105.84230444386695 > 90.64137684604759.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -1461.9650551161437 (T) = (0 -105893.31132268632) / Math.Sqrt((798.4362108671548 / (36)) + (1726.6312510297107 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.22667308876295955 = (86325.61706352799 - 105893.31132268632) / 86325.61706352799 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_HashCode.Combine_4

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 96.05094325153374 > 80.0786290958738.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -651.1588558119554 (T) = (0 -96073.40610182252) / Math.Sqrt((30368.200004328413 / (35)) + (720.5226259625829 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.2611042594369195 = (76181.96939935727 - 96073.40610182252) / 76181.96939935727 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_HashCode.Combine_1

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 77.93597014925372 > 61.394557486007464.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -2307.442054512058 (T) = (0 -77962.71964457598) / Math.Sqrt((232.1936856732065 / (36)) + (714.1438843655179 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.3334026607235099 = (58468.99960607029 - 77962.71964457598) / 58468.99960607029 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_HashCode.Combine_2

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 91.89288744343892 > 76.0131236255787.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -2157.783958825201 (T) = (0 -91887.25776551165) / Math.Sqrt((547.8023933766647 / (36)) + (729.7528939552269 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.2691533592370463 = (72400.43695014906 - 91887.25776551165) / 72400.43695014906 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_HashCode.Combine_6

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 123.34208333333335 > 114.01688313802083.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -178.67373744209118 (T) = (0 -124021.8661256358) / Math.Sqrt((1649.6126945473984 / (36)) + (81334.61100368263 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.14189666011141489 = (108610.41148288759 - 124021.8661256358) / 108610.41148288759 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Char>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
IndexOfAnyThreeValues - Duration of single invocation	14.13 ns	16.93 ns	1.20	0.03	False
SequenceEqual - Duration of single invocation	15.89 ns	20.54 ns	1.29	0.10	False
IndexOfAnyTwoValues - Duration of single invocation	11.89 ns	15.20 ns	1.28	0.03	False
LastIndexOfValue - Duration of single invocation	9.93 ns	15.03 ns	1.51	0.10	True

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.IndexOfAnyThreeValues(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.92964290554546 > 14.838862054288155.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 4/19/2023 8:38:27 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -113.08395291490251 (T) = (0 -16.919073516146486) / Math.Sqrt((0.020411996217386458 / (34)) + (3.138997315495306E-05 / (11))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (11) - 2, .025) and -0.19639584366111765 = (14.141702017596494 - 16.919073516146486) / 14.141702017596494 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.SequenceEqual(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.53598976571988 > 16.69734151532932.
IsChangePoint: Marked as a change because one of 3/3/2023 11:43:56 AM, 3/29/2023 12:02:44 PM, 4/19/2023 8:38:27 PM, 5/9/2023 7:00:05 PM, 5/16/2023 5:40:18 PM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -30.55832482434292 (T) = (0 -20.369248688979884) / Math.Sqrt((0.4012049304677225 / (24)) + (0.05813476786962719 / (23))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (23) - 2, .025) and -0.26281278469364583 = (16.13006214054239 - 20.369248688979884) / 16.13006214054239 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.IndexOfAnyTwoValues(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.198106703964061 > 12.467150094090082.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 4/4/2023 8:16:00 PM, 4/6/2023 1:55:21 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -18.71160801459279 (T) = (0 -15.664633494316313) / Math.Sqrt((0.26349205495593536 / (35)) + (0.26625923722333517 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.27030923766171266 = (12.331354468578503 - 15.664633494316313) / 12.331354468578503 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.LastIndexOfValue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.034664567127335 > 10.450002426419863.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -14.470481429296132 (T) = (0 -14.508311877830522) / Math.Sqrt((0.029148312756134154 / (36)) + (1.0949523132768257 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.46189410635425776 = (9.92432476112244 - 14.508311877830522) / 9.92432476112244 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
BinarySearch - Duration of single invocation	5.64 ns	7.29 ns	1.29	0.10	False
StartsWith - Duration of single invocation	15.30 ns	21.24 ns	1.39	0.14	False
LastIndexOfValue - Duration of single invocation	17.51 ns	20.08 ns	1.15	0.03	False
BinarySearch - Duration of single invocation	12.19 ns	13.69 ns	1.12	0.05	False
IndexOfAnyFiveValues - Duration of single invocation	33.46 ns	49.34 ns	1.47	0.14	False
IndexOfAnyFourValues - Duration of single invocation	29.84 ns	36.58 ns	1.23	0.21	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.BinarySearch(Size: 4)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.2907181314952965 > 5.923116799915878.
IsChangePoint: Marked as a change because one of 4/19/2023 8:38:27 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -7.0630634563813315 (T) = (0 -6.8146056163686834) / Math.Sqrt((0.02611501073424612 / (36)) + (0.31082356691744295 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.2142527702400282 = (5.612180415303151 - 6.8146056163686834) / 5.612180415303151 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.StartsWith(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.236517576461363 > 16.077262542338808.
IsChangePoint: Marked as a change because one of 3/29/2023 12:02:44 PM, 4/9/2023 11:04:55 PM, 4/12/2023 8:39:35 PM, 5/5/2023 3:18:41 PM, 5/9/2023 1:22:51 PM, 5/16/2023 5:40:18 PM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -9.81713902114575 (T) = (0 -21.43047951538953) / Math.Sqrt((5.490723291789415 / (24)) + (0.3681066890852158 / (23))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (23) - 2, .025) and -0.2930664450384157 = (16.57337841966261 - 21.43047951538953) / 16.57337841966261 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.LastIndexOfValue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.07994168412729 > 18.373182717495958.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 1:57:48 PM, 5/19/2023 3:32:16 PM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -19.931617156266785 (T) = (0 -19.94861607930257) / Math.Sqrt((0.09454348510726432 / (35)) + (0.14042806441506445 / (10))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (10) - 2, .025) and -0.14848607986291865 = (17.369488780990366 - 19.94861607930257) / 17.369488780990366 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.BinarySearch(Size: 33)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.688515028988311 > 12.718221453217263.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -10.82207175538672 (T) = (0 -14.624340625858292) / Math.Sqrt((0.06763424569931516 / (36)) + (0.4979277364772434 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.1914352235524659 = (12.274557891828431 - 14.624340625858292) / 12.274557891828431 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.IndexOfAnyFiveValues(Size: 33)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 49.34275302339757 > 34.58990514931742.
IsChangePoint: Marked as a change because one of 4/19/2023 1:39:34 AM, 4/19/2023 8:38:27 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -12.732400365103391 (T) = (0 -46.20533258610484) / Math.Sqrt((0.9709324053819218 / (35)) + (10.738895171850062 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.38142996262768536 = (33.447466636828565 - 46.20533258610484) / 33.447466636828565 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 33)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.582489593709255 > 34.75811278502811.
IsChangePoint: Marked as a change because one of 4/22/2023 6:52:03 AM, 4/25/2023 9:59:52 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -9.093726350937773 (T) = (0 -36.17240036969219) / Math.Sqrt((14.949812964680369 / (36)) + (0.2181728391800804 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.19879598335866916 = (30.173941914910245 - 36.17240036969219) / 30.173941914910245 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

performanceautofiler · 2023-05-23T05:14:03Z

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
SerializeToUtf8Bytes - Duration of single invocation	11.14 μs	15.13 μs	1.36	0.36	False
SerializeObjectProperty - Duration of single invocation	12.40 μs	16.51 μs	1.33	0.24	False
SerializeToStream - Duration of single invocation	10.94 μs	16.06 μs	1.47	0.30	False
SerializeToString - Duration of single invocation	12.23 μs	16.45 μs	1.35	0.25	False
SerializeToWriter - Duration of single invocation	10.04 μs	13.82 μs	1.38	0.30	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.13358332754308 > 11.382016451111388.
IsChangePoint: Marked as a change because one of 4/17/2023 11:29:15 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -31.823667801498345 (T) = (0 -15388.827624336549) / Math.Sqrt((170548.96111794934 / (36)) + (163271.97710265598 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.4072043434468904 = (10935.74483052137 - 15388.827624336549) / 10935.74483052137 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeObjectProperty(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.512235257475087 > 12.589232679621931.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -21.863171052722617 (T) = (0 -16432.254885043152) / Math.Sqrt((202870.52839810145 / (34)) + (340844.0544747123 / (11))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (11) - 2, .025) and -0.34365884435253363 = (12229.484406781345 - 16432.254885043152) / 12229.484406781345 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.06487872176591 > 11.779065092427983.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -31.35336418423422 (T) = (0 -15691.882998552499) / Math.Sqrt((193288.89809170505 / (34)) + (169320.5458830041 / (11))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (11) - 2, .025) and -0.4086138827849868 = (11139.946290695287 - 15691.882998552499) / 11139.946290695287 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToString(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.452783400809714 > 12.045666364862518.
IsChangePoint: Marked as a change because one of 4/17/2023 11:29:15 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -30.706437736355863 (T) = (0 -16091.719255588976) / Math.Sqrt((180994.54239712656 / (36)) + (202866.95464538987 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.41310261383506586 = (11387.509369837713 - 16091.719255588976) / 11387.509369837713 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToWriter(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.823383525243582 > 10.864599876340007.
IsChangePoint: Marked as a change because one of 4/17/2023 7:48:00 AM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -27.4893648055806 (T) = (0 -15116.061636073226) / Math.Sqrt((231147.2351075417 / (36)) + (237552.5841044789 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.43759032765617895 = (10514.860419740147 - 15116.061636073226) / 10514.860419740147 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in BenchmarksGame.ReverseComplement_1

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
RunBench - Duration of single invocation	460.33 μs	583.48 μs	1.27	0.50	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BenchmarksGame.ReverseComplement_1*'

Payloads

Baseline
Compare

Histogram

BenchmarksGame.ReverseComplement_1.RunBench

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 583.4797364672365 > 448.17233605238977.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -27.08468968791916 (T) = (0 -584794.0794159545) / Math.Sqrt((1161628318.9045773 / (35)) + (4212444.741121477 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.36678531930959 = (427860.96042599727 - 584794.0794159545) / 427860.96042599727 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SerializeToWriter - Duration of single invocation	7.08 μs	10.11 μs	1.43	0.23	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToWriter(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.112696848290598 > 7.42753940054223.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -8.82710716943464 (T) = (0 -9071.036742445136) / Math.Sqrt((412711.5126014737 / (36)) + (326612.7968572838 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.24597351456279573 = (7280.280548843051 - 9071.036742445136) / 7280.280548843051 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Get

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
GetGuid - Duration of single invocation	4.52 μs	11.82 μs	2.61	0.02	True
GetByte - Duration of single invocation	627.98 ns	1.41 μs	2.24	0.00	True
GetString - Duration of single invocation	3.05 μs	3.26 μs	1.07	0.01	False
GetUInt64 - Duration of single invocation	621.41 ns	711.59 ns	1.15	0.02	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Get.GetGuid

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.8162285840262 > 4.75266484251315.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -27.45831314977342 (T) = (0 -11236.50479678782) / Math.Sqrt((2947.927189906697 / (36)) + (648452.7843622875 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -1.4613997431935761 = (4565.0873361223585 - 11236.50479678782) / 4565.0873361223585 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetByte

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4083349973590302 > 659.6318169946792.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -1746.4051414837845 (T) = (0 -1407.5686627530145) / Math.Sqrt((0.37375237991405524 / (36)) + (2.0781939653975714 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -1.2416960812704416 = (627.9034319207534 - 1407.5686627530145) / 627.9034319207534 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetString

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.262550829341991 > 3.179800493021845.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -12.581704956263671 (T) = (0 -3249.0110473001027) / Math.Sqrt((9863.962480453869 / (36)) + (478.8299543601909 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.07411951737913458 = (3024.813342213287 - 3249.0110473001027) / 3024.813342213287 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetUInt64

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 711.5895648344597 > 652.9629298453265.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -143.51831654453952 (T) = (0 -714.4480823889135) / Math.Sqrt((7.556371959960399 / (35)) + (1.9422990575067125 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.14396315795189618 = (624.5376675137261 - 714.4480823889135) / 624.5376675137261 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SequenceCompareToDifferent - Duration of single invocation	5.26 ns	6.31 ns	1.20	0.01	False
IndexOfAnyFiveValues - Duration of single invocation	14.50 ns	16.88 ns	1.16	0.01	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.SequenceCompareToDifferent(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.312776158995189 > 5.5213566724762995.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 4/25/2023 6:46:56 PM, 5/4/2023 10:11:14 AM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -21.538907705180172 (T) = (0 -6.05122088792769) / Math.Sqrt((0.0068282713863926795 / (36)) + (0.015690381005491163 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.16698450601759712 = (5.185348097360637 - 6.05122088792769) / 5.185348097360637 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfAnyFiveValues(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.879760568627635 > 15.204835099094481.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 4/19/2023 8:38:27 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -17.792440203731655 (T) = (0 -16.55268984752416) / Math.Sqrt((0.09370464658458359 / (36)) + (0.08864954331320306 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.1248472356037212 = (14.71550031292925 - 16.55268984752416) / 14.71550031292925 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-05-23T05:14:10Z

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_String

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
TrimStart - Duration of single invocation	9.86 ns	13.20 ns	1.34	0.06	False
TrimStart_CharArr - Duration of single invocation	11.90 ns	14.14 ns	1.19	0.04	False
CtorCharCount - Duration of single invocation	8.22 ns	9.55 ns	1.16	0.08	False
Remove_IntInt - Duration of single invocation	12.06 ns	13.27 ns	1.10	0.03	False
Trim_CharArr - Duration of single invocation	13.84 ns	15.66 ns	1.13	0.07	False
Trim - Duration of single invocation	14.01 ns	17.33 ns	1.24	0.05	False
Trim - Duration of single invocation	12.96 ns	16.45 ns	1.27	0.05	False
TrimEnd_CharArr - Duration of single invocation	13.58 ns	14.74 ns	1.09	0.11	False
Trim_CharArr - Duration of single invocation	13.12 ns	15.90 ns	1.21	0.03	False
IndexerCheckPathLength - Duration of single invocation	123.46 ns	146.93 ns	1.19	0.06	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.TrimStart(s: " Test")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.200704308649891 > 10.638138180164972.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -21.834809257339423 (T) = (0 -13.323210659767591) / Math.Sqrt((0.6159902302533001 / (34)) + (0.024929481298602795 / (11))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (11) - 2, .025) and -0.30545172300106704 = (10.20582410289308 - 13.323210659767591) / 10.20582410289308 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.TrimStart_CharArr(s: " Test", c: [' ', ' '])

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.143545398278574 > 12.71544809316802.
IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 4/17/2023 6:40:18 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -36.89505082584341 (T) = (0 -14.079802851770829) / Math.Sqrt((0.08815692328792635 / (34)) + (0.004044811289854276 / (11))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (11) - 2, .025) and -0.16628953118498416 = (12.072304925403333 - 14.079802851770829) / 12.072304925403333 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.CtorCharCount(size: 1)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.545277590487762 > 9.259646789024131.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -4.375311426762168 (T) = (0 -9.81593440062914) / Math.Sqrt((0.12051633319940874 / (36)) + (0.8083533252753817 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.14097093955987258 = (8.603141465124096 - 9.81593440062914) / 8.603141465124096 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 7, i2: 4)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.272768311318803 > 12.89915160323769.
IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -11.157759825272663 (T) = (0 -13.703676168749967) / Math.Sqrt((0.06947969532898317 / (36)) + (0.146956491258229 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.11195005021990319 = (12.324003372309646 - 13.703676168749967) / 12.324003372309646 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Trim_CharArr(s: "Test ", c: [' ', ' '])

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.662763048840672 > 14.861488413080306.
IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 4/17/2023 6:40:18 PM, 4/25/2023 1:57:48 PM, 5/2/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -20.434897770521754 (T) = (0 -15.579997793931968) / Math.Sqrt((0.20622558679955663 / (35)) + (0.024626746393726896 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.134134470803975 = (13.737346139287597 - 15.579997793931968) / 13.737346139287597 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Trim(s: " Te st ")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.325326164108525 > 15.020688274294903.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/4/2023 4:23:54 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -17.437602302739506 (T) = (0 -17.084059137567184) / Math.Sqrt((0.6967605445974735 / (36)) + (0.020793346834967898 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.17477360122023666 = (14.542426830005358 - 17.084059137567184) / 14.542426830005358 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Trim(s: "Test ")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.44569421450896 > 13.819438902222965.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/21/2023 10:36:00 PM, 4/4/2023 4:23:54 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -20.707125852854357 (T) = (0 -16.381238231423) / Math.Sqrt((0.6573587953741246 / (35)) + (0.015021551500215063 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.21865660798316627 = (13.442046039969679 - 16.381238231423) / 13.442046039969679 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.TrimEnd_CharArr(s: "Test ", c: [' ', ' '])

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.740921681747519 > 14.124001282079787.
IsChangePoint: Marked as a change because one of 3/25/2023 2:21:39 PM, 3/26/2023 1:36:36 AM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -12.348524983160601 (T) = (0 -14.565405901017236) / Math.Sqrt((0.48533486868531867 / (34)) + (0.0549109321156556 / (11))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (11) - 2, .025) and -0.13337243403919521 = (12.851385355392827 - 14.565405901017236) / 12.851385355392827 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Trim_CharArr(s: " Test", c: [' ', ' '])

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.901408698237097 > 14.04041766023009.
IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 4/17/2023 6:40:18 PM, 4/25/2023 1:57:48 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -42.74499000492254 (T) = (0 -15.633439311911207) / Math.Sqrt((0.05303095207002123 / (36)) + (0.016953480895448216 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.17662995973238185 = (13.286623532403466 - 15.633439311911207) / 13.286623532403466 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.IndexerCheckPathLength

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 146.93121515844825 > 129.60780191372908.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 5/9/2023 1:22:51 PM, 5/16/2023 12:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -212.91065367111312 (T) = (0 -146.658053420386) / Math.Sqrt((0.187122243886619 / (21)) + (0.07878669486745718 / (26))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (26) - 2, .025) and -0.18854992049043184 = (123.39242205314392 - 146.658053420386) / 123.39242205314392 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Utf8Encoding

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetBytes - Duration of single invocation	92.50 μs	108.45 μs	1.17	0.09	False
GetString - Duration of single invocation	176.12 μs	186.39 μs	1.06	0.01	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Utf8Encoding.GetBytes(Input: Chinese)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 108.45099753694582 > 95.96944295058141.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/6/2023 11:12:53 AM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -16.216053377782387 (T) = (0 -110046.74480094534) / Math.Sqrt((16325722.701345239 / (36)) + (7595232.767391533 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.18709407832631736 = (92702.63141746956 - 110046.74480094534) / 92702.63141746956 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetString(Input: Cyrillic)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 186.39436813186813 > 184.65470575842696.
IsChangePoint: Marked as a change because one of 5/1/2023 3:42:23 PM, 5/6/2023 11:12:53 AM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -23.440443062482963 (T) = (0 -186272.50327301666) / Math.Sqrt((7428154.9318784205 / (36)) + (334705.5330618154 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.06522555320695363 = (174866.72443430143 - 186272.50327301666) / 174866.72443430143 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_UInt32

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
TryFormat - Duration of single invocation	4.39 ns	6.03 ns	1.37	0.03	False
TryParseHex - Duration of single invocation	9.82 ns	11.49 ns	1.17	0.02	False
TryFormat - Duration of single invocation	7.02 ns	9.11 ns	1.30	0.02	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt32.TryFormat(value: 12345)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.031775157879428 > 4.605680995080933.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -29.61018325851917 (T) = (0 -5.9591645289276265) / Math.Sqrt((0.009504180386061635 / (36)) + (0.025184903209004343 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.3352716396040178 = (4.462885567385262 - 5.9591645289276265) / 4.462885567385262 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.TryParseHex(value: "3039")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.492387880254501 > 10.305175439213851.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -95.61814631516819 (T) = (0 -11.450348201807484) / Math.Sqrt((0.005421395090340535 / (36)) + (0.0017674503895222533 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.17278675071685218 = (9.763367632528755 - 11.450348201807484) / 9.763367632528755 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.TryFormat(value: 4294967295)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.11057999154967 > 7.339131783507776.
IsChangePoint: Marked as a change because one of 5/1/2023 3:42:23 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -32.837698811821845 (T) = (0 -8.965259877967712) / Math.Sqrt((0.08981571486102512 / (36)) + (0.020649176017052844 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.3195896659104384 = (6.793975513427665 - 8.965259877967712) / 6.793975513427665 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
BitArrayCopyToByteArray - Duration of single invocation	94.42 ns	121.20 ns	1.28	0.07	True
BitArrayByteArrayCtor - Duration of single invocation	10.10 ns	13.75 ns	1.36	0.08	False
BitArrayIntArrayCtor - Duration of single invocation	14.29 ns	15.82 ns	1.11	0.07	False
BitArrayLeftShift - Duration of single invocation	143.46 ns	151.95 ns	1.06	0.00	True

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArrayCopyToByteArray(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 121.1976428268472 > 99.19713391305552.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -96.38609440227934 (T) = (0 -121.9475763496905) / Math.Sqrt((0.15268451007097372 / (34)) + (0.8292977768170345 / (11))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (11) - 2, .025) and -0.2876479300776375 = (94.70568274228327 - 121.9475763496905) / 94.70568274228327 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayByteArrayCtor(Size: 4)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.751186934234116 > 10.244971563433905.
IsChangePoint: Marked as a change because one of 3/6/2023 8:17:12 PM, 5/5/2023 3:18:41 PM, 5/10/2023 6:05:58 AM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -16.198408695577427 (T) = (0 -13.172876963826297) / Math.Sqrt((0.13887364309437272 / (36)) + (0.32633262077122477 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.2905715027272702 = (10.207010565465781 - 13.172876963826297) / 10.207010565465781 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayIntArrayCtor(Size: 4)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.821694551883793 > 14.570822926244821.
IsChangePoint: Marked as a change because one of 4/5/2023 6:15:03 PM, 4/6/2023 9:09:21 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -10.379818858229253 (T) = (0 -16.432405673722855) / Math.Sqrt((0.3869278234008287 / (36)) + (0.3665537120496994 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.15287951502073424 = (14.253359054113579 - 16.432405673722855) / 14.253359054113579 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayLeftShift(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 151.9501496334039 > 150.628881155551.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -115.51402031509578 (T) = (0 -151.9739687565354) / Math.Sqrt((0.14817550547224495 / (36)) + (0.008533632717603202 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.0561465042163811 = (143.8947798906877 - 151.9739687565354) / 143.8947798906877 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
EncodeUtf16 - Duration of single invocation	55.69 ns	58.94 ns	1.06	0.01	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Payloads

Baseline
Compare

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,&lorem ipsum=dolor sit amet,16)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 58.940062941377235 > 57.61116619742747.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -18.59738545336607 (T) = (0 -58.951808338853105) / Math.Sqrt((0.44354936366961817 / (34)) + (0.2859164476331918 / (11))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (11) - 2, .025) and -0.06647348655922482 = (55.277331393441365 - 58.951808338853105) / 55.277331393441365 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-05-23T05:14:12Z

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.ReadOnlySequence

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Slice_Start_And_Length - Duration of single invocation	10.04 ns	12.62 ns	1.26	0.04	True
Slice_Start_And_Length - Duration of single invocation	5.03 ns	6.27 ns	1.25	0.04	False
Slice_Repeat - Duration of single invocation	46.70 ns	61.17 ns	1.31	0.01	True

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySequence*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySequence.Slice_Start_And_Length(Segment: Multiple)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.623822415362667 > 10.545031560753616.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -64.09450720193517 (T) = (0 -12.773879985176935) / Math.Sqrt((0.017154509265285188 / (35)) + (0.01590169066926252 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.2833023318288403 = (9.953913172566917 - 12.773879985176935) / 9.953913172566917 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySequence.Slice_Start_And_Length(Segment: Single)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.269162247869717 > 5.28351209734677.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -12.633060682950179 (T) = (0 -6.072775245904013) / Math.Sqrt((0.021297706733732553 / (36)) + (0.044832318248273655 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.1656631092206085 = (5.209717282692786 - 6.072775245904013) / 5.209717282692786 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySequence.Slice_Repeat(Segment: Multiple)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 61.16926066904937 > 48.99585907897534.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -111.20979205296433 (T) = (0 -61.29055375819971) / Math.Sqrt((0.3180565276615822 / (36)) + (0.09524404054728851 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.31576652138769284 = (46.581633414383205 - 61.29055375819971) / 46.581633414383205 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Parse - Duration of single invocation	1.52 μs	1.73 μs	1.14	0.02	False
ToByteArray - Duration of single invocation	46.49 ns	50.73 ns	1.09	0.31	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.Parse(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.725997904544639 > 1.592842052261791.
IsChangePoint: Marked as a change because one of 4/25/2023 1:57:48 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -8.961658780445225 (T) = (0 -1679.2505732229974) / Math.Sqrt((998.5847191659759 / (34)) + (2257.9292213058407 / (11))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (11) - 2, .025) and -0.08902421614237532 = (1541.9772566411489 - 1679.2505732229974) / 1541.9772566411489 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_BigInteger.ToByteArray(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 50.73186669301123 > 49.63495949064831.
IsChangePoint: Marked as a change because one of 4/7/2023 2:04:47 AM, 4/11/2023 10:53:42 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -9.269477429433733 (T) = (0 -51.62253518712435) / Math.Sqrt((1.8760666296190944 / (35)) + (2.8221971031914963 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.11111466756123912 = (46.46013295858069 - 51.62253518712435) / 46.46013295858069 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Enum

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ToString_NonFlags_Large - Duration of single invocation	10.58 ns	12.36 ns	1.17	0.12	False
ToString_NonFlags_Large - Duration of single invocation	10.68 ns	12.95 ns	1.21	0.12	False
ToString_NonFlags_Large - Duration of single invocation	10.60 ns	12.41 ns	1.17	0.13	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.ToString_NonFlags_Large(value: Control)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.363525151350022 > 11.260088704832858.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/16/2023 8:14:39 PM, 3/31/2023 8:05:26 PM, 4/5/2023 6:15:03 PM, 4/9/2023 11:04:55 PM, 4/17/2023 6:40:18 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -9.089892426000498 (T) = (0 -11.789517048214732) / Math.Sqrt((0.11493791617779216 / (35)) + (0.1331555299259624 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.1057619392891445 = (10.661894418064161 - 11.789517048214732) / 10.661894418064161 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_NonFlags_Large(value: Format)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.953831765499721 > 11.256308258345168.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/31/2023 8:05:26 PM, 4/5/2023 6:15:03 PM, 4/9/2023 11:04:55 PM, 4/17/2023 6:40:18 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -11.889707337858095 (T) = (0 -12.133290788906208) / Math.Sqrt((0.16852003560183548 / (35)) + (0.10256941848133205 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.13188987878466502 = (10.719497555657966 - 12.133290788906208) / 10.719497555657966 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_NonFlags_Large(value: UppercaseLetter)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.406946794733793 > 11.268134596161946.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/31/2023 8:05:26 PM, 4/5/2023 6:15:03 PM, 4/9/2023 11:04:55 PM, 4/17/2023 6:40:18 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -10.917234773202294 (T) = (0 -11.89320884305452) / Math.Sqrt((0.10964505395013277 / (36)) + (0.10517737394023777 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.11491183213791772 = (10.66739853343246 - 11.89320884305452) / 10.66739853343246 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.BenchF.Whetsto

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	545.56 ms	588.75 ms	1.08	0.00	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.Whetsto*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchF.Whetsto.Test

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 588.7522769230769 > 573.1807865.
IsChangePoint: Marked as a change because one of 3/26/2023 5:20:11 AM, 4/6/2023 1:55:21 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -15.97774641176929 (T) = (0 -588701388.5081584) / Math.Sqrt((88999578710026.25 / (36)) + (17142685767003.828 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.05762893649446206 = (556623753.5627799 - 588701388.5081584) / 556623753.5627799 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Count - Duration of single invocation	22.97 μs	31.38 μs	1.37	0.04	False
Count - Duration of single invocation	23.24 μs	41.91 μs	1.80	0.08	False
Count - Duration of single invocation	29.01 μs	46.56 μs	1.61	0.04	False
Count - Duration of single invocation	28.89 μs	46.64 μs	1.61	0.06	False
Count - Duration of single invocation	22.85 μs	31.37 μs	1.37	0.03	False
Count - Duration of single invocation	29.51 μs	47.07 μs	1.60	0.12	False
Count - Duration of single invocation	23.01 μs	31.41 μs	1.37	0.06	False
Count - Duration of single invocation	5.32 ms	6.97 ms	1.31	0.17	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.378060704742815 > 24.088310423516283.
IsChangePoint: Marked as a change because one of 3/10/2023 12:53:36 AM, 3/14/2023 8:42:09 PM, 3/20/2023 8:31:51 AM, 4/25/2023 3:46:39 AM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -909.305446548811 (T) = (0 -31343.093638575847) / Math.Sqrt((2413.6810980342225 / (35)) + (188.64557776901887 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.36839187482689667 = (22905.056815351807 - 31343.093638575847) / 22905.056815351807 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.90589030384272 > 24.403640131081946.
IsChangePoint: Marked as a change because one of 3/10/2023 12:53:36 AM, 3/14/2023 8:42:09 PM, 3/20/2023 8:31:51 AM, 4/10/2023 1:33:42 PM, 4/18/2023 7:42:00 PM, 5/1/2023 3:42:23 PM, 5/5/2023 3:18:41 PM, 5/9/2023 1:22:51 PM, 5/16/2023 5:40:18 PM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -1959.8878090434062 (T) = (0 -41937.80382486636) / Math.Sqrt((199.3106926067191 / (24)) + (1902.8198564048623 / (23))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (23) - 2, .025) and -0.8047097451854052 = (23237.977152141946 - 41937.80382486636) / 23237.977152141946 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 46.56226872519842 > 30.46107102272728.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/20/2023 8:31:51 AM, 4/10/2023 1:33:42 PM, 4/25/2023 3:46:39 AM, 5/5/2023 3:18:41 PM, 5/9/2023 1:22:51 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -327.04568720637656 (T) = (0 -46406.890563935965) / Math.Sqrt((64197.346601844685 / (36)) + (11630.268610614934 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.6015421560261623 = (28976.37779269163 - 46406.890563935965) / 28976.37779269163 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 46.63956039186508 > 30.45414471243043.
IsChangePoint: Marked as a change because one of 3/10/2023 12:53:36 AM, 3/14/2023 8:42:09 PM, 3/20/2023 8:31:51 AM, 5/1/2023 3:42:23 PM, 5/5/2023 3:18:41 PM, 5/9/2023 1:22:51 PM, 5/16/2023 5:40:18 PM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -22.73027661348872 (T) = (0 -46785.64005749909) / Math.Sqrt((13330075.664648924 / (24)) + (24077.874051180544 / (22))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (22) - 2, .025) and -0.5684678714021038 = (29828.880087722744 - 46785.64005749909) / 29828.880087722744 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: None)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.367859886439543 > 24.089446585903087.
IsChangePoint: Marked as a change because one of 3/10/2023 12:53:36 AM, 4/25/2023 3:46:39 AM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -906.0229688489045 (T) = (0 -31353.7612832359) / Math.Sqrt((766.6694780583043 / (34)) + (701.0848684346896 / (11))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (11) - 2, .025) and -0.36690456658194404 = (22937.783697393395 - 31353.7612832359) / 22937.783697393395 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 47.07428463855421 > 31.09019627754831.
IsChangePoint: Marked as a change because one of 3/10/2023 12:53:36 AM, 3/14/2023 8:42:09 PM, 3/20/2023 8:31:51 AM, 4/10/2023 1:33:42 PM, 4/18/2023 7:42:00 PM, 5/2/2023 12:34:35 AM, 5/16/2023 5:40:18 PM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -12.517438825918635 (T) = (0 -47230.10924968848) / Math.Sqrt((36066658.360915944 / (24)) + (26901.728714359055 / (23))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (23) - 2, .025) and -0.48152984289330053 = (31879.283077721968 - 47230.10924968848) / 31879.283077721968 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.410707831325297 > 24.15747412683824.
IsChangePoint: Marked as a change because one of 3/10/2023 12:53:36 AM, 3/20/2023 8:31:51 AM, 3/25/2023 6:47:18 AM, 4/25/2023 3:46:39 AM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -940.9533745483557 (T) = (0 -31394.433118685658) / Math.Sqrt((622.2941077750725 / (36)) + (685.8253793124502 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.36511901036392586 = (22997.579610525125 - 31394.433118685658) / 22997.579610525125 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-zA-Z]+ing", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.966698333333334 > 5.642191701896392.
IsChangePoint: Marked as a change because one of 3/24/2023 2:55:56 PM, 4/9/2023 11:04:55 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -27.335524602450892 (T) = (0 -7022258.282828283) / Math.Sqrt((80240563615.04079 / (36)) + (14358616222.373774 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.301098324043337 = (5397177.2563703535 - 7022258.282828283) / 5397177.2563703535 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Count - Duration of single invocation	4.31 ms	4.73 ms	1.10	0.01	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Tom|Sawyer|Huckleberry|Finn", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.733728333333333 > 4.532689.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -15.486931742179156 (T) = (0 -4697714.197469197) / Math.Sqrt((769913794.3264085 / (35)) + (6772343849.480897 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.0908078497104966 = (4306637.689411553 - 4697714.197469197) / 4306637.689411553 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-05-23T05:14:14Z

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ImmutableArray - Duration of single invocation	18.94 μs	21.39 μs	1.13	0.21	False
ICollection - Duration of single invocation	17.85 μs	20.40 μs	1.14	0.01	True
List - Duration of single invocation	16.61 μs	20.40 μs	1.23	0.02	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<Int32>.ImmutableArray(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.393624601003197 > 19.61657502644875.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -32.04259232953253 (T) = (0 -21394.74694382545) / Math.Sqrt((188266.03993022983 / (36)) + (34.89749193004331 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.12150311384957767 = (19076.850237524202 - 21394.74694382545) / 19076.850237524202 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsFalse<Int32>.ICollection(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.39954906440382 > 18.756119163911062.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -66.31632340972878 (T) = (0 -20496.910859876672) / Math.Sqrt((39398.50660481132 / (36)) + (4130.276552709118 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.14160895829001455 = (17954.40611343699 - 20496.910859876672) / 17954.40611343699 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsFalse<Int32>.List(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.39705124020888 > 17.434940444627088.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -13.615696739381121 (T) = (0 -20145.664865503095) / Math.Sqrt((14932.756187647705 / (34)) + (711839.6913103869 / (11))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (11) - 2, .025) and -0.20847818404017374 = (16670.275997993016 - 20145.664865503095) / 16670.275997993016 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in LinqBenchmarks

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Count00ForX - Duration of single invocation	101.81 ms	123.94 ms	1.22	0.00	True

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'

Payloads

Baseline
Compare

Histogram

LinqBenchmarks.Count00ForX

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 123.9424 > 106.9041789375.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -74.67602816082078 (T) = (0 -124018347.83882785) / Math.Sqrt((2633650699291.9434 / (35)) + (27707913096.714497 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.20180076307002878 = (103193766.92857143 - 124018347.83882785) / 103193766.92857143 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.BenchF.Trap

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	131.30 ms	142.03 ms	1.08	0.00	True

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.Trap*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchF.Trap.Test

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 142.03180714285713 > 138.2693930625.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -95.54696462551239 (T) = (0 -142244253.4681985) / Math.Sqrt((73983832421.82773 / (35)) + (111611713685.99838 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.0803524132672104 = (131664678.78571428 - 142244253.4681985) / 131664678.78571428 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Globalization.Tests.StringSearch

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
IsPrefix_FirstHalf - Duration of single invocation	11.48 ns	12.64 ns	1.10	0.01	False
LastIndexOf_Word_NotFound - Duration of single invocation	458.94 ns	577.93 ns	1.26	0.02	False
IsSuffix_SecondHalf - Duration of single invocation	11.39 ns	14.46 ns	1.27	0.08	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, Ordinal, False))

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.638692659249632 > 12.03901122307721.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -20.173377420054486 (T) = (0 -12.642008331681565) / Math.Sqrt((0.05092914990232646 / (36)) + (0.03559149850743519 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.12210581421021133 = (11.266324593977423 - 12.642008331681565) / 11.266324593977423 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False))

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 577.9310954063604 > 482.13528878687464.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -20.422265055230664 (T) = (0 -578.1293911496094) / Math.Sqrt((849.3842144148304 / (36)) + (0.04429739756761201 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.2071460056971799 = (478.9225068227884 - 578.1293911496094) / 478.9225068227884 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, Ordinal, False))

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.455516717685047 > 11.95738678151914.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -12.50309948527115 (T) = (0 -14.324010667366013) / Math.Sqrt((1.4106175799928677 / (35)) + (0.016748999358461664 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.21730858280983628 = (11.766951182010732 - 14.324010667366013) / 11.766951182010732 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.CtorDefaultSize<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	19.11 ns	22.90 ns	1.20	0.14	False
List - Duration of single invocation	10.85 ns	11.90 ns	1.10	0.08	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorDefaultSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorDefaultSize<String>.Dictionary

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.901499315679434 > 20.231788761282143.
IsChangePoint: Marked as a change because one of 3/2/2023 1:15:31 AM, 3/2/2023 11:15:04 AM, 4/9/2023 7:17:29 PM, 4/25/2023 1:57:48 PM, 5/3/2023 11:54:24 PM, 5/5/2023 9:43:39 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -12.66103659471125 (T) = (0 -23.189848086098326) / Math.Sqrt((0.106401511674418 / (35)) + (1.022903646801376 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.2036463502790644 = (19.266330247852103 - 23.189848086098326) / 19.266330247852103 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorDefaultSize<String>.List

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.899690074513689 > 11.558784983981411.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/9/2023 11:04:55 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -16.87966463969552 (T) = (0 -11.681723095932034) / Math.Sqrt((0.03524689945821217 / (36)) + (0.007019271450281091 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.06169319864160887 = (11.0029178965056 - 11.681723095932034) / 11.0029178965056 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Random

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
NextSingle - Duration of single invocation	5.51 ns	10.28 ns	1.87	0.02	True

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Random*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Random.NextSingle

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.277675896707205 > 5.797831795620796.
IsChangePoint: Marked as a change because one of 4/25/2023 1:57:48 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -734.2926831807051 (T) = (0 -10.255555999973655) / Math.Sqrt((0.00017220360114348577 / (36)) + (0.0004026606355803684 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.8540146970720147 = (5.53153975325542 - 10.255555999973655) / 5.53153975325542 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Runtime.InteropServices.Tests.SafeHandleTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
AddRef_GetHandle_Release - Duration of single invocation	11.35 ns	12.53 ns	1.10	0.05	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.InteropServices.Tests.SafeHandleTests*'

Payloads

Baseline
Compare

Histogram

System.Runtime.InteropServices.Tests.SafeHandleTests.AddRef_GetHandle_Release

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.526611646794949 > 11.938363677406574.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -59.42651382092841 (T) = (0 -12.559562936547279) / Math.Sqrt((0.013695551092980094 / (36)) + (0.00014658740029925495 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.10361906601372396 = (11.380342478054919 - 12.559562936547279) / 11.380342478054919 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Ascii

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
FromUtf16 - Duration of single invocation	15.23 ns	16.53 ns	1.09	0.10	True

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Ascii*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Ascii.FromUtf16(Size: 128)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.528041875503785 > 15.992053995268005.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -13.71371683696431 (T) = (0 -17.111453616288145) / Math.Sqrt((1.9098822285086303 / (35)) + (0.09543499810908487 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.25242296723154845 = (13.66267951322596 - 17.111453616288145) / 13.66267951322596 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ctor_string - Duration of single invocation	23.33 ns	26.20 ns	1.12	0.04	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_StringBuilder.ctor_string(length: 100)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.19664384508619 > 25.994330203924395.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/17/2023 1:30:52 PM, 3/20/2023 8:31:51 AM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -11.463907224051743 (T) = (0 -26.329062868418518) / Math.Sqrt((0.8573746685993155 / (36)) + (0.08711768802949536 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.08408845713045483 = (24.286821518338687 - 26.329062868418518) / 24.286821518338687 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.BenchF.InvMt

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	2.22 ms	3.14 ms	1.41	0.02	True

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.InvMt*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchF.InvMt.Test

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.144514761904762 > 2.341147064732143.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -32.576841372636366 (T) = (0 -3155709.370941559) / Math.Sqrt((3440166100.194197 / (34)) + (6953269957.870692 / (11))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (11) - 2, .025) and -0.3880119183324313 = (2273546.3069602842 - 3155709.370941559) / 2273546.3069602842 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.BlockCopyPerf

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
CallBlockCopy - Duration of single invocation	6.41 ns	7.87 ns	1.23	0.02	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.BlockCopyPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.BlockCopyPerf.CallBlockCopy(numElements: 10)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.873114414487551 > 6.736376669007747.
IsChangePoint: Marked as a change because one of 3/21/2023 12:39:01 PM, 5/16/2023 5:40:18 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -23.399612296885437 (T) = (0 -7.87141970408957) / Math.Sqrt((0.09117764847270715 / (36)) + (0.006628878012636959 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.19969578022032575 = (6.5611797872990545 - 7.87141970408957) / 6.5611797872990545 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-05-23T05:14:15Z

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_UInt64

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryParseHex - Duration of single invocation	6.06 ns	7.45 ns	1.23	0.01	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.TryParseHex(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.447627670212659 > 6.360015358710146.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -779.8495247581623 (T) = (0 -7.4490103294170815) / Math.Sqrt((8.35780925379187E-05 / (34)) + (7.471877873106035E-06 / (11))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (11) - 2, .025) and -0.2276544531583494 = (6.0676767067909365 - 7.4490103294170815) / 6.0676767067909365 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.BenchI.BubbleSort

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	13.25 μs	14.75 μs	1.11	0.06	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.BubbleSort*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.BubbleSort.Test

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.748966921205568 > 13.916632735373858.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 3/23/2023 12:23:24 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -16.85152117820739 (T) = (0 -14914.175978718251) / Math.Sqrt((45540.60990964404 / (35)) + (71813.31640715184 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.11108576992681546 = (13423.064521563094 - 14914.175978718251) / 13423.064521563094 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Text.Tests.Base64EncodeDecodeInPlaceTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Base64DecodeInPlace - Duration of single invocation	190.14 ms	350.66 ms	1.84	0.02	True
Base64EncodeInPlace - Duration of single invocation	138.85 ms	371.98 ms	2.68	0.00	True

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Base64EncodeDecodeInPlaceTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Base64EncodeDecodeInPlaceTests.Base64DecodeInPlace(NumberOfBytes: 200000000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 350.6560894736842 > 199.26713625.
IsChangePoint: Marked as a change because one of 5/9/2023 9:55:32 AM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -35.037837829805504 (T) = (0 -358219202.3444977) / Math.Sqrt((395965226149553.5 / (36)) + (152125902085986.9 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.950751585198942 = (183631378.31717598 - 358219202.3444977) / 183631378.31717598 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Base64EncodeDecodeInPlaceTests.Base64EncodeInPlace(NumberOfBytes: 200000000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 371.9764 > 145.70822700000002.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -5528.856101012597 (T) = (0 -372002426.7066267) / Math.Sqrt((4549727939.995043 / (34)) + (18102030754.428307 / (11))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (11) - 2, .025) and -1.6806065822680245 = (138775465.660418 - 372002426.7066267) / 138775465.660418 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.LowLevelPerf

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
StaticDelegate - Duration of single invocation	250.59 μs	278.61 μs	1.11	0.01	True

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.LowLevelPerf.StaticDelegate

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 278.6052558479532 > 263.18379761904765.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -843.9790274131028 (T) = (0 -278535.5988912403) / Math.Sqrt((7301.367260899878 / (35)) + (9703.00740822576 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.11119757049215799 = (250662.53408732233 - 278535.5988912403) / 250662.53408732233 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Float

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetHashCodeBenchmark - Duration of single invocation	14.35 ns	15.68 ns	1.09	0.03	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Float*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GetHashCodeBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.681490525713906 > 15.040623978496853.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 1:22:51 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -11.38000597897063 (T) = (0 -15.471140568549783) / Math.Sqrt((0.05366519119398824 / (36)) + (0.05895682230646476 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.06482725412647038 = (14.529249236057085 - 15.471140568549783) / 14.529249236057085 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Reflection.Activator<EmptyClass>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
CreateInstanceType - Duration of single invocation	9.01 ns	11.19 ns	1.24	0.14	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Activator&lt;EmptyClass&gt;*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Activator<EmptyClass>.CreateInstanceType

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.192079165404119 > 9.524256138857426.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/9/2023 11:04:55 PM, 4/17/2023 6:40:18 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -28.551385558276962 (T) = (0 -10.940905576599517) / Math.Sqrt((0.07635381665583121 / (35)) + (0.021905613605461265 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.2027558275194771 = (9.096530921960836 - 10.940905576599517) / 9.096530921960836 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Uri

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
EscapeDataString - Duration of single invocation	10.57 μs	12.51 μs	1.18	0.01	True

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Uri*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Uri.EscapeDataString(input: "a{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{üa{ü")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.514916302278541 > 11.156996802245555.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -13.804463275184107 (T) = (0 -12538.591387621855) / Math.Sqrt((53687.19866637545 / (36)) + (257876.93082614505 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.2104326970195386 = (10358.767917039719 - 12538.591387621855) / 10358.767917039719 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.MathBenchmarks.Single

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Asinh - Duration of single invocation	88.87 μs	94.78 μs	1.07	0.01	True

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Single*'

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Single.Asinh

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 94.7771561771562 > 93.43822230113636.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -77.93540352971148 (T) = (0 -94815.28345644254) / Math.Sqrt((194736.1044136468 / (35)) + (918.3352478033387 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.06583707501911572 = (88958.51502889598 - 94815.28345644254) / 88958.51502889598 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IterateForEach<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ImmutableHashSet - Duration of single invocation	17.49 μs	20.49 μs	1.17	0.08	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<Int32>.ImmutableHashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.486638742882175 > 18.412267862535693.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -16.819995008749682 (T) = (0 -20983.11939157122) / Math.Sqrt((4476.66443533617 / (36)) + (446396.90647039795 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.19292980265464524 = (17589.567588031718 - 20983.11939157122) / 17589.567588031718 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TrimEnd - Duration of single invocation	7.67 ns	9.84 ns	1.28	0.03	True

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Primitives.StringSegmentBenchmark.TrimEnd

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.83946338727325 > 8.11117708405195.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -35.74395707337658 (T) = (0 -9.685432994814574) / Math.Sqrt((0.02236278917946311 / (35)) + (0.0277218931125111 / (11))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (11) - 2, .025) and -0.2617138910127003 = (7.676409892769486 - 9.685432994814574) / 7.676409892769486 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Ctor_Span - Duration of single invocation	11.04 ns	12.36 ns	1.12	0.08	False
Ctor_Bytes - Duration of single invocation	10.93 ns	12.54 ns	1.15	0.05	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*'

Payloads

Baseline
Compare

Histogram

System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Span(address: [16, 32, 48, 64, 80, ...])

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.357086942251025 > 11.711064946349595.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/16/2023 8:14:39 PM, 4/26/2023 4:28:25 PM, 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -19.781962631393842 (T) = (0 -12.384593743157302) / Math.Sqrt((0.15333947887884078 / (36)) + (0.022605865743949526 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.14538089154690573 = (10.812642182663938 - 12.384593743157302) / 10.812642182663938 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Bytes(address: [16, 32, 48, 64, 80, ...])

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.53595660131922 > 11.220335940299336.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/16/2023 8:14:39 PM, 3/25/2023 12:18:49 AM, 3/31/2023 8:05:26 PM, 5/2/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 2:06:57 AM falls between 5/13/2023 6:35:24 AM and 5/22/2023 2:06:57 AM.
IsRegressionStdDev: Marked as regression because -25.823590799167594 (T) = (0 -12.373797756958936) / Math.Sqrt((0.09120713594387084 / (36)) + (0.016037401429216724 / (11))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (11) - 2, .025) and -0.15187479961003433 = (10.742311370253146 - 12.373797756958936) / 10.742311370253146 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler bot assigned EgorBo May 23, 2023

performanceautofiler bot added arch-x64 branch-refs/heads/main kind-micro os-windows perf-regression runtime-coreclr untriaged labels May 23, 2023

AndyAyersMS mentioned this issue Jun 6, 2023

Dynamic PGO Microbenchmark Regressions dotnet/runtime#87194

Closed

[Perf] Windows/x64: 100 Regressions on 5/19/2023 3:32:16 PM #17994

[Perf] Windows/x64: 100 Regressions on 5/19/2023 3:32:16 PM #17994

Comments

performanceautofiler bot commented May 23, 2023 • edited Loading

Run Information

Regressions in System.Collections.ContainsTrue<Int32>

Repro

Payloads

Payloads

Histogram

System.Collections.ContainsTrue<Int32>.List(Size: 512)

Description of detection logic

JIT Disasms

System.Collections.ContainsTrue<Int32>.ICollection(Size: 512)

Description of detection logic

JIT Disasms

System.Collections.ContainsTrue<Int32>.Array(Size: 512)

Description of detection logic

JIT Disasms

System.Collections.ContainsTrue<Int32>.Span(Size: 512)

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.Tests.Perf_HashCode

Repro

Payloads

Payloads

Histogram

System.Tests.Perf_HashCode.Combine_5

Description of detection logic

JIT Disasms

System.Tests.Perf_HashCode.Combine_3

Description of detection logic

JIT Disasms

System.Tests.Perf_HashCode.Combine_4

Description of detection logic

JIT Disasms

System.Tests.Perf_HashCode.Combine_1

Description of detection logic

JIT Disasms

System.Tests.Perf_HashCode.Combine_2

Description of detection logic

JIT Disasms

System.Tests.Perf_HashCode.Combine_6

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.Memory.Span<Char>

Repro

Payloads

Payloads

Histogram

System.Memory.Span<Char>.IndexOfAnyThreeValues(Size: 512)

Description of detection logic

JIT Disasms

System.Memory.Span<Char>.SequenceEqual(Size: 512)

Description of detection logic

JIT Disasms

System.Memory.Span<Char>.IndexOfAnyTwoValues(Size: 512)

Description of detection logic

JIT Disasms

System.Memory.Span<Char>.LastIndexOfValue(Size: 512)

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.Memory.Span<Int32>

Repro

Payloads

Payloads

Histogram

System.Memory.Span<Int32>.BinarySearch(Size: 4)

Description of detection logic

JIT Disasms

System.Memory.Span<Int32>.StartsWith(Size: 512)

Description of detection logic

JIT Disasms

System.Memory.Span<Int32>.LastIndexOfValue(Size: 512)

performanceautofiler bot commented May 23, 2023 •

edited

Loading

performanceautofiler bot commented May 23, 2023 •

edited

Loading