[Perf] Windows/arm64: 83 Regressions on 6/8/2023 3:52:42 PM #18852

performanceautofiler · 2023-06-15T08:46:49Z

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
TryParseUInt64 - Duration of single invocation	26.51 ns	29.58 ns	1.12	0.01	False
TryParseDateTimeOffset - Duration of single invocation	4.29 ns	6.50 ns	1.52	0.29	False
TryParseInt64 - Duration of single invocation	2.73 ns	4.22 ns	1.55	0.43	False
TryParseDecimal - Duration of single invocation	36.28 ns	45.71 ns	1.26	0.02	False
TryParseUInt16 - Duration of single invocation	1.78 ns	2.79 ns	1.57	0.57	False
TryParseUInt32 - Duration of single invocation	7.06 ns	8.12 ns	1.15	0.15	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value: 18446744073709551615)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.57514584987845 > 28.24336856683063.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -32.53849434952265 (T) = (0 -29.42411845996003) / Math.Sqrt((0.03837008363438911 / (12)) + (0.03648330145112784 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.09371648944070958 = (26.90287541975942 - 29.42411845996003) / 26.90287541975942 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseDateTimeOffset(value: 12/30/2017 3:45:22 AM -08:00)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.499294486842408 > 4.498855472332188.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -24.632217580733972 (T) = (0 -6.660764811897213) / Math.Sqrt((0.05703100663844913 / (12)) + (0.06716527185134046 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.5830613207946008 = (4.2075216698200375 - 6.660764811897213) / 4.2075216698200375 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: 12345)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.217932597285427 > 3.0333266009413267.
IsChangePoint: Marked as a change because one of 4/28/2023 1:33:49 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -14.810869511086683 (T) = (0 -4.0111446471294165) / Math.Sqrt((0.0387852753582227 / (12)) + (0.029887244533084653 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.3785719234350773 = (2.9096375596672437 - 4.0111446471294165) / 2.9096375596672437 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseDecimal(value: 123456.789)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 45.7103609648865 > 38.70256030332696.
IsChangePoint: Marked as a change because one of 3/30/2023 2:02:58 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -27.097436637882208 (T) = (0 -45.01909670610535) / Math.Sqrt((0.633425518261271 / (12)) + (0.42690647834215983 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.21378188669854264 = (37.08993946890755 - 45.01909670610535) / 37.08993946890755 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt16(value: 0)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.791028431745073 > 2.006919215636941.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -11.633139268856493 (T) = (0 -3.041553889173428) / Math.Sqrt((0.04165784167647684 / (12)) + (0.04949562194910929 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.48436586091791983 = (2.0490594463635508 - 3.041553889173428) / 2.0490594463635508 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32(value: 4294967295)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.116825501907527 > 7.41833122266938.
IsChangePoint: Marked as a change because one of 4/26/2023 5:34:01 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -6.16887819152795 (T) = (0 -8.139057244249786) / Math.Sqrt((0.30282833194281794 / (12)) + (0.020754704836957055 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.14175308095486558 = (7.128561665402267 - 8.139057244249786) / 7.128561665402267 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Double

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ToStringWithCultureInfo - Duration of single invocation	104.55 ns	116.81 ns	1.12	0.02	False
ToStringWithFormat - Duration of single invocation	115.57 ns	138.47 ns	1.20	0.01	False
ToStringWithFormat - Duration of single invocation	193.47 ns	205.60 ns	1.06	0.01	False
ToStringWithFormat - Duration of single invocation	115.30 ns	130.63 ns	1.13	0.01	False
ToStringWithFormat - Duration of single invocation	116.24 ns	133.90 ns	1.15	0.01	False
ToStringWithFormat - Duration of single invocation	108.88 ns	120.83 ns	1.11	0.01	False
ToString - Duration of single invocation	195.86 ns	208.15 ns	1.06	0.01	False
ToString - Duration of single invocation	193.23 ns	210.76 ns	1.09	0.00	False
ToStringWithFormat - Duration of single invocation	166.29 ns	182.08 ns	1.09	0.01	False
ToString - Duration of single invocation	107.75 ns	120.44 ns	1.12	0.01	False
ToStringWithFormat - Duration of single invocation	108.12 ns	121.42 ns	1.12	0.01	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.ToStringWithCultureInfo(value: 12345, culture: zh)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 116.80720542191257 > 108.71921567709721.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -34.1927597376135 (T) = (0 -116.52264152716673) / Math.Sqrt((1.0303283234000291 / (12)) + (0.603786540335472 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.11949060127093981 = (104.08541295021185 - 116.52264152716673) / 104.08541295021185 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "E")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 138.4676168160732 > 122.01003324569348.
IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -35.99698372731682 (T) = (0 -134.15899592334875) / Math.Sqrt((1.143601894689089 / (12)) + (1.7784710729572 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.14845825797332882 = (116.81660608205091 - 134.15899592334875) / 116.81660608205091 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 205.60452783239361 > 205.09955420096986.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -23.86608476943617 (T) = (0 -206.14775944296784) / Math.Sqrt((1.5749504164622152 / (12)) + (1.0431520399735017 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.05623494823248566 = (195.17225763826278 - 206.14775944296784) / 195.17225763826278 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "E")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 130.63410714285712 > 121.0593899211226.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -64.07609421349449 (T) = (0 -131.7691941104574) / Math.Sqrt((0.562022732179779 / (12)) + (0.2996185683101651 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.1475109107643386 = (114.83044986708497 - 131.7691941104574) / 114.83044986708497 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "E")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 133.898790058862 > 122.65790810682634.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -83.02566640793171 (T) = (0 -134.02461371871712) / Math.Sqrt((0.4489452371807319 / (12)) + (0.062326612469853764 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.1458263726192868 = (116.96764616470234 - 134.02461371871712) / 116.96764616470234 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "G")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 120.83141846956089 > 113.42344653876793.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -34.438051914022516 (T) = (0 -121.17414102740699) / Math.Sqrt((0.9190409521657333 / (12)) + (0.7996247342723737 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.11808493905208606 = (108.37650771875936 - 121.17414102740699) / 108.37650771875936 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToString(value: -1.7976931348623157E+308)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 208.14857496396615 > 205.83354405146903.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -30.680546114740476 (T) = (0 -208.07847870696017) / Math.Sqrt((1.203201043482124 / (12)) + (0.441387019180788 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.057103316075121424 = (196.83835585676414 - 208.07847870696017) / 196.83835585676414 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToString(value: 1.7976931348623157E+308)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 210.7552370652299 > 203.5268495971351.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -24.557724941237186 (T) = (0 -206.63460063792354) / Math.Sqrt((0.7763773583529353 / (12)) + (2.4029453724717187 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.06311544600084604 = (194.36703832610772 - 206.63460063792354) / 194.36703832610772 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G17")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 182.08184121488085 > 175.00749894322834.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -34.74714639940897 (T) = (0 -181.67580418755864) / Math.Sqrt((0.4105684839004004 / (12)) + (1.9315006045988632 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.08905298440721897 = (166.8199865284299 - 181.67580418755864) / 166.8199865284299 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToString(value: 12345)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 120.43639826836635 > 114.04921270629187.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -36.62490690055699 (T) = (0 -120.55803475517662) / Math.Sqrt((0.9788256174793966 / (12)) + (0.3678460057635083 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.11197021394984692 = (108.41840297766657 - 120.55803475517662) / 108.41840297766657 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "R")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 121.41930410043405 > 113.93745142291195.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/12/2023 9:12:45 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -74.14918734351902 (T) = (0 -121.53407920688383) / Math.Sqrt((0.21202273768367377 / (12)) + (0.15592319895647563 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.11741627971453941 = (108.7634764350592 - 121.53407920688383) / 108.7634764350592 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

performanceautofiler · 2023-06-15T08:46:52Z

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Single

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ToString - Duration of single invocation	143.37 ns	157.09 ns	1.10	0.01	False
ToStringWithFormat - Duration of single invocation	144.06 ns	157.97 ns	1.10	0.01	False
ToStringWithFormat - Duration of single invocation	146.09 ns	159.76 ns	1.09	0.01	False
ToStringWithFormat - Duration of single invocation	109.18 ns	121.45 ns	1.11	0.01	False
ToString - Duration of single invocation	148.55 ns	159.05 ns	1.07	0.01	False
ToStringWithCultureInfo - Duration of single invocation	142.63 ns	151.67 ns	1.06	0.01	False
ToString - Duration of single invocation	109.95 ns	120.50 ns	1.10	0.01	False
ToStringWithFormat - Duration of single invocation	145.00 ns	156.39 ns	1.08	0.01	False
ToStringWithCultureInfo - Duration of single invocation	143.36 ns	154.62 ns	1.08	0.01	False
ToStringWithFormat - Duration of single invocation	109.87 ns	121.10 ns	1.10	0.01	False
ToStringWithFormat - Duration of single invocation	148.08 ns	159.53 ns	1.08	0.01	False
ToStringWithCultureInfo - Duration of single invocation	102.42 ns	114.86 ns	1.12	0.01	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Single.ToString(value: 3.4028235E+38)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 157.08663114276615 > 151.42090106665606.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/12/2023 9:12:45 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -44.02286340238594 (T) = (0 -157.22829373009014) / Math.Sqrt((0.4805611680388116 / (12)) + (0.5464178463494488 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.08721229618233116 = (144.61600028088915 - 157.22829373009014) / 144.61600028088915 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 157.96871445999875 > 151.82889188044584.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/12/2023 9:12:45 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -38.10517430096889 (T) = (0 -157.83576319351567) / Math.Sqrt((0.484912359534822 / (12)) + (0.9421829059716914 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.08827445319946113 = (145.0330500081924 - 157.83576319351567) / 145.0330500081924 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "R")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 159.7550715688621 > 153.571883539265.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/12/2023 9:12:45 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -43.46005968910126 (T) = (0 -159.3778742869819) / Math.Sqrt((0.581040696108174 / (12)) + (0.4625676956948726 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.08581504693246338 = (146.78178824030888 - 159.3778742869819) / 146.78178824030888 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "R")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 121.44913605559873 > 114.27068470920108.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/12/2023 9:12:45 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -46.624861705583236 (T) = (0 -121.24043059782319) / Math.Sqrt((0.5106338893477148 / (12)) + (0.36880010803624 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.11409979420370496 = (108.82367201627476 - 121.24043059782319) / 108.82367201627476 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToString(value: -3.4028235E+38)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 159.04502328925568 > 153.9936479408777.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -23.71356419443588 (T) = (0 -159.14463289161648) / Math.Sqrt((2.4300537674326215 / (12)) + (0.9898002110956643 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.08537029988984465 = (146.6270386316709 - 159.14463289161648) / 146.6270386316709 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithCultureInfo(value: 3.4028235E+38, culture: zh)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 151.6677083753377 > 147.68591861555387.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -33.72902587162347 (T) = (0 -152.79223252712893) / Math.Sqrt((1.0678539599983257 / (12)) + (0.6143007483448817 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.0887093852013818 = (140.3425327309606 - 152.79223252712893) / 140.3425327309606 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToString(value: 12345)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 120.49653282340354 > 115.21013212648592.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -35.58303338065645 (T) = (0 -120.72229471518372) / Math.Sqrt((0.8484565473862571 / (12)) + (0.46169592967090617 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.10627234084569503 = (109.12529424978392 - 120.72229471518372) / 109.12529424978392 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "R")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 156.38837364447278 > 151.36954278937156.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/12/2023 9:12:45 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -46.69228536490276 (T) = (0 -157.14931216107453) / Math.Sqrt((0.5826505884594407 / (12)) + (0.3390022363277427 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.0883400836079505 = (144.39357194316474 - 157.14931216107453) / 144.39357194316474 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithCultureInfo(value: -3.4028235E+38, culture: zh)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 154.6232821316979 > 150.01175322007694.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -26.16064766460413 (T) = (0 -154.46129588777143) / Math.Sqrt((1.487577052804786 / (12)) + (0.791786099951662 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.07854175852279253 = (143.2130881045597 - 154.46129588777143) / 143.2130881045597 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "G")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 121.10476729023809 > 114.79241939028873.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -36.59096195948437 (T) = (0 -120.86218444840648) / Math.Sqrt((0.822941141561209 / (12)) + (0.4682288167381133 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.10854070342602963 = (109.02818820713816 - 120.86218444840648) / 109.02818820713816 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 159.5349674750702 > 155.47976068407104.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/12/2023 9:12:45 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -43.83650545923254 (T) = (0 -159.89263345165057) / Math.Sqrt((0.684679031223823 / (12)) + (0.31545846590789794 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.08481418890051774 = (147.39172393541898 - 159.89263345165057) / 147.39172393541898 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithCultureInfo(value: 12345, culture: zh)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 114.86430464606097 > 109.87303847867892.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -19.629231266489047 (T) = (0 -116.38933138512651) / Math.Sqrt((3.1998624988489124 / (12)) + (0.6543535658465781 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.10491613467852147 = (105.3377063943322 - 116.38933138512651) / 105.3377063943322 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Ascii

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
FromUtf16 - Duration of single invocation	13.16 ns	14.58 ns	1.11	0.13	False
FromUtf16 - Duration of single invocation	5.15 ns	6.58 ns	1.28	0.18	False
ToUtf16 - Duration of single invocation	5.07 ns	6.55 ns	1.29	0.18	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Ascii*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Ascii.FromUtf16(Size: 128)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.583458941646201 > 13.892613240389057.
IsChangePoint: Marked as a change because one of 5/12/2023 2:01:50 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -10.07934224224609 (T) = (0 -14.585402931216025) / Math.Sqrt((0.08134589302865369 / (12)) + (0.07545467536246639 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.08403279343501711 = (13.454761718968564 - 14.585402931216025) / 13.454761718968564 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.FromUtf16(Size: 6)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.579692081904745 > 5.340649662153314.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -51.61750898558922 (T) = (0 -6.645600971417181) / Math.Sqrt((0.0029364157961079885 / (12)) + (0.00873327019850644 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.3074292252031072 = (5.082952746742216 - 6.645600971417181) / 5.082952746742216 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.ToUtf16(Size: 6)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.552539197822115 > 5.2927231587930805.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -15.26309168911853 (T) = (0 -6.5080361726043545) / Math.Sqrt((0.12892456513752093 / (12)) + (0.006781433524514633 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.33142152804162006 = (4.888035859069356 - 6.5080361726043545) / 4.888035859069356 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Get

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
GetByte - Duration of single invocation	824.46 ns	1.14 μs	1.39	0.01	False
GetDecimal - Duration of single invocation	4.44 μs	4.96 μs	1.12	0.01	False
GetInt64 - Duration of single invocation	737.24 ns	831.25 ns	1.13	0.01	False
GetUInt64 - Duration of single invocation	689.95 ns	779.54 ns	1.13	0.01	False
GetUInt32 - Duration of single invocation	591.24 ns	752.88 ns	1.27	0.01	False
GetUInt16 - Duration of single invocation	835.73 ns	1.15 μs	1.38	0.01	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Get.GetByte

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1439860586536326 > 876.1907162317733.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -130.92679391345783 (T) = (0 -1141.930862510808) / Math.Sqrt((31.970687970368417 / (12)) + (37.82178313493646 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.3711404032512439 = (832.8329176232172 - 1141.930862510808) / 832.8329176232172 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetDecimal

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.962463939776795 > 4.681769113169542.
IsChangePoint: Marked as a change because one of 3/31/2023 8:29:50 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -41.89839442190464 (T) = (0 -4966.212268575676) / Math.Sqrt((842.6745117164827 / (12)) + (1009.0070591681355 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.11430651723034353 = (4456.773959214928 - 4966.212268575676) / 4456.773959214928 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetInt64

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 831.2547246168764 > 774.5372606498729.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -61.29607792642008 (T) = (0 -831.4788131534527) / Math.Sqrt((26.740268384283755 / (12)) + (0.9000023925338607 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.12579936483167525 = (738.5674917997239 - 831.4788131534527) / 738.5674917997239 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetUInt64

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 779.5443430140892 > 733.9027378218697.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -17.16172893341946 (T) = (0 -779.4523890754747) / Math.Sqrt((250.96714743343102 / (12)) + (2.2977618057739737 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.11248969294111184 = (700.6378522166982 - 779.4523890754747) / 700.6378522166982 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetUInt32

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 752.877246916436 > 621.4606947200311.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/22/2023 11:33:11 PM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -99.83195142006652 (T) = (0 -754.1217020352747) / Math.Sqrt((12.126723180829627 / (12)) + (20.673278940479335 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.2714437081372465 = (593.1223672812974 - 754.1217020352747) / 593.1223672812974 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetUInt16

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1536998527788003 > 880.1057536899962.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -127.83769205588376 (T) = (0 -1143.0067574455038) / Math.Sqrt((45.33004577041334 / (12)) + (25.12491213654096 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.3647421843661415 = (837.5257763255678 - 1143.0067574455038) / 837.5257763255678 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.ReadOnlySpan

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
IndexOfString - Duration of single invocation	9.49 ns	11.57 ns	1.22	0.01	False
IndexOfString - Duration of single invocation	11.05 ns	12.96 ns	1.17	0.01	False
IndexOfString - Duration of single invocation	22.74 ns	24.61 ns	1.08	0.01	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!", value: "y", comparisonType: Ordinal)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.56714072737889 > 9.96736264437114.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -294.083571754722 (T) = (0 -11.545324992065305) / Math.Sqrt((0.0004698068049488299 / (12)) + (0.00011324681590266102 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.21391884450859214 = (9.510788175249871 - 11.545324992065305) / 9.510788175249871 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだすだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "す", comparisonType: Ordinal)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.961752322765998 > 11.604796755177198.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -369.2759343220304 (T) = (0 -12.963026864460087) / Math.Sqrt((0.00025780143917074366 / (12)) + (7.081710168258811E-05 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.1734783497043438 = (11.046668963024416 - 12.963026864460087) / 11.046668963024416 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.605466350967337 > 23.885874595048506.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -220.35851443624347 (T) = (0 -24.615681893952015) / Math.Sqrt((0.00023658062351481594 / (12)) + (0.0006797540610953521 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.08220644497221011 = (22.745828218186336 - 24.615681893952015) / 22.745828218186336 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-06-15T08:46:53Z

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	2.24 μs	4.07 μs	1.82	0.09	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<Int32, Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.070379283089807 > 2.355419365968948.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -17.315061704472505 (T) = (0 -4501.868579724471) / Math.Sqrt((18415.054219686943 / (12)) + (188535.4550756072 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.9496079845893326 = (2309.1147632290545 - 4501.868579724471) / 2309.1147632290545 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_String

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Replace_String - Duration of single invocation	63.32 ns	71.07 ns	1.12	0.01	False
Join_List - Duration of single invocation	55.01 ns	68.21 ns	1.24	0.05	False
Trim_CharArr - Duration of single invocation	17.09 ns	19.89 ns	1.16	0.02	False
Replace_String - Duration of single invocation	30.15 ns	34.79 ns	1.15	0.06	False
Remove_IntInt - Duration of single invocation	11.48 ns	14.31 ns	1.25	0.29	False
Insert - Duration of single invocation	14.73 ns	17.96 ns	1.22	0.33	False
TrimStart_CharArr - Duration of single invocation	12.19 ns	14.77 ns	1.21	0.08	False
Join_Array - Duration of single invocation	50.95 ns	63.93 ns	1.25	0.06	False
Replace_String - Duration of single invocation	11.73 ns	15.21 ns	1.30	0.19	False
Replace_String - Duration of single invocation	49.95 ns	61.26 ns	1.23	0.01	False
Split - Duration of single invocation	493.36 ns	547.07 ns	1.11	0.04	False
TrimEnd_CharArr - Duration of single invocation	11.77 ns	14.71 ns	1.25	0.09	False
Replace_Char - Duration of single invocation	19.68 ns	22.36 ns	1.14	0.12	False
Insert - Duration of single invocation	13.12 ns	17.33 ns	1.32	0.38	False
CtorCharCount - Duration of single invocation	21.54 ns	23.36 ns	1.08	0.05	False
Trim_CharArr - Duration of single invocation	14.85 ns	16.63 ns	1.12	0.12	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 71.0685712440624 > 66.43265004573114.
IsChangePoint: Marked as a change because one of 3/23/2023 6:11:55 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -24.75783210462966 (T) = (0 -70.88670497856178) / Math.Sqrt((0.8913089384460107 / (12)) + (0.5341967791468388 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.1345957303266683 = (62.47750020895316 - 70.88670497856178) / 62.47750020895316 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Join_List

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 68.20762149897642 > 56.93204259763223.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -62.07656481071606 (T) = (0 -68.65959514628639) / Math.Sqrt((0.20007511232757533 / (12)) + (0.44708560960423577 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.25676696586994374 = (54.631922234492926 - 68.65959514628639) / 54.631922234492926 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Trim_CharArr(s: " Te st ", c: [' ', ' '])

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.89220046218289 > 17.91175902007894.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 4/6/2023 1:55:21 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -20.842520558053895 (T) = (0 -20.078118927277576) / Math.Sqrt((0.15806001815450116 / (12)) + (0.09022054401523528 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.172628343334897 = (17.122321016202285 - 20.078118927277576) / 17.122321016202285 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "b")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.78638135727987 > 32.83133385860904.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -13.272990105920968 (T) = (0 -34.26164521447613) / Math.Sqrt((0.755699731466911 / (12)) + (0.17701015460634947 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.1200887327931846 = (30.58833127357444 - 34.26164521447613) / 30.58833127357444 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 7, i2: 4)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.306607217454408 > 13.22480743188355.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -8.548435610093048 (T) = (0 -14.487387686226928) / Math.Sqrt((0.37411032359372015 / (12)) + (0.259614845539205 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.15399528487019284 = (12.554113414645833 - 14.487387686226928) / 12.554113414645833 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Insert(s1: "dzsdzsDDZSDZSDZSddsz", i: 7, s2: "Test")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.9597464315748 > 15.688529163261936.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -18.851213912355604 (T) = (0 -17.854372723474476) / Math.Sqrt((0.14082102929063345 / (12)) + (0.14508337064819574 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.19013629201034718 = (15.001956366959734 - 17.854372723474476) / 15.001956366959734 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.TrimStart_CharArr(s: " Test", c: [' ', ' '])

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.770530938644283 > 12.773133564319524.
IsChangePoint: Marked as a change because one of 4/11/2023 2:19:19 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -21.785828901535112 (T) = (0 -14.89762316538372) / Math.Sqrt((0.12241901047777154 / (12)) + (0.0820983877589202 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.2314370361778941 = (12.097754678243737 - 14.89762316538372) / 12.097754678243737 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Join_Array

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 63.92952005401367 > 52.42847526340588.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -29.950611816035728 (T) = (0 -63.44312451212591) / Math.Sqrt((1.0130262198464186 / (12)) + (1.4409083415638315 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.2635921114890935 = (50.208547469769094 - 63.44312451212591) / 50.208547469769094 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "bad", newValue: "nice")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.20779841186198 > 13.370655525556117.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 4/25/2023 1:02:54 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -13.010041311778185 (T) = (0 -15.318592357440561) / Math.Sqrt((0.2778005566649823 / (12)) + (0.15842776586956164 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.18998270155844196 = (12.87295381468891 - 15.318592357440561) / 12.87295381468891 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "nice", newValue: "bad")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 61.258061759651454 > 52.146815453765036.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -51.07211459350747 (T) = (0 -61.45296104748085) / Math.Sqrt((0.43876277383453055 / (12)) + (0.22093587439522297 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.2381353616321031 = (49.63347542749599 - 61.45296104748085) / 49.63347542749599 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Split(s: "A B C D E F G H I J K L M N O P Q R S T U V W X Y Z", arr: [' '], options: RemoveEmptyEntries)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 547.0722360442668 > 510.82171583101615.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -9.478593199611081 (T) = (0 -537.8566024912966) / Math.Sqrt((153.33120908367547 / (12)) + (126.69445494857219 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.09126950175525374 = (492.8723854429914 - 537.8566024912966) / 492.8723854429914 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.TrimEnd_CharArr(s: "Test ", c: [' ', ' '])

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.71106885942104 > 12.432826171343537.
IsChangePoint: Marked as a change because one of 4/11/2023 2:19:19 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -30.00270331819729 (T) = (0 -14.962501751380072) / Math.Sqrt((0.02585516851098903 / (11)) + (0.09801755271449812 / (13))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (13) - 2, .025) and -0.24908852485902405 = (11.978736057213228 - 14.962501751380072) / 11.978736057213228 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'i', newChar: 'I')

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.36465368586285 > 20.67533196061923.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 5/25/2023 4:37:39 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -9.343063192436416 (T) = (0 -22.369085453414844) / Math.Sqrt((0.38096229512721264 / (12)) + (0.2085550864891442 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.10048256836055443 = (20.326614974682627 - 22.369085453414844) / 20.326614974682627 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Insert(s1: "Test", i: 2, s2: " Test")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.331396188422207 > 14.055761544550217.
IsChangePoint: Marked as a change because one of 3/22/2023 1:13:01 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -8.728238741979691 (T) = (0 -16.433348397008697) / Math.Sqrt((0.7652078283815786 / (12)) + (0.4624008190210582 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.20105680457028047 = (13.682407305363292 - 16.433348397008697) / 13.682407305363292 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.CtorCharCount(size: 100)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.36126053377406 > 22.706686717563535.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -11.206381640242856 (T) = (0 -23.590051304940218) / Math.Sqrt((0.29865506273410075 / (12)) + (0.03915727382945532 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.08618734836639182 = (21.71821586802615 - 23.590051304940218) / 21.71821586802615 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Trim_CharArr(s: " Test", c: [' ', ' '])

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.62845653902864 > 15.022266590535766.
IsChangePoint: Marked as a change because one of 4/26/2023 2:24:59 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -20.351702799719767 (T) = (0 -16.7053630597007) / Math.Sqrt((0.07717783719171945 / (12)) + (0.09040767767246011 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.16407831897432476 = (14.350720898590295 - 16.7053630597007) / 14.350720898590295 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Enum

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ToString_Flags - Duration of single invocation	59.83 ns	72.33 ns	1.21	0.06	False
TryParseGeneric_Flags - Duration of single invocation	93.03 ns	107.42 ns	1.15	0.02	False
TryParseGeneric_Flags - Duration of single invocation	13.42 ns	17.10 ns	1.27	0.03	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.ToString_Flags(value: Red, Orange, Yellow, Green, Blue)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 72.33186395860119 > 62.577586626149085.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -4.413532465340377 (T) = (0 -67.88232678056605) / Math.Sqrt((12.730531125714844 / (12)) + (20.083606239725373 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.11726061472563251 = (60.757826675234604 - 67.88232678056605) / 60.757826675234604 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.TryParseGeneric_Flags(text: "Red, Orange, Yellow, Green, Blue")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 107.42484977862114 > 97.68021054305719.
IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 4/25/2023 6:46:56 PM, 5/11/2023 10:01:37 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -43.95447153996991 (T) = (0 -107.41408261980627) / Math.Sqrt((1.0267115446396529 / (12)) + (0.12884611651194208 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.14473801911139833 = (93.83289523587793 - 107.41408261980627) / 93.83289523587793 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.TryParseGeneric_Flags(text: "Red")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.09522617235743 > 14.070387579989891.
IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -62.327776403786444 (T) = (0 -17.066840038460484) / Math.Sqrt((0.02559207758623687 / (12)) + (0.020292660823922188 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.2852653399851644 = (13.278845626271602 - 17.066840038460484) / 13.278845626271602 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.Perf_Path

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ChangeExtension - Duration of single invocation	22.76 ns	25.00 ns	1.10	0.13	False
GetExtension - Duration of single invocation	16.44 ns	18.49 ns	1.12	0.17	False
GetFileName - Duration of single invocation	24.85 ns	31.98 ns	1.29	0.07	False
GetFileNameWithoutExtension - Duration of single invocation	31.86 ns	39.24 ns	1.23	0.05	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Path.ChangeExtension

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.996661568904834 > 23.904587804488692.
IsChangePoint: Marked as a change because one of 3/17/2023 1:20:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -13.476251786488792 (T) = (0 -24.869383869592355) / Math.Sqrt((0.11310561900830161 / (12)) + (0.1630371836529727 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.08732678618896011 = (22.872041952317407 - 24.869383869592355) / 22.872041952317407 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_Path.GetExtension

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.485927700362183 > 16.803743280146975.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -14.555652760206769 (T) = (0 -19.128451770417517) / Math.Sqrt((0.2066854718743568 / (12)) + (0.2973167092357644 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.17975698281366287 = (16.213891546373468 - 19.128451770417517) / 16.213891546373468 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_Path.GetFileName

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.984704558021956 > 26.539866231821374.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -22.831427673062525 (T) = (0 -31.575717990530542) / Math.Sqrt((0.5311679228099321 / (12)) + (0.34472606947583245 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.23819212286646615 = (25.501468962208744 - 31.575717990530542) / 25.501468962208744 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_Path.GetFileNameWithoutExtension

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 39.24252130877096 > 33.391005926518645.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -78.18510625782088 (T) = (0 -39.123109927030406) / Math.Sqrt((0.06293101972696963 / (12)) + (0.04341022660557728 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.2272191855226086 = (31.879480363868264 - 39.123109927030406) / 31.879480363868264 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-06-15T08:46:55Z

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Tests.DictionarySequentialKeys

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
TryGetValue_3k_Int_32ByteRefsValue - Duration of single invocation	4.05 ns	6.66 ns	1.64	0.01	False
TryGetValue_3k_Int_Int - Duration of single invocation	3.76 ns	6.24 ns	1.66	0.01	False
TryGetValue_3k_Int_32ByteValue - Duration of single invocation	4.05 ns	6.67 ns	1.65	0.01	False
TryGetValue_17_Int_Int - Duration of single invocation	3.69 ns	6.22 ns	1.69	0.03	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.DictionarySequentialKeys*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_32ByteRefsValue

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.662505708805262 > 4.257484022647746.
IsChangePoint: Marked as a change because one of 5/4/2023 12:54:08 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -93.75743017894571 (T) = (0 -6.655369530556292) / Math.Sqrt((0.0015955107786948543 / (12)) + (0.008126980130055613 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.6336730145995065 = (4.073868804271 - 6.655369530556292) / 4.073868804271 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_Int

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.243503741909384 > 3.935855756567385.
IsChangePoint: Marked as a change because one of 5/4/2023 12:54:08 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -79.95401195593487 (T) = (0 -6.271588681139822) / Math.Sqrt((0.00208824396554828 / (12)) + (0.010404584750375886 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.6609835362228542 = (3.775828323621844 - 6.271588681139822) / 3.775828323621844 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_32ByteValue

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.674170623488405 > 4.251035251546461.
IsChangePoint: Marked as a change because one of 5/4/2023 12:54:08 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -125.34165509130278 (T) = (0 -6.636611942026581) / Math.Sqrt((0.0007654921490445222 / (12)) + (0.004669075912737598 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.6350906262490099 = (4.058864894388969 - 6.636611942026581) / 4.058864894388969 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_Int

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.2210360557885895 > 3.905968645745382.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -44.99873999235952 (T) = (0 -6.21882341392138) / Math.Sqrt((0.0014481399461652813 / (12)) + (0.038060815606167496 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.6653138801218017 = (3.73432509519859 - 6.21882341392138) / 3.73432509519859 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_Encoding

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetChars - Duration of single invocation	28.13 ns	32.29 ns	1.15	0.10	False
GetString - Duration of single invocation	20.40 ns	22.95 ns	1.13	0.13	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetChars(size: 16, encName: "utf-8")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.28863434794331 > 29.69805247868836.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -8.491605644038692 (T) = (0 -31.684882575531372) / Math.Sqrt((0.7122183348750092 / (12)) + (0.7185575315296244 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.09979012184531295 = (28.809935592409236 - 31.684882575531372) / 28.809935592409236 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_Encoding.GetString(size: 16, encName: "utf-8")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.94718439924276 > 21.067022991216696.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -10.346531793907745 (T) = (0 -23.09559970004029) / Math.Sqrt((0.5282888943999148 / (12)) + (0.9359524657465976 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.1800695944935813 = (19.57138782984372 - 23.09559970004029) / 19.57138782984372 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Insert_Strings - Duration of single invocation	549.76 ns	644.03 ns	1.17	0.17	False
Insert_Primitives - Duration of single invocation	24.64 μs	29.99 μs	1.22	0.10	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_StringBuilder.Insert_Strings

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 644.0328758818342 > 578.0295455470888.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -20.00261900528355 (T) = (0 -651.9561212718504) / Math.Sqrt((16.129493499944523 / (12)) + (311.33651492942926 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.18245413986209533 = (551.3584834232009 - 651.9561212718504) / 551.3584834232009 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_StringBuilder.Insert_Primitives

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.986517171647932 > 25.72889670307347.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -117.4732291930289 (T) = (0 -29936.81906263461) / Math.Sqrt((11834.18342249258 / (12)) + (15022.992584950438 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.22190046603317123 = (24500.210855817702 - 29936.81906263461) / 24500.210855817702 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Sort<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
List - Duration of single invocation	5.50 μs	6.28 μs	1.14	0.27	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<Int32>.List(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.280855000000001 > 5.792986561764706.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -13.639524758854387 (T) = (0 -6329.226676113359) / Math.Sqrt((29401.377213163418 / (12)) + (22611.156156247464 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.16209426413635528 = (5446.396967475879 - 6329.226676113359) / 5446.396967475879 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Decimal

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	60.25 ns	70.54 ns	1.17	0.01	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Decimal*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Decimal.ToString(value: 123456.789)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 70.53819228846456 > 63.701263192003594.
IsChangePoint: Marked as a change because one of 2/17/2023 12:28:21 PM, 5/12/2023 9:12:45 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -58.59248829728597 (T) = (0 -70.45969599636017) / Math.Sqrt((0.23116791094506528 / (12)) + (0.12018838406361859 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.1633435899844934 = (60.566539931078616 - 70.45969599636017) / 60.566539931078616 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Depth

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ReadSpanEmptyLoop - Duration of single invocation	21.91 μs	24.76 μs	1.13	0.01	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Depth*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Depth.ReadSpanEmptyLoop(Depth: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.76216961356179 > 23.19307990014252.
IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -70.05166869801845 (T) = (0 -24747.661444829122) / Math.Sqrt((15066.353867820942 / (12)) + (3334.8601740758963 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.12368371306754807 = (22023.689724282285 - 24747.661444829122) / 22023.689724282285 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	2.53 μs	5.60 μs	2.21	0.10	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueTrue<Int32, Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.601744551163651 > 2.6592702878990746.
IsChangePoint: Marked as a change because one of 5/4/2023 12:54:08 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -16.95723683080062 (T) = (0 -5675.955541913432) / Math.Sqrt((358290.8977522186 / (12)) + (10682.715490069993 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -1.0976930785931174 = (2705.8083948678454 - 5675.955541913432) / 2705.8083948678454 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in BenchmarksGame.KNucleotide_1

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
RunBench - Duration of single invocation	305.27 ms	350.64 ms	1.15	0.04	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BenchmarksGame.KNucleotide_1*'

Payloads

Baseline
Compare

Histogram

BenchmarksGame.KNucleotide_1.RunBench

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 350.6394933333333 > 320.340806625.
IsChangePoint: Marked as a change because one of 5/3/2023 1:18:01 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -5.332818717174185 (T) = (0 -338824057.01465195) / Math.Sqrt((176942112120.26514 / (12)) + (515162252766951 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.10999831317838585 = (305247362.0833333 - 338824057.01465195) / 305247362.0833333 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Globalization.Tests.Perf_NumberCultureInfo

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ToString - Duration of single invocation	124.24 ns	134.45 ns	1.08	0.01	False
ToString - Duration of single invocation	123.49 ns	133.34 ns	1.08	0.01	False
ToString - Duration of single invocation	122.80 ns	132.67 ns	1.08	0.01	False
ToString - Duration of single invocation	125.59 ns	133.74 ns	1.06	0.01	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_NumberCultureInfo*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: fr)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 134.45495926760046 > 131.2259096964906.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -25.308755013802067 (T) = (0 -134.94927237898102) / Math.Sqrt((0.9788790702014226 / (12)) + (1.0479098861601466 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.08169713970271146 = (124.75698365632162 - 134.94927237898102) / 124.75698365632162 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: )

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 133.3428440158849 > 130.22011109851726.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -32.91912789980444 (T) = (0 -134.76833889784078) / Math.Sqrt((0.8364732892361921 / (12)) + (0.3974569825378998 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.08383600596715239 = (124.34384736792475 - 134.76833889784078) / 124.34384736792475 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: da)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 132.67459194306758 > 130.55155025586228.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -29.521577329187185 (T) = (0 -134.79852423594548) / Math.Sqrt((0.7320768075549657 / (12)) + (0.7436600564819944 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.08142946911802637 = (124.64846583650262 - 134.79852423594548) / 124.64846583650262 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: ja)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 133.74161085502212 > 130.83502911100825.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -46.8870776517239 (T) = (0 -135.06696926841568) / Math.Sqrt((0.21617767821745187 / (12)) + (0.39384783490491637 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.08260289762247515 = (124.76132251727648 - 135.06696926841568) / 124.76132251727648 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-06-15T08:46:57Z

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	dd715a0747ebeb85a234c8abec7e01bb6652c8ce
Compare	9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.BlockCopyPerf

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
CallBlockCopy - Duration of single invocation	5.89 ns	7.93 ns	1.35	0.15	False

Test Report

Repro

General Docs link: /~https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone /~https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.BlockCopyPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.BlockCopyPerf.CallBlockCopy(numElements: 10)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.927813000119963 > 7.3860958177773215.
IsChangePoint: Marked as a change because one of 5/2/2023 7:50:39 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -7.175399691720614 (T) = (0 -7.923551202234628) / Math.Sqrt((0.19145004068439 / (12)) + (0.07559579091114059 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.15421822461324 = (6.864864055399647 - 7.923551202234628) / 6.864864055399647 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

AndyAyersMS · 2023-06-15T16:21:17Z

Also likely from dotnet/runtime#87115

@EgorBo FYI

performanceautofiler bot added arch-arm64 branch-refs/heads/main kind-micro os-windows perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-coreclr untriaged labels Jun 15, 2023

AndyAyersMS removed the untriaged label Jun 15, 2023

[Perf] Windows/arm64: 83 Regressions on 6/8/2023 3:52:42 PM #18852

[Perf] Windows/arm64: 83 Regressions on 6/8/2023 3:52:42 PM #18852

Comments

performanceautofiler bot commented Jun 15, 2023 • edited Loading

Run Information

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Repro

Payloads

Payloads

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value: 18446744073709551615)

Description of detection logic

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseDateTimeOffset(value: 12/30/2017 3:45:22 AM -08:00)

Description of detection logic

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: 12345)

Description of detection logic

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseDecimal(value: 123456.789)

Description of detection logic

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt16(value: 0)

Description of detection logic

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32(value: 4294967295)

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.Tests.Perf_Double

Repro

Payloads

Payloads

Histogram

System.Tests.Perf_Double.ToStringWithCultureInfo(value: 12345, culture: zh)

Description of detection logic

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "E")

Description of detection logic

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G")

Description of detection logic

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "E")

Description of detection logic

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "E")

Description of detection logic

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "G")

Description of detection logic

JIT Disasms

System.Tests.Perf_Double.ToString(value: -1.7976931348623157E+308)

Description of detection logic

JIT Disasms

System.Tests.Perf_Double.ToString(value: 1.7976931348623157E+308)

Description of detection logic

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G17")

Description of detection logic

JIT Disasms

System.Tests.Perf_Double.ToString(value: 12345)

Description of detection logic

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "R")

Description of detection logic

JIT Disasms

Docs

performanceautofiler bot commented Jun 15, 2023 • edited Loading

Run Information

Regressions in System.Tests.Perf_Single

Repro

Payloads

Payloads

Histogram

System.Tests.Perf_Single.ToString(value: 3.4028235E+38)

Description of detection logic

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G")

performanceautofiler bot commented Jun 15, 2023 •

edited

Loading

performanceautofiler bot commented Jun 15, 2023 •

edited

Loading