signature approved metrics e2e test #19628

brad-decker · 2023-06-15T20:38:42Z

Explanation

Part of #19261

Implements E2E tests to ensure that signing signature requests results in requested and approved events being fired to segment. This is important so that if anything changes we are alerted before mixpanel reflects a disturbance in our business metrics. These events are only fired for users who opt-in to metametrics so the controller for MetaMetrics is mocked out to have the opt-in flag set to true.

Manual Testing Steps

No functional changes were made in this PR, this should be primarily code review and review of the resulting test runs in CI

Pre-merge author checklist

I've clearly explained:
- What problem this PR is solving
- How this problem was solved
- How reviewers can test my changes
Sufficient automated test coverage has been added

Pre-merge reviewer checklist

PR is linked to the appropriate GitHub issue
IF this PR fixes a bug in the release milestone, add this PR to the release milestone

If further QA is required (e.g. new feature, complex testing steps, large refactor), add the Extension QA Board label.

In this case, a QA Engineer approval will be be required.

codecov · 2023-06-15T21:01:07Z

Codecov Report

Merging #19628 (4c5675b) into develop (89cec53) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop   #19628   +/-   ##
========================================
  Coverage    69.83%   69.83%           
========================================
  Files          979      979           
  Lines        36905    36905           
  Branches      9909     9909           
========================================
  Hits         25769    25769           
  Misses       11136    11136

metamaskbot · 2023-06-15T21:43:56Z

Builds ready [3172ffd]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4
- common: 0, 1, 2, 3, 4
- content-script: 0
- contentscript: contentscript
- disable-console: disable-console
- inpage: inpage
- policy-load: policy-load
- sentry-install: sentry-install
- ui: 0, 1, 10, 11, 12, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (1583 ± 39 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	102	339	131	50	24
		domContentLoaded	1455	1868	1564	82	39
		load	1485	1869	1583	82	39
		domInteractive	1455	1868	1564	82	39

Bundle size diffs

background: 0 bytes
ui: 0 bytes
common: 0 bytes

metamaskbot · 2023-06-20T16:00:18Z

Builds ready [5d35243]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4
- common: 0, 1, 2, 3, 4
- content-script: 0
- contentscript: contentscript
- disable-console: disable-console
- inpage: inpage
- policy-load: policy-load
- sentry-install: sentry-install
- ui: 0, 1, 10, 11, 12, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (1688 ± 95 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	98	174	129	21	10
		domContentLoaded	1452	2339	1665	200	96
		load	1466	2340	1688	197	95
		domInteractive	1452	2339	1665	200	96

Bundle size diffs

background: 0 bytes
ui: 0 bytes
common: 0 bytes

metamaskbot · 2023-06-22T17:39:41Z

Builds ready [20e1ebd]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4
- common: 0, 1, 2, 3, 4, 5
- content-script: 0
- contentscript: contentscript
- disable-console: disable-console
- inpage: inpage
- policy-load: policy-load
- sentry-install: sentry-install
- ui: 0, 1, 10, 11, 12, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (1532 ± 28 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	100	137	111	9	4
		domContentLoaded	1427	1583	1500	46	22
		load	1436	1642	1532	58	28
		domInteractive	1427	1583	1500	46	22

Bundle size diffs

background: 0 bytes
ui: 0 bytes
common: 0 bytes

test/e2e/metrics/signature-approved.spec.js

metamaskbot · 2023-06-22T19:55:40Z

Builds ready [62f1010]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4
- common: 0, 1, 2, 3, 4, 5
- content-script: 0
- contentscript: contentscript
- disable-console: disable-console
- inpage: inpage
- policy-load: policy-load
- sentry-install: sentry-install
- ui: 0, 1, 10, 11, 12, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (1597 ± 48 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	115	223	146	25	12
		domContentLoaded	1403	1817	1596	101	48
		load	1403	1817	1597	101	48
		domInteractive	1403	1817	1596	101	48

Bundle size diffs

background: 0 bytes
ui: 0 bytes
common: 0 bytes

metamaskbot · 2023-06-22T20:53:32Z

Builds ready [4c5675b]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4
- common: 0, 1, 2, 3, 4, 5
- content-script: 0
- contentscript: contentscript
- disable-console: disable-console
- inpage: inpage
- policy-load: policy-load
- sentry-install: sentry-install
- ui: 0, 1, 10, 11, 12, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (1443 ± 23 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	107	166	124	14	7
		domContentLoaded	1346	1535	1443	48	23
		load	1346	1535	1443	48	23
		domInteractive	1346	1535	1443	48	23

Bundle size diffs

background: 0 bytes
ui: 0 bytes
common: 0 bytes

PeterYinusa

LGTM

brad-decker changed the title ~~signature approved e2e~~ signature approved metrics e2e test Jun 15, 2023

brad-decker force-pushed the test/e2e-test-permission-metricss-19261 branch from d1eedd6 to 5d35243 Compare June 20, 2023 15:28

brad-decker force-pushed the test/e2e-test-permission-metricss-19261 branch from 7211b8e to efcb891 Compare June 21, 2023 16:56

brad-decker marked this pull request as ready for review June 22, 2023 17:32

brad-decker requested a review from a team as a code owner June 22, 2023 17:32

brad-decker requested a review from DDDDDanica June 22, 2023 17:32

PeterYinusa previously approved these changes Jun 22, 2023

View reviewed changes

DDDDDanica previously approved these changes Jun 22, 2023

View reviewed changes

test/e2e/metrics/signature-approved.spec.js Outdated Show resolved Hide resolved

test/e2e/metrics/signature-approved.spec.js Outdated Show resolved Hide resolved

brad-decker added 4 commits June 22, 2023 13:58

signature approved e2e

5534630

improvements

b57b8b3

updates

8ab451b

fixup

d1ef16d

brad-decker force-pushed the test/e2e-test-permission-metricss-19261 branch from 20e1ebd to d1ef16d Compare June 22, 2023 18:59

use default ganache options

62f1010

brad-decker dismissed stale reviews from PeterYinusa and DDDDDanica via 62f1010 June 22, 2023 19:24

extract swithToNotifiationWindow to helpers

4c5675b

brad-decker force-pushed the test/e2e-test-permission-metricss-19261 branch from c831b7c to 4c5675b Compare June 22, 2023 20:21

DDDDDanica approved these changes Jun 22, 2023

View reviewed changes

PeterYinusa approved these changes Jun 23, 2023

View reviewed changes

brad-decker merged commit b86326b into develop Jun 23, 2023

brad-decker deleted the test/e2e-test-permission-metricss-19261 branch June 23, 2023 14:48

github-actions bot locked and limited conversation to collaborators Jun 23, 2023

metamaskbot added the release-10.34.0 Issue or pull request that will be included in release 10.34.0 label Jun 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

signature approved metrics e2e test #19628

signature approved metrics e2e test #19628

brad-decker commented Jun 15, 2023 •

edited

Loading

codecov bot commented Jun 15, 2023 •

edited

Loading

metamaskbot commented Jun 15, 2023

metamaskbot commented Jun 20, 2023

metamaskbot commented Jun 22, 2023

metamaskbot commented Jun 22, 2023

metamaskbot commented Jun 22, 2023

PeterYinusa left a comment

signature approved metrics e2e test #19628

signature approved metrics e2e test #19628

Conversation

brad-decker commented Jun 15, 2023 • edited Loading

Explanation

Manual Testing Steps

Pre-merge author checklist

Pre-merge reviewer checklist

codecov bot commented Jun 15, 2023 • edited Loading

Codecov Report

metamaskbot commented Jun 15, 2023

metamaskbot commented Jun 20, 2023

metamaskbot commented Jun 22, 2023

metamaskbot commented Jun 22, 2023

metamaskbot commented Jun 22, 2023

PeterYinusa left a comment

Choose a reason for hiding this comment

brad-decker commented Jun 15, 2023 •

edited

Loading

codecov bot commented Jun 15, 2023 •

edited

Loading