ci: Migrate metamaskbot PR comment #29373

Gudahtt · 2024-12-20T05:36:17Z

Description

Migrate the metamaskbot PR comment from CircleCI to GitHub Actions. CircleCI is still used for artifact storage for linked artifacts.

Related issues

Relates to #28572

These changes were extracted from #29256

Manual testing steps

Test that all links in the metamaskbot comment work correctly, except those that are already broken. Links already broken on main include:
- Some build links (fixed in ci: Fix metamaskbot comment test build links #29403 ):
  - Beta builds
  - Firefox test build, and Firefox flask-test build
- MV3 performance stats reports (fixed in chore: Remove broken MV3 perf stats #29408 ):
  - mv3: Background Module Init Stats (link works, page is broken)
  - mv3: UI Init Stats (link works, page is broken)
  - mv3: Module Load Stats (link works, page is broken)
- Coverage report (fixed by chore: Remove broken coverage report link #29410)
- mv3 bundle size stats (fixed by fix: Fix bundle size tracking #29486)

Screenshots/Recordings

N/A

Pre-merge author checklist

I've followed MetaMask Contributor Docs and MetaMask Extension Coding Standards.
I've completed the PR template to the best of my ability
I’ve included tests if applicable
I’ve documented my code using JSDoc format if applicable
I’ve applied the right labels on the PR (see labeling guidelines). Not required for external contributors.

Pre-merge reviewer checklist

I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

metamaskbot · 2024-12-20T07:01:01Z

Builds ready [6a4182e]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:

Bundle size diffs

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

metamaskbot · 2024-12-20T13:54:51Z

Builds ready [dc9fd81]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:

Page Load Metrics (1422 ± 42 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	1327	1610	1431	83	40
		domContentLoaded	1292	1575	1402	78	38
		load	1301	1610	1422	87	42
		domInteractive	20	62	32	14	7
		backgroundConnect	7	61	22	17	8
		firstReactRender	15	96	37	28	13
		getState	4	91	20	25	12
		initialActions	0	1	0	0	0
		loadScripts	919	1204	1039	77	37
		setupStore	6	90	13	20	9
		uiStartup	1465	2217	1713	222	107

Bundle size diffs [🚀 Bundle size reduced!]

background: -113.98 KiB (-2.03%)
ui: 197 Bytes (0.00%)
common: -272.89 KiB (-3.27%)

metamaskbot · 2024-12-20T18:48:25Z

Builds ready [fb737b2]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:

Page Load Metrics (1766 ± 68 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	246	2055	1605	464	223
		domContentLoaded	1463	1975	1733	144	69
		load	1516	2040	1766	142	68
		domInteractive	23	66	37	12	6
		backgroundConnect	8	79	34	22	10
		firstReactRender	16	86	36	23	11
		getState	6	70	21	20	9
		initialActions	0	1	0	0	0
		loadScripts	1097	1482	1301	105	51
		setupStore	7	52	14	10	5
		uiStartup	1691	2301	2008	165	79

Bundle size diffs [🚀 Bundle size reduced!]

background: -113.98 KiB (-2.03%)
ui: 1.18 KiB (0.01%)
common: -273.38 KiB (-3.28%)

metamaskbot · 2024-12-20T19:22:44Z

Builds ready [44f95d4]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:

Page Load Metrics (1610 ± 85 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	1390	2238	1615	175	84
		domContentLoaded	1385	2207	1588	172	82
		load	1390	2241	1610	177	85
		domInteractive	23	127	37	25	12
		backgroundConnect	8	62	22	16	8
		firstReactRender	16	96	34	26	12
		getState	4	61	19	19	9
		initialActions	0	1	0	0	0
		loadScripts	1009	1687	1194	142	68
		setupStore	6	54	14	16	8
		uiStartup	1617	2476	1888	203	98

Bundle size diffs [🚀 Bundle size reduced!]

background: -114.89 KiB (-2.04%)
ui: -9.25 KiB (-0.12%)
common: -317.6 KiB (-3.79%)

metamaskbot · 2024-12-20T20:46:52Z

Builds ready [0bce5e0]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5, 6, 7
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12
- ui: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
- content-script: 0
- offscreen: 0

Page Load Metrics (1771 ± 75 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	413	2067	1702	335	161
		domContentLoaded	1521	2038	1747	164	79
		load	1569	2046	1771	156	75
		domInteractive	26	88	43	17	8
		backgroundConnect	9	67	25	18	9
		firstReactRender	16	77	44	24	11
		getState	5	55	15	16	8
		initialActions	0	1	0	0	0
		loadScripts	1091	1477	1287	120	58
		setupStore	6	61	15	16	8
		uiStartup	1794	2317	2017	161	77

Bundle size diffs

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

Gudahtt · 2024-12-20T20:50:02Z

development/highlights/index.js

@@ -9,7 +9,7 @@ async function getHighlights({ artifactBase }) {
  // here we assume the PR base branch ("target") is `main` in lieu of doing
  // a query against the github api which requires an access token
  // see https://discuss.circleci.com/t/how-to-retrieve-a-pull-requests-base-branch-name-github/36911
-  const changedFiles = await getChangedFiles({ target: 'main' });
+  const changedFiles = await getChangedFiles({ target: 'origin/main' });


This is now running on GitHub Actions rather than CircleCI, which doesn't checkout the main branch locally. origin/main works across both CI systems.

Gudahtt · 2024-12-20T20:52:47Z

.github/workflows/publish-prerelease.yml

+          BRANCH: ${{ github.head_ref }}
+          HEAD_COMMIT_HASH: ${{ github.event.pull_request.head.sha }}
+        run: |
+          pipeline_id=$(curl --silent "https://circleci.com/api/v2/project/gh/$OWNER/$REPOSITORY/pipeline?branch=$BRANCH" | jq -r ".items | map(select(.vcs.revision == \"${HEAD_COMMIT_HASH}\" )) | first | .id")


Same strategy as used elsewhere for getting the CircleCI workflow corresponding to the current branch+commit:

metamask-extension/.github/workflows/wait-for-circleci-workflow-status.yml

Line 21 in 6f11eda

pipeline_id=$(curl --silent "https://circleci.com/api/v2/project/gh/$OWNER/$REPOSITORY/pipeline?branch=$BRANCH" | jq -r ".items | map(select(.vcs.revision == \"${HEAD_COMMIT_HASH}\" )) | first | .id")

we could probably refactor this later into something reusable, to avoid this duplication. But it is a temporary workaround anyway - since we intend to migrate away from CircleCI.

Gudahtt · 2024-12-20T20:53:49Z

.github/workflows/publish-prerelease.yml

+
+          echo "Getting artifacts from pipeline '${pipeline_id}', workflow '${workflow_id}', build number '${build_num}', job ID '${job_id}'"
+
+      - name: Get CircleCI job artifacts


Most of the artifacts referenced in the metamaskbot comment are referenced by CircleCI artifact link. We didn't need to download those.

These three are pulled from the filesystem by the script though, so we needed to download these.

Technically we could also fetch it in the .js script and keep it in memory instead of having it on the filesystem, but in my original implementation I did not want to make any "breaking changes" to the script.

Gudahtt · 2024-12-20T20:54:31Z

.github/workflows/publish-prerelease.yml

+          PR_COMMENT_TOKEN: ${{ secrets.PR_COMMENT_TOKEN }}
+          PR_NUMBER: ${{ github.event.pull_request.number }}
+          HEAD_COMMIT_HASH: ${{ github.event.pull_request.head.sha }}
+          MERGE_BASE_COMMIT_HASH: ${{ steps.get-merge-base.outputs.MERGE_BASE }}


This is equivalent to the PARENT_COMMIT used previously, they both use git merge-base to find the base commit between the current branch and the target branch.

this is a much better name, PARENT_COMMIT had me confused before.

Gudahtt · 2024-12-20T20:56:36Z

development/metamaskbot-build-announce.js

  const sourceMapRoot = '/build-artifacts/source-map-explorer/';
-  const bundleFiles = await glob(`.${sourceMapRoot}*${fileType}`);


The strategy used to collect these links was changed. Previously we used glob to look for these files on the filesystem. It would have been tedious to migrate each file one-by-one between CI systems, so instead we're hard-coding the 5 bundle types and testing each bundle index until we get a 404.

This ensures the links are constructed correctly even as the number of bundles fluctuates (which is common), and it doesn't require transferring the files between CI systems.

how many requests are we making here? It seems like a lot

Yeah it's a lot. A bit regrettable. But at least we can replace this soon, and in the overall comment building this is very fast in comparison to the other steps.

metamaskbot · 2024-12-20T21:26:08Z

Builds ready [2727f61]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5, 6, 7
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12
- ui: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
- content-script: 0
- offscreen: 0

Page Load Metrics (1783 ± 125 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	1550	2822	1789	262	126
		domContentLoaded	1522	2811	1754	265	127
		load	1550	2822	1783	260	125
		domInteractive	20	128	41	25	12
		backgroundConnect	8	80	28	23	11
		firstReactRender	15	81	32	23	11
		getState	4	58	22	22	10
		initialActions	0	1	0	0	0
		loadScripts	1115	2293	1327	243	117
		setupStore	7	71	18	20	10
		uiStartup	1750	3091	2012	273	131

Bundle size diffs

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

itsyoboieltr · 2025-01-06T21:28:52Z

development/metamaskbot-build-announce.js

+ */
+async function artifactExists(url) {
+  // Using a regular GET request here rather than HEAD because for some reason CircleCI always
+  // returns 404 for HEAD requests.


I can feel the frustration you must have felt when you discovered this fact :DDD

itsyoboieltr

Good job on this one! Seems good overall

Migrate the `metamaskbot` PR comment from CircleCI to GitHub Actions. CircleCI is still used for artifact storage for linked artifacts. Relates to #28572 These changes were extracted from #29256

metamaskbot · 2025-01-07T16:44:30Z

Builds ready [df4b0c5]

builds: chrome, firefox
builds (flask): chrome, firefox
builds (MMI): chrome
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Bundle Size Stats
mv2: E2e Actions Stats
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5, 6, 7
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12
- ui: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
- content-script: 0
- offscreen: 0

Page Load Metrics (1966 ± 102 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	1691	2596	1972	221	106
		domContentLoaded	1682	2526	1935	206	99
		load	1693	2581	1966	211	102
		domInteractive	30	133	51	25	12
		backgroundConnect	10	61	29	19	9
		firstReactRender	18	102	43	26	12
		getState	5	87	28	27	13
		initialActions	0	1	0	0	0
		loadScripts	1206	2019	1439	183	88
		setupStore	7	86	25	25	12
		uiStartup	1993	2907	2309	276	133

Gudahtt · 2025-01-07T16:56:56Z

Strange. Had a LavaMoat policy validation failure (unexpected end of JSON input), but it passed upon retry. No policy changes here though, or even build system changes.

Edit: We discovered the likely cause, tracked here: #29482

Gudahtt · 2025-01-07T20:55:26Z

Discovered another broken link, details here: #29485

This was broken in another PR, and I will fix it separately from this as well. Just noting for anyone testing this. I'll update the manual testing instructions as well.

Gudahtt · 2025-01-09T17:02:14Z

.github/workflows/main.yml

@@ -76,6 +76,15 @@ jobs:
    name: Wait for CircleCI workflow status
    uses: ./.github/workflows/wait-for-circleci-workflow-status.yml

+  publish-prerelease:


Intentionally not required for all-jobs-completed because this is informational only, not a quality gate

desi

I checked all the links and the ones I found that were not working were covered in other issues for that reason I am fine to approve this.

Gudahtt added the team-extension-platform label Dec 20, 2024

metamaskbot added the team-wallet-framework label Dec 20, 2024

Gudahtt force-pushed the migrate-metamaskbot-comment branch 4 times, most recently from 051cd52 to 3cbc139 Compare December 20, 2024 05:40

Gudahtt force-pushed the migrate-metamaskbot-comment branch from 54e05e1 to 0bce5e0 Compare December 20, 2024 20:16

Gudahtt force-pushed the migrate-metamaskbot-comment branch from 0bce5e0 to e10477e Compare December 20, 2024 20:47

Gudahtt commented Dec 20, 2024

View reviewed changes

Gudahtt marked this pull request as ready for review December 20, 2024 21:29

Gudahtt requested a review from a team as a code owner December 20, 2024 21:29

itsyoboieltr reviewed Jan 6, 2025

View reviewed changes

itsyoboieltr approved these changes Jan 6, 2025

View reviewed changes

Gudahtt added 7 commits January 7, 2025 12:35

ci: Migrate metamaskbot PR comment

8778a7d

Migrate the `metamaskbot` PR comment from CircleCI to GitHub Actions. CircleCI is still used for artifact storage for linked artifacts. Relates to #28572 These changes were extracted from #29256

Temporarily disable e2e tests

0bd48c6

Fetch entire git history

118eafb

Fix benchmark artifact processing

0bbc3c5

Fix highlight script (no local main branch)

b535197

Ensure flask test build is available to prepublish

ba153d5

Build bundles without glob

1374f33

Gudahtt added 4 commits January 7, 2025 12:35

Add debug logs

4836725

Fix bundle validation

1906d2f

Initialize bundle entries

acf625a

Undo temporary CircleCI config changes

df4b0c5

Gudahtt dismissed davidmurdoch’s stale review via df4b0c5 January 7, 2025 16:05

Gudahtt force-pushed the migrate-metamaskbot-comment branch from 2727f61 to df4b0c5 Compare January 7, 2025 16:05

davidmurdoch previously approved these changes Jan 7, 2025

View reviewed changes

Gudahtt commented Jan 9, 2025

View reviewed changes

itsyoboieltr requested a review from davidmurdoch January 9, 2025 17:09

desi approved these changes Jan 9, 2025

View reviewed changes

itsyoboieltr added this pull request to the merge queue Jan 9, 2025

danjm approved these changes Jan 9, 2025

View reviewed changes

Merged via the queue into main with commit 87c524e Jan 9, 2025
90 checks passed

itsyoboieltr deleted the migrate-metamaskbot-comment branch January 9, 2025 18:24

github-actions bot locked and limited conversation to collaborators Jan 9, 2025

metamaskbot added the release-12.11.0 Issue or pull request that will be included in release 12.11.0 label Jan 9, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: Migrate metamaskbot PR comment #29373

ci: Migrate metamaskbot PR comment #29373

Gudahtt commented Dec 20, 2024 •

edited by itsyoboieltr

Loading

metamaskbot commented Dec 20, 2024

metamaskbot commented Dec 20, 2024

metamaskbot commented Dec 20, 2024

metamaskbot commented Dec 20, 2024

metamaskbot commented Dec 20, 2024

Gudahtt Dec 20, 2024

Gudahtt Dec 20, 2024

itsyoboieltr Jan 6, 2025 •

edited

Loading

Gudahtt Dec 20, 2024

itsyoboieltr Jan 6, 2025

Gudahtt Dec 20, 2024

itsyoboieltr Jan 6, 2025 •

edited

Loading

Gudahtt Dec 20, 2024 •

edited

Loading

itsyoboieltr Jan 6, 2025

Gudahtt Jan 7, 2025

metamaskbot commented Dec 20, 2024

itsyoboieltr Jan 6, 2025

itsyoboieltr left a comment

metamaskbot commented Jan 7, 2025

Gudahtt commented Jan 7, 2025 •

edited

Loading

Gudahtt commented Jan 7, 2025

Gudahtt Jan 9, 2025

desi left a comment


		echo "Getting artifacts from pipeline '${pipeline_id}', workflow '${workflow_id}', build number '${build_num}', job ID '${job_id}'"

		- name: Get CircleCI job artifacts

		const sourceMapRoot = '/build-artifacts/source-map-explorer/';
		const bundleFiles = await glob(`.${sourceMapRoot}*${fileType}`);

ci: Migrate metamaskbot PR comment #29373

ci: Migrate metamaskbot PR comment #29373

Conversation

Gudahtt commented Dec 20, 2024 • edited by itsyoboieltr Loading

Description

Related issues

Manual testing steps

Screenshots/Recordings

Pre-merge author checklist

Pre-merge reviewer checklist

metamaskbot commented Dec 20, 2024

metamaskbot commented Dec 20, 2024

metamaskbot commented Dec 20, 2024

metamaskbot commented Dec 20, 2024

metamaskbot commented Dec 20, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

itsyoboieltr Jan 6, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

itsyoboieltr Jan 6, 2025 • edited Loading

Choose a reason for hiding this comment

Gudahtt Dec 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

metamaskbot commented Dec 20, 2024

Choose a reason for hiding this comment

itsyoboieltr left a comment

Choose a reason for hiding this comment

metamaskbot commented Jan 7, 2025

Gudahtt commented Jan 7, 2025 • edited Loading

Gudahtt commented Jan 7, 2025

Choose a reason for hiding this comment

desi left a comment

Choose a reason for hiding this comment

Gudahtt commented Dec 20, 2024 •

edited by itsyoboieltr

Loading

itsyoboieltr Jan 6, 2025 •

edited

Loading

itsyoboieltr Jan 6, 2025 •

edited

Loading

Gudahtt Dec 20, 2024 •

edited

Loading

Gudahtt commented Jan 7, 2025 •

edited

Loading