[CI] Build PR images on GHCR #3432

jlestel · 2023-11-15T02:51:42Z

I modified the CI so that the PR images are build then pushed into the GitHub packages (GHCR).

This does not interfere with how current CI works with Docker Hub.

Functioning :

When a PR is created or modified: the image is built on the GHCR from fork repo.
When a branch is pushed: the image is built on the GHCR from main repo.
When a PR is completed: the image is deleted in the fork repo. Additionally, untagged images are also deleted on this fork repo.

Once this PR is merged, each new / updated PR will trigger CI on the fork repo.

Users will then see all these images on their fork. Eg: https://github.com/jlestel/teslamate/pkgs/container/teslamate

If necessary, anyone can then modify their docker-compose to use a PR image (eg: ghcr.io/jlestel/teslamate:master) for testing this PR instead of teslamate/teslamate:latest.

JakobLichterfeld · 2023-11-15T07:50:08Z

Thank you for this PR. Can you elaborate on why you chose to modify the Elixir CI rather than create your own workflow?

jlestel · 2023-11-15T08:24:20Z

Thank you for this PR. Can you elaborate on why you chose to modify the Elixir CI rather than create your own workflow?

No reason, I just separated them.

Now I trigger the GHCR build on a pull_request or push master in a separate workflow.

JakobLichterfeld

lgtm, my findings see inline comments

.github/workflows/ghcr_build.yml

DrMichael · 2023-11-15T11:29:28Z

If necessary, anyone can then modify their docker-compose to use a PR image (eg: ghcr.io/teslamate-org/teslamate:pr-123) for testing instead of teslamate/ teslamate:latest

Would that also be the case for grafana?

jlestel · 2023-11-15T15:49:31Z

If necessary, anyone can then modify their docker-compose to use a PR image (eg: ghcr.io/teslamate-org/teslamate:pr-123) for testing instead of teslamate/ teslamate:latest

Would that also be the case for grafana?

yes I added grafana also

adriankumpf · 2023-11-15T17:39:42Z

Could this be used by a malicious PR to leak build secrets? Or does building the images require maintainer approval?

brianmay · 2023-11-15T23:05:52Z

I don't like to be the pessimist, but wondering if this will work. "With the exception of GITHUB_TOKEN, secrets are not passed to the runner when a workflow is triggered from a forked repository." https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions#using-secrets-in-a-workflow

My experience, not even jobs triggered by dependabot created pull requests have access to secrets. I had to update all of my github workflows as a result.

But nothing wrong with trying it. :-)

Also see: https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions

JakobLichterfeld · 2023-11-16T07:20:13Z

Could this be used by a malicious PR to leak build secrets? Or does building the images require maintainer approval?

Good point you raise. As far as my brief research shows, you can extract secrets using malicious github workflows and actions. Building a docker image with a verified pipeline should not be an issue, but PRs related to .github should be excluded from auto-run. But this is nothing introduced by this PR, but more of a general issue in my opinion.

JakobLichterfeld

Thank you for your dedicated work! I really like the process of refactoring into separate actions. I added my findings as inline comments.

.github/actions/build/action.yaml

.github/actions/grafana/action.yml

.github/actions/merge/action.yml

JakobLichterfeld

lgtm

jlestel · 2023-11-16T12:02:38Z

Could this be used by a malicious PR to leak build secrets? Or does building the images require maintainer approval?

Yes there was a flaw there.

Now, I prevent workflows from being launched when there are changes to the .github directory: only a maintainer can run it manually.

jlestel · 2023-11-16T16:55:45Z

So, i'll try to replace pull_request by pull_request_target to only run an action in the repo of the requester of a PR.

Voilà @adriankumpf @JakobLichterfeld : the GHCR build workflow is now based on the PR source user. The CI is played on the fork repo instead of the teslamate-org repo (and the generated images too).

Eg for this PR, you'll find image(s) on my fork:

TeslaMate image: https://github.com/jlestel/teslamate/pkgs/container/teslamate
Grafana image: https://github.com/jlestel/teslamate/pkgs/container/teslamate

It build only amd64 as others need a thrid party not everyone has install.

Let me know what you think about that.

JakobLichterfeld

Thanks again for refactoring and your dedicated work.

I added once again my small findings.

Additionally, two general points still prevent me from merging:

I would vote to include - '!.github/**' # Important: Exclude PRs related to .github from auto-run in all workflows
Please lint the files, as there are some trailing whitespaces for example

.github/workflows/ghcr_build.yml

.github/actions/merge/action.yml

.github/workflows/buildx.yml

.github/workflows/ghcr_build.yml

import

JakobLichterfeld

lgtm

This reverts commit b65b3f3.

JakobLichterfeld · 2023-11-17T08:08:49Z

@jlestel Sorry, merging on mobile in the middle of the night without testing in our repo was not the best idea. I suggest opening a CI branch, including the branch in the workflow triggers, and testing it in our repo, as local testing of GitHub workflows is limited in terms of secrets and GitHub usernames. The issue lies within the login to pull buildx.

jlestel · 2023-11-17T08:50:14Z

@jlestel Sorry, merging on mobile in the middle of the night without testing in our repo was not the best idea. I suggest opening a CI branch, including the branch in the workflow triggers, and testing it in our repo, as local testing of GitHub workflows is limited in terms of secrets and GitHub usernames. The issue lies within the login to pull buildx.

No problem. Yes it will be better to test it with a dedicated branch as well as on our repositories. Keep me informed.

* Upgrade CI * Keep lint and test on tags * Keep actions/checkout@v3 * Consider CI on branches named v* * Fix write packages * Distinct Elixir CI and PR Build * Rename build * Fix type * Add workflow_dispatch * Add timeout to prevent stuck jobs * Add actions for buildx and merge * fix username ? * Add grafana images from PR * fix typo * Finih merge action refactoring * Exclude PRs related to .github from auto-run * Don't try to auth on DockerHub when we can't * tmp allow myself to execute * Update ghcr_build.yml to be able to write on packages from public fork * Revert ghcr_build.yml * Update ghcr_build.yml * Update to build on source repo * Update ghcr_build.yml to test * Use correct repo with REGISTRY_IMAGE * Update ghcr_build.yml * Update ghcr_build.yml * Update ghcr_build.yml * Update ghcr_build.yml * Update ghcr_build.yml * Use dynamic variables to work in all repos * Purge in source repo * Update ghcr_build.yml * Update ghcr_build.yml * Update ghcr_build.yml * Update action.yaml * Update ghcr_build.yml * Update buildx.yml * Update action.yaml * Fix vars * Trace digests * Update action.yml * Update action.yml * Fix typo * Fox repo name * Update buildx.yml * Update action.yaml * Fix repository name * Run on any branch pushed * Exclude PRs related to .github from auto-run * Add buildjet dependency * lint * lint actions * fix: links in dashboards --------- Co-authored-by: Julien <[email protected]>

This reverts commit b65b3f3.

Julien added 5 commits November 15, 2023 03:30

Upgrade CI

3df37d3

Keep lint and test on tags

66fd791

Keep actions/checkout@v3

600f480

Consider CI on branches named v*

39861fd

Fix write packages

6e7af96

Julien added 2 commits November 15, 2023 09:20

Distinct Elixir CI and PR Build

5aaedf8

Rename build

476caf5

Fix type

3a9fb01

JakobLichterfeld requested changes Nov 15, 2023

View reviewed changes

.github/workflows/ghcr_build.yml Show resolved Hide resolved

.github/workflows/ghcr_build.yml Outdated Show resolved Hide resolved

.github/workflows/ghcr_build.yml Show resolved Hide resolved

JakobLichterfeld assigned jlestel Nov 15, 2023

JakobLichterfeld added the github_actions Pull requests that update Github_actions code label Nov 15, 2023

Julien added 2 commits November 15, 2023 10:13

Add workflow_dispatch

54a0e10

Add timeout to prevent stuck jobs

598d639

Julien added 3 commits November 15, 2023 14:30

Add actions for buildx and merge

1e071ab

fix username ?

ad6d00c

Add grafana images from PR

c028a3e

JakobLichterfeld requested changes Nov 16, 2023

View reviewed changes

fix typo

4985f78

JakobLichterfeld reviewed Nov 16, 2023

View reviewed changes

Julien added 2 commits November 16, 2023 12:40

Finih merge action refactoring

e3d7222

Exclude PRs related to .github from auto-run

20b84bb

jlestel requested a review from JakobLichterfeld November 16, 2023 12:03

jlestel added 12 commits November 16, 2023 17:03

Update buildx.yml

e51fd86

Update action.yaml

65f9582

Fix vars

c82e416

Trace digests

91421f5

Update action.yml

36ee0b6

Update action.yml

a9c8861

Fix typo

3746d9f

Fox repo name

e771097

Update buildx.yml

4033fcf

Update action.yaml

107e5c6

Fix repository name

c45e127

Run on any branch pushed

8e7a14d

JakobLichterfeld requested changes Nov 16, 2023

View reviewed changes

Julien and others added 6 commits November 16, 2023 19:50

Exclude PRs related to .github from auto-run

1b3824e

Add buildjet dependency

c291627

lint

ce2944b

lint actions

0a832d8

Merge pull request #2 from teslamate-org/master

ac00f1f

import

fix: links in dashboards

c34b8aa

JakobLichterfeld approved these changes Nov 17, 2023

View reviewed changes

JakobLichterfeld merged commit b65b3f3 into teslamate-org:master Nov 17, 2023
1 of 2 checks passed

JakobLichterfeld added a commit that referenced this pull request Nov 17, 2023

Revert "[CI] Build PR images on GHCR (#3432)"

6294402

This reverts commit b65b3f3.

JakobLichterfeld mentioned this pull request Nov 17, 2023

Revert "[CI] Build PR images on GHCR" #3441

Merged

JakobLichterfeld added a commit that referenced this pull request Nov 17, 2023

Revert "[CI] Build PR images on GHCR (#3432)" (#3441)

fdb8b6c

This reverts commit b65b3f3.

JakobLichterfeld mentioned this pull request Nov 18, 2023

CI Build PR images on GHCR #3450

Closed

JakobLichterfeld added a commit that referenced this pull request Feb 9, 2024

Revert "[CI] Build PR images on GHCR (#3432)" (#3441)

7ddf24f

This reverts commit b65b3f3.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CI] Build PR images on GHCR #3432

[CI] Build PR images on GHCR #3432

jlestel commented Nov 15, 2023 •

edited

Loading

JakobLichterfeld commented Nov 15, 2023

jlestel commented Nov 15, 2023

JakobLichterfeld left a comment

DrMichael commented Nov 15, 2023

jlestel commented Nov 15, 2023

adriankumpf commented Nov 15, 2023

brianmay commented Nov 15, 2023

JakobLichterfeld commented Nov 16, 2023

JakobLichterfeld left a comment •

edited

Loading

JakobLichterfeld left a comment

jlestel commented Nov 16, 2023

jlestel commented Nov 16, 2023

JakobLichterfeld left a comment

JakobLichterfeld left a comment

JakobLichterfeld commented Nov 17, 2023 •

edited

Loading

jlestel commented Nov 17, 2023

[CI] Build PR images on GHCR #3432

[CI] Build PR images on GHCR #3432

Conversation

jlestel commented Nov 15, 2023 • edited Loading

JakobLichterfeld commented Nov 15, 2023

jlestel commented Nov 15, 2023

JakobLichterfeld left a comment

Choose a reason for hiding this comment

DrMichael commented Nov 15, 2023

jlestel commented Nov 15, 2023

adriankumpf commented Nov 15, 2023

brianmay commented Nov 15, 2023

JakobLichterfeld commented Nov 16, 2023

JakobLichterfeld left a comment • edited Loading

Choose a reason for hiding this comment

JakobLichterfeld left a comment

Choose a reason for hiding this comment

jlestel commented Nov 16, 2023

jlestel commented Nov 16, 2023

JakobLichterfeld left a comment

Choose a reason for hiding this comment

JakobLichterfeld left a comment

Choose a reason for hiding this comment

JakobLichterfeld commented Nov 17, 2023 • edited Loading

jlestel commented Nov 17, 2023

jlestel commented Nov 15, 2023 •

edited

Loading

JakobLichterfeld left a comment •

edited

Loading

JakobLichterfeld commented Nov 17, 2023 •

edited

Loading