Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weโ€™ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add actionlint to workflow linter #39

Draft
wants to merge 73 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
d7f741b
adding actionlint to default workflow-linter rules
AmyLGalles Dec 27, 2024
04bad82
making output more specific
AmyLGalles Dec 27, 2024
d016a6d
making output more specific
AmyLGalles Dec 27, 2024
afc6f8b
committing for troubleshooting
AmyLGalles Dec 30, 2024
490499b
fixing typo
AmyLGalles Dec 30, 2024
58b2fa8
newline
AmyLGalles Dec 30, 2024
61d09ca
setting fail level to warning at first
AmyLGalles Dec 30, 2024
b11a498
fixing error on what the rule is compatible with
AmyLGalles Dec 30, 2024
3a666ac
debugging type annotation error
AmyLGalles Dec 30, 2024
dbe827f
debugging type annotation error
AmyLGalles Dec 30, 2024
adf8f4c
debugging type annotation error
AmyLGalles Dec 30, 2024
51b6ad3
code is better with notes
AmyLGalles Dec 30, 2024
d81c5ac
forcing breaking change to test
AmyLGalles Dec 30, 2024
d1fc89e
undo breaking change
AmyLGalles Dec 30, 2024
b19c121
Merge branch 'main' into agalles/bwwl-actionlint
AmyLGalles Dec 31, 2024
07c488f
adding error messages
AmyLGalles Dec 31, 2024
81667f5
fixed error messages
AmyLGalles Dec 31, 2024
0993f38
fixing windows detection
AmyLGalles Dec 31, 2024
bd7987a
catching unknown errors
AmyLGalles Dec 31, 2024
6b93e8d
catching unknown errors
AmyLGalles Dec 31, 2024
0187ed5
fixng string interpolation
AmyLGalles Dec 31, 2024
74228da
printing install errors
AmyLGalles Dec 31, 2024
a8526d4
catching more windows installations
AmyLGalles Dec 31, 2024
b07f9b3
Catching FileNotFound exception
pixman20 Dec 31, 2024
90a3988
Adding filename for action lint to run against
pixman20 Dec 31, 2024
f3cd785
removing debug print
pixman20 Dec 31, 2024
1b346d8
Updating comments about filename
pixman20 Dec 31, 2024
14240e3
Fixing tests
pixman20 Dec 31, 2024
5f3192f
Reverting package...json file changes
pixman20 Jan 2, 2025
0d1e5b4
Fixing typing issue
pixman20 Jan 2, 2025
69e8e02
removing extra step of running actionlint now that its part of bwwl
AmyLGalles Jan 2, 2025
966eec8
removing test print
AmyLGalles Jan 2, 2025
b06fd4e
adding a test for actionlint
AmyLGalles Jan 3, 2025
f0211d3
fixed tests
AmyLGalles Jan 3, 2025
23b8518
removing unnecessary no filename condition
AmyLGalles Jan 3, 2025
8dec317
Fixing incorrect workflow test fn name
pixman20 Jan 3, 2025
27cdbd0
adding error message for missing filename
AmyLGalles Jan 3, 2025
5859da3
removing test text
AmyLGalles Jan 3, 2025
b5b99fb
loading test content from a file
AmyLGalles Jan 3, 2025
4580470
loading test content from a file
AmyLGalles Jan 3, 2025
190669a
testing outcome of ci run
AmyLGalles Jan 3, 2025
06a20db
adding back correct test content
AmyLGalles Jan 3, 2025
dc83955
apply black
Eeebru Jan 6, 2025
fce0766
trying apt over apt get
AmyLGalles Jan 6, 2025
f0f0442
adding apt update
AmyLGalles Jan 6, 2025
0ad73ea
adding apt update
AmyLGalles Jan 6, 2025
4a83b5b
adding apt update
AmyLGalles Jan 6, 2025
32974a4
trying actionlint from source
AmyLGalles Jan 6, 2025
41b84e6
importing shutil
AmyLGalles Jan 6, 2025
a39f6ad
attempting to install actionlint from script
AmyLGalles Jan 6, 2025
6e9bb9a
fixing import for urllib
AmyLGalles Jan 6, 2025
fd1d66a
fixing import for urllib
AmyLGalles Jan 6, 2025
18e8e29
opening url as binary
AmyLGalles Jan 6, 2025
12a0488
debugging url open
AmyLGalles Jan 6, 2025
f14a1d4
fixing subprocess run
AmyLGalles Jan 6, 2025
05a1a1b
expanding error message
AmyLGalles Jan 6, 2025
f9ec8fd
fixing location of error
AmyLGalles Jan 6, 2025
868ad0a
fixing location of error
AmyLGalles Jan 6, 2025
afa2f85
fixing location of error
AmyLGalles Jan 6, 2025
ea111f2
fixing location of error
AmyLGalles Jan 6, 2025
42bddac
fixing location of error
AmyLGalles Jan 6, 2025
be462fb
fixing location of error
AmyLGalles Jan 6, 2025
eddc816
fixing location of error
AmyLGalles Jan 6, 2025
3ae7b8e
fixing location of error
AmyLGalles Jan 6, 2025
dd6e1ef
works but wonky
AmyLGalles Jan 7, 2025
0b74d96
works but wonky
AmyLGalles Jan 7, 2025
73fd662
separated installation into a separate function
AmyLGalles Jan 7, 2025
8018bb5
added an additional test
AmyLGalles Jan 7, 2025
6d3aef0
fixing return
AmyLGalles Jan 7, 2025
3803764
removing failure if actionlint fails
AmyLGalles Jan 8, 2025
6cd2900
adding back return
AmyLGalles Jan 8, 2025
25d8b89
Improve the test coverage for run_actionlint rule
Eeebru Jan 10, 2025
a5ff366
catching unknown error
AmyLGalles Jan 10, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Thumbs.db
*.sublime-workspace

# Visual Studio Code
.vscode/*
.vscode
!.vscode/settings.json
!.vscode/tasks.json
!.vscode/launch.json
Expand All @@ -35,3 +35,6 @@ flake.*
# Python
**/__pycache__/**
*.pyc
.venv/
.pytest_cache
.pytype
5 changes: 2 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
# Bitwarden Workflow Linter

Bitwarden's Workflow Linter is an extensible linter to apply opinionated organization-specific
GitHub Action standards. It was designed to be used alongside tools like
[action-lint](https://github.com/rhysd/actionlint) and
[yamllint](https://github.com/adrienverge/yamllint) to check for correct Action syntax and enforce
GitHub Action standards. It was designed to be used alongside
[yamllint](https://github.com/adrienverge/yamllint) to enforce
specific YAML standards.

To see an example of Workflow Linter in practice in GitHub Action, see the
Expand Down
1 change: 1 addition & 0 deletions settings.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,6 @@ enabled_rules:
- bitwarden_workflow_linter.rules.step_approved.RuleStepUsesApproved
- bitwarden_workflow_linter.rules.step_pinned.RuleStepUsesPinned
- bitwarden_workflow_linter.rules.underscore_outputs.RuleUnderscoreOutputs
- bitwarden_workflow_linter.rules.run_actionlint.RunActionlint

approved_actions_path: default_actions.json
1 change: 1 addition & 0 deletions src/bitwarden_workflow_linter/default_settings.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,6 @@ enabled_rules:
- bitwarden_workflow_linter.rules.step_approved.RuleStepUsesApproved
- bitwarden_workflow_linter.rules.step_pinned.RuleStepUsesPinned
- bitwarden_workflow_linter.rules.underscore_outputs.RuleUnderscoreOutputs
- bitwarden_workflow_linter.rules.run_actionlint.RunActionlint

approved_actions_path: default_actions.json
24 changes: 19 additions & 5 deletions src/bitwarden_workflow_linter/load.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,20 +39,32 @@ def __load_workflow_from_file(cls, filename: str) -> CommentedMap:
objects (depending on their location in the file).
"""
with open(filename, encoding="utf8") as file:
return yaml.load(file)
if not file:
raise WorkflowBuilderError(f"Could not load {filename}")
try:
return yaml.load(file)
except Exception as e:
raise WorkflowBuilderError(f"Error loading YAML file {filename}: {e}")

@classmethod
def __build_workflow(cls, loaded_yaml: CommentedMap) -> Workflow:
def __build_workflow(cls, filename: str, loaded_yaml: CommentedMap) -> Workflow:
"""Parse the YAML and build out the workflow to run Rules against.

Args:
filename:
The name of the file that the YAML was loaded from
loaded_yaml:
YAML that was loaded from either code or a file

Returns
A Workflow to run linting Rules against
"""
return Workflow.init("", loaded_yaml)
try:
if not loaded_yaml:
raise WorkflowBuilderError("No YAML loaded")
return Workflow.init("", filename, loaded_yaml)
except Exception as e:
raise WorkflowBuilderError(f"Error building workflow: {e}")

@classmethod
def build(
Expand All @@ -76,9 +88,11 @@ def build(
be loaded from disk
"""
if from_file and filename is not None:
return cls.__build_workflow(cls.__load_workflow_from_file(filename))
return cls.__build_workflow(
filename, cls.__load_workflow_from_file(filename)
)
elif not from_file and workflow is not None:
return cls.__build_workflow(workflow)
return cls.__build_workflow("", workflow)

raise WorkflowBuilderError(
"The workflow must either be built from a file or from a CommentedMap"
Expand Down
4 changes: 3 additions & 1 deletion src/bitwarden_workflow_linter/models/workflow.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,16 @@ class Workflow:
"""

key: str = ""
filename: Optional[str] = None
name: Optional[str] = None
on: Optional[CommentedMap] = None
jobs: Optional[Dict[str, Job]] = None

@classmethod
def init(cls: Self, key: str, data: CommentedMap) -> Self:
def init(cls: Self, key: str, filename: str, data: CommentedMap) -> Self:
init_data = {
"key": key,
"filename": filename,
"name": data["name"] if "name" in data else None,
"on": data["on"] if "on" in data else None,
}
Expand Down
108 changes: 108 additions & 0 deletions src/bitwarden_workflow_linter/rules/run_actionlint.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
"""A Rule to run actionlint on workflows."""

from typing import Optional, Tuple
import subprocess
import platform
import urllib.request
import os

from ..rule import Rule
from ..models.workflow import Workflow
from ..utils import LintLevels, Settings


def install_actionlint(platform_system: str) -> Tuple[bool, str]:
"""If actionlint is not installed, detects OS platform
and installs actionlint"""

error = f"An error occurred when installing Actionlint on {platform_system}"

if platform_system.startswith("Linux"):
return install_actionlint_source(error)
elif platform_system == "Darwin":
try:
subprocess.run(["brew", "install", "actionlint"], check=True)
return True, ""
except (FileNotFoundError, subprocess.CalledProcessError):
return False, f"{error} : check Brew installation"
elif platform_system.startswith("Win"):
try:
subprocess.run(["choco", "install", "actionlint", "-y"], check=True)
return True, ""
except (FileNotFoundError, subprocess.CalledProcessError):
return False, f"{error} : check Choco installation"
return False, error


def install_actionlint_source(error) -> Tuple[bool, str]:
"""Install Actionlint Binary from provided script"""
url = "https://raw.githubusercontent.com/rhysd/actionlint/main/scripts/download-actionlint.bash"
version = "1.6.17"
request = urllib.request.urlopen(url)
with open("download-actionlint.bash", "wb+") as fp:
fp.write(request.read())
try:
subprocess.run(["bash", "download-actionlint.bash", version], check=True)
return True, os.getcwd()
except (FileNotFoundError, subprocess.CalledProcessError):
return False, error


def check_actionlint(platform_system: str) -> Tuple[bool, str]:
"""Check if the actionlint is in the system's PATH."""
try:
subprocess.run(
["actionlint", "--version"],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
check=True,
)
return True, ""
except subprocess.CalledProcessError:
return (
False,
"Failed to install Actionlint, \
please check your package installer or manually install it",
)
except FileNotFoundError:
return install_actionlint(platform_system)


class RunActionlint(Rule):
"""Rule to run actionlint as part of workflow linter V2."""

def __init__(self, settings: Optional[Settings] = None) -> None:
self.message = "Actionlint must pass without errors"
self.on_fail = LintLevels.WARNING
self.compatibility = [Workflow]
self.settings = settings

def fn(self, obj: Workflow) -> Tuple[bool, str]:
if not obj or not obj.filename:
raise AttributeError(
"Running actionlint without a filename is not currently supported"
)

installed, location = check_actionlint(platform.system())
if installed:
if location:
result = subprocess.run(
[location + "/actionlint", obj.filename],
capture_output=True,
text=True,
check=False,
)
else:
result = subprocess.run(
["actionlint", obj.filename],
capture_output=True,
text=True,
check=False,
)
if result.returncode == 1:
return False, result.stdout
if result.returncode > 1:
return False, result.stdout
return True, ""
else:
return False, self.message
32 changes: 32 additions & 0 deletions tests/fixtures/test_workflow.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: test
on:
workflow_dispatch:
pull_request:

jobs:
job-key:
name: Test
runs-on: ubuntu-latest
steps:
- name: Test
run: echo test

call-workflow:
uses: bitwarden/server/.github/workflows/workflow-linter.yml@master

test-normal-action:
name: Download Latest
runs-on: ubuntu-20.04
steps:
- name: Checkout
uses: actions/checkout@2541b1294d2704b0964813337f33b291d3f8596b

- run: |
echo test

test-local-action:
name: Testing a local action call
runs-on: ubuntu-20.04
steps:
- name: local-action
uses: ./version-bump
36 changes: 36 additions & 0 deletions tests/fixtures/test_workflow_incorrect.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: test
on:
push:
branches:
-
path:
- "src/**"
workflow_dispatch:

jobs:
job-key:
name: Test
runs-on: ubuntu-latest
steps:
- name: Test
run: echo test

call-workflow:
uses: bitwarden/server/.github/workflows/workflow-linter.yml@master

test-normal-action:
name: Download Latest
runs-on: ubuntu-20.04
steps:
- name: Checkout
uses: actions/checkout@2541b1294d2704b0964813337f33b291d3f8596b

- run: |
echo test

test-local-action:
name: Testing a local action call
runs-on: ubuntu-20.04
steps:
- name: local-action
uses: ./version-bump
Loading