Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix CI after the connector package split #464

Closed
wants to merge 4 commits into from
Closed

Fix CI after the connector package split #464

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
97168a5
Fix CI after the connector package split
jackyhu-db Nov 1, 2024
bb4e5dc
Fix syntax
jackyhu-db Nov 1, 2024
915e550
Refactor action to add resuable build workflow
jackyhu-db Nov 1, 2024
21c982f
Change checkout to v3
jackyhu-db Nov 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 76 additions & 0 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
name: Build and validate package

on:
workflow_call:
inputs:
target_path:
description: "The path of the package or service to build"
required: true
type: string
run_test:
description: "Run lint and test"
required: true
type: boolean

jobs:
build_package:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [ 3.8, 3.9, "3.10", "3.11" ]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should consider the change we made in dbt, as Python 3.8 is end of life, Python 3.12 has been out for a while, and python 3.13 was released last month.

steps:
- name: Set up python ${{ matrix.python-version }}
id: setup-python
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
#----------------------------------------------
# ----- install & configure poetry -----
#----------------------------------------------
- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
installer-parallel: true

#----------------------------------------------
# load cached venv if cache exists
#----------------------------------------------
- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v2
with:
path: .venv
key: venv-${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-${{ github.event.repository.name }}-${{ hashFiles('**/poetry.lock') }}
#----------------------------------------------
# install dependencies if cache does not exist
#----------------------------------------------
- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: poetry install --no-interaction --no-root
working-directory: ${{ github.workspace }}/${{ github.event.inputs.target_path }}

#----------------------------------------------
# install your root project, if required
#----------------------------------------------
- name: Install library
run: poetry install --no-interaction
working-directory: ${{ github.event.inputs.target_path }}

- name: Check-linting
if: ${{ inputs.run_test }}
run: poetry run black --check src
working-directory: ${{ github.event.inputs.target_path }}

- name: Run unit test
if: ${{ inputs.run_test }}
run: poetry run python -m pytest tests/unit
working-directory: ${{ github.event.inputs.target_path }}

# - name: Check types
# if: ${{ inputs.run_test }}
# run: |
# mkdir .mypy_cache # Workaround for bad error message "error: --install-types failed (no mypy cache directory)"; see https://github.com/python/mypy/issues/10768#issuecomment-2178450153
# poetry run mypy --install-types --non-interactive src
# working-directory: ${{ github.event.inputs.target_path }}
160 changes: 8 additions & 152 deletions .github/workflows/code-quality-checks.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,164 +2,20 @@ name: Code Quality Checks
on:
push:
branches:
- main
- PECO-1803/connector-split
pull_request:
branches:
- main
- PECO-1803/connector-split
jobs:
run-unit-tests:
validate-connector-core:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [3.8, 3.9, "3.10", "3.11"]
steps:
#----------------------------------------------
# check-out repo and set-up python
#----------------------------------------------
- name: Check out repository
uses: actions/checkout@v2
- name: Set up python ${{ matrix.python-version }}
id: setup-python
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
#----------------------------------------------
# ----- install & configure poetry -----
#----------------------------------------------
- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
installer-parallel: true

#----------------------------------------------
# load cached venv if cache exists
#----------------------------------------------
- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v2
with:
path: .venv
key: venv-${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-${{ github.event.repository.name }}-${{ hashFiles('**/poetry.lock') }}
#----------------------------------------------
# install dependencies if cache does not exist
#----------------------------------------------
- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: poetry install --no-interaction --no-root
#----------------------------------------------
# install your root project, if required
#----------------------------------------------
- name: Install library
run: poetry install --no-interaction
#----------------------------------------------
# run test suite
#----------------------------------------------
- name: Run tests
run: poetry run python -m pytest tests/unit
check-linting:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [3.8, 3.9, "3.10"]
steps:
#----------------------------------------------
# check-out repo and set-up python
#----------------------------------------------
- name: Check out repository
uses: actions/checkout@v2
- name: Set up python ${{ matrix.python-version }}
id: setup-python
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
#----------------------------------------------
# ----- install & configure poetry -----
#----------------------------------------------
- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
installer-parallel: true

#----------------------------------------------
# load cached venv if cache exists
#----------------------------------------------
- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v2
with:
path: .venv
key: venv-${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-${{ github.event.repository.name }}-${{ hashFiles('**/poetry.lock') }}
#----------------------------------------------
# install dependencies if cache does not exist
#----------------------------------------------
- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: poetry install --no-interaction --no-root
#----------------------------------------------
# install your root project, if required
#----------------------------------------------
- name: Install library
run: poetry install --no-interaction
#----------------------------------------------
# black the code
#----------------------------------------------
- name: Black
run: poetry run black --check src
uses: actions/checkout@v3

check-types:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [3.8, 3.9, "3.10"]
steps:
#----------------------------------------------
# check-out repo and set-up python
#----------------------------------------------
- name: Check out repository
uses: actions/checkout@v2
- name: Set up python ${{ matrix.python-version }}
id: setup-python
uses: actions/setup-python@v2
- name: Build
uses: ./.github/workflows/build.yml
with:
python-version: ${{ matrix.python-version }}
#----------------------------------------------
# ----- install & configure poetry -----
#----------------------------------------------
- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
installer-parallel: true
target_path: "databricks_sql_connector_core"
run_test: true

#----------------------------------------------
# load cached venv if cache exists
#----------------------------------------------
- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v2
with:
path: .venv
key: venv-${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-${{ github.event.repository.name }}-${{ hashFiles('**/poetry.lock') }}
#----------------------------------------------
# install dependencies if cache does not exist
#----------------------------------------------
- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: poetry install --no-interaction --no-root
#----------------------------------------------
# install your root project, if required
#----------------------------------------------
- name: Install library
run: poetry install --no-interaction
#----------------------------------------------
# mypy the code
#----------------------------------------------
- name: Mypy
run: |
mkdir .mypy_cache # Workaround for bad error message "error: --install-types failed (no mypy cache directory)"; see https://github.com/python/mypy/issues/10768#issuecomment-2178450153
poetry run mypy --install-types --non-interactive src
Loading