Automated CI/CD pipeline that deploys a Python application to a Compute Engine VM and runs it on a schedule — built entirely on Google Cloud services with no manual steps after code push.
| Component | Service |
|---|---|
| Version Control | GitHub |
| CI/CD Engine | Google Cloud Build |
| Scheduler | Google Cloud Scheduler |
| Compute | Google Compute Engine |
| Secrets | Google Secret Manager |
Deployment Pipeline (cloudbuild.yaml)
Triggered on every push to the repository. Fetches config from Secret Manager, copies main.py, run.sh, and config.json to the VM at /data/Mine, and runs the script once to verify.
Scheduled Execution (cloudbuild-scheduler.yaml)
Triggered by Cloud Scheduler every 15 minutes via HTTP + OIDC authentication. SSHes into the VM and runs run.sh → main.py.
Git Push
│
▼
Cloud Build (Deploy)
├── Fetch config from Secret Manager
├── Copy files to VM /data/Mine
└── Run script
Cloud Scheduler (every 15 min)
│ HTTP + OIDC
▼
Cloud Build (Run)
│ SSH
▼
VM → run.sh → main.py → output.log
- All credentials stored in Secret Manager — nothing hardcoded in code or pipelines
- Cloud Scheduler authenticates via OIDC tokens — no static keys
- IAM least-privilege roles used across all services
- Config deleted from build environment after each deployment
Detailed pipeline documentation with architecture diagrams:
- VM Pipeline v1 — Basic deploy + scheduled execution
- VM Pipeline v2 — Adds dev→prod test gate with auto-promotion
- Airflow Pipeline — Cloud Composer / Airflow variant
├── main.py # Python application
├── run.sh # Shell wrapper for execution
├── requirements.txt # Python dependencies
├── cloudbuild.yaml # Deployment pipeline
├── cloudbuild-scheduler.yaml # Scheduled execution pipeline
└── docs/
├── pipeline-vm-v1.md
├── pipeline-vm-v2.md
├── pipeline-airflow.md
└── pipeline_diagram.png