Repository navigation
HDDS-17602. Use different runners for some jobs - #11977
Conversation
chihsuan
left a comment
chihsuan
left a comment
There was a problem hiding this comment.
Thanks @adoroszlai looks good to me!
I left two questions mainly to confirm the intent.
| exclude: | ||
| - check: errorprone | ||
| fail-fast: false | ||
| fail-fast: true |
There was a problem hiding this comment.
- Enable
fail-fastforbasicchecks matrix.
Just curious, what's the reason for enabling fail-fast? I don't feel strongly either way, just want to understand the trade-off. 🙂
There was a problem hiding this comment.
Basic checks are mostly deterministic. Intermittency may be mainly due to infrastructure. Failures usually require new commit, and thus a new full run. They are also quick, so re-run does not cost much.
On the other hand, tests are more prone to flakiness, and some splits are very long. Cancelling a few 40 minute runs at 30 minutes because some flaky test in one split failed is a waste of resources.
We turned off fail-fast (HDDS-6464) when GitHub introduced "Re-run failed checks" -- before that, all checks had to be re-run anyway, so waiting for some to finish successfully did not make sense. I don't recall why we did not keep it for basic checks.
| include: | ||
| - os: ubuntu-24.04 | ||
| - java: 21 | ||
| os: macos-15 |
There was a problem hiding this comment.
Dropping this for forks makes sense given GitHub runner capacity. Just a thought, if we still want this coverage, maybe we could keep it for scheduled builds in master? Totally fine to drop it if it's no longer needed.
There was a problem hiding this comment.
Compilation on macos was added to catch regression in Shaded Jar build failure in case-insensitive filesystem (HDDS-10178) because new plexus-archiver changed behavior. I thought it may no longer be worth the runner minutes.
Keeping it in the scheduled build sounds like a good idea.
There was a problem hiding this comment.
Updated to run ubuntu-24.04 on push, but macos-15 on other events, so that fork runs can try macos via workflow_dispatch.
There was a problem hiding this comment.
* master: (892 commits) HDDS-16784. Run rebot in pod in kubernetes check (apache#11938) HDDS-17705. Stop Recon OM sync before closing the metadata manager (apache#11936) HDDS-17755. Use isSupportedBy() for OzoneManagerVersion check (apache#12002) HDDS-17770. Bump zizmorcore/zizmor-action to 0.8.1 (apache#11987) HDDS-16701. S3G: Avoid calling getS3Volume in DELETE_KEY (apache#11740) HDDS-16616. Replace Pair/Triple with domain objects in SCM node decommission code (apache#11984) HDDS-17500. Return x-amz-server-side-encryption on GET and HEAD for encrypted objects (apache#11972) HDDS-17478. Share aws-chunked parsing between SignedChunksInputStream and UnsignedChunksInputStream (apache#11992) HDDS-17410. Record STS revocation cutoff when revoking tenant access id (apache#11796) HDDS-17635. Datanode ReadBlock: a cancelled streaming call is not a container failure (apache#11891) HDDS-17706. Avoid retrying datanode usage query for unknown UUID (apache#11947) HDDS-17407. Allow for clock skew with revocation (apache#11952) HDDS-17421. SCM terminates when it applies Raft log entry written by pre-ZDU SCM finalization (apache#11925) HDDS-16951. Remove the per-call InfoBucket from rooted-ofs rename on FSO buckets (apache#11674) HDDS-17602. Use different runners for some jobs (apache#11977) HDDS-17409. Record STS revocation cutoff when revoking S3 secrets (apache#11794) HDDS-17515. Move KeyLifecycleService properties to a reconfigurable config group (apache#11912) HDDS-17465. Recon keeps the old container mappings of an overwritten key (apache#11762) HDDS-17503. [Recon] Report OM pending deletion as purgeable and snapshot-held size (apache#11855) HDDS-17581. Add TracingUtil span helpers and span-kind aware tracing proxies (apache#11852) ... Conflicts: hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/TestTrashOzoneFileSystem.java hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/service/TestDirectoryDeletingService.java hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/service/TestQuotaRepairTask.java
What changes were proposed in this pull request?
Minor CI improvements, posted in one PR to avoid full build for each small change.
ubuntu-slimupdate-jar-report.sh, used independencycheck, needed Maven only for getting Ozone version frompom.xml. Change the implementation to avoid this, so we can apply the improvements described in (1) todependency, too.rat.shfails ifmvnis not found (encountered this when trying to run it onubuntu-slim).fail-fastforbasicchecks matrix.ubuntu-26.04for acceptance and some basic checks. Different runner labels have separate queues, using different ones helps distribute the load a bit (wrt. only this workflow).macosrunner, which frequently fails to start due to resource shortage recently. Examples:https://issues.apache.org/jira/browse/HDDS-17602
How was this patch tested?
https://github.com/adoroszlai/ozone/actions/runs/42354416560