provider: added k3d provider and node lifecycle handlers #441

harshanarayana · 2024-07-13T14:35:25Z

What type of PR is this?

/kind feature

What this PR does / why we need it:

This PR enables a new E2EClusterProvider for k3d based infra.

This change includes the following changes and features.

Added a new Interface type E2EClusterProviderWithImageLoaderAndNodeLifecycle which can be used to setup providers that extend the cluster lifecycle function around the nodes.
Enabled a k3d based provider with support for Node lifecycle management.
Existing Image loader related function and interfaces were augmented with args ...string to be able to provide additional arguments in case if the image load handlers need some of the additional config.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

I have also extended the E2EClusterProviderWithImageLoader 's Image load related methods to take an additional arg ...string value to account for additional options that one can pass while performing image load operation alone.

This comes in handy with k3d where it provides a few different mode of loading images and cleanup workflows.

This change also doesn't break any existing API contract extended by the envfuncs.

Does this PR introduce a user-facing change?

enabled a new `E2eClusterProvider` for `k3d` that can be used as a new cluster provider for running e2e tests.

Additional documentation e.g., Usage docs, etc.:

NA

harshanarayana · 2024-07-13T14:41:35Z

/cc @cpanato

harshanarayana · 2024-07-14T16:10:56Z

go install of k3d is failing due to an issue similar to k3d-io/k3d#1324

harshanarayana · 2024-07-14T16:30:43Z

/hold

Holding this under the k3d install issue is resolved.

harshanarayana · 2024-07-16T05:51:03Z

support/utils/command.go

+	p := commandRunner.NewProc(installCommand)
+	p.SetStdout(&stdout)
+	p.SetStderr(&stderr)
+	result := p.Run()
+	if result.Err() != nil {
+		return "", fmt.Errorf("failed to install %s: %s: \n %s", pPath, result.Result(), stderr.String())


Adding these to be able to capture and log the install failures better during the provider install.

harshanarayana · 2024-07-16T05:53:52Z

/unhold

cpanato

/lgtm
thanks

k8s-ci-robot · 2024-07-16T11:27:39Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cpanato, harshanarayana

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [cpanato,harshanarayana]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

cpanato

/lgtm

vladimirvivien

@harshanarayana this a great (massive) PR.
Thanks for doing this and I am sure the community will appreciate the k3d integration.
I left some comments and we can work on getting this to the finish line.

examples/k3d/main_test.go

pkg/stepfuncs/nodelifecycle_funcs.go

support/k3d/k3d.go

support/kind/kind.go

support/types.go

k8s-ci-robot · 2024-07-22T05:19:47Z

New changes are detected. LGTM label has been removed.

harshanarayana · 2024-07-22T05:19:57Z

@vladimirvivien PTAL. I has pushed the requested changes.

ShwethaKumbla

LGTM

vladimirvivien

Making progress @harshanarayana
Added some more comments.

support/k3d/k3d.go

vladimirvivien · 2024-08-01T13:17:06Z

pkg/utils/nodelifecycle.go

+
+// PerformNodeLifecycleOperation performs a node operation on a cluster. These operations can range from Add/Remove/Start/Stop.
+// This helper is re-used in both node lifecycle handler used as types.StepFunc or env.Func
+func PerformNodeLifecycleOperation(ctx context.Context, action support.NodeOperation, node *support.Node, args ...string) error {


Does this function only applies when using K3d or do you think orther providers will use that.
If only k3d, maybe it belongs in third_party package path if that is the case.

Other providers like minikube can support this.. And kwok technically can be simulated to enable node add and remove workflows. So it can work with other providers as well. We can do that even for kind. But we will have to use adhoc docker command to simulate power on and off kind of workflows

support/k3d/k3d.go

harshanarayana · 2024-08-15T13:39:08Z

@vladimirvivien PTAL when you can. Addressed the required changes in the message for wait for control plane.

vladimirvivien

More comments.

support/k3d/k3d.go

vladimirvivien · 2024-08-27T13:45:16Z

support/utils/command.go

-	p = commandRunner.RunProc("ls $GOPATH/bin")
-	if p.Err() != nil {
-		return "", fmt.Errorf("failed to install %s: %s", pPath, p.Err())
+	p = commandRunner.NewProc("ls $GOPATH/bin")


Do you think gexe could be improved to make output caputure easier ? If so, please open an issue.

k8s-triage-robot · 2024-11-29T13:00:13Z

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Mark this PR as fresh with /remove-lifecycle stale
Close this PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

harshanarayana · 2024-12-01T00:35:46Z

/remove-lifecycle stale

vladimirvivien · 2024-12-19T15:21:48Z

@harshanarayana is this still on your radar for 2025?

harshanarayana · 2024-12-20T12:47:33Z

I will get this done before we cut the christmas release..

This change includes the following changes and features. 1. Added a new Interface type `E2EClusterProviderWithLifeCycle` which can be used to setup providers that extend the cluster lifecycle function around the nodes. 2. Enabled a `k3d` based provider with support for Node lifecycle management. 3. Existing Image loader related function and interfaces were augmented with `args ...string` to be able to provide additional arguments in case if the image load handlers need some of the additional config.

harshanarayana · 2025-01-08T14:16:19Z

@vladimirvivien @cpanato PTAL. I have reorganised all the providers under third_party as suggested in the comments, and added backward compatibility type alias to make sure we do not break existing behaviours for the users who are already consuming this in some form.

harshanarayana · 2025-01-08T15:55:29Z

/test pull-e2e-framework-test

harshanarayana · 2025-01-09T13:03:27Z

/test pull-e2e-framework-test

k8s-ci-robot · 2025-01-09T13:51:52Z

@harshanarayana: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-e2e-framework-test	`6b424cb`	link	true	`/test pull-e2e-framework-test`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

k8s-ci-robot added kind/feature Categorizes issue or PR as related to a new feature. sig/testing Categorizes an issue or PR as relevant to SIG Testing. labels Jul 13, 2024

k8s-ci-robot requested review from ShwethaKumbla and vladimirvivien July 13, 2024 14:35

k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 13, 2024

harshanarayana force-pushed the feature/k3d-provider-support branch 2 times, most recently from 0c3a6e3 to d25b6ab Compare July 13, 2024 14:38

k8s-ci-robot requested a review from cpanato July 13, 2024 14:41

harshanarayana mentioned this pull request Jul 14, 2024

[BUG] Cannot install k3d v5.5.1 using Go 1.22.x due to replace directive k3d-io/k3d#1468

Closed

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 14, 2024

harshanarayana force-pushed the feature/k3d-provider-support branch 2 times, most recently from 3b76b81 to a70ae18 Compare July 16, 2024 05:50

k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 16, 2024

harshanarayana commented Jul 16, 2024

View reviewed changes

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 16, 2024

harshanarayana force-pushed the feature/k3d-provider-support branch from a70ae18 to afe220a Compare July 16, 2024 11:21

harshanarayana changed the title ~~provider: enable k3d based cluster provider~~ provider: added k3d provider and node lifecycle handlers Jul 16, 2024

harshanarayana force-pushed the feature/k3d-provider-support branch 2 times, most recently from 46696b8 to 6a755b5 Compare July 16, 2024 11:24

cpanato approved these changes Jul 16, 2024

View reviewed changes

k8s-ci-robot assigned cpanato Jul 16, 2024

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 16, 2024

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 16, 2024

cpanato reviewed Jul 16, 2024

View reviewed changes

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 16, 2024

vladimirvivien reviewed Jul 20, 2024

View reviewed changes

examples/k3d/main_test.go Show resolved Hide resolved

pkg/stepfuncs/nodelifecycle_funcs.go Outdated Show resolved Hide resolved

support/k3d/k3d.go Outdated Show resolved Hide resolved

support/kind/kind.go Outdated Show resolved Hide resolved

support/types.go Outdated Show resolved Hide resolved

harshanarayana force-pushed the feature/k3d-provider-support branch from 4c0f410 to 3a6934c Compare July 22, 2024 05:19

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 22, 2024

harshanarayana force-pushed the feature/k3d-provider-support branch from 3a6934c to 1597cf5 Compare July 22, 2024 05:25

ShwethaKumbla reviewed Jul 27, 2024

View reviewed changes

vladimirvivien reviewed Aug 1, 2024

View reviewed changes

harshanarayana force-pushed the feature/k3d-provider-support branch from 1597cf5 to 3d1be3f Compare August 15, 2024 13:38

vladimirvivien reviewed Aug 27, 2024

View reviewed changes

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Nov 29, 2024

k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Dec 1, 2024

harshanarayana added 2 commits January 8, 2025 19:20

refactor: re-organize the provider under third_party

6b424cb

harshanarayana force-pushed the feature/k3d-provider-support branch from 3d1be3f to 6b424cb Compare January 8, 2025 14:15

k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jan 8, 2025

harshanarayana mentioned this pull request Jan 8, 2025

runtimes: enable log export feature k3d-io/k3d#1471

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

provider: added k3d provider and node lifecycle handlers #441

provider: added k3d provider and node lifecycle handlers #441

harshanarayana commented Jul 13, 2024 •

edited

Loading

harshanarayana commented Jul 13, 2024

harshanarayana commented Jul 14, 2024

harshanarayana commented Jul 14, 2024

harshanarayana Jul 16, 2024

harshanarayana commented Jul 16, 2024

cpanato left a comment

k8s-ci-robot commented Jul 16, 2024

cpanato left a comment

vladimirvivien left a comment

k8s-ci-robot commented Jul 22, 2024

harshanarayana commented Jul 22, 2024

ShwethaKumbla left a comment

vladimirvivien left a comment

vladimirvivien Aug 1, 2024

harshanarayana Aug 1, 2024 •

edited

Loading

harshanarayana commented Aug 15, 2024

vladimirvivien left a comment

vladimirvivien Aug 27, 2024

k8s-triage-robot commented Nov 29, 2024

harshanarayana commented Dec 1, 2024

vladimirvivien commented Dec 19, 2024

harshanarayana commented Dec 20, 2024

harshanarayana commented Jan 8, 2025

harshanarayana commented Jan 8, 2025

harshanarayana commented Jan 9, 2025

k8s-ci-robot commented Jan 9, 2025

provider: added k3d provider and node lifecycle handlers #441

Are you sure you want to change the base?

provider: added k3d provider and node lifecycle handlers #441

Conversation

harshanarayana commented Jul 13, 2024 • edited Loading

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Additional documentation e.g., Usage docs, etc.:

harshanarayana commented Jul 13, 2024

harshanarayana commented Jul 14, 2024

harshanarayana commented Jul 14, 2024

harshanarayana Jul 16, 2024

Choose a reason for hiding this comment

harshanarayana commented Jul 16, 2024

cpanato left a comment

Choose a reason for hiding this comment

k8s-ci-robot commented Jul 16, 2024

cpanato left a comment

Choose a reason for hiding this comment

vladimirvivien left a comment

Choose a reason for hiding this comment

k8s-ci-robot commented Jul 22, 2024

harshanarayana commented Jul 22, 2024

ShwethaKumbla left a comment

Choose a reason for hiding this comment

vladimirvivien left a comment

Choose a reason for hiding this comment

vladimirvivien Aug 1, 2024

Choose a reason for hiding this comment

harshanarayana Aug 1, 2024 • edited Loading

Choose a reason for hiding this comment

harshanarayana commented Aug 15, 2024

vladimirvivien left a comment

Choose a reason for hiding this comment

vladimirvivien Aug 27, 2024

Choose a reason for hiding this comment

k8s-triage-robot commented Nov 29, 2024

harshanarayana commented Dec 1, 2024

vladimirvivien commented Dec 19, 2024

harshanarayana commented Dec 20, 2024

harshanarayana commented Jan 8, 2025

harshanarayana commented Jan 8, 2025

harshanarayana commented Jan 9, 2025

k8s-ci-robot commented Jan 9, 2025

harshanarayana commented Jul 13, 2024 •

edited

Loading

harshanarayana Aug 1, 2024 •

edited

Loading