Working towards a single API

[bcl]

These are my notes on discussions about moving to a single API for local use with composer-cli, and for remote like cloudapi and cockpit-composer. Updated on November 15, 2024 with current status and links to open PRs

Overall goal is to have a single API, probably an extended cloudapi, that all the tools can use, resulting in less code duplication.

Some of the features that the weldr api were designed around no longer make sense to support. This includes:

• blueprint storage/history/diff
• sources that apply to every build for a distro
• storage of compose results longer than what's needed to be uploaded to a final location (one of which could be a local path)

The epic linking to all the jira issues is here - https://issues.redhat.com/browse/COMPOSER-2131

Current status as of osbuild-composer v125

• cloudapi listens to /run/cloudapi/api.socket when osbuild-composer is used on-premisis
• blueprints can be used to start cloudapi composes instead of the customizations
• cloudapi composes can be accessed locally, in the /var/lib/osbuild-composer/artifacts/, if the service is started with OSBUILD_LOCALSAVE=1 in the environment, and an upload type of 'localsave' is used in the cloudapi compose request.

There are a considerable number of pending PRs that need reviews, I've listed them in order of priority. I've tried to keep them as stand-alone as possible. The most entangled one is the job deletion PR which is currently left as a draft.

osbuild-composer

Add Architecture field to cloudapi blueprint schema

• cloudapi: Add architecture support to the optional Blueprint osbuild-composer#4374
• https://issues.redhat.com/browse/RHEL-60125

This makes it match the behavior of the blueprint in the weldr api.

List compose jobs

• List cloudapi compose jobs osbuild-composer#4425
• https://issues.redhat.com/browse/RHEL-60120

Cloudapi needs to be able to list the jobs it knows about. Job deletion depends on this one being merged first.

Include compose request with metadata response

• Return compose request details as part of metadata response osbuild-composer#4451
• https://issues.redhat.com/browse/RHEL-60120

Currently there is no way to know anything about the blueprint used to create a cloudapi compose, which makes it impossible to differentiate between composes without looking at the manifest.

List the distro/arch/image-types supported on cloudapi

• cloudapi: Add /distributions/{arch} to cloudapi osbuild-composer#4336
• https://issues.redhat.com/browse/RHEL-60133

This returns a map of all the supported distros so that users know what's possible to use when creating a compose.

Depsolve blueprints and search for packages

• Add depsolving and package search to cloudapi osbuild-composer#4390
• https://issues.redhat.com/browse/RHEL-60125
• https://issues.redhat.com/browse/RHEL-60136

This allows users to see what a blueprint will pull in, and allows searching for packages in a similar way to the weldr api.

Delete compose jobs

• Add cloudapi routes for listing and deleting composes osbuild-composer#4154
• https://issues.redhat.com/browse/RHEL-60120

This depends on osbuild/osbuild-composer#4425 and already has some stalled discussions on it. I'd prefer to get the above PRs sorted out before revisiting this.

weldr-client

Basic cloudapi support, starting compose, waiting for compose

• Add basic cloudapi support weldr-client#150
• https://issues.redhat.com/browse/RHEL-60123
• https://issues.redhat.com/browse/RHEL-60124
• https://issues.redhat.com/browse/RHEL-60134

This lays the groundwork for supporting the other new cloudapi routes. I have some of that working, but it all depends on this piece first. With this change you can start cloud composes using a local blueprint file instead of the ones stored in the weldr api.

[supakeen]

Some (random) thoughts on this:

1. Is anyone 'else' using the weldr API that we know of?
2. @croissanne is working on the webservice blueprints, I'm not a 100% sure on how far along this is but it might make sense to have a blueprint -> webservice blueprint -> composerequest translation internally.

Will we move blueprint parsing into the API or keep it in the client(s)?

[bcl]

I know of at least one user of weldr-client, who filed a bug a while back. It exists so we have to assume it is used.
Blueprints are not parsed in the client, one of my goals for composer-cli was to keep it loosely coupled to the backend so that it doesn't end up getting out of sync. The client passes toml/json to the server, the server parses it and returns errors to the client.

[croissanne]

1. random thought but if we really wanted to we could just let the weldr api translate & forward requests to the cloudapi, probably not worth it though. We should remove it in rhel 10 maybe but keep in rhel9? That said do we support direct talking to the unix socket? I always thought we only supported weldr-client or cockpit-composer.
2. Personally I think it's fine if the client parses the blueprint/deals with it, as long as 'parsing' for the weldr-client means just converting toml to json.

[teg]

Love to see this making progress!

I have a comment on a very small point: the support for freezing blueprints might be implemented via a feature I had hoped to do ages and ages ago:

Our current "compose" end point does two things internally: "depsolve" (which is essentially the same as "freeze")and "build". If we were to expose these as two separate calls (requiring state between them), we would support freezing (roughly) and it would also allow our cloud UI to be more powerful: we could depsolve before the review screen allowing us to show more info before building the image as well as handling more errors (caused by depsolve) before the build even starts.

[bcl]

Some code to add blueprints to cloudapi compose requests - osbuild/osbuild-composer#3757