Add Triggers to provider information

Body

We do not have triggers information exposed in provider info but some Providerst already expose some (for example cncf.kubernetes).

What needs to be done:

add triggers to provider.yaml schema add triggers to provider.yaml schema #29950
add triggers to provider_info schema Addd triggers and transfers to provider_info.schema #33654
expose triggers via Providers Manager Add more checks for trigger module #31278
add CLI triggers command to airflow providers Add more checks for trigger module #31278
add automated exposure of triggers in the docs
check trigge classes are present in provider.yaml in unit tests

Committer

I acknowledge that I am a maintainer/committer of the Apache Airflow project.

@eladkal

cc: @eladkal

@potiuk

hi @potiuk I don't see trigger information as part of providers.yaml for cncf.kubernetes file too. Am i looking at the wrong place?

https://github.com/apache/airflow/blob/main/airflow/providers/cncf/kubernetes/provider.yaml#L76-L112

Yep. That's the point. We don't have them there and we need to add them :)

I don't think I understood the changes in provider_info_schema
do you mean the below file? what changes have to go into this file to support triggers?
https://github.com/apache/airflow/blob/main/airflow/provider_info.schema.json

I will add changes in the below file to include the triggers change.
https://github.com/apache/airflow/blob/main/airflow/provider.yaml.schema.json

Those Schemas (JSON schemas) define what is the allowed content of the provider.yaml (provider.yaml.schema.json) and what kind of information can be exposed via provider_info entrypoint in the provider packages (provier_info.schema.json). Both need to be updated to add optional "triggerer" information.

@potiuk

the provider_info.schema.json file doesn't contain information about hooks, sensors, transfers, or operators as such. Does it have to contain triggers information alone?
could you explain to me why triggers alone have to be included? @potiuk

We expose the information there also for connections and other things:

    "connection-types": {
      "type": "array",
      "description": "Map of connection types mapped to hook class names.",
      "items": {
        "type": "object",
        "properties": {
          "connection-type": {
            "description": "Type of connection defined by the provider",
            "type": "string"
          },
          "hook-class-name": {
            "description": "Hook class name that implements the connection type",
            "type": "string"
          }
        }
      },
      "required": ["connection-type", "hook-class-name"]
    },
    "extra-links": {
      "type": "array",
      "description": "Operator class names that provide extra link functionality",
      "items": {
        "type": "string"
      }
    },
   ...

This is what I call "core extensions" - i.e. the ways how providers are adding implementation of capabilities that Airflow exposes as public interface. It is nicely summarized here (this page is automatically generated based on the provider.yaml information, but also it is exposed for runtime query via CLI (airflow providers subcommands such as hooks, auth etc.)

This information is automatically discovered via ProvidersManager based on the information exposed via "get_provider_info" entrypoint (Providers Manager automatically queries the installed providers to get that information and it uses "provider_info.schema" to know how to query (simply get_provider_info returns a dict that conforms to the schema which is selected subset of information in provider.yaml.

https://airflow.apache.org/docs/apache-airflow-providers/core-extensions/index.html

This is in order to be able to actively "query" the providers and figure out in automated way wha

@Bowrna

@Bowrna are you working on any of the other sub tasks listed in the issue?

@eladkal

I will pick one more task from this list and start working on it before today @eladkal

@eladkal

can expect delay at my side ( for a week) as i am relocating home. sorry @eladkal i thought i could find time in between and check this, but i am wrong.

I think we can mark the below task as done PR #30822

expose triggers via Providers Manager
add CLI triggers command to airflow providers

Marked :)

check trigge classes are present in provider.yaml in unit tests -> that is already there.

Addd triggers and transfers to provider_info.schema

Addd triggers and transfers to provider_info.schema (#33654)

Hi, I took a dig at finishing this in #71975. Would appreciate any feedback!

Navigation Menu

Repository navigation

Add Triggers to provider information #29918

Body

Committer

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Add Triggers to provider information #29918

Description

Body

Committer

Activity

potiuk commented on Mar 4, 2023

Bowrna commented on Mar 5, 2023

potiuk commented on Mar 5, 2023

Bowrna commented on Mar 7, 2023

potiuk commented on Mar 7, 2023

Bowrna commented on Mar 8, 2023

potiuk commented on Mar 8, 2023

eladkal commented on May 5, 2023

Bowrna commented on May 5, 2023

Bowrna commented on May 11, 2023

pankajastro commented on May 15, 2023

potiuk commented on May 15, 2023

potiuk commented on Aug 23, 2023

subhramit commented on Aug 22, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions