Skip to content

Commit

Permalink
docs(kafka): add example for ingestion from confluent cloud (#3789)
Browse files Browse the repository at this point in the history
  • Loading branch information
anshbansal authored Dec 28, 2021
1 parent e7b9379 commit 895af09
Showing 1 changed file with 24 additions and 0 deletions.
24 changes: 24 additions & 0 deletions metadata-ingestion/source_docs/kafka.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,30 @@ sink:
# sink configs
```

### Connecting to Confluent Cloud

If using Confluent Cloud you can use a recipe like this. In this `consumer_config.sasl.username` and `consumer_config.sasl.password` are the API credentials that you get (in the Confluent UI) from your cluster -> Data Integration -> API Keys. `schema_registry_config.basic.auth.user.info` has API credentials for Confluent schema registry which you get (in Confluent UI) from Schema Registry -> API credentials.

```yml
source:
type: "kafka"
config:
connection:
bootstrap: "abc-defg.eu-west-1.aws.confluent.cloud:9092"
consumer_config:
security.protocol: "SASL_SSL"
sasl.mechanism: "PLAIN"
sasl.username: "CLUSTER_API_KEY_ID"
sasl.password: "CLUSTER_API_KEY_SECRET"
schema_registry_url: "https://abc-defgh.us-east-2.aws.confluent.cloud"
schema_registry_config:
basic.auth.user.info: "REGISTRY_API_KEY_ID:REGISTRY_API_KEY_SECRET"

sink:
# sink configs
```


## Config details

Note that a `.` is used to denote nested fields in the YAML recipe.
Expand Down

0 comments on commit 895af09

Please sign in to comment.