Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Features are disabled when broker increases minimum RPC versions #4948

Open
5 of 7 tasks
emasab opened this issue Jan 8, 2025 · 0 comments
Open
5 of 7 tasks

Features are disabled when broker increases minimum RPC versions #4948

emasab opened this issue Jan 8, 2025 · 0 comments

Comments

@emasab
Copy link
Contributor

emasab commented Jan 8, 2025

Description

with KIP-896: it's planned that older
RPC versions. When connecting to such broker, librdkafka deactivates most features because the maximum version is almost always the same as the minimum version and corresponds to a version being removed.

How to reproduce

In rd_kafka_broker_set_api_versions can be overridden this way:

        size_t i;
        for (i = 0; i < api_cnt; i++) {
                if (apis[i].ApiKey == RD_KAFKAP_Produce)
                        apis[i].MinVer = 8;
                if (apis[i].ApiKey == RD_KAFKAP_ApiVersion)
                        apis[i].MinVer = 2;
                if (apis[i].ApiKey == RD_KAFKAP_FindCoordinator)
                        apis[i].MinVer = 1;
                if (apis[i].ApiKey == RD_KAFKAP_JoinGroup) {
                        apis[i].MinVer = -2;
                        apis[i].MaxVer = -2;
                }
                if (apis[i].ApiKey == RD_KAFKAP_SaslHandshake)
                        apis[i].MinVer = 2;
                if (apis[i].ApiKey == RD_KAFKAP_InitProducerId)
                        apis[i].MinVer = 1;
                if (apis[i].ApiKey == RD_KAFKAP_ListOffsets)
                        apis[i].MinVer = 1;
        }

It happens that all features are disabled.
test_minver_increased.log

Minimum required MinVer values are currently:

Produce 3
Fetch 4
ApiVersion 0
FindCoordinator 0
OffsetCommit 2
OffsetFetch 1
*JoinGroup 0
SyncGroup 0
Heartbeat 0
LeaveGroup 0
SaslHandshake 0
ListOffsets 1
InitProducerId 0
SaslAuthenticate 1

by comparing it to KIP 896 removed versions (non-baseline), all versions described in the KIP can be removed except for JoinGroup v0 because it disables the feature RD_KAFKA_FEATURE_BROKER_BALANCED_CONSUMER that isn't used in the code but it's public and users can detect it to decide if they can call subscribe or assign manually (even though it seems uncommon that an application supports both cases).

Checklist

Please provide the following information:

  • librdkafka version (release number or git tag): v1.x+ up to latest
  • Apache Kafka version: <REPLACE with e.g., 0.10.2.3>
  • librdkafka client configuration: any
  • Operating system: any
  • Provide logs (with debug=.. as necessary) from librdkafka
  • Provide broker log excerpts
  • Critical issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant