Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log the hbase attributes for one failing data point #1610

Conversation

breedx-splk
Copy link
Contributor

Relates to #1605

Because we're having intermittent test failures, I thought it might be nice to see the attributes for the given data point(s) that are failing. The output looks something like this:

Dec 16, 2024 3:25:34 PM io.opentelemetry.contrib.jmxscraper.assertions.MetricAssert lambda$logAttributes$4
INFO: hbase.region_server.queue.length -> attrs = [{region_server=> string_value: "1fdbc6238e8a"},{state=> string_value: "flush"}]
Dec 16, 2024 3:25:34 PM io.opentelemetry.contrib.jmxscraper.assertions.MetricAssert lambda$logAttributes$4
INFO: hbase.region_server.queue.length -> attrs = [{region_server=> string_value: "1fdbc6238e8a"},{state=> string_value: "compaction"}]

and the idea then is that you can turn this on for a while until we can figure out what aspect of the test is failing. Also open to other ideas on how to narrow it down.

@robsunday
Copy link
Contributor

robsunday commented Dec 18, 2024

This is needed functionality and it is good you brought this up. On my PR I added some test stabilization changes, but also included dumping metric data points together with attributes in case of attribute matching failure. This will provide us with complete unprocessed input data for failing metric.
If my changes will produce too verbose output then we may switch to your solution, for now I'd suggest put this PR on hold if you don't mind.

@@ -120,6 +120,7 @@ protected MetricsVerifier createMetricsVerifier() {
"hbase.region_server.queue.length",
metric ->
metric
.logAttributes()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is something that should be done automatically. Maybe just call it inside hasDataPointsWithAttributes() ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nah, this design assumed that you'd only want to turn this on in specific cases for certain metrics or whatever. Doing this automatically for every metric assertion would be too verbose.

@breedx-splk
Copy link
Contributor Author

Superseded by #1615

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants