Upgrade storage integration test: Use V2 Archive ReaderWriter #6489

ekefan · 2025-01-06T00:51:30Z

Which problem is this PR solving?

Part of Upgrade storage integration tests to Storage v2 API #6366

Description of the changes

Incrementally swaps the fields of StorageIntegration to align with v2 storage api
- replaced ArchiveSpanReader with ArchiveTraceReader
- replaced ArchiveSpanWriter with ArchiveTraceWriter
Updates test functions accordingly to work with the updated fields

How was this change tested?

Checklist

I have read https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
I have signed all commits
I have added unit tests for the new functionality
I have run lint and test steps successfully
- for jaeger: make lint test
- for jaeger-ui: npm run lint and npm run test

plugin/storage/integration/cassandra_test.go

yurishkuro · 2025-01-06T02:54:16Z

looks like some archive tests are failing

    --- ❌ FAIL: TestElasticsearchStorage/ArchiveTrace (0.00s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x22fe57b]

goroutine 65 [running]:
testing.tRunner.func1.2({0x2532840, 0x3a10eb0})
	/opt/hostedtoolcache/go/1.23.4/x64/src/testing/testing.go:1632 +0x3fc
testing.tRunner.func1()
	/opt/hostedtoolcache/go/1.23.4/x64/src/testing/testing.go:1635 +0x6b6
panic({0x2532840?, 0x3a10eb0?})
	/opt/hostedtoolcache/go/1.23.4/x64/src/runtime/panic.go:785 +0x132
github.com/jaegertracing/jaeger/plugin/storage/integration.(*StorageIntegration).testArchiveTrace.func1(0x6c0f40?)
	/home/runner/work/jaeger/jaeger/plugin/storage/integration/integration.go:214 +0x47b
github.com/jaegertracing/jaeger/plugin/storage/integration.(*StorageIntegration).waitForCondition(0x2a7de18?, 0xc000523040, 0xc000439e58)
	/home/runner/work/jaeger/jaeger/plugin/storage/integration/integration.go:128 +0x165

ekefan · 2025-01-06T08:47:04Z

there's no new change with the recent push... just trying to update the branch.

I will get back to these issues later in my day.

ekefan · 2025-01-07T12:16:24Z

Hello @yurishkuro,

I can't find the cause of this bug, but for elastic search and open search, the testArchiveTrace test fails; the archive trace reader returns an empty trace...
I want to try updating the reader before the writer, but I'd be glad to get any help or way forward from you.
Thanks alot.

yurishkuro · 2025-01-07T14:46:54Z

Did you look at the logs? I see nil pointer segfaults, that should be easy to troubleshoot.

codecov · 2025-01-08T06:52:27Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 50.29%. Comparing base (b3be011) to head (54a0a07).

❗ There is a different number of reports uploaded between BASE (b3be011) and HEAD (54a0a07). Click for more details.

HEAD has 2 uploads less than BASE

Flag BASE (b3be011) HEAD (54a0a07)

cassandra-4.x-v1-manual 1 0

unittests 1 0

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #6489       +/-   ##
===========================================
- Coverage   96.26%   50.29%   -45.98%     
===========================================
  Files         372      188      -184     
  Lines       21282    11410     -9872     
===========================================
- Hits        20488     5739    -14749     
- Misses        607     5213     +4606     
- Partials      187      458      +271

Flag	Coverage Δ
badger_v1	`10.67% <ø> (ø)`
badger_v2	`2.78% <ø> (ø)`
cassandra-4.x-v1-manual	`?`
cassandra-4.x-v2-auto	`2.71% <ø> (ø)`
cassandra-4.x-v2-manual	`2.71% <ø> (ø)`
cassandra-5.x-v1-manual	`16.57% <ø> (ø)`
cassandra-5.x-v2-auto	`2.71% <ø> (ø)`
cassandra-5.x-v2-manual	`2.71% <ø> (ø)`
elasticsearch-6.x-v1	`20.23% <ø> (-0.01%)`	⬇️
elasticsearch-7.x-v1	`20.30% <ø> (-0.01%)`	⬇️
elasticsearch-8.x-v1	`20.46% <ø> (-0.01%)`	⬇️
elasticsearch-8.x-v2	`2.77% <ø> (ø)`
grpc_v1	`12.31% <ø> (ø)`
grpc_v2	`9.08% <ø> (ø)`
kafka-3.x-v1	`10.35% <ø> (ø)`
kafka-3.x-v2	`2.78% <ø> (ø)`
memory_v2	`2.78% <ø> (ø)`
opensearch-1.x-v1	`20.36% <ø> (+<0.01%)`	⬆️
opensearch-2.x-v1	`20.36% <ø> (+<0.01%)`	⬆️
opensearch-2.x-v2	`2.77% <ø> (-0.01%)`	⬇️
tailsampling-processor	`0.51% <ø> (ø)`
unittests	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Emmanuel Emonueje Ebenezer <[email protected]>

--------- Signed-off-by: Emmanuel Emonueje Ebenezer <[email protected]>

Signed-off-by: Emmanuel Emonueje Ebenezer <[email protected]>

ekefan · 2025-01-08T09:04:23Z

plugin/storage/integration/integration.go

+		if len(traces) > 0 {
+			actualTrace = traces[0]
+		} else {
+			actualTrace, err = s.ArchiveSpanReader.GetTrace(context.Background(), spanstore.GetTraceParameters{TraceID: tID})


with span reader, the archive span test(specifically) doesn't fail,
I have had no luck figuring out why it does fail with the archive trace reader,

ekefan · 2025-01-08T09:04:48Z

plugin/storage/integration/integration.go

+	TraceWriter        tracestore.Writer
+	TraceReader        tracestore.Reader
+	ArchiveTraceReader tracestore.Reader
+	ArchiveSpanReader  spanstore.Reader


the plan is not to get this merged, I just wanted to show you somthing:

the segfaults occured because the ArchiveTraceReader wasn't reading the spans, so `actualTrace1 remained nil, hence the errors when trying to read the spans

but that doesn't occur with the archive span reader...

a tip - if you don't think the PR is ready for merge, you can convert it to a Draft. I would still get a notification of all changes because I subscribe to all, but it won't be saying "Ready for review".

ekefan · 2025-01-08T09:20:29Z

I'll get more familiar with time, but git messed with me alot for this pr... alot of commits🙃🙂

ekefan requested a review from a team as a code owner January 6, 2025 00:51

ekefan requested a review from pavolloffay January 6, 2025 00:51

dosubot bot added area/storage v2 labels Jan 6, 2025

ekefan force-pushed the archive-reader-writer branch from 2ca2f23 to 9ad6870 Compare January 6, 2025 00:53

yurishkuro reviewed Jan 6, 2025

View reviewed changes

plugin/storage/integration/cassandra_test.go Outdated Show resolved Hide resolved

ekefan marked this pull request as draft January 6, 2025 02:26

yurishkuro approved these changes Jan 6, 2025

View reviewed changes

yurishkuro added the changelog:ci Change related to continuous integration / testing label Jan 6, 2025

yurishkuro marked this pull request as ready for review January 6, 2025 02:51

dosubot bot added storage/cassandra storage/elasticsearch labels Jan 6, 2025

yurishkuro enabled auto-merge (squash) January 6, 2025 02:52

yurishkuro mentioned this pull request Jan 6, 2025

Upgrade storage integration tests to Storage v2 API #6366

Open

6 tasks

yurishkuro disabled auto-merge January 6, 2025 02:54

ekefan force-pushed the archive-reader-writer branch from aaab620 to a43bc9a Compare January 7, 2025 09:16

ekefan added 9 commits January 8, 2025 09:37

Resolve merge conflict

94ad08c

Signed-off-by: Emmanuel Emonueje Ebenezer <[email protected]>

Resolve merge conflict

a24df6a

Signed-off-by: Emmanuel Emonueje Ebenezer <[email protected]>

Resolve merge conflicts

1802271

--------- Signed-off-by: Emmanuel Emonueje Ebenezer <[email protected]>

Resolve merge conflict

549aff2

Signed-off-by: Emmanuel Emonueje Ebenezer <[email protected]>

Resolve merge conflict

f807b46

Signed-off-by: Emmanuel Emonueje Ebenezer <[email protected]>

format code properly

9d8bd3a

Signed-off-by: Emmanuel Emonueje Ebenezer <[email protected]>

Fix lint errors

3315b6f

Signed-off-by: Emmanuel Emonueje Ebenezer <[email protected]>

Debug: Use archive span reader when trace reader fails

9370cae

Signed-off-by: Emmanuel Emonueje Ebenezer <[email protected]>

fix lint

42572fa

Signed-off-by: Emmanuel Emonueje Ebenezer <[email protected]>

Debug: Use archive span reader when trace reader fails

54a0a07

Signed-off-by: Emmanuel Emonueje Ebenezer <[email protected]>

ekefan force-pushed the archive-reader-writer branch from ccf0435 to 54a0a07 Compare January 8, 2025 08:46

ekefan commented Jan 8, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade storage integration test: Use V2 Archive ReaderWriter #6489

Upgrade storage integration test: Use V2 Archive ReaderWriter #6489

ekefan commented Jan 6, 2025 •

edited

Loading

yurishkuro commented Jan 6, 2025

ekefan commented Jan 6, 2025 •

edited

Loading

ekefan commented Jan 7, 2025 •

edited

Loading

yurishkuro commented Jan 7, 2025

codecov bot commented Jan 8, 2025 •

edited

Loading

ekefan Jan 8, 2025

ekefan Jan 8, 2025 •

edited

Loading

yurishkuro Jan 8, 2025

ekefan commented Jan 8, 2025

Upgrade storage integration test: Use V2 Archive ReaderWriter #6489

Are you sure you want to change the base?

Upgrade storage integration test: Use V2 Archive ReaderWriter #6489

Conversation

ekefan commented Jan 6, 2025 • edited Loading

Which problem is this PR solving?

Description of the changes

How was this change tested?

Checklist

yurishkuro commented Jan 6, 2025

ekefan commented Jan 6, 2025 • edited Loading

ekefan commented Jan 7, 2025 • edited Loading

yurishkuro commented Jan 7, 2025

codecov bot commented Jan 8, 2025 • edited Loading

Codecov Report

ekefan Jan 8, 2025

Choose a reason for hiding this comment

ekefan Jan 8, 2025 • edited Loading

Choose a reason for hiding this comment

yurishkuro Jan 8, 2025

Choose a reason for hiding this comment

ekefan commented Jan 8, 2025

ekefan commented Jan 6, 2025 •

edited

Loading

ekefan commented Jan 6, 2025 •

edited

Loading

ekefan commented Jan 7, 2025 •

edited

Loading

codecov bot commented Jan 8, 2025 •

edited

Loading

ekefan Jan 8, 2025 •

edited

Loading