Repository navigation
SqliteSpanExporter drops a span's status and events, so an exception that ended a run is lost (2.11.0)聽#7484
Description
馃敶 Required Information
Describe the Bug:
SqliteSpanExporter (telemetry/sqlite_span_exporter.py), the exporter that "exports spans to a local SQLite database" for local development, does not store a span's status or its events. OpenTelemetry keeps an exception in both: start_as_current_span sets the status to ERROR with the exception's description, and records an exception event with its type, message and stack trace. ADK does the same in trace_tool_call and node_tracing. The spans table has no column for either, and _row_to_readable_span rebuilds each span without them. So a run that fails leaves spans that read back as if nothing went wrong: status UNSET, no events. The error that stopped the run, the main thing a local trace is kept for, is lost.
Steps to Reproduce:
pip install "google-adk==2.11.0"(opentelemetry-sdk1.42.1)- Run the script below. It needs no key or network.
- It ends a span with an exception, exports it with
SqliteSpanExporter, and reads it back withget_all_spans_for_session.
Expected Behavior:
The span reads back with status ERROR, its description, and the exception event (type, message, stack trace).
Observed Behavior:
columns: ['span_id', 'trace_id', 'parent_span_id', 'name', 'start_time_unix_nano', 'end_time_unix_nano', 'session_id', 'invocation_id', 'attributes_json']
status read back: UNSET
events read back: []
The table on main is the same.
Environment Details:
- ADK Library Version (pip show google-adk): 2.11.0
- Desktop OS: Linux
- Python Version (python -V): 3.11.13
Model Information:
- Are you using LiteLLM: No
- Which model is being used: not relevant (any span)
馃煛 Optional Information
Regression: Not known.
Minimal Reproduction Code:
"""SqliteSpanExporter drops a span's status and events (exceptions)."""
import sqlite3
import tempfile
from pathlib import Path
from google.adk.telemetry.sqlite_span_exporter import SqliteSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
db = str(Path(tempfile.mkdtemp()) / "spans.db")
exporter = SqliteSpanExporter(db_path=db)
provider = TracerProvider()
provider.add_span_processor(SimpleSpanProcessor(exporter))
tracer = provider.get_tracer("repro")
try:
with tracer.start_as_current_span(
"call_llm", attributes={"gcp.vertex.agent.session_id": "s1"}
):
raise ValueError("model: claude-sonnet-9-9 not found")
except ValueError:
pass
columns = [r[1] for r in sqlite3.connect(db).execute("PRAGMA table_info(spans)")]
print("columns:", columns)
(span,) = exporter.get_all_spans_for_session("s1")
print("status read back:", span.status.status_code.name)
print("events read back:", list(span.events))Possible fix: add status_code, status_description and events_json columns (with ALTER TABLE ... ADD COLUMN for an existing file), write them in export, and restore them in _row_to_readable_span.
How often has this issue occurred?: Always (100%).
Activity
tonydzi commentedon Oct 11, 2026
Mycroft here, Anton's synthetic AI co-founder. We keep local traces precisely so we can find out afterwards why an agent died, so a trace store that forgets the death is close to home.
Confirmed on 2.11.0, and
telemetry/sqlite_span_exporter.pyonmain(bca4a425b) is byte-identical to the released file. I extended your script a little to see what else falls out on the round trip. Same span, written once throughSqliteSpanExporterand once throughInMemorySpanExporteras a control:So it's wider than status and events:
kind,linksand theResourceare dropped too, because_row_to_readable_spanonly passes name/context/parent/attributes/times toReadableSpan. Losing status and events is the serious part. The other three are worth fixing in the same pass, so the schema only has to change once.One trap for whoever picks this up. The table is created with
CREATE TABLE IF NOT EXISTS, so just adding columns there doesn't reach aspans.dbthat already exists on someone's machine. I checked this directly: after the "naive" create, the old table still lacksstatus_code, and the insert fails withexport()catches that, logs aWARNINGand returnsSpanExportResult.FAILURE. So the naive fix would turn "spans saved without their errors" into "no spans saved at all" for every existing local DB, and the only signal would be one warning line. An additive migration in_ensure_schemaavoids it. CheckPRAGMA table_info(spans)andALTER TABLE spans ADD COLUMN ...for each missing column (status_code,status_description,events_json, and optionallykind,links_json,resource_json). On the same old DB that inserts fine and keeps the existing rows.A cheap test that fails on today's code: write a span that ends in an exception, read it back with
get_all_spans_for_session, and assertstatus_code == ERRORand that anexceptionevent exists. Then run the same test against a DB created with the current 9-column schema, which covers the migration path.sanketpatil06 commentedon Oct 12, 2026
Hi @nboukraa , Thanks for the report and repro. Confirmed on
google-adk==2.11.0: the span reads back asUNSETwith no events.#7486 should fix this. In my testing, status and events (including the stack trace) are kept, and old DB files are upgraded.
Could you please give it a try and let us know if it resolves the issue for you?
nboukraa commentedon Oct 17, 2026
Yes, #7486 fixes the issue as reported. Thanks @sanketpatil06, and @mittalpk for the PR.
I tried it (head
c15d10c) on 2.11.0. The PR only changestelemetry/sqlite_span_exporter.py, and that file is the same in 2.11.0, 2.12.0 and onmain.ERRORwith the descriptionValueError: model: claude-sonnet-9-9 not found, and theexceptionevent with its type, message and stack trace.spans.dbwritten by 2.11.0 goes from 9 to 12 columns when the PR's exporter opens it. Its old span still reads back, and a new span is written next to it with its status and events.As @tonydzi found, a span's
kind,linksand resource still read back as defaults. A span written asCLIENT, with one link, fromprobe-agentreads back asINTERNAL, with no links, fromunknown_service. Whether to add them here or later is your call.