Chart spec valid for rendering in notebook fails in export to PNG #3713

TSSlade · 2024-12-11T18:55:54Z

What happened?

I have created a chart specification that renders perfectly in my notebook. I can then right-click, save to PNG, and get what I need. The full expected visualization (using the entire dataset) is as follows:

If I attempt to temps.save("temps.png") I get an error about the Vega-Lite to PNG conversion failing:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[138], line 1
----> 1 temps.save("temps.png")

File [~/miniconda3/envs/ttemp/lib/python3.12/site-packages/altair/vegalite/v5/api.py:2092](http://localhost:8888/lab/tree/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/altair/vegalite/v5/api.py#line=2091), in TopLevelMixin.save(self, fp, format, override_data_transformer, scale_factor, mode, vegalite_version, vega_version, vegaembed_version, embed_options, json_kwds, engine, inline, **kwargs)
   2090 if override_data_transformer:
   2091     with data_transformers.disable_max_rows():
-> 2092         save(**kwds)
   2093 else:
   2094     save(**kwds)

File [~/miniconda3/envs/ttemp/lib/python3.12/site-packages/altair/utils/save.py:224](http://localhost:8888/lab/tree/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/altair/utils/save.py#line=223), in save(chart, fp, vega_version, vegaembed_version, format, mode, vegalite_version, embed_options, json_kwds, scale_factor, engine, inline, **kwargs)
    217 else:
    218     # Temporarily turn off any data transformers so that all data is inlined
    219     # when calling chart.to_dict. This is relevant for vl-convert which cannot access
    220     # local json files which could be created by a json data transformer. Furthermore,
    221     # we don't exit the with statement until this function completed due to the issue
    222     # described at https://github.com/vega/vl-convert/issues/31
    223     with data_transformers.enable("default"), data_transformers.disable_max_rows():
--> 224         perform_save()

File ~/miniconda3/envs/ttemp/lib/python3.12/site-packages/altair/utils/save.py:175, in save.<locals>.perform_save()
    171     write_file_or_filename(
    172         fp, mb_html["text/html"], mode="w", encoding=encoding
    173     )
    174 elif format == "png":
--> 175     mb_png = spec_to_mimebundle(
    176         spec=spec,
    177         format=format,
    178         mode=inner_mode,
    179         vega_version=vega_version,
    180         vegalite_version=vegalite_version,
    181         vegaembed_version=vegaembed_version,
    182         embed_options=embed_options,
    183         scale_factor=scale_factor,
    184         engine=engine,
    185         **kwargs,
    186     )
    187     write_file_or_filename(fp, mb_png[0]["image/png"], mode="wb")
    188 elif format in {"svg", "pdf", "vega"}:

File [~/miniconda3/envs/ttemp/lib/python3.12/site-packages/altair/utils/mimebundle.py:134](http://localhost:8888/lab/tree/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/altair/utils/mimebundle.py#line=133), in spec_to_mimebundle(spec, format, mode, vega_version, vegaembed_version, vegalite_version, embed_options, engine, **kwargs)
    131 embed_options = preprocess_embed_options(final_embed_options)
    133 if format in {"png", "svg", "pdf", "vega"}:
--> 134     return _spec_to_mimebundle_with_engine(
    135         spec,
    136         cast(Literal["png", "svg", "pdf", "vega"], format),
    137         internal_mode,
    138         engine=engine,
    139         format_locale=embed_options.get("formatLocale", None),
    140         time_format_locale=embed_options.get("timeFormatLocale", None),
    141         **kwargs,
    142     )
    143 elif format == "html":
    144     html = spec_to_html(
    145         spec,
    146         mode=internal_mode,
   (...)
    151         **kwargs,
    152     )

File [~/miniconda3/envs/ttemp/lib/python3.12/site-packages/altair/utils/mimebundle.py:244](http://localhost:8888/lab/tree/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/altair/utils/mimebundle.py#line=243), in _spec_to_mimebundle_with_engine(spec, format, mode, format_locale, time_format_locale, **kwargs)
    236     png = vlc.vega_to_png(
    237         spec,
    238         scale=scale,
   (...)
    241         time_format_locale=time_format_locale,
    242     )
    243 else:
--> 244     png = vlc.vegalite_to_png(
    245         spec,
    246         vl_version=vl_version,
    247         scale=scale,
    248         ppi=ppi,
    249         format_locale=format_locale,
    250         time_format_locale=time_format_locale,
    251     )
    252 factor = ppi / default_ppi
    253 w, h = _pngxy(png)

ValueError: Vega-Lite to PNG conversion failed:
Error: Expression parse error: ("Video time (m's\"): " + (timeFormat(datum["datetime"], '%M'%S"')) + "; Temp, °C: " + (format(datum["temp"], "")) + "; roi: " + (isValid(datum["roi"]) ? datum["roi"] : ""+datum["roi"]) + "; Frame: " + (isValid(datum["frameId"]) ? datum["frameId"] : ""+datum["frameId"]) + "; Video Time: " + (isValid(datum["time_label"]) ? datum["time_label"] : ""+datum["time_label"]) + "; Temp (°C): " + (format(datum["temp"], ".2f")) + "; ROI: " + (isValid(datum["roi"]) ? datum["roi"] : ""+datum["roi"]) + "; xy: " + (isValid(datum["xy"]) ? datum["xy"] : ""+datum["xy"]))
    at b (https://cdn.skypack.dev/-/vega-util@v1.17.2-LUfkDhormMyfWqy3Ts6U/dist=es2020,mode=imports,min/optimized/vega-util.js:1:324)
    at Rn (https://cdn.skypack.dev/-/vega-functions@v5.15.0-Bjrw9nnQutKMtsMi1DSI/dist=es2020,mode=imports,min/optimized/vega-functions.js:1:12384)
    at Bn (https://cdn.skypack.dev/-/vega-parser@v6.4.0-nwGMLAa2L4N1N7f1iRh9/dist=es2020,mode=imports,min/optimized/vega-parser.js:1:4443)
    at An (https://cdn.skypack.dev/-/vega-parser@v6.4.0-nwGMLAa2L4N1N7f1iRh9/dist=es2020,mode=imports,min/optimized/vega-parser.js:1:4271)
    at ze (https://cdn.skypack.dev/-/vega-parser@v6.4.0-nwGMLAa2L4N1N7f1iRh9/dist=es2020,mode=imports,min/optimized/vega-parser.js:1:4175)
    at Mt (https://cdn.skypack.dev/-/vega-parser@v6.4.0-nwGMLAa2L4N1N7f1iRh9/dist=es2020,mode=imports,min/optimized/vega-parser.js:1:24664)
    at https://cdn.skypack.dev/-/vega-parser@v6.4.0-nwGMLAa2L4N1N7f1iRh9/dist=es2020,mode=imports,min/optimized/vega-parser.js:1:39141
    at Array.forEach (<anonymous>)
    at hn (https://cdn.skypack.dev/-/vega-parser@v6.4.0-nwGMLAa2L4N1N7f1iRh9/dist=es2020,mode=imports,min/optimized/vega-parser.js:1:39130)
    at al (https://cdn.skypack.dev/-/vega-parser@v6.4.0-nwGMLAa2L4N1N7f1iRh9/dist=es2020,mode=imports,min/optimized/vega-parser.js:1:39804)

The most relevant part of which appears to be this expression:

Error: Expression parse error: ("Video time (m's\"): " + (timeFormat(datum["datetime"], '%M'%S"')) + "; Temp, °C: " + (format(datum["temp"], "")) + "; roi: " + (isValid(datum["roi"]) ? datum["roi"] : ""+datum["roi"]) + "; Frame: " + (isValid(datum["frameId"]) ? datum["frameId"] : ""+datum["frameId"]) + "; Video Time: " + (isValid(datum["time_label"]) ? datum["time_label"] : ""+datum["time_label"]) + "; Temp (°C): " + (format(datum["temp"], ".2f")) + "; ROI: " + (isValid(datum["roi"]) ? datum["roi"] : ""+datum["roi"]) + "; xy: " + (isValid(datum["xy"]) ? datum["xy"] : ""+datum["xy"]))

The output of pprint.pprint(temps.to_dict()) is as follows:

{'config': {'view': {'continuousWidth': 300, 'continuousHeight': 300}},
 'data': {'url': 'vegafusion+dataset://table_2db42a3b_b982_4250_b7c0_1e9c488ebf27'},
 'mark': {'type': 'circle', 'opacity': 0.5, 'size': 20},
 'encoding': {'color': {'condition': {'param': 'param_1',
    'field': 'roi',
    'legend': None,
    'scale': {'domain': ['forehead',
      'nose',
      'sinus_left',
      'sinus_right',
      'face'],
     'range': ['lightcoral',
      'navajowhite',
      'lightgreen',
      'lightseagreen',
      'gold']},
    'type': 'nominal'},
   'value': 'lightgray'},
  'opacity': {'condition': {'param': 'param_3', 'value': 0.8}, 'value': 0.0},
  'shape': {'field': 'roi', 'type': 'nominal'},
  'tooltip': [{'field': 'frameId', 'title': 'Frame', 'type': 'ordinal'},
   {'field': 'time_label', 'title': 'Video Time', 'type': 'nominal'},
   {'field': 'temp',
    'format': '.2f',
    'title': 'Temp (°C)',
    'type': 'quantitative'},
   {'field': 'roi', 'title': 'ROI', 'type': 'nominal'},
   {'field': 'xy', 'type': 'nominal'}],
  'x': {'axis': {'format': '%M\'%S"', 'gridColor': '#EEE', 'tickColor': 'red'},
   'field': 'datetime',
   'title': 'Video time (m\'s")',
   'type': 'temporal'},
  'y': {'field': 'temp',
   'scale': {'domainMin': 31, 'domainMax': 37},
   'title': 'Temp, °C',
   'type': 'quantitative'}},
 'height': 480.0,
 'name': 'TempsChart',
 'params': [{'name': 'param_1',
   'select': {'type': 'interval', 'encodings': ['x', 'y']}},
  {'name': 'param_3', 'select': {'type': 'point', 'fields': ['roi']}},
  {'name': 'param_5',
   'select': {'type': 'interval', 'encodings': ['x', 'y']},
   'bind': 'scales'}],
 'title': 'Temps (°C) over time, by ROI',
 'transform': [{'calculate': "'(' + datum.x1 + ', ' + datum.y1 + ')'",
   'as': 'xy'}],
 'width': 640.0,
 '$schema': 'https://vega.github.io/schema/vega-lite/v5.20.1.json'}

Here's a reduced dataset that reproduces the problem:

MWE

mwe_data.to_dict()

{'frameId': {0: 'frame_10721', 1: 'frame_10721', 2: 'frame_10721'},
 'roi': {0: 'forehead', 1: 'nose', 2: 'sinus_left'},
 'raw_timestamp': {0: 464623.04981108126,
  1: 464623.04981108126,
  2: 464623.04981108126},
 'temp': {0: 35.967320261437905, 1: 31.96078431372549, 2: 31.88888888888889},
 'coords': {0: '[304,242,308,246]',
  1: '[299,345,303,349]',
  2: '[367,356,371,360]'},
 'x1': {0: 304, 1: 299, 2: 367},
 'y1': {0: 242, 1: 345, 2: 356},
 'x2': {0: 308, 1: 303, 2: 371},
 'y2': {0: 246, 1: 349, 2: 360},
 'datetime': {0: Timestamp('1970-01-01 00:07:44.623049811'),
  1: Timestamp('1970-01-01 00:07:44.623049811'),
  2: Timestamp('1970-01-01 00:07:44.623049811')},
 'time_label': {0: '7\'44.623"', 1: '7\'44.623"', 2: '7\'44.623"'}
 'regimes': {0: 'stimulus', 1: 'stimulus', 2: 'stimulus'}}

res_dict = {"height": 640, "width": 320}
brush = alt.selection_interval(encodings=["x", "y"]) 
legend_selection = alt.selection_multi(fields=["roi"])

(
        alt.Chart(mwe_data, title="Temps (°C) over time, by ROI", name="TempsChart")
        .transform_calculate(
            xy="'(' + datum.x1 + ', ' + datum.y1 + ')'",
            area="(datum.x1 + datum.w) * (datum.y1 + datum.h)"
        )
        .mark_circle(opacity=0.5, size=20)
        .encode(
            alt.X(
                "datetime:T",
                axis=alt.Axis(format="%M'%S\"", tickColor="red", gridColor="#EEE"),
            ).title("Video time (m's\")"),
            alt.Y("temp:Q")
            .title("Temp, °C")
            .scale(domainMin=min_temp, domainMax=max_temp),
            shape=alt.Shape("roi:N"),
            color=alt.condition(
                brush,
                alt.Color(
                    "roi:N", scale=alt.Scale(domain=ttemp_rois, range=ttemp_colors),
                    legend=None
                ), #.legend(title="ROIs"),
                alt.value("lightgray"),
            ),
            opacity=alt.condition(legend_selection, alt.value(0.8), alt.value(0.0)),
            tooltip=base_tooltip,
        )
        .add_params(brush)
        .add_params(legend_selection)
        .properties(width=res_dict["width"], height=res_dict["height"])
        .interactive()
    )
mwe.show()

The outcome is as follows:

The result of mwe.save("temps_test.png") remains the same as with the full dataset:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[142], line 1
----> 1 mwe.save("temps_test.png")

File [~/miniconda3/envs/ttemp/lib/python3.12/site-packages/altair/vegalite/v5/api.py:2092](http://localhost:8888/lab/tree/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/altair/vegalite/v5/api.py#line=2091), in TopLevelMixin.save(self, fp, format, override_data_transformer, scale_factor, mode, vegalite_version, vega_version, vegaembed_version, embed_options, json_kwds, engine, inline, **kwargs)
   2090 if override_data_transformer:
   2091     with data_transformers.disable_max_rows():
-> 2092         save(**kwds)
   2093 else:
   2094     save(**kwds)

File [~/miniconda3/envs/ttemp/lib/python3.12/site-packages/altair/utils/save.py:224](http://localhost:8888/lab/tree/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/altair/utils/save.py#line=223), in save(chart, fp, vega_version, vegaembed_version, format, mode, vegalite_version, embed_options, json_kwds, scale_factor, engine, inline, **kwargs)
    217 else:
    218     # Temporarily turn off any data transformers so that all data is inlined
    219     # when calling chart.to_dict. This is relevant for vl-convert which cannot access
    220     # local json files which could be created by a json data transformer. Furthermore,
    221     # we don't exit the with statement until this function completed due to the issue
    222     # described at https://github.com/vega/vl-convert/issues/31
    223     with data_transformers.enable("default"), data_transformers.disable_max_rows():
--> 224         perform_save()

File ~/miniconda3/envs/ttemp/lib/python3.12/site-packages/altair/utils/save.py:175, in save.<locals>.perform_save()
    171     write_file_or_filename(
    172         fp, mb_html["text/html"], mode="w", encoding=encoding
    173     )
    174 elif format == "png":
--> 175     mb_png = spec_to_mimebundle(
    176         spec=spec,
    177         format=format,
    178         mode=inner_mode,
    179         vega_version=vega_version,
    180         vegalite_version=vegalite_version,
    181         vegaembed_version=vegaembed_version,
    182         embed_options=embed_options,
    183         scale_factor=scale_factor,
    184         engine=engine,
    185         **kwargs,
    186     )
    187     write_file_or_filename(fp, mb_png[0]["image/png"], mode="wb")
    188 elif format in {"svg", "pdf", "vega"}:

File [~/miniconda3/envs/ttemp/lib/python3.12/site-packages/altair/utils/mimebundle.py:134](http://localhost:8888/lab/tree/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/altair/utils/mimebundle.py#line=133), in spec_to_mimebundle(spec, format, mode, vega_version, vegaembed_version, vegalite_version, embed_options, engine, **kwargs)
    131 embed_options = preprocess_embed_options(final_embed_options)
    133 if format in {"png", "svg", "pdf", "vega"}:
--> 134     return _spec_to_mimebundle_with_engine(
    135         spec,
    136         cast(Literal["png", "svg", "pdf", "vega"], format),
    137         internal_mode,
    138         engine=engine,
    139         format_locale=embed_options.get("formatLocale", None),
    140         time_format_locale=embed_options.get("timeFormatLocale", None),
    141         **kwargs,
    142     )
    143 elif format == "html":
    144     html = spec_to_html(
    145         spec,
    146         mode=internal_mode,
   (...)
    151         **kwargs,
    152     )

File [~/miniconda3/envs/ttemp/lib/python3.12/site-packages/altair/utils/mimebundle.py:244](http://localhost:8888/lab/tree/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/altair/utils/mimebundle.py#line=243), in _spec_to_mimebundle_with_engine(spec, format, mode, format_locale, time_format_locale, **kwargs)
    236     png = vlc.vega_to_png(
    237         spec,
    238         scale=scale,
   (...)
    241         time_format_locale=time_format_locale,
    242     )
    243 else:
--> 244     png = vlc.vegalite_to_png(
    245         spec,
    246         vl_version=vl_version,
    247         scale=scale,
    248         ppi=ppi,
    249         format_locale=format_locale,
    250         time_format_locale=time_format_locale,
    251     )
    252 factor = ppi / default_ppi
    253 w, h = _pngxy(png)

ValueError: Vega-Lite to PNG conversion failed:
Error: Expression parse error: ("Video time (m's\"): " + (timeFormat(datum["datetime"], '%M'%S"')) + "; Temp, °C: " + (format(datum["temp"], "")) + "; roi: " + (isValid(datum["roi"]) ? datum["roi"] : ""+datum["roi"]) + "; Frame: " + (isValid(datum["frameId"]) ? datum["frameId"] : ""+datum["frameId"]) + "; Video Time: " + (isValid(datum["time_label"]) ? datum["time_label"] : ""+datum["time_label"]) + "; Temp (°C): " + (format(datum["temp"], ".2f")) + "; ROI: " + (isValid(datum["roi"]) ? datum["roi"] : ""+datum["roi"]) + "; xy: " + (isValid(datum["xy"]) ? datum["xy"] : ""+datum["xy"]))
    at b (https://cdn.skypack.dev/-/vega-util@v1.17.2-LUfkDhormMyfWqy3Ts6U/dist=es2020,mode=imports,min/optimized/vega-util.js:1:324)
    at Rn (https://cdn.skypack.dev/-/vega-functions@v5.15.0-Bjrw9nnQutKMtsMi1DSI/dist=es2020,mode=imports,min/optimized/vega-functions.js:1:12384)
    at Bn (https://cdn.skypack.dev/-/vega-parser@v6.4.0-nwGMLAa2L4N1N7f1iRh9/dist=es2020,mode=imports,min/optimized/vega-parser.js:1:4443)
    at An (https://cdn.skypack.dev/-/vega-parser@v6.4.0-nwGMLAa2L4N1N7f1iRh9/dist=es2020,mode=imports,min/optimized/vega-parser.js:1:4271)
    at ze (https://cdn.skypack.dev/-/vega-parser@v6.4.0-nwGMLAa2L4N1N7f1iRh9/dist=es2020,mode=imports,min/optimized/vega-parser.js:1:4175)
    at Mt (https://cdn.skypack.dev/-/vega-parser@v6.4.0-nwGMLAa2L4N1N7f1iRh9/dist=es2020,mode=imports,min/optimized/vega-parser.js:1:24664)
    at https://cdn.skypack.dev/-/vega-parser@v6.4.0-nwGMLAa2L4N1N7f1iRh9/dist=es2020,mode=imports,min/optimized/vega-parser.js:1:39141
    at Array.forEach (<anonymous>)
    at hn (https://cdn.skypack.dev/-/vega-parser@v6.4.0-nwGMLAa2L4N1N7f1iRh9/dist=es2020,mode=imports,min/optimized/vega-parser.js:1:39130)
    at al (https://cdn.skypack.dev/-/vega-parser@v6.4.0-nwGMLAa2L4N1N7f1iRh9/dist=es2020,mode=imports,min/optimized/vega-parser.js:1:39804)

What would you like to happen instead?

No response

Which version of Altair are you using?

5.4.1

The text was updated successfully, but these errors were encountered:

TSSlade · 2024-12-11T19:33:48Z

I suspected the culprit was the desire for the "Video time (m's")" label in the axis title and the format="%M'%S\"" for the x-axis label.

I looked to the Vega docs re: Field for some insight. That led to the following approach and results:

Escapes	NB Spec	PNG exported?	Note
existing `format="%M'%S\""` `.title("Video time (m's\")")`	✅	❌
double escapes `format="%M\\'%S\\""` `.title("Video time (m\\'s\\")")`	❌	N/A	Syntax Error (unterminated string literal)
triple escapes `format="%M\\\'%S\\\""` `.title("Video time (m\\\'s\\\")")`	✅	✅	Visual has extraneous `\`
quad escapes `format="%M\\\\'%S\\\\""` `.title("Video time (m\\\\'s\\\\")")`	✅	❌	Syntax Error (unterminated string literal)
sidestep this nonsense `format="%M:%S"` `.title("Video time (m:s))`	✅	✅	Just not what I actually want, label-wise

Is there some other approach to this that would be better? Some flavor of URL-encoding approach?

TSSlade · 2024-12-12T22:00:42Z

Have confirmed that resorting to unicode strings to pass the spec in generates an acceptable in-notebook visual, but does not pass the compilation process:

...
alt.X("datetime:T",
    axis=alt.Axis(format="%M\u0027%S\u0022", tickColor="red", gridColor="#EEE"),
).title("Video time (m\u0027s\u0022)"),
...

ValueError: Vega-Lite to PNG conversion failed:
Error: Expression parse error: ("Video time (m's\"): " + (timeFormat(datum["datetime"], '%M'%S"')) + "; Temp, °C: " + (format(datum["temp"], "")) + "; roi: " + (isValid(datum["roi"]) ? datum["roi"] : ""+datum["roi"]) + "; Frame: " + (isValid(datum["frameId"]) ? datum["frameId"] : ""+datum["frameId"]) + "; Video Time: " + (isValid(datum["time_label"]) ? datum["time_label"] : ""+datum["time_label"]) + "; Temp (°C): " + (format(datum["temp"], ".2f")) + "; ROI: " + (isValid(datum["roi"]) ? datum["roi"] : ""+datum["roi"]) + "; xy: " + (isValid(datum["xy"]) ? datum["xy"] : ""+datum["xy"]))

...and stacking slashes with the unicode point yields a similar set of outcomes as repeated previously.

dangotbanned · 2025-01-03T13:41:03Z

@TSSlade could you please reduce this into a minimal repro that a maintainer can work with you on?

If the problem here is related to escaping characters - then you should be able to reduce this into a few lines.

TSSlade added the bug label Dec 11, 2024

dangotbanned added the needs-info label Jan 3, 2025

dangotbanned added needs-repro Issues that **need** a Minimal, Reproducible Example vega: vl-convert Requires upstream/integration action w/ `vl-convert` and removed needs-info labels Jan 7, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Chart spec valid for rendering in notebook fails in export to PNG #3713

Chart spec valid for rendering in notebook fails in export to PNG #3713

TSSlade commented Dec 11, 2024

TSSlade commented Dec 11, 2024 •

edited

Loading

TSSlade commented Dec 12, 2024

dangotbanned commented Jan 3, 2025

Chart spec valid for rendering in notebook fails in export to PNG #3713

Chart spec valid for rendering in notebook fails in export to PNG #3713

Comments

TSSlade commented Dec 11, 2024

What happened?

MWE

What would you like to happen instead?

Which version of Altair are you using?

TSSlade commented Dec 11, 2024 • edited Loading

TSSlade commented Dec 12, 2024

dangotbanned commented Jan 3, 2025

TSSlade commented Dec 11, 2024 •

edited

Loading