ARIA 1.1: Remove children-presentational true from math role

[takenspc]

The math role should not be children-presentational true. There are a few reasons to remove children-presentational true from math role.

1. Any MathML elements can be links

A MathML elements which has href attribute is a link. If a descendant of <math> (which has an implicit role="math") has an href attribute and UAs don't expose descendants, users of ATs can't operate that link. This should be avoid.

Mozilla has supported the href attribute long ago and WebKit recently support the attribute.

2. Compatibility of WebKit

WebKit already exposes descendants of a <math>. If other implementations don't expose descendants of a <math> element (which has an implicit role="math") per current ARIA 1.1 Draft, they will not work with ATs which work with WebKit.

[cookiecrook]

If ARIA 1.2 is still limited to HTML Role Parity, this issue is not in scope.

[mcking65]

This issue is more like an editorial correction than an actual change. If browsers have implemented this, they have made the math role pretty useless.

The other possibility is that I don't yet understand the purpose of the math role.

[cookiecrook]

The ARIA math role is wholly insufficient to represent the <math> element. (Cc @stevefaulkner who may disagree.) MathML defines many, many, many, sub-level elements that would also need to be mapped. It's a good goal, but complete support for Math should be out-of-scope for a point release.

The ARIA implementation is basically a subclass of "image" and IMO, should remain as-is for ARIA 1.2. Changing it to make children non-presentational could break some of the LaTeX test cases.

[mcking65]

@cookiecrook wrote:

MathML defines many, many, many, sub-level elements that would also need to be mapped. It's a good goal, but complete support for Math should be out-of-scope for a point release.

I agree. However, this issue is not to provide complete support for math. It is to remove children presentational true.

The ARIA implementation is basically a subclass of "image" and IMO, should remain as-is for ARIA 1.2.

No, it is a subclass of section.

My understanding has been that the math role is more like a region or figure that is designated as containing math content, which could include one or more images as well as widgets or structures. However, this is not possible if children presentational is true.

The definition of the math role and its position in the ontology do not jive with children presentational.

Changing it to make children non-presentational could break some of the LaTeX test cases.

Would love to learn more about these test cases and, if you can point to them, actual implementations where the math role is providing benefit.

[cookiecrook]

@mcking65 wrote:

The ARIA implementation is basically a subclass of "image" and IMO, should remain as-is for ARIA 1.2.

No, it is a subclass of section.

Sorry for not being more clear. I intended to say that the implementations of the math role (in browsers) treat it very similarly to how they treat flat images, where children are also presentational. I did not intend to imply that the superclass ARIA role was img, but perhaps it should be.

@takenspc wrote

WebKit already exposes descendants of a <math>. If other implementations don't expose descendants of a element (which has an implicit role="math") per current ARIA 1.1 Draft, they will not work with ATs which work with WebKit.

IMO, this is an error in the mapping. @stevefaulkner? The math role is incomplete for MathML, so the <math> element should not have an implicit mapping to the math role. If ARIA were to allow non-presentational children these math images, we'd need to duplicate the entirety of MathML in ARIA roles. This strikes me as foolish, since MathML is better supported in modern browsers.

Perhaps the best argument against changing it is that most of the math role description could be summed up as: "This role is intended to support legacy images of math. Use MathML instead of this. It's way better."

I believe that this should be resolved by updating the HTML-AAM to divest <math> of role=math

[cookiecrook]

Would love to learn more about these test cases

The following example gives a human-readable plain text aria-label (not TeX but nearly so), and the contents are basically an ASCII layout of Math that would not make sense linearized.

<pre role="math" aria-label="x=⟮−b±√⟮b²−4ac⟯⟯÷2a">
      −b±√⟮b²−4ac⟯
x =  -------------
          2a
</pre>

and, if you can point to them, actual implementations where the math role is providing benefit.

The math role has a very limited utility, it's effectively an image with "math" as the role description instead of "image."

MathML on the other hand, provides a lot of utility. IIRC, we've supported it in WebKit and VoiceOver for the last 4 or 5 versions of iOS and macOS. Users can navigate to and explore each individual part of equations, and VoiceOver converts the formula to Nemeth on connected braille displays.

[pkra]

Following some encouragement from @stevefaulkner:

This strikes me as foolish, since MathML is better supported in modern browsers.

This seems a rather extreme exaggeration.

1. Gecko and WebKit still have rather partial implementations that only enthusiasts use. The implementations have stalled for a long time now (last year's minimal changes to WebKit not withstanding).
2. The Chrome team has repeatedly made it clear (e.g., most recently at last year's TPAC) that MathML will not happen in Chrome. Even Edge (which, according to its developers, could integrate the capable math rendering engine from Office) has never seriously suggested it will support MathML.
3. MathML support is not (and really never has been) actively developed by browser vendors. The partial implementations in Gecko and WebKit were developed almost exclusively by lone-wolf volunteers (even if Igalia backed Fred Wang for the WebKit work last year). Those volunteers have repeatedly made questionable design decisions and never sought feedback from the developer community.
4. Presentation MathML (which the only relevant kind on the web) carries virtually no semantic information, it merely provides rather vague information about layout structure, not math. In particular, implementations differ so widely that developers cannot rely on their content being even visually reliable. (And of course it doesn't help that the layout structure of MathML is frequently incompatible with CSS or SVG.)
5. Presentation MathML is fundamentally unsuitable for accessibility. If the lack of semantics is not convincing enough, this is easily demonstrated by the fact that all high-quality MathML accessibility solutions (e.g., MathPlayer, speech-rule-engine) use complex heuristics to generate a completely different data structure internally so as to provide non-visual rendering. (Apologies to @cookiecrook; VO's MathML support does not seem to have seen an update since its partial, initial release years ago; in particular not a single bug report has been as much as acknowledged on bugzilla.)
6. Finally, MathML is a dead spec without working group or any other active development.

So for all intents and purposes, developers should consider MathML deprecated for the web.

And they do -- all public crawler data indicates that MathML is used exclusively with JavaScript rendering solutions; this includes every significant STEM publisher out there and even Wikipedia (though it now could deliver MathML).

To end on a positive note. The still relatively new MathOnTheWeb Community Group focuses on pragmatic solutions hopes to provide feedback on potential incremental improvements to standards. There's already agreement on our a11y task force that a modified math role could greatly help real world solutions for math rendering (using CSS and SVG) when it comes to enabling better and more consistent representations of on the accessibility tree. Certainly, not removing all children from the accessibility tree is a must.

We are currently gathering input from the CG (and beyond) and plan to connect with the ARIA WG once we have useful list of ideas.

[cookiecrook]

I appreciate the detailed reply, but I think it paints a bleaker picture of MathML than is fair. Especially this bit.

@pkra wrote:

MathML support is not (and really never has been) actively developed by browser vendors. The partial implementations in Gecko and WebKit were developed almost exclusively by lone-wolf volunteers (even if Igalia backed Fred Wang for the WebKit work last year).

Apple engineers did a lot of work to support MathML in iBooks and VoiceOver, as well as the API mappings in WebKit. There's also an entirely different parser and renderer in the iBooks Author context. There are some third party products used to help generate the source MathML or LaTeX for iBA.

AFAIK, Apple engineers created the only live conversion implementation of any open math standards (LaTeX and MathML) to Nemeth braille. (Microsoft's Word implementation being a propriety format.) As far as I know, it's also the only accessible solution that doesn't take the "self-voicing web application" approach. Please correct me if I'm wrong.

Certainly, not removing all children from the accessibility tree is a must.

Backing up to my point about the current math role. It's woefully insufficient in it's current state, but unhiding the children won't solve anything in the short term, and could break things. While MathML support is far from perfect, it has the widest accessible implementations of any math standard.

What do you think about this counter?

1. Change the math role to mathimage or something more representative of the original ARIA intention for that role. Leave mathimage as childrenPresentational.
2. Free up the math role for a deeper implementation in the future, including—at a minimum—an annotation block or attribute.

At a high level, I'd ask: Are you recommending using legacy images of math instead of the only Math solution with reasonable level of accessibility support? Regardless of accessibility considerations, even a polyfilled implementation in Chrome strikes me as a more robust solution than raster images.

[pkra]

To address the more important part of #425 (comment):

What do you think about this counter? [...]

Speaking for myself, I think both points would open up a path for improvements that help developers and users in the real world.

Are you recommending using legacy images of math instead of the only Math solution with reasonable level of accessibility support?

I'm guessing with "only Math solution with reasonable level of accessibility" you are referring to MathML. Given the lack of support for it in browsers or AT, this feels like a trick question (or perhaps like setting the bar really low).

Raster images are clearly not a reasonable solution. HTML (+CSS) and SVG markup has been the best choice for visual rendering of mathematics for years now (essentially can be delivered down to IE8) and visual rendering is a major accessibility concern after all. These formats also provide a good base for providing a better presentation on the accessibility tree, starting with flat, top-level labels and improving from there. For example, my own focus is on deeply nested labels that allow quite decent exploration on some AT (and a role=math could help AT provide a better exploration experience). Enriched HTML and SVG can easily be created from tools that enrich MathML but do not limit developers to MathML (e.g., work at Desmos or Khan Academy).

Regarding the other parts of #425 (comment).

I think it paints a bleaker picture of MathML than is fair.

It is what it is. In 18 years, MathML has tried and failed on the web twice. The WG is gone and the spec dead in the water (i.e., no way to improve it even if it wasn't as problematic as it is).

Better solutions have been developed, in part based on MathML's experience but also using completely different approaches. Those solutions are rely on successful and continuously improving web standards and benefit from increasing stable browser implementations (and again starting with IE8 here).

Apple engineers did a lot of work to support MathML in iBooks and VoiceOver, as well as the API mappings in WebKit.

I don't want to denigrate the work you and others at Apple are doing. For me, the lack of quality says more about MathML than about Apple's engineers -- it's garbage in, garbage out. Still, bugzilla does not send a good message.

FWIW, I have the impression that MathPlayer's really excellent heuristics have warped the perception in the a11y community regarding MathML's ability to provide accessible rendering. It's if we people raster images by recent improvements in machine learning generated image descriptions; those do not make raster images an accessible format either.

There's also an entirely different parser and renderer in the iBooks Author context.

If iBA is still using the Pages engine, I don't see how it applies to this context. Though, since you bring it up, the fact that iBA does not convert TeX to MathML but to SVG using BlahTeX seems to indicate a lack of confidence in its MathML rendering.

Apple engineers created the only live conversion implementation of any open math standards (LaTeX and MathML) to Nemeth braille.

I I don't know what "live" means in this context and I admit I'm not a braille expert. I'm aware of liblouis which is quite old and widely used (e.g., used by MathPlayer). I know speech-rule-engine just received funding for braille support. So hopefully there'll be more (and open-source) competition soon.

But this it brings up an interesting point about MathML. If it's true that there's only one implementation, it indicated that it's too expensive to provide support for MathML. To come back to on my earlier comparison, nobody expects ATs to implement machine learning features that generate image descriptions on the fly; yet for a niche like MathML we somehow assume they'll make such an investment.

This is actually a general problem of MathML (just talk to STEM publishers about their costly math workflows). MathML solutions are far and few. Yet the number of web-based based tools for math (not using MathML) is slowly but surely increasing, even expanding the notion of what math on the web can be.

As far as I know, it's also the only accessible solution that doesn't take the "self-voicing web application" approach.

I'm not sure what this describes. Could you explain? But perhaps that's really off topic now.

[joanmarie]

As a data point, given:

<div role="math">
  <div>First div inside role math</div>
  <div>Second div inside role math</div>
</div>

WebKit respects that children-presentational is true for the math role. Firefox and Chrome/Chromium each expose the children. So....

@cookiecrook, given that the other two multi-platform user agents are not respecting children-presentational is true, how would you feel about our making children-presentational false for the math role and then updating the WebKit implementation?

[cookiecrook]

@joanmarie

WebKit respects that children-presentational is true for the math role. Firefox and Chrome/Chromium each expose the children. So....

In Chrome, I believe that's because they specifically removed Blink support for MathML, so these are just rendered as generic elements.

@cookiecrook, given that the other two multi-platform user agents are not respecting children-presentational is true, how would you feel about our making children-presentational false for the math role and then updating the WebKit implementation?

Even if WebKit updated to match other browsers, I don't think that would solve the Math problem, so I don't believe a change is warranted until a better proposal is in the works...

VoiceOver users can interact with sub-section contents of MathML, but the math role is not yet sufficient to represent math structurally. If there was a property that allowed AT to know the math formula explicitly, a change to "children presentational" may not be necessary.

Spitballing:

• aria-format="latex" aria-label="x=⟮−b±√⟮b²−4ac⟯⟯÷2a"
• aria-mathvalue="x=⟮−b±√⟮b²−4ac⟯⟯÷2a"
• aria-formula="x=⟮−b±√⟮b²−4ac⟯⟯÷2a"

Perhaps there should be an "accessible math" incubator in WICG?

[bkardell]

In Chrome, I believe that's because they specifically removed Blink support for MathML, so these are just rendered as generic elements.

I don't know if this is helpful as a comment in this context, but mathml seems under-specified here in general and all report as generic Element in all the browsers I tested. We're working on that as part of the effort to explain magic/implementation of mathml as much as possible in terms of the platform mathml/issues/83

[joanmarie]

This issue had fallen off my radar somehow. But as I stated earlier in this issue:

WebKit respects that children-presentational is true for the math role. Firefox and Chrome/Chromium each expose the children.

If we were to test children-presentational=true today using the W3C's minimum-two-implementations standard, it would fail testing. Doing what is suggested in the opening report, on the other hand, would pass. Our spec is not in alignment with reality, and we have lack of interoperability. How shall we proceed?

[joanmarie]

This issue had fallen off my radar somehow. But as I stated earlier in this issue:

WebKit respects that children-presentational is true for the math role. Firefox and Chrome/Chromium each expose the children.

If we were to test children-presentational=true today using the W3C's minimum-two-implementations standard, it would fail testing. Doing what is suggested in the opening report, on the other hand, would pass. Our spec is not in alignment with reality, and we have lack of interoperability. How shall we proceed?

@cookiecrook: I get that you don't like the idea of children-presentational=false on math, but in light of the above, I have to ask: Can you live with it?

[cookiecrook]

@joanmarie wrote:

@cookiecrook: I get that you don't like the idea of children-presentational=false on math, but in light of the above, I have to ask: Can you live with it?

No. This change proposal feels like a workaround for the fact that Blink dropped its Math support entirely, and now forces web authors to use polyfills like MathJAX.

If WebKit were to implement it this way, it would break accessible MathML support on iOS and macOS. With the tree representation of Math, VoiceOver users get 1) Nemeth or other math braille formats natively, and 2) sub-formula exploration (e.g. with or without braille, users can interact and dig into the denominator portion of a complex fraction in a formula).

I don't think it makes sense to break the best native implementation of accessible math in order to work around the fact that other browsers don't support math at all.

[cookiecrook]

Apologies. I may have gotten my wires crossed here with the phrasing of the specific question. For MathML, WebKit needs to expose the children nodes (MathML-based roles like mfrac) in order for native support to be utilized.

However, for ARIA math (which is still under-specified), it could go either way. Ideally, there would be a new, disambiguous string (LaTeX seems most logical because it is linearizable) that would define the intent of the mathematical formula. Once this is supported, childrenpresentational=false could be fine on generic ARIA math, but possibly not on MathML.

aria-math="x=⟮−b±√⟮b²−4ac⟯⟯÷2a"

[joanmarie]

Sorry for any wiring-crossing I've done. To be clear, the scenario we're talking about is NOT MathML.

Consider as a contrived example:

    <div role='math'><input /> + 1 = 4</div>

Should the input element be in the accessibility tree? I think it should. It is for Gecko and Blink; it's not for WebKit. As a result, when you use Safari with VoiceOver and tab into the input, you're told you're in a Math. And I find it difficult to navigate amongst the contents. But if I change the above to:

    <div role='group'><input /> + 1 = 4</div>

Suddenly the input is in the accessibility tree in Safari, tabbing to the input causes VoiceOver to tell me I'm in the input, and navigation between the input and the rest of the content in the div works smoothly.

Mind you, I'm not trying to suggest that authors should do the above. Nor am I suggesting the math role as it appears in the spec is not in desperate need of some attention and a re-write. And do we need some authoring guidance, heck yeah.

What I am saying is this and only this: If an author slaps the 'math' role on a div, the children therein should be in the accessibility tree, i.e. children-presentational for math needs to be false; not true.

I don't think this has to be a big deal. Couldn't you just treat the math role like a group rather than something akin to img and hope the author did something that makes sense with the content?

[joanmarie]

I also think we might wish to consider repurposing the math role. Creating a role as a hackaround in which to embed MathML (or TeX or LaTeX) in the hopes that the native browser implementations or a polyfill library tidy things up for the AT seems rather unfortunate.

[joanmarie]

The more I read the 1.0, 1.1, and now 1.2-in-progress math role section, the more I wish to weep. 😉 I've opened a new issue (#940) to address what the role should be during the 1.2 cycle.

For this issue, let's try to reach consensus on whether or not children of an element with role='math' should be included or excluded from the accessibility tree.

[cookiecrook]

To be clear, the scenario we're talking about is NOT MathML.

I think this is point of confusion or contention. Both the ARIA spec and the HTML-AAM refer to these as if they are interchangeable. As long as it's made clear they are not the same, I don't have a strong preference about what happens to the ARIA "math" role.

Consider as a contrived example:

Thanks https://webkit.org/b/196756

I also think we might wish to consider repurposing the math role. Creating a role as a hackaround in which to embed MathML (or TeX or LaTeX) in the hopes that the native browser implementations or a polyfill library tidy things up for the AT seems rather unfortunate.

👍

[joanmarie]

To be clear, the scenario we're talking about is NOT MathML.

I think this is point of confusion or contention. Both the ARIA spec and the HTML-AAM refer to these as if they are interchangeable. As long as it's made clear they are not the same, I don't have a strong preference about what happens to the ARIA "math" role.

I hope that what we come up with to solve issue #940 will solve the confusion. In the meantime I'll do a pull request to set children-presentational to false for the ARIA math role.

Thanks https://webkit.org/b/196756

Thanks for filing it!

[AmeliaBR]

In #940 (comment), I investigated the use case where switching from "children presentational: true" to "false" would actually break things: when the equation is drawn to screen with presentational markup (e.g., @cookiecrook's example of pre-formatted ASCII, or a mix of SVG text and drawing code) but author has provided a readable version of the equation with aria-label or aria-labelledby.

With the proposed change (unless ARIA adds some special rules for backwards compatibility), the extra garbage markup gets exposed in addition to the label.

But, the child markup is already exposed in Chrome and Firefox, so that the change wouldn't break anything that isn't already broken for many users. If an author was testing properly they would already be using a wrapper with aria-hidden="true" to hide the presentational stuff.

It may be worth having an example of using aria-hidden on presentational markup within an equation, with a warning about the change in definition from ARIA 1. E.g., working from the pre-formatted example:

<div id="eq-1" role="math" aria-labelledby="eq-1-linear">
<p id="eq-1-linear" hidden>x=⟮−b±√⟮b²−4ac⟯⟯÷2a</p>
<pre aria-hidden="true">
      −b±√⟮b²−4ac⟯
x =  -------------
          2a
</pre>
</div>

[joanmarie]

It may be worth having an example of using aria-hidden on presentational markup within an equation, with a warning about the change in definition from ARIA 1.

Excellent point! Could you please add that to #940? I want to collect all the input from what the new math role will look like and everything we'll wish to cover in the spec.

Thanks!

[joanmarie]

The group approved merging the change during today's ARIA WG meeting. It's been merged into the Editor's Draft. I need to write the tests, obtain the results, etc. before merging into the Working Draft as per our work flow.