Are you in the Right Room?

• Are you interested in Specialized Markup?
  • Math
  • Music
  • Graphics
  • Chemistry
  • Biology
• Do you use, develop, or interface with AT?
• Do you like developing standards?

The Dance

• Speech Output
• Braille
• Magnification
• Navigation
• Alternative modes of Input
• Your turn to lead...

Speech Output Use Cases for Expert Handlers

• Today's Screen readers Are Good At...
  • Highly effective with textual, linear content
  • On screen widgets also effectively handled as list items
  • Tabular content can also work well when screen readers provide cell contents plus row and column labels
• Today's Screen Readers Need To Improve With...
  • more complex, nonlinear content such as
    • Mathematical Expressions
    • Chemical Formulae
    • Biological Structures
    • Musical Scores
    • Graphics (SVG)

Speech Output Use Cases for Expert Handlers

• To place knowledge domain specific content under tight screen reader control, Screen Readers Need To ...
• Voice in logical order — i.e. what's next and/or previous?
• Support substructure focus at several levels of granularity
• Appropriately voice specialized symbols and symbolic expressions

Speech Output Use Cases for Expert Handlers

• TTS based synthetic speech is widely used by blind and low vision persons in lieu of a monitor
• Tight control of what is spoken and when is critical when TTS is used in lieu of a monitor. Therefore, these users rely on a class of AT called "Screen Readers"
• Users require systematic structural access to screen content becauseof the fundamental difference between speech and display media:
  • Monitor Display = Static
  • TTS Rendering = Temporal

Braille Display, Embossing and Tactile Output

Why is custom braille handling needed?

• Different braille codes
  
  • different braille codes are used to represent some non text info
  • most AT has no knowledge of special areas
  • Ex: math has differing codes in different countries and agencies
• Different ways to represent information
  • Some areas such as mathematics can benefit from displaying information at different level of granularity
  • low level: summary of expression
  • high level: complete Nemeth translation spanning several lines
• Different devices to represent information
  • no way to represent info in braille (eg, graphics)
  • tactile and sound methods are possible, such as tactile printout placed on touchpad.
  • standard AT has to give total control to expert handler

Braille Display, Embossing and Tactile Output

Smooth AT/Expert Handler Communication

• Expert handler may need some service from AT, such as speech output using the same voice AT is using and Braille display output to the same braille display.
• There are many different codes (braille tables) used to represent braille dot patterns. Expert handler and AT should agree on which braille code is used.

Magnification

General Cases

• Text-based Content: use larger fonts
  • mathematics
  • music
  • labels for plots/graphics
• Non-textual Content: enlarge or fill this area
  • images
  • SVG

Magnification

Specialized Cases

• Magnification while speaking or navigating
  • music — magnify current and next measures
  • mathematics — magnify the current term
  • plots — magnify curve, data, or labels
• Math and Chemistry Scripts
  • control over minimum shrink amount (minimum font size)
  • control over percent to shrink

Alternative Input Use Cases

• There are users with disabilities who do not require accommodation in order to read domain specific markup. Rather, these users require assistive technologies to facilitate their scrolling and/or editing of content.
• Highly effective assistive technologies exist to accommodate alternative input strategies ranging from:
  • speech input technology, such as DragonDictate or NaturallySpeaking;
  • mouse and keyboard alternative systems, such as the GNOME On-Screen Keyboard (GOK), Jambu, which provides improved web accessibility for switch and alternative pointer users, and OpenEyes, an open-source open-hardware toolkit for real-time eye-tracking;
  • context-aware word-prediction technologies, such as Dasher

Alternative Input Use Cases

• Users of alternative input assistive technologies accessing knowledge domain specific content require:
  • Context aware expedited scrolling and navigation. The Navigation Use Cases outlined elsewhere in these slides will serve this requirement.
  • Knowledge domain context aware command and content vocabulary for speech based navigation systems and for word-prediction systems.

Navigability: The General Use Case

Need:

• Ability to navigate within sub-components of documents containing specialized content
• Examples: math, music, genomic or chemical markup

Navigability: The General Use Case

Why an Expert Handler?

• AT doesn't know how to deal with specialized content directly;
• specialized components have content which needs to receive focus at different levels of granularity
• Examples: a numerator within a fraction, an expression, a term, a bar of music

Navigability: The General Use Case

How?

• Each level of granularity needs to be able to provide the AT commands to inspect and navigate to and from "items" -- by word, bar, expression, clause, term, etc. -- depending upon the type of content being expressed.
• Three system components need to interact:
  1. the user agent or editor
  2. the AT
  3. the expert handler

Reading & Editing

Three Conditions for Expert Handlers

1. level of granularity
2. mode of output, and
3. which item (first, last, previous, current, next)

Reading & Editing

Read-only:

1. the AT responds a Point of Regard change event, using the role of the object which received focus
2. the AT fetches accessibility information pertinent to that role and then formats/outputs a response tailored to an AT user

Reading & Editing

Editable content

• Separate cursors are needed:
  1. one to remain at the POR (the caret, if editing)
  2. one to move around for review purposes

Navigability: The General Use Case

Where Am I?

• The ability to query the AT to determine one's point of regard within a document and within containers in the document is essential.
• A user must have a means of obtaining:
  1. all available information about the object/character with focus, beginning with the repetition of the character or the textual binding which describes the object with focus.
  2. information about the current point of regard from the most generic level -- what percentage of the document or section has been read, how much of the document or section remains to be read -- to the most atomic.
• ATs must create a UI where successive "Where Am I?" queries by the user generate more verbose or more terse responses.

Navigability: The General Use Case

Document Summary

• A user may find it necessary to consult a Document Summary, containing a list of the types of elements and containers in the document.
• A user needs to know the document title and natural language of the document's content, as well as the type, tab index order, and number of structural elements appropriate to the content being rendered.
• An application may implement a document summary feature natively through its own UI instead of an accessibility API, but in the case of specialized markup, may need the assistance of an expert handler in order to present a document summary appropriate to the content being summarized.

Putting It All Together

Expert Handlers & the Flow of Control

• Expert Handler Changes the DOM (like Greasemonkey)
  1. During install, EH registers itself with app (browser, PDF viewer)
  2. When page is loaded, EH is invoked by app to convert the DOM to text, braille, etc.
• AT Acts as Middle Man
  1. AT traverses the DOM and consults a resource to find a handler for specialized content
  2. AT copies node from DOM and passes it to EH
  3. AT issues requests to handler
• AT Hands off to Expert Handler
  1. AT traverses the DOM and consults a resource to find a handler for specialized content
  2. AT points EH to DOM so that EH interacts with DOM directly
  3. AT issues requests to handler

Reading & Editing

How it Works:

1. the browser/editor provides the input UI for the user.
2. the AT will handle the UI commands
3. the AT will call the handler to return an item at the current level of granularity
4. the AT informs handler about the output mode

Your turn to lead..