Using today's accessibility APIs, developers of assistive technologies face difficult challenges trying to implement logical navigation of the content and structures of complex documents for users with vision, learning, physical, and cognitive impairments. Some of the challenges include:
To address these challenges, IBM will work with an open source standards working group in a standards organization such as Open Accessibility (formerly part of FSG, now part of The Linux Foundation) to do the following:
As input, a "gap analysis" chart will be developed to categorize and compare all the different accessibility APIs and other APIs used by assistive technologies on different platforms today to access document content and structure information (including AT-SPI and ATK, Microsoft's MSAA and UI Automation, various Document Object Models, IBM Home Page Reader Web Access Technology (WAT), and others). Use cases will also be developed as input. Then the group should start drafting an API recommendation for extending cross-platform APIs (such as DOMs) which should map to accessibility APIs (AT-SPI and potentially MSAA/UI Automation and others such as for the Mac) to provide a richer accessible document interface that addresses performance and semantic information issues when navigating today's complex documents. Acessibility APIs need to include the content, structure, attributes, and event handling of a DOM as well as additional services such as descriptive relationships and structure navigation.
Today, applications and assistive technologies use a combination of DOMs, accessibility APIs, native platform APIs, and hard-coded heuristics to provide an accessible user interface. To achieve portability and interoperability, applications need to be able to implement a cross-platform API like a DOM interface that can connect to platform-specific accessibility APIs.
Three types of APIs will be compared and classified: Document Objects Models (DOMs), accessibility APIs, and assistive technology services.
The DOMs will include the W3C XHTML DOM, OMG DOMs, Adobe's PDF DOM, and word processing, spreadsheet, and presentation DOMs from Open Office and Microsoft Office.
Accessibility APIs will include Microsoft's Active Accessibility (MSAA) and User Interface (UI) Automation, the Gnome Accessibility Toolkit (ATK) and Assistive Technology Service Provider Interface (AT-SPI), and Mac Accessibility API.
Assistive technology services in addition to those provided by accessibility APIs will include example interfaces like the IBM Home Page Reader Web Access Technology (WAT) and desirable use cases for accessing complex documents.
The classifications for the comparisons will include:
ID |
MSAA |
AT-SPI |
W3C DOM |
HPR WAT |
|---|---|---|---|---|
| NavigateByUnit |
|
|
|
|
|
NavigateChild |
|
|
|
N/A |
|
NumberOfChildren |
|
N/A |
|
N/A |
NavigateParent |
|
|
N/A |
N/A |
NavigateAll |
N/A |
N/A |
N/A |
|
Activate |
|
|
|
|
CollectAll |
N/A |
N/A |
N/A |
|
Search |
N/A |
N/A |
N/A |
|
|
NavigateLists |
N/A |
N/A |
|
|
|
NavigateWithinContainer |
N/A |
|
|
Direction: Right, Left, Up, Down, EdgeRight, EdgeLeft, EdgeUp, EdgeDown, SpanRight, SpanLeft, SpanUp, SpanDown, First, Last, Current |
SubmitForm |
N/A |
|
|
|
NavigateDocument |
N/A |
|
N/A |
|
| ID | MSAA |
AT-SPI |
W3C DOM |
HPR WAT |
|---|---|---|---|---|
Action |
|
|
|
|
Attributes |
|
|
|
|
Help |
|
|||
Keys |
|
|
|
|
|
Name |
|
|
|
|
| Role |
|
|
|
|
State |
|
|
N/A |
|
Value |
|
|
|
N/A |
ID Value Missing |
N/A |
|
N/A |
N/A |
Locale |
N/A |
|
N/A |
|
HyperLink |
N/A |
|
N/A |
N/A |
HyperText |
N/A |
|
N/A |
N/A |
Image |
N/A |
|
N/A |
N/A |
Table |
N/A |
|
N/A |
|
Component |
N/A |
|
N/A |
N/A |
Frames |
N/A |
N/A |
N/A |
|
Style |
N/A |
|
N/A |
|
ID |
MSAA |
AT-SPI |
W3C DOM |
HPR WAT |
|---|---|---|---|---|
Selection |
|
|
|
|
FocusPOR |
|
|
N/A |
|
Get Selection |
|
|
N/A |
|
Table Selection |
N/A |
|
N/A |
N/A |
ID |
MSAA |
AT-SPI |
W3C DOM |
HPR WAT |
|---|---|---|---|---|
Clipboard |
N/A |
|
N/A |
(uses Windows APIs) |
Attributes |
N/A |
|
N/A |
N/A |
Contents |
N/A |
|
|
N/A |
Selection |
N/A |
|
N/A |
N/A |
Range |
N/A |
|
N/A |
N/A |
Character Offset |
N/A |
|
N/A |
N/A |
ID |
MSAA |
AT-SPI |
W3C DOM |
HPR WAT |
|---|---|---|---|---|
Location |
|
|
N/A |
N/A |
Hit Test |
|
|
N/A |
N/A |
ID |
MSAA |
AT-SPI |
W3C DOM |
HPR WAT |
|---|---|---|---|---|
Labels |
N/A |
|
|
|
Targets |
N/A |
|
N/A |
N/A |
Table Headers |
N/A |
|
|
|
Document Summary |
N/A |
N/A |
N/A |
|
ID |
MSAA |
AT-SPI |
W3C DOM |
HPR WAT |
|---|---|---|---|---|
Listeners |
|
|
|
|
Document |
|
|
|
|
Pause Resume |
N/A |
|
N/A |
N/A |
Keyboard |
N/A |
|
|
|
Mouse |
N/A |
|
|
|
Mutations |
N/A |
|
|
N/A |
Application |
N/A |
|
N/A |
N/A |
Desktop |
N/A |
|
N/A |
N/A |
ID |
MSAA |
AT-SPI |
W3C DOM |
HPR WAT |
|---|---|---|---|---|
String |
N/A |
N/A |
N/A |
|
Integer |
N/A |
N/A |
N/A |
|
Boolean |
N/A |
N/A |
N/A |
|
Some further descriptions for some of the HPR WAT APIs.
/** * Perform the indicated jump and render content at landing spot. * * Repositions the document cursor as specified, then renders * any jump announcement (such as "table row 2") that is defined * for the jump. Will also usually render the next renderable * content that follows the landing spot. * * Caller is responsible for freeing the memory in the * returned packet sequence. */ DOMResult RenderJump(IDispatch**ppdispCollection, NavDirection direction, NavUnit unit); /** * Do a non-rendering jump across select boxes * * This is a jump that can be called on a select or option element. * This jump will land on a selected option, or if no selected options * in the direction of the jump, will land on the select element as * a start tag or end tag as appropriate. */ void JumpSelectOption(NavDirection direction); /** * Provides the "Play" function. * * Causes us to enter asynchronous mode. Collects PacketCollections * and returns them via the FirePacketsReady event, and also schedules * subsequent collections on future time slices. * @param fromWhere Indicates whether to start the Play from * the current POR or from the next item. */ HRESULT PlayBegin(long cookie,WATPLAYPARAMfromWhere); // activates access key: returns false if the char is not access key bool ActivateAccesskey( UINT nChar ); /** * Activate the DOM node * * For Anchor nodes with 'href' attributes, we will follow the * hyperlink. For form controls, we operate the control. * @param keys Enumeration representing important key combinations * pressed with Enter */ virtual void Activate(KeyCombokeys=KEY_None) = 0; /** * Perform an asynchronous CollectAll operation. * * Collected packets are broadcast back to all registered listeners * via the Collector.FireOnPacketsReady interface. */ STDMETHOD(CollectAllAsync)(long cookie); /* * * Search for text string, update document position when found. * * Search both web page content and metatext which we generate. * Render the result, with the serach string in its own packet with * unique attribute set (HPR 2.5 speaks the search text in link voice). */ STDMETHOD(RenderSearchAsync)(long iCookie, BSTR text, enum WATNAVDIRECTION direction, VARIANT_BOOL bMatchCase, VARIANT_BOOL bFirstSearch); /* * * Perform an asynchronous CollectLinks operation. * * Collected packets are broadcast back to all registered listeners * via the Collector.FireOnPacketsReady interface. */ STDMETHOD(CollectLinksAsync)(long cookie); /** * Perform the indicated jump for AdvTableReading Mode. * * Repositions the POR as specified, then renders * any jump announcement (such as "width 2") that is defined * for the jump. Will also render the cell content. */ HRESULT RenderATRJump(WATNAVATRDIRECTION direction, IDispatch** ppSeq);
/** * Retrieves the string value of a certain attribute * (instance version of class method of same name). * @param strName Name of the attribute to retrieve * @param strValue String to return the value in * @return true if the attribute was retrievable, false otherwise */ bool GetAttributeString( const CWatStr& strName, CWatStr& strValue) const; static bool GetAttributeString(IHTMLElement* pElem, const CWatStr& strName, CWatStr& strValue); /** * Collect the Accesskeys in the DOM and transmit them, one by one, to the * registered event listener. */ class CCollectAccesskeysIterator : public CIterator // CCoWatPacket /** * A CoWatPacket represents text from the DOM and is the basic class which * is used to exchange textual content between objects. * * Packets contain the text which will ultimately be rendered in one or more * views for the end user. The packet has "start" and "length" attributes * associated with it. The intent here is that Character and Word mode * navigation will be accomplished by selecting a subrange within the packet. * Language identification is provided with the "lang" attribute. * * All text retrieved from the DOM will have an association with a node * in the DOM and any character within that text has a specific offset. * Thus, any character within the text has a point of regard (POR) - which is * defined as having a node in the DOM and an offset. * */ class ATL_NO_VTABLE CCoWatPacket
/* * * Collect the content, asynchronously, between the POR saved by a SetMark * operation, and the current POR. * * Collected packets are broadcast back to all registered listeners * via the Collector.FireOnPacketsReady interface. */ STDMETHOD(CollectMarkAsync)(long cookie); /* * * Moves backwards, one step, through the jump history. * * A history of node selectors (aka node handles) associated with * jump departure points is maintained by the Collector. This * method moves the POR back one jump in the jump history. Only the * node handle is saved, the offset is lost. * * A separate history is maintained for each frame. * This method backs up only within the current frame. * That is, switches between frames are not maintained as part of the history. * * The jump history facility doesn't keep track of jumps in table * navigation mode, character, or word reading mode. * */ STDMETHOD(MovePorToUndoJump)(); /* * * Moves the POR to the position the event fired from * @param pEvt Pointer to the Event object */ DOMResult MovePorToEventElement(IHTMLEventObj* pEvt, int iFrame); /* * * Moves the POR to the mouse location the event fired from * * Updates the POR location and the offset to an x,y event position */ DOMResult MovePorToMouseEvent(IHTMLEventObj* pEvt, int iFrame); /* * * Collect the content, asynchronously, between SelBegin and SelEnd * Collected packets are broadcast back to all registered listeners */ DOMResult CollectSelectionAsync(int cookie);
LoginHelper,StateSet, andStreamableContentDOMException,createxxxx methods,insertBefore, replaceChild, removeChild, appendChild),NamedNodeMap,CharacterData,Comment, CDATASection,DocumentType,Notation,Entity,EntityReference,ProcessingInstruction,DOMImplementation,DocumentView,AbstractView.http://www.linuxbase.org/~gk4/a11y/idl/index.htmlhttp://msdn2.microsoft.com/en-us/library/ms697707.aspxhttp://www.w3.org/TR/2004/NOTE-DOM-Requirements-20040226/http://www.w3.org/DOM/DOMTR#dom2http://www.w3.org/TR/DOM-Bindings/http://www.w3.org/TR/2000/REC-DOM-Level-2-Style-20001113/http://www.w3.org/DOM/DOMTR#dom3http://www.w3.org/Math/DOM/webmaster@a11y.org. Please send any comments upon this document to accessibility-rfc@a11y.org.