Catherine Laws
IBM Accessibility Center, WW Strategic Platform Enablement
11400 Burnet Road, Bldg 903 Office 5B004, Austin, Texas 78758
Phone: (512) 838-4595, FAX: (512) 838-9367
E-mail: claws@us.ibm.com, Web: http://www.ibm.com/able
Initial draft: April, 2005
5/6/05 AT-SPI updates from Bill Haneman
Using today's accessibility APIs, developers of assistive technologies face difficult challenges trying to implement logical navigation of the content and structures of complex documents for users with vision, learning, physical, and cognitive impairments. Some of the challenges include:
To address these challenges, IBM will work with an open source standards working group in a standards organization such as FSG to do the following:
Create and propose a standard cross-platform, cross-document accessibility API extension for navigating all content and structure and for handling events and changes in complex documents.
Include new techniques for handling performance issues
Provide for extensibility and scriptability
As input, a "gap analysis" chart will be developed to categorize and compare all the different accessibility APIs and other APIs used by assistive technologies on different platforms today to access document content and structure information (including AT-SPI and ATK, Microsoft's MSAA and UI Automation, various Document Object Models, IBM Home Page Reader Web Access Technology (WAT), and others). Use cases will also be developed as input. Then the group should start drafting an API recommendation for extending cross-platform APIs (such as DOMs) which should map to accessibility APIs (AT-SPI and potentially MSAA/UI Automation and others such as for the Mac) to provide a richer accessible document interface that addresses performance and semantic information issues when navigating today's complex documents. Accessibility APIs need to include the content, structure, attributes, and event handling of a DOM as well as additional services such as descriptive relationships and structure navigation.
Today, applications and assistive technologies use a combination of DOMs, accessibility APIs, native platform APIs, and hard-coded heuristics to provide an accessible user interface. To achieve portability and interoperability, applications need to be able to implement a cross-platform API like a DOM interface that can connect to platform-specific accessibility APIs.
Three types of APIs will be compared and classified: Document Objects Models (DOMs), accessibility APIs, and assistive technology services.
The DOMs will include the W3C XHTML DOM, OMG DOMs, Adobe’s PDF DOM, and word processing, spreadsheet, and presentation DOMs from Open Office and Microsoft Office.
Accessibility APIs will include Microsoft’s Active Accessibility (MSAA) and User Interface (UI) Automation, the Gnome Accessibility Toolkit (ATK) and Assistive Technology Service Provider Interface (AT-SPI), and Mac Accessibility API.
Assistive technology services in addition to those provided by accessibility APIs will include example interfaces like the IBM Home Page Reader Web Access Technology (WAT) and desirable use cases for accessing complex documents.
The classifications for the comparisons will include: Navigations and Actions, Attributes, Selection and Focus, Editing, Spatial Mapping, Events, Relationships, and Preferences.
|
ID |
MSAA |
AT-SPI |
W3C DOM |
HPR WAT |
|---|---|---|---|---|
|
NavigateByUnit |
IAccessible::accNavigate NAVDIR_UP, NAVDIR_DOWN, NAVDIR_FIRSTCHILD, NAVDIR_LASTCHILD , NAVDIR_LEFT, NAVDIR_ RIGHT, NAVDIR_NEXT, NAVDIR_PREVIOUS |
Accessible::getChildAtIndex |
Node or TreeWalker parentNode firstChild lastChild previousSibling nextSibling TreeWalker previousNode nextNode CharacterData::substringData NodeIterator:: nextNode previousNode |
RenderJump Direction: Previous, Current, Next, First, Last Unit: Word, Character, Item, Link, Form Control, Table, Heading, Block JumpSelectOption |
|
NavigateChild |
IAccessible::get_accChild |
Accessible::getChildAtIndex |
Node::childNodes |
|
|
NumberOfChildren |
IAccessible::get_accChildCount |
|
Node::hasChildNodes |
|
|
NavigateParent |
IAccessible::get_accParent |
Accessible::getIndexInParent |
|
|
|
NavigateAll |
|
|
|
PlayBegin fromPOR, FromNextItem, FromCursor |
|
Activate |
IAccessible::accDoDefaultAction |
Action::doAction |
|
ActivateLink ActivateAccesskey IWatNode::Activate |
|
CollectAll |
|
|
|
CollectAllAsync (for printing, saving) |
|
Search |
|
|
|
RenderSearchAsync First, Previous , Current, Next Match Case |
|
NavigateLists |
|
|
Document::GetElementsByTagName
|
CollectLinksAsync (Links List) |
|
NavigateWithinContainer |
|
Accessible::getChildAtIndex |
|
RenderATRJump (for table nav) Direction: Right, Left, Up, Down, EdgeRight, EdgeLeft, EdgeUp, EdgeDown, SpanRight, SpanLeft, SpanUp, SpanDown, First, Last, Current |
|
SubmitForm |
|
Action::doAction |
HTMLFormElement::Submit HTMLFormElement::Reset |
RunImpliedSubmission (for forms without submit button) RunFormSubmission ConfigureFormSubmission |
|
NavigateDocument |
|
Action::doAction |
|
Back Forward Refresh NextFrame |
|
ID |
MSAA |
AT-SPI |
W3C DOM |
HPR WAT |
|---|---|---|---|---|
|
Action |
IAccessible::get_accDefaultAction (such as check, press) |
Action::getDescription Action::getName |
|
IWatNode:IsActivatable |
|
Attributes |
IAccessible::get_accDescription (such as alt, title) |
Accessible:: |
Element:: getAttributes setAttributes removeAttributes getAttributeNode setAttributeNode removeAttributeNode Node::hasAttributes
|
GetActivateableNode (then use IHTMLElement (DOM interface) to get object attributes) IWatNode:GetAttributes IWatNodeSelector:: GetAttributeString
|
|
Help |
IAccessible::get_accHelp IAccessible::get_accHelpTopic BSTR* pszHelpFile |
|
|
|
|
Keys |
IAccessible::get_accKeyboardShortcut |
Action::getKeyBinding |
Element:: getAttributes |
CollectAccesskeysIterator GetAccessKeysList IWatNodeSelector:: GetAttributeString IsTabindexPresented |
|
Name |
IAccessible::get_accName |
Accessible:: getName
|
Node::nodeName Attr::name |
IWatNodeSelector:: GetAttributeString |
|
Role |
IAccessible::get_accRole ROLE_SYSTEM_ANIMATION, ROLE_SYSTEM_APPLICATION, ROLE_SYSTEM_BORDER, ROLE_SYSTEM_BUTTONDROPDOWN , ROLE_SYSTEM_BUTTONDROPDOWNGRID, ROLE_SYSTEM_BUTTONMENU , ROLE_SYSTEM_CHART, ROLE_SYSTEM_CHECKBUTTON, ROLE_SYSTEM_CLIENT , ROLE_SYSTEM_COLUMN , ROLE_SYSTEM_COLUMNHEADER , ROLE_SYSTEM_COMBOBOX , ROLE_SYSTEM_CURSOR, ROLE_SYSTEM_GROUPING , ROLE_SYSTEM_HELPBALLOON , ROLE_SYSTEM_HOTKEYFIELD , ROLE_SYSTEM_INDICATOR, ROLE_SYSTEM_IPADDRESS, ROLE_SYSTEM_LINK, ROLE_SYSTEM_MENUITEM, ROLE_SYSTEM_MENUPOPUP, ROLE_SYSTEM_OUTLINE, ROLE_SYSTEM_OUTLINEBUTTON, ROLE_SYSTEM_OUTLINEITEM, ROLE_SYSTEM_PAGETAB, ROLE_SYSTEM_PAGETABLIST, ROLE_SYSTEM_PANE , ROLE_SYSTEM_PROGRESSBAR, ROLE_SYSTEM_PROPERTYPAGE, ROLE_SYSTEM_PUSHBUTTON, ROLE_SYSTEM_RADIOBUTTON, ROLE_SYSTEM_ROW, ROLE_SYSTEM_ROWHEADER, ROLE_SYSTEM_SCROLLBAR, ROLE_SYSTEM_SEPARATOR, ROLE_SYSTEM_SLIDER, ROLE_SYSTEM_SOUND, ROLE_SYSTEM_SPINBUTTON, ROLE_SYSTEM_SPLITBUTTON, ROLE_SYSTEM_STATICTEXT, ROLE_SYSTEM_STATUSBAR, ROLE_SYSTEM_TABLE, ROLE_SYSTEM_TEXT, ROLE_SYSTEM_TITLEBAR, ROLE_SYSTEM_TOOLBAR, ROLE_SYSTEM_TOOLTIP, ROLE_SYSTEM_WHITESPACE, ROLE_SYSTEM_WINDOW |
Accessible:: getLocalizedRoleName |
Node::nodeName (depends on nodeType) Node::nodeType Element Attribute Text CDATASection EntityReference Entity ProcessingInstruction Comment Document DocumentType DocumentFragment Notation
|
IWatNodeSelector:: GetAttributeString
CoWatPacketAttributes: get_ActivateableEvents put_ActivateableEvents get_LinkHref put_LinkHref get_EltClass put_EltClass EltClass_DEFAULT EltClass_AREA, EltClass_EmbededObject, EltClass_BUTTON, EltClass_CAPTION, EltClass_DD, EltClass_DL, EltClass_DT, EltClass_FORM, EltClass_Heading, EltClass_IMG, EltClass_LI, EltClass_INPUT_Button, EltClass_INPUT_Checkbox, EltClass_INPUT_File, EltClass_INPUT_Image, EltClass_INPUT_Password, EltClass_INPUT_Radio, EltClass_INPUT_Reset, EltClass_INPUT_Submit, EltClass_INPUT_Text, EltClass_LABEL, EltClass_MAP, EltClass_OL, EltClass_OPTION, EltClass_SELECT, EltClass_TABLE, EltClass_TD, EltClass_TH, EltClass_TR, EltClass_TEXTAREA, EltClass_UL get_SearchResult put_SearchResult get_Item put_Item get_Header (H1, H2, etc) put_Header get_MetaText put_MetaText get_ListItem put_ListItem get_Link put_Link put_VisitedLink WatBrowser::isFrameSet |
|
State |
STATE_SYSTEM_CHECKED, STATE_SYSTEM_COLLAPSED, STATE_SYSTEM_DEFAULT, STATE_SYSTEM_EXPANDED, STATE_SYSTEM_EXTSELECTABLE, STATE_SYSTEM_FOCUSABLE, STATE_SYSTEM_FOCUSED, STATE_SYSTEM_HASPOPUP, STATE_SYSTEM_HOTTRACKED, STATE_SYSTEM_INVISIBLE, STATE_SYSTEM_MOVEABLE, STATE_SYSTEM_MULTISELECTABLE, STATE_SYSTEM_NORMAL , STATE_SYSTEM_OFFSCREEN, STATE_SYSTEM_PRESSED, STATE_SYSTEM_PROTECTED, STATE_SYSTEM_READONLY, STATE_SYSTEM_SELECTABLE, STATE_SYSTEM_SELECTED, STATE_SYSTEM_SELFVOICING, STATE_SYSTEM_SIZEABLE, STATE_SYSTEM_TRAVERSED, STATE_SYSTEM_UNAVAILABLE |
STATE_INVALID STATE_OPAQUE STATE_LAST_DEFINED |
|
IWatNode::IsDisabled IWatNode:IsReadOnly IWatNodeSelector:: GetAttributeString IsVisible |
|
Value |
IAccessible::get_accValue |
AccessibleValue:: |
Node::nodeValue Attr::value Attr::specified |
|
|
|
|
Accessible::isEqual |
|
|
|
Locale |
|
Application::getLocale |
|
IWatNodeSelector::Language |
|
HyperLink |
|
Hyperlink:: getObject getURI isValid |
|
|
|
HyperText |
|
Hypertext:: getLink (returns Hyperlink) getLinkIndex getNLinks |
|
|
|
Image |
|
Image:: getImageExtents getImagePosition getImageSize |
|
|
|
Table |
|
Table:: getAccessibleAt getColumnAtIndex getColumnExtentAt getIndexAt getRowAtIndex getRowExtentAt |
|
RenderWhereAmI getTableSummary RenderATRJump (includes headers) |
|
Component |
|
Component:: getSize getPosition getMDIZOrder getLayer getExtents getAccessibleAtPoint contains |
|
|
|
Frames |
|
|
|
GetNumberOfDocuments (frames) |
|
Style |
|
AccessibleText:: |
|
GetBodyStyleAttributes (for Zoom) IWatNodeSelector:: put_StyleAttributes get_StyleAttributes |
|
ID |
MSAA |
AT-SPI |
W3C DOM |
HPR WAT |
|---|---|---|---|---|
|
Selection |
IAccessible::accSelect SELFLAG_NONE SELFLAG_TAKEFOCUS SELFLAG_TAKESELECTION SELFLAG_EXTENDSELECTION SELFLAG_ADDSELECTION SELFLAG_REMOVESELECTION |
Selection:: clearSelection selectAll selectChild deselectSelectedChild Text:: addSelection removeSelection setSelection |
Range:: setStart setEnd setStartBefore setStartAfter Collapse selectNode selectNodeContents CompareBoundaryPoints deleteContents insertNode surroundContents cloneRange toString detach DocumentRange:: createRange |
CollectMarkAsync (a range) AsyncSelectAll |
|
FocusPOR |
IAccessible::get_accFocus |
Component:: grabFocus registerFocusHandler deregisterFocusHandler |
|
SetPOR GetPOR ResetPOR MovePorToRenderable MovePorToUndoJump MovePorToEventElement MovePorToMouseEvent |
|
Get Selection |
IAccessible::get_accSelection |
Selection:: getSelectedChild isChildSelected Text:: getNSelections getSelection |
|
MovePORToSelection get_Selection put_Selection CollectSelectionAsync |
|
Table Selection |
|
Table:: addColumnSelection addRowSelection getSelectedColumns getSelectedRows isColumnSelected isRowSelected isSelected removeColumnSelection removeRowSelection |
|
|
|
ID |
MSAA |
AT-SPI |
W3C DOM |
HPR WAT |
|---|---|---|---|---|
|
Location |
IAccessible::accLocation pxLeft and pyTop screen coordinates pcxWidth and pcyHeight in pixels |
Component:: contains getExtents getPosition getSize getLayer [z order] getMDIZOrder [z order] |
|
|
|
Hit Test |
IAccessible::accHitTest xLeft and yTop |
getAccessibleAtPoint |
|
|
Other AT-SPI classes not included in comparison: LoginHelper, StateSet, StreamableContent
Other DOM interfaces not included in comparison: DOMException, Document createxxxx methods, Node methods (insertBefore, replaceChild, removeChild, appendChild), NamedNodeMap, CharacterData, Comment, CDATASection, DocumentType, Notation, Entity, EntityReference, ProcessingInstruction, namespace methods in DOM Level 2, DOMImplementation, DocumentView, AbstractView
Still to do:
MSAA Extensions for OpenOffice and DHTML
DOM Level 2 Style, HTML
DOM Level 3
DOMs for Adobe PDF, OpenOffice Write/Calc/Impress, MS Office Word/Excel/PowerPoint, Firefox DOM, IE DOM
MS UI Automation
Mac Accessibility API
MS Active Accessibility Text Services?
Some further descriptions for some of the HPR WAT APIs.
/**
* Perform the indicated jump and render content at landing spot.
*
* Repositions the document cursor as specified, then renders
* any jump announcement (such as "table row 2") that is defined
* for the jump. Will also usually render the next renderable
* content that follows the landing spot.
* Caller is responsible for freeing the memory in the
* returned packet sequence.
*/
DOMResult RenderJump(IDispatch** ppdispCollection, NavDirection direction, NavUnit unit);
/**
* Do a non-rendering jump across select boxes
*
* This is a jump that can be called on a select or option element.
* This jump will land on a selected option, or if no selected options
* in the direction of the jump, will land on the select element as
* a start tag or end tag as appropriate.
*/
void JumpSelectOption(NavDirection direction);
/**
* Provides the "Play" function.
*
* Causes us to enter asynchronous mode. Collects PacketCollections
* and returns them via the FirePacketsReady event, and also schedules
* subsequent collections on future time slices.
* @param fromWhere Indicates whether to start the Play from
* the current POR or from the next item.
*/
HRESULT PlayBegin(long cookie,WATPLAYPARAM fromWhere);
// activates access key: returns false if the char is not access key
bool ActivateAccesskey ( UINT nChar );
/**
* Activate the DOM node
*
* For Anchor nodes with 'href' attributes, we will follow the
* hyperlink. For form controls, we operate the control.'
* @param keys Enumeration representing important key combinations
* pressed with Enter
*/
virtual void Activate(KeyCombo keys=KEY_None) = 0;
/**
* Perform an asynchronous CollectAll operation.
*
* Collected packets are broadcast back to all registered listeners
* via the Collector.FireOnPacketsReady interface.
*/
STDMETHOD(CollectAllAsync)(long cookie);
/**
* Search for text string, update document position when found.
*
* Search both web page content and metatext which we generate.
* Render the result, with the serach string in its own packet with
* unique attribute set (HPR 2.5 speaks the search text in link voice).
*/
STDMETHOD(RenderSearchAsync)(long iCookie, BSTR text, enum WATNAVDIRECTION direction, VARIANT_BOOL bMatchCase, VARIANT_BOOL bFirstSearch);
/**
* Perform an asynchronous CollectLinks operation.
*
* Collected packets are broadcast back to all registered listeners
* via the Collector.FireOnPacketsReady interface.
*/
STDMETHOD(CollectLinksAsync)(long cookie);
/**
* Perform the indicated jump for AdvTableReading Mode.
*
* Repositions the POR as specified, then renders
* any jump announcement (such as "width 2") that is defined
* for the jump. Will also render the cell content
*/
HRESULT RenderATRJump(WATNAVATRDIRECTION direction, IDispatch** ppSeq);
/**
* Retrieves the string value of a certain attribute
* (instance version of class method of same name).
* @param strName Name of the attribute to retrieve
* @param strValue String to return the value in
* @return true if the attribute was retrievable, false otherwise
*/
bool GetAttributeString( const CWatStr& strName, CWatStr& strValue) const;
static bool GetAttributeString(IHTMLElement* pElem, const CWatStr& strName, CWatStr& strValue);
/**
* Collect the Accesskeys in the DOM and transmit them, one by one, to the
* registered event listener.
*/
class CCollectAccesskeysIterator : public CIterator
// CCoWatPacket
/**
* A CoWatPacket represents text from the DOM and is the basic class which
* is used to exchange textual content between objects.
*
* Packets contain the text which will ultimately be rendered in one or more
* views for the end user. The packet has "start" and "length" attributes
* associated with it. The intent here is that Character and Word mode
* navigation will be accomplished by selecting a subrange within the packet.
* Language identification is provided with the "lang" attribute.
*
* All text retrieved from the DOM will have an association with a node
* in the DOM and any character within that text has a specific offset.
* Thus, any character within the text has a point of regard(POR) - which is
* defined as having a node in the DOM and an offset.
*
*/
class ATL_NO_VTABLE CCoWatPacket
/**
* Collect the content, asynchronously, between the POR saved by a SetMark
* operation, and the current POR.
*
* Collected packets are broadcast back to all registered listeners
* via the Collector.FireOnPacketsReady interface.
*/
STDMETHOD(CollectMarkAsync)(long cookie);
/**
* Moves backwards, one step, through the jump history.
*
* A history of node selectors (aka node handles) associated with
* jump departure points is maintained by the Collector. This
* method moves the POR back one jump in the jump history. Only the
* node handle is saved, the offset is lost.
* <P>
* A separate history is maintained for each frame.
* This method backs up only within the current frame.
* That is, switches between frames are not maintained as part of the history.
* <P>
* The jump history facility doesn't keep track of jumps in table
* navigation mode, character, or word reading mode.
*
*/
STDMETHOD(MovePorToUndoJump)();
/**
* Moves the POR to the position the event fired from
* @param pEvt Pointer to the Event object
*/
DOMResult MovePorToEventElement(IHTMLEventObj* pEvt, int iFrame);
/**
* Moves the POR to the mouse location the event fired from
*
* Updates the POR location and the offset to an x,y event position
*/
DOMResult MovePorToMouseEvent(IHTMLEventObj* pEvt, int iFrame);
/**
* Collect the content, asynchronously, between SelBegin and SelEnd
* Collected packets are broadcast back to all registered listeners
*/
DOMResult CollectSelectionAsync(int cookie);
AT_SPI_IDL Documentation
http://www.linuxbase.org/~gk4/a11y/idl/index.html
Microsoft Active Accessibility User Interfaces
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/msaa/msaaccrf_5q05.asp
W3C Document Object Model (DOM) Specifications