Express

Making screen readers in unity easy

September 2018

c-sharp, c++, java, native plugins, Unity, Tool, screen reader,

About

For my grad work (that is happening at the moment) I’m looking at the accessibility landscape.
As case study I’m making screen reader support easy in unity.
To do this I have needed to create an c++ dll to interact with windows and an java aar to interact with android.

This it still being worked on and is not final.

Code

Csharp

This is how a root object gets selected. A root object is the presumed root of an element, if you have a test label that is part of a button object then the button is the presumed root. This gets done for all default ui objects.

  /// <summary> tries to find the root of an element
  /// <para>It could be that the current gameobject is part of a button or slider. if this is the case we need to work from them and not the current gameobject</para>
  /// </summary>
  /// <param name="go">the gameobject we need to root</param>
  /// <returns>the root gameobject. could be the same as input<returns>
  public static GameObject GetUIElementRoot(GameObject go)
  {
    if (go == null)
      return go;
    
    // if the object has an selectable then it is a root
    var selectable = go.GetComponent<Selectable>();
    if (selectable)
      return go;

    // if it doesnt have a parent then it returns null
    // this is so that we dont keep looking for parents until we get the canvas
    // happens with image and raw image
    var parent = go.transform.parent;
    if (parent == null)
      return null;
    
    // recursively look upwards for a root
    var newRoot = GetUIElementRoot(parent.gameObject); 
    if (newRoot != null)
      return newRoot;
    
    // if newRoot returned null because we found no valid parent then assume this object is the root
    return go;
  }

The rest of the code can be found here, Native Plugins or Unity Code