Audiom: The World’s Most Inclusive Map Viewer

Cross-Sensory Digital Map Project Development
XR Navigation in partnership with The Smith-Kettlewell Eye Research Institute
Submitted by Brandon Biggs and developed in collaboration with Michelle Zorrilla of IMPACT and Katie Guyot and Kathleen Murphy of the Center on KTDRR

Focus

This project consists of developing a digital map creation platform, Audiom, that will allow blind and visually impaired (BVI) individuals to access maps equally in audio and visual formats. Audiom is designed by and for BVI individuals. After developing an initial prototype, the Audiom team engaged in codesign sessions and interviews with BVI individuals to determine the product’s usefulness, test its usability, and identify desirable features. The team also engaged potential customers, such as businesses and government offices, to understand how this technology could be deployed beyond a research setting. Future efforts will focus on identifying and improving the features that would make Audiom useful to BVI individuals in their everyday lives.

Context

BVI individuals have limited access to spatial map information. Commonly used existing tools include tactile maps and turn-by-turn directions on navigation apps. These tools, when available, tend to present highly simplified map information. In codesign sessions conducted with BVI individuals, the project team found that BVI users desire more detailed information than they can typically obtain from existing tools. Audiom will enable BVI individuals to access detailed map information entirely in audio or in a high-contrast map viewer. These maps may be used to virtually navigate a particular space, like a museum, or to represent geographic information, like the features of a country.

An example of a map created with Audiom appears below. Click on the text box that says “Audio Map ON” to begin using the map, then navigate with the arrow keys. Audiom’s maps can be embedded in any webpage.

Audiom is currently under development by researchers at XR Navigation in partnership with the Smith-Kettlewell Eye Research Institute.

“Natural Laboratory” of Audio Games

The project was initially inspired by audio games, which are designed for blind users and can be played completely in audio. These games have been available since 1972 and often use audio maps to present geographical information about game worlds. The project treats audio games as a “natural laboratory” for designing audio maps (Biggs et al., 2018). By purchasing and playing audio games, BVI consumers test a variety of approaches to audio maps and express their preferences for different map designs. Games that provide a better user experience are more likely to survive in a demanding consumer market. In this way, each iteration of audio game technology incorporates market feedback from BVI consumers.

The Audiom team built an initial prototype of an audio map based on conventions observed in audio games. For instance, one of the prototype’s navigational modes involves moving between tiles using the arrow keys on a computer, with tiles represented by short sound and speech clips, much like the grid-based maps used in some audio games.

KT Activities

The project’s knowledge translation (KT) strategies focus primarily on assistive technology tech transfer (ATTT). ATTT is “the process of moving, selling, transferring or disseminating an assistive technology from the university setting, a business or organization to the market and into the hands of those that may benefit from its use” (Zorrilla, 2021). Successful ATTT involves working with users and potential customers to establish whether there is a need for the product and to identify what features the product should have. To address these questions, the project has used the following KT strategies:

  1. Lead researcher shares lived experience of target audience: The research is participant-led. The lead developer is blind and has designed solutions based in part on his own lived experience.
  2. Codesign with ongoing product testing: Researchers conducted codesign sessions with BVI individuals about their experiences with maps and what they would look for in an “ideal” map experience. As part of the codesign approach, the project is performing ongoing testing of Audiom with BVI individuals to understand how they use the tool and what features should be added or improved.
  3. Review of existing map technology to identify unmet needs: Researchers reviewed the map tools already available in commercial and research settings to identify unmet needs that their product can address.
  4. Consult with targeted product distributors: Researchers interviewed individuals who work with map data in business or government settings to find out what would make them want to use Audiom to design inclusive maps.

Strategies 2 through 4 are discussed in more detail below.

Codesign and Testing

The project has conducted two codesign sessions to develop Audiom in a way that will address users’ navigational needs. Codesign involves partnering with stakeholders—in this case, BVI individuals—to develop usable solutions that meet their needs. Codesign sessions consisted of both focus groups and individual interviews that were analyzed using an inductive coding approach. In addition, some sessions involved testing a prototype to ensure usability.

The first codesign session took place in 2019, after an initial Audiom prototype already existed. This session had two goals: first, to understand participants’ ideal navigational experiences; and second, to test the usability of the initial Audiom prototype. The study recruited 10 blind male participants from a forum post on audiogames.net (Biggs, 2019; Biggs et al., 2019). Participants were asked to codesign their ideal nonvisual navigational experiences without considering whether their ideas were feasible given current technology. Researchers identified several desirable features that the developers could incorporate into a future iteration of Audiom, including a mobile version and more detailed information, such as bushes, sidewalks, poles, and doors. Researchers also identified a need for long-term research into dynamic virtual reality and holographic tactile displays.

Participants were then asked to use the prototype map to complete eight tasks, such as finding routes between locations or obtaining an overview of what is near a location. Researchers timed how long it took participants to complete each task and evaluated participants’ “mental task load” using the National Aeronautics and Space Administration (NASA) Task Load Index, an established method of obtaining a subjective score for mental load when completing a task. Researchers also interviewed participants about their feedback on the prototype. Results indicated that the interface was easy to learn and navigate, that participants had unique navigational styles and preferred using their own screen readers, and that participants needed user interface features that made it easier to understand and answer questions about spatial properties and relationships. Participants were able to learn to use the interface in approximately 9 minutes, much faster than the hour-plus training time for an interactive data sonification product used for comparison (Biggs et al., 2019).

In March 2021, the team held a codesign session over Zoom with six BVI participants to investigate attitudes regarding travel during the COVID-19 pandemic and to develop map-based solutions. As part of this study, participants engaged with a map created with the Audiom prototype and shared their feedback on specific features. Analysis of this codesign is in progress.

Review of Existing Technology to Identify Unmet Needs

To contextualize Audiom among other inclusive map products, researchers reviewed existing auditory or cross-sensory map tools in both research settings and the commercial market. Currently, there is no commercial nonvisual map viewing tool that shows points, polygons, and lines. Several prototypes have been designed in research settings, but these products have not been disseminated to a consumer market, and none can be used on a mobile platform. In addition, very few related projects have engaged in technology transfer activities to promote their use and usability among BVI consumers. Although one project did use a codesign approach, the interview process was not described in detail. This review served as a foundation for Audiom’s KT activities by identifying the gaps in current technology and building an understanding of how Audiom can address the unmet needs of potential users.

Customer Engagement: Consult With Targeted Product Distributors

The project’s goal is to make inclusive maps free and widely available to BVI individuals. To do this, the team has engaged potential customers—in this case those intermediaries who would purchase the software for further distribution to users—to find out how Audiom can address their needs and, in turn, their business and agency customers. This project’s target customers, i.e., those who would be the initial entities paying for the tool, are businesses, government agencies, and nonprofits that would use inclusive maps on their websites. Researchers have interviewed approximately 30 individuals in the public and private sectors—generally individuals who work with geographic information system information—about their experience with building accessible maps. The research team also conducted a survey of government resources to assess existing approaches to making accessible maps.

These efforts revealed that the current state of inclusive maps is, in a word, dismal. Interview sources broadly agreed that a lack of accessible map technology is a problem that requires a solution. Businesses and government sources suggested that they would pay for a product like Audiom if it would reduce their costs or improve the experiences of BVI visitors in their spaces. For instance, museums have expressed an interest in developing stronger connections with BVI visitors, and a climate research center hopes to build inclusive maps about climate-related information.

A key finding from these interviews was that potential customers want a product that would be useful to low-vision users as well as blind users. As a direct result of this recommendation, the team added a high-contrast map viewer and now considers the product to be an “inclusive” rather than a purely auditory map viewer.

Next Steps

Next, the team plans to share a new version of its auditory map tool with BVI individuals to find out what features would make them want to use Audiom maps in their everyday lives. A new codesign study will involve 15 blind individuals interviewed repeatedly over a period of 6 months, with usage data tracked on a weekly basis, to identify additional features or improvements to existing features based on user preferences. Specific research questions may compare the level of spatial knowledge people obtain from different modes of the map viewer. The team hopes to release the tool to the general public in the near future after incorporating lessons learned from the codesign.

Acknowledgements

  • "Leveraging Maps and Computer Vision to Support Indoor Navigation for Blind Travelers - COVID Supplement", National Eye Institute of the National Institutes of Health, R01EY029033
  • "Cross-Sensory Digital Map Project Development", National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR), 90IFDV0020
  • "Rehabilitation Engineering Research Center (RERC) on Low Vision and Blindness", National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR), 90RE5024
  • IMPACT Startup through the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR), 90DPKT0002

Contact Information

Organization: XR Navigation
Website: https://audiom.xrnavigation.io/

Organization: Smith-Kettlewell Eye Research Institute
Mailing Address: 2318 Fillmore Street, San Francisco, CA 94115-1813
Website: https://www.ski.org/
Key contact: Brandon Biggs brandon.biggs@ski.org

The Smith Kettlewell SKERI Eye Research Institute

References

Biggs, B. (2019). Designing accessible nonvisual maps. [Master’s thesis, Ontario College of Art & Design University]. http://openresearch.ocadu.ca/id/eprint/2606/5/Biggs_Brandon_2019_MDES_INCD_MRP.pdf

Biggs, B., Coughlan, J., & Coppin, P. (2019, June 23–27). Design and evaluation of an audio game-inspired auditory map interface [Paper presentation]. The 25th International Conference on Auditory Display (ICAD 2019) Northumbria University, Newcastle upon Tyne, UK. https://smartech.gatech.edu/handle/1853/61525

Biggs, B., Yusim, L., & Coppin, P. (2018, June 10–15). The audio game laboratory: Building maps from games [Paper presentation]. The 24th International Conference on Auditory Display (ICAD 2018), Houghton, MI. http://icad2018.icad.org/wp-content/uploads/2018/06/ICAD2018_paper_51.pdf

Zorrilla, M. (2020). Increasing assistive technology tech transfer success through the IMPACT Center’s three-phase assistive technology training program. In Center on Knowledge Translation for Disability & Rehabilitation Research (Ed.), KT casebook (8th ed.). https://ktdrr.org/products/ktcasebook/impact-assistive-technology-training-program.html