Stellenbosch University
Welcome to Stellenbosch University
PhD study can help the visually impaired read mathematical equations, diagrams
Author: Corporate Communication & Marketing / Korporatiewe Kommunikasie & Bemarking [Alec Basson]
Published: 13/02/2023

​Blind or visually impaired persons access documents by using a screen reader that reads a text out loud or displays it on a special hardware braille device. However, a large body of technical and scientific material remains inaccessible to them because mathematical equations and diagrams are represented as images and not as text that a screen reader can read.

This is according to Dr Rynhardt Kruger who recently obtained his doctorate in Electronic Engineering at Stellenbosch University (SU). For his PhD, titled Technical document accessibility, Kruger developed methods that could make it easier for blind or visually impaired people to read non-textual graphical information in electronic documents (see video). His supervisors were Profs Thomas Niesler and Febe de Wet from the Digital Signal Processing Research Group in SU's Department of Electrical and Electronic Engineering. Kruger works as a researcher and developer in voice computing at the Council for Scientific and Industrial Research.

 


  • Cellphone users click here for the video.

As someone who was born blind, Kruger says he knows first-hand how difficult it is to read equations and diagrams in scientific documents. “During my studies, it was not always easy to access the material in my textbooks. I realised that many technical documents do not adhere to existing accessibility standards for blind or visually impaired people." Despite this, Kruger became the first blind person at SU to obtain both his BSc honours and master's degrees in computer science. For his Honours project, he developed a programme that allows blind musicians to study music notation. For his master's degree, Kruger researched methods to allow blind people to access online virtual worlds.

​He is passionate about broadening access for people with disabilities and says it is important to continuously search for assistive technologies that blind or visually impaired people can use to read equations and diagrams in scientific texts.RynhardtKruger_pic_orig.jpg

“Currently, graphical material must first be converted to an accessible format before they can read it. The problem is that diagrams are still only accessible through textual descriptions or tactile diagrams, which are costly to produce. Textual descriptions are often inadequate to describe the complex relations between components of diagrams such as lines and rectangles or the contours of a geographical map.

“Also, when equations are available in an accessible format, they must be read in a linear fashion since this is how mathematics is expressed in currently accessible formats. This hinders blind or visually impaired readers from accessing the visual layout of the equation."

According to Kruger, blind or visually impaired readers can use his methods to explore diagrams, geometric shapes and equations through sounds and as gestures on a touch screen or by text-based navigation.

He explains: “Diagram content is played to users as a special form of sound from which they could interpret the content being explored. For example, in the case of a descending line from upper-left to lower-right on the screen, the resulting sound would be a tone descending in pitch.

“I devised two gestures that blind or visually impaired people could use to explore content on a touch screen. When one finger is dragged across the screen, the content directly under the finger turns into a sound. When two fingers are dragged across the screen, the line between the two fingers acts as a scanner, and all content encountered by that line becomes a sound."

Kruger says that to convey an equation to readers, the different elements of the equation are represented as “rooms that they can walk through". “The directions between rooms reflect the visual layout of the equation, so that, for example, the reader would have to move right in order to visit an element which is situated visually right of the element where the user is currently located."

He points out that the equations used in his study were similar in format to other inaccessible equations that readers might encounter in published material.

Kruger emphasises the significance of his study and says it showed that blind or visually impaired readers are able to identify the basic components of a diagram and to explore previously inaccessible equations using his approach.

“My methods allow blind or visually impaired readers to access graphical material directly and read equations and explore diagrams by just using a tablet instead of having to pay for costly hardware or conversion services.

“These methods will help to get an idea of a diagram's shape where the shape itself is important (such as a previously unknown geometrical shape), and for accessing already published material which was produced without taking accessibility standards into account."

Kruger says he would like to see his methods built into a book reader application that will allow readers to study diagrams and equations directly from books.

  • Main photo by Gerd Altmann from Pixabay. Photo 1: Dr Rynhardt Kruger​