This App Translates Speech for the Deaf

Skinner Cheng MS ’14 did the same things his classmates did. He participated in class discussions, gave presentations, and even co-authored a research paper with one of his professors. But in one way, his university experience was very different: Cheng is deaf.

Now he’s built a voice-recognition app that aims to transform the way the deaf and those who hear communicate. It’s called Ava, and it transcribes conversations in real time on a smartphone and allows deaf users to respond with text (similar to a chat conversation) that can be read on the screen or aloud by the phone. It’s the first app of its kind.

“We’re trying to build a bridge between deaf people and hearing people,” says the master’s in computer science alumnus, who is co-founder and chief technology officer of Ava, which is also the name of the company that makes the app.

Software engineer to startup star

Cheng, who is from Taiwan, came to USF after a decade as a software engineer and programmer in Asia. He wanted to take his computer science skills to the next level by working with coders in the Bay Area, home to the world’s most innovative tech engineers. USF was the best road to take him there, Cheng says.

In USF’s Master of Computer Science program, he bolstered his Android programming knowledge and learned iOS programming — skills he uses every day at his San Francisco-based startup. In his classes, captions — paid for by Student Disability Services — typed everything professors and students said so that Cheng could follow the lessons. He wants Ava to do the same thing, but with the added convenience and affordability of being able to pull it out of your pocket.

Cheng was still at USF when he connected with Ava, then a fledgling company looking to hire a student or graduate from the university who could make its app idea a reality. Cheng was the perfect match, both a longtime programmer and a potential app user. The company’s founders were so impressed by Cheng’s app prototype that they asked him to join them as a co-founder.

'Ava is part of me'

Ava launched Oct. 31, but the work is far from complete. Speech recognition technology is years from making perfect transcriptions and it hasn’t been able to overcome issues like accents and background chatter. Nevertheless, Cheng is dedicated to making his product as effective and user-friendly as possible — and he hopes that it will make a difference for the world’s 360 million deaf and hard-of-hearing people.

“When we started the company most of my time, except sleeping and eating, was spent coding,” he says. “Ava is part of me.”