Why voice AI still struggles to hear Africa

Imagine talking to a voice assistant the way you talk to a friend. It answers in your language. It says "ehe" while you explain what you need, waits while you search for the right word, and lets you cut in. You would hardly think of it as technology. You would think of it as a conversation.

Talking to a voice assistant in most of South Africa's languages is not like this, yet.

Many voice systems today treat a conversation as a series of turns at a microphone. They wait for you to stop, then reply. People do not talk that way: in conversation the next speaker usually comes in within a fraction of a second, in languages across the world [1]. And when you mix languages in one sentence, as many people across Africa do [2, 3], these systems lose you altogether.

This is not a small problem for the people who would use these systems most. For many, speaking to someone is the easiest way to reach the services they need, and the language they are most at ease in is often not the one those services answer in. A voice that cannot follow them in their own language, at the pace they actually talk, leaves them with a worse service, or none at all.

To make voice AI as natural in Africa's languages as a conversation with a neighbour, we need models that are full-duplex: models that listen and speak at the same time, the way people do. A full-duplex model does not wait for silence. This is what we are building at Pan-African AI Lab, beginning with the eleven official spoken languages of South Africa.

None of this is solved. We believe it can be done, and that doing it well will change who is served by the voice systems now being built everywhere else. We will share what we learn here as the work moves forward.

References

1. Stivers, T., Enfield, N. J., Brown, P., Englert, C., Hayashi, M., Heinemann, T., Hoymann, G., Rossano, F., de Ruiter, J. P., Yoon, K.-E., and Levinson, S. C. (2009). Universals and cultural variation in turn-taking in conversation. Proceedings of the National Academy of Sciences 106(26), 10587–10592.
2. Myers-Scotton, C. (1993). Social Motivations for Codeswitching: Evidence from Africa. Oxford: Clarendon Press.
3. Slabbert, S., and Finlayson, R. (2000). 'I'm a cleva!': the linguistic make-up of identity in a South African urban environment. International Journal of the Sociology of Language 144, 119–135.