Ga11y: An Automated GIF Annotation System for Visually Impaired Users


Animated GIF images have become prevalent in internet culture, often used to express richer and more nuanced meanings than static images. But animated GIFs often lack adequate alternative text descriptions, and it is challenging to generate such descriptions automatically, resulting in inaccessible GIFs for blind or low-vision (BLV) users. To improve the accessibility of animated GIFs for BLV users, we provide a system called Ga11y (pronounced “galley”), for creating GIF annotations. Ga11y combines the power of machine intelligence and crowdsourcing and has three components: an Android client for submitting annotation requests, a backend server and database, and a web interface where volunteers can respond to annotation requests. We evaluated three human annotation interfaces and employ the one that yielded the best annotation quality. We also conducted a multi-stage evaluation with 12 BLV participants from the United States and China, receiving positive feedback.

In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems
Mingyuan Zhong
Mingyuan Zhong
PhD Student
Computer Science & Engineering

Currectly, I conduct research in accessibility, user interface, interaction techniques, and the intersection of these areas.