Rewriting Text for a Sign Language Avatar with Several Hundred Signs, What It Saves in Fingerspelling and What It Costs in Meaning

Abstract

Some sign language avatars work like a phrase book. They play a recorded sign for each word in their dictionary and fingerspell the other meaningful words, such as nouns, verbs and names, spelling them letter by letter with the hand alphabet, which takes longer than signing. This paper measures how much an International Sign avatar with several hundred signs fingerspells in English meeting transcripts and written sentences, and tests whether rewording the text into words that the avatar knows avoids the fingerspelling without changing the meaning. Lists of the most frequent English words stood in for larger dictionaries. Three kinds of rewording were compared, replacing each missing word by a similar word that has a sign, asking a small language model to simplify the text, and restricting that model to words with a sign. With the avatar’s own dictionary, most content words had to be fingerspelled, which took most of the signing time. Word replacement removed more than half of the fingerspelling almost instantly, at some cost in meaning, while simplification barely helped. The restricted model removed almost all fingerspelling and more than halved the signing time but lost much of the meaning, partly because it completed the first letters of a missing word into an unrelated word. With a dictionary of the 5,000 most frequent words, every method tested with it kept most of the meaning. Word replacement is the practical choice for small dictionaries, which should grow, starting with the most frequent words.