Click above for the video version of this post!
Chimpanzees, like these in Uganda, can learn to understand human language as well as a human 2-year-old⁹. Photo by USAID Africa Bureau
Captive chimpanzees understand English as well as a human 2 year old¹¹ and use signs from Human sign languages⁵. Dolphins jointly coordinate their actions to open containers¹⁴ and perform novel tricks⁹. A parrot can reliably report the number or color of an item¹⁰. And prairie dogs sound the alarm that a tall human wearing white is approaching fast¹²!
Do animals use language? And if so, can we use A.I. to talk with them?
Human Language
Humans use language to communicate. Animals also communicate both visually and verbally on a variety of topics, from where to find food, to a desire to mate, or warnings of danger.
But human language is more than mere communication. Human language productively combines a discrete inventory of mostly arbitrary units of sound, each with individual fixed meanings, to jointly convey novel information. Language can describe things immediately present, remote in time or space, or even hypothetical — and sophisticated rules govern the manner in which we combine these units. Words come in categories: nouns, verbs, prepositions. Some categories, such as verbs, demand that other words fill subservient semantic roles: the verb give requires several arguments: a giver, a thing given, and a recipient. You cannot say *Sarah gave John, without the hearer asking “what did Sarah give?”
But perhaps the most distinctive aspect of human language is its use of recursive structure to encode meaning. In “the girl that saw the man that drove the truck that blew a tire left already”, each “that” introduces a recursive clause that contributes to the meaning of the full sentence.
A.I. Talks covers humanity’s attempts to teach artificial systems to communicate using language. Understanding the subtle differences and surprising similarities between human languages and animal communication systems can help us zero in on what exactly an artificial language system needs to accomplish.
Animal Communication
At first glance, most animal communication might seem simplistic compared to human language, but actually, various animals tick the boxes for nearly all of the properties of language!
Many animals have a repertoire of discrete vocalizations that carry fixed meanings. For example,
A low-ranking rhesus monkey will make a “noisy scream” sound when confronted by a higher-ranking member of the social group, while the higher-ranking member will make “arched screams” — a separate distinct sound¹².
Many songbirds have a high-pitched pure-toned “seet” call that warns of an approaching predator and a separate harsher “mobbing” call that marshals nearby birds to swarm a predator¹².
Animals can communicate about events not immediately present. Bees dance to tell other members of their hive where to find food some distance away. Prairie dog alarm calls have been shown to encode the information that a human is approaching who shot a gun sometime in the past¹². And captive dolphins who’ve been taught separate commands to “perform a novel acrobatic trick” and to “perform a trick with another dolphin”, when asked to perform a novel trick together, somehow coordinate their future actions to do so⁹!
Rhesus monkeys, song birds, such as this yellow robin, honey bees, prairie dogs, and dolphins all exhibit some features of human language in their communication. Photo Credits: (Top Left) Rhesus Macaques by Timothy Gonsalves, (Top Center) Yellow Robin by JJ Harrison, (Top Right) Honey Bee by Maciej A. Czyzewski, (Bottom left) Prairie Dog by Joe Ravi, (Bottom Right) Dolphin by sheilapic76
The physical form of some signals that animals produce definitely influences the meaning of that signal. For example, the warning “seet” of songbirds may be high and pure-toned because it is more difficult to locate the source of high-pitched sounds. But many signals that animals use to communicate have an arbitrary meaning. Con Slobodchikoff has shown that prairie dogs encode significant amounts of information by modulating the frequencies of their short chirping alarm calls in specific, but arbitrary, ways¹².
Some animals, such as Slobodchikoff’s prairie dogs, seem capable of productively describing novel ideas. In one experiment Slobodchikoff used a clothesline pulley system to send lifesize plywood cutouts into the middle of a prairie dog colony. One cutout was of a coyote’s silhouette while another was an abstract oval shape of about the same size. Each silhouette elicited distinct calls from the Prairie Dogs. The coyote cutout received calls similar to those the prairie dogs used for real coyotes, but in response to the oval shape, the Prairie Dogs produced an entirely novel call¹². Similarly, when a young captive chimpanzee named Washoe did not know the sign for her potty chair, she productively combined a novel sequence of signs she did know: DIRTY and GOOD⁵.
Going even further into the realm of human language, Chimps and also Dolphins have been shown to understand at least a limited form of semantic argument structure. A captive chimpanzee named Ally understood prepositional phrases like “Toothbrush on blanket” which require two arguments: an item and a location for that item. In formal tests of his understanding, Ally would place the correct item on the correct target up to 60% of the time, despite a hyperactive chimp personality that often led him to lose interest before completing tasks 😬.
In another set of experiments, researchers also taught prepositional phrases to two captive dolphins, Ake and Phoenix, using two very different command structures. Ake learned a gesture based language, while Phoenix was given verbal commands; meanwhile, just as different human languages use different word orders, Phoenix was given commands in the form “objectA on objectB” while Ake’s instructions came as “objectA objectB on”. Even with the differences in the form of their commands, Ake and Phoenix each correctly placed objects on other objects better than 60% of the time⁶— much better than the 4% correct that random guessing would achieve!
Of all the proposed properties of human language, the only one that has not been conclusively documented in animal communication systems is recursive structure. And even here, there is ongoing debate on whether some song birds combine calls with different meanings using recursive syntax¹ ¹³.
So, with their many parallels to human languages, do animal communication systems qualify as full fledged language?
Animal Language?
Humans and animals use fashion to signify many things — without resorting to language. (Left) The author and his daughter in typical plumage. Among other things, humans’ dress marks age, gender, mating status, and social class. Photo by author. (Right) A Peacock’s brilliant crown displays his sexual fitness. Photo by Wilfredor
Both humans and animals direct traffic using communication systems that are not language. (Left) Photo by author. (Right) Photo by Dean Beeler
Some scientists, including Slobodchikoff, call animal communication “animal language” exactly because, just as humans use language to communicate, animals’ verbal and gestural signals are the way in which animals communicate. If human language evolved from the communication systems of our distant ancestors then shouldn’t animal communication also qualify as language? If we systematically separate human language from animal communication, we risk placing a barrier between us lofty humans and the natural world we’re a part of. After all, animals’ communication systems serve them in their environment just as human language serves us in ours. And it seems terribly unfair to banish animals from the language club just because they might not communicate in the same way we do.
But both animals and humans use communication systems that are clearly not language. Road signs communicate information using non-arbitrary symbols that indicate aspects of the immediate environment but that very rarely can be combined in any intelligible way. Animals also mark territory and paths without resorting to language. Humans and animals both use physical appearance to signal age, gender, and social status. But our fashion statements do not follow the grammatical rules of a natural language.
Can We Talk to Animals?
Still, while there is currently no overwhelming evidence of any animal communication system that exhibits all the features of human language, this does not guarantee that we might not yet discover one that does! Which is why I’m so excited about current projects that use techniques from A.I. and machine learning to analyze and decipher the vocalizations of some of the most intelligent and largest-brained non-human species on the planet — whales.
Project CETI³, led by Michael Bronstein from the Imperial College London, is analyzing the bursts of clicks sperm whales use to communicate over long distances, while another group led by Denise Herzing, Research Director of the Wild Dolphin Project, and Thad Stamer of Georgia Tech is analyzing the sounds produced by dolphins⁷. The techniques that both of these groups use come directly from modern Natural Language Processing. Specifically, they use deep neural networks to build language models of animal vocalizations.
A language model is a computational system that, given the first few items of a sequence, predicts what will come next. Over the past few years, big-budget tech companies have trained enormous neural networks to predict the next word of an English (or other natural language) sentence. While 10 years ago, the best language models were unable to generate even a single coherent sentence; now, deep language models, like GPT-3⁴, can generate entire paragraphs that largely hang together.
There are many challenges that must be overcome to apply deep language models to whale vocalizations: How to find the whale’s equivalent of a word? How to scale up the number of recorded whale vocalizations? Current successful deep learning algorithms were trained on orders of magnitude more data than the number of sperm whale recordings we currently have.
But perhaps the largest challenge will be that even if we are able to build a language model over whale vocalizations, the model might produce output that, while sounding natural and correct to a sperm whale, to us humans is just another unintelligible series of clicks!
A recent paper by Emily Bender and Alexander Koller² argues that modern language models like GPT-3 that are trained solely on text are inherently unable to connect the words they produce to objects and entities in the real world. These models do not “understand” either the input or the output text. They simply produce sequences of the clicks that humans are able to interpret!
To get around this problem, it is vital to connect audio recordings with contextual information about the environment. In his work with prairie dogs, for example, Con Slobodchikoff carefully recorded which predator was approaching at the time of each prairie dog’s alarm call.
I personally hope these challenges can be overcome. That we discover whales really do communicate in full-fledged language. That whales, who already clearly manipulate and exploit their environment, communicate meaning by combining arbitrary symbols in complex — even recursive — ways. I hope we discover not only the proper response to a whale’s vocalization but an interpretation of that vocalization in language that humans understand.
I, for one, look forward to the day we have a meaningful conversation with whales — When A.I. Talks!
Bibliography
¹Johan J. Bolhuis, Gabriel J. L. Beckers, Marinus A. C. Huybregts, Robert C. Berwick, Martin B. H. Everaert. 2018. Meaningful syntactic structure in songbird vocalizations? PLoS Biol 16(6): e2005157. https://doi.org/10.1371/journal.pbio.2005157
²Emily M. Bender and Alexander Koller. 2020. Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. https://aclanthology.org/2020.acl-main.463.pdf
³Michael Bronstein. 2020. Project CETI Next Steps: Industrial-Scale Whale Bioacoustic Data Collection and Analysis. Workshop on Decoding Communication in Nonhuman Species. https://www.youtube.com/watch?v=6B8Fg2kZrxA&ab_channel=SimonsInstitute
⁴Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, Dario Amodei. 2020. Language Models are Few-Shot Learners. NeurIPS 2020. https://papers.nips.cc/paper/2020/file/ 1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf
⁵Roger Fouts. 1997. Next of Kin: My Conversations with Chimpanzees. HarperCollins, New York.
⁶Louis M. Herman. 1986. Cognition and language competencies of bottlenosed dolphins. In Dolphin cognition and behavior: A comparative approach, 221–252.
⁷Denise Herzing. 2017. Cracking the Dolphin Communication Code. Talks at Google. https://www.youtube.com/watch?v=Mfb6zoB_yII&ab_channel =TalksatGoogle
⁸Edward Kako. 1999. Elements of syntax in the systems of three language-trained animals. Animal Learning & Behavior, 27 (1), 1–14.
⁹Nova ScienceNow. 2011. How Smart Are Dolphins? Season 5, Episode 4. https://www.pbs.org/video/nova-sciencenow-how-smart-are-dolphins/
¹⁰Irene M. Pepperberg. 2002. In search of king Solomon’s ring: cognitive and communicative studies of Grey parrots (Psittacus erithacus). Brain, Behavior and Evolution, 59(1–2), 54–67.
¹¹E. Sue Savange-Rumbaugh, Jeannine Murphy, Rose A. Sevcik, Karen E. Brakke, Shelly L. Williams, and Duane M. Rumbaugh. 1993. Language comprehension in ape and child, Monographs of the Society for Research in Child Development, 58 (Nos. 3–4).
¹²Con Slobodchikoff. 2012. Chasing Doctor Dolittle: Learning the Language of Animals. St. Martin’s Press, New York.
¹³Toshitaka N. Suzuki, David Wheatcroft, Michael Griesser. 2018. Call combinations in birds and the evolution of compositional syntax. PLoS Biol 16(8): e2006532. https://doi.org/10.1371/journal.pbio.2006532
¹⁴Yamamoto, Nobuyuki Kashiwagi, Mika Otsuka, Mai Sakai, and Masaki Tomonaga. 2019. Cooperation in bottlenose dolphins: bidirectional coordination in a ropepulling task. PeerJ 7:e7826 http://doi.org/10.7717/peerj.7826