Is Artificial Intelligence Made in Humanity’s Image? Lessons for an AI Military Education
Artificial intelligence is not like us. For all of AI’s diverse applications, human intelligence is not at risk of losing its most distinctive characteristics to its artificial creations.
Yet, when AI applications are brought to bear on matters of national security, they are often subjected to an anthropomorphizing tendency that inappropriately associates human intellectual abilities with AI-enabled machines. A rigorous AI military education should recognize that this anthropomorphizing is irrational and problematic, reflecting a poor understanding of both human and artificial intelligence. The most effective way to mitigate this anthropomorphic bias is through engagement with the study of human cognition — cognitive science.
This article explores the benefits of using cognitive science as part of an AI education in Western military organizations. Tasked with educating and training personnel on AI, military organizations should convey not only that anthropomorphic bias exists, but also that it can be overcome to allow better understanding and development of AI-enabled systems. This improved understanding would aid both the perceived trustworthiness of AI systems by human operators and the research and development of artificially intelligent military technology.
For military personnel, having a basic understanding of human intelligence allows them to properly frame and interpret the results of AI demonstrations, grasp the current natures of AI systems and their possible trajectories, and interact with AI systems in ways that are grounded in a deep appreciation for human and artificial capabilities.
Artificial Intelligence in Military Affairs
AI’s importance for military affairs is the subject of increasing focus by national security experts. Harbingers of “A New Revolution in Military Affairs” are out in force, detailing the myriad ways in which AI systems will change the conduct of wars and how militaries are structured. From “microservices” such as unmanned vehicles conducting reconnaissance patrols to swarms of lethal autonomous drones and even spying machines, AI is presented as a comprehensive, game-changing technology.
As the importance of AI for national security becomes increasingly apparent, so too does the need for rigorous education and training for the military personnel who will interact with this technology. Recent years have seen an uptick in commentary on this subject, including in War on the Rocks. Mick Ryan’s “Intellectual Preparation for War,” Joe Chapa’s “Trust and Tech,” and Connor McLemore and Charles Clark’s “The Devil You Know,” to name a few, each emphasize the importance of education and trust in AI in military organizations.
Because war and other military activities are fundamentally human endeavors, requiring the execution of any number of tasks on and off the battlefield, the uses of AI in military affairs will be expected to fill these roles at least as well as humans could. So long as AI applications are designed to fill characteristically human military roles — ranging from arguably simpler tasks like target recognition to more sophisticated tasks like determining the intentions of actors — the dominant standard used to evaluate their successes or failures will be the ways in which humans execute these tasks.
But this sets up a challenge for military education: how exactly should AIs be designed, evaluated, and perceived during operation if they are meant to replace, or even accompany, humans? Addressing this challenge means identifying anthropomorphic bias in AI.
Anthropomorphizing AI
Identifying the tendency to anthropomorphize AI in military affairs is not a novel observation. U.S. Navy Commander Edgar Jatho and Naval Postgraduate School researcher Joshua A. Kroll argue that AI is often “too fragile to fight.” Using the example of an automated target recognition system, they write that to describe such a system as engaging in “recognition” effectively “anthropomorphizes algorithmic systems that simply interpret and repeat known patterns.”
But the act of human recognition involves distinct cognitive steps occurring in coordination with one another, including visual processing and memory. A person can even choose to reason about the contents of an image in a way that has no direct relationship to the image itself yet makes sense for the purpose of target recognition. The result is a reliable judgment of what is seen even in novel scenarios.
An AI target recognition system, in contrast, depends heavily on its existing data or programming which may be inadequate for recognizing targets in novel scenarios. This system does not work to process images and recognize targets within them like humans. Anthropomorphizing this system means oversimplifying the complex act of recognition and overestimating the capabilities of AI target recognition systems.
By framing and defining AI as a counterpart to human intelligence — as a technology designed to do what humans have typically done themselves — concrete examples of AI are “measured by [their] ability to replicate human mental skills,” as De Spiegeleire, Maas, and Sweijs put it.
Commercial examples abound. AI applications like IBM’s Watson, Apple’s SIRI, and Microsoft’s Cortana each excel in natural language processing and voice responsiveness, capabilities which we measure against human language processing and communication.
Even in military modernization discourse, the Go-playing AI “AlphaGo” caught the attention of high-level People’s Liberation Army officials when it defeated professional Go player Lee Sedol in 2016. AlphaGo’s victories were viewed by some Chinese officials as “a turning point that demonstrated the potential of AI to engage in complex analyses and strategizing comparable to that required to wage war,” as Elsa Kania notes in a report on AI and Chinese military power.
But, like the attributes projected on to the AI target recognition system, some Chinese officials imposed an oversimplified version of wartime strategies and tactics (and the human cognition they arise from) on to AlphaGo’s performance. One strategist in fact noted that “Go and warfare are quite similar.”
Just as concerningly, the fact that AlphaGo was anthropomorphized by commentators in both China and America means that the tendency to oversimplify human cognition and overestimate AI is cross-cultural.
The ease with which human abilities are projected on to AI systems like AlphaGo is described succinctly by AI researcher Eliezer Yudkowsky: “Anthropomorphic bias can be classed as insidious: it takes place with no deliberate intent, without conscious realization, and in the face of apparent knowledge.” Without realizing it, individuals in and out of military affairs ascribe human-like significance to demonstrations of AI systems. Western militaries should take note.
For military personnel who are in training for the operation or development of AI-enabled military technology, recognizing this anthropomorphic bias and overcoming it is critical. This is best done through an engagement with cognitive science.
The Relevance of Cognitive Science
The anthropomorphizing of AI in military affairs does not mean that AI is always given high marks. It is now cliché for some commentators to contrast human “creativity” with the “fundamental brittleness” of machine learning approaches to AI, with an often frank recognition of the “narrowness of machine intelligence.” This cautious commentary on AI may lead one to think that the overestimation of AI in military affairs is not a pervasive problem. But so long as the dominant standard by which we measure AI is human abilities, merely acknowledging that humans are creative is not enough to mitigate unhealthy anthropomorphizing of AI.
Even commentary on AI-enabled military technology that acknowledges AI’s shortcomings fails to identify the need for an AI education to be grounded in cognitive science.
For example, Emma Salisbury writes in War on the Rocks that existing AI systems rely heavily on “brute force” processing power, yet fail to interpret data “and determine whether they are actually meaningful.” Such AI systems are prone to serious errors, particularly when they are moved outside their narrowly defined domain of operation.
Such shortcomings reveal, as Joe Chapa writes on AI education in the military, that an “important element in a person’s ability to trust technology is learning to recognize a fault or a failure.” So, human operators ought to be able to identify when AIs are working as intended, and when they are not, in the interest of trust.
Some high-profile voices in AI research echo these lines of thought and suggest that the cognitive science of human beings should be consulted to carve out a path for improvement in AI. Gary Marcus is one such voice, pointing out that just as humans can think, learn, and create because of their innate biological components, so too do AIs like AlphaGo excel in narrow domains because of their innate components, richly specific to tasks like playing Go.
Moving from “narrow” to “general” AI — the distinction between an AI capable of only target recognition and an AI capable of reasoning about targets within scenarios — requires a deep look into human cognition.
The results of AI demonstrations — like the performance of an AI-enabled target recognition system — are data. Just like the results of human demonstrations, these data must be interpreted. The core problem with anthropomorphizing AI is that even cautious commentary on AI-enabled military technology hides the need for a theory of intelligence. To interpret AI demonstrations, theories that borrow heavily from the best example of intelligence available — human intelligence — are needed.
The relevance of cognitive science for an AI military education goes well beyond revealing contrasts between AI systems and human cognition. Understanding the fundamental structure of the human mind provides a baseline account from which artificially intelligent military technology may be designed and evaluated. It possesses implications for the “narrow” and “general” distinction in AI, the limited utility of human-machine confrontations, and the developmental trajectories of existing AI systems.
The key for military personnel is being able to frame and interpret AI demonstrations in ways that can be trusted for both operation and research and development. Cognitive science provides the framework for doing just that.
Lessons for an AI Military Education
It is important that an AI military education not be pre-planned in such detail as to stifle innovative thought. Some lessons for such an education, however, are readily apparent using cognitive science.
First, we need to reconsider “narrow” and “general” AI. The distinction between narrow and general AI is a distraction — far from dispelling the unhealthy anthropomorphizing of AI within military affairs, it merely tempers expectations without engendering a deeper understanding of the technology.
The anthropomorphizing of AI stems from a poor understanding of the human mind. This poor understanding is often the implicit framework through which the person interprets AI. Part of this poor understanding is taking a reasonable line of thought — that the human mind should be studied by dividing it up into separate capabilities, like language processing — and transferring it to the study and use of AI.
The problem, however, is that these separate capabilities of the human mind do not represent the fullest understanding of human intelligence. Human cognition is more than these capabilities acting in isolation.
Much of AI development thus proceeds under the banner of engineering, as an endeavor not to re-create the human mind in artificial ways but to perform specialized tasks, like recognizing targets. A military strategist may point out that AI systems do not need to be human-like in the “general” sense, but rather that Western militaries need specialized systems which can be narrow yet reliable during operation.
This is a serious mistake for the long-term development of AI-enabled military technology. Not only is the “narrow” and “general” distinction a poor way of interpreting existing AI systems, but it clouds their trajectories as well. The “fragility” of existing AIs, especially deep-learning systems, may persist so long as a fuller understanding of human cognition is absent from their development. For this reason (among others), Gary Marcus points out that “deep learning is hitting a wall.”
An AI military education would not avoid this distinction but incorporate a cognitive science perspective on it that allows personnel in training to re-think inaccurate assumptions about AI.
Human-Machine Confrontations Are Poor Indicators of Intelligence
Second, pitting AIs against exceptional humans in domains like Chess and Go are considered indicators of AI’s progress in commercial domains. The U.S. Defense Advanced Research Projects Agency participated in this trend by pitting Heron Systems’ F-16 AI against a skilled Air Force F-16 pilot in simulated dogfighting trials. The goals were to demonstrate AI’s ability to learn fighter maneuvers while earning the respect of a human pilot.
These confrontations do reveal something: some AIs really do excel in certain, narrow domains. But anthropomorphizing’s insidious influence lurks just beneath the surface: there are sharp limits to the utility of human-machine confrontations if the goals are to gauge the progress of AIs or gain insight into the nature of wartime tactics and strategies.
The idea of training an AI to confront a veteran-level human in a clear-cut scenario is like training humans to communicate like bees by learning the “waggle dance.” It can be done, and some humans may dance like bees quite well with practice, but what is the actual utility of this training? It does not tell humans anything about the mental life of bees, nor does it gain insight into the nature of communication. At best, any lessons learned from the experience will be tangential to the actual dance and advanced better through other means.
The lesson here is not that human-machine confrontations are worthless. However, whereas private firms may benefit from commercializing AI by pitting AlphaGo against Lee Sedol or Deep Blue against Garry Kasparov, the benefits for militaries may be less substantial. Cognitive science keeps the individual grounded in an appreciation for the limited utility without losing sight of its benefits.
Human-Machine Teaming Is an Imperfect Solution
Human-machine teaming may be considered one solution to the problems of anthropomorphizing AI. To be clear, it is worth pursuing as a means of offloading some human responsibility to AIs.
But the problem of trust, perceived and actual, surfaces once again. Machines designed to take on responsibilities previously underpinned by the human intellect will need to overcome hurdles already discussed to become reliable and trustworthy for human operators — understanding the “human element” still matters.
Be Ambitious but Stay Humble
Understanding AI is not a straightforward matter. Perhaps it should not come as a surprise that a technology with the name “artificial intelligence” conjures up comparisons to its natural counterpart. For military affairs, where the stakes in effectively implementing AI are far higher than for commercial applications, ambition grounded in an appreciation for human cognition is critical for AI education and training. Part of “a baseline literacy in AI” within militaries needs to include some level of engagement with cognitive science.
Even granting that existing AI approaches are not intended to be like human cognition, both anthropomorphizing and the misunderstandings about human intelligence it carries are prevalent enough across diverse audiences to merit explicit attention for an AI military education. Certain lessons from cognitive science are poised to be the tools with which this is done.
Vincent J. Carchidi is a Master of Political Science from Villanova University specializing in the intersection of technology and international affairs, with an interdisciplinary background in cognitive science. Some of his work has been published in AI & Society and the Human Rights Review.
