Cognition Keyword

Katherine Hayles

James B. Duke Professor of Literature

Duke University


            Cognition and information are intimately entwined. Their relation is specified by the definition I proposed (Hayles 2017:22): “cognition is a process that interprets information in contexts that connect it with meaning.” This definition overcomes a major shortcoming of information as developed by the Shannon-Wiener theory; namely, its decontextualized and purely quantitative character. Not only does my definition explicitly include context, but it also makes clear that cognition includes interpretive acts that ground information, thereby connecting it with meaning. Broadening the scope beyond the anthropocentric bias toward “thinking,” this framework invites explorations of cognition beyond the brain in humans, beyond the human species in nonhumans, and beyond biological lifeforms in computational media. It raises provocative questions about what constitutes “meaning,” including how meaning-making practices may be understood outside the human realm and in media other than biological entities.

Human Cognition

An important step in going beyond “thought” is the realization that cognition in humans occurs in nonconscious as well as conscious neuronal processes, as indicated by the last two decades of research in neuroscience, cognitive science, and cognitive psychology (Damasio 2000; Edelman and Tononi 2000; Dehaene 2009; Dresp-Langley 2012; Kouider and Dehaene 2007). Although these nonconscious processes are inaccessible to consciousness, they nevertheless perform functions essential for consciousness to operate. These include creating a coherent body image, processing information faster than consciousness, and recognizing patterns too noisy and subtle for consciousness to discern (Lewicki, Hill and Cryzewska 1992). Unlike the Freudian unconscious (if one believes in it), which manifests through symptoms and dreams, nonconscious processes appear as subtle intuitions to which consciousness can attend if the context is appropriate. Otherwise, the results of these processes die out within half a minute if not reinforced by “top-down” signals (Dehaene 2014). This timline indicates that a primary contribution of nonconscious cognition is to keep consciousness that is operating relatively slowly and with limited processing power from being overwhelmed by the information flooding into sensory and neuronal systems every microsecond.

Recent research has made increasingly clear that the massive and complex feedback loops involved in conscious and nonconscious neuronal processes are thoroughly embodied, integrating information across different brain regions and connecting with sensing and information-processing performed by organs, glands, and tissues throughout the body (Dehaene 2016; Edelman and Tononi 2000). Cognitive activities, including abstract symbol manipulation, are grounded in simulations of bodily actions (Barsalou 2008; Varela, Thompson and Rosch 2017). Moreover, embodied and embedded cognitions also draw significantly on environmental resources to support and extend their reach (Clark 2008).

Animal Cognition

The study of animal cognition  is a rapidly expanding area of inquiry that includes advances in biosemiotics; research into core consciousness in nonhuman mammalian species such as primates, dogs, cats, pigs, horses, and some cephalopods such as octopi; along with the study of the cognitive capacities of birds, reptiles, fish, bees, and species that do not have central nervous systems, such as mollusks. Around 1960, a “cognitive turn” opened the field to inferences about mental states from observed behaviors, representing a sharp departure from the constraints of behaviorism. Among the capacities that have been studied are navigational abilities in migrating species, spatial recognition in mammals and bees, tool use in primate species and ravens, allocation of attentional resources in pigeons and other species, and the ability among many species to form categories.   Closely related to evolutionary psychology, animal cognition studies often invoke what an animal needs to know to survive to explain why a given species developed certain cognitive abilities rather than others.

That many nonhuman species possess consciousness, a historically contentious issue, is now accepted by many experts in the field. At a Cambridge conference in 2012 on “Consciousness in Human and Nonhuman Animals,” the attending scientists signed this declaration:

Convergent evidence indicates that non-human animals have the neuroanatomical, neurochemical, and neurophysiological substrates of conscious states along with the capacity to exhibit intentional behaviors . . .  Non-human animals, including all mammals and birds, and many other creatures including octopuses, also possess these neurological substrates. (“The Cambridge Declaration on Consciousness”)

The declaration shows an increasing awareness that species without neocortices may nevertheless demonstrate affective behaviors similar to humans:

Systems associated with affect are concentrated in subcortical regions where neural homologies abound. Young humans and non-human animals without neocortices retain these brain-mind functions. Furthermore, neural circuits supporting behavior/electrophysiological states of attentiveness, sleep and decision making appear to have arisen in evolution as early as the invertebrate radiation, being evident in insects and cephalopod mollusks (e.g., octopus).

The conclusions extend beyond mammals as well:

Birds appear to offer, in their behavior, neurophysiology, and neuroanatomy a striking case of parallel evolution of consciousness. Evidence of near human-like levels of consciousness has been most dramatically observed in African grey parrots. Mammalian and avian emotional networks and cognitive microcircuitries appear to be far more homologous than previously thought.

A notorious problem in the philosophy of thought, of course, has been the difficulty of proving that another creature, including another human, has awareness of mind comparable to oneself. It seems reasonable to conclude, however, that if similar neural architectures are present, so are the dynamics of neural processes identified with human consciousness. The declaration therefore does not speak of consciousness as such but rather the “substrates”—anatomical, chemical, physiological—associated with consciousness in humans. Unstated, yet fundamental to this research, is the assumption that these substrates work by creating, transmitting, receiving, storing, and processing information.  This is the common framework that allows homologies to emerge across species while nevertheless emphasizing that each species has its distinctive sensory, affective, and neuronal capacities. The link between information and cognition forms the bridge that supports inferences about the ability of nonhuman species to grasp the world, interact with it, and understand it in their own terms and contexts. “Context” in this sense includes both the internal capabilities and constraints of the organism’s biology as well as how interact with the organism’s environment.

With the concept of umwelt, proposed in the 1920s-30s, Jakob von Uexküll recognized the importance of organismic specificity (Uexküll 2010). According to Uexküll, each organism constructs its own view of the world—its umwelt—according to its sensing organs, neuronal processes, and the kinds of interactions in which it can engage. Opening the door to thinking about biological processes from an organism’s viewpoint, umwelt introduces a subjective component (in the sense of considering the organism as a subject), although, as Thomas Nagel famously pointed out (1974), there remains an ineradicable difference between knowledge of an organism from the outside and the experience of being that organism.

Subsequent commentators have observed that humans tend to be more sympathetic to lifeforms whose umwelten are similar to our own. This anthropomorphic bias can be seen in the Cambridge declaration, which assumes that humans represent the standard for consciousness against which other lifeforms are measured. This makes sense from a certain perspective, since we (presumably) know that humans are conscious and the evidence for other species is indirect. Nevertheless, the anthropocentric bias continues to permeate discussions of cognition.

Plant Cognition

Plant intelligence research is an emerging area whose controversial status is reflected in the field’s names. The more controversial is “plant neurobiology”; the more conservative, “plant sensing and behavior.” At issue is the extent to which plant behavior can be understood as adaptive, variable, and responsive to environmental contingencies rather than genetically programmed and inflexible. Plant research challenges notions of cognition based on human/animal neurologies and thus is especially important for understanding the full scope of what “cognition” may mean.

Anthony Trewavas, a plant biologist who has critically examined the question of plant cognition (2003, 2005), makes three central claims in his book-length study (2014): plant cells possess knowledge about themselves; they respond to environmental variations with changes in behavior; and their responses imply that they can assess their situation, anticipate environmental variations, and make decisions to optimize their survival—in brief, that they are at least minimally cognitive. As early as 1880, Charles Darwin had suggested that plant roots act similarly to brains in animals: “the tip of the [root] radicle thus endowed [with sensitivity] and having the power of directing the movements of the adjoining parts, acts like the brain of one of the lower animals” [Charles and Francis Darwin, 574]. In plant neurobiology, this sensitive plant component  is termed a “root-brain.”

What is the evidence for a root-brain? The mechanisms for plant information-processing capacities include the presence of the phytohormone auxin, which acts similarly to neurotransmitters in animal brains. Brenner et al. (2008) explore long-distance electrical signaling through action potentials that travel through vascular bundles along the plant axis, a mechanism that allows roots to communicate with the upper portion of the plant as well as with other of the plant’s roots. Grazón and Keijzer (2011) emphasize the importance of the root apices, the area where roots converge just under the soil:  root apices are “the forward command centers .  . .[It is] the distinctive decision-making capacities of the transition zone within the root-apex that led to the formulation of the root-brain concept.” They continue, “The transition zone is special. It is the one and only plant area where electrical activity is known to synchronize . . . and where—brain-like—decision-making takes place that controls phenotype changes by exchanging information vascularly all the way up from the roots themselves to the shoots and organs at the opposite end of the plant” (161).   Research into these plant capacities indicate how a plant can achieve cognition even though it lacks neurons or a brain.

To distinguish between simple tropisms and more complex behaviors, researchers have focused on instances in which plants integrate competing signals to optimize their fitness. For example, Li and Zhang (2008) examined instances in which plants, which are normally gravitropic (preferentially growing downward), were faced with abnormal saline conditions. The roots demonstrated phenotypic plasticity, exhibiting flexibility that went beyond a simple graviresponse, such as avoiding salty areas. Similar research has explored how plant parts preferentially expand and contract in response to changing environmental conditions, including chemical mechanisms that allow root tips to communicate with each other and with the above-ground parts. The stilt palm, for example, grows toward sunlight by extending its roots in that direction and allowing its shade-direction roots to diminish and die (Trewavas 2003). Other responses include root segregation (how the roots are distributed in space), which has been shown to respond to the amount of available soil, irrespective of nutrients and other variables (Koller, 2000).  In addition, roots can distinguish themselves from those of competing species, this includes the ability to recognize themselves even from their clones, indicating that the mechanisms go beyond genotype discriminations (Gruntman and Novoplansky, 2004).

Cognitive plant behaviors that have been empirically verified include associative learning, memory (for example, habituation (Laskow) , off-line anticipation (as when a plant moves during the night to position itself for optimal light when the sun rises), territoriality, and perceptions of predators that initiate defensive responses. Garzón emphasizes that these responses, particularly off-line competencies, “mark the borderline between reactive, noncognitive cases of covariation and the cognitive case of intentional systems” (211).

Since plants compose 80% of the world’s biomass, the cognitive capacities of plants suggest the possibility of a planetary cognitive ecology in which humans, animals, and plants all participate. The next step in creating this kind of framework is to connect the cognitive capacities of the biosphere with meaning-making practices in humans, nonhuman animals, and plants. As defined above, cognition implies interpretation, and interpretation implies the possibility of choice or selection. Moreover, these meaning-making practices happen within contexts specific to organisms. To elucidate these issues, we turn next to biosemiotics and its emphasis on the biosphere as interconnecting and interacting networks of signs.

Cognition and Meaning

Meaning-making requires the sharing of information within parts or systems of an organism and between organisms, and this in turn requires the creation, transmission, and processing of signs. According to C. S. Peirce, a sign “is something which stands  to somebody for something n some respect or capacity” (Peirce 2.228)., . Dividing signs into the three categories of indexical (related to the object by cause and effect), iconic (related by morphology or form), and symbolic (related by abstract representation), Peirce opened semiotics to a much broader scope beyond symbol-dominated modes of communication, such as language, mathematics, and logic. Jesper Hoffmeyer, a biochemist, used Peirce’s ideas to pioneer the study of biosemiotics. He argues that “life is based entirely on semiosis, on sign operations” (24), starting with DNA code necessary for biological reproduction. Other biosemiotic sign vehicles include chemicals, electrical signals, morphological formations (e.g., protein folding). as well as many kinds of indexical and iconic sign carriers.

Central to biosemiotics is Peirce’s triadic logic of sign operations, in which the sign vehicle (for example, the genetic code) is related to the object (the organism) through the actions of an interpretant (genetic processes that read, transcribe, copy, and enact the code into the phenotype or organism). The interpretant, a feature not present in Saussurean linguistics, ensures that sign operations take place within specific contexts and that their meaning-creation is relevant to these contexts. Building on umwelt theory, Hoffmeyer illustrates  the contextual qualities of sign relations using Uexküll’s example of different lifeforms encountering a flower in a meadow: a bee that sips its nectar, an ant that crawls up its stem, a cow that eats it.  Each, Uexküll writes, “imprints its meaning on the meaningless object, thereby turning it into a conveyor of meaning in each respective umwelt” (qtd. in Hoffmeyer, 54).  “As the two parts of a duet must be composed in harmony—note for note, point for point—so in nature the meaning-factors are related contrapuntally to the meaning-utilizers in its life” (qtd. in Hoffmeyer, 55).

Philosophy has long insisted that cognition requires “aboutness,” a sense of intentionality that implies purposeful, directed action. Hoffmeyer accepts this idea but argues that mental “aboutness” grows out of a pre-existing bodily “aboutness,” in particular behaviors necessary for an organism’s survival and reproduction (47). He suggests that this implies an “evolutionary intentionality, the anticipatory power inherent in all living things” (47), thus re-positioning the mental activities of human intentionality within a more capacious context of what “intention” means in an evolutionary context.  Moreover, he turns the emphasis away from survival of the phenotype to survival in semiotic terms: organisms “survive semiotically inasmuch as they bequeath DNA self-referential messages to the next generation” (48).  This emphasis on the inter-relationality of sign exchanges enables him to extrapolate beyond biological niches to niches occupied by specific kinds of sign relations, for example, the species-specific bird song that cooperates/competes with the many different kinds of birdsongs and other sign exchanges in a given territory.

Intentions, he continues, “presuppose temporality. If the present second were to last forever all our intentions would be to no purpose. Intentions are dependent upon being able to anticipate the future” (48). Each organism, even a unicellular one, has experiences that constitute its life history; this history conditions its responses and changes how it interacts with its environment. By reacting selectively, an organism carries the past into the present and anticipates the future, enacting a fold in time (an expression Bruno Latour uses in another context). This temporal fold exists on another level inasmuch as every organism carries within itself its self-representation, the DNA that enables the future to unfold out of the past. In this sense, all organisms not only exist in time but also manipulate temporalities, exhibiting “anticipatory power” (47).

Anticipation is also essential to learning. Hoffmeyer observes that “any process of selection presupposes an intention or ground rule that determines what will be selected” (57), suggesting that the umwelt concept provides such a theory.

The umwelt is the representation of the surrounding world within the creature.  The umwelt could also be said to be the creature’s way of opening up to the world around it, in that it allows selected aspects of that world to slip through in the form of signs . . . the specific character of its umwelt allows the creature to become part of the semiotic network found in that particular ecosystem. ()

As noted above, organisms exist not only within ecological niches but also within semiotic niches, the specific flows of information coded into signs by the organism’s senses and receptors, signs that are meaningful in that creature’s specific umwelt. Thus it is not only organisms that survive but also their “patterns of interpretation” (58).

The biosemiotic approach is essential, this observation  implies, because it reveals these patterns in ways that other biological fields focusing on phenotypic survival may not. “It seems more appropriate and more satisfactory to speak of living creatures as messages rather than as vehicles for survival,” Hoffmeyer suggests (46), although of course he recognizes that message and organism are inextricably bound together. Similarly to the competing/cooperating dynamics between different species that create an ecological niche, the flows of information and signs within and among creatures create a semiotic niche, and niches interact with one another to form a semiosphere, networks of signs and meanings, each specific to the umwelten of the participating creatures.

A speculation emerging from this line of thought is whether evolutionary dynamics can be said to have directionality. Hoffmeyer believes so, suggesting that evolution in general proceeds toward greater complexity, a thesis that Stuart Kauffman has developed in another context (notwithstanding that many species, for example sharks, have not significantly changed for millennia). Complementing this physiological directionality is a corresponding increase in semiotic complexity, a tendency toward greater “logical depth” of biosemiotic networks.  “Logical depth” is a term proposed by Charles Bennett (1988), who defines it as the number of steps in an inferential process or chain of cause and effect linking something with its probable source. Bennett developed the term as a measure of complexity in computer algorithms, but Hoffmeyer appropriates it for biosemiotics, suggesting that it corresponds to the “depth of meaning” that an individual or species is capable of communicating (62).  An evolutionary tendency toward greater semiotic complexity thus implies a tendency toward greater depth of meanings, each specific to the creature that converts information flows into signs meaningful to it.

The creation of meaning is also level-specific, operating not only with a species-specific context but also within the context at which an organismic unit creates and processes signs. In a multicellular organism, for example, the cell processes information according to receptors on its surface, for example proteins that lock together with specific molecules to perform operations such as self-other recognition essential to the operation of the organism’s immune system. An advantage of a biosemiotic approach, Hoffmeyer suggests, is that “it leads us away from the standard ‘chain of command models’ of the brain-controlling-body or DNA-controls-embryogeny type. The whole essence of the sign process is that the decentralized units at tissue or cell level can interpret their own environment and act accordingly” (94). Thus, while causal reasoning leads to a top-down model of control and command, a model that Evelyn Keller critiqued decades ago, “semiotics paves the way for self-organized chaos” (p. 94). “What emerges,” he writes, “when the authority for interpreting and making decisions is delegated to organs, tissues, and cells, is a hierarchical network of sign processes the accumulated output of which constitutes the coordinated actions of the organism.  No single body controls this autonomous chaos, the efficacy of which can only by explained by its actual history throughout all the various stages of discoveries and conquests made by other life-forms” (95).   Terrence Deacon points out that this modularity is essential to prevent what he calls a “complexity catastrophe,” “too many components, needing to interact in a highly constrained manner in a finite time, despite vast possible degrees of freedom—setting an upper limit on the complexity of self” (Deacon, 473).   Modularity also creates a finer-grained sense of contexts, extending not only down to organs and tissues but even to individual cells.

This distributed model of sign-processing and meaning-creation leads to the issue of where the process ends, since the cell contains components such as mitochondria that also process information. “All my instincts,” he writes, “tell me that the cell forms the boundary, the lowest level at which it is reasonable to talk about true sign processes. At the receptor level, on the other hand, we are dealing with proto-sign processes, which are covered quite perfectly by the term categorical perception” (78). While he  justifies this intuition (somewhat problematically, in my view) by arguing that triadic sign-processes require the presence of a self-description within the unit, there are other good reasons to select the cell as the minimal level at which meaning-creation occurs, since it is the smallest unit capable of living on its own.

Applied to organisms with consciousness, including humans, the biosemiotic approach leads to a “swarm” model, where hundreds or thousands of networks of sign-processors within the brain all interpret and make decisions, in a “self-organizing chaos of elements, cells, or pieces of tissue all working their way, more or less independently, to a plan of action that will work for the survival of the organism” (113). In this autonomous self-organized chaos, “consciousness acts . . . as a continuous means of taking stock . . . or an interpretation” within “a body which is at all times involved in one actual life, one true story. What I am trying to say,” he concludes, “is that even though consciousness is a neurological phenomenon its unity is a function of the body’s own historical oneness”  (120).  It is this historical trajectory that makes every cognizer distinct from every other, since no two trajectories will align exactly.  “During every second of a human life, the body is effecting an interpretation of its situation vis-à-vis the biosemiotically rooted narrative which the individual sees him- or herself as being involved in at that moment.  This interpretation is what we experience as consciousness” (120-121).  Consciousness in this view, then, is built on top of, and is a result of, all the myriad sign-processes and relations within the body and between the body and the environment.  The enormous mistake that has dogged centuries of commentary about consciousness is to envision it as a self-contained faculty independent of all the bodily processes that co-constitute it.  Citing Hoffmeyer, Wendy Wheeler observes that mental “aboutness” grows out of bodily “aboutness,” and bodily “aboutness” in turn grows out of all the sign processes enacted within organs, tissues, glands and cells (Wheeler, 159).

Considered as a whole, what do these multi-level networks of sign processes interpret? “The total sensory input in which the organism is immersed at any given moment,” both from within (endosemiosis), via communications between body components to the brain and back again, and from without (exosemiosis), via the sensory receptors through the peripheral nervous system to the central nervous system to the brain and back again.  Just as an ecological niche is constituted through interactions between an organism and its environment (which includes the contributions of all the other organisms occupying that niche}, so the sign exchanges within an organism are in constant interaction with sign exchanges within its semiotic niche (which contains the contributions of all the other organisms occupying that niche).  The distinctive advantage of focusing on sign-exchanges rather than, say, energy or matter exchanges, is that it provides an opening to understanding how meanings are created, exchanged and interpreted between all living organisms and not simply between humans.

This view of meaning-creation provides a compelling alternative to a traditional view of meaning as associated solely with consciousness, an anthropocentric orientation that inevitably tends to privilege humans above all other organisms. “How could it have come about,” Hoffmeyer asks, “that this self-consciousness could glorify itself to such an extent that it could eventually imagine that nothing else in this world had any real meaning” (146). Acknowledging that he is leaving this question to others, he summarizes, “What I wanted to demonstrate is simply that this idea and all of its destructive side effects are an illusion. We did not invent meaning.  This world has always meant something. It just did not know it” (146).  The territory toward which he gestures here is now being extensively explored in the environmental humanities, commentaries on the anthroposcene, and myriad other areas connecting the dots between massive environmental damage, neoliberal capitalism, and ideologies of human dominion over the earth.

Returning now to the initial definition of cognition, we can understand what the key terms “information,” “interpret,” and “meaning” imply in a biosemiotic perspective. Unlike the computationalist view of meaning and interpretation, which postulates that the brain operates through symbols in a way similar to the computer, the biosemiotic approach works through biological processes such as neurotransmitters, immune responses, and many  other chemical, electrical, and physiological dynamics that have already been extensively researched in humans, nonhuman animals, and plants, without needing to postulate representations  (symbols in the brain) that have not been shown to exist.  The key move is to connect information flows with sign-creations through an organism’s specific responses and actions within its environment. The result is a view of cognition that is embodied and embedded both within the dynamics of a creature’s umwelt and within the networks of signs that comprise a semiotic niche, which combine with other niches to form the semiosphere.

The next challenge this essay undertakes is to relate the biological realm of meaning-creation to artificially created networks of sign-creation generated by computational and programmable media. The path will not be through a computationalist view that leaves out embodiment (“the brain operates like a computer”), but rather through a critical and close examination of how computational media, with their profoundly different embodiments, nevertheless also engage in meaning-making practices.

Machine Cognition

A persistent heresy haunting biology has been the impression that organisms evolve toward some purpose: the dolphin’s sleek body for fast swimming, the giraffe’s long neck for eating tall vegetation, the lion’s fierce canines for tearing flesh. No, biologists keep saying, that’s an illusion created by the imperative to survive and reproduce, a second-order effect of the fitness criteria matching an organism to its environment. Computational media, by contrast, are designed precisely for human-assigned purposes. They have no imperative to survive and reproduce (except in special cases when programs give them that purpose); rather, they make choices, interpret information, and create meanings in the contexts defined by their assigned purposes and enacted through their specific embodiments. This situation suggests that machine cognition stands in relation to biological cognition like those equivocal figures that can be seen either as a young or old woman, depending on the observer’s perceptual orientation: is it purpose and design or survival and reproduction? If purpose, then it leans toward machine cognition; if survival, toward the biological.  This kind of perceptual flip has led to important developments in computational design, such as evolutionary and genetic algorithms, neural network programs, and artificial life, all of which use design parameters geared toward selection and reproduction.  For our purposes here, the equivocal relation between machine and biological cognition suggests that crucial aspects of biosemiotics may have inverse equivalents in machine cognition, especially the specificity of an organism’s umwelt, the multi-level hierarchy of sign interpretations, and the importance of context and embodiment for connecting those interpretations with meanings, now understood in the overall framework of purpose and design rather than survival and reproduction as the overarching imperative.

Machine cognition in general includes mechanical calculation as well as computational media.  For the purposes of this essay, however, “machine cognition” will be taken to refer to networked and programmable machines and systems, because they demonstrate the potential for increasingly complex choices and decisions, and also because, in the contemporary world, they are connected to a very large variety of sensors and actuators to perform myriad tasks, from data mining to satellite imaging to controlling pacemakers and insulin pumps.

To connect a computer’s selection procedures with the kinds of selections or choices made by biological organisms, we need a theory of meaning that emphasizes the connection of meaning with consequences. John Dewey developed such a theory in the context of his work as a pragmatist, aimed at overcoming (paraphrasing Mark Twain) “the luminous fog that passes for clarity” in linguistically oriented philosophical discourses. Many of the problems that philosophers ponder, Dewey believed, disappear when the focus shifts to the consequences of decisions. Dewey thought that language, and hence meaning-making, evolved in social contexts of use and relied on a set of expectations or anticipations based on past experiences. Although Dewey was specifically concerned with language, his usage-focused approach accords well with the kind of learning that biological systems undergo, for example when an immune system learns to recognize foreign bacteria and develops antibodies to attack them. “The very conception of cognitive meaning,” he wrote in Experience and Nature, “. . . is that things in their immediacy are subordinated to what they portend and give evidence of. An intellectual sign denotes that a thing is not taken immediately but is referred to something that may come in consequence of it. . . the character of intellectual meaning is instrumental” (128). In terms of computer programs, this focus is exemplified by the family of branching conditional commands, such as “if/else,” which in addition to providing for choice or selection also specify consequences depending on whether the test expression is evaluated as true or false. If true, then statements inside the body of IF are executed; if false, then those statements are skipped. The advantage of this approach for meaning-making is that it does not depend on consciousness and therefore can be extended to nonhuman lifeforms; by the same token, it can also be extended to computational cognition.

The biosemiotic framework discussed above provides a springboard to launch a a parallel exploration into machine cognition.  As we have seen, the key move in biosemiotics is to connect biological processes to meaning through the creation, interpretation, and transmission of signs. With computational cognition, signs are explicitly present. The issue, then, is how sign operations work in the contexts of embodied machines and real-world interactions to create interpretations and decisions meaningful first to the machine and then to human designers and users as well. These parallels are as follows.

1) The umwelt. Media archeology, as articulated by Wolfgang Ernst (2012), Jussi Parikka (2012), Ian Bogost (2012), and others, emphasizes the specificities of computational devices, the sensors through which they know their environments and the actuators that enable them to interact with it. Similarly to an organism developing an umwelt, a computational device senses its surroundings, which may be a set of data, sub-routines nested within functions, or other internal (endosemiotic) systems. Many computational systems also have sensors that reach out into the world (exosemiosis), including sensors for vision, motion, location, temperature, pressure, and many other parameters. Actuators may include such devices as stepper motors, traffic lights, controllers that open or close valves, and myriad other kinds of capabilities.

2) Multileveled interpretations and decisions. Similar to the multiple levels at which decisions are made in biological organisms, computers also have multiple levels of codes, starting from the electronics that sense whether an incoming signal counts as zero or 1, through logic gates, to machine, on up to high-level languages such as C++, Java, and Python developed because they are easy (or easier) for humans to understand. Every command, however, must ultimately be translated into machine code, for that is all the computer’s CPU (central processing unit) understands. The software that carries out these translations, conventionally called the compiler and/or interpreter, interprets  the high-level commands in its  contexts, which include equivalent statements for the lower-level code. Nested loops, functions and subroutines may make decision trees extremely complex, as for example when some commands wait on others to be evaluated before they can proceed. The degree to which flexibility is built into the program varies with the complexity and logical depth of the algorithms. Even at the level of logic gates, however, selection  may be emphasized through such architectures as programmable gate arrays, which enable the computer to configure the logic gates itself through evolutionary trial and error to determine the most efficient way to solve a problem. When a computer is networked with other programmable systems and interfaced with a wide variety of sensors and actuators, the possibilities for choice and interpretation increase still more, resulting in deeper and more complex cognitions.

3) Context and embodiment. Although it is common in computer science departments to regard computers as abstract schemas based on formal logic, computers, like biological organisms, must be instantiated in physical form to exist in the world, and the details of this embodiment matter, as the recent vulnerabilities discovered in Intel computer chips demonstrate. Each CPU and each computer architecture has distinctive characteristics that define the specific contexts in which programs are read, interpreted, executed, and understood. Additionally, the emerging areas of context-aware computation and tangible computing, in which everyday objects such as tabletops, vases, chairs, and walls are invested with sensors and computational capabilities, emphasize the physical interfaces through which humans can interact with computational devices through everyday activities rather than through space-restricted devices such as screen and cursors. . Paul Dourish (2004), among others, suggests that such capabilities connect human embodied and embedded cognition with homologous capacities in the computational realm.

4) Cognition as anticipation and learning. Many computer programs anticipate and learn; examples are the auto-complete features of text processing, or, on a more complex level, dictation programs, such as Dragon Dictate, that learn to recognize the distinctive pronunciations of a specific user and become more accurate as the user gives input and corrections. The development of deep learning algorithms has taken this aspect of computer cognition much further. Inspired by the neurological features of biological cognition, deep learning programs typically employ a cascade of multiple layers of nonlinear processing units, each layer of which uses the output from a previous layer for input. Multiple levels of representations are also used, moving from less to more abstract and arranged in a hierarchy of concepts, much as a cell in a multicellular organism communicates with larger systems and, in species with brains and central nervous systems, with brain neurons in complex systems of feedforward and feedback loops.

The potential of neural net architecture is exemplified in the development of AlphaGo by DeepMind (recently acquired by Google). Go is considered a more “intuitive” game than chess because the possible combinations of moves are exponentially greater. Employing neural net architecture, AlphaGo used cascading levels of input/output feedback, training on human-played games .  . It became progressively better until it beat the human Go champions, Lee Sedol in 2016 and Ke Jie in 2017. Now DeepMind has developed a new version that “learns from scratch,” AlphaGoZero, that uses no human input at all, starting only with the basic rules of the game. Combining neural net architecture with a powerful search algorithm, it plays against itself and learns strategies through trial and error. At three hours, AlphaGoZero was at the level of a beginning player, focusing on immediate advances rather than long-term strategies; at 19 hours, it had advanced to an intermediate level, able to evolve and pursue long-term goals; and at 70 hours, it was playing at a superhuman level, able to beat AlphaGo 100 games to 0, and arguably becoming the best Go player on the planet.

Cognitive Planetary Ecologies

The belief that evolution has directionality, that it is moving to create more complex organisms and consequently a deeper and more complex semiosphere, is frankly speculative and would have to be carefully evaluated in view of the huge diversity of lifeforms. Furthermore, special caution would be necessary to avoid anthropocentric bias and to take into account, for example, bacteria and insects as well as more complex animals. It is beyond question, however, that the depth and complexity of human-originated signs are expanding exponentially, primarily through the myriad technological devices that are generating, processing, storing, and interpreting signals of all types.  Since “semiosphere” was developed specifically to describe sign-exchanges between biological organisms, I am reluctant to appropriate it to describe this expanding complexity. To avoid confusion, I prefer “cognisphere” (Hayles 2006), which explicitly includes technological as well as biological cognitions.

From a bird’s-eye view, the expanding cognisphere looks like this: in its tendency toward greater complexity, evolution produced Homo sapiens, the first species to find ways to exteriorize cognition. Progress was understandably slow at first: imagine the effort it took to produce the first tools, technically defined as artifacts that create other artifacts. Their creation was tedious because, by definition, there were no other tools to aid in their construction, only natural objects such as rocks and plant materials. But, as the depth and complexity of the tool-world increased, each layer made it possible to accelerate the development of further layers. Sometime in the early twentieth century, a qualitative leap occurred with the development of computational media. Embodied and embedded in very different ways than human cognition, computational cognition nevertheless performed many of the same functions, including (and especially) the creation and interpretation of signs within contexts that connected them with meaning.

The result is nothing less than a planetary cognitive ecology that includes, for the first time, artificial cognizers more numerous than the human population (Cisco estimates that by 2018, devices connected to the internet will exceed 20 billion, compared to a human population of 7.2 billion). It remains to be seen, of course, whether this exponential expansion will continue, or whether countervailing forces, such as environmental degradation, nuclear war, or other human-caused disasters, will change the dynamics catastrophically. What is certain is that cognitive activities across the bio-techno-spectrum have never been as deep, complex, and pervasive as they are at present.