In his 25th of July 2024 Youtube podcast, Michael Millerman used quotes and ideas from an older book written by Winograd and Flores in order to support his arguments on “Heidegger and AI.”
In “Understanding Computers and Cognition“, Terry Winograd and Fernando Flores presented a theory of language as a basis for the understanding of cognition and of computers: language does not convey information. It evokes an understanding, or “listening,” which is an interaction between what was said and the pre-understanding present in the listeners` mind. The same words are perceived differently by the respective listeners according to their pre-understanding ability. This background grows out of challenges, habits and disruptions. People interpret language in a way that suits their understanding of the scope of their actions. The background of challenges and habits is not purely individual, but is generated within a shared social tradition. All humans are unique, yet they share this background, to varying degrees, with other people of the same culture.
Winograd and Flores claim that our knowledge is not represented in the brain at all, but consists of an unformalized shared background from which we articulate representations in order to cope with new situations. Computer programs, like AI, contain only pre-selected objects and properties, and there is no basis for moving beyond this initial formalization when a disruption occurs.
Winograd and Flores object to how computers are described in popular literature, how Al researchers talk about intelligence, and the kinds of programs Al researchers, particularly in the area of Natural Language, are trying to develop. They reject the commonly accepted belief that expert systems or any program (AI included?) could be intelligent, that representations can be used to model intelligent behaviour, and that developing autonomous agents is an effective use of computers – understanding the nature of human cognition and what computers can do is supposed to enable us to use them more effectively.
Winograd and Flores approach cognition and computers in terms of what it means to “understand language in the way people understand it.” They conclude that computers will never be able to understand natural language. This is because all programs, all representations and abstractions are based on pre-selected objects and properties. The background that motivates representations, the experience behind the designer’s analysis, has been cut out. Thus, when breakdowns occur (an inability to cope due to the fact that the action requests are different from what the computer expected), there will be no basis for moving beyond the initial formalization. Yes, the programmer can anticipate typical breakdowns and provide for representational changes, but these will themselves be limited and prone to break down. The only way out is to generate new programming from outside the representational realm. The key to this argument is that new representations spring from a shared, unformalized background. Coping with a breakdown involves articulating the basis of a representation. If you don’t have this background, you can’t speak with commitment, that is, with an implicit promise to clarify your meaning if questioned. Since you can’t negotiate meaning, you can’t engage in language. According to Winograd and Flores, the view that we codify and store experiences in representations that exist in the brain is naive. They claim that representation is a post-hoc interpretation of history. What we articulate has meaning within a context, and what we say has been shaped historically by that context. But it is only formalized (represented) when we speak. We are not translating what we have already formalized.
Is this book an insider’s attempt to articulate Al practice and to understand what an Al researcher is doing when he writes a program? The authors try to understand what it means to be human, how programs relate to life, what they choose while trying to identify our nature, and what they leave out. While the stated objective of the book is “how to design computer tools suited to human use and purposes,” the authors are most interested in understanding what it means to be human.
“In trying to understand a tradition, the first thing we must become aware of is how it is concealed in its obviousness”
All behavior proceeds from the subconscious: “To exist historically means that knowledge of oneself can never be complete”. Language is necessarily blind to its context because it involves a formalization based on the historical structure of interactions.” As carriers of a tradition, we cannot be objective observers of it. The power of language is to articulate recurrence, to identify patterns, to claim structure, to explain.
“Something gets recognised.”
Instead, there is a “pre-orientation.” “We are always already oriented to a certain direction of possibilities.”
All reasoning involves reinterpretation. Winograd and Flores validate Heidegger’s “Ready-to-hand” concept: the ontological difference between the Dasein and the perceptible world.
Intelligent reasoning is reinterpretation.
Unlike AI, humans are not following “plans.” There is extreme plasticity in their behavior. Every action is an interpretation of the current situation based on the entire history of their interactions.
An AI-like functional simulation of the cognitive system in terms of manipulated representations cannot generate the range of reinterpretation an unformalized background allows. Winograd and Flores conclude that manipulating a representation formally is not understanding.