Introduction to the Symbol grounding problem

Motion planning in the past was treated as np hard problem because of a large state space. Instead of using faster hardware and improved algorithm, the recommended attempt in solving these problems is to use an abstraction mechanism realized in natural language. The original problem gets converted into written sentences and these sentences are used as facts for the planner. Scene recognition and other image to text translation systems are discussed under the umbrella term "symbol grounding problem" which is about labelling sensory data with natural language. The label space consists of a certain vocabulary list and a grammar which defines the word order of the sentences, while the generated sentences are used in a dialog which captures expert knowledge of a domain. There is no single algorithm or paradigm available how to convert images into text, but the input data is a dataset which contains of example annotations. A certain program has to replicate the dataset which means, it has to generate the same anchored speech acts.

The core element of the symbol grounding problem is the instruction following task which is a dialogue based language game. The speaker gives a sequence of commands which have to be interpreted by the hearer. The dialog allows to solve complex problems from robotics which can't be addressed with backtracking algorithms from the past. From a programming perspective, the vocabulary of an instruction following game has much in common with the opcodes in Assembly language used to program computers on the lowest level. The idea is that a high level input language gets converted into a low level machine language. This transformation is done by an interpreter which is a grounding mechanism. In addition to the previous two articles about the symbol grounding problem, this part III gives an overview about the origin and how it may help to build future robots.

This article is the second step in our research into the Symbol Grounding Problem (SGP). In a previous work, we defined the main condition that must be satisfied by any strategy in order to provide a valid solution to the SGP, namely the zero semantic commitment condition (Z condition). We then showed that all the main strategies proposed so far fail to satisfy the Z condition, although they provide several important lessons to be followed by any new proposal. Here, we develop a new solution of the SGP. It is called praxical in order to stress the key role played by the interactions between the agents and their environment. It is based on a new theory of meaning -Action-based Semantics (AbS) -and on a new kind of artificial agents, called twomachine artificial agents (AM²). Thanks to their architecture, AM 2 s implement AbS, and this allows them to ground their symbols semantically and to develop some fairly advanced semantic abilities, including the development of semantically grounded communication and the elaboration of representations, while still respecting the Z condition.

AI Magazine

n order for robots to engage in dialog with human teammates, they must have the ability to map between words in the language and aspects of the external world. A solution to this symbol grounding problem (Harnad, 1990) would enable a robot to interpret commands such as “Drive over to receiving and pick up the tire pallet.” In this article we describe several of our results that use probabilistic inference to address the symbol grounding problem. Our specific approach is to develop models that factor according to the linguistic structure of a command. We first describe an early result, a generative model that factors according to the sequential structure of language, and then discuss our new framework, generalized grounding graphs (G3). The G3 framework dynamically instantiates a probabilistic graphical model for a natural language input, enabling a mapping between words in language and concrete objects, places, paths and events in the external world. We report on corpus-based experime...

The integration of text alongside sensor perception offers a significant advantage through its abstraction mechanism. Textual descriptions require minimal memory space and greatly simplify decision planning. By implementing a symbol grounding data structure in software, visual dictionaries effectively map language to sensory perception. This structure typically consists of two layers: 1. The first layer presents visual information, such as a 2D map of the environment. 2. The second layer provides textual annotations for this map, including location names, directional terms, and object labels. This approach significantly enhances human-machine interaction. Users can input textual commands, which the robot can then locate and reference within the map, creating a more intuitive and efficient communication process. For simplification, it's useful to conceptualize symbol grounding as a multimodal data structure comprising both text and images. In this context, grounding can be understood as the process of translating between different layers of the data structure in both directions. This bidirectional translation enables the robot to connect abstract linguistic concepts with concrete sensory data, and vice versa, facilitating more sophisticated and context-aware interactions.

KI - Künstliche Intelligenz, 2013

This paper gives an overview of the research papers published in Symbol Grounding in the period from the beginning of the 21st century up 2012. The focus is in the use of symbol grounding for robotics and intelligent system. The review covers a number of subtopics, that include, physical symbol grounding, social symbol grounding, symbol grounding for vision systems, anchoring in robotic systems, and learning symbol grounding in software systems and robotics. This review is published in conjunction with a special issue on Symbol Grounding in the Künstliche Intelligenz Journal.

2016

Proceedings of the 2nd Conference on Artificial General Intelligence (2009), 2009

A great deal of contention can be found within the published literature on grounding and the symbol grounding problem, much of it motivated by appeals to intuition and unfalsifiable claims. We seek to define a formal framework of representation grounding that is independent of any particular opinion, but that promotes classification and comparison. To this end, we identify a set of fundamental concepts and then formalize a hierarchy of six representational system classes that correspond to different perspectives on the representational requirements for intelligence, describing a spectrum of systems built on representations that range from symbolic through iconic to distributed and unconstrained. This framework offers utility not only in enriching our understanding of symbol grounding and the literature, but also in exposing crucial assumptions to be explored by the research community.

Robotics in the past was described as algorithm centric problems. The idea was that a certain computer program is able to determine the optimal action. This decision is made, according to the computational paradigm, by the CPU which is running some sort of intelligent program. The result of this understanding was to locate the intelligence inside the Turing machine which gets executed on a physical computer. In contrast, the symbol grounding problem assumes that the bottleneck is located in man to machine communication which is realized with a protocol. The protocol gets implemented with semantic tagging which maps keywords to certain game states. The sum of all keywords are forming a cognitive map which can be extended to an activity language and possible voice commands. The shared vocabulary between man and machine has much in common with a text adventure. The following paper introduces the symbol grounding problem for the example of a kitchen robot and the pong videogame.

International Journal of Signs and Semiotic Systems, 2011

This article discusses the current progress and solutions to the symbol grounding problem and specifically identifies which aspects of the problem have been addressed and issues and scientific challenges that still require investigation. In particular, the paper suggests that of the various aspects of the symbol grounding problem, the transition from indexical representations to symbol-symbol relationships requires the most research. This analysis initiated a debate and solicited commentaries from experts in the field to gather consensus on progress and achievements and identify the challenges still open in the symbol grounding problem.