Project topics and materials
Get Complete Project Material File(s) Now! »
Environment Type
Articial intelligence applications, since their advent in the second half of the twentieth century, have diversied to tackle many areas of human intelligence. Research in this eld has led to optimal algorithms on a number of problems and super-human performance on others such as in the 90s, when the Deep Blue computer [Hsu (2002)] won against a chess world champion.
However, humans still excel in many quotidian tasks such as vision, physical interaction, spoken language, environmental and behavioural anticipation or adapting ourselves in the constantly changing conditions of the natural world in which we live. To this end, hard problems have often been idealised in computer simulations (virtual reality) where research could focus on the essentials
of the articial agents’ intelligence without the need to solve low-level problems like noisy perception, motor fatigue or failure to name a few. Once matured in virtual reality, such agents would be ready for embodiment into a robotic implementations where, only few prove to be feasible. From this point of view, we can categorise existing approaches through the prism of environment complexity, namely those that have been implemented in virtual environments (Subsection 2.1.1) and those that have a robotic embodiment (Subsection 2.1.2)
Virtual World
The challenge for an intelligent agent in a virtual world is to cope with potentially complex behaviour, but in an accessible sandbox context. The virtual world is a controlled environment where observing events and object properties is simplied so that agent development can focus on behaviour while neglecting problems that arise from interfacing with the world. An example of such simplicity is given by the trajectory of an object moving under the eects of gravity, whose exact coordinates can be directly sampled by the agent without requiring to capture, segment and analyse an image. The main characteristic that describes this environment type is the focus on behaviour, but this brings the drawback of possible poor scaling of developed methods towards the real environment due to noise, uncertainty and interface issues. Regarding computational approaches to mental simulation, most existing works have been evaluated in virtual environments of varying complexity. Discrete environments provide a simple but informative view of the behaviour ofan agent [Ustun and Smith (2008)], under controllable circumstances. As complexity rises, namely the transition from discrete to continuous space, the challenge for intelligent agents to perform tasks increases signicantly, but it also enables a wider range of behaviour. Only now does the use of mental simulation begin to nd its applications, and advantages over traditional methods, in agent decision-making. Literature provides mental simulation approaches to constrained 2-dimensional continuous space [Kennedy et al. (2008, 2009); Svensson et al. (2009); Buche et al. (2010)] which focus on developing models to cope with the increased complexity of the environment. Other works go even further, to continuous 3-dimensional space where trajectories are more dynamic as human users intervene [Buche and De Loor (2013)] and collisions [Kunze et al. (2011a)] or occlusions [Gray and Breazeal (2005)] take place.
A recent trend that relates to the paradigm of mental simulation is the application of Monte Carlo Tree Search (MCTS) to real time video games, as an enhancement to its history of success in turn based games. Succinctly, if given the capability to simulate the outcomes of its actions, an agent can rely on MCTS planning algorithms such as UCT [Kocsis and Szepesvari (2006)]to perform more eciently in real-time strategy (RTS) games [Balla and Fern (2009); Churchill and Buro (2013)] or similar scenarios. We note that MCTS consists in planning algorithms reliant on a simulator { i.e. a way to obtain the eects of performed actions { while mental simulation encompasses the mechanisms for constructing such a simulator, which could eventually be used together with heuristic planning techniques.
Real World
In the real world, agents require a physical embodiment (or interface) in order to interact with the environment. The challenge of performing mental simulation within a real setup is to anticipate the behaviour of real entities which are perceived through noisy and fault-prone sensory input. In addition to issues that exist in virtual worlds, reality poses further obstacles to object detection and recognition, and therefore can be viewed as a signicantly more com- plex version of a continuous 3-dimensional virtual world. Systems that aim to achieve functionality in the real world must also solve interface problems in order to exhibit their behaviour. Interface issues include acquiring adequate information from sensors and eectors, the possibility of externally caused damage to the system and environment noise. Several systems using mental simulation have been developed as duallycompatible with both virtual environments and robotic embodiments. This allowed the authors to evaluate the cognitive process of their approach [Gray and Breazeal (2005); Breazeal et al. (2009)] in virtual reality where the agent can perform more dexterous actions that its robotic counterpart. Computer simulations were also used by Kennedy et al. (2008, 2009) to evaluate their
approach to improving a robot’s performance within a team. Other researchers have directly approached reality with robots that use mental simulation to support their natural language skills [Roy et al. (2004)], reasoning [Cassimatis et al. (2004)] and resilience [Bongard et al. (2006)]. We note that these are dicult problems in robotics, and it is interesting that mental simulation is able in these cases to decrease complexity of the original tasks and allow robots to perform better in the real world.
Mental Simulation Targets
Depending on what the agent encounters in its environment, the use of mental simulation in existing research can be divided into two categories: inanimate objects and entities which exhibit some form of behaviour. In the following, we explore what existing approaches use mental simulation for, namely the environmental (Subsection 2.2.1) and behavioural (Subsection 2.2.2) aspects of their environment.
Environmental Aspects
One aspect of mental simulation is represented by anticipating how insentient systems evolve based on a model of the laws that govern their behaviours. Such systems can be composed objects that move according to the laws of physics or deterministic mechanisms such as, for example, a light switch that can be used to turn a light bulb on and o. Such systems are \simple » in the sense that the underlying rules are deterministic and exhibit little or no change over time, for example applying a force to an object will always trigger a mass-dependent acceleration on that object; this does not exclude the potential complexity of such system. Possessing a mental model of physical phenomena allows humans to anticipate the consequences of actions that are performed in the environment [Hegarty (2004)]. Having a representation of properties such as mass, gravity, elasticity and friction are necessary in successful predictions of mechanical outcomes. Humans tend to construct a mental image of a given scenario, as it would visually appear in reality, in order to reason in certain contexts [Bergen (2005)]. The ability of humans to analyse environmental information has been linked to their capability of focusing on relatively small sets of data, through the process of attention management [Gross et al. (2004)], due to not being able to process the entire depth of the observable world. Nevertheless, humans are procient at high precision tasks such as anticipating and counterbalancing weights using body movements [Dufosse et al. (1985)]. A wide range of approaches have been proposed to control an agent’s behaviour in complex physical environments, with arguably one of the most successful being those based on Reinforcement Learning (RL) [Sutton and Barto (1998); Kormushev et al. (2013)]. While particularly well suited for noisy, real world data, these approaches usually assume a direct function between the agent’s sensors and eectors, which can ultimately lead to limited scalability of their adaptiveness [Atkeson and Santamaria (1997)] in novel scenarios. In this sense, our focus turns to adaptability using internal model based approaches Pezzulo et al. (2013).
Table of contents :
Abstract
Contents
List of Figures
List of Tables
1 Introduction
1.1 Cognitive Approach Towards Computational Decision Making .
1.1.1 Mental Simulation: a Common Framework
1.1.2 Assessing the Applicability of Mental Simulation
1.2 Objectives
1.3 Manuscript Organization
2 Related Work
2.1 Environment Type
2.1.1 Virtual World
2.1.2 Real World
2.2 Mental Simulation Targets
2.2.1 Environmental Aspects
2.2.2 Behavioural Aspects
2.3 Cognitive Function
2.3.1 Prospection
2.3.2 Navigation
2.3.3 Theory of Mind
2.3.4 Counterfactual Reasoning
2.4 Computational Implementation
2.5 Discussion
3 Proposition: ORPHEUS
3.1 Generic Agent Architecture
3.1.1 Perception
3.1.2 Imaginary World September 4th, 2015 vii
3.1.3 Abstract World
3.2 Implementation
3.2.1 Software Architecture
3.2.2 Distributed Processing and Visualization
3.3 Discussion
4 Applications
4.1 Introduction
4.2 Angry Birds Agent
4.2.1 Perception
4.2.2 Abstract world
4.2.3 Imaginary World
4.2.4 Decision and Control
4.2.5 Preliminary Results
4.2.6 Competition Results
4.3 Orphy the Cat
4.3.1 Perception
4.3.2 Abstract world
4.3.3 Imaginary World
4.3.4 Decision and Control
4.3.5 Results
4.4 Nao Robot Goalkeeper
4.4.1 Perception
4.4.2 Abstract world
4.4.3 Imaginary World
4.4.4 Decision and Control
4.4.5 Results
4.5 Discussion
5 Conclusion
5.1 Discussion
5.1.1 Contributions of Orpheus
5.1.2 Limitations of Orpheus
5.2 Future Work
5.2.1 Nao Soccer Lab
5.2.2 Learning
5.2.3 Planning
5.2.4 Agent’s Goals
5.2.5 Emotions
5.3 Publications
Bibliography
