Graduate School of Informatics /Multimedia Informatics

In partial fulfillment of the requirements for the degree of Master of Science Ali Kaan Sungur will defend his thesis.

Title: HIERARCHICAL TEMPORAL MEMORY BASED AUTONOMOUS AGENT FOR PARTIALLY OBSERVABLE VIDEO GAME ENVIRONMENTS

Date: 15th August2017

Time: 11:00 AM

Place: A-212

Thesis Abstract : Believable non-player characters (NPC) can have a profound impact on the experience that a video game provides. This thesis presents an online, unsupervised and lifelong learning autonomous agent that the player can interact with. It has an architecture that is based on a combination of Hierarchical Temporal Memory and Temporal Difference Learning Lambda with the guidance of neurobiological research. The agent has a visual sensor with online data stream. Input from the sensor is fed to the architecture in order to model the surrounding environment. The goal of the agent is to learn rewarding sequences of behavior based on the stimulation it receives caused by its own actions. It navigates in a procedurally generated three dimensional environment and is in a continuous learning state adapting the synapses of its neural connectum. In addition, the architecture is capable of being stored and loaded at any point allowing for persistent learning through multiple simulation sessions. The study presents the characteristics of the architecture on a video game related learning task. Data collected from the experiments with varying parameters are compared along with the runtime and serialization performance. The proposed methodology results in an autonomous NPC that can learn rewarding behaviors without any supervision. Moreover, it is also capable of learning specific action sequences via player guidance. End result is a promising and novel NPC architecture that is also fairly open to incremental improvements through the relevant neurobiological studies and the advancements on the theory of Hierarchical Temporal Memory.