The ReJAM project aims to develop Robots engaging elderly in Joint Activities with Music.

The objective of ReJAM is the promotion of physical, cognitive, emotional and social wellbeing through various music-related activities, e.g. physical exercises, games, reminiscence, and making music together. The activities are designed especially for group activities: ReJAM is well-suited for use in meeting centers or together with visitors at home.

More information about ReJAM can be found on the project website:


People prefer to live independently in their own homes for as long as possible. However, demographic changes, rising costs and new family structures are increasing the demands for new care solutions. Self-care and informal carers will play a key role in these new solutions. In order to support the ability of informal carers it becomes essential to reduce the impact of physical distance, increase informal carer presence in the daily life of elderly and to support cooperation between formal and informal carers to foster complementarity. To address these challenges, new care solutions should go beyond state of the art and enhance presence, awareness and complementarity.

The SALIG++ project offers novel solutions based on ICT-support for self-care by elderly and the bidirectional awareness and interaction between elderly and informal carers in collaboration with formal care in order to promote and prolong the well-being of elderly in living at home. SALIG++ makes it possible for carers to, for example, visit the home of the elderly from a distance and experience it as if they were actually there. The primary benefit is that carers become fully informed about the status of the elderly, her medical status as well as her home and devices (such as stove and faucets).

Military Human Enhancement

The Military Human Enhancement project was funded by NWO as part of the Responsible Innovation research  program. The proposed research (a) formulated design principles for computer assisted combat systems that are compliant with relevant legal and moral doctrines (b) investigated whether the principles of Just War Theory and ensuing rules of engagement can be reconciled with advanced forms of user enhancement by means of IT and (c) established whether the moral doctrines of war should be reformulated given the ubiquity of cognitive enhancement by computer systems.

The project specifically looked at a future generation of Automated, Intelligent Combat and Decision Support Systems for Command and Control developed by The Netherlands Navy, the Netherlands Defence Academy and the CAMS Force Vision team of the Dutch Ministry of Defence.

Personalised Educational Games

In 2009, Utrecht University and TNO joined forces in a research project aimed at the automation of personalized scenario-based training. I’m still thankful that I got to work on this topic under the supervision of Dr. Karel van den Bosch, Prof. dr. John-Jules Meyer, and Prof. dr. Mark Neerincx.

Scenario-based training engages learners in the reenactment of short storylines ecompassing real-life situations, i.e. situations that learners may actually experience in their future profession. Even though scenario-based training is useful for any type of profession, it is particularly feasible for the training of complex high-risk professions that heavily rely on decision-making and situation awareness.  Examples of such professions are the military, police, emergency health care, crisis management, and the fire department.

Virtual worlds are increasingly used as a medium for scenario-based training so as to create more training opportunities for learners. To take away the need for additional people to control, for instance, non-player characters (e.g. opponents or team players), recent efforts have focused on the inclusion of Artificial Intelligence (AI) to automatically control such parts of the virtual world.

This project aimed to add yet another layer of AI on top of the virtual world inhabited by AI-controlled characters: a director system. The director system monitors the learner while playing the game. Based on the performance of the learner, the director system selects and manipulates the content presented in the virtual world and/or the behavior of the characters in the storyline. The aim of these manipulations is to maximize the effectiveness of the training, i.e. to provide the learner with suitable exercises so as to maximize the learner’s training progress. The director system offers the learner an adaptive and personalized training curriculum. The personalized training curriculum is constructed with the use of solid didactic principles.  In addition, the design of the director system leaves room for the instructor to take over control when and where he/she sees fit.

Scenario authoring tool

Pauline Hovers worked on an authoring tool for instructors to exert control over the automated scenario generation process for an agent-based adaptive educational game. The tool allows instructors to determine the setting of the scenario, specify the learning goal addressed in the scenario, add characters to the scenario, and define the attitudes and goals for those characters.



As part of my research on the design and development of Personalized Educational Games, I developed INDIGO: the `Integrated Didactics for Games’ Ontology.

INDIGO serves as a common vocabulary for (automated) scenario-based training (SBT). It specifies the core concepts of SBT and their relationships, and is applicable across training domains and applications.

INDIGO contains information about the training domain, events taking place in the simulated environment, the behavior of the participating characters, and teaching strategies for effective learning. In addition, INDIGO provides a theoretically sound, generic, and consistent knowledge base for an automated reasoner. A frame-based approach to ontology engineering was used to model the relevant knowledge. The ontology was implemented in Protege Frames 3.5 and is available for download here.

Multi-Agent Director System

Ruben de Jong implemented a multi-agent system (MAS) that controls the adaptivity of a personalized educational game. The MAS was implemented in 2APL (for more information about 2APL, please contact Mehdi Dastani). It was designed using the OperA model (for more information about OperA, please contact Virginia Dignum).

The organization consists of:
(1) the director, coordinating the adaptation in the system,
(2) the planner, planning a complete storyline for a given learning goal,
(3) the monitor, tracing the behaviour of the learner within the game,
(4) the learner model, keeping track of the learner’s personal characteristics,
(5) the role player, controlling non-player characters in the game environment.

The figure below shows the text-based user interface for the learner. It enables the learner to interact with the agents and to perform actions by clicking on the respective buttons. In the current scenario, Sylvia – the victim NPC – has fallen over the chair and broken her arm. The trainee can choose to respond to these events by clicking on one of the available actions in the menu on the left.

Agent-based Learner Model

Christian van Rooij investigated how BDI agents can be used to model users. As a case study, he developed an agent-based learner model which was implemented in 2APL (for more information about 2APL, contact Mehdi Dastani). The learner model supports adaptivity in an agent-based adaptive educational game.

Learner modele-Partners that Care

TNO initiated a research programme called `ePartners that care’. An ePartner is an electronic, interactive partner that offers people tailored advice on how to set and achieve personal goals in a wide range of possible areas, such as developing a healthy lifstyle, career planning, or adopting an eco-driving style.

Within this project I investigated the conceptual/functional architecture of the user model, i.e. how to gather information from the user, how to respond to gathered information with meaningful advice, and how to assess the effects of the advice provided.

More information about this project can be found in these reports.

Automated scenario generation

Gwen Ferdinandus investigated the automatic generation of First Aid training scenarios for a personalized educational game. For this purpose she implemented an HTN planner to operate on an ontology (implemented in Protégé).

The ontology contains (among other things) a library of so-called semantic (or `smart’) objects and `decomposition methods’. The planner decomposes abstract tasks into actions and fulfills the prerequisites of those actions by selecting semantically annotated objects that meet those prerequisites.

For instance, decomposing the task `treat victim’ into `cool burn’ and `dress burn’ has the prerequisite of a `victim with a burn’ being present in the scenario. This prerequisite is fulfilled by adding a `victim with a burn’ to the scenario. In the figure added below, part of the code is visible along with the ontology editor (Protégé) and a list of objects that are available to the planner for use in a scenario.


More about this project can be found in this conference publication and in Gwen’s thesis. The source code is available for download in the downloads section.

My first experiment

Between May and August 2010, I conducted my first experiment. I had trainees interact with actors in training scenarios. The actors behaved according to scripts. I was able to influence the behaviors of the actors, like a true director, through in-ear portophones. I’m happy to have been able to conduct a true experiment as an addition to my literature research.

A trainee is moving a victim in a First Aid scenario
The director (me) is giving instructions to the actors from an adjacent room.

Confluence Constructor

Confluence Constructor is a software program I wrote in the Java programming language for my Master’s thesis in Cognitive Artificial Intelligence. The program constructs the common reduct for any two diverging sequences of steps in any orthogonal or WCR & SN term rewriting system and modular combinations of these. The program as well as the manual can be downloaded here.