The Artificial Intelligence Research Institute (IIIA) offers up to 5 scholarships for the introduction to a research career, in the context of CSIC's JAE Intro Programme.

Training plans and mentors:

1. Multi-objective and multi-agent reinforcement learning for ethical and safe environments (JAEIntroICU-2021-IIIA-07)
   Mentor: Dr. Juan Antonio Rodríguez-Aguilar (jar@iiia.csic.es)
   
   The purpose of this project is to investigate how to exploit multi-objective reinforcement learning to help agents learn ethical and safe behaviours in a variety of state-of-the-art environments. On the one hand, we will investigate how to extend a state-of-the-art algorithm to create environments where the learning of safe behaviours is guaranteed. We will evaluate the performance of the resulting algorithm on state-of-the art environments (Doors and Sokoban) and will compare it with recent multi-objective reinforcement learning algorithms proposed in the literature for the learning of safe policies. On the other hand, we will extend another state-of-the-art algorithm to create multi-agent environments where the learning of ethical behaviours is guaranteed. We will empirically evaluate the resulting algorithm on two state-of-the-art multi-agent environments that have been proposed to investigate the learning of ethical policies (Deepmind’s Gathering Game and the Public Civility Game). As an important aspect of this empirical evaluation, we will investigate whether in such environments it is enough for the agents to employ low-cost learning algorithms (e.g. Q-learning), or whether instead it is more convenient to resort to Deep Reinforcement Learning.
    
2. Faster cost effective disaster management through active learning (JAEIntroICU-2021-IIIA-08.)
   Mentor: Dr. Jesús Cerquides (cerquide@iiia.csic.es)
   
   In the CROWD4SDG EU project at IIIA-CSIC we are working together with UN and University of Geneva (among other partners) in providing AI tools that can help fulfill the Sustainable Development Goals (SDGs). One of the tools which development is planned makes use of information obtained from social media (specifically, images captured from Twitter) to help assist disaster relief when a disaster, such as a flooding, an earthquake or a volcano eruption occurs. In particular we are interested in the determination of the relevance of images and an automatic assignment of the level of damage shown in the images, whether directly affected humans are shown and so on. This is currently done by means of human annotators. We are interested in minimizing the amount of information requested to human annotators and that decreases the time to annotation, by designing a pipeline that uses active learning, thus only requesting to the human experts those images that the machine learning algorithm is unable to annotate carefully. The student will be responsible for the evaluation of active learning strategies in this scenario, under the direction of the tutors and in close collaboration with PhD student Hafiz Firmansyah from University of Geneva.
    
3. Implementing a negotiation bot for the game of Diplomacy (JAEIntroICU-2021-IIIA-09)
   Mentor: Dr. Dave de Jonge (davedejonge@iiia.csic.es)
   
   Diplomacy is a strategic board game for seven players, which is also commonly used as a testbed for artificial intelligence. Unlike traditional 2-player games like chess and go, it involves coalition formation, negotiation, and deception between the players, which makes it a much more interesting, but also more challenging game. Now that modern chess and go programs are already superior to humans, Diplomacy is often considered the next frontier in artificial intelligence, and large companies such as Google's DeepMind are currently working on it.
   The goal of this project is to implement a negotiation algorithm (in Java) that will allow an existing Diplomacy-playing bot to negotiate and strike deals with other players. For example, it could discover that England should propose to France to jointly attack Germany. Dr. Dave de Jonge already has a basic idea of how such a negotiation algorithm could work, so the main goal for the student would be to implement this idea, perform experiments, and determine how well it works. And perhaps, based on the results of those experiments, the student could even figure out ways to improve the idea.
    
4. Inference solvers for maximum and minimum satisfiability (JAEIntroICU-2021-IIIA-10)
   Mentor: Dr. Felip Manyà (felip@iiia.csic.es)
   
   This project aims to define logical calculi, prove their completeness and develop efficient solvers for solving the maximum and minimum satisfiability problems in propositional logic, also known as MaxSAT and MinSAT problems. In contrast to the solvers defined so far for this problem, which combine inference and search, the solvers to be developed in this project will be exclusively inference solvers. Moreover, the project aims to identify NP-hard optimization problems that can be solved efficiently by first reducing them to MaxSAT/MinSAT and then deriving a solution to the original problem with a MaxSAT/MinSAT solver.
   Knowledge of Python programming, computational logic and SAT/MaxSAT/MinSAT solvers will be valued.
    
5. Agent-based simulations and immersive technologies for fire department training (JAEIntroICU-2021-IIIA-11)
   Mentor: Dr. Jordi Sabater-Mir (jsabater@iiia.csic.es)
   
   The use of simulations in virtual environments for the training of emergency forces has proven to be very useful to improve these units' operational level when faced with a real problem. However, the use of these generic tools has its limitations. One of them is that they do not allow multilevel simulations in which low-ranking units located in the focus of the emergency, middle managers, and high-level managers can participate simultaneously in a single scenario.
   A scenario of this complexity requires a set of federated simulations that communicate with each other, providing a different level of granularity according to the participant's tasks and using different interfaces to interact with him/her (CAVE, Virtual Reality, Augmented Reality, standard screens…). During the grant, the candidate will work on different tasks associated to the project. For example how to adapt the information that flows from one simulation to another simulation that is at a different level of abstraction (adding/removing detail) or developing internal agent models for different actors in the simulations (firefighters, fire, smoke, vehicles, etc.).
   Courses in Artificial Intelligence, advanced knowledge of Python programming, and knowledge of the Unity and/or Unreal Engine video game development environments (specially in the development of VR applications) will be valued.
    
6. Integrating machine learning and optimization for problems in graphs and social networks (JAEIntroICU-2021-IIIA-12)
   Mentor: Dr. Christian Blum (christian.blum@iiia.csic.es)
   
   This project will deal with a recent line of research in optimization, which explores the potential of integrating machine learning (ML) techniques (in particular, deep learning) with optimization algorithms. In this project, we are especially interested in the use of deep learning techniques for the discovery of new methods for the generation of valid solutions to hard combinatorial optimization problems. In other words, our goal is to make use of deep learning in order to learn how to generate good solutions for a given optimization problem, as opposed to learning how to generate good solutions for a specific instance of that optimization problem. This is potentially interesting in the context of optimization problems that arise in graphs and social networks because the greedy functions available for such problems are sometimes misleading. Given that this line of research is very recent, the research that we contemplate in this JAE INTRO project is a step toward territories that are still largely unexplored.