Intern - Research - High-Dimensional Inputs For Rl Agents in Dynamic Video Games Environments - LA Forge

Job Description

Research Internship (F/M/NB) – High-Dimensional Inputs for RL agents in Dynamic Video Games Environments - La Forge

About Ubisoft

Ubisoft's 19,000 team members, working across more than 30 countries around the world, are bound by a common mission to enrich players' lives with original and memorable gaming experiences. Their commitment and talent have brought to life many acclaimed franchises such as Assassin's Creed, Far Cry, Watch Dogs, Just Dance, Rainbow Six, and many more to come. Ubisoft is an equal opportunity employer that believes diverse backgrounds and perspectives are key to creating worlds where both players and teams can thrive and express themselves. If you are excited about solving game-changing challenges, cutting edge technologies and pushing the boundaries of entertainment, we invite you to join our journey and help us create the unknown.

Ubisoft Bordeaux

Founded in 2017, Ubisoft Bordeaux works with passion on the biggest AAA titles to deliver the best gaming experiences. Today, the studio is composed of 400 talents from 20 different nationalities, working on licenses such as Assassin's Creed, Beyond Good & Evil 2 and a free to play game, BattleCore Arena. At the same time, the studio has set up a Tech branch which works on all Ubisoft's online services (named Online Services) as well as on the Anvil game engine. Ubisoft Bordeaux is also home to a R&D team, La Forge, which brings together engineers and researchers to work together on prototypes for game production, particularly around AI topics.

Job Description

In video games, navigating and interacting within dynamic environments presents a unique challenge for NPCs and bots. Current methods for navigation, such as using absolute or relative positions, often fail to generalize effectively across new environments. This internship will explore innovative approaches to designing machine learning (ML)-based agents that can operate in complex, dynamic, and adversarial settings by leveraging high-dimensional inputs.

Recent advances in AI, such as DeepMind's SIMA agent, show that it is possible for an agent to learn from pixel inputs across multiple environments. However, we aim to push these ideas further by taking advantage of the structured data available in video games. Specifically, this internship will investigate whether object-centric observations, semantic depth maps and potentially neural environment representations can be used to develop a more efficient, generalist agent capable of navigating dynamic environments (e.g., following a car in motion regardless of texture, color and surroundings).

This internship will help pave the way for the development of foundation models-generalist agents that can be deployed across various games and settings, moving beyond simplistic small-scale navigation, aiming to create more efficient and adaptable NPCs and bots.

Relevant literature:
• https://general-navigation-models.github.io/
• DeepMind's SIMA Agent: https://deepmind.google/discover/blog/sima-generalist-ai-agent-for-3d-virtual-environments/
• Learning Approximate Raycasts: https://dl.acm.org/doi/10.1145/3651289
Objectives:
• Explore how high-dimensional observations (e.g., semantic depth maps, asset lists) can improve bot/NPC performance in dynamic environments.
• Investigate the design of a generalist agent capable of learning from simpler, more structured inputs (such as depth maps) while efficiently handling complex navigation tasks (e.g. attention layers over large asset lists, CNNs for symbolic images)
• Develop a prototype to test neural environment representations that embed richer semantic information, enabling the agent to generalize across various environments.
• Integrate these techniques with Ubisoft's game development pipelines, aiming to optimize bot/NPC learning for real-time, large-scale environments.

Internship Structure:

The internship will be based at Ubisoft's La Forge research lab, where cutting-edge resources and large datasets are available for research. You will work closely with internal teams and researchers, leveraging multiplayer 3D game environments and state-of-the-art tools. Collaboration with other departments, including the production side, will help test findings in real-world applications.

Qualifications
• You are in the final year of your engineering degree or pursuing a research master's in computer science, AI, or a related field.
• You have strong skills in machine learning, computer vision, and reinforcement learning. Knowledge of neural representations is a plus.
• You are proficient in Python and familiar with machine learning libraries such as TensorFlow or PyTorch.
• Experience with 3D graphics, game engines, and neural radiance fields (NeRF) is a plus.
• You can communicate effectively in English in a collaborative, international team environment.
• Knowledge of game development and interest in video game production is a plus.

Skills and competencies show up in different forms and can be based on different experiences, that's why we strongly encourage you to apply even though you may not have all the requirements listed above.

Jobcode: Reference SBJ-r7xyq6-3-145-201-122-42 in your application.