Description

Recent innovations in sequence modeling have introduced Mamba, a state-space model (SSM)-based architecture that achieves state-of-the-art performance on various long-range tasks while maintaining linear-time complexity.

This thesis investigates how causal reasoning and intervention simulation can be integrated into the Mamba architecture. The key idea is to introduce a structured latent space into Mamba's architecture that enables counterfactual simulation and causal inference. The focus lies in simulating interventions in latent space, which reflects the hidden dynamics of temporal systems.

Tasks

• Review the state of the art in Mamba, state-space models, and causal latent variable models.

• Analyze causal modeling techniques suitable for integration into sequence architectures.

• Design and implement a Mamba-based model with a latent space suitable for causal interventions.

• Develop your own ideas to improve existing approaches and investigate them.

Requirements

• Good understanding of the fundamentals of deep learning.

• Ideally: Experience with Python and one or more deep learning libraries (PyTorch, TensorFlow).

• Willingness to familiarize yourself with new subject areas.

Supervision

Shahenda Youssef

If you are interested, please send your CV to shahenda.youssef@iosb.fraunhofer.de