About the Role

As a Research Engineer for Foundational Models, you will be responsible for investigating, developing and experimenting privacy preserving techniques on Transformer Models.

We are looking for a mid-to-senior level Research Engineer who can build and modify transformer-based architectures from scratch (Initially LLMs). This is a hands-on role for someone excited about digging into the inner workings of LLMs and experimenting with novel neural network layers.

You are passionate about the mathematical elegance that goes into building neural networks and are able to explain

You’ll also have opportunities to occasionally work on-site with our research partners at SUPSI and IDSIA in Switzerland to accelerate our joint innovations.

What you’ll do

• Research and develop novel privacy-preserving techniques in collaboration with our extended research teams
• Implement and integrate privacy-preserving machine learning methods to existing various architectures (LLaMA, Mistral, Gemma, Qwen, etc) for LLM inference.
• Benchmark performance of privacy-preserving techniques on multiple different datasets and different tasks.
• Cross-team collaboration: Work closely with fellow research engineers and scientists on designing experiments and interpreting results.
• Research and innovate: Stay up-to-date with the latest research in deep learning and LLMs and Privacy Preservation

Required Qualifications

• Deep Learning Expertise: Proven hands-on experience in designing and implementing deep neural network models (especially transformer architectures) from scratch. You understand the internals of modern sequence models (attention, transformer blocks, etc.) and can modify or create new components as needed.
• Mathematical Foundations: Strong foundation in mathematics relevant to machine learning (linear algebra, calculus, probability). Ability to understand and implement complex algorithms from research papers.
• Communication: Excellent written and verbal communication skills. Ability to document your work clearly, explain complex technical concepts to team members, and communicate with external collaborators.
• Programming & Frameworks: Excellent programming skills in Python and deep learning frameworks (PyTorch)
• Cryptograph : Some knowledge of cryptography or security principles as they apply to data privacy will be important in practice (e.g., basic understanding of encryption, secure protocols).