Google DeepMind Pre-Training Lead: How To Land a Job at a Frontier Lab | Vlad Feinberg

### TL;DR Google DeepMind Pre-training Lead Vlad Feinberg details the key skills needed to land a job at a top AI lab, emphasizing infrastructure engineering, understanding scaling laws, and research intuition (i.e., "research as an MDP"). He notes that all labs have huge demand for diverse skill sets. ## The Shape of the Work and Required Skills At top labs, large language models tightly couple research and product, requiring a mix of skills. Vlad Feinberg highlights several specific directions, with **kernel development and low-level engineering** being a core need — all labs and projects are desperate for this kind of talent. Concretely, when a research project involves changing neural network architectures, improving KV caches, etc., you must be able to implement those new techniques efficiently. The entire tech stack cycle is about creating software artifacts that run at scale with high throughput and low latency — closely tied to classic engineering backend thinking — and is a very open specialization area. ## The Spectrum of Research and Application ### Internal Team Division Within GDM (Google DeepMind) there are different focus areas. For example, one team focuses on how to use the Gemini large language model to improve search results. This may look like an applied LLM version, but it actually requires a lot of hard research — ensuring factual accuracy, citing sources, evaluating source quality (avoiding references to sarcasm or jokes). Even in the so-called "applied AI" vertical, research is still happening. ### Continuum from Pure Research to Product Research There are also very classic LLM research teams (e.g., pre-training, post-training) that independently create state-of-the-art models. But at GDM, the importance of "pure research" depends on implementation; teams must both deliver models, ensure stable training (acting as "guardians"), and also own the recipe for building LLMs. These two roles cannot be separated. So along a spectrum from research to application, everyone needs to be flexible. ## The Spectrum of Software Engineering and AI Researcher ### Infrastructure Investment Drives New Techniques Vlad emphasizes that many new techniques are built on infrastructure investment. Take his team's work on **distillation** as an example: to transfer knowledge from a teacher model to a student model (involving statistical information over trillions of tokens), millions of dollars of floating-point operations are required. This forces teams to optimize the system, because every second and every byte matters. Distillation infrastructure has evolved through three or four generations, each time rethinking system design and broadening capabilities — for instance, spending four months rewriting distillation infrastructure led to new understandings of distillation scaling laws, which ultimately translated into powerful models (e.g., Flash 3.0). These investments start with classic engineering design documents: thinking about the right abstractions, designing storage systems, supporting reads and writes across data centers — these are all classic distributed systems problems. ### New Cross-Cutting Skills If you drop a typical backend engineer into a research team, adjusting the model architecture is harder than pure architecture work. There is a crossover point: **research taste** — a high-level intuition for how to navigate a DAG (directed acyclic graph) of milestones in a project. #### Research as an MDP Professor Jacob Steinhardt's article "Research as an MDP" (Markov Decision Process) is a great framework. In a research project, transitions between nodes are stochastic: some ideas may succeed or not, and some nodes may be unknown beforehand (a hidden MDP). This differs from software engineering, where the DAG is deterministic — you can list all paths and find the shortest one. In research, you must consider success probabilities, time invested, and prior estimates of different ratios. This intuition of estimating the likelihood of a method's success before trying it is "research taste" — a skill that needs deliberate cultivation. ## Shortcomings of Backend Engineers in Research Teams If a backend engineer goes directly into a research team, the first problem they face is **lack of background context in the research area**. Research work requires a humble perspective: you must understand the sum of human knowledge at the frontier on that topic before you can push it forward. Therefore, you need the ability to efficiently browse the history reference tree, quickly assess high-value papers without reading each one fully. Additionally, you need background knowledge in machine learning and computer science (including basic math and coursework) to really understand existing methodologies. Without deep understanding of what's already been done, it's hard to improve upon it. ## Scaling Laws: A Core Pre-Training Concept Vlad's team studies distillation, and the key to deeply understanding LLM distillation is **scaling laws**. People often focus on power-law structure and exponents, but the important thing is not the functional form — it's: **for a given scaling LLM recipe, as you invest more and more FLOPs, you must be able to predict the final test loss**. Why do we need to predict generalization error? In classical machine learning (e.g., ImageNet), you can iterate on different ideas using training and validation sets. But in the language model world, each pre-training run involves far more FLOPs than ever before — this is like a **one-shot version** of the ImageNet problem: you never see the full training dataset. You have to practice on MNIST and CIFAR, then based on that come up with methods that work directly on ImageNet. If you just do it independently, many people have tried that. ## How to Get Started: Vlad's Interview Advice If you complete the scaling law exercise and send the video of you doing it to Vlad, he would be happy to interview you. --- Source: Google DeepMind Pre-Training Lead: How To Land a Job at a Frontier Lab | Vlad Feinberg (https://www.youtube.com/watch?v=cDyi91onoJ8)