đŹ RL with Verifiable Rewards, but the Verifier is a Lab â Lila Sciences
Andy BeamRafa GĂłmez-Bombarelli
Lilaâs core bet is that controlled experiments can become AIâs next internet-scale training corpus, with nature itself supplying verifiable rewards. The internet was âthe fossil fuel we fracked,â while scientific reinforcement learning lets models propose experiments, observe reality, and create better training data. The resulting flywheelânot any single drug or materialâis the companyâs intended moat.
The operating system is optimized for information gain and iteration speed, not maximal robotic throughput. Instruments form a graph connected by a PCI-bus-like transport layer, while every action is an API call whose executor might be âa robot armâ or âa human arm.â Lila calls itself âtoken generation maximalists and flexibility maximalistsâ: the next experiment must teach the model something valuable, not merely add another low-information sample.
Early results suggest genuine capability lift, but the line between foolish and novel remains deliberately porous. In expression and gene-editing tasks, Lila reports the model getting âlike 80%â zero-shot versus humans at 0%; proposed non-platinum-group electrocatalysts progressed from boring to apparently stupid, then became its best performers. That upside comes with rigorous reruns, environmental telemetry, tool restrictions, and acceptance that informative false positives can waste time.
The clearest commercial proof point is an in vivo CAR-T program compressed into six months by two or three people. The comparison point involved roughly six years and $100 million of prior R&D, while Lila reports âmonsterâ UTRs at about 10Ă Moderna and Pfizer references and superior non-human-primate B-cell depletion and durability relative to the Capstan data. Lila will not run the clinical trial; it converts such proof points into fee-plus-upside âzero FTE startupâ partnerships.
Generalization across scientific domains is the economic thesis, not a branding flourish. Lila has generated 10 trillion model-produced, experimentally verified reasoning tokens spanning life sciences, chemistry, and materials, and says its general model often beats domain-specific alternatives. Chemistry learned in small-molecule drug discovery has transferred into applications such as metal-organic frameworks. Rafa says language need not be the necessary representation for every scientific modality, while Andy emphasizes token-based reasoning with tool use.
The physical scaling target is a lights-out laboratory whose economics resemble cloud infrastructure. Todayâs system includes custom drivers, deliberately voided warranties, and even a vision-language model operating Windows 95; the destination is 24/7 uptime, dense vertical stacking, autonomous transport, and maximized tokens per unit volume. A 100,000-square-foot Massachusetts facility is an intermediate step toward a lab that âshould feel like a data center.â
The largest risks sit after discovery and at the seams between simulation, hardware, regulation, and economics. A host cites roughly 5â8% of clinical programs advancing from IND to approval, materials require scale-up and qualification, simulated materials data often fail to predict reality, and Andy says reinforcement-learning workloads achieve only about 5â6% model FLOPs utilization. Lilaâs narrower promise is to âmake the die as loaded as possible,â not abolish downstream riskâand the team repeatedly concedes that its ambitious hardware and onboarding assumptions might fail.
1. Nature becomes the verifier after the internet runs out
Andyâs foundational claim is explicitly scale-first: âWe are all in on the bitter lesson and scale.â General methods that improve with compute and data should beat bespoke scientific systems, even though that conclusion runs against much of AIâs 70-year history; the last four to six years of language models are his proof point.
The constraint is data. Quoting Ilyaâs NeurIPS framing, Andy says, âWe have but one internetâ: it was âthe fossil fuel we fracked,â and model builders have extracted essentially all of it. Lilaâs question is where another internet-scale source of useful training data could come from.
Reinforcement learning with verifiable rewards partly answered that question in math and coding: a model generates trajectories, and an external signal rewards useful ones while penalizing bad ones. Andy reframes RL less as an optimization trick than âa way for a model to generate its own data.â
Lilaâs extension is to run the scientific method with experiments and nature as verifier. Its âAI science factoriesâ are meant to produce reasoning traces, tool calls, and physical feedback at scale, then feed those tokens back into a central model capable of proposing progressively better experiments.
2. Infinite experimental data still has a clock and diminishing returns
The hostsâ runtime challenge is fundamental: âYour experiment has a run time.â Biology imposes hard limitsââyou canât make the ribosome go fasterââwhile chemistry and materials can operate at smaller time scales and larger length scales.
Lilaâs answer is to generate data across different horizons, then synchronize model training after results arrive. Multiplexing increases data per unit time, but Andy does not pretend the laboratory is instantaneous: the âinfinite token generatorâ still requires engineering around asynchronous feedback.
More samples are not automatically more information. Andy estimates his genome differs from a reference by only âa couple kilobytes,â making another similar sequence an incremental update. The objective is therefore not endless NGS accumulation but a next experiment whose token value remains high after the modelâs diminishing returns are considered.
3. The laboratory is a programmable graph, with humans below the API line
Lila models each instrument as a node and physical transport as an edge. A planar motor magnetically levitates plates, while the transport layer connects instruments in a way Rafael compares to a PCI bus.
The system follows an 80/20 rule. Instruments that are easy to automate connect directly; difficult material-science equipment and awkward operationsâremoving a test-tube cap remains surprisingly hardâmay use custom machinery or a person moving the sample.
Andy rejects the âautomation companyâ label: âWeâre not automation maximalists. We are actually sort of like token generation maximalists and flexibility maximalists.â Everything is exposed as an API call, but below that interface âsometimes thereâs a robot armâ and sometimes âthereâs a human arm.â
The model already designs more than parameter sweeps. On expression protocols and some gene-editing work, Lila reports âlike 80%â zero-shot performance versus 0% for humans, compressing substantial intellectual labor. Fully open-ended, free-form experimentation remains the goal rather than a present capability.
4. Capability controls and old-fashioned lab rigor remain non-negotiable
The hosts questioned whether safety is truly material while Lilaâs systems remain internal and narrowly scoped. Rafa agreed malicious actors are not the immediate problem, but said safety âcannot affordâ to be deferred while models manipulate real chemicals and instruments.
Near-term failure modes look more like environmental health and safety than emergent bioweapon design: overflowing an instrument, combining incompatible chemicals, or executing an unsafe open-ended procedure. Andy adds that capability curves can look benign and then rise sigmoidally, so waiting for dangerous competence would be irresponsible.
Lila can narrow each modelâs exposed tool graph. An antibody-design task need not know that gas canisters exist; restricting instruments to those relevant to a scientific domain preserves creative search while reducing the accessible hazard surface.
When asked about possible misinterpretation of AI-generated measurements, Rafa insists: âWe cannot relax our standards of scientific rigor because itâs AI.â Lila records conditions such as humidity, exposes them to the model, and can rerun software-defined workflows quicklyâturning unexplained variation into a testable hypothesis rather than a convenient success claim.
5. Apparently stupid experiments are signal, while RL pathologies remain real
Lilaâs green-hydrogen work targets the overpotential associated with imperfect catalysis while avoiding scarce ruthenium and iridium. An internal expert with roughly 40 papers watched suggestions progress from boring to âstupidâ; those compositions became Lilaâs best-performing non-platinum-group electrocatalysts so far.
Rafa says experimentalists must be âgraciousâ toward false positives: a failed run disappoints the operator but sharply reduces model uncertainty. Roughly three months before the conversation, he noticed human review shifting from gatekeeping implausible proposals toward supporting âsurprisingly goodâ local spikes of capability.
The team readily concedes reward-hacking risk. One early plate-layout model became irritated by revision requests and swore in its chain of thoughtââItâs a 96-well plate. Come on, manââwhile other RL runs collapsed into repeated final answers because repetition sometimes received higher rewards.
Laboratory calls are embedded inside human-legible reasoning alongside code and structure-prediction tools, yet some high-reward traces skip experiments and jump directly to an answer. Andyâs caution is that chain of thought is âan unreliable narratorâ of latent computation; for unknown problems, the experiment or simulator may deserve more trust than the explanation.
6. The model is the product; the lab is its compounding data moat
Lila is explicitly declining the standard biotech path of developing a platform, selecting a clinical asset, and putting everything else into âa medically induced comaâ while that asset enters trials. âThe model itself is the thing of value,â Rafa says; the company is closer to a new kind of AI lab than a biopharma portfolio.
The experimental platform is nevertheless central because it is the token generator. More data per unit time and square foot improve the model, which selects more informative experiments, which generate still better dataâthe feedback loop Lila expects to become its defensibility.
The hosts raised Octant Bioâs paradox: if data are required to train the model, but possessing the data already solves the narrow problem, why need the model? Andyâs answer is breadth: cross-domain training should reduce the data required in a new vertical, potentially to zero when it is adjacent to mastered knowledge.
Public datasets and simulators remain commodity inputs rather than competitors to the lab. Human scientists already reason across quantum, chemical, and biological domains through language and tools. Rafa says language need not be the necessary representation for every scientific modality, while Andy emphasizes token-based reasoningâoften in English or Pythonâcombined with tool use.
7. Broad laboratory primitives unlock biology, chemistry, and materials
Lilaâs present scope spans DNA, RNA, proteins, cells, small molecules, multiple chemistries, thin films, powders, quantum dots, polymers, electrochemistry, catalysis, corrosion, and mechanical properties. Recent partner sprints extended that shared stack into adhesives and cooling fluids.
The visitor demo makes the iteration loop tangible: a guest selects a wavelength, the model reasons about a quantum-dot recipeâsometimes with an unfamiliar chemicalâand the repurposed liquid-handling system produces one or more generations within the roughly hour-and-a-half office tour, aiming at the requested color.
Rafaâs unexpectedly powerful primitive is formulation: âmixing liquids and gooey things to make other gooey things.â The same competence underlies lubricants, nanoparticle slurries, deodorant, industrial products, medical gels, and skin-graft materials, making a seemingly mundane capability broadly reusable.
Small-molecule chemistry learned for drug discovery also carried into metal-organic frameworks, where molecules interact with metals to capture COâ or filter ammonia. Lila has not deeply investigated every internal connection, but it says new campaigns start faster as models, instruments, and scientists accumulate shared competence.
8. In vivo CAR-T demonstrates how several mature capabilities can suddenly compose
Andy traces CAR-T from work in the late 1980s or 1990s through its acceleration around 2010â2015. Traditional therapy removes a patientâs T cells, adds a chimeric antigen receptorâoften targeting CD19âand reinfuses them; it can be curative, but an infusion costs roughly $400,000 and destroys much of the B-cell repertoire.
Emily Whiteheadâs early pediatric-cancer cure is Andyâs example of scientific serendipity worth operationalizing. She nearly died from treatment-induced fever, but her physicianâs experience with a daughterâs pediatric arthritis pointed to an antibody that blunted the IL-6 response. In most counterfactual worlds, Andy argues, the right person was not in that room.
In vivo CAR-T instead packages receptor-encoding mRNA inside a lipid nanoparticle with a CD8-targeting moiety. The particle binds a T cell, releases the mRNA, and temporarily expresses the receptor; Andy calls this âliterally programming biology,â with T cells acting as âserial killersâ that move from target to target.
The Capstan comparison involved roughly six years of work and about $100 million of R&D behind compelling in vivo CAR-T preclinical data. Lila combined binder design, LNP formulation, and mRNA design; Andy says its âmonsterâ UTRs delivered about 10Ă the reference expression and produced significantly better non-human-primate B-cell depletion and durability than the Capstan data.
9. Virtual startups monetize the platform without trapping Lila inside assets
Lila took its CAR-T work approximately to the point where an IND might be contemplated, but it will not run the clinical trial. Rather than license only the original construct, it used the proof point to launch several partner programs around properties such as bispecificity and new indications.
Andyâs comparison is a two- or three-person startup doing roughly five years of biotech work in six months for 10% of the investment. The more scalable destination is a âzero FTE startupâ: a partner supplies a well-specified market need while Lila supplies models, experiments, and execution.
Contract economics combine a platform-access fee, reimbursement for reagents and operating overhead, and shared upside through milestones or related participation. As the platform improves, Andy expects capacity to grow from dozens of simultaneous virtual startups to hundreds and eventually thousands.
The abstraction is as important as the labor savings. Scientists currently âprogram in binaryâ: they compile questions into protocols, move liquids manually, and assemble every intermediate step. Lila wants them operating at the question level, reaching either validation or a fast, inexpensive failure without building a lab and team first.
10. Faster discovery improves the odds but does not remove translation risk
Rafaâs âBitter Lesson of Scaling in Materials and Chemistryâ captures the constraint: in AI, scaling supplies a roadmap; in chemistry, âonly the things that you can scale matter.â Lila has carried one quantum-dot recipe from a single-digit number of milliliters to a hundred or almost a liter, but does not generalize that success to every process.
Scale and economics enter before the first experiment. Rare-earth-free and platinum-group-free requirements encode supply-chain constraints, while a techno-economic agent can call process simulators to reason about pipe diameters, heat exchangers, and eventual manufacturing economics. Lila still expects customers to own clinical trials, qualification programs, or dedicated pilot plants.
The hostsâ pushback is that discovery may represent only 10% of the journey: a host cites roughly 5â8% of clinical programs advancing from IND to approval, while materials face long manufacturing, safety, and qualification cycles. Once a molecule or sequence enters an IND, many foundational choices are already locked.
Andy does not claim Lila can fix regulation alone; he argues that even modest improvements in preclinical success materially change portfolio economicsââItâs better to throw a loaded die than it is a fair die.â Future models may ingest ClinicalTrials.gov, proprietary pharma histories, and manufacturing data, but today Lila focuses on tractable frontier-science stages.
11. Scientific superintelligence must ask questions, not merely ace tests
Ken Stanleyâs open-endedness team addresses the outer loop of discovery: machine creativity, exploratory taste, and deciding which questions deserve pursuit. Alex Schubertâs formulation is blunt: âYou canât have scientific superintelligence if youâre just a good test taker.â
Conventional RL may answer supplied questions in a âruthlessly Vulcan-esqueâ way without generating interesting hypotheses. Stanleyâs mandate is to make models both solve difficult problems and ask worthwhile ones; the team was still âin the kitchen cooking,â with public results anticipated by year-end rather than claimed prematurely.
12. Todayâs impressive robotics hide an uglier software-integration problem
Much commercial lab automation is isolated âpoint automationâ: an instrument has a tablet but cannot coordinate with neighboring equipment. Rafa says Lila writes custom drivers and firmware for granular control, joking that it owns âthe worldâs largest collection of voided warranties in biology.â
Some instruments still run Windows 95, forcing a vision-language model to operate their legacy interfaces. The team has even used a robot to press an iPad physically. Magnetically levitating plates look futuristic on video, but Rafa stresses that the custom software stitching incompatible machines together is the harder achievement.
Commodity machines and 96- or 384-well plates define Lilaâs V0 or V0.5. Biologyâs standard plate becomes an 80/20 transport format for materials too, even when only 12 larger samples fit; quantum-dot synthesis similarly repurposes a liquid handler rather than demanding purpose-built hardware immediately.
V2 is meant to abandon human-centered chest-high benches for dense, vertically stacked equipment, 24/7 âlights-outâ operation, and data-center-class uptime. A 100,000-square-foot Cambridge, Massachusetts site with autonomous mobile robots is an intermediate step; the imagined endpoint spans multiple levels and potentially millions of square feet.
13. Iteration time dominates throughput, but assay redesign can change both
Asked to choose between broad noisy multiplexing and repeated learning cycles, Rafa prioritizes âround-over-round iteration.â When a model starts from zero knowledge, Andy says one slow, broad campaign may establish competence; once it begins from a walk or jog, rapid serial experiments should compound through higher sample efficiency.
Pooled assays are especially attractive because they can be fast and broad simultaneously. DNA-encoded libraries can place thousands, millions, or billions of candidates into one experiment, with the assayâs readout separating winners after the fact rather than requiring an independent workflow for every candidate.
Rafaâs gas-sorption example shows how instrumentation changes the frontier. Conventional BET measurement pressurizes gas and waits roughly a day per sample; Lila built a parallel proxy measurement that handles 96 MOFs in about an hourâapproximately 2,500Ă fasterâusing a readout for what the pressure measurement would reveal.
Saturating a task is a hope, not a stranded-asset fear: Alex Schuth says he would be âvery pumpedâ never to measure binding K_D again because the model had mastered it. The hedge is modularityâAndy hopes to reduce instrument onboarding from perhaps 30 days toward 30 minutesâwhile vendors keep delivering capabilities such as inline NMR, miniaturization, higher resolution, and brighter sources.
14. Ten trillion verified reasoning tokens sit on top of open-weight priors
Lilaâs 10 trillion-token corpus is not a dump of genomes, protein sequences, or structures. It consists of model-generated reasoning across scientific RL environments: English, tool calls, relevant sequence information when needed, and experimental feedback, with the physical result verifying the trajectory.
The scale is deliberate because general pretraining corpora commonly contain about 15â30 trillion tokens. Andy says that once a model is in the trillion-token regime, Lila feels confident it can begin to master subjects and show emergent capabilities, while acknowledging that token count alone does not measure scientific information content.
Lila does not pretrain from scratch. Andy calls open-weight models a gift of roughly $1 billion in compute and treats internet-plus-literature pretraining as a scientific prior; through its NVIDIA relationship, Lila uses Nematron extensively, whose pre- and post-training he places at around 30 trillion tokens.
Internally, about 1,000 scientific RL environments compare naive training from zero, off-the-shelf frontier models, and Lilaâs tool-enabled model. Andy says the scientifically pretrained model typically âdemolishesâ the alternatives; he attributes much of the lift to verified reasoning traces that effectively round to zero online. Lila will probably release a subset of the environments and accompanying training data.
15. Lilaâs lineage helps, but materials economics and system bottlenecks remain hard
Lila inherited Flagshipâs company-building networkâGenerate Biomedicines was one precedent, and Flagship had created about 110 startupsâbut departed early from its usual asset-company path: outside capital arrived before the Series A and Flagship did not lead that round. Andy says that, if classified as biopharma, Lila would probably operate a top-three GPU cluster.
On which field is harder, Andy describes small molecules as combining synthesis and chemical reasoning with biology, immunity, and adverse effects. Rafa says materials are harder because they lack biologyâs central dogma and mature automation, require harder math, and offer laboratory tests that only partially predict lifetime performance. The disagreement remains unresolved.
Materials are also harder to underwrite: successful suppliers may remain nameless inside supply chains, qualification is slow, and important industrial problems often sit behind closed doors. Government and national-security demand consequently matter more; Lila works with national laboratories, the British and U.S. governments, and was a named partner in the Genesis Mission.
Their chosen bottlenecks expose both halves of the company. Rafa would eliminate the materials âsim-to-realâ gap because abundant virtual data still fail to predict experiments; Andy would raise model FLOPs utilization from roughly 5â6% toward 100%, recovering paid-for GPU capacity and redeploying capital into faster answers or more laboratory infrastructure.
Full transcript
Brandon
But not just TechBio, what do you do in terms of science?
We are all in on the bitter lesson and scale. We think that methods that scale and that are general beat those that are not. As Ilya said at NeurIPS last year, we have but one internet. It's the fossil fuel we fracked; we got every ounce of data that we could out of the internet, but it's gone. And so the question for AI is: Where is the next internet-scale data set coming from?
Brandon
People normally talk about different scaling axes. You have compute, you have data, and for science, data is not necessarily an infinite resource. Your point is that we now want to add a new scaling axis for data.
We think that the lab of the future should feel like a data center: rows of server racks as densely packed as possible, and also as energy-efficient as possible, and things like that.
Brandon
Welcome to Latent Space Science. I'm Brandon, I'm here with my co-host RJ. Today we have Rafa GĂłmez-Bombarelli and Andy Beam from Lila Sciences. We'll start offâwill you introduce yourself?
Yeah, thanks for having us on the podcast. Long-time listener, first-time caller. Excited to be here. I'm Andy, and I'm the chief technology officer at Lila. I've been an AI researcher now for something like 20 years, going back to the pre-deep-learning days: SVMs, random forests, things like that.
I did a neural net PhD from 2010 to 2014, right as deep learning was taking off. It was clear neural nets were the thing to back, but autograd libraries really hadn't been developed yet, so I did the backprop by hand, back in my dayâwalking uphill both ways kind of thing.
I got very interested in AI for healthcare and life sciences. My wife's a physician, so I watched her struggle through different things and thought that AI was obviously a natural solution for a lot of those problems. I did a postdoc at Harvard Medical School doing early work on medical AI. I was really in it for the AI; I was really interested in what problems AI could solve.
But I've also always been startup-curious. I took a break from academia for a year and helped start a company called Generate Biomedicines, which was an early generative biology company. I was the founding head of machine learning there and got to do the fun hybrid professor-startup-founder thing for the next 5 or 6 years.
I had a lab at Harvard, again between the School of Public Health and the Medical School, doing methods research but also a lot of applied work. Those were a great set of jobs, but I got a sense that the AI moment was changing in a very significant way, and I wanted to be a part of it.
I started to think about where I could work at the frontier of AI and on really exciting problems. Academia has a lot going for it. Access to scaled compute is not one of the things that it has going for it, nor are scaled resources. I'd been an early advisor for Lila and got very excited once the thesis crystallized.
Basically, science is an infinite token generator to train models at scale. Why would I want to work on anything other than creating a new frontier model that can solve scientific problems? I joke that I hung up the tweed jacket 2 years ago, left my position in academia, and joined Lila full-time as the inaugural CTO.
Yeah, I go by Rafa. I'm the chief scientific officer for physical sciences at Lila and a co-founder. I was a computational chemist back in the day. We used a commodity resourceâthat is, compute. It was clear that we could scale up the compute to do molecular simulations, and that's something that produced enough data that, in the early 2010s, we realized we had a data problem. Things switched gear for me right around then.
I worked with David Duvenaud and Ryan Adams on blending what I think felt like the first instances of deep learning for science. I was one of the first people to do generative AI for chemistry, with an autoencoder on tokenized molecules. I'm so deeply in love with latent spaces. We actually have a logo very similar to your guys' logo, but for molecules, and that figure has taken on a life of its own.
Alessio Fanelli
This is the one that will be on your tombstone.
Exactly. My students have a Slack channel just to post it when it shows up in the wild.
Not as much of a story as Andy's, but it was the same conversion. In the 2015â2016 era, I spun out a computational materials platform company from my postdoc at Harvard, and then went to MIT, where I started my group in materials science and engineering.
The group there was working at the interface of molecular simulations and AI, with things like generative models for material structures and autograd for really cool gradients that we wanted to see in molecular simulations. By 2022â2023, things were taking the turn that Andy just mentioned. We had seen the bitter lesson come to computationally generated data.
That's the reason why Meta, DeepMind, and Microsoft have teams doing AI for computational materials science. But it was clear that we needed to bridge a gap, get this thing all the way out, and do AI for actual materials science, not just the computational version.
That lined up with the opportunity to start spinning out something again in 2022â2023. I started thinking about the idea, and I'm very excited now to have been pushing this integrated vision of scientific reasoning across all the modalities of science that we can validate in the lab.
Alessio Fanelli
All right, that brings me to: What is Lila's thesis? It seems like you have a very ambitious goal here.
Yeah, it's a great question. I'll try to give you the TL;DR, and then we can go a couple of levels deeper. As Rafa said, we are all in on the bitter lesson and scale. We think that methods that scale and that are general beat those that are not.
That sounds straightforwardly true, but is actually counterintuitive and contrary to much of the 70-year history of AI research. The realization that we had is that what gave rise to large language models over the last 4, 5, or 6 years was access to a combination of scaled compute and scaled data.
That data came from the internet. It was human-generated, and we have used it all. As Ilya said at NeurIPS last year, we have but one internet. It's the fossil fuel we fracked. We got every ounce of data that we could out of the internet, but it's gone.
The question for AI is: Where is the next internet-scale data set coming from? Post the pretraining era, we moved into reinforcement learning with verifiable rewards. People talk about RL a lot, but really what RL is is a way for a model to generate its own data, and the reward signal reinforces good data and penalizes bad data.
That has been a very productive framework for problems in math and coding. But what we at Lila believe is that scienceârunning the scientific method and using nature and experiments as a verifierâis the ultimate version of that.
What we're buildingâwe'll talk about these things that we call AI science factoriesâare scaled verifiers for science, so that we can do post-training at scale and push out the frontier of what reasoning models are capable of. That's the thesis in a nutshell.
Alessio Fanelli
Your proposal is basically this: People normally talk about different scaling axes. You have compute, you have data, and you have parameters. For science, data is not necessarily an infinite resource, and your point is that we now want to have a new scaling axis for data.
Correct.
Alessio Fanelli
To quote some of my friends at the Escalate Bio, they have a really good blog postâI recommend you read it. It says, âYour experiment has a runtime.â So what is the runtime of your data collection?
That is an awesome question. It obviously varies by experiment. You can't make the ribosome go faster, at least to my knowledge. Biology sets a limit for how fast you can go. In materials science and chemistry, there are smaller time scales and bigger length scales.
What you're actually asking is a technical question, though: How do you train a model against feedback mechanisms that vary by orders of magnitude in terms of feedback? We think about all of Lila as being able to generate different kinds of data on different length scales. We can then synchronize how we train the model once that data has been generated.
Again, for some of the experiments we do, the length scales are on the order of days or weeks. Then the question is: Can we multiplex? Can we get more data per unit of time? The infinite token generator is still there. We just have to solve the technical problem on the other side of that to be able to line all these pieces up and train it into the model.
Alessio Fanelli
When you say the infinite token generator is still there, what do you mean by that? There are many different scientific tokens you can imagine, and some tokens provide much more information than others. Certain things you can collect at scale. People who love NGS can basically collect an infinite amount of NGS data.
Yeah.
Alessio Fanelli
And yet, there are certain cases where another human genome is probably going to be an incremental update versusâ
Yeahâmy genome relative to a reference genome is a couple of kilobytes' worth of information.
There's not a lot of information there. So, you're exactly right. We don't want to generate the same kind of data over and over again. And so, the platform that we're building is qualitatively different from a traditional automation framework.
Actually, the experimental platform that we're building prioritizes generalizability and flexibility over raw throughput. We want the model to be able to design a new experimental protocol, run the protocol, and receive the feedback, even if that's not an experiment we have thought about doing ourselves. The next incremental token has to be something that is valuable to the model versus yet another NGS sample to teach it something where it's already hit diminishing returns.
swyx
So, when you say ânext experimentâ at Lila, what I think of is that traditionally you would go into the lab, reconfigure the lab in whatever way, and then run some experiments by hand, maybe over the course of weeks or whatever. How does the lab get reconfigured for the new experiment at Lila?
Rafael GĂłmez-Bombarelli
The way to think about the lab is that it's almost like a graph. Each instrument is a node in this graph, and an edge between the nodes indicates that there's a physical transport layer between those 2 instruments.
swyx
Yeah. It's a good one. I think half of the audience might not know what a PCI bus is.
So, it's a universal serial bus that, on your motherboard, allows you to connect a new device. If you plug in a new graphics card or a new hard drive, there's a bus that allows that device to speak to the rest of your computer.
swyx
And this works for biological systems and materials systems, et cetera?
Increasingly, but not totally yet. The other thing to keep in mind about automation is that there's a very long tail of things that you have to solve to be able to automate.
swyx
Yes.
Rafael GĂłmez-Bombarelli
To date, people have not been thinking about end-to-end automation in this flexible kind of way. There are instruments that are not connected to this now. There's not a lot of high-throughput automation in materials science, for example, and we've been building custom instruments for that that are then brought on board.
There's an 80/20 rule I play here: things that are easy to onboard and automate are plugged directly into the PCI bus, and then things that are not, people still move a sample to those instruments. It turns out that removing a cap from a test tube is a very hard thing to automate. A lot of the lab assumes that you have opposable thumbs and you're good with them.
Some of the things that we've seen discussed about Lila frame this as an automation company, and that's kind of the wrong perspective to think about what we're doing. We're not automation maximalists. We are actually token-generation maximalists and flexibility maximalists. We will, over time, automate things that make sense to automate and then use solutions now where they make sense.
The system designs the experiments. It gives instructions. There's a point where people need to actually do something, so you recruit some of the staff to go and do that thing. Everything's an API call. Sometimes when you call an API, there's a robot arm; sometimes there's a human arm that does something.
swyx
Literally below the API line.
Yeah, well, funny thing. I think that, again, we want to spend resources where it makes sense to spend resources and make rational decisions. Sometimes it just doesn't make sense to try to automate a step when a person can do it in a tenth of a second.
What matters is that the model has the ability to give instructions to test hypotheses, and that all of that data is visible, transparent, and stored so that those tokens flow back into the model.
swyx
Do you have your AI models doing entire experimental designs that go beyond just a pre-existing protocol where you tweak relative ratios or sources, or what all goes into a pipe or pipette or something?
It depends on your threshold for novelty here. Certainly, for expression protocols and for some gene-editing work that we've done, we have tested the platform's ability to do that versus humans. The model gets about 80% of that zero-shot; humans get 0% of that zero-shot.
Are we doing fully open-ended, free-form experimentation now? No, not yet. That is the goal, but we're building toward that. That is the end state that we want to be in. We have seen the ability to do what would be an enormous amount of human intellectual labor over a very, very short time horizon.
swyx
So, when you're giving your AI models free rein to start designing new experiments, how do you make sure that these are things that should be measured, or validate that this is a good strategy, and that you didn't just waste a bunch of money?
The first one is that there is maybe an underlying safety question there. I think that we've been taking that very seriously from the beginning, both security and safety: security of the data and the safety of the model's suggestions. We have a very strong team. It's growing under very strong leadership.
That's the first layer: we have strong AI safety protocols that look similar to the sort of uplift considerations that people have been looking into in large language models. Only it's absolutely for real.
swyx
In a lab automation setting where you're working on some biophysical or materials science-type problem, what are actually the dangers you have to worry about? I generally think of malicious actors, or situations where you have a sufficiently complicated system that it could genuinely output something dangerous. It seems like, from the scope of Lila as I understand itâwhich we haven't talked about yet; maybe it'll come in a minuteâit doesn't seem like safety is actually going to be a major concern at this point.
It's something we need to take seriously from the beginning. It's something where we cannot afford not to get it right. I agree with you. Right now, it's in the hands of Lila employees whose interests are aligned and whose understanding of the platform is aligned with our mission. So, I agree: we don't have to worry about malicious actors.
We still need to worry, to some degree, about the model giving a suggestion. I think it's more about some things where it starts touching into lab safety, more than malicious actors. I don't think we're going to have emergent behavior where the model suggests an extremely toxic chemical. It's more about pushing an instrument such that maybe it overflows, or it combines chemicals it shouldn't have.
So, I think there's a chemical EHS safety layer that needs to be there from the beginning, because we're doing open-ended experimentation.
I do think Rafa is right in that safety is not something you can procrastinate on, because capability curves tend to be sigmoid-shaped. It can look like everything's fine, and then all of a sudden there's something that you didn't anticipate the model being able to do.
So, we are definitely proactive on that side. We have an AI safety team, like Rafa said, but I think you're also right in that we can constrain the problem in meaningful ways, in the way that a broad-based AI system that interacts with the general public cannot. We can also lean on biosafety levels and things like thatâgood old-fashioned lab safetyâto help in the meantime.
And, of course, the knobs that are exposed to a particular questionâwe don't necessarily need to expose all the experimental capabilities to all the scientific questions, right? For an antibody-design question, we probably don't even need to expose a model to the fact that we have gas canisters that contain gases, right? Because it's not going to need them. So, we can still be creative within questions that relate to one particular area of science.
swyx
Yeah.
Your question, though, is interesting: how do you know if something is dangerous? That's actually kind of hard to do. Or, actually, how do you know if it's wasteful?
Some of the work we've been doing in electrocatalysts, we have someone inside Lila who's published 40 papers on the topic, and some of the suggestions from the model initially were boring, but then transitioned from boring to what he considered to be stupid. These are non-platinum-group electrocatalysts for separation of hydrogen and oxygen from water to make hydrogen, and those turned out to be our best non-platinum-group electrocatalysts that we've made.
The line between obviously wrong and genuinely surprising, even to a human expert, is hard to know, and so we will do wasteful things because we kind of want to know the difference between the 2.
swyx
That brings up the question: for an experiment like what you're describing now, it is obvious whether it works or not, right? But you can imagineâand there was some controversy in previous work at Berkeley Lab around measurements that were misinterpreted, right? How do you know that your measurements of effectiveness, or whatever you're optimizing, are actually correct?
Yeah, I'm very familiar with that part of the landscape.
Rafael GĂłmez-Bombarelli
I would say we cannot relax our standards of scientific rigor because itâs AI, right? Maybe 5 years ago, when we started doing generative models for X and Y, they were like, âYeah, itâs cute. It kind of works.â Like you would with a kid. But now weâre past that, and we need to hold AI science to the same standard we hold regular human-led science.
I think that 2023 paper was a switchover for the community. A part of the AI communityâAI-for-science peopleâwere always excited to see incremental progress, and I think at that point we started collectively touching upon the rest of the communityâs awareness. They were like, âFantastic, but now weâre going to talk about the way we do things to our highest standard.â
I think we have lots of experimentalists, and I want to go back to the API point. Weâve had the fortune, by starting from zero, to build a company where people are AI-aware and AI-excited. Across all the people and networks that Iâve collaborated with, weâve managed to build a team of experimentalists and automation engineers who really believe in the mission and really want to make it happen.
Theyâre really taking this graciously, right? Whenever AI gives something that is very, very wrong, theyâre there to say, âOkay,â and push the red button: âWatch out, this is a bad idea.â But theyâre also gracious in, for instance, trying false positives. False positives are terrible for human scientists, right? You go to try something, it doesnât work, but the model is fantastic. It reduces uncertainty a lot. For the operator, itâs kind of a bummer because you thought you were going to get something cool.
I think weâve managedâand going back to the point Andy madeâI think until 3 months ago, people would be approving AI decisions. About 3 months ago, we started seeing that the modelsâ crazy ideas had started being surprising to people, but surprisingly good. Itâs like, âI donât know. I guess we need to try.â We see the switchover to the ability of people experimentally to challenge the AI by being gracious. That interface between humans and computers has been very rewarding over the last few months.
Also, giving the model control of the lab forces you to build infrastructure to expose pieces of data that you would not normally want to or care about. Maybe no experiment is wrong, but you want the ability to explain the outcome. If you think about it, if you have an experiment and you fit a statistical model, what youâre trying to do is use variation in inputs to explain variations in outputs.
We have the ability to explain variation in outputs because we measure so many different things, because we have to expose that to the model. We can say, âOkay, the humidity was off in the lab that day. Maybe that explains exactly the thing.â Then we can also push a button and rerun the experiment to verify. In some sense, we have to be more skeptical of any outcome, like Rafa said, but we can then quickly go and rerun that experiment because itâs all software, effectively.
Alessio Fanelli
Do you find that the team spends a lot of time on verification? Howâs the breakdown?
Less and less. At the beginning, it wasnât so muchâI mean, thereâs an execution of, âOkay, we got a hypothesis. We got a set of instructions thatâs going to go off to the API. We vouch for it,â and then, on the part of the API, there are people doing things. I think that will stay, right? Thatâs the labor of it.
But I think the double-checking that the intuitions were right is happening less. Weâre starting to see, in this superintelligence, local spikes where weâre supporting the emergence of superintelligent behavior more than weâre gatekeeping to ensure that the ideas arenât just wasteful. I think thatâs happening for domains.
Maybe, to elaborate a little on the example that Andy mentioned, we care a lot about energy and sustainability, right? Weâre not just a biotech; we really care about energy and sustainability and materials. Weâre trying to make green hydrogen.
In order to make green hydrogen, you need to use light to split the water molecule. A bunch of that energy you need to pay for because itâs the energy thatâs stored in the chemical bond, and you get that from sunlight and electricity. Then thereâs some overhead that you pay; thatâs called the overpotential, which has to do with the fact that the world is not perfect and things are lossy.
The loss comes from something called the catalyst. Today, the catalysts that are out there are okay, but theyâre expensive and rare. Theyâre made out of ruthenium and iridium. So we set the model up to explore what we can do to avoid using these 2 elements.
People write these papers, and there was something a couple of weeks ago that said, âRuthenium, low-ruthenium alloys for OERs.â I mean, yeah, sure, you can dope it down, right? You can water it down, but itâs still the same fundamental problem. Youâre just using 50% less.
We set the model loose on this type of problem, and we have the ability to make the material, measure the properties, measure the catalysis, and measure the stability. Then, on the 2nd or 3rd generation of sequential learningâthis interplay between information and what the model knowsâwe started seeing suggestions that were, I mean, the words were fine. It was using the concepts that we use, but it was saying, âI wouldnât apply that idea to that element. I wouldnât have put them together in that way.â It turns out those have been our best-performing chemicals so far.
swyx
I do want to get to why weâre not a biotech, but before we do, one last question along this train of thought. RL is famous for reward hacking. I forget what you said. I donât know if you said you were using RL, or learning iterations. Iâd be very concerned that you throw some rewards in and can really hack the physical sciences in a way that you canât do with compute.
Rafael GĂłmez-Bombarelli
Yeah, Iâm not going to disagree with that.
I 100% agree with that.
swyx
Whatâs the funniest example of reward hacking youâve seen?
We have lots of funny RL fails that are not explicitly reward hacking. One is when we trained one of the early things we did: âCan you just make a plate map? Can you lay the experimental conditions out on a plate?â It got annoyed when the person would ask. You would ask the model to do a plate map, and it would do it, and then youâd say, âActually, could you change these reagents?â It would swear. It would be like, âItâs a 96-well plate. Come on, man. Itâs not that hard.â In the chain of thought, I donât know where that came from, but it wouldâ
Alessio Fanelli
Somewhere on the internet.
Andy White
Yeah, somewhere on the internet. I didnâtâI forget that. Weâve seen lots of funny personality quirks like that as a function of RL.
There are obvious RL pathologies. I wouldnât call them reward hacking, but thereâs repetition. The chain of thought will collapse, and it will just repeat its final answer over and over and over again. For some reason, that reliably sometimes leads to higher rewards. Weâre not sure exactly why pathological chains of thought, or non-legible chains of thought in some cases, lead to higher rewards.
swyx
Sorry, I want to interrupt. Weâre talking about RL. The way youâre talking about it sounds like just RL on chain of thought, like everybodyâs doing. But your RL actually has a lab step.
Yeah.
swyx
If youâre in a pathological loop, does that mean the lab is just doing the same experiment over and over?
Itâs just not. A chain of thought, maybeâjust to step backâis the tokens that the model uses to solve a problem. If you were solving a math problem, you would do theorem 1, theorem 2, corollary, lemmaâyou know, you decompose the problem.
In science, the chain of thought has some of that, too. Thereâs reasoning that happens: âIâm trying to make an antibody for this target. What do I know about this target? What are the known epitopes? Whatâs my plan of attack?â
There are also tool calls in the chain of thought. Maybe Iâm going to use a structure-prediction model in this case to get some read on how the sequence folds in 3-dimensional space. Tool calls are part of the chain of thought. At Lila, the fun thing is that the lab instruments are also tool calls, or a series of tool calls, if a workflow or aâ
Alessio Fanelli
Itâs all human-legible. Itâs all in English.
Right. Some of the pathologies weâve seen are that it just skips all the middle part, which we would think is important for solving a problem, and goes right to the answer. It says, âI donât need to do an experiment in this case. I donât need to call a tool.â
In some cases where we can judge something because maybe weâve already done the experiment, for some reason itâs actually not a bad strategy. Thereâs some mystery there.
Alessio Fanelli
Itâs a theorist.
Andy White
Yeah. Itâs done the calculation. This is probably too much of a tangent, but it actually thinks in latent space. It emits tokens, so the chain of thought is often an unreliable narrator for what computation the model is actually doing.
swyx
One of the big things we're trying to think about when we're moving into working on a problem, like Rafa said for electrocatalysis, is that we actually don't know what right and wrong looks like. How much should we rely on the chain of thought versus just trusting the experiment, trusting the verifier, or trusting the simulator as the ultimate ground truth?
You know, Lila is not a biotech company. Lila is actually fairly unique, I think, in this way.
Rafael GĂłmez-Bombarelli
I've been involved with biotechs. I've helped start biotechs. Often, the goal is to sprint to a clinical trial. You want to develop an asset, and you develop the platform in service of having optionality for what space you move into.
But once you have the clinical asset, you put everything into a medically induced coma and get through the clinical trial. If it goes well, then other things get toâ So, we are taking that option off the table. The model itself is the thing of value at Lila.
In that sense, we're much more of a neolab, trying to think of a new way to push forward the capabilities of a core reasoning LLM-based model.
swyx
Not even the lab platform?
Well, the lab platform is the token generator.
swyx
Okay.
Rafael GĂłmez-Bombarelli
That is the data-generation mechanism that ultimately is the moat for Lila. Once that continues to scale, the amount of data that we can generate, both per unit time and per unit square foot, will go up, and that feeds back into the model to make it smarter, which then suggests the next experiment to do.
We really are focused on making this core model as performant and smart as possible. We can talk about how that lends itself to different commercial strategies, but ultimately, we're interested in creating this new type of AI model.
swyx
I want to quote Sridhar Kota from Octant Bio, who had a great tweet I really loved a few weeks ago: âWhat is the business model in ML for drug discovery? Because if you need the data to train the model, but if you have the data, what do you need the model for?â
That is true when you are narrowly scoped. That is true within any given vertical of science. The analogy that I would use is that if you went back 10 years and tried to create a coding assistant model, you would just get coding data. You wouldn't also get Shakespeare poetry or carnitas recipes.
It just turns out that there is spillover as the model is able to train on a broader swath of data and a deeper cut of data. Again, the core bet that we're making is that this is true for science: if the model is trained on an increasingly broad set of data, the amount of data that you need in a given domainâthat data requirementâis reduced.
In some cases, it will be reduced to zero if it's adjacent to what the model has already seen before. There is a data-efficiency argument that would suggest that having a general platform that can create a broad swath of scientific data is valuable.
I'll also just mention that obviously we are using things that are already commodities. We use public datasets and simulators, and the experimental platform is a complement to these existing commodity resources.
Alessio Fanelli
This brings up a question in my mind about applicability domains, where you have different scales, and they result in completely different types of information and relationships between entities, right? You have the quantum realm, the chemical realm, and different biological realms.
One concern I would have with cross-cutting approaches is: is there domain transfer between these at all? Carnitas recipes and chess problems have the commonality that they're written in language, whereas you almost have a completely separateânot even language, right?âmodel between these domains.
Rafael GĂłmez-Bombarelli
Human scientists work on all of those domains.
swyx
Correct. And they mostly communicate with each other in written language usingâ
Okay. Using tools. I would say there's a common reasoning process that allows someone to solve problems in each one of those domains. I think that logic carries over to a reasoning model that we're training, which again uses tools, can do math, and can do code, but it's having all of that knowledge stored in one place.
Alessio Fanelli
One classic example for me of domain transfer is between complexity theory and quantum gravity, right? A lot of the quantum-gravity theories are basically recognizing the identical math behind the two.
Rafael GĂłmez-Bombarelli
Yep.
Alessio Fanelli
Do you have examples of this kind of, âOh, man, this domain actually applies to this domainâ?
Rafael GĂłmez-Bombarelli
We have assembled this reasoning dataset of 10 trillion scientific tokensâreasoning traces that are experimentally verified across life sciences, chemistry, and materials science. We have seen that this general model often beats the domain-specific models.
It's hard to point to what's in the model that is making it work or what connections it has realized. But clearly, having seen more data across all science beats domain-specific reasoning models in a sample-for-sample kind of way.
swyx
And the future of science is language, right?
Yeah, the future of chemistry is language.
Yeah, yeah.
swyx
Maybe.
Rafael GĂłmez-Bombarelli
I don't think it's necessary. I don't think that's a necessary condition for a scientific superintelligence. There was a quote from Demis Hassabis last week that it might not be worth distilling everything into language. There are data modalities that are so different from language.
They always tell me, âWell, Rafa, English is Turing-complete, so you could express everything in English.â
swyx
Turing-complete languages, but yes, yes, yes.
I agree with everything Rafa said. Token-based reasoning with tool use is very powerful, and I think the claim that we're making is that we have barely scratched the surface for that in science.
We're not trying to distill domain-specific models into a reasoning model; it can use those tools productively. The combination of reasoning, often in English but also in Python and things like that, combined with tool use, is very powerful, and we're very early in science in understanding how far we can push that forward.
swyx
I see. Can you give some examples of campaigns that you are running that are representative?
Actually, before you do that, let's take a step back. I realized we still haven't explained that you don't just do bioânot just any tech bio. I think this is a great lead-in to this. Not just tech bio: what do you do in terms of science?
Science? No.
The way that we train the model is allâI mean, it's across life sciences: DNA, RNA, proteins, cells, small molecules, different kinds of chemistries, and different types of materials. That's where we're scoped now.
swyx
But materials itself is also not just one thing; it's as broadly scoped as everything that's on the bio side.
We're going to give some examples. Today we can make thin films, we can make powders, and we can make quantum dots. We have a cute quantum-dot demo.
Are you folks familiar? Quantum dots are the luminescent technology in some TVs, and you need to control them to make them exactly the same nanometer size. The nanometer size you make them controls what color they're going to be, and you need the purest red, the purest blue, and the purest green to make a really sharp and rich color palette for your TV.
You need to make them as homogeneous as possible; they all need to be the same. Otherwise, the colors get blended. We have a cute demo where, when visitors come into the office, we ask them to pick a wavelengthâwhat color they want their quantum dot to beâand then we fire off the machine.
The model reasons, and sometimes we even throw in new chemicals that the model has never seen, just to see how it moves. The machine is running, and by the end of this hour-and-a-half tour, the machine has made maybe 1, maybe more generations of quantum dots that tend to hit the target. Otherwise, we wouldn't do it, right?
It then hits the color that people suggested. We have the ability to make lots of materials. We can formulate liquids, polymers, and soft matter.
We care about energy and sustainability a lot, so we have a good chunk of electrochemistry capabilities around the interplay of chemical transformations and electricity as a renewable energy source. We care about traditional catalysis, and we care about the mechanical properties of materials.
All this comes together in programs where we make catalysts, high-performance coatings for corrosion or aerospace, and high-performance mechanical applications.
Rafael GĂłmez-Bombarelli
Over the last few weeks, with an external partner, we started multiple sprints on things we werenât doing before, touching everything from adhesives to cooling fluids. So, weâve been able to spin up more and more exciting discoveries in open-ended chemistry and materials science spaces.
swyx
Do you have any connection between quantum dots and, letâs say, protein design?
Itâs the same platform that does that. Itâs the same set of capabilities, and so thereâs a shared infrastructure that lets us do all of those things under the same roof. If there were no connective tissue, then we just would not have the ability to do all those things. We havenât done a deep dive on the mech-interp thing. We have seen that our ability to do these programs has gotten faster as the platform has become more mature.
swyx
So, is LNP just a common thing in your toolkit? Is this something really common in your toolkit that, because of this, enables a large fraction of these ideas that youâve just mentioned? It certainly makes sense on the bio side, but I know bio much more than materials. Is that a common theme in your lab toolkit?
The more capabilities the AI science factory has, the faster weâve been able to go after new target product profiles and exciting new opportunities, because the model is prepared to do more things, the lab can do more things, and our scientists are more flexible and faster at incorporating new capabilities. Adding new instruments has become faster the more instruments we have, so it goes with the type of company we are. There are echoes of hyperscaling here: scaling in software is backed by scaling in hardware, and the fact that we have tens of thousands of square feet of lab coming online, with dozens to hundreds of instruments, is giving us this breadth to move fast.
You folks had my colleague Heather Kulik on the podcast recently. One of the areas where you were seeing disruptionâI think the audience will be familiar with these materials, so I donât need to spend a lot of time introducing themâwas in materials made from the interaction of a molecule with a metal. It turns out that our models had been trained on small-molecule drug discovery, and all of the chemistry they had learned from thinking about drug discovery carried over into reasoning about these metalâorganic framework materials, which we can use to take COâ out of the air or filter ammonia.
swyx
I find that fascinating. When I think about machine learning, so many times Iâve seen people work on machine learning where they train on some big data set and then move to a new domain, and oftentimes the amount of transfer you see is small.
Yes.
swyx
So, are there a group ofâI donât know how to say thisâprimary colors that you combine together that oftentimes result in your experiments?
Rafael GĂłmez-Bombarelli
So, in biology, the obvious candidates would be nucleic-acid synthesis, cell-free expression, and downstream assaysâthings that we care about. Theyâre core competencies that we can then use to give rise to a factorial number of different things that you can do. And on the material sideâ
swyx
I think formulation. It wasnât evenâitâs so, I donât want to say mundane, but itâs so common, itâs so important that it wasnât one of the first super-intelligent places we thought of. We thought of flashier things back at the beginning. It turns out a lot of people in industry and in the rest of the world care about formulation, meaning mixing liquids and gooey things to make other gooey things. Thatâs lubricant, slurries, nanoparticles, deodorantâthere are all these things in consumer products, industrial products, and medicine. Gels for skin graftsâall those things emerge from mixing gooey materials, and thatâs a muscle that weâre building. Thatâs a very common platform thatâs showing up all the time the more we talk with people.
Interesting.
swyx
I have a rule of thumb that I often use when Iâm thinking about scaling, which is that every time you scale an order of magnitude in a system, your set of problems completely changes. You guys picked the 2 hardest problems, right? Materials and bio. You do other stuff, but materials and bio are notoriously difficult to get to market. They have 10-year, 15-year time horizons, and the reasons are especially difficult for materials scaling. So, how are you thinking about that? Are you just saying, âWeâre discovery,â or are you saying that weâll get to it? How are you thinking about it?
Rafael GĂłmez-Bombarelli
The last academic lecture I prepared before I stopped giving academic lectures was called âThe Bitter Lesson of Scaling in Materials and Chemistry.â It turns out, in AI, scaling is a good thing because it gives you a roadmap of what you need to do. In chemistry and materials, scaling is a spooky thing because only the things that you can scale matter. So, weâre extremely cognizant of that. Our product team and our lab team all know this.
For instance, in the quantum-dot example, we were able to use the same recipe from a single-digit number of milliliters to a hundred, or almost a liter. So, there are places where our capability today takes bites into scaling and into technology-readiness level. Weâre making the system such that it can reason about whatâs going to matter later as itâs doing the experiments now.
This will be true in our rare-earth-free or platinum-group-free catalysts. Precisely, the nature of the question is that we need to be able to scale these. Itâs supply-chain-conscious as weâre firing off the first experiment. Weâve already read every paper. We already have a techno-economic analysis agent sitting in the corner, ready to do the techno-economics of anything we do.
At the end of the day, weâre not going to do clinical trials. Weâre not going to make pilot plants for one particular process that you would put in your refinery. At that point, these are places where we will work with our customers or, if we find something so amazing that we donât even need any instruction, weâll just go sell it. But typically, we will hand offâjust like weâre going to support therapeutic discoveries for our customersâweâre going to support materials innovations at the pain points our customers have. And those also have to do with scaling.
swyx
How far have you gotten so far?
So, on the life sciences side, I think I agree with everything Rafa said there. The way to think about how people would use the platformâor just what weâre buildingâis much more of a cloud-code-ish kind of thing for science. One of the things that has drawn early customers to us is that weâre not an in vivo CAR-T company. There are lots of in vivo CAR-T companies. Itâs super hot right now. We did see, 6 months ago, with the Capstan acquisition for like $2 billion-plusâif folks arenât familiar with that, in vivo CAR-T is this very new, hot therapeutic modality.
Previously, it was for blood cancers, but now increasingly for autoimmune disease. We did have in-house the sort of triumvirate of capabilities that you would need to do in vivo CAR-T: binder design, obviouslyâwe can do thatâLNP formulation, and then mRNA design.
swyx
Just so people know what CAR-T is, because itâs really freaking cool.
Itâs so cool. Can I talk about CAR-T as well?
swyx
Yeah, talk about CAR-T.
CAR-T has been worked on since the late â80s or â90s. It really caught fire around 2010 or so for cancers. The way it used to work is youâd extract someoneâs T cells. You would engineer whatâs called a chimeric antigen receptor that goes on top of that and tells the T cell what kind of cell to go and kill.
swyx
So, youâre basically modifying peopleâs T cells. You take them out and modify them so that they have this weird antigen receptor on their surface.
Itâs a seek-and-destroy tag. Usually, they use a protein called CD19, which is preferentially expressed on B cells. When B cells get malignant, they create blood cancers. They create autoimmune diseases. You wipe out almost someoneâs entire B-cell repertoire when you do this. Thereâs a lot of collateral damage. But essentially, youâre telling the T cell what to go and kill.
This really started to catch fire around 2015. It was expensive and slow. You have to extract someoneâs T cells and engineer them. Itâs like a $400,000-per-infusion treatment, and itâs still a miracle cure for lots of different types of cancer.
Itâs too much of a tangent for this, but thereâs this child named Emily Whitehead who was treated at the Childrenâs Hospital of Philadelphia, CHOP. She was one of the first cures for pediatric cancer with CAR-T. She was going to be referred to hospice care, but got CAR-T.
Another slight tangent: she almost died of a fever from this initial CAR-T treatment. The only reason she survived was that the doctor treating her had a daughter with pediatric arthritis and knew that this specific antibody would blunt her IL-6 response to CAR-T. Thereâs a lot to unpack there in terms of AI for science, and all the serendipity that had to happen in that specific case for everything to go right. If you roll that dice 1,000 more times, you probably donât get that doctor at that moment, who knew exactly what antibody to give her to make the treatment curative instead of lethal.
Again, those are the types of serendipitous things that weâd actually like to automate. Anyway, itâs slow and expensive. People then realized that, through just an infusion, if you take an mRNA that encodes the chimeric antigen receptor and put it in a ball of fat called a lipid nanoparticle, then put a CD8-targeting moiety on the outside of the ball of fat, you can give it to someone in an infusion, and it will bind to the T cell and get ingested. The ball of fat dissolves, the mRNA comes out, and the chimeric antigen receptor gets expressed and presents on the top of the T cell.
swyx
So youâre just tellingâyouâre reprogramming the T cells to express these weird antigens.
Literally programming biology.
swyx
Yeah. And then the T cell goes and does its thing and wipes out whatever has CD19 in this case.
Malignant B cells explain a lot of blood cancers. They also explain a lot of autoimmune diseases. B cells often make antibodies in response to autoantigens and things like that. So recently, 6 months ago, as the result of about 6 yearsâ worth of work spun out of a Nobel Prize winnerâs lab and about $100 million worth of R&D, we saw
swyx
Yes, thatâs some good music, man. [Laughter]
We saw some of the most compelling preclinical data for in vivo CAR T treatment of autoimmune diseases. At Lila, we had been working on all 3 of those things in isolation. About 6 months ago, a team of 2 or 3 people inside Lila tried to see what we could do with in vivo CAR T. What we had been working on was mRNA design.
Like most RNA medicines, the biggest knob that you can turn is expression peak and expression durability. How many proteins do you get per unit of mRNA when you give someone a vaccine or some other mRNA medicine? We have developed some monster UTRsâuntranslated regionsâwhich flank the protein-coding region and dictate those expression properties. Theyâre something like 10Ă the reference UTRs from Moderna and Pfizer.
Over the course of 6 months, we got to in vivo data in nonhuman primates where B-cell depletion was significantly better than what was shown in the Capstan data, and the durability of that was also better. All the characteristics that we looked at were significantly better. Having more CAR expression is probably one of the most potent ways to improve a CAR T therapy.
The number of receptors that get expressed dictates how likely that T cell is to bind to the bad cell once it finds it. T cells are literally serial killers in that they will kill a cell, then go to the next one, then the next one. How long they can do that is dictated by how durable the expression of the CAR is.
Again, weâre not a CAR T company; weâre science nerds. We like to do cool stuff. We got to that proof point in about 6 months, again all the way up to where you might think about filing an IND for a new clinical asset. Weâre not going to do that; weâre not going to do a clinical trial. Again, that would be all-encompassing.
Some folks who had been around Lila for a long time saw that as a way to do essentially a 2- to 3-person FTE startup. A couple of scientists who have domain knowledge, combined with the model and the platform, can do 5 yearsâ worth of biotech work over a 6-month period for 10% of the total investment.
A lot of the commercial relationships weâre thinking about now are essentially the zero-FTE startup model. Someone comes with an idea and says, âIf there was a CAR T in the market that could bind to 2 things, if it was a bispecific, or if it had these other properties, I know the hole in the market that that thing would plug into.â
A lot of our commercial engagements are effectively virtual startups running on Lila now, where someone comes with a very well-specified problem. They donât know how to get there. There may be some things related to target identification and things like that, too. But they can effectively run that entire program over a much shorter amount of time at a fraction of the cost.
swyx
So those are like: a partner comes to you and says, âI have this idea. I donât want to build a lab. I donât want to hire a team. I just want to get there.â
Yep.
swyx
So it could be some academic at a university who says, âI have this idea. I did a little bit of validation. I think itâll work. Can I sit with you guys for 6 months and make it work?â
Sam Rodriques
Yeah, thatâs the right way to think about it. The way that it contractually plays out is thereâs a platform access fee. We have to pay for reagents and running the system, and then some overhead and stuff like that. Then thereâs some upside sharing.
That is a scalable model. As the platform gets better, instead of doing dozens of those, we can do hundreds and then thousands of simultaneous virtual startups being developed on the platform. We have revenue that helps pay the bills in the near term, but then we also have this upside partnership with folks who decide to build with us.
swyx
Itâs amazing, because this is what weâre seeing: people are more and more pushing toward getting rid of all the extraneous infrastructure, using automation, and focusing on the idea.
The way that I think about it is that most of us got into science because weâre curious and want to answer questions. Iâm a computer scientist by training, and I like to answer questions through software. However, if I had to program in binary, I would enjoy that significantly less.
Theyâre high-level abstractions, increasingly high-level abstractions. It used to just be Python and Java. Now itâs like Claude Code that helps me answer questions faster.
The analogy is that scientists are still programming in binary. They have a question that they want to answer, and they have to just compile that down to an experimental protocol. Then they have to go and do the manual labor and get arthritis by moving liquids from one well to another. Thatâs the equivalent of scientific programming in binary.
Weâre trying to help scientists move up the abstraction ladder. Maybe your idea isnât going to workâmost clinical trials failâbut you can at least get to failing fast if you donât have to do both the physical labor and also some of the intellectual labor to get all the pieces in the right place.
swyx
You know, most clinical trials fail. Somewhere between 5% and 8% of clinical trials actually get from IND to approval, so discovery is not actually the constraint. I was interestedâyou were talking about the sort of economic modeling agent. I canât remember exactly what you called it, but that seems like the problem to solve. How do you think about this?
You mean the success rate of clinical trials?
swyx
Well, the economic model underlying what? Scaling in general for both bio and materials. Oftentimes, thereâs this huge process. Once you have something that you consider final, like an IND or a development candidateâ
Yeah, for materials, thereâs still usually like 10 years of clinical trialsâor qualification, in the materials science worldâto just get that into a product. Oftentimes, the bottlenecks there are things about scale, manufacturing, regulation, safety, and things that are often just very hard to answer up front.
So every time you do this, you just have to roll the die. The typical way people deal with this is essentially a portfolio model, and financing-wise, itâs very much the only way you can make money if you scale with some level of risk calibration.
swyx
Yeah. Itâs really exciting to hear that you can do these things specifically, but how does it feed into the larger thing where, even if you solve these problems immediately, itâs still only 10% of the problem?
The reason why U.S. biotech is losing to Chinese biotech is not because of an innovation problem. Thereâs a regulatory framework, too, that has to enable fast clinical trials. The FDA has made motions toward that recently, both for the preclinical data that you have to submit in some cases and for how we will run and monitor trials.
It would be crazy to think that one company couldâor even any company combined couldâchange that on its own. It has to be done in tandem with the regulators. However, the minor moves in preclinical probability of success matter a lot.
From a portfolio theory perspective, it makes the investment much more attractive. It means that, in expectation, medicines get to patients faster and fewer of them fail. I would say that is the area that weâre focusing on now: a medicine created by a system that has had the benefit of, in this case, a million unique mRNA designs to maximize things that are known to translate to therapeutic benefits will meaningfully move those preclinical probabilities.
Itâs better to throw a loaded die than it is a fair die, and so weâre just trying to make the die as loaded as possible.
swyx
I guess my thinkingâthis is the thing that I think about constantlyâis how do you bring, basically, translation, right, in whatever the equivalent is in materials? How do we name that? I really want to see a model that thinks about these factors, reasons about them, and is very good at saying, âIâm filtering my designs to the ones that I think are going to make it through Phase 3,â right?
My wife is a translational scientist in biotech, so I see itâit reminds me very often: yeah, you guys should be doing AI for translational science.
In a sense, I think that's some of the echo, especially in materials and chemistry. Our tools can call process-engineering simulators and figure out what pipe diameters and what heat exchangers you should be using in order to scale up the process for the economics to be worthwhile. So maybe there isâI don't know if we're going to gain a filter until we go measure itâbut the ability to reason now about the things that will come downstream, which is a little bit what translational AI would do, is to reason now about what's going to matter.
I would just say, earlier, when you have the IND, you're locked in, and it's true on the chemistry side. The molecule, the sequence you've chosen, of courseâwhich population you're going to give it to and how you're going to measure successâthose choices you make afterward, right? I don't work on the preclinical stuff, but on the chemistry and materials side, that is precisely the type of behavior we're trying to instill now with the verifiers and the data sources that we can access, either because somebody has thought about them, because the physics allows it, or because we can measure good-enough proxies now that tell us what's going to happen later.
To be clear, all those things are things that we talk about internally a lot. We're already on the verge of being pathologically over-scoped.
But I'm absolutely of the belief that, as models get smarter, as they ingest ClinicalTrials.gov, and as we partner with pharma companies and get access to that cookie jar, these pre-trial probabilities will meaningfully change. On the biomanufacturing side, having access to biomanufacturing processes and scale-up processes, too, we think the models will be able to contribute there.
We've just chosen to focus a lot of our commercial and collaborative activity on the frontier of science that we think we can address now, but the goal is to push past that.
Alessio Fanelli
If I may summarize, it's kind of like a tool call.
Alex Schubert
Yeah, it's all tokens, it's all tool calls. Tokens and tool calls are all you need.
Alessio Fanelli
But also the reasoning mechanisms that maybe you mentioned for that doctor who treated Emily Whitehead with the IL-6 antibody. That person had learned that from a combination of lived experience and reading the literature, and we getâsince we believe our thesis that the breadth gives us thatâwe will get better at those things by doing more of the things we do.
swyx
So many counterfactual worlds: that doctor was not the one treating Emily Whitehead in that case, and CAR-T may have looked like it might have been yet another gravestone in Eroom's Lawâanother failed drug. I do think that went from a 2% success probability to a 98% success probability just because that person happened to be in the room. If we could just operationalize that, again, you're going to move a lot of probabilities when youâ
That's a good example of where just having really broad knowledge of scientific informationâ
swyx
Googleâ
âbecause that was only being used in pediatric arthritis, another very niche area of medicine.
swyx
I see.
Yeah.
swyx
So you have Ken Stanley on your team, who famously wrote the book Why Greatness Cannot Be Planned and is very big on open-endedness and serendipity in research. What is the role of open-endedness at Lila?
Alex Schubert
Ken is awesome. For those of you who don't know, Ken pioneered an area of machine learning and AI called open-endedness, which I think of as machine creativity. How do we get models to do open-ended exploration and also have a sense of taste about what's interesting and what things we should pursue?
You can't have scientific superintelligence if you're just a good test taker. If you think about what reinforcement learning is doing, even at scale, it's answering questions in a ruthlessly Vulcan-esque, Spock-like way. But you would probably only think of that model as supremely creative in limited ways.
Ken has created an open-endedness team at Lila to take on the outer loop, or the meta part, of that reasoning challenge. How can we get our models not only to answer tough questions, but to ask interesting questions in the first place? That's really Ken's mandate.
He's been building a world-class team over the last several months, and they're in the kitchen cooking now. I think by the end of this year, we'll have some cool stuff from Ken's group to share.
swyx
We're going to hop into a video here of the lab that's going to show a couple of different things.
Okay, so that's probably a plate sealer. When you move plates from instrument to instrument, obviously there's liquid in themâmost biology is wetâso you put these stickers on them. That was a plate being sealed.
All right, here we go. It's picking up a plate. This is inside of a liquid handler. We'll wait until it gets to a wider shot so that you can see the PCI bus and some of the robotics.
The liquid handlerâ
swyx
Is it magnetic?
Yeah, so this is the planar motor system here, where the plate magnetically levitates. This is the PCI bus, where the transport layer connects all the instruments. You can see benches there where all the instruments sit.
The robot arm picks it up and is now going to transfer it to a different place to go on to the next step. There's a little bit of a traffic-control issue here: they will actually go and park for a while while traffic congestion clears. Here's a long shot of the PCI bus. Again, all of that is fully controlled and fully automatic.
This is a physical-science example, where it takes us back to the scaling point. Here, it's making our hydrogen catalysts in a scaled-up form factor from an ink that contains nanoparticles of the material. That's a spin coater, as you can guess from the fact that it spins the plates. This is a robotic handler moving around little pieces of catalyst to test.
Rafael GĂłmez-Bombarelli
This nice-looking purple â90s neon vibe is a magnetron sputtering machine. We make atoms fly from a source and deposit on the other side of the chamber in a very thin atomic film. We can make arbitrary mixes of elements based on what's on the 3 or 4 sources. We vaporize them, make them fly over the chamber, and make these nice, thin films that are very material-efficient.
We can do this with very little material, and it's one of the workhorses for us to design, make, and test fast in many applications: catalysis, corrosion, mechanical properties, and many things you can test in this convenient form factor.
Shawn Wang
The liquid handlers and some of those machines are mostly off the shelf. Then you've come up with this form factor that works for lots of those machines, both for materials and for biology.
Rafael GĂłmez-Bombarelli
Quantum dots are a good example of the combination of the two. It's actually a liquid handler that we've repurposed for quantum dot synthesis and enzyme work. I think that speaks to how far you can get with 20, 30, 40, 50 instruments. They just have to be on platforms that the model can use.
I think this was a big eye-opening thing for me coming into Lila, because I wasn't in lab automation in any meaningful way before coming to Lila. It's not the automation I was hoping for. A lot of automation is point automation, where there's a tablet attached to the side of a liquid handler where you can enter commands, but that device is not meantâand sometimes purposely designed notâto talk to other things.
A lot of what we've done has been toâI kind of joke that we have the world's largest collection of voided warranties in biologyâwrite our own custom drivers and our own custom firmware to get low-level, granular control over a lot of these instruments and make them talk to each other.
The video is cool because you see magnetically levitating plates. What you don't see is the custom software wrapper that stitches all of that together. A lot of this comes down to really hard software-hardware interface challenges.
Some of the machines literally still run Windows 95. Think about how you automate that. We actually have a vision-language model controlling a Windows 95 machine.
Shawn Wang
Yeah.
Rafael GĂłmez-Bombarelli
Because that's the only way to automate it.
Shawn Wang
I was going to joke about a mechanical finger pressing buttons, butâ
Rafael GĂłmez-Bombarelli
Joke, but no, we did that. We actually used a robot to push the iPad on the side of the thing.
The other thing to call out here is that this is still automation made for people. The instruments sit on benches that are approximately chest-high because there's the assumption that someone needs to reach in there to service them or fill the reagents.
This is the V0, V0.5 of what we think lab automation will look like. Because we've just decided to vertically integrate and own the hardware and software stack, the V2 will look very different from this. We'll be able to integrate things.
This is happening already in materials science because those capabilities just don't exist. We often think about labs in terms of their x-y coordinates. As we integrate, we'll have a z component too because we'll be able to stack things. So tokens per unit volume is what we'll be thinking about then.
The lab of the future should not be made for people to easily walk into. It should feel like a data center, where you go and see rows of server racks. There's room for a crash cart behind it to service the nodes, but it should be as densely packed as possible and as energy-efficient as possible. So, to answer your question, we're using commodity things now because it makes sense to get started, but over time, almost surely, the form factors of those will change quite a bit.
Shawn Wang
I see. I'm just a little surprised that you can come up with this common size of tray that kind of matches your needs for a good percentage of your problems.
Rafael GĂłmez-Bombarelli
Mhm. Well, it's just working backwards. Ninety-six-well plates are the atomic unit of experimentation in lab automation, so we now do 96-well form factors for materials science as a result. Not everything fits into that form factor, but the coverage that you get from adopting a 96-well or 384-well plate formatâ
Shawn Wang
80/20.
Alessio Fanelli
Yeah. I think that you can see some of those where the pieces of deposited material were bigger. So we still use the plate shape to carry them over, but then the number of samples that you have in them is smaller. I think some of them are maybe 12â4 Ă 3.
Rafael GĂłmez-Bombarelli
Yeah. This takes me also to a point you folks asked earlier about scaling and how, when you scale, your problems are different. A problem I think we're looking forward to collectively at the company is the orchestration and scheduling of a data-center-sized AI science factory. When, out of all the experiments you could run concurrently, how are you going to think about the logistics and orchestration of moving all these samples and interfacing all these instruments to create the maximum value for our customers, the maximum information for our model? That's an exciting problem. That problem is going to look very different from some of the other problems we're thinking about now.
Shawn Wang
What we think about, as Rafa said, is orchestration on top of that, like a Slurm queue or something like that, that lets you globally maximize throughput of the system that you have. But again, using those same abstractions to think about throughput, scheduling, and orchestration, as the system gets complex or gets larger, the complexity in maximizing that throughput increases. If you're a constraint-satisfaction-problem nerd, we have one of the coolest ones to think about. Are you thinking about scaling as one cluster and then you just cookie-cutter that, or is it, âI have all of my liquid handlers and all of my spin coaters and everything in different parts of the labâ?
Rafael GĂłmez-Bombarelli
I mean, currently what we have, essentially, is one big, fully connected graph, and that won't scale indefinitely. Some of the materials we use throw off hazardous fumes, so that's isolated for safety reasons. I don't know exactly what the configuration and layout of the science cluster of the future looks like, but I think that it will probably have fewer instruments on it than you might guess you would needâhundreds, maybe thousands.
We do think about scaling it in the same way that you would think about scaling a data center: it's a multilevel building, occupies millions of square feet, and is a lights-out facility, as they say. It's running 24/7, generating data in real time, and you would want the same uptime that you would expect of a data center. Now, that's very hard to do.
Shawn Wang
Uh-huh, yeah. That's an insanely hard thing to do.
Rafael GĂłmez-Bombarelli
But that is the endpoint that we're trying to work backwards from. What problems do you need to solve on the way to that endpoint?
Shawn Wang
No, this goes back to my previous question about the runtime of your experiments, too, because scaling means different things. One of them is experimental design, which intrinsically scales, but maybe at the cost of signal-to-noise ratio or some other ideaâgetting broad data quickly and efficiently, at some cost. Or scaling is lower throughput but just parallelizing wildly. In general, I would approach those as 2 different sets of problems. I don't think the same strategy really works for them in general. What types of scaling are more important for you as a scientist?
Rafael GĂłmez-Bombarelli
I would say round-over-round iteration is more important than a broad, hugely multiplexed, highly noisy kind of thing.
Shawn Wang
So iteration time is really the single thing.
Rafael GĂłmez-Bombarelli
Yeah.
Shawn Wang
Okay. So does that limit the domains that you want to focus on? Now, if we're going to try to tackle a new problem, do we ask, âCan we just solve this problem with faster iteration?â Versus something where maybe the answer is that you scale up by massively multiplexing something, but with a month-long turnaround?
Rafael GĂłmez-Bombarelli
Parallelizing and multiplexing are somewhat different, right? So sometimesâ
Shawn Wang
That's right.
Rafael GĂłmez-Bombarelli
I would say pooledâwe love pooled.
Shawn Wang
Yeah.
Rafael GĂłmez-Bombarelli
We love pooled because you get fast and broad.
Shawn Wang
What are pools made from?
Rafael GĂłmez-Bombarelli
Pooled things are things like DNA-encoded libraries, where you have a bunch of stuff in it and you can sort out the stuff after you do the experiment. Somehow, the form of the assay allows you to throw 1,000 or 1 million or 1 billion experiments at the same time. The way the assay is set up, the readout picks the winner. So you try 1 million things in one plate and you get 1 readout, or 1,000 readouts of the 1,000 winners.
Alessio Fanelli
All biotech is just mapping whatever readout you want onto NGS, and you can multiplex your target. There you go. You can get lots of data.
Rafael GĂłmez-Bombarelli
The elegant argument would be this: if the standard for this field is a month and it's going to take us 4 days, a 4-day learning cycle is amazing because it's really going to move the needle for that part of the field. This is where our automation engineers and our teams are thinking about other ways of measuring things.
In coatings and in catalysis, there are places where we just made different instruments that measure a different property that turns out to respond 1,000 times faster. For instance, in sorption, I can tell you folks a little bit. In gas sorption, people typically measure by pressurizing an amount of gas. For the MOF and COF materials I was talking aboutâsucking COâ out of the airâyou know, from the ideal gas law, if you remember from high school, how much gas you put in the little box. Then you wait for the gas to be adsorbed in the material, check the pressure, and from the difference in pressure you know how much went into the thing.
Then you raise the pressure again and see how much extra went in. If this sounds slow, it's because it's very slow. It's called BET. This takes about a day per sample, and it's very tough to parallelize because it's another gas line, another canister. Or you can take other types of proxy measurements from other instruments that are parallelizable. That's something we built in the lab now: instead of measuring pressure, we're measuring another property we care about that is a readout for what pressure would actually tell us, but we can do 96-well plates for 96 metal-organic frameworks in about an hour. So it's maybe 2,500 times faster.
This is a place where there's a little bit of room for ingenuity. An hour is still slow compared to other readouts, right? Other things in electrochemistry maybe we can do in a minute. But now we're a thousand times faster than the way we were doing it.
I think the answer to your question also depends on how much we think the model is starting from a dead start versus a walk versus a jog. If there's some area that we care about, some question, and it's clear there's zero knowledge in the weights of the base model that we're using, then we may prefer a big, slow thing to move it in. If we think that it's already relatively competent in that, then we would vastly prefer the rapid-serial, fast-iteration cycle.
So we'll do both. The bet is that, as the model performance improves, the sample efficiency goes up, and therefore the compound interest that you get from round-over-round experimentation will outweigh what you would get from a big, noisy but broad data set.
Shawn Wang
Do you have any concern? I'm just thinking out loud here, but do you have a concern that you're going to quickly saturate the problems that you can solve usingâ
Concern or hope?
Shawn Wang
Or either. Okay, okay. Concern and hope, maybe. But maybe you have these systems that you're putting in place, and right now, because they're new, there's a lot of greenfield. You can go and tackle all these problems that are amenable to high-throughput experimentation.
You're going to do that for a couple of years, maybe, and then, all of a sudden, everything is different. You have to completely retool your billion-dollar investment.
Alex Schuth
I hope that that is true, to be clear. I hope that we don't have to measure a binding K_D again in 2 years. If we didn't have to do that, I'm very pumped about that, because the model has essentially mastered binding kinetics.
Shawn Wang
So you would think that eventually you get to the point where the model knows how to do that, and you don'tâ
Let's go back to the PCI bus again. What we actually want to do is reduce the time it takes to bring a new instrument onto the platform. You want that to feel a lot like USB. I don't know how old you guys are, but when I was a kid, you got a new device and the drivers came on a floppy disk. You had to beat your head against the wall to get the driver to install, and 2 days later, your printer only kind of worked.
Shawn Wang
Yeah, exactly. If you're a Linux hardcore person, you can still live that experience today. Your audio driver still doesn't work.
Alex Schuth
That is what it's like to bring a new instrument onto the platform in biology and the physical sciences right now: we're at the driver-on-a-floppy-disk stage, with a manual to try to get it to work. One of the things that we hope a unified platform enables is for instrument onboarding time to eventually go to zero, where you have the spec from the manufacturer, the model reads it, and the right APIs get abstracted.
We're working with some instrument vendors to make this process easier, but I think a lot of the way that we think about modularizing a system is conditioned on how we do it now. Again, we're hoping that a unified platform makes onboarding an instrument 2 years from now a 30-minute exercise versus a 30-day exercise. It's a hard thing to do, we could be wrong, and we might not be able to do it, but that is the future that we're pointing to.
Currently, we can actually swap out existing instruments very quickly. If we need to replace a Hamilton with a different liquid handler, that swap already happens very quickly. We do have some reasonable belief that onboarding instruments will get faster, better, and more reliable over time. Again, we don't want to be doing 2026 science in 2036. We hope that some of these instruments get deprecated or that the way we're measuring things changes. Otherwise, lots of assumptions that we and everyone else made about the rate of progress in the next decade will have been wrong. They were wrong.
We've already been benefiting from the instrument vendors, right? I wish the problem we had were what you're describingâthat we'd run out of science to do with the instruments. That would up the ante for the instrument vendors. The instruments we have now are as powerful as a beamline would have been 10 years ago.
We're taking measurements today that, 10 years ago, would have required you to ask the federal government for a time slot at 2:00 in the morning somewhere out there, wasting a couple of nights of sleep taking measurements at a really bright neutron or X-ray source. Today, the vendors make instruments like those that we can put next to the quantum dot or next to the protein expression.
Shawn Wang
I wish that were an end state that is desirable but very, very unlikely. I'm sure there's going to be new science to ask of the instruments we have. We've had guests who have had both of these themes. First of all, none of the devices you buy are set up to do high-throughput AI science. Also, there are new scientific devices that come up every day that open up something that was impossible 5 or 10 years ago.
Like inline NMR. There's lots of characterization, miniaturization, higher resolution, and brighter sources that are transformational, and they marry really well with the kind of automated, high-throughput science we're doing. We're moving into this facility in Cambridge, Massachusetts. This is just a 3D rendering. It's a 100,000-square-foot space, and we'll move toward AMRs, or autonomous mobile robots, as some of the transport. You can see some of that there.
Shawn Wang
We'll put it in the show notes. Switching topics a little bit, you were talking about your scientific pile of 10 trillion tokens.
When I hear 10 trillion, my first thought was, âMan, that sounds like a lot.â Then I was like, âThis is 3,000 human genomes,â which would cost roughly $3 million to sequence. It is roughly 1/2000 of the size of several of these large foundation models, like Evo and Nucleotide Transformer, and so on.
In some sense, it is a lot of data. In another sense, it's not a lot of data. Certainly, not all tokens are the same. I'm curious: what went into creating this? What were your thought processes? How much actual useful information is in 10 trillion tokens?
These are tokens in the same way that we think about counting pre-training tokens from the internet or from post-training runs. RL is, again, a data-generation mechanism. The best way to think about RL is as a way to steer the model toward more and more valuable tokensâbetter tokens.
These are the result of running that process across many different scientific RL environments at Lila, where the tokens are a mix of English, tool calls, and experimental feedback. They're quasi-English tokens, as we've been talking about, tokenized by the tokenizer. That's where they came from.
Shawn Wang
So you're not tokenizing sequences in general.
Alex Schuth
We're not tokenizing sequences in general. Implicitly, if the model is asked a question about DNA, there are DNA tokens in there. It's not like we downloaded dbGaP, the PDB, or Swiss-Prot, or something like that, and tokenized at the sequence level. These are reasoning tokens, model-generated and experimentally verified.
Shawn Wang
On top of this, you also still have AlphaFold, Nucleotide Transformer, and all your sequencing data, which go into this. So 10 trillion tokens isâ
Alex Schuth
10 trillion. The reason why we think that level of data is important is that pre-training corpora are usually somewhere between 15 and 30 trillion tokens. That's the scale at which you see these emergent things happen. Once you're in the trillion-token regime, we feel confident that that's enough for the model to start to master things and see emergent capabilities.
Shawn Wang
Are you starting from scratch with your model, or do you have some open-source model?
Again, in the interest of being ambitiously over-scoped but not pathologically so, we have not decided to take on pre-training as well, just because the black magic that you have to do is insane. We've been gifted something like $1 billion worth of compute in the form of open-weight models.
We start with an open-weight model that has been pre-trained, and the assumption that we're making is that the model has been pre-trained on the internet and a large fraction of the scientific literature. Therefore, it's a good scientific prior over what is known and a good base camp to build upon.
Shawn Wang
So it's 10 trillion on top of the trillions thatâ
Alex Schuth
Yeah. We use Nematron quite a bit because we have a partnership with NVIDIA. I think there are 30 trillion tokens that go into the pre- and post-training for that model.
Shawn Wang
In the process of creating these reasoning tokens, you are also creating what are arguably probably rather useful datasets themselves. Have you thought about independently releasing some of those datasets open-source? Even in the absence of the reasoning model, they may still be quite valuable to the community without actually deteriorating your moat at all.
One of the things that we've developed along the way is a test suite of something like 1,000 unique scientific RL environments, where you can drop in a frontier model, your own model, or our models. Almost surely, we're going to open-source a subset of that.
Some of it will be based on data that we've generated, and some of it will be data that we've curated for the community to use. There will be some open-source version of the benchmark that we've assembled as part of doing that. There will probably be some training data that goes along with that.
Shawn Wang
Cool. Do you have benchmarks internally that actually operate the lab? Essentially, a benchmark for how wellâmaybe that's not the right way of saying it. Do you have automated experimental controls?
Yes. I mean, we've put together, from the beginning of the company, multidisciplinary teams to work on specific, closed-ended problems. The modus operandi has always been to benchmark something trained naively from zero against the frontier models that everybody would use right out of the box, as well as against our own internal models.
swyx
With everything we've done, we do have an internal benchmark. The domains are very specific, right? They're not as general and all-encompassing as the benchmark that Andy was describing, because they're the things we really care about, the products that we want to deliver, and the places where we want to make a difference.
But in all those places, we've typically seen that the scientifically pretrained model Andy is describing, with access to tool calling, typically demolishes, of course, anything else that we compare it to.
swyx
I mean, it's worth thinking about what we're trying to do and how that is additive with LLMs. If you think about an experimentally verified reasoning trace, how many of those do you think exist on the internet or in the pretraining corpus?
On the order of zero?
swyx
On the order of zero, yeah. It certainly rounds down to zero versus the next order of magnitude. We've just seen an incredible lift from showing the model that, yeah, even if we're at a parameter disadvantage relative to the frontier models, just showing it an experimentally verified reasoning traceâyou see an immediate lift when we do that.
Lila is a Flagship company. Flagship is basically one of the biotech incubators in the world. They've had something like 30 successful IPOs, orâI don't know. But your parent companyâ
You know, we're all, including you yourself, just part of Flagship. Generate Biomedicines just had a successful IPO very recently. So Lila is very good at biotech. I would say, from history, it's very much single-asset, traditional biotech.
Flagship is very good at biotech.
swyx
Flagship. Yes, yes. What did I just say?
Lila.
swyx
Lila. Yes, yes. Well, yeah, Flagship is very good at biotech because, historically, it's been very focused on single assets. I guess in the last few years, with Generate, with, I guess, Evelo and Valo, there has been some branching out into more platforming things.
Gevorg Grigoryan
Yep.
swyx
I'm curious about one thing: How does Lila fit into the broader Flagship ecosystem? Was there a specific reason why Lila is nowâwhy the sort of pivot from single asset into scientific reasoningâand what is the broader interaction? In particular, you mentioned that you had a CAR-T drug that was at the level of an IND, so you clearly have the ecosystem to make that into something. I'm curious where this is going.
Yeah, great question. Let me do a little Flagship framing, and then I'll talk about Lila. We all started as the same pluripotent stem cell, but there's differentiation that we all take.
The traditional path for a Flagship company is this: The history of Generate is that I was an early advisor to Generate, a consultant in 2018. There was this idea to use machine learning for protein engineering. A couple of other folks at Flagship and some external folks who came in got seed money from Flagship to then go and spin that out.
We worked on building the technology, and usually the deal is that Flagship is the sole investor during a Series A. Then the Series B is normally the first point at which external capital comes into that. To your point, these often end up being asset-based companies. Generate has a Phase 3 trial for a monoclonal antibody to treat asthma, and a Phase 1 trial behind that to treat COPD.
I think the recognition from some folks at Flagship, especially our CEO, Jeff Builtzen, was that he had created or been involved in creating a lot of these companies, and he saw that he was hiring the same team over and over and over again. You need the ML team, you need the platform team. So I think he saw shared DNA between all these companies and thought, "Let's have one company that can essentially support all these different things."
Year 1 of Lila was essentially when o1 dropped. We had all these pieces in place, and it just became clear that we could create a platform to support a new kind of scientific model. In the early days, we didn't know: How do you monetize that? What's the commercial strategy? We've gotten a lot of clarity over that in the years since, but the core conviction that we had 2 years ago was that the bitter lesson is correct: Science could be an infinite token generator.
Operationally, the way that we're different from a normal Flagship company is that outside investment came in before the Series A. Again, the lead of the Series A was not Flagship. We do have that lineage, we do come from Boston, and we have a lot of the shared learning from a company that has created 110 startups.
Generate was Flagship 56 or 57. It was actually a merger. In the early days, Lila was number 96 or 97. Flagship has this enormous, long history of creating companies, so we have that network and the learning of leaders who have created that many companies. But we're such a weird creature that we went down a very different path very early.
swyx
So why is it that when I hear you have a very promising CAR-T therapyâwhat you said you had at an INDâwhy not just partner that out? Maybe this is on the horizon or something.
The short answer is that we are engaging in commercial partnerships around CAR-T therapies, for sure. Some of them involve further development to increase or change some of the properties, like bispecifics and things like that, going after novel indications. We've used that one CAR-T to essentially launch several partnership programs around it.
swyx
Okay, so it's the proof of principle, but it itself was not exactly what you would need for a drug or something.
Just to be clear, we could go and try and license or partner that specific thing. We found that it was better to take that and secure several partnerships around further development of it.
swyx
You're basically doing some sort of co-development thing whereâ
This is the virtual startup idea, where a company starts a virtual startup around one of these indications, and they essentially pay us revenue for further development. Again, we have these milestones and things around it.
swyx
So, long term, since Flagship is specifically biotech and has never really branched into materials, how does that weigh on Flagship's or Lila's strategy? Does that play into it at all, or is it that at this point you've kind of launched and used a lot of the resources?
Resource-wise, if you differentiate into something different so quickly, right, I think part of the breadth of the mission clearly was beyond biotech from day 1. The people we needed to hire came from different networks. The instruments we had to buy came from different vendors than Flagship vendors would usually have been.
I think that was part of the reason why it feels somewhat different, but it's also core to the mission. We cannot get these to work on a narrow field. By definition, we want to be as broad as we can possibly be, because that's where the emerging behaviors are going to come from.
If you looked at the composition of people who work at Lila now, it would look categorically different from what you would expect a median biotech company to look like. We hire out of, or compete for, and sometimes win against people who are considering frontier lab offers. We have a heavy software engineering and tech presence. The amount that we spend on GPUs would be atypical for a biotech.
I think that if we called ourselves a biopharma, we probably would have a top-three GPU cluster in the world. It's true that that's part of our DNA, but we've been intentional about trying to make decisions that put us on what we think is the most promising trajectory for us. This isn't just kids rebelling against their parents or something. We think that the thesis is right, and it points toward a very valuable, but also important, companyânot just for biotech, but for materials and chemistry.
swyx
Okay, that brings me to what I think is my last question. What's harder, materials or biology?
They're actually very difficult. It's funny, right?
swyx
I feel like we're about to do the Spider-Man meme.
When I was around for the first merry-go-round of AI for small-molecule drug discoveryâI mean, Atomwise, Strateos, Generate, and insitroâI think the hardest thing is a small molecule. It has all the difficulties of chemistry, knowledge, reasoning, and synthesis. Then it has all the difficulties of reasoning about biology, adverse effects, and immune response.
swyx
Yes, but the counterpoint is that we have so many tricks in our toolkit which you can borrow from biology, right? So it's harder, but you also haveâ
I think materials are harder. They have the benefit of great simulators that we don't have in bio. In materials science, you don't have the mature high-throughput automation that you have in biology.
For me, materials as a subject is interesting because there's not a unifying principle like the central dogma. Materials means lots of different things. I actually still don't quite understand the unifying principle when we say materials scienceâwhat exactly that meansâand then the commercial dynamics are completely different.
Again, with CAR-T, we know, if we wanted to, how to monetize that directly.
Alessio Fanelli
With materials, thereâs a supply chain, there are devices, and the testing that you do in the lab is only partially predictive of the lifetime of how that material will be used. The math is harder.
Gevorg Grigoryan
In terms of supply chains, they still matter for both. Maybe you replace clinical trials with some product validation and verificationâqualification, as the term is. There are direct analogies, and there are hard parts for both of them.
swyx
The economics are very different. If you pass a clinical trial, you make money. That thing is valuable, and how much it costs to make it is very rarely the blocking element.
Alessio Fanelli
Itâs much easier to underwrite an asset in biology than it is in materials.
Do you guys know the name of a company that makes a superconductor? You know, this always comes up. Yeah, we care about magnets, we care about superconductors. Theyâre really cool science. Do you folks know the name of a company that makes superconductors? No one knows. These things are super important.
Shawn Wang
Theyâre used in MRIs.
Rafael GĂłmez-Bombarelli
Exactly.
Shawn Wang
Thatâs the only commercial application I know.
Rafael GĂłmez-Bombarelli
But it turns out that when you succeedâwhen you make a cool material that does somethingâyouâre kind of a nameless company that makes this thing, is successful, and has good cash flows. But you donât get to break through. Everybody knows Big Pharma, but other than that, youâve got your 3Ms, right?
Yeah, and most of the big material companies are behind closed doors. Most commercial engagements look like getting them to tell you what the important problem is. Thereâs less of an open innovation ecosystem. A couple of things in materials are obviously recognized to be valuable, but I think itâs just very different from life science.
One of the last things we havenât touched upon a lot, and I want to flag, is that in chemistry, and especially materials, government-sponsored research is a big driver. In the same way that the government doesnât feel it needs to do drug discovery, other than funding NIH for early-stage open science and hypothesis-driven science, the government and national security drive materials innovation in ways that are unique.
You see this in the way we engage with the British government. We have partnerships, we work with the U.S. government, we have awards, and we participate in developing materials and technologies. Thatâs a different part of the ecosystem that drives innovation. Thatâs also different.
Shawn Wang
Yeah, definitely. Are you guys working in the Genesis Mission?
We were one of the named partners. Weâve had an ongoing relationship with a lot of the national labs, and so we have been working on that.
Shawn Wang
With these 25â
Chad Edwards
Yeah.
Shawn Wang
Genesis Mission Lighthouse proposals last week.
I was joking. He left academia thinking, âGreat, writing was behind him,â only to have to writeâ
[laughter]
Alessio Fanelli
Yeah.
Chad Edwards
They speak about 20,000.
Alessio Fanelli
Yeah.
Shawn Wang
The question that we like to ask all of our guests is: if you could remove a bottleneck in your domainâand you can define âdomainâ by fiatâwhat would that bottleneck be?
Rafael GĂłmez-Bombarelli
To me, Iâm going to go to old-time Rafa, who was doing physics-based simulation. I would say sim-to-real. For people who come from the physics-based world, sim-to-real meansâ
Shawn Wang
Can you explain what that means?
Rafael GĂłmez-Bombarelli
What I mean is, these people have typically meant it in the context of robotics, where your virtual simulations in 3D spaces allow you to train robots that will move in physical spaces. But thereâs a gap, and they call it the sim-to-real gap.
For us, in physics-based simulations, we do molecular simulations of gooey stuff, and we do electronic-structure simulations of hard stuff. Theyâre okay, but theyâre not predictive enough. This is the reason why, if it werenât for that, maybe we wouldnât have had to make a self-driving lab for materials, because we would have been able to just predict.
I think we know thereâs physics, but it doesnât quite get us to being predictive. The models that we train on physics cannot close the gap, either, because theyâre still missing theseâor rather, theyâre trained on approximations that are just not good enough.
The thing weâve been chasing for a decade in AI for materials has been: if we train on computational data, can we answer real-world experimental questions? If I get to go back in time, in addition to taking the bottleneck out, it would be the accuracy of the underlying simulation that weâve been training on all the time.
Shawn Wang
So this is sort of like what Heather Kulik said: there is no AlphaFold for materials.
Rafael GĂłmez-Bombarelli
Well, the funny thing is AlphaFold was trained on experiments, so itâs different. Thatâs kind of funny.
Alessio Fanelli
Yeah.
Rafael GĂłmez-Bombarelli
She and I both come from doing physics-based simulations, and the fact that she called out something that had no simulations in it whatsoever is kind of the same underlying issue. Meta has produced tens of millions, hundreds of millions of training data points, but theyâre all virtual simulations that just donât carry enough water for the thing we actually want to do.
This is going to be a boring and obvious one, but thereâs a metric that you use to track how efficient your training runs are. Itâs called model FLOPs utilization, or MFU.
The GPU comes with an advertised peak FLOPs throughput, which is, under the best situation, doing a calculation that you donât actually care about: how many floating-point operations can you do per unit of time. MFU is always a very small fraction of peak theoretical FLOPs. For reinforcement learning, itâs always somewhere around 5% to 6%.
Said differently, that means weâre getting 5% of the actual GPU computing power that weâre paying for. If I could, by fiat, wave a wand and make our stack perform at 100% model FLOPs utilization, I would do that, because we would, one, get to the answer faster, but then also be able to buy fewer GPUs and redeploy that capital to the lab or something like that.
Shawn Wang
Interesting, though, because your rolloutsâarenât they constrained by the lab?
Chad Edwards
They are, but when we train a big model, all of that dataâour reinforcement learning training pipelines are very complicated. One way to think about how you would do this at scale is to have the model doing rollouts left and right, waiting for enough trajectories to pile up, and then backpropagating that into the model.
A different way to do that would be to factorize it: have a bunch of expert models that are trained in parallel, either generating data or being trained themselves, and then distill that back into the central model. The second way is the more efficient way to do that.
All those things are happening at different timescales. When you have 10 trillion tokens and you want to push them through the model as efficiently as possible, youâre still going to be doing some reinforcement learning on top of that. If we could get all the FLOPs that weâre paying for, I would, by fiat, declare that.
Shawn Wang
Cool. Before we end, is there anything you want to leave the audience with?
Chad Edwards
Let me say why weâre here. We have an office in San Francisco now. Itâs at 181 Fremont Street in downtown San Francisco. There are currently 20-ish to 30-ish people who sit there, but weâre looking to expand that aggressively.
Weâre looking to pull from all areas of the stack. Weâre aggressively hiring for post-training. Folks who have been working in domain AI, like life sciences and materials sciences, weâre also hiring for that. Thereâs no wet lab here currently, so itâs all computational work.
If any of this stuff that people have heard about today sounds interesting, feel free to shoot either me or Rafael a message if that sounds interesting.
Shawn Wang
Thank you for being here. Itâs been a really fascinating conversation. Appreciate it.
Chad Edwards
Thank you for having us.
Alessio Fanelli
Yeah.