BioByte 114: reinforcement learning for LLMs, new 3D printing technique enables fully biologic tissue systems, Dual-Inf explains differential diagnoses, and a call for greater interpretability in AI
Welcome to Decoding Bio’s BioByte: each week our writing collective highlight notable news—from the latest scientific papers to the latest funding rounds—and everything in between. All in one place.
What we read
Blogs
The Urgency of Interpretability [Dario Amodei, April 2025]
In a new essay, Dario Amodei, founder and CEO of Anthropic, urges society to prioritize the interpretability of the many AI systems we are rapidly building and deploying. While we cannot stop the momentum of AI development, we can still influence how it unfolds by understanding the inner workings of these systems and steering their impact toward positive outcomes.
Amodei explains that much of the danger around AI comes from its opacity. If we are to ensure AI works for human good, we must understand how these models produce their outputs and find ways to guide their behavior through design.
Historically, early efforts in AI interpretability focused on vision models. Researchers discovered that some neurons represented human-recognizable concepts, such as detecting wheels. When Anthropic was founded, the team turned its attention to large language models, seeking to uncover how they process and generate language.
Initial work found a few interpretable neurons tied to individual words and concepts. However, most neurons showed a phenomenon called superposition, where many concepts overlap within the same neurons. Superposition allows models to represent far more ideas than the number of neurons would suggest but makes their internal logic extremely difficult to understand.
Progress accelerated when researchers applied techniques from sparse autoencoders. This allowed them to isolate clearer combinations of neurons tied to more distinct features, such as genres of music expressing discontent or linguistic hedging. Using these methods, Anthropic identified over 30 million features in a medium-sized model. Even so, this is only a small fraction of the billions of features that likely exist inside these systems.
Beyond finding features, researchers showed they could manipulate them by increasing or decreasing their influence over the model’s output. One example involved amplifying a model’s internal concept of the Golden Gate Bridge, resulting in a version called "Golden Gate Claude," which brought up the bridge even in unrelated conversations.
More recently, researchers have moved from studying individual features to studying groups of features called circuits. Circuits reveal how models combine concepts to reason through problems. Interpretability is also essential for models applied across biotechnology. Understanding for example how specific model predictions can be attributed to specific residues can highlight which positions are truly driving the effect.
Papers
3D bioprinting of collagen-based high-resolution internally perfusable scaffolds for engineering fully biologic tissue systems [Shiwarski et al., Science Advances, April 2025]
Organ-on-a-chip systems—which can be used to accurately model biological processes—are insightful tools for studying disease, vasculogenesis, and pharmaceutical responses. Despite their usefulness, reconstructing natural vasculature within engineered tissues to better simulate conditions present in living organisms has been a major limiting factor in the manufacturing of these devices. Additionally, the use of non-biological materials, like plastics and elastomers, as proxy for natural substrates imparts poor mechanical properties on the devices and often results in undesired interactions with lipophilic biomolecules. To address these limitations, a group out of Carnegie Mellon is facilitating significant progress via the development of a comprehensive tissue engineering platform.
The researchers in this study utilized a 3D printing technique called freeform reversible embedding of suspended hydrogels (FRESH) to bioprint cells, collagen, fibrin, vital components of the extracellular matrix (ECM), and growth factors to collectively form what they dubbed ‘collagen-based high-resolution internally perfusable scaffolds (CHIPS)’. The use of collagen—a naturally occurring structural protein—to create highly complex 3D constructs affords biological compatibility, improved cell health and function, and tunable mechanical properties with the incorporation of different types of collagen. To round out the platform, a novel bioreactor termed vasculature and perfusion organ-on-a-chip reactor (VAPOR) was also created in order to perfuse the cells and ensure they received the necessary nutrients. Using this system, they were able to manufacture a pancreatic-like system that carries out glucose-stimulated insulin secretion (GSIS), failure of which leads to type I diabetes. Over a 24 hour period, they were able to demonstrate that it could produce 8 ng of constitutive insulin.
These devices are not without their limits, and resolution proves to be a challenging problem in particular. CHIPS fabricated using FRESH have a lower feature size limit of about ~100 μm, an order of magnitude larger than those fabricated using photolithography. Issues related to methods of cell seeding are also an ongoing problem, as there are tradeoffs between adhesion and uniformity.
As demonstrated by the pancreatic-like CHIPS, advances in bioprinting have significant therapeutic potential. In fact, FluidForm Bio, a Carnegie Mellon spinout, is seeking to do exactly this in humans having already proven they can cure type I diabetes in an animal model in vivo. It is an exciting time to be in the organ-on-a-chip space, and further advances in the field will likely bring about a whole new slew of treatment options for patients in need.
Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? [Yue et al., arXiv, April 2025]
A team from Tsinghua’s LeapLab set out to test whether a reinforcement learning with verifiable reward (RLVR) model could unlock fundamentally new reasoning strategies that do not exist in the base model. They started with a base large language model and fine-tuned it by rewarding outputs that passed an automatic verifier. The authors measured accuracy over multiple attempts, increasing the number of tries (k) from 1 up to 256.
Their key finding was that RL-trained models performed significantly better at pass@1, meaning they found correct solutions more often on the first try. However, as k increased, the base model eventually caught up. This suggests that all the reasoning paths discovered by the RLVR model already existed in the base model's distribution. In effect, RLVR made the model more likely to select the best existing chain of reasoning, rather than creating new ones.
This aligns with the broader intuition behind reinforcement learning: it reduces the probability of wrong paths rather than inventing new strategies. While this improves efficiency, it also narrows the model’s capacity for creative exploration. As we look toward the next era of AI, fostering creativity will require new training paradigms that encourage exploration beyond the pretrained prior.
Explainable differential diagnosis with dual-inference large language models [Zhou et al., Nature npj Health Systems, April 2025]
While LLMs have shown promise in diagnostic accuracy, their ability to provide high-quality explanations for their reasoning remains largely unexplored. To address this, Zhou et al. collected an expert-annotated DDx (differential diagnosis) dataset and designed a novel LLM reasoning strategy called Dual-Inf to guide LLMs in determining accurate diagnoses with truthful reasoning chains.
The authors first curated the Open-XDDx dataset, the first publicly available structured dataset with DDx explanations, comprising 570 clinical notes across nine specialties annotated by domain experts. They then proposed Dual-Inf, a customized framework designed to optimize LLMs' explanation generation capabilities through a dual-inference mechanism (from symptoms to diagnoses and vice versa) and backward verification to boost correctness. The framework includes modules for forward inference, backward inference, examination (for prediction assessment), and iterative self-reflection to refine diagnoses and explanations. They evaluated Dual-Inf against existing prompting strategies using their dataset, assessing performance based on diagnostic accuracy (whether the correct diagnosis was included) and interpretation performance (how well the generated explanations semantically aligned with expert explanations). One thing to note is that the evaluation of diagnosis reasoning quality involved using other LLMs to measure semantic similarity between the Dual-Inf diagnosis and the expert-labeled diagnosis, which raises issues of how to trust the quality of those metrics.
Zhou et al.’s work demonstrates that there’s still juice to squeeze from existing LLMs with creative prompting and CoT methods to make them more reliable and interpretable in the clinical setting. Earlier this month, Google Research (McDuff et al., Nature, April 2025) demonstrated that their medical LLM AMIE outperformed doctors in generating accurate DDx - even doctors that had access to AMIE itself! Zhou et al.’s work demonstrates one way to systematically evaluate LLM DDx through careful dataset curation (Open-XDDx) and evaluation. Future work in the space to assess model quality, improve LLM DDx generations, and incorporate cross-verification with external knowledge sources like literature search will become increasingly important as these LLMs become implemented in real-world healthcare systems.
Notable deals
Axiom has raised a $15 million seed round to develop AI models that predict drug candidate toxicity.The round was led by Amplify Partners, Dimension Capital, and Zetta Venture Partners. The timing is important as the FDA plans to phase out animal testing over the next three to five years. Axiom’s first model focuses on predicting small molecule toxicity in the liver, trained on data from 100,000 molecules and their effects on liver injury.
10x Genomics and Ultima Genomics partner with Arc Institute to accelerate development of the Arc Virtual Cell Atlas. After their recent release of their Virtual Cell Atlas, a dataset containing observational and perturbational data of over 300 million cells, Arc Institute is looking to expand the Atlas project in hopes of “potentially compressing decades of drug discovery research into just a few years,” as reported by the institute’s official account on X. In this newly announced partnership, Arc Institute will reportedly be leveraging 10x's Chromium Flex technology as well as Ultima's UG 100 sequencing system and newly launched Solaris chemistry to further said Virtual Cell Atlas aims.
Merck KGaA announced Monday morning that it is set to acquire SpringWorks Therapeutics for $47 per share, valuing the deal at approximately $3.9 billion. This acquisition aims to enhance Merck's KGaA presence in the U.S. and expand the reach of SpringWorks Therapeutics' innovative therapies for rare tumors globally.
Creyon Bio and Lilly Enter into RNA-Targeted Oligo Therapy Development Collaboration. Eli Lilly has invested $13M in cash and equity purchase to work with Creyon Bio on optimizing oligonucleotide therapies.
Biohaven Announces Investment up to $600 Million by Oberland Capital. Oberland has invested $250M upfront to support the commercialization of troriluzole in anticipation of FDA approval for patients with spinocerebellar ataxia.
In case you missed it
DreamLLM-3D: Affective Dream Reliving using Large Language Model and 3D Generative AI [Liu et al., arXiv, February 2025]
Salvador Dalí’s iconic melting clocks were inspired by a dream that embodies the fluidity of time and reality, while Niels Bohr discovered the structure of the atom and credited a dream where he saw the electrons revolving around the nucleus like the solar system.
The authors presented DreamLLM-3D, a composite multimodal AI system that enables automated dream content analysis for immersive dream-reliving, by integrating a LLM with text-to-3D diffusion model.
“Dreamwork” strategies have been put forth by psychologists and neuroscientists to support people in deeply engaging with their dreams to attain personal insights, for instance, by re-experiencing dreams as if reliving the memory, feelings and bodily sensations it created. More quantitative approaches, such as the Hall-Van de Castle (HVDC), have been used for dream content analysis. The HVDC consists of ten categories (from characters to social interactions to elements from the past), together with detailed rules to recognize and measure those elements from written reports. Automatic scoring of dreams according to HVDC rules using LLMs has been used extensively by dream researchers already as well as work on dream text to 2D and 3D objects; yet without including the “rich emotional and social information from the dream narration”. The inclusion of such information could allow for a more affective dream-reliving experience for deepened personal insights.
An LLM pipeline analyzes and categorizes key dream entities, sentiments, and social interaction in real-time. The output is then fed into the 3D model. The social interaction information is mapped into the soundscape, while the sentiment information is mapped into the visual and soundscape.
What we listened to
What we liked on socials channels
Field Trip
Did we miss anything? Would you like to contribute to Decoding Bio by writing a guest post? Drop us a note here or chat with us on Twitter: @decodingbio.
 |
|
👏