Sparse Autoencoders, Additive Decision Trees, and Other Emerging Topics in AI Interpretability

Feeling inspired to write your first TDS post? We’re always open to contributions from new authors.

As LLMs get bigger and AI applications more powerful, the quest to better understand their inner workings becomes harder — and more acute. Conversations around the risks of black-box models aren’t exactly new, but as the footprint of AI-powered tools continues to grow, and as hallucinations and other suboptimal outputs make their way into browsers and UIs with alarming frequency, it’s more important than ever for practitioners (and end users) to resist the temptation to accept AI-generated content at face value.

Our lineup of weekly highlights digs deep into the problem of model interpretability and explainability in the age of widespread LLM use. From detailed analyses of an influential new paper to hands-on experiments with other recent techniques, we hope you take some time to explore this ever-crucial topic.

• Deep Dive into Anthropic’s Sparse Autoencoders by Hand
  Within a few short weeks, Anthropic’s “Scaling Monosemanticity” paper has attracted a lot of attention within the XAI community. Srijanie Dey, PhD presents a beginner-friendly primer for anyone interested in the researchers’ claims and goals, and in how they came up with an “innovative approach to understanding how different components in a neural network interact with one another and what role each component plays.”
• Interpretable Features in Large Language Models
  For a high-level, well-illustrated explainer on the “Scaling Monosemanticity” paper’s theoretical underpinnings, we highly recommend Jeremi Nuer’s debut TDS article—you’ll leave it with a firm grasp of the researchers’ thinking and of this work’s stakes for future model development: “as improvements plateau and it becomes more difficult to scale LLMs, it will be important to truly understand how they work if we want to make the next leap in performance.”
• The Meaning of Explainability for AI
  Taking a few helpful steps back from specific models and the technical challenges they create in their wake, Stephanie Kirmer gets “a bit philosophical” in her article about the limits of interpretability; attempts to illuminate those black-box models might never achieve full transparency, she argues, but are still important for ML researchers and developers to invest in.

• Additive Decision Trees
  In his recent work, W Brett Kennedy has been focusing on interpretable predictive models, unpacking their underlying math and showing how they work in practice. His recent deep dive on additive decision trees is a powerful and thorough introduction to such a model, showing how it aims to supplement the limited available options for interpretable classification and regression models.
• Deep Dive on Accumulated Local Effect Plots (ALEs) with Python
  To round out our selection, we’re thrilled to share Conor O’Sullivan’s hands-on exploration of accumulated local effect plots (ALEs): an older, but dependable method for providing clear interpretations even in the presence of multicollinearity in your model.

Interested in digging into some other topics this week? From quantization to Pokémon optimization strategies, we’ve got you covered!

• In a fascinating project walkthrough, Parvathy Krishnan, Joaquim Gromicho, and Kai Kaiser show how they’ve combined several geospatial datasets and some Python to optimize the process of selecting healthcare-facility locations.
• Learn how weight quantization works and how to apply it in real-world deep learning workflows — Chien Vu’s tutorial is both thorough and accessible.
• The knapsack problem is a classic optimization challenge; Maria Mouschoutzi, PhD approaches it with a fun new twist, showing how to create the most powerful Pokémon team with the aid of modeling and PuLP, a Python optimization framework.
• Squeezing the most value out of RAG systems continues to be a top priority for many ML professionals. Leonie Monigatti takes a close look at potential solutions for measuring context relevance.
• After more than a decade as a data leader at tech giants and high-growth startups, Torsten Walbaum offers the insights he’s accumulated around a fundamental question: how do we make sense of data?
• Data analysts might not often think of themselves as programmers, but there’s still a lot of room for cross-disciplinary learning—as Mariya Mansurova demonstrates in a data-focused roundup of software-engineering best practices.

Thank you for supporting the work of our authors! We love publishing articles from new authors, so if you’ve recently written an interesting project walkthrough, tutorial, or theoretical reflection on any of our core topics, don’t hesitate to share it with us.

Until the next Variable,

TDS Team

Sparse Autoencoders, Additive Decision Trees, and Other Emerging Topics in AI Interpretability was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.