Spotify R&D

Don't miss Kristian Lindwall, Vijay Edupuganti, and new addition, Vaibhav Srivastav of OpenAI as they discuss what it actually looks like to build and validate product when AI is writing the code. The livestream starts shortly (2pm ET / 7pm BST)! Watch it live 👇 https://lnkd.in/gEFRxxGE

Your best data expert can't be everywhere at once. At Spotify, thousands of teams are moving fast — and the demand for data insights had quietly outpaced what any individual could handle. So we built Vedder. Ask a question in plain English, get a reliable answer in seconds. 2,100+ Spotifiers have used it since August 2025 — more than a quarter of them had never written SQL. The interesting part isn't the model. It's what sits behind it. Dumping schemas into an LLM wasn't working at Spotify's scale, so we built a context layer owned by the people who actually understand the data. When we tested auto-generating examples from query history, domain experts accepted only 12.5% of them. Most of it was noise. The signal doesn't label itself. Scaling expertise doesn't mean replacing it. It means giving the people who know the most more leverage. Check out the full writeup from Pavlina Mitsou and Jonathan Warburton👇 https://hubs.li/Q04lwyJ50 #SpotifyEngineering #AIEngineering #DataPlatform

🎧 Hear what happens when a venerable institution like the BBC embraces a culture of experimentation in order to inform smarter product decisions. 💡 Want to hear more from the Confidence, by Spotify team? Join the upcoming livestream on June 16 to learn how the best teams using AI to build product. 👉 https://lnkd.in/enzpmUKb

Confidence, by Spotify is going live. 🎙️ On June 16, Kristian Lindwall (Spotify) and Vijay Edupuganti (Sierra) sit down for a conversation on what it actually looks like to build and ship in the AI era — moderated by Operators & Friends. Watch the free livestream 👉 https://lnkd.in/enzpmUKb #Experimentation #AIEngineering #SpotifyEngineering

The geese are unleashed! 2.5 million automated PRs merged and counting.🚀 In his talk last month at “Code with Claude 2026” in London, Spotify's Chief Architect & VP of Engineering Niklas Gustavsson shared how we’ve been using Fleetshift to automate simple code changes across our fleet of components for years. Now, with our background coding agent 🪿Honk powering these shifts, making even complex code changes across hundreds or even thousands of repos at once has become routine. How routine? In the attached video, you can see Goose Farm: our internal real-time dashboard that shows current activity in Spotify’s Fleet Management system. Each goose represents an active background coding session. As silly as the scene looks, it’s a real picture of effective agents at work. As Niklas says in his talk, “What used to be hundreds of teams doing migrations for their components, taking weeks and weeks or months, now can be done by a single engineer in a few days.” Learn more about how we’re scaling developer experience for more effective teams and agents. Link to Niklas’ full talk in the comments👇 🪿🪿🪿👩💻🪿🪿🪿👨💻🪿🪿🪿 #Backstage #DeveloperExperience #AI #ClaudeCode #CodeWithClaude #AgenticFirst #FleetFirst #PlatformEngineering #Honk

We're excited to announce a new experimentation miniseries on the NerdOut@Spotify podcast, hosted by two of our finest experimentation nerds 🚀 Luke Frake and Mårten Schultzberg. For the first episode, they sat down with Bhavic Patel, Data Director at Huel, to talk about building experimentation culture at a company where not every question can be answered with a traditional A/B test. Huel sells direct-to-consumer, on Amazon, in supermarkets, and on TikTok Shop — so measurement means synthetic control groups, geo-lift tests, difference-in-difference analysis, and a lot of cross-platform data stitching. They also get into why experiment relevancy matters more than velocity, how AI is accelerating experiment design, and what it takes to make experimentation everyone's job, not just the data team's. Whether you're scaling an experimentation program or just getting started, this one's worth a listen. 🎧 Listen now: https://hubs.li/Q04jbslh0 🧪🎧🧪🎧

📣🎧 New on NerdOut@Spotify: we're launching a special miniseries on product experimentation. Now that you can build almost anything faster than ever, what matters most is learning what to build, what to keep, and what to put back on the shelf. Over the next few episodes, we go beyond A/B testing basics to explore how teams build experimentation culture, move faster with data, and make smarter product bets in practice. Hosting this series are two of Spotify's finest experimentation nerds: 🚀 Luke Frake and Mårten Schultzberg. In this first episode, they’re joined by Bhavik Patel, Data Director at Huel, to talk about how the UK-founded nutrition brand approaches experimentation across a rapidly growing omnichannel business. Unlike pure digital products, Huel sells direct-to-consumer on its own site, on Amazon, in supermarkets, and on TikTok Shop — which means you can't always rely on a classical A/B test to measure what's working. Bhav shares how his lean team rebuilt Huel's experimentation program from the ground up — including how they use synthetic controls and geo-lift tests to navigate the messy reality of measuring across channels, why relevancy matters more than experiment velocity, how AI is fast-tracking experiment design, and more. If you care about building better products with data, check it out — and stay tuned for more episodes in this series dropping soon. 🎧 Listen now: https://hubs.li/Q04jbvq_0 🧪🎧🧪🎧

Teams often treat LLM evals and A/B tests as alternatives where they might pick one and skip the other. But evals and experiments measure different things. Evals verify, does the output meet quality standards? Experiments validate, do real users respond the way you predicted? The right relationship is a funnel, not a fork. A strong eval stack means you don't run an experiment to find out if a change does what you intended, evals already told you that. You run the experiment to validate that the intended change drives the business outcome it was meant to, and to catch regressions in the metrics you weren't optimizing for. At Spotify ~42% of launched experiments are rolled back due to unexpected regressions in secondary metrics. No eval caught those. That's exactly the job experiments exist to do. Matilda Ankargren and Mårten Schultzberg just published a piece on how we think about combining LLM evals with online experimentation, and how the two make each other smarter over time. https://hubs.li/Q04h2qhq0

Coding is no longer the constraint. At Spotify 96% of developers are using AI-assisted coding — and 70% of PRs are now AI-assisted. But scaling engineering in the age of agents isn’t just about tool adoption and faster code generation. It’s about improving developer experience, establishing stronger standards, and building systems for both humans and agents. Tomorrow at Code with Claude in London, Chief Architect & VP of Engineering Niklas Gustavsson will share how we’re approaching agentic-first development at Spotify — from using Honk and our Fleet Management tooling to orchestrate large-scale code changes, to using Backstage as a shared source of truth for both humans and agents. A preview of what Niklas will dive into: 🪿 How Honk, our Claude Code–powered agent, ships 1,000+ PRs to production every month 🚀 How Fleetshift has driven 2.5M+ merged PRs across our software fleet — saving 90% of engineering time on complex migrations 🧭 How Backstage gives humans and agents shared, queryable context across every component, from ownership and lifecycle to security and operational metadata If you're a platform leader, engineering manager, or devex practitioner thinking about how to scale engineering in an agentic world, this one's for you — and if you couldn’t make it to London, you can still sign up to attend virtually: https://hubs.li/Q04h0Jx80 “Coding is no longer the constraint: Scaling devex to teams and agents at Spotify” 📍Code with Claude, London, Main Stage 🗓️ Tues, May 19 🕚 11:15am–11:45am (BST) #Backstage #DeveloperExperience #AI #ClaudeCode #CodeWithClaude #AgenticFirst #FleetFirst #PlatformEngineering

ICYMI, Spotify’s developer experience team joined The New Stack to discuss what it’s really like to adopt agentic-first development at scale — sharing their firsthand experiences and lessons from inside Spotify. 📺 Watch the full conversation here: https://hubs.li/Q04gpnmD0 You’ll hear how we’re handling the increase in PR reviews, the role Backstage and Fleet Management play in scaling developer experience to both our teams and agents, and lots more in this wide-ranging conversation. Thanks to everyone who joined live and asked great questions! Hosted by Jennifer Riggins for The New Stack With Spotify’s Sanjana Seetharam + Stefan Särne 🤖🚀