Fabric Spark: Custom Live Pools, Resource Profiles & Performance Fixes | Fabric Insider Ep.4

Is Spark slow in Microsoft Fabric — or is it your configuration choices that are slowing it down? In Episode 4 of Fabric Insider, I sit down with Santhosh Kumar Ravindran from Microsoft Fabric Data Engineering team to go deep on everything Spark performance in Microsoft Fabric. If you work with notebooks, Spark pools, or data engineering at scale in Fabric, this episode is for you. 🔥 What we cover in this episode: ▶ Introduction — Santhosh Kumar & his role in Fabric Data Engineering ▶ What is Apache Spark and why Microsoft Fabric uses it for both Data Engineering and Data Science ▶ Is Fabric Spark the same as open-source Spark? — customizations and Microsoft-specific optimizations explained ▶ Compute Strategy: Fixed vs Autoscale — what's the real difference, and is Bursting the same as Autoscale? ▶ Hybrid Compute — what it is and when to use it ▶ Governance hierarchy for Data Engineering compute: Capacity → Workspace → Environments → Session-level ▶ Custom Live Pools (NEW at FabCon 2025) — how they differ from Starter/Default Spark pools and the right use cases ▶ Resource Profiles (NEW at FabCon 2025) — what they are, how they work, and why writeHeavy & readHeavyForPBI matter ▶ Why does Spark feel slow? — the policy choices you're making that cause cold starts, missed session-sharing, and capacity underutilization ▶ Diagnostics & resolving throttling in Microsoft Fabric ▶ Max Job Lifetime — the hard ceiling you need to know about ▶ Recommended actionable rollout plan for your organization ▶ What's on the roadmap for Fabric Spark performance 💡 Whether you're a data engineer tuning Spark notebooks or an architect designing your Fabric capacity strategy, the insights in this episode will save you hours of trial and error. 🎙️ Guest: Santhosh Kumar Ravindran | Microsoft Fabric Data Engineering Team 🔗 Related Resources & Community: - Microsoft Fabric Documentation: https://learn.microsoft.com/en-us/fabric/ - RADACAD Blog — Microsoft Fabric: https://radacad.com - Fabric Ideas (submit & vote on feature requests): https://ideas.fabric.microsoft.com - Microsoft Fabric Reddit Community: https://www.reddit.com/r/MicrosoftFabric/ - Santhosh Kumar Ravindran on LinkedIn: https://www.linkedin.com/in/thisissanthoshr/ ━━━━━━━━━━━━━━━━━━━━━━━━━ 📺 Watch the full Fabric Insider Playlist: https://www.youtube.com/playlist?list=PLMXQvYI7QV6f8vUxtlFuGzQk-tYj3uOTn 🎙️ Listen on Spotify (Fabric Insider Podcast): https://open.spotify.com/show/4eJrCr5UOWZ7ggmrVUce6l?si=Jx18sgdaScCgR3P1oDz-dw 📖 Read the Fabric Insider Blog Series: https://radacad.com/category/fabric-insider-2026/ ━━━━━━━━━━━━━━━━━━━━━━━━━ 🔗 Related Resources: - Microsoft Fabric Documentation: https://learn.microsoft.com/en-us/fabric/ - RADACAD Blog — Microsoft Fabric: https://radacad.com ━━━━━━━━━━━━━━━━━━━━━━━━━ 👤 About Reza Rad: Reza Rad is an author, trainer, speaker, and consultant with 20+ years of experience in Microsoft data platform, Power BI, and Microsoft Fabric. He is the founder of RADACAD and has trained thousands of professionals globally. 🌐 Website: https://radacad.com 💼 LinkedIn: https://www.linkedin.com/in/rezarad/ ━━━━━━━━━━━━━━━━━━━━━━━━━ #MicrosoftFabric #Spark #FabricDataEngineering #ApacheSpark #FabricInsider #PowerBI #DataEngineering #MicrosoftFabricSpark #CustomLivePools #ResourceProfiles #FabCon #DataScience #CloudComputing #FabricSpark #MicrosoftData