End to End Data Engineering Project using Databricks Free Edition | FMCG Domain
Nov 28, 2025•Channel
AI Analysis
Data from YouTube Data API v3•Updated Just now
Video Overview
Video Details
Published6 months ago
DurationN/A
Video IDU6ZUKWdfSLY
Languageen
CategoryEducation
PrivacyPublic
Made for KidsNo
Video TypeRegular Video
Performance Metrics
Views678
Likes89
Comments5
Engagement Rate13.86%
Likes per 100 views13.13
Comments per 1K views7.37
Video Tags
Description
This is an end end to data engineering project using databricks free edition that is perfect for beginners as well as advanced users. In FMCG domain, we are building an ETL pipeline for a real life industry use case where a big retail company acquires a small one and we will build the data pipeline to consolidate data from both the companies in a single lakehouse architecture.
The tech stack data engineering project is: Python, SQL, Amazon S3, Medallion architecture, Spark, BI dashboard and Genie.
Signup to Databricks Free Edition: https://bit.ly/4nK0NTN
Databricks foundations video: https://www.youtube.com/watch?v=761SQ9Hxbic
Code and other resources: https://codebasics.io/resources/data-engineering-project-using-databricks-free-edition
Get Certified by Codebasics with this Databricks Mini Course - https://codebasics.io/courses/Databricks-Mini-Course
⭐️ Timestamps ⭐️
00:00:57 Project Introduction
00:02:57 Stakeholder Discussion
00:04:22 2 Learning Paths
00:04:40 Tech Architecture
00:09:47 Data Model
00:16:42 Signup to Databricks free edition
00:17:26 Setup Catalog. Data Import
00:27:36 AWS Account Setup
00:30:48 Upload Data to S3
00:32:50 Databricks S3 Connection
00:34:34 Dimension Data Processing
01:40:23 Fact Processing - Historical Load
02:02:11 Fact Processing - Incremental Load
02:25:51 Orchestration
02:38:07 Denormalized Table and Genie
02:43:24 Dashboarding
02:54:46 Stakeholder Review Meeting
02:55:55 Conclusion
Do you want to learn technology from me? Check https://codebasics.io/?utm_source=description&utm_medium=yt&utm_campaign=description&utm_id=description for my affordable video courses.
Need help building software or data analytics/AI solutions? My company https://www.atliq.com/ can help. Click on the Contact button on that website.
🎥 Codebasics Hindi channel: https://www.youtube.com/channel/UCTmFBhuhMibVoSfYom1uXEg
#️⃣ Social Media #️⃣
🧑🤝🧑 Discord for Community Support: https://discord.gg/r42Kbuk
📸 Codebasics' Instagram: https://www.instagram.com/codebasicshub/
📝 Codebasics' Linkedin : https://www.linkedin.com/company/codebasics/
------
📝 Dhaval's Linkedin : https://www.linkedin.com/in/dhavalsays/
📝 Hem's Linkedin: https://www.linkedin.com/in/hemvad/
📽️ Hem's Instagram for daily tips: https://www.instagram.com/hemvadivel/
📸 Dhaval's Personal Instagram: https://www.instagram.com/dhavalsays/
🔗 Patreon: https://www.patreon.com/codebasics?fan_landing=true