Accelerated Data‑Engineering & Visualization Learning Plan
A top‑down, link‑rich roadmap to get you ready for Databricks + Azure data pipeline and Power BI dashboard in just two weeks.
Week 1 — Data‑Engineering Core
| Day | Goal | What to Do | Instant Resources |
|---|
| 1 (AM) | Understand the Lakehouse | Skim the Medallion (Bronze‑Silver‑Gold) diagram & watch a 10‑min overview video. | Medallion Architecture — Databricks |
| 1 (PM) | Spin‑up Databricks | Open a free Community workspace; install and run the retail CDC pipeline demo (dbdemos.install("lakehouse-retail-c360")). | dbdemos repo |
| 2 | PySpark 101 | Work through the first Databricks Quickstart notebook (load → transform → Delta). Keep the PySpark cheat‑sheet open. | Databricks Quickstart |
| 3 | Azure plumbing | Create a free Azure subscription; complete the Copy Data with Azure Data Factory quickstart (CSV → Data Lake). | ADF Copy‑Data Quickstart |
| 4 | Glue ADF ⇄ Databricks | In ADF, add a Databricks notebook activity that triggers yesterday’s pipeline; verify tables in Azure Synapse. | Retail ADF Pipeline Example |
| 5 – 6 | Mini project | Build a micro‑pipeline: ingest sample retail CSV → Lake → clean with PySpark → write Delta Gold → query with Databricks SQL → expose to Power BI. | Retail Lakehouse Demo |
| 7 | Checkpoint & certify | Sit the free Databricks Lakehouse Fundamentals accreditation (≈3 hrs). | Lakehouse Fundamentals Accreditation |
Week 2 — Visualization & Storytelling
| Day | Goal | What to Do | Instant Resources |
|---|
| 8 | Power BI crash course | Install Power BI Desktop; connect to Azure Databricks with Partner Connect. | Connect Power BI ⇄ Databricks |
| 9 | First retail dashboard | Clone a ready‑made Retail Power BI repo, swap in your Gold table, rebuild visuals (sales vs store, YoY growth, stockouts). | Retail Dashboard Repo |
| 10 | DAX & polish | Add DAX measures (Total Sales, % Margin, 12‑Mo Rolling Avg); apply Chanel‑style theme (muted palette, luxury font). | Sales Dashboard w/ DAX |
| 11 | Real‑time flair (optional) | Read the Databricks blog on streaming → Power BI; test with a tiny Kafka simulator. | Real‑time Streaming to Power BI |
| 12 | Dry‑run demo | Craft a 5‑slide deck (Problem, Architecture, Pipeline, Dashboard live tour, ROI). | — |
| 13 – 14 | Cert prep & wrap | Skim study guides: PL‑300 (Power BI Data Analyst) & DP‑203 (Azure Data Engineer). | PL‑300 · DP‑203 |
Cheat‑Codes (Top‑Down Shortcuts)
| Need it fast? | Shortcut |
|---|
| Architecture slides | Re‑label the Lakehouse Build a Simple Pipeline PDF from Databricks. |
| Code snippets | Re‑use cells from the Retail_DLT_CDC_Python notebook inside dbdemos. |
| Dashboard inspiration | Browse screenshots in the Power BI retail repos for layout ideas. |
| Video refreshers | YouTube playlist: Master Databricks & Spark |
Resume‑Ready Badges (Next 30 Days)
| Badge / Cert | Why it Helps | Effort |
|---|
| Databricks Lakehouse Fundamentals | Shows Lakehouse basics. | 3–4 hrs |
| Microsoft PL‑300 (Power BI) | Validates dashboard & DAX chops. | 1 week study |
| Microsoft DP‑203 (Azure DE) | Deep Azure pipeline mastery. | 3–4 weeks |
When You Have More Runway
- Automation / MLOps — explore Databricks Workflows & Delta Live Tables for scheduled refreshes.
- dbt on Databricks — clone the dbt‑Databricks C360 demo for modular SQL models.
- Advanced visuals — try custom visuals like Deneb or Charticulator inside Power BI for high‑fashion storytelling.
Pro tip: keep a single Notion page with these links and one‑sentence takeaways; check items off as you go. Two weeks from now you’ll have a live Lakehouse pipeline and a polished dashboard ready to impress Chanel.