excelworld

[excelworld]

What a session! 🙌 A massive thank you to Miguel Angel Escobar for delivering such an insightful and practical walkthrough on Dataflows Gen2 in Microsoft Fabric during our latest meetup!

Your clarity, real-world tips, and passion for empowering data professionals were felt throughout the session. From understanding the new architecture to demystifying how everything fits in the end-to-end pipeline—it was truly eye-opening for all attendees.

🎥 Missed it or want to rewatch? Here's the full recording: 🔗 https://www.youtube.com/live/kUbJgcLNKcA?si=YkxTEFsHAedyCAgQ

We’re grateful for your time and knowledge, Miguel. Looking forward to more collaborations in the future!

[excelworld]

📬 Where Can Your Events Go in Microsoft Fabric?

Q: What types of event destinations are available in Event Stream?

✅ A: You can route your streaming data to: 🔹 KQL Database – For interactive querying and time-series analysis 🔹 Lakehouse – To store raw or transformed data for long-term analytics 🔹 Custom App – Send events to your own application via API 🔹 Reflex – Trigger real-time actions based on event logic

⚡ These destinations help turn raw streams into actionable insights, stored datasets, or automated workflows—with no-code setup.

💬 Which destination do you use most—and why? Let’s share real-time use cases!

[excelworld]

🌊 Streaming Starts Here: Spark Structured Streaming Sources

Q: What are the types of streaming sources that Spark Structured Streaming can read from?

✅ A: Spark Structured Streaming can ingest data from: 🔹 Network ports (e.g., socket streams) 🔹 Real-time message brokers (like Kafka, Azure Event Hubs) 🔹 File system locations (e.g., new files in cloud storage or local directories)

⚡ This flexibility allows you to build powerful, scalable pipelines for both real-time analytics and event-driven applications.

💬 What’s your go-to source for streaming data in Spark? Let’s exchange experiences in the comments!

[excelworld]

🎛️ Getting to Know the Event Stream Editor in Microsoft Fabric

Q: What’s the main editor in Event Stream used for?

✅ A: It’s your control center for real-time data! You can: 🔹 Establish sources (like IoT or logs) and destinations (Lakehouse, KQL, Power BI) 🔹 View data in-flight as it streams 🔹 Capture, transform, and route events using a no-code interface

⚡ Whether you're building real-time dashboards or automated responses, this editor makes streaming data pipelines visual and intuitive.

💬 What’s your first impression of the Event Stream experience in Fabric? Share your use case or feedback below!

[excelworld]

🧩 Power Query Online Tip: Diagram View

Q: What does the Diagram View in Power Query Online allow you to do?

✅ A: It gives you a visual representation of how your data sources are connected and what transformations have been applied.

🔍 Perfect for understanding query logic, debugging complex flows, and documenting your data prep process—especially in Dataflows Gen2 within Microsoft Fabric.

👀 If you're more of a visual thinker, this view is a game-changer!

💬 Have you tried Diagram View yet? What’s your experience with it?

[excelworld]

📂 Managed vs. External Tables in Microsoft Fabric

Q: What’s the difference between managed and external tables?

✅ A:

Managed tables: Both the table definition and data files are fully managed by the Spark runtime for the Fabric Lakehouse.

External tables: Only the table definition is managed, while the data itself resides in an external file storage location.

🧠 Use managed tables for simplicity and tight Fabric integration, and external tables when referencing data stored elsewhere (e.g., OneLake, ADLS).

💬 Which one do you use more in your projects—and why?

[excelworld]

🔄 Mastering Dataflows (Gen2): Transform Like a Pro

Q: What are some common data transformations in Dataflows Gen2?

✅ A: Here are some of the most used transformations:

🔹 Filter and Sort rows

🔹 Pivot and Unpivot

🔹 Merge and Append queries

🔹 Split and Conditional Split

🔹 Replace values and Remove duplicates

🔹 Add, Rename, Reorder, or Delete columns

🔹 Rank and Percentage calculators

🔹 Top N and Bottom N selections

🧠 These transformations help clean, shape, and enrich your data—making your downstream reporting more effective and insightful.

💬 Which transformation do you use the most in your projects?

Drop your favorite (or most underrated) one in the comments!

#DataPlatform #LowCodeTools

[excelworld]

🔍 Quick Fabric Insight

Q: What is the purpose of workspace roles?

A: Workspace roles are used to control access and manage the lifecycle of data and services in Microsoft Fabric.

🎯 Whether you're publishing reports, setting up pipelines, or managing Lakehouses—assigning the right role ensures smooth collaboration and secure data handling.

👥 Are you using workspace roles effectively in your Fabric projects?

💬 Comment below with how your team structures roles—or any best practices you follow!

[excelworld]

🔍 Quick Fabric Insight

Q: What is the purpose of workspace roles?

A: Workspace roles are used to control access and manage the lifecycle of data and services in Microsoft Fabric.

🎯 Whether you're publishing reports, setting up pipelines, or managing Lakehouses—assigning the right role ensures smooth collaboration and secure data handling.

👥 Are you using workspace roles effectively in your Fabric projects?

💬 Comment below with how your team structures roles—or any best practices you follow!

[excelworld]

🔐 Fabric Access Tip

💭 Q: What is the purpose of workspace roles in Microsoft Fabric?

✅ A: Workspace roles are used to control access and manage the lifecycle of data and services in Fabric.

Each role (Admin, Member, Contributor, Viewer) comes with its own permissions—knowing the difference can save your project!

🧩 Are you assigning workspace roles strategically in your Fabric environment?

💬 Share how you're using roles to keep your data ecosystem secure and collaborative.

[excelworld]

🧠 Fabric Question of the Day

📂 Have you ever explored the mysterious _delta_log folder in your Fabric Lakehouse tables?

👉 Q: What is stored in the _delta_log folder for each table?

✅ A: Transaction details are logged in JSON format inside the _delta_log folder. This is how Delta Lake tracks all changes and ensures ACID compliance.

🔁 Have you checked the _delta_log in your environment?

💬 Drop a comment if you’ve used it—or if you’ve ever wondered how version control works behind the scenes in Delta Tables!

[excelworld]

✨ Have you heard of the Medallion Architecture in modern data engineering?

It’s a layered approach to organizing data in a Lakehouse environment, helping ensure quality and scalability at every stage:

🥉 Bronze – Raw, unprocessed data 🥈 Silver – Cleaned and enriched data 🥇 Gold – Business-ready, refined data for reporting and analytics

This structure supports better data governance, performance, and reusability across the enterprise.

Do you use this approach in your projects? Let’s discuss how it’s working for you! 💬

[excelworld]

🔍 Ever wondered how to speed up repeated queries in your KQL database?

Meet Materialized Views — a powerful feature that stores the precomputed results of a query as a reusable schema entity. Instead of recalculating every time, your query pulls from a ready-made snapshot, saving both time and compute.

⚡️ Faster insights, optimized performance.

Have you used materialized views in your projects? Share your experience below! 👇

[excelworld]

📊 Why are dataflows important in end-to-end analytics? Dataflows (Gen2) play a key role by helping you: ✅ Prepare consistent data ✅ Stage it in your preferred destination ✅ Reuse it across reports ✅ Easily refresh and update it

They streamline your analytics process from raw data to actionable insights!

💬 How are you using dataflows in your projects?

[excelworld]

🚀 What makes Delta Lake so powerful in a Lakehouse architecture? Delta Lake combines the reliability and performance of relational databases with the scalability and flexibility of data lakes. It's the best of both worlds — structured data management meets open data storage.

💡 Curious how this transforms your data strategy? Let’s discuss!

👇 Drop your thoughts in the comments.

[excelworld]

🔍 What are Dataflows (Gen 2)? 💬 They are a cloud-based ETL solution designed to create and run scalable data transformation workflows.

📣 Now it’s your turn! How would you define Dataflows Gen 2 in your own words? Drop your answer below 👇

[excelworld]

Diagram View in Power Query Online lets you visually explore and manage your data transformation steps and dependencies. It's great for understanding the flow and structure of your queries. Have you tried it yet? What do you like or wish it had?