I recently took and passed the DP-700 exam, which is required for the Microsoft Certified: Fabric Data Engineer Associate certification. It’s the second Fabric certification, and it focuses more on the data engineering aspect of Fabric, as well as a bit of governance and administration. My preparation for this exam is probably fairly atypical, because: […]
Tag: fabric
How to use a Script Activity in ADF as a Lookup
In Azure Data Factory (ADF, but also Synapse Pipelines and Fabric Pipelines), you have a typical scenario where you retrieve some metadata from a database and then loop over the metadata with a ForEach activity. For example, to copy all the table from a source database in a metadata-driven pipeline. The obvious choice would be […]
Dynamic Warehouse & Lakehouse Connections in Microsoft Fabric Data Pipelines
When you develop data pipelines in Microsoft Fabric (the Azure Data Factory equivalent in Fabric, not to be confused with deployment pipelines), you will most likely have some activities with a connection to a warehouse, a lakehouse or a KQL database (for the remainder of the blog post I’ll talk about a warehouse, but it […]
System Views in Microsoft Fabric – Query references an object that is not supported in distributed processing mode
I have a metadata-driven ELT framework that heavily relies on dynamic SQL to generate SQL statements that load data from views into a respective fact or dimension. Such a task is well suited for generation, since the pattern to load a type 1 SCD, type 2 SCD or a fact table is always the same. […]
PostgreSQL error in Fabric Pipelines – No pg_hba.conf Entry for Host
I came across a bit of a weird error when I was trying to connect to a PostGreSQL database (hosted as an Azure DB for PostGreSQL), where “xx.xx.xx.xxx” is some IP address: No pg_hba.conf entry for host “xx.xx.xx.xxx” Some context to get us started. I’m using Pipelines in Microsoft Fabric to copy data from an […]
Techorama 2024 – Slides
You can find the slidedeck for my Techorama session “Microsoft Fabric for Dummies” on github.
Check your regions people
Today I was having a nice discussion with some colleagues about Fabric and pricing/licensing came up. I mentioned an F2 is only around €250 a month, but a colleague said “no no, it’s over €300”. A bit confused, I pulled up the sizing table of my Fabric capacity in Azure, which looks like this: Similarly, […]
Fabulous Stuff in Fabric – Part 5: Folders in Workspaces
A while ago I had a little blog post series about cool stuff in Snowflake. I’m doing a similar series now, but this time for Microsoft Fabric. I’m not going to cover the basics of Fabric, hundreds of bloggers have already done that. I’m going to cover little bits & pieces that I find interesting, that […]
Speaking at dataMinds Saturday & Techorama 2024
I have the pleasure to announce I’ll be presenting at two conferences this spring. The first one is at dataMinds Saturday 2024 (back in-person!): Yes, it’s a long abstract 🙂 You can register for this free event here. The other session is at Techorama Belgium 2024 where I’ll present about Microsoft Fabric: Tickets for the […]
Change Notebook Spark Session Time-Out in Microsoft Fabric
You might know the feeling: you’re writing code in a Notebook in Microsoft Fabric and suddenly you have to leave your workstation for a while. Someone ran the doorbell (you’re working from home and you get some parcels delivered), or you took a coffee break with some colleagues. When you return to your notebook, the […]