In the blog post Call a Fabric REST API from Azure Data Factory I explained how you can call a Fabric REST API endpoint from Azure Data Factory (or Synapse if you will). Let’s go a step further and execute a Fabric Data Pipeline from an ADF pipeline, which is a common request. A Fabric […]
Author: Koen Verbeeck
Koen Verbeeck is a Microsoft Business Intelligence consultant at AE, helping clients to get insight in their data. Koen has a comprehensive knowledge of the SQL Server BI stack, with a particular love for Integration Services. He's also a speaker at various conferences.
Azure Data Factory Pipeline Debugging Fails with BadRequest
I recently had a new pipeline fail. It was actually a copy of an old pipeline where I had made some adjustments into as part of a database migration. When triggered during an execution run, it failed saying some expression could not be parsed. When I went into the pipeline and triggered a debug, it […]
Call a Fabric REST API from Azure Data Factory
Suppose you want to call a certain Microsoft Fabric REST API endpoint from Azure Data Factory (or Synapse Pipelines). This can be done using a Web Activity, and most Fabric APIs now support service principals or managed identities. Let’s illustrate with an example. I’m going to call the REST API endpoint to create a new […]
Cool Stuff in Snowflake – Part 14: Asynchronous Execution of SQL Statements
I’m doing a little series on some of the nice features/capabilities in Snowflake (the cloud data warehouse). In each part, I’ll highlight something that I think it’s interesting enough to share. It might be some SQL function that I’d really like to be in SQL Server, it might be something else. It’s been a while since I […]
How I passed the DP-700 Exam
I recently took and passed the DP-700 exam, which is required for the Microsoft Certified: Fabric Data Engineer Associate certification. It’s the second Fabric certification, and it focuses more on the data engineering aspect of Fabric, as well as a bit of governance and administration. My preparation for this exam is probably fairly atypical, because: […]
Take over Ownership in Microsoft Fabric
When you create an item in Microsoft Fabric (a notebook, a lakehouse, a warehouse, a pipeline …), you are also the owner of the item. There has been an outstanding issue in Microsoft Fabric, where if a user’s account was deactivated, other users could no longer access the item. Whoopsies. When a consultant came in, […]
Book Review – Agile Data Warehouse Design
I recently read the book Agile Data Warehouse Design – Collaborative Dimensional Modeling, from Whiteboard to Star Schema (quite the title) by Lawrence Corr and Jim Stagnitto. The book was recommended to my years ago by a former colleague (the book is from 2011, with the latest revision from 2014) and it has been sitting […]
Cloudbrew 2024 – Slides
You can find the slides for the session Building the €100 data warehouse with the Azure Data Platform in this GitHub repo.
Book Review – Microsoft Power BI Performance Best Practices
I was asked to do a review of the book Microsoft Power BI Performance Best Practices 2nd edition (Amazon link), which means I got a copy for free. The book is written by Thomas Leblanc and Bhavik Merchant and published by Packt. As the title obviously implicates, the book aims to give you a set […]
Create a Numbers Table in Power Query
This is a quick blog post, mainly so I have the code available if I need it somewhere in the future 🙂 Sometimes you need a numbers table (also called tally table) when you’re working with data. It’s a table with one single integer column that contains incremental numbers (0,1,2,3,4,…). In a database, it’s fairly […]