Recently I had a piece of JSON of which I needed to extract data from to put into a table in Azure SQL DB. Unfortunately, the structure of the JSON itself was too complex to map automatically in Azure Data Factory. It might be possible though, but after a few tries to map it manually […]
Author: Koen Verbeeck
Koen Verbeeck is a Microsoft Business Intelligence consultant at AE, helping clients to get insight in their data. Koen has a comprehensive knowledge of the SQL Server BI stack, with a particular love for Integration Services. He's also a speaker at various conferences.
Book Review – Mastering Snowflake Solutions: Supporting Analytics and Data Sharing
Disclaimer: Apress gave me a free copy of the book to review. People that know me, also know I always give honest feedback 🙂 I had the pleasure of reading the book Mastering Snowflake Solutions: Supporting Analytics and Data Sharing. The book is aimed at data professionals who have already taken their first steps developing […]
Execute SQL Scripts on Snowflake using Azure Data Factory
Azure Data Factory has a new activity introduced this week (around the 10th of March 2022 for you future readers): the Script activity! This is not to be confused with the script task/component of SSIS, which allows you to execute .NET script (C# for most people, or VB if you’re Ben Weissman). No, this task […]
Extracting Data from the AFAS REST API with Azure Data Factory
AFAS is a Dutch business software company, proving companies with ERP and HR software-as-a-service. You can extract data from the service using a REST API (there are also SOAP web services available for those who appreciate the trip down memory lane). In this blog post I’ll show you how you can get data out of […]
Dynamically Map JSON to SQL in Azure Data Factory
We’re reading in some JSON files in Azure Data Factory (ADF), for example for a REST API. We’re storing the data in a relational table (SQL Server, Azure SQL DB…). The data volume is low, so we’re going to use a Copy Data activity in a pipeline, rather than a mapping data flow (or whatever […]
T-SQL Linting in an Azure Devops Build Pipeline
Linting is a process where a code analyzer is run to find programmatic errors, bugs, “stylistic constructs” or anti-patterns. For example, in Python there are specific modules that help you with linting when the code is build. If the code is not indented correctly (or whatever rule your code has to obey in Python), the […]
DirectQuery to Power BI Datasets
A new feature has been added to the Power BI Desktop toolset and it’s a very nice one. The official name is “DirectQuery for PBI datasets and AS”. It allows you to create a connection to an existing model (either Analysis Services or a Power BI dataset) and link tables from that model into your […]
Speaking at Techorama 2022
I’ve received some good news: Techorama is back with an in-person conference! I’m selected as a speaker and I’ll be talking about Azure Data Factory. Azure Data Factory (ADF) is the tool in the Azure cloud for creating your data pipelines. It allows you to move and transform data from a multitude of sources to […]
Convert Local Datetime to UTC (pre-SQL Server 2016)
Recently I was working with a client who was still on SQL Server 2012. Support issues aside, there are some very useful T-SQL functions/clauses who didn’t exist in 2012, but were introduced in a later version. One of them is AT TIME ZONE, which you can use the convert dates between time zones, as demonstrated […]
Speaking at SQLBits 2022
I’m delighted to announce I’ll be speaking at the SQLBits conference in March 2022! I’m not delivering one session, but two this time. Well, actually, for one session I’m co-presenting with Mala Mahadevan. I’m really looking forward to this session, as it’s a not-well known topic (I hadn’t heard about it till Mala told me […]