Yet another book review! I just finished the book Pro Serverless Data Handling with Microsoft Azure: Architecting ETL and Data-Driven Applications in the Cloud (what a title :), written by German MVPs (and metalheads) Benjamin Kettner and Frank Geisler. The book handles the topic of “serverless” computing in the Microsoft Data Platform, where serverless is […]
Book Review – Snowflake Security
When I was working with Snowflake on a project, it seemed security (assigning roles to users, assigning permissions etc.) is not as straight forward as it is in SQL Server. That’s why I bought the book Snowflake Security – Securing Your Snowflake Data Cloud, written by Ben Herzberg and Yoav Cohen, and published by Apress. […]
Book Review – Azure Data Factory by Example
I just finished the book Azure Data Factory by Example: Practical Implementation for Data Engineers by Richard Swinbank. The book was not exactly what I was expecting, but that’s actually my fault and not the book’s. The book itself is great and is a great introduction to Azure Data Factory (ADF) and it covers almost […]
Parsing JSON in SQL Server with OPENJSON
Recently I had a piece of JSON of which I needed to extract data from to put into a table in Azure SQL DB. Unfortunately, the structure of the JSON itself was too complex to map automatically in Azure Data Factory. It might be possible though, but after a few tries to map it manually […]
Book Review – Mastering Snowflake Solutions: Supporting Analytics and Data Sharing
Disclaimer: Apress gave me a free copy of the book to review. People that know me, also know I always give honest feedback 🙂 I had the pleasure of reading the book Mastering Snowflake Solutions: Supporting Analytics and Data Sharing. The book is aimed at data professionals who have already taken their first steps developing […]
Execute SQL Scripts on Snowflake using Azure Data Factory
Azure Data Factory has a new activity introduced this week (around the 10th of March 2022 for you future readers): the Script activity! This is not to be confused with the script task/component of SSIS, which allows you to execute .NET script (C# for most people, or VB if you’re Ben Weissman). No, this task […]
Extracting Data from the AFAS REST API with Azure Data Factory
AFAS is a Dutch business software company, proving companies with ERP and HR software-as-a-service. You can extract data from the service using a REST API (there are also SOAP web services available for those who appreciate the trip down memory lane). In this blog post I’ll show you how you can get data out of […]
Dynamically Map JSON to SQL in Azure Data Factory
We’re reading in some JSON files in Azure Data Factory (ADF), for example for a REST API. We’re storing the data in a relational table (SQL Server, Azure SQL DB…). The data volume is low, so we’re going to use a Copy Data activity in a pipeline, rather than a mapping data flow (or whatever […]
T-SQL Linting in an Azure Devops Build Pipeline
Linting is a process where a code analyzer is run to find programmatic errors, bugs, “stylistic constructs” or anti-patterns. For example, in Python there are specific modules that help you with linting when the code is build. If the code is not indented correctly (or whatever rule your code has to obey in Python), the […]
DirectQuery to Power BI Datasets
A new feature has been added to the Power BI Desktop toolset and it’s a very nice one. The official name is “DirectQuery for PBI datasets and AS”. It allows you to create a connection to an existing model (either Analysis Services or a Power BI dataset) and link tables from that model into your […]