I came across a bit of a weird error when I was trying to connect to a PostGreSQL database (hosted as an Azure DB for PostGreSQL), where “xx.xx.xx.xxx” is some IP address: No pg_hba.conf entry for host “xx.xx.xx.xxx” Some context to get us started. I’m using Pipelines in Microsoft Fabric to copy data from an […]
Book Review – Fundamentals of Data Engineering
This book was recommended by some of my colleagues, so I decided to give it a try. After finishing the book, I was pleased that I did. As the title might give away, it’s a book about data engineering, but not from a pure technical perspective. It won’t teach you how to write Python or […]
Planview Portfolios REST API with ADF – Retrieving Actual Data
In the introduction of this blog post series, I explained the use case: extracting data from the Planview Portfolios REST API using Azure Data Factory. Any tool that can send HTTPS requests can work with the REST API, but our focus here is ADF (which can also be Synapse Pipelines or Fabric Pipelines). In the […]
Planview Portfolios REST API with ADF – Fetching the Access Token
In the introduction of this blog post series, I explained the use case: extracting data from the Planview Portfolios RESET API using Azure Data Factory. Any tool that can send HTTPS requests can work with the REST API, but our focus here is ADF (which can also be Synapse Pipelines or Fabric Pipelines). In the […]
Planview Portfolios REST API with ADF – The OData Feed
In the introduction, I explained the setup of the use case and why we need to use the OData feed: to get the list of Project IDs, because the REST API doesn’t have an endpoint for this. Weird, right? In this part of the series, we’ll dive into the Lookup activity, which uses the OData […]
Extracting Data from the Planview Portfolios REST API using ADF – Introduction
I have a client using Planview Portfolios to manage their PMO. As part of the data warehouse, we want to extract the project data from the Planview REST API. There are other methods to get data, such as using OData (which is used in the Power BI connector), SOAP Web Services (2005 called) and a […]
Techorama 2024 – Slides
You can find the slidedeck for my Techorama session “Microsoft Fabric for Dummies” on github.
Webinar – Microsoft Fabric for Dummies
On Wednesday May 15th 2024 I will give a free webinar on MSSQLTips.com about Microsoft Fabric (at 6PM UTC).Abstract: Microsoft Fabric is the new all-encompassing data platform of Microsoft. But there are so many components, services, and features to learn, it may seem overwhelming. In this session, we step back and start from the beginning. […]
Get row counts of all tables in a Microsoft Fabric warehouse
I loaded some built-in sample data from Wide World Importers into a Fabric warehouse. You get an option to load sample data when you create a new pipeline in Fabric. It says the data is 352MB in size, but after loading the data I was curious about how many rows were actually in that sample […]
dataMinds Saturday 2024 – Session Materials
It was great being at dataMinds Saturday 2024 this past weekend. A great crowd of data enthusiasts was present to learn from a bunch of local speakers (both old and new :)). I was some interesting sessions about Fabric, Purview, finding and fixing issues in a data lake with KQL, and Benni‘s great session on […]