Call a Fabric REST API from Azure Data Factory

Suppose you want to call a certain Microsoft Fabric REST API endpoint from Azure Data Factory (or Synapse Pipelines). This can be done using a Web Activity, and most Fabric APIs now support service principals or managed identities. Let’s illustrate with an example. I’m going to call the REST API endpoint to create a new lakehouse. First, create a system-managed identity for Azure Data Factory in the Azure Portal:

Next, add this managed identity to a security group in Azure Entra ID.

This is important, as we cannot directly give permissions to a service principal (or managed identity) for the Fabric REST APIs in the Fabric Admin portal; this needs to be a security group.

Once the security group can access the APIs, we can add the managed identity to a Fabric workspace:

To create a lakehouse, the managed identity needs to be at least a contributor in the workspace, as specified in the documentation. Depending on the specific REST API endpoint, different permissions might be needed. For example, listing existing objects typically only requires viewer permissions.

Now we can add a Web Activity to an ADF pipeline. For the URL, you need to use https://api.fabric.microsoft.com/v1/workspaces/{workspaceId}/lakehouses, with workspaceId as the ID of the workspace you want to create the lakehouse in. You can find this ID in the URL when you’re inside the workspace in the Fabric service:

I use the following expression, with a global parameter containing the ID of the workspace:

@concat('https://api.fabric.microsoft.com/v1/workspaces/',pipeline().globalParameters.fabric_workspace,'/lakehouses')

Unfortunately, at the time of writing we cannot specify a specific folder so the lakehouse will be created in the root folder of the workspace. For the body, the following expression can be specified:

@json(concat('{
  "displayName": "my_lakehouse",
  "description": "This is my first lakehouse",
  "creationPayload": {
    "enableSchemas": true
  }',''))

I added the concat function in case some parameterization is needed. As authentication, the managed identity is chosen and the method is set to POST. For the resource URL, you can specify https://api.fabric.microsoft.com (I found this through trial&error, this isn’t mentioned in the documentation). The following header is added:

If everything is set correctly, the pipeline can be executed and the REST API call should be successful.


------------------------------------------------
Do you like this blog post? You can thank me by buying me a beer 🙂
Koen Verbeeck

Koen Verbeeck is a Microsoft Business Intelligence consultant at AE, helping clients to get insight in their data. Koen has a comprehensive knowledge of the SQL Server BI stack, with a particular love for Integration Services. He's also a speaker at various conferences.

Share
Published by
Koen Verbeeck

Recent Posts

Cool Stuff in Snowflake – Part 14: Asynchronous Execution of SQL Statements

I’m doing a little series on some of the nice features/capabilities in Snowflake (the cloud data warehouse).…

2 weeks ago

How I passed the DP-700 Exam

I recently took and passed the DP-700 exam, which is required for the Microsoft Certified:…

4 weeks ago

Take over Ownership in Microsoft Fabric

When you create an item in Microsoft Fabric (a notebook, a lakehouse, a warehouse, a…

2 months ago

Book Review – Agile Data Warehouse Design

I recently read the book Agile Data Warehouse Design - Collaborative Dimensional Modeling, from Whiteboard…

4 months ago

Cloudbrew 2024 – Slides

You can find the slides for the session Building the €100 data warehouse with the…

4 months ago

Book Review – Microsoft Power BI Performance Best Practices

I was asked to do a review of the book Microsoft Power BI Performance Best…

4 months ago