Execute SQL Scripts on Snowflake using Azure Data Factory

Azure Data Factory has a new activity introduced this week (around the 10th of March 2022 for you future readers): the Script activity! This is not to be confused with the script task/component of SSIS, which allows you to execute .NET script (C# for most people, or VB if you’re Ben Weissman). No, this task executes SQL, so it’s more akin to the Execute SQL Task of SSIS.

This is great news, because:

  1. You previously could only execute SQL scripts through the “Stored procedure” activity. If you don’t have a stored procedure, but you rather wanted to execute a hardcoded script (or a parameterized script through dynamic content and ADF expressions), you had to wrap it in the sp_executesql stored proc (you know, like dynamic SQL). Or you could (ab)use the Lookup component to execute some scripts.
  2. But even then, the Stored proc activity only supports Microsoft relational sources (Azure SQL DB, Azure Synapse and SQL Server). For all other sources, you either had to use the Lookup component (feels dirty) or use something else like an Azure Function.

In other words, existing options in ADF were limited, especially if you were using a non-Microsoft database like Snowflake. Luckily, those days are over. Well, not entirely, because for the moment the Script activity supports the Microsoft databases (no not Access), and only Oracle and Snowflake are added to the list. I don’t know if other databases are in the pipeline.

The Script activity actually resembles the SSIS Execute SQL Task a lot. You can just specify a query, like in this example:

You have two options: Query and NonQuery (who comes up with this stuff?). NonQuery is for when you want to execute SQL scripts that don’t return a result set. DDL-style queries, such as TRUNCATE TABLE, CREATE TABLE, ALTER WAREHOUSE etc. But also INSERT, UPDATES, DELETE, MERGE, COPY INTO etc.

The Query option is for when – you already guessed it – the script is returning result sets.

That’s one big advantage over the Lookup activity: you have the ability to have more than one result set. Like the Execute SQL Script, you can also specify parameters.

For Snowflake and Oracle, you have to use question marks as placeholder (just like in SSIS).

If instead of parameterizing values, you need more dynamic SQL, you can just use dynamic content on the script itself. You can compare it as using a variable with expressions in SSIS.

Be sure to check out the official announcement, it has a great comparison between the new Script activity, and the Stored procedure and Lookup activities. If you don’t see the new Script activity yet, it’s possible it’s not available in your region yet, as updates are rolled out gradually.


------------------------------------------------
Do you like this blog post? You can thank me by buying me a beer 🙂
Koen Verbeeck

Koen Verbeeck is a Microsoft Business Intelligence consultant at AE, helping clients to get insight in their data. Koen has a comprehensive knowledge of the SQL Server BI stack, with a particular love for Integration Services. He's also a speaker at various conferences.

View Comments

  • Thank You so much..I was struggling since long time to achieve this.
    You made this so easy!!

    Thanks Again!

Recent Posts

Free webinar – Tackling the Gaps and Islands Problem with T-SQL Window Functions

I'm hosting a free webinar at MSSQLTips.com at the 19th of December 2024, 6PM UTC.…

5 days ago

dataMinds Connect 2024 – Session Materials

The slides and scripts for my session "Tackling the Gaps & Islands Problem with T-SQL…

4 weeks ago

Connect to Power BI as a Guest User in another Tenant

Sometimes your Microsoft Entra ID account (formerly known as Azure Active Directory) is added as…

2 months ago

How to use a Script Activity in ADF as a Lookup

In Azure Data Factory (ADF, but also Synapse Pipelines and Fabric Pipelines), you have a…

4 months ago

Database Build Error – Incorrect syntax near DISTINCT

I wrote a piece of SQL that had some new T-SQL syntax in it: IS…

4 months ago

Speaking at dataMinds Connect 2024

I'm very excited to announce I've been selected as a speaker for dataMinds Connect 2024,…

5 months ago