Categories: SSIS

The Azure-SSIS IR Startup Time

UPDATE: here’s an official blog post going in more technical detail on how they achieved the improvements in startup time

If you want to run SSIS packages in Azure Data Factory, you need the Azure SSIS Integration Runtime (quite the mouthful), which is basically a cluster of virtual machines handling the packages like an SSIS scale-out cluster. You can read more about it in the article Configure an Azure SQL Server Integration Services Integration Runtime.

Previously, it took about 20-30 minutes to start the runtime, which was less than ideal. If you wanted to run your ETL multiple times a day, you’d lose quite some time, unless you keep the runtime running the entire time, which costs money.

Luckily, the team behind the IR made some changes and the runtime now starts in about 4-5 minutes. Quite the improvement! Now it’s easier to have multiple batches in a day and still save money. Normally you don’t have to do anything, the change is automatic, but I did recently upgrade the virtual machines of the runtime to a newer version:

v3 baby!

The startup time depends on the size of the cluster and on any custom setup you have configured.

yep, that’s fast

------------------------------------------------
Do you like this blog post? You can thank me by buying me a beer 🙂

Koen Verbeeck

Koen Verbeeck is a Microsoft Business Intelligence consultant at AE, helping clients to get insight in their data. Koen has a comprehensive knowledge of the SQL Server BI stack, with a particular love for Integration Services. He's also a speaker at various conferences.

Next What's in the pipeline for Power BI? »

Previous « Cool Stuff in Snowflake – Part 9: ILIKE

View Comments

Chris says:

May 22, 2020 at 9:04 am

I was really excited to hear this but it must be pointed out (info in the provided link to Microsoft's blog) which explains the benefits can only be seen on those SSIS-IR provisioned outside a VNet. Unfortunately the start up time for those inside a VNet will remain at 20-30 mins.
- Koen Verbeeck says:
  
  May 25, 2020 at 8:08 am
  
  It probably has to do something with the pool of VMs they have standing idle. Much harder to do for a VNet.