Categories: Azure Data Factory

#TSQL2sday 171 – Describe the Most Recent Issue You Closed

It’s #TSQL2sday time! This month’s invitation has been sent out by Brent Ozar and he asks us to describe the most recent issue – or the last ticket – we closed.

In my case, I actually blogged about it in this blog post: Azure Function App doesn’t retrieve most recent secret value from Azure Key Vault. But this wasn’t the actual issue. The real issue was that the data warehouse from a client wasn’t being updated with recent data. The ELT (not a typo) is implemented in Azure Data Factory, so I went to take a look at the monitoring section of ADF. All runs in the past week had failed. Whoops. For some reason, the alerting emails were not being sent out if something failed. The root cause of the issue was an app registration that was being used in a PowerShell script to send an email (because ADF, strangely enough, cannot send an email) and its secret had expired. As explained in the blog post, the secret issue is resolved by configuring a new secret for the app registration in Entra ID, and then waiting for 24 hours for the Azure Function App to refresh its cache.

The original issue that caused the ELT not to refresh was – as often is the case I’ve noticed in my career – a permission issue. The ERP system thought it was a good idea to reset permissions after a software update, so suddenly the account used to extract the data had no longer permissions to the HR module.

Conclusion

One crucial part of monitoring your processes is monitoring if your monitoring actually works 🙂


------------------------------------------------
Do you like this blog post? You can thank me by buying me a beer 🙂
Koen Verbeeck

Koen Verbeeck is a Microsoft Business Intelligence consultant at AE, helping clients to get insight in their data. Koen has a comprehensive knowledge of the SQL Server BI stack, with a particular love for Integration Services. He's also a speaker at various conferences.

Recent Posts

Techorama 2024 – Slides

You can find the slidedeck for my Techorama session "Microsoft Fabric for Dummies" on github.

18 hours ago

Webinar – Microsoft Fabric for Dummies

On Wednesday May 15th 2024 I will give a free webinar on MSSQLTips.com about Microsoft…

3 days ago

Get row counts of all tables in a Microsoft Fabric warehouse

I loaded some built-in sample data from Wide World Importers into a Fabric warehouse. You…

7 days ago

dataMinds Saturday 2024 – Session Materials

It was great being at dataMinds Saturday 2024 this past weekend. A great crowd of…

1 week ago

Check your regions people

Today I was having a nice discussion with some colleagues about Fabric and pricing/licensing came…

3 weeks ago

Book Review – Deciphering Data Architectures

I recently purchased and read the book Deciphering Data Architectures - Choosing Between a Modern…

1 month ago