Categories: Azure Data Factory

#TSQL2sday 171 – Describe the Most Recent Issue You Closed

It’s #TSQL2sday time! This month’s invitation has been sent out by Brent Ozar and he asks us to describe the most recent issue – or the last ticket – we closed.

In my case, I actually blogged about it in this blog post: Azure Function App doesn’t retrieve most recent secret value from Azure Key Vault. But this wasn’t the actual issue. The real issue was that the data warehouse from a client wasn’t being updated with recent data. The ELT (not a typo) is implemented in Azure Data Factory, so I went to take a look at the monitoring section of ADF. All runs in the past week had failed. Whoops. For some reason, the alerting emails were not being sent out if something failed. The root cause of the issue was an app registration that was being used in a PowerShell script to send an email (because ADF, strangely enough, cannot send an email) and its secret had expired. As explained in the blog post, the secret issue is resolved by configuring a new secret for the app registration in Entra ID, and then waiting for 24 hours for the Azure Function App to refresh its cache.

The original issue that caused the ELT not to refresh was – as often is the case I’ve noticed in my career – a permission issue. The ERP system thought it was a good idea to reset permissions after a software update, so suddenly the account used to extract the data had no longer permissions to the HR module.

Conclusion

One crucial part of monitoring your processes is monitoring if your monitoring actually works 🙂


------------------------------------------------
Do you like this blog post? You can thank me by buying me a beer 🙂
Koen Verbeeck

Koen Verbeeck is a Microsoft Business Intelligence consultant at AE, helping clients to get insight in their data. Koen has a comprehensive knowledge of the SQL Server BI stack, with a particular love for Integration Services. He's also a speaker at various conferences.

Recent Posts

Book Review – Microsoft Power BI Performance Best Practices

I was asked to do a review of the book Microsoft Power BI Performance Best…

2 days ago

Create a Numbers Table in Power Query

This is a quick blog post, mainly so I have the code available if I…

7 days ago

Microsoft finally adds Tenant Switcher for Fabric / Power BI

Praise whatever deity you believe in, because it's finally here, a tenant switcher for Microsoft…

1 week ago

Book Review – Humanizing Data Strategy by Tiankai Feng

This book was making its rounds on social media, and the concept seems interesting enough…

1 week ago

Free webinar – Tackling the Gaps and Islands Problem with T-SQL Window Functions

I'm hosting a free webinar at MSSQLTips.com at the 19th of December 2024, 6PM UTC.…

3 weeks ago

dataMinds Connect 2024 – Session Materials

The slides and scripts for my session "Tackling the Gaps & Islands Problem with T-SQL…

1 month ago