Categories: Azure Data Factory

#TSQL2sday 171 – Describe the Most Recent Issue You Closed

It’s #TSQL2sday time! This month’s invitation has been sent out by Brent Ozar and he asks us to describe the most recent issue – or the last ticket – we closed.

In my case, I actually blogged about it in this blog post: Azure Function App doesn’t retrieve most recent secret value from Azure Key Vault. But this wasn’t the actual issue. The real issue was that the data warehouse from a client wasn’t being updated with recent data. The ELT (not a typo) is implemented in Azure Data Factory, so I went to take a look at the monitoring section of ADF. All runs in the past week had failed. Whoops. For some reason, the alerting emails were not being sent out if something failed. The root cause of the issue was an app registration that was being used in a PowerShell script to send an email (because ADF, strangely enough, cannot send an email) and its secret had expired. As explained in the blog post, the secret issue is resolved by configuring a new secret for the app registration in Entra ID, and then waiting for 24 hours for the Azure Function App to refresh its cache.

The original issue that caused the ELT not to refresh was – as often is the case I’ve noticed in my career – a permission issue. The ERP system thought it was a good idea to reset permissions after a software update, so suddenly the account used to extract the data had no longer permissions to the HR module.

Conclusion

One crucial part of monitoring your processes is monitoring if your monitoring actually works 🙂


------------------------------------------------
Do you like this blog post? You can thank me by buying me a beer 🙂
Koen Verbeeck

Koen Verbeeck is a Microsoft Business Intelligence consultant at AE, helping clients to get insight in their data. Koen has a comprehensive knowledge of the SQL Server BI stack, with a particular love for Integration Services. He's also a speaker at various conferences.

Recent Posts

Free webinar – Tackling the Gaps and Islands Problem with T-SQL Window Functions

I'm hosting a free webinar at MSSQLTips.com at the 19th of December 2024, 6PM UTC.…

5 days ago

dataMinds Connect 2024 – Session Materials

The slides and scripts for my session "Tackling the Gaps & Islands Problem with T-SQL…

4 weeks ago

Connect to Power BI as a Guest User in another Tenant

Sometimes your Microsoft Entra ID account (formerly known as Azure Active Directory) is added as…

2 months ago

How to use a Script Activity in ADF as a Lookup

In Azure Data Factory (ADF, but also Synapse Pipelines and Fabric Pipelines), you have a…

4 months ago

Database Build Error – Incorrect syntax near DISTINCT

I wrote a piece of SQL that had some new T-SQL syntax in it: IS…

4 months ago

Speaking at dataMinds Connect 2024

I'm very excited to announce I've been selected as a speaker for dataMinds Connect 2024,…

5 months ago