Categories: SSIS

An update on AutoAdjustBufferSize in SSIS 2016

This week SQL Server 2016 Cumulative Update 1 was released and it contains a lot of updates/fixes for SSIS, SSAS, SSRS and MDS. If you are already working on SQL Server 2016, I’d urge to install this update as soon as possible.

Anyway, one of the particular updates that caught my attention was Adds the adjusted buffer size to the BufferSizeTuning event when AutoAdjustBufferSize is enabled in SSIS 2016. In simple terms, it allows you to log  the size of the data flow buffer set automatically by the AutoAdjustBufferSize property. This property basically automatically calculates the buffer size needed to reach the amount of rows in the buffer specified by DefaultBufferMaxRows. Therefore, it ignores  the DefaultBufferSize property. Unfortunately, this new property is set to false by default.

Back on topic: since the size of the buffer is calculated automatically for you, it might be interesting to know its final size, especially when doing some performance tuning. There are two options to get the log information:

  • Run the package using the Verbose logging level.
  • Add a custom log provider to the package and log the new custom BufferSizeTuning event of the data flow.

Let’s take a look at the first one. When you execute a package in the SSIS Catalog, certain information is logged automatically to the Catalog. Which information is decided by the logging level. In order to view the size of the data flow buffer, you need to use the Verbose logging level. Keep in mind that this logging level should be used for debugging only, since it generates quite some logging overhead! I ran a very simple package and it already resulted in over 6400 rows of logging. When you take a look at the All Messages drillthrough of the All Executions report in the catalog, you can find the necessary information under the message type NonDiagnostic.

In the screenshot above, you can see that the buffer was adjusted to 50MB.

The other method is to enable logging inside the SSIS package itself. The data flow has a new custom event, called BufferSizeTuning:

When we run the package, the following message is logged:

The buffer size of buffer type 0 is automatically adjusted to 50160000 bytes.

Now, when we change the DefaultBufferMaxRows property from its default value of 10,000 to 20,0000, we can see the following message being logged:

The buffer size of buffer type 0 is automatically adjusted to 100320000 bytes.

In other words, the buffer sized doubled to 100MB to incorporate the increase in the number of rows per buffer. By the way, the package ran only a few seconds faster although more memory was used.

Conclusion: a very nice trick to the tool belt for some SSIS performance tuning. Don’t use the Verbose logging level too much though.


------------------------------------------------
Do you like this blog post? You can thank me by buying me a beer 🙂
Koen Verbeeck

Koen Verbeeck is a Microsoft Business Intelligence consultant at AE, helping clients to get insight in their data. Koen has a comprehensive knowledge of the SQL Server BI stack, with a particular love for Integration Services. He's also a speaker at various conferences.

Recent Posts

Free webinar – Tackling the Gaps and Islands Problem with T-SQL Window Functions

I'm hosting a free webinar at MSSQLTips.com at the 19th of December 2024, 6PM UTC.…

5 days ago

dataMinds Connect 2024 – Session Materials

The slides and scripts for my session "Tackling the Gaps & Islands Problem with T-SQL…

4 weeks ago

Connect to Power BI as a Guest User in another Tenant

Sometimes your Microsoft Entra ID account (formerly known as Azure Active Directory) is added as…

2 months ago

How to use a Script Activity in ADF as a Lookup

In Azure Data Factory (ADF, but also Synapse Pipelines and Fabric Pipelines), you have a…

4 months ago

Database Build Error – Incorrect syntax near DISTINCT

I wrote a piece of SQL that had some new T-SQL syntax in it: IS…

4 months ago

Speaking at dataMinds Connect 2024

I'm very excited to announce I've been selected as a speaker for dataMinds Connect 2024,…

5 months ago