The monthly blog party is back and we’ve reached the mythical number 100. The host of this month is the Creator himself: Adam Machanic (blog | twitter). The topic is the future: what will the world be like when it’s T-SQL Tuesday #200?
The very first T-SQL Tuesday was over eight years ago, which was also around the time I started working in business intelligence (no apparent correlation though). When I look back, there are some changes, sure, but a lot has stayed the same as well. Some of the changes are:
- people are getting tired of manually creating SSIS packages. They are either looking for automation (like Biml) or like replacements that work smarter. Power Query is a good example and there are other tools as well that allow you to wrangle data easier and with much less code. The assistance of machine learning to aid you in ETL tasks will be an interesting evolution (such as the column by example in Power BI).
- there has been a more general interest in machine learning and “advanced analytics”. This has also been fueled by hypes like big data, IOT and A.I. As Adam mentioned in his invitation, most of the time it’s existing technologies in a new marketing package. However, due to much more computing power being available to us, we are able to put more theory into practice, with less effort and less cost.
- people are slowly realizing the value of proper data visualization (I cringe when I think about those reports created back in the day with SSRS 2005).
- self-service and mobile BI are taking off (although in some cases it still seems to be driven by IT or BI pros and not really the business users themselves).
But a lot has stayed the same. My guess is that in another decade, these will still be the same:
- data needs to be protected. Even more so with regulations like the GDPR.
- in order to do proper analysis of data, a decent model is needed. Self-service BI can be awesome, but without proper models you’re going to drive into a wall.
- we’re still going to be building data warehouses (but maybe they’ll be virtualized instead) and traditional reporting systems. “Yeah, it’s totally cool what your machine learning model can do and how it will predict everything our customers will need, but the CFO still needs that financial report on last quarter”.
- business users will still export everything to Excel
You get the point 🙂 Every company will always have a need for proper curated data to report and analyse on (and most likely it will serve as an input for the advanced analytics systems. They don’t train themselves ;). What will change though:
- for some tasks, less coding will be required. If Power Query (and the other Power tools) continues to evolve at the same rate, writing ETL and creating data models will become a breeze. Even today it’s remarkable how much you can achieve in those tools without writing a single line of code. I think our focus will shift more to creating easy-to-understand data models that integrate seamlessly into other systems.
- Mo’ cloud, mo’ data.
Okay, enough rambling. I think the future for data professionals is bright and I hope that we can write about all the interesting evolutions we had in T-SQL Tuesday #200.