Skip to main content

Extending Microsoft Management Data Warehouse

Working with Management Data Warehouse, a reporting tool for Data Warehouses introduced in SQL Server 2008 and never changed since, makes a lot of limitations visible and also the lack of in-depth documentation by Microsoft. They tried to make the documentation better accessible in SQL Server 2012 but the product itself did not change. 

What are the main limitations?

MDW uses snapshots, which is actually a 'easy' way to handle all the issues involved in collecting ongoing activity. It is like taking 10 pictures per second of a dancing girl: you see the changes in every movement but if you want to know what is the total distance the girl covered then it can get rather complex. So you have for 1 minute of pictures and I bet you will break your teeth on that one. If you want to see the distance the girl danced in a week, month or year go for it!

This gets us into the basic question of BI. Why are we collecting data? What do we want to do with it? Apparently taking 10 pictures a second with a Nikon F4, which is a great camera, but it did not help us finding a reply to the question: what distance did the girl cover. We need other tools like a simple step counter.

It is the same with MDW: we collect a lot of data but Microsoft never asked it's customers what they wanted to collect. 

Questions like what query did blow the performance in my data warehouse at day x can be answered but you will never find the user involved. 

The same with question on server availability. A valid question in a database world where words like bronze, silver or gold SLA's are standard now. Microsoft is talking of 99.999% uptime for a clustered server but can you show it? No, you have to use TSQL code and create your own report.

MDW is designed from a technical point of view trying to make it easy to examine issues on your data warehouse but the result kind of drowned in a waterfall of data poured out over us. 

Questions like: when did my DWH drown in the TSQL requests fired and who did it are impossible to answer without diving into a day to day analysis like the pictures of the dancing girl above.

So Microsoft tells you you can do this yourself. Of cause you can extend MDW but that is not an esy task. Queries are stored in XML (why did they choose that) and XML does not like '', >, <, etc. And TSQL does not like
&lt;, &gt;, &#39;,etc. 

More about this issue in this article: XML parsing errors

That will not resolve your issues in MDW but it simply shows you that you have to change all your comparisons in your custom extensions.

If you look at the article Microsoft published on extending the MDW you will run into trouble within the first minute. Check it out: Create a Custom Collection Set That Uses the Generic T-SQL Query Collector Type (Transact-SQL)

If you do exactly what is written in that example you create a new data collection set. If you collect the data in that example over 3 months using the snapshot mechanism your table will be humongous.

Comments

Popular posts from this blog

Privacy and the liberty to express yourself on LinkedIn

Unaware that LinkedIn has such a strong filtering policy that it does not allow me posting a completely innocent post on a Chinese extreme photography website I tried to post the following: "As an Mpx lover I was suprised to find out that the M from Million is now replaced by the B from Billion. This picture is 24 Bpx! Yes you read this well, 24 billion pixels.  Searching on the picture I stumbled on a fellow Nikon lover. If you want to search for him yourself you can find him here: http://www.bigpixel.cn/t/5834170785f26b37002af46a " In my eyes nothing is wrong with this post, but LinkedIn considers it as offending. I changed the lover words, but I could not post it.  Even taking a picture and post it will not let this pass:  Or my critical post on LinkedIn crazy posting policy: it will not pass and I cannot post it.  The technology LinkedIn shows here is an example what to expect in the near future.  Newspapers will have a unified reporting using

Windows Server 2016 with ... XBox extensions !!!

Microsoft must have been thinking that the live of a Windows administrator gets so boring that they need a distraction and they have integrated the XBox Live extensions standard in Windows Server 2016.  No kidding: take a look. I did not select it as a feature and it is there. As you can see the XBox Live service is started automatically even in mode manual (see error log).  To my opinion these 3 XBox Live services should never be available on a server. Unless the server is part of the XBox Live platform of cause but honestly I don't think that Microsoft will allow that.  Every IT manager with a serious Windows production environment would fire any administrator playing XBox Live on the production servers. So though it might be tempting don't do it Windows administrator. It is a trap! :-)

How to run SQL Server 2016 with In-Databasse R on Windows 2016 CTP5

For those who like me tried to run SQL Server 2016 with In-Database R might have run into the same problem as me: In-Database R or the LaunchPad service gives a timeout and won't start. I did several clean installations with different configuration options - for instance I like to put my data on another disk than the system disk - but in the end I tried to do the next, next, next, finish install to see if it something in the setup options is hard coded in there (yes, it happens developers!). For some reason this problem is related to Windows 2016 and not on Windows 2012R2 and I hope the SQL Server team will soon resolve these issues because they are in one word a bit sloppy.  There are 2 issues (maybe even 3 so I give this one also):  The R setup does not create the ExtensibilityLog directory in the "C:\Program Files\Microsoft SQL Server\MSSQL13.MSSQLSERVER\MSSQL\Log" directory The R setup sets the number of users in the SQL Server Launchpad service to 0 it is pos