I haven’t been blogging much lately, actually I haven’t been blogging at all in the last 4 months. The reason behind is I have been putting all my efforts in a new project I started recently, which absorbed all my attention and spare time.
I am proud to announce that my project is now live and available to everyone for download.
The project name is ExtendedTSQLCollector and you can find it at http://extendedtsqlcollector.codeplex.com. As you may have already guessed, it’s a bridge between two technologies that were not meant to work together, that could instead bring great advantages when combined: Extended Events and Data Collector.
ExtendedTSQLCollector is a set of two Collector Types built to overcome some of the limitations found in the built-in collector types and extend their functionality to include the ability to collect data from XE sessions.
The first Collector Type is the “Extended T-SQL Query” collector type, which was my initial goal when I started the project. If you have had the chance to play with the built-in “Generic T-SQL Query” collector type, you may have noticed that not all datatypes are supported. For instance, it’s impossible to collect data from XML or varchar(max) columns. This is due to the intermediate format used by this collector type: the SSIS raw files.
The “Extended T-SQL Query” collector type uses a different intermediate format, which allows collecting data of any data type. This is particularly useful, because SQL Server exposes lots of information in XML format (just think of the execution plans!) and you no longer need to code custom SSIS packages to collect that data.
The second Collector Type is the “Extended XE Reader” collector type, which takes advantage of the Extended Events streaming APIs to collect data from an Extended Events session, without the need to specify additional targets such as .xel files or ring buffers. This means no file system bloat due to .xel rollover files and no memory consumption for additional ring buffers: all the events are read directly from the session and processed in near real-time.
In addition to the filter predicates defined in the XE session, you can add more filter predicates on the data to collect and upload to the MDW and decide which columns (fields and actions) to collect. The collector will take care of creating the target table in your MDW database and upload all the data that satisfies the filter predicates.
The near real-time behavior of this collector type allowed me to include an additional feature to the mix: the ability to fire alerts in response to Extended Events. The current release (1.5) allows firing email alerts when the events are captured, with additional filter predicates and the ability to include event fields and actions in the email body. You can find more information on XE alerts in the documentation.
Here is an example of the email alerts generated by the XEReader collector type for the blocked_process event:
Another part of the project is the CollectionSet Manager, a GUI to install the collector types to the target servers and configure collection sets and collection items. I think that one of the reasons why the Data Collector is very underutilized by DBAs is the lack of a Graphical UI. Besides the features specific to the ExtendedTSQLCollector, such as installing the collector type, this small utility aims at providing the features missing in the SSMS Data Collector UI. This part of the project is still at an early stage, but I am planning to release it in the next few months.
My journey through the ins and outs of the Data Collector allowed me to understand deeply how it works and how to set it up and troubleshoot it. Now I am planning to start a blog series on this topic, from the basics to the advanced features. Stay tuned :-)
I don’t want to go into deep details on the setup and configuration of this small project: I just wanted to ignite your curiosity and make you rush to codeplex to download your copy of ExtendedTSQLCollector.
What are you waiting for?
Some days ago I was talking with my friend Davide Mauri about the uniquifier that SQL Server adds to clustered indexes when they are not declared as UNIQUE.
We were not completely sure whether this behaviour applied to duplicate keys only or to all keys, even when unique.
The best way to discover the truth is a script to test what happens behind the scenes:
-- ============================================= -- Author: Gianluca Sartori - @spaghettidba -- Create date: 2014-03-15 -- Description: Checks whether the UNIQUIFIER column -- is added to a column only on -- duplicate clustering keys or all -- keys, regardless of uniqueness -- ============================================= USE tempdb GO IF OBJECT_ID('sizeOfMyTable') IS NOT NULL DROP VIEW sizeOfMyTable; GO -- Create a view to query table size information -- Not very elegant, but saves a lot of typing CREATE VIEW sizeOfMyTable AS SELECT OBJECT_NAME(si.object_id) AS table_name, si.name AS index_name, SUM(total_pages) AS total_pages, SUM(used_pages) AS used_pages, SUM(data_pages) AS data_pages FROM sys.partitions AS p INNER JOIN sys.allocation_units AS AU ON P.hobt_id = AU.container_id INNER JOIN sys.indexes AS si ON si.index_id = p.index_id AND si.object_id = p.object_id WHERE si.object_id = OBJECT_ID('#testUniquifier') GROUP BY OBJECT_NAME(si.object_id), si.name GO IF OBJECT_ID('#testUniquifier') IS NOT NULL DROP TABLE #testUniquifier; -- Create a test table CREATE TABLE #testUniquifier ( i int NOT NULL ) -- Results table: will receive table size -- in different scenarios DECLARE @results TABLE( description varchar(500), table_name sysname, index_name sysname, total_pages int, used_pages int, data_pages int ); -- INSERTS 100K UNIQUE VALUES INSERT INTO #testUniquifier SELECT TOP(100000) ROW_NUMBER() OVER(ORDER BY (SELECT NULL)) FROM sys.all_columns AS AC CROSS JOIN sys.all_columns AS AC1; -- ----------------------------------------------------------------- -- TEST1: CREATES A UNIQUE CLUSTERED INDEX (NO UNIQUIFIER) -- ----------------------------------------------------------------- CREATE UNIQUE CLUSTERED INDEX UK_test ON #testUniquifier(i); INSERT @results SELECT 'Unique clustered index' AS description, * FROM sizeOfMyTable; DROP INDEX UK_test ON #testUniquifier -- ----------------------------------------------------------------- -- TEST2: CREATES A NON-UNIQUE CLUSTERED INDEX -- NO DUPLICATES ARE PRESENT YET -- ----------------------------------------------------------------- CREATE CLUSTERED INDEX IX_test ON #testUniquifier(i) INSERT @results SELECT 'Non-Unique clustered index, no duplicates' AS description, * FROM sizeOfMyTable DROP INDEX IX_test ON #testUniquifier -- ----------------------------------------------------------------- -- TEST3: CREATES A NON-UNIQUE CLUSTERED INDEX -- 10000 DUPLICATE VALUES ARE PRESENT -- ----------------------------------------------------------------- UPDATE TOP(10000) #testUniquifier SET i = 1 CREATE CLUSTERED INDEX IX_test ON #testUniquifier(i) INSERT @results SELECT 'Non-Unique clustered index, some duplicates' AS description, * FROM sizeOfMyTable DROP INDEX IX_test ON #testUniquifier -- ----------------------------------------------------------------- -- TEST4: CREATES A NON-UNIQUE CLUSTERED INDEX -- ALL ROWS CONTAIN THE SAME VALUE (1) -- ----------------------------------------------------------------- UPDATE #testUniquifier SET i = 1 CREATE CLUSTERED INDEX IX_test ON #testUniquifier(i) INSERT @results SELECT 'Non-Unique clustered index, all duplicates' AS description, * FROM sizeOfMyTable -- ----------------------------------------------------------------- -- Display results -- ----------------------------------------------------------------- SELECT * FROM @results;
As you can see, the uniquifier is added only to the keys that are duplicated:
Another way to discover the same results would be looking at the output of DBCC PAGE().
Looking at the text output of DBCC PAGE, uniquifiers are displayed as 0 (zero) when the values are not set, but the values are actually missing from the page.
This becomes even clearer when using DBCC PAGE WITH TABLERESULTS:
IF OBJECT_ID('tempdb..#formatteddata') IS NOT NULL DROP TABLE #formatteddata; SELECT *, ROWNUM = ROW_NUMBER() OVER (ORDER BY page_id, slot_id) INTO #formatteddata FROM #testUniquifier CROSS APPLY sys.fn_PhysLocCracker(%%physloc%%); IF OBJECT_ID('tempdb..#dbccpage') IS NOT NULL DROP TABLE #dbccpage; CREATE TABLE #dbccpage ( page_id int, ParentObject varchar(128), Object varchar(128), Field varchar(128), value varchar(4000), Slot AS SUBSTRING(Object, NULLIF(CHARINDEX('Slot ',Object,1),0) + 5, ISNULL(NULLIF(CHARINDEX(' ',Object,6),0),0) - 5) ) DECLARE @current_page_id int; DECLARE pages CURSOR STATIC LOCAL FORWARD_ONLY READ_ONLY FOR SELECT DISTINCT page_id FROM #formatteddata OPEN pages FETCH NEXT FROM pages INTO @current_page_id WHILE @@FETCH_STATUS = 0 BEGIN INSERT INTO #dbccpage (ParentObject, Object, Field, value) EXEC sp_executesql N'DBCC PAGE (2, 1, @pageid, 3) WITH TABLERESULTS;', N'@pageid int', @current_page_id UPDATE #dbccpage SET page_id = @current_page_id WHERE page_id IS NULL FETCH NEXT FROM pages INTO @current_page_id END CLOSE pages; DEALLOCATE pages; WITH PageData AS ( SELECT page_id, slot, field, value FROM #dbccpage WHERE field IN ('i', 'UNIQUIFIER') ), Uniquifiers AS ( SELECT * FROM PageData PIVOT (MAX(value) FOR field IN ([i], [UNIQUIFIER])) AS pvt ), sourceData AS ( SELECT * FROM #formatteddata ) SELECT src.ROWNUM, src.i, src.page_id, src.slot_id, UNIQUIFIER FROM sourceData AS src LEFT JOIN Uniquifiers AS unq ON src.slot_id = unq.slot AND src.page_id = unq.page_id ORDER BY ROWNUM;
If you run the code in the different situations outlined before (unique clustered index, non-unique clustered index with or without duplicate keys) you will find the uniquifiers associated with each duplicate key and you will also notice that no uniquifier is generated for the keys that are unique.
Last week I was in the process of migrating a couple of SQL Server instances from 2008 R2 to 2012.
In order to let the migration complete quickly, I set up log shipping from the old instance to the new instance. Obviously, the existing backup jobs had to be disabled, otherwise they would have broken the log chain.
That got me thinking: was there a way to keep both “regular” transaction log backups (taken by the backup tool) and the transaction log backups taken by log shipping?
The first thing that came to my mind was the COPY_ONLY option available since SQL Server 2005.
You probably know that COPY_ONLY backups are useful when you have to take a backup for a special purpose, for instance when you have to restore from production to test. With the COPY_ONLY option, database backups don’t break the differential base and transaction log backups don’t break the log chain.
My initial thought was that I could ship COPY_ONLY backups to the secondary and keep taking scheduled transaction log backups with the existing backup tools.
I was dead wrong.
Let’s see it with an example on a TEST database.
I took 5 backups:
- FULL database backup, to initialize the log chain. Please note that COPY_ONLY backups cannot be used to initialize the log chain.
- LOG backup
- LOG backup with the COPY_ONLY option
- LOG backup
- LOG backup with the COPY_ONLY option
The backup information can be queried from backupset in msdb:
SELECT ROW_NUMBER() OVER(ORDER BY bs.backup_start_date) AS [backup #] ,first_lsn ,last_lsn ,backup_start_date ,type ,is_copy_only ,DENSE_RANK() OVER(ORDER BY type, bs.first_lsn) AS sequence FROM msdb.dbo.backupset bs WHERE bs.database_name = 'TEST'
As you can see, the COPY_ONLY backups don’t truncate the transaction log and losing one of those backups wouldn’t break the log chain.
However, all backups always start from the first available LSN, which means that scheduled log backups taken without the COPY_ONLY option truncate the transaction log and make significant portions of the transaction log unavailable in the next COPY_ONLY backup.
You can see it clearly in the following picture: the LSNs highlighted in red should contain no gaps in order to be restored successfully to the secondary, but the regular TLOG backups break the log chain in the COPY_ONLY backups.
That means that there’s little or no point in taking COPY_ONLY transaction log backups, as “regular” backups will always determine gaps in the log chain.
When log shipping is used, the secondary server is the only backup you can have, unless you keep the TLOG backups or use your backup tool directly to ship the logs.
Why on earth should one take a COPY_ONLY TLOG backup (more than one at least) is beyond my comprehension, but that’s a whole different story.
I exchanged some emails about my post with Jonathan Kehayias and looks like I was wrong on many of the points I made.
I don’t want to keep misleading information around and I definitely need to fix my wrong assumptions.
Unfortunately, I don’t have the time to correct it immediately and I’m afraid it will have to remain like this for a while.
Sorry for the inconvenience, I promise I will try to fix it in the next few days.
Now that my SQL Server 2014 CTP1 virtual machine is ready, I started to play with it and some new features and differences with the previous versions are starting to appear.
What I want to write about today is a T-SQL enhancement to DDL statements that brings in some new interesting considerations.
SQL Server 2014 now supports a new T-SQL syntax that allows defining an index in the CREATE TABLE statement without having to issue separate CREATE INDEX statements.
Up to now, the same could be achieved only with PRIMARY KEY and UNIQUE constraints, thus allowing UNIQUE indexes only.
For instance, the following statement creates a table with a unique clustered index on order_id and a unique nonclustered index on PO_number:
CREATE TABLE #orders ( order_id uniqueidentifier NOT NULL PRIMARY KEY CLUSTERED DEFAULT NEWSEQUENTIALID() ,PO_number varchar(50) NOT NULL UNIQUE ,order_date datetime NOT NULL ,total_amount decimal(18,3) )
OK, but what if I want to add a non-unique index to my table?
SQL Server 2014 offers a new syntax to do that inline with the table DDL:
CREATE TABLE #orders ( order_id uniqueidentifier NOT NULL PRIMARY KEY CLUSTERED DEFAULT NEWSEQUENTIALID() ,PO_number varchar(50) NOT NULL UNIQUE -- creates a nonclustered index on order_date ,order_date datetime NOT NULL INDEX IX_order_date ,total_amount decimal(18,3) )
A similar syntax can be used to create a compound index:
CREATE TABLE #orders ( order_id uniqueidentifier NOT NULL PRIMARY KEY CLUSTERED DEFAULT NEWSEQUENTIALID() ,PO_number varchar(50) NOT NULL UNIQUE ,order_date datetime NOT NULL INDEX IX_order_date ,total_amount decimal(18,3) -- creates a compound index on PO_number and order_date ,INDEX IX_orders_compound(PO_number, order_date) )
An interesting aspect of this new syntax is that it allows creating non-unique nonclustered indexes to table variables, which is something that couldn’t be done in the previous versions.
The syntax to use is the same as for permanent tables:
DECLARE @orders TABLE ( order_id uniqueidentifier NOT NULL PRIMARY KEY CLUSTERED DEFAULT NEWSEQUENTIALID() ,PO_number varchar(50) NOT NULL UNIQUE ,order_date datetime NOT NULL INDEX IX_order_date ,total_amount decimal(18,3) )
Cool! But, wait: does this mean that table variables will now behave in the same way permanent tables do?
Table variables don’t have statistics, and being able to create indexes on them won’t change anything in this regard.
Do you want a proof? OK, the skeptics can run the following code. Please make sure you capture the actual execution plan.
SET NOCOUNT ON; -- create the table variable DECLARE @orders TABLE ( order_id uniqueidentifier NOT NULL PRIMARY KEY CLUSTERED DEFAULT NEWSEQUENTIALID() ,PO_number varchar(50) NOT NULL UNIQUE ,order_date datetime NOT NULL INDEX IX_order_date ,total_amount decimal(18,3) ) -- insert some data INSERT INTO @orders (order_date, PO_number, total_amount) SELECT order_date = DATEADD(second, CHECKSUM(NEWID()), GETDATE()) ,PO_number = CAST(NEWID() AS varchar(50)) ,total_amount = CHECKSUM(NEWID()) / 1000.0 FROM sys.all_columns - SELECT COUNT(*) FROM @orders WHERE order_date > GETDATE() OPTION ( -- activate some (undocumented) trace flags to show -- statistics usage. More information on the flags -- can be found on Paul White's blog: -- http://sqlblog.com/blogs/paul_white/archive/2011/09/21/how-to-find-the-statistics-used-to-compile-an-execution-plan.aspx -- redirect output to the messages tab QUERYTRACEON 3604 -- show "interesting" statistics ,QUERYTRACEON 9292 -- show loaded statistics ,QUERYTRACEON 9402 -- add RECOMPILE to let the optimizer "see" -- the table cardinality ,RECOMPILE )
The output of the above batch is empty. Looks like no stats were loaded.
The actual execution plan confirms that no stats were loaded and the estimated cardinality of the table variable is way off:
If we repeat the test with a temporary table, we see a different behaviour.
SET NOCOUNT ON; IF OBJECT_ID('tempdb..#orders') IS NOT NULL DROP TABLE #orders; CREATE TABLE #orders ( order_id uniqueidentifier NOT NULL PRIMARY KEY CLUSTERED DEFAULT NEWSEQUENTIALID() ,PO_number varchar(50) NOT NULL UNIQUE ,order_date datetime NOT NULL INDEX IX_order_date ,total_amount decimal(18,3) ) INSERT INTO #orders (order_date, PO_number, total_amount) SELECT order_date = DATEADD(second, CHECKSUM(NEWID()), GETDATE()) ,PO_number = CAST(NEWID() AS varchar(50)) ,total_amount = CHECKSUM(NEWID()) / 1000.0 FROM sys.all_columns SELECT COUNT(*) FROM #orders WHERE order_date > GETDATE() OPTION ( QUERYTRACEON 3604 ,QUERYTRACEON 9292 ,QUERYTRACEON 9402 )
This time the messages tab contains some output:
Stats header loaded: DbName: tempdb, ObjName: #orders, IndexId: 2, ColumnName: order_date, EmptyTable: FALSE Stats header loaded: DbName: tempdb, ObjName: #orders, IndexId: 2, ColumnName: order_date, EmptyTable: FALSE
The optimizer identified the statistics on the oder_date column as “interesting” and then loaded the stats header.
Again, the actual execution plan confirms that a better estimation is available:
The lack of statistics has always been the most significant difference between table variables and temporary tables and SQL2014 doesn’t appear to change the rules (yet).
As a DBA, you probably have your toolbox full of scripts, procedures and functions that you use for the day-to-day administration of your instances.
I’m no exception and my hard drive is full of scripts that I tend to accumulate and never throw away, even if I know I will never need (or find?) them again.
However, my preferred way to organize and maintain my administration scripts is a database called “TOOLS”, which contains all the scripts I regularly use.
One of the challenges involved in keeping the scripts in a database rather than in a script file is the inability to choose a database context for the execution. When a statement is encapsulated in a view, function or stored procedure in a database, every reference to a database-specific object is limited to the database that contains the programmable object itself. The only way to overcome this limitation is the use of dynamic sql.
For instance, if I want to query the name of the tables in a database, I can use the following statement:
SELECT name FROM sys.tables
The statement references a catalog view specific to a single database, so if I enclose it in a stored procedure, the table names returned by this query are those found in the database that contains the stored procedure itself:
USE TOOLS; GO CREATE PROCEDURE getTableNames AS SELECT name FROM sys.tables; GO EXEC getTableNames;
This is usually not an issue, since most stored procedures will not cross the boundaries of the database they are created in. Administration scripts are different, because they are meant as a single entry point to maintain the whole SQL server instance.
In order to let the statement work against a different database, you need to choose one of the following solutions:
- dynamic SQL
- marking as a system object
- … an alternative way
Each of these techniques has its PROs and its CONs and I will try to describe them in this post.
1. Dynamic SQL
It’s probably the easiest way to solve the issue: you just have to concatenate the database name to the objects names.
USE TOOLS; GO ALTER PROCEDURE getTableNames @db_name sysname AS BEGIN DECLARE @sql nvarchar(max) SET @sql = 'SELECT name FROM '+ QUOTENAME(@db_name) +'.sys.tables'; EXEC(@sql) END GO EXEC getTableNames 'msdb';
- very easy to implement for simple statements
- can rapidly turn to a nightmare with big, complicated statements, as each object must be concatenated with the database name. Different objects have different ways to be related to the database: tables and views can be concatenated directly, while functions such as OBJECT_NAME accept an additional parameter to specify the database name.
- the statement has to be treated as a string and enclosed in quotes, which means that:
- quotes must be escaped, and escaped quotes must be escaped again and escaped and re-escaped quotes… ok, you know what I mean
- no development aids such as intellisense, just-in-time syntax checks and syntax coloring
It’s a neater way to avoid concatenating the database name to each object referenced in the statement.
USE TOOLS; GO ALTER PROCEDURE getTableNames @db_name sysname AS BEGIN -- use a @sql variable to store the whole query -- without concatenating the database name DECLARE @sql nvarchar(max); SET @sql = 'SELECT name FROM sys.tables'; -- concatenate the database name to the -- sp_executesql call, just once DECLARE @cmd nvarchar(max); SET @cmd = 'EXEC '+ QUOTENAME(@db_name) +'.sys.sp_executesql @sql'; EXEC sp_executesql @cmd, N'@sql nvarchar(max)', @sql END GO EXEC getTableNames 'msdb';
- the dynamic sql is taken as a whole and does not need to be cluttered with multiple concatenations
- needs some more work than a straight concatenation and can be seen as “obscure”
- suffers from the same issues found with plain dynamic sql, because the statement is, again, treated as a string
3. System object
Nice and easy: every stored procedure you create in the master database with the “sp_” prefix can be executed from any database context.
Using the undocumented stored procedure sp_MS_marksystemobject you can also mark the stored procedure as a “system object” and let it reference the tables in the database from which it is invoked.
USE master; GO ALTER PROCEDURE sp_getTableNames AS BEGIN SELECT name FROM sys.tables END GO EXEC sys.sp_MS_marksystemobject 'sp_getTableNames' GO USE msdb; GO EXEC sp_getTableNames;
- no need to use dynamic sql
- requires creating objects in the “master” database, which is something I tend to avoid
- works with stored procedures only (actually, it works with other objects, such as tables and views, but you have to use the “sp_” prefix. The day I will find a view named “sp_getTableNames” in the master database it won’t be safe to stay near me)
An alternative method:
It would be really helpful if we could store the statement we want to execute inside an object that doesn’t involve dynamic sql and doesn’t need to be stored in the master database. In other words, we need a way to get the best of both worlds.
Is there such a solution? Apparently, there isn’t.
The ideal object to store a statement and reuse it later is a view, but there is no way to “execute” a view against a different database. In fact you don’t execute a view, you just select from it, which is quite a different thing.
What you “execute” when you select from a view is the statement in its definition (not really, but let me simplify).
So, what we would need to do is just read the definition from a view and use the statement against the target database. Sounds straightforward, but it’s not.
The definition of a view also contains the “CREATE VIEW” statement and stripping it off is not just as easy as it seems.
Let’s see the issue with an example: I will create a view to query the last update date of the index statistics in a database, using the query from Glenn Berry’s Diagnostic Queries.
USE TOOLS; GO -- When were Statistics last updated on all indexes? (Query 48) CREATE VIEW statisticsLastUpdate AS SELECT DB_NAME() AS database_name ,o.NAME AS stat_name ,i.NAME AS [Index Name] ,STATS_DATE(i.[object_id], i.index_id) AS [Statistics Date] ,s.auto_created ,s.no_recompute ,s.user_created ,st.row_count FROM sys.objects AS o WITH (NOLOCK) INNER JOIN sys.indexes AS i WITH (NOLOCK) ON o.[object_id] = i.[object_id] INNER JOIN sys.stats AS s WITH (NOLOCK) ON i.[object_id] = s.[object_id] AND i.index_id = s.stats_id INNER JOIN sys.dm_db_partition_stats AS st WITH (NOLOCK) ON o.[object_id] = st.[object_id] AND i.[index_id] = st.[index_id] WHERE o.[type] = 'U';
I just had to remove ORDER BY and OPTION(RECOMPILE) because query hints cannot be used in views.
Querying the object definition returns the whole definition of the view, not only the SELECT statement:
SELECT OBJECT_DEFINITION(OBJECT_ID('statisticsLastUpdate')) AS definition
definition ------------------------------------------------------------------- -- When were Statistics last updated on all indexes? (Query 48) CREATE VIEW statisticsLastUpdate AS SELECT DB_NAME() AS database_name ,o.NAME AS stat_name ,i.NAME AS [Index Name] ,STATS_DATE(i.[object_id], i.index_id) AS [Statistics Date] ,s.auto_created ,s.no_recompute (1 row(s) affected)
In order to extract the SELECT statement, we would need something able to parse (properly!) the view definition and we all know how complex it can be.
Fortunately, SQL Server ships with an undocumented function used in replication that can help solving the problem: its name is fn_replgetparsedddlcmd.
This function accepts some parameters, lightly documented in the code: fn_replgetparsedddlcmd (@ddlcmd, @FirstToken, @objectType, @dbname, @owner, @objname, @targetobject)
Going back to the example, we can use this function to extract the SELECT statement from the view definition:
SELECT master.sys.fn_replgetparsedddlcmd( OBJECT_DEFINITION(OBJECT_ID('statisticsLastUpdate')) ,'CREATE' ,'VIEW' ,DB_NAME() ,'dbo' ,'statisticsLastUpdate' ,NULL ) AS statement
statement --------------------------------------------------------------------- AS SELECT DB_NAME() AS database_name ,o.NAME AS stat_name ,i.NAME AS [Index Name] ,STATS_DATE(i.[object_id], i.index_id) AS [Statistics Date] ,s.auto_created ,s.no_recompute ,s.user_created ,st.row_count (1 row(s) affected)
The text returned by the function still contains the “AS” keyword, but removing it is a no-brainer:
DECLARE @cmd nvarchar(max) SELECT @cmd = master.sys.fn_replgetparsedddlcmd( OBJECT_DEFINITION(OBJECT_ID('statisticsLastUpdate')) ,'CREATE' ,'VIEW' ,DB_NAME() ,'dbo' ,'statisticsLastUpdate' ,NULL ) SELECT @cmd = RIGHT(@cmd, LEN(@cmd) - 2) -- Removes "AS" SELECT @cmd AS statement
statement ------------------------------------------------------------------- SELECT DB_NAME() AS database_name ,o.NAME AS stat_name ,i.NAME AS [Index Name] ,STATS_DATE(i.[object_id], i.index_id) AS [Statistics Date] ,s.auto_created ,s.no_recompute ,s.user_created ,st.row_count (1 row(s) affected)
Now that we are able to read the SELECT statement from a view’s definition, we can execute that statement against any database we like, or even against all the databases in the instance.
-- ============================================= -- Author: Gianluca Sartori - spaghettidba -- Create date: 2013-04-16 -- Description: Extracts the view definition -- and runs the statement in the -- database specified by @db_name -- If the target database is a pattern, -- the statement gets executed against -- all databases matching the pattern. -- ============================================= CREATE PROCEDURE [dba_execute_view] @view_name sysname ,@db_name sysname AS BEGIN SET NOCOUNT, XACT_ABORT, QUOTED_IDENTIFIER, ANSI_NULLS, ANSI_PADDING, ANSI_WARNINGS, ARITHABORT, CONCAT_NULL_YIELDS_NULL ON; SET NUMERIC_ROUNDABORT OFF; DECLARE @cmd nvarchar(max) DECLARE @sql nvarchar(max) DECLARE @vw_schema sysname DECLARE @vw_name sysname IF OBJECT_ID(@view_name) IS NULL BEGIN RAISERROR('No suitable object found for name %s',16,1,@view_name) RETURN END IF DB_ID(@db_name) IS NULL AND @db_name NOT IN ('[USER]','[SYSTEM]') AND @db_name IS NOT NULL BEGIN RAISERROR('No suitable database found for name %s',16,1,@view_name) RETURN END SELECT @vw_schema = OBJECT_SCHEMA_NAME(OBJECT_ID(@view_name)), @vw_name = OBJECT_NAME(OBJECT_ID(@view_name)) SELECT @cmd = master.sys.fn_replgetparsedddlcmd( OBJECT_DEFINITION(OBJECT_ID(@view_name)) ,'CREATE' ,'VIEW' ,DB_NAME() ,@vw_schema ,@vw_name ,NULL ) SELECT @cmd = RIGHT(@cmd, LEN(@cmd) - 2) -- Removes "AS" -- CREATE A TARGET TEMP TABLE SET @sql = N' SELECT TOP(0) * INTO #results FROM ' + @view_name + '; INSERT #results EXEC [dba_ForEachDB] @statement = @cmd, @name_pattern = @db_name; SELECT * FROM #results;' EXEC sp_executesql @sql ,N'@cmd nvarchar(max), @db_name sysname' ,@cmd ,@db_name END
The procedure depends on dba_ForEachDB, the stored procedure I posted a couple of years ago that replaces the one shipped by Microsoft. If you still prefer their version, you’re free to modify the code as you wish.
Now that we have a stored procedure that “executes” a view, we can use it to query statistics update information from a different database:
EXEC [dba_execute_view] 'statisticsLastUpdate', 'msdb'
We could also query the same information from all user databases:
EXEC [dba_execute_view] 'statisticsLastUpdate', '[USER]'
That’s it, very easy and straightforward.
Just one suggestion for the SELECT statements in the views: add a DB_NAME() column, in order to understand where the data comes from, or it’s going to be a total mess.
This is just the basic idea, the code can be improved in many ways.
For instance, we could add a parameter to decide whether the results must be piped to a temporary table or not. As you probably know, INSERT…EXEC cannot be nested, so you might want to pipe the results to a table in a different way.
Another thing you might want to add is the ability to order the results according to an additional parameter.
To sum it up, with a little help from Microsoft, we can now safely create a database packed with all our administration stuff and execute the queries against any database in our instance.
Recently I had to assess and tune quite a lot of SQL Server instances and one the things that are often overlooked is the location of the system databases.
I often see instance where the system databases are located in the system drives under the SQL Server default installation path, which is bad for many reasons, especially for tempdb.
I had to move the system databases so many times that I ended up coding a script to automate the process.
The script finds all system databases that are not sitting in the default data and log paths and issues the ALTER DATABASE statements needed to move the files to the default paths.
Obviously, to let the script work, the default data and log paths must have been set in the instance properties:
You may also point out that moving all system databases to the default data and log paths is not always a good idea. And you would be right: for instance, if possible, the tempdb database should be working on a fast dedicated disk. However, very often I find myself dealing with low-end servers where separate data and log disks are a luxury, not to mention a dedicated tempdb disk. If you are concerned about moving tempd to the default data and log paths, you can modify the script accordingly.
-- ============================================= -- Author: Gianluca Sartori - spaghettidba -- Create date: 2013-03-22 -- Description: Moves the system databases to the -- default data and log paths and -- updates SQL Server startup params -- accordingly. -- ============================================= SET NOCOUNT ON; USE master; -- Find default data and log paths -- reading from the registry DECLARE @defaultDataLocation nvarchar(4000) DECLARE @defaultLogLocation nvarchar(4000) EXEC master.dbo.xp_instance_regread N'HKEY_LOCAL_MACHINE', N'Software\Microsoft\MSSQLServer\MSSQLServer', N'DefaultData', @defaultDataLocation OUTPUT EXEC master.dbo.xp_instance_regread N'HKEY_LOCAL_MACHINE', N'Software\Microsoft\MSSQLServer\MSSQLServer', N'DefaultLog', @defaultLogLocation OUTPUT -- Loop through all system databases -- and move to the default data and log paths DECLARE @sql nvarchar(max) DECLARE stmts CURSOR STATIC LOCAL FORWARD_ONLY FOR SELECT ' ALTER DATABASE '+ DB_NAME(database_id) + ' MODIFY FILE ( ' + ' NAME = '''+ name +''', ' + ' FILENAME = '''+ CASE type_desc WHEN 'ROWS' THEN @defaultDataLocation ELSE @defaultLogLocation END + '\'+ RIGHT(physical_name,CHARINDEX('\',REVERSE(physical_name),1)-1) +'''' + ' )' FROM sys.master_files WHERE DB_NAME(database_id) IN ('master','model','msdb','tempdb') AND ( physical_name NOT LIKE @defaultDataLocation + '%' OR physical_name NOT LIKE @defaultLogLocation + '%' ) OPEN stmts FETCH NEXT FROM stmts INTO @sql WHILE @@FETCH_STATUS = 0 BEGIN PRINT @sql EXEC(@sql) FETCH NEXT FROM stmts INTO @sql END CLOSE stmts DEALLOCATE stmts -- Update SQL Server startup parameters -- to reflect the new master data and log -- files locations DECLARE @val nvarchar(500) DECLARE @key nvarchar(100) DECLARE @regvalues TABLE ( parameter nvarchar(100), value nvarchar(500) ) INSERT @regvalues EXEC master.dbo.xp_instance_regenumvalues N'HKEY_LOCAL_MACHINE', N'SOFTWARE\Microsoft\MSSQLServer\MSSQLServer\Parameters' DECLARE reg CURSOR STATIC LOCAL FORWARD_ONLY FOR SELECT * FROM @regvalues WHERE value LIKE '-d%' OR value LIKE '-l%' OPEN reg FETCH NEXT FROM reg INTO @key, @val WHILE @@FETCH_STATUS = 0 BEGIN IF @val LIKE '-d%' SET @val = '-d' + ( SELECT physical_name FROM sys.master_files WHERE DB_NAME(database_id) = 'master' AND type_desc = 'ROWS' ) IF @val LIKE '-l%' SET @val = '-l' + ( SELECT physical_name FROM sys.master_files WHERE DB_NAME(database_id) = 'master' AND type_desc = 'LOG' ) EXEC master.dbo.xp_instance_regwrite N'HKEY_LOCAL_MACHINE', N'SOFTWARE\Microsoft\MSSQLServer\MSSQLServer\Parameters', @key, N'REG_SZ', @val FETCH NEXT FROM reg INTO @key, @val END CLOSE reg DEALLOCATE reg
After running this script, you can shut down the SQL Server service and move the data and log files to the appropriate locations.
When the files are ready, you can bring SQL Server back online.
BE CAREFUL! Before running this script against a clustered instance, check what the xp_instance_regread commands return: I have seen cases with SQL Server not reading from the appropriate keys.
If you are one among the many that downloaded my consistency check stored procedure called “dba_RunCHECKDB”, you may have noticed a “small” glitch… it doesn’t work on SQL Server 2012!
This is due to the resultset definition of DBCC CHECKDB, which has changed again in SQL Server 2012. Trying to pipe the results of that command in the table definition for SQL Server 2008 produces a column mismatch and it obviously fails.
Fixing the code is very easy indeed, but I could never find the time to post the corrected version until today.
Also, I had to discover the new table definition for DBCC CHECKDB, and it was not just as easy as it used to be in SQL Server 2008. In fact, a couple of days ago I posted a way to discover the new resultset definition working around the cumbersome metadata discovery feature introduced in SQL Server 2012.
Basically, the new output of DBCC CHECKDB now includes 6 new columns:
CREATE TABLE ##DBCC_OUTPUT( Error int NULL, [Level] int NULL, State int NULL, MessageText nvarchar(2048) NULL, RepairLevel nvarchar(22) NULL, Status int NULL, DbId int NULL, -- was smallint in SQL2005 DbFragId int NULL, -- new in SQL2012 ObjectId int NULL, IndexId int NULL, PartitionId bigint NULL, AllocUnitId bigint NULL, RidDbId smallint NULL, -- new in SQL2012 RidPruId smallint NULL, -- new in SQL2012 [File] smallint NULL, Page int NULL, Slot int NULL, RefDbId smallint NULL, -- new in SQL2012 RefPruId smallint NULL, -- new in SQL2012 RefFile smallint NULL, -- new in SQL2012 RefPage int NULL, RefSlot int NULL, Allocation smallint NULL )
If you Google the name of one of these new columns, you will probably find a lot of blog posts (no official documentation, unfortunately) that describes the new output of DBCC CHECKDB, but none of them is strictly correct: all of them indicate the smallint columns as int.
Not a big deal, actually, but still incorrect.
As usual, suggestions and comments are welcome.
This means that now it’s officially supported and you can use it in production code.
After reading the post on the CSS blog, I started to wonder whether there is some actual use in production for this query hint, given that it requires the same privileges as DBCC TRACEON, which means you have to be a member of the sysadmin role.
In fact, if you try to use that hint when connected as a low privileged user, you get a very precise error message, that leaves no room for interpretation:
SELECT * FROM [AdventureWorks2012].[Person].[Person] OPTION (QUERYTRACEON 4199)
Msg 2571, Level 14, State 3, Line 1
User ‘guest’ does not have permission to run DBCC TRACEON.
How can a query hint available to sysadmins only be possibly useful for production?
My concerns were not about the usefulness of the hint per se, but about the usefulness in production code. Often 140 chars are not enough when you want to express your thoughts clearly, in fact I decided to write this blog post to clarify what I mean.
As we have seen, the QUERYTRACEON query hint cannot be used directly by users not in the sysadmin role, but it can be used in stored procedures with “EXECUTE AS owner” and in plan guides.
While it’s completely clear what happens when the hint is used in procedures executed in the context of the owner, what happens in plan guides is not so obvious (at least, not to me). In fact, given that the secuirty context is not changed when the plan guide is matched and applied, I would have expected it to fail miserably when executed by a low privileged user, but it’s not the case.
Let’s try and see what happens:
First of all we need a query “complex enough” to let the optimizer take plan guides into account. A straight “SELECT * FROM table” and anything else that results in a trivial plan won’t be enough.
SELECT * FROM [Person].[Person] AS P INNER JOIN [Person].[PersonPhone] AS H ON P.BusinessEntityID = H.BusinessEntityID INNER JOIN [Person].[BusinessEntity] AS BE ON P.BusinessEntityID = BE.BusinessEntityID INNER JOIN [Person].[BusinessEntityAddress] AS BEA ON BE.BusinessEntityID = BEA.BusinessEntityID WHERE BEA.ModifiedDate > '20080101'
Then we need a plan guide to apply the QUERYTRACEON hint:
EXEC sp_create_plan_guide @name = N'[querytraceon]', @stmt = N'SELECT * FROM [Person].[Person] AS P INNER JOIN [Person].[PersonPhone] AS H ON P.BusinessEntityID = H.BusinessEntityID INNER JOIN [Person].[BusinessEntity] AS BE ON P.BusinessEntityID = BE.BusinessEntityID INNER JOIN [Person].[BusinessEntityAddress] AS BEA ON BE.BusinessEntityID = BEA.BusinessEntityID WHERE BEA.ModifiedDate > ''20080101''', @type = N'SQL', @hints = N'OPTION (QUERYTRACEON 4199)'
If we enable the plan guide and try to issue this query in the context of a low privileged user, we can see no errors thrown any more:
CREATE LOGIN testlogin WITH PASSWORD = 'testlogin123'; GO USE AdventureWorks2012; GO CREATE USER testlogin FOR LOGIN testlogin; GO GRANT SELECT TO testlogin; GO EXECUTE AS USER = 'testlogin'; GO SELECT * FROM [Person].[Person] AS P INNER JOIN [Person].[PersonPhone] AS H ON P.BusinessEntityID = H.BusinessEntityID INNER JOIN [Person].[BusinessEntity] AS BE ON P.BusinessEntityID = BE.BusinessEntityID INNER JOIN [Person].[BusinessEntityAddress] AS BEA ON BE.BusinessEntityID = BEA.BusinessEntityID WHERE BEA.ModifiedDate > '20080101'; GO REVERT; GO
If we open a profiler trace and capture the “Plan Guide Successful” and “Plan Guide Unsuccessful” events, we can see that the optimizer matches the plan guide and enforces the use of the query hint.
Lesson learned: even if users are not allowed to issue that particular query hint directly, adding it to a plan guide is a way to let anyone use it indirectly.
Bottom line is OPTION QUERYTRACEON can indeed be very useful when we identify some queries that obtain a decent query plan only when a specific trace flag is active and we don’t want to enable it for the whole instance. In those cases, a plan guide or a stored procedure in the owner’s context can be the answer.