Monthly Archives: May 2011

Setup failed to start

Yesterday I ran into this error message while installing a new SQL Server 2005 instance on a Windows 2003 cluster:

Setup failed to start on the remote machine. Check the Task scheduler event log on the remote machine.

SQL Server 2005 setup, in a clustered environment, relies on a remote setup process started on the passive nodes through a scheduled task:

For some weird reason, the remote scheduled task refuses to start if there is an active RDP session on the passive nodes. This KB article describes symptoms and resolution in detail:

If you look into the Scheduled Tasks log you will find something similar to this:

sql server Task did not appear to start on machine <machine_name> 267015

Needless to say that one of my co-workers was logged on the desktop of one of the passive nodes.  So, if you need another good reason to lock everyone out of your servers’ desktop, here it is.

After resetting the offending session, remember to log on to all the passive nodes and kill the zombie setup processes you may have left. Also, delete the scheduled tasks from each node.

Changing Server Collation

In order to avoid collation conflict issues with TempDB, all user databases on a SQL Server instance should be set to the same collation.

Temporary tables and table variables are stored in TempDB, that means that, unless explicitly defined, all character-based columns are created using the database collation, which is the same of the master database. If user databases have a different collation, joining physical tables to temporary tables may cause a collation conflict.

For instance you could have a user database with collation SQL_Latin1_General_CP1_CI_AS on a server with collation Latin1_General_CI_AS:

SELECT  name,
        is_system =
            WHEN name IN ('master', 'model', 'tempdb', 'msdb') THEN 1
            ELSE 0
FROM    sys.databases
ORDER BY is_system DESC, name ASC

Every time you create a temporary table and join it to a permanent table in this database, you have to ensure that the collations match, or you will run into a collation conflict:

USE dblocal

    testColumn varchar(10)

INSERT INTO TestTable VALUES ('conflict')

    tempColumn varchar(10)

INSERT INTO #TempTable VALUES ('conflict')

FROM TestTable
    ON testColumn = tempColumn

The only ways to avoid the conflict are:

  • Define explicitly the collation on the temporary table with a collation name or database_default
  • Add the COLLATE clause to the join predicate, using a collation name or database_default
-- OPTION 1: define the collation when creating the table
    tempColumn varchar(10) COLLATE database_default

-- OPTION 2: force a specific collation when querying the table
FROM TestTable
    ON testColumn = tempColumn COLLATE database_default

The first option must be preferred when possible, because it doesn’t need performing scalar calculations that end up making any index referencing the column virtually useless.

However, both methods require editing the code, which is not always possible. In those cases, changing the server collation is the only possible solution.

In the SQL Server 2000 days, there was a “nice” tool called rebuildm, that recreated the master database and could achieve this task simply. From SQL Server 2005 on, the only tool to rebuild system databases is the setup utility.

This is what Microsoft recommends to rebuild system databases:

/SQLSYSADMINACCOUNTS=accounts /[ SAPWD= StrongPassword ]

Be warned that blogs and forums are filled with bad advice on this topic. The most frequent ones are these:

  1. Change the collation of the model database
    TempDB is created from model when the service starts, which makes this method look smart. It’s not! Changing the database collation for model doesn’t change the collation of the objects inside the database: you would end up with a messy template for every new database.
  2. Copy the model database from another instance with the desired collation.
    It could work, given that you copy the database from an instance with the exact same @@version. Even if you managed to copy the database, tempdb would have a collation other than those of master and msdb, which would mean moving the original problem among the system databases. Believe me, this is not what you want.

The recommended method requires running the setup and usually is not a big deal. This blog on MSDN (CSS Support Engineers) describes in depth how the setup works and how to troubleshoot some common issues:

An undocumented method

There seems to be another undocumented method, mentioned (and advertised as safe) by Paul Randal (blog | twitter) in one of his presentations: (page 17). The method relies on a startup parameter that allows changing the server collation when combined with a couple of trace flags:

 sqlservr -m -T4022 -T3659 -q"new collation" 

Trace flag 3659 allows logging all errors to sql server logs
Trace flag 4022 forces SQL Server to skip startup stored procedures (if you have any).
Startup option “-m” forces single user mode.
Startup option “-q” rebuilds all databases and contained objects into the specified collation, without reinstalling the instance.

Besides being undocumented, this method seems to have some pitfalls. Gail Shaw (blog | twitter) raises a couple of warnings on this thread at SqlServerCentral:

So, why recommend a method that could fail or cause harm to your instance? There is a scenario where this method comes extremely handy, mainly because the documented and supported method fails, which is a big SQL Server 2005 cluster.

One of the biggest innovations in SQL Server 2008 setup concerned the cluster installation: in SQL Server 2005 the setup process was run against all passive nodes and then against the active node for the instance, while SQL Server 2008 setup runs only against the local machine and has to be repeated for each node in the cluster.

Running the setup on all the passive nodes, as SQL Server 2005 setup does, can be a pain in the rear. The more nodes you have, more things can go wrong.

Despite all my efforts, I was unable to run the setup successfully with rebuild database action on a SQL Server 2005 cluster. The log files did not report anything useful for troubleshooting: they were only saying that one of the passive nodes could not complete the setup. I was ready to uninstall the instance and install from scratch when I came across the undocumented –q parameter. I tried (I had nothing to lose, after all) and it worked.

I hope it can work for you too.

Trace Flag 3659

Many setup scripts for SQL Server include the 3659 trace flag, but I could not find official documentation that explains exactly what this flag means.
After a lot of research, I found a reference to this flag in a script called AddSelfToSqlSysadmin, written by Ward Beattie, a developer  in the SQL Server product group at Microsoft.

The script contains a line which suggests that this flag enables logging all errors to errorlog during server startup. I’m unsure of what kind of errors are not logged without setting the flag, but I didn’t find any further reference. The BOL page for Trace Flags doesn’t list this one, so if you happen to know something more, feel free to add a comment here.