[Looking for Charlie's main web site?]

Helpful info on SQL Server Diagnostics: Part II

Last week I pointed out a useful blog entry on SQL Server troubleshooting. Now there's a Part II, which addresses such things as:

  • Are there SQL batches that actually run for a long time?
  • Is there blocking in the database?
  • Are there any problems in the configuration options of SQL Server?
  • Is there a disk IO problem?
  • Are there any indications of memory problems?

Good stuff that even we CFers can benefit understanding. Again, while it's from a Microsoft support engineer who focuses on .NET debugging in her blog, there's really nothing about this blog entry or the last that's .NET specific.

Helpful info on SQL Server Diagnostics

Here's a useful blog entry, "What if I suspect that my performance problem is in SQL server?". It's easy to blame CF for performance problems, but sometimes the problem is in the database--and it could be configuration, or database setup, all in addition to your own (or someone else's) SQL coding.

The entry focuses on using the tools known variously as PSSDIAG and SQLDIAG, either built-into SQL Server 2005 or available for free download for older editions, and it does a nice job of walking through it in a friendly way, with screenshots and more.

The entry is on the blog of Tess, a Microsoft support engineer, called If broken it is, fix it you should. While the majority of her entries are on .NET, this was a guest blog entry with a SQL Server support engineer. Despite her blog being mostly about .NET, there are occasional gems like this which are of equal value to CFers.

Resources for getting a much greater understanding of the SQL Server query plan/procedure cache

Ever wanted to understand the SQL Server query plan/procedure cache better? It can be vital to good performance of SQL statements, especially with respect to the concept of "bind variables" or "parameterized queries" (using CFQUERYPARAM in a CFML context, or Parameters.Add() in a C# context, among other ways).

Yet we so often just kind of assume the database will do what's best. Or we blindly use (or recommend use of) bind variables without fully understanding why, nor understanding some implications when it may not always be such a good idea.

And did you know that if you can't (or don't) change your code to cause parameterization, did you know that the database can do it for you, either automatically in some simple cases, or by way of an enforced parameterization, either for an entire table of for queries that meet certain "plan guides"? There are many implications to understand in all this.

And how do you track the plan cache (using DMVs, the profiler, perfmon)? And how does it work (allocation of memory, flushing the cache automatically or manually)?

Fortunately, I've found many great resources to help you understand, and I've not found them listed (all together) anywhere else yet.

The info applies to 2005, 2000, and 7, though some aspects may differ, as the resources indicate. (There are even substantive changes in 2005 SP2 that are important to note over the RTM and SP1 releases.)

Huge Plan Caching Article Series by MS Engineers

First, I'll point out that some MS engineers have put together a long and very resourceful series of article-length blog posts on the topic, with explanations, code, demos, troubleshooting techniques, and lots more.

They do kindly offer a "table of contents" page listing all the topics covered, with bullet points about topics within each entry.

Sadly, that page doesn't offer URLs to the articles, nor have any that I found there, nor do the articles link to each other. You can dig around and find the URLs, even finding a category page that lists them in rather random order. But I've looked around the web and can't find any page that lists them all with their links, in order, so here you go:

  1. Structure of the Plan Cache and Types of Cached Objects
  2. Sql_Handle and Plan_Handle Explained
  3. How Cache Lookups Work
  4. Query Parameterization
  5. Retrieving Query Plans from Plan Cache DMV's
  6. Best Programming Practices
  7. Costing Cache Entries
  8. Factors that affect Batch Cache-ability
  9. Memory Pressure Limits
  10. Plan Cache Flush
  11. Temporary Tables, Table Variables and Recompiles
  12. Plan Cache Trace Events and Performance

They also offer a series of extensions to that, on troubleshooting:

Chapter in "Practical SQL Server 2005 Troubleshooting" book

Next, I'd point out that one of the contributors to that, Bart Duncan, is a Microsoft Support engineer who I saw speak on the topic at the SQL Server PASS 2006 conference, where I first learned a lot of this info.

He identified then that a lot of the info was in a chapter of a new book, SQL Server 2005 Practical Troubleshooting: The Database Engine, which I've since gotten and was indeed very useful.

If you're a member of the O'Reilly Safari service (or join for their trial), you can find the chapter online.

Of course, both the talk and the book came out before the article series above, and certainly before SP2, so I would recommend you consider both. (I've not yet done a comparison to determine what may be in the chapter that's not in the articles.)

SQL Server BooksOnline

Of course, it always pays to read the docs, and there are indeed some discussions of this concept and related features in the SQL Server Books Online, but I honestly found the info above either easier to find or just more complete.

My Own Talks

I've made my own meager attempts to communicate some of this info myself in a couple of talks I've givento both CFML and SQL Server audiences. But really, there is just too much to communicate in any one hour. If this topic seems of interest, take a few hours to digest all the above. I think you'll be AMAZED at what you learn.

New (free) Performance Dashboard for SQL Server 2005 SP2

Those using SQL Server 2005 may want to take note of a new "Dashboard Report" option for SP2, to help monitor and resolve performance problems, including capturing diagnostic info when a problem is detected.

Common performance problems that the dashboard reports may help to resolve include:

  • CPU bottlenecks (and what queries are consuming the most CPU)
  • IO bottlenecks (and what queries are performing the most IO).
  • Index recommendations generated by the query optimizer (missing indexes)
  • Blocking
  • Latch contention

The report is an extension of the Custom Reports feature introduced in the SQL Server 2005 SP2 release of SQL Server Management Studio. Note that Reporting Services does not need to be installed.

The reports retrieve info from dynamic management views. They don't poll performance counters or require tracing be enabled. They also do not store a history of performance over time. So it's a lightweight (yet powerful) monitoring option.

You can get the extension itself at:

Performance Dashboard Reports

There's also a complete article about how to install it from Black Belt Administration: Performance Dashboard for Microsoft SQL Server, Part I , at Database Journal (and from which I obtained the image above).

Have you sought a keyboard shortcut to "open table" in SQL Server 2005 Mgt Studio?

I'm a huge fan of keyboard shortcuts, so imagine my dismay when I noticed that the new SQL Server 2005 "open table" option, available in Management Studio when you're viewing the tables in a database, had no keyboard shortcut (or Admin menu equivalent). The feature opens an editable grid of data in the table, which is a great when you need to do a quick fix of the data. But you have to right-click to see the option--I wonder how many never even notice it?

So I asked around and got an answer to my keyboard dilemma which actually is a generic windows solution. Did you know that you can get the equivalent of the right-click by using Shift-f10? Whatever you have the keyboard focus on, it will open its corresponding context menu. Very nice.

So in SQL Mgt Studio, open the database, then its tables, then select the table (all of which can be done with the keyboard), and then use shift-F10. You'll suddenly see that each context menu option shows the standard underline under the key to hit to execute that command (it's the "o" for open table).

Hope that may help others.

Need to migrate an Access DB into SQL Server? Here's a solution you may have missed?

If you have need to migrate an Access DB to SQL Server 2005, Microsoft has a free tool to help, which it seems many may miss.

Check out: http://www.microsoft.com/sql/solutions/migration/default.mspx#EYC

Thanks to Teddy Payne for pointing it out in his blog entry, which also offers an additional document he's put together on the migration steps.

Solving error connecting to SQL Server 2005 from CFMX 6.1/7 on Localhost

Are you getting the error, "Connection refused" or "Error establishing socket to host and port", trying to connect to a SQL Server 2005 database in CFMX 6.1 or 7? Or the error, "SQL Server does not exist or access denied", in CF5 using OLEDB? I have a solution.

No, I'm not just going to recommend creating an ODBC datasource, as many others have. :-) There is a real solution. The short answer is to open the "SQL Server Configuration Manager" in SQL Server 2005, and ensure that TCP/IP is enabled as a protocol.

The rest of this entry explains additional details, such as how to find and make that change, what specific errors you get, and how I found the information, in case any of it helps others.

I should add that I don't have SQL 2000 against which to test how things are similar or different, but I do point to some CFMX docs that may apply to that version. In any case, hope this will serve all those CFML developers making the move to SQL Server 2005.

The Errors You May See

To help ensure that future readers can find this entry more readily when doing web searching, let me offer details on the error. The problem I'm referring to is if you get any of the following errors.

In the CFMX 7 admin, when adding, editing, or verifying a SQL Server 2005 datasource, you get the following error:

Connection verification failed for data source: blogcfcsql java.sql.SQLException: [Macromedia][SQLServer JDBC Driver]Error establishing socket. Connection refused: connect The root cause was that: java.sql.SQLException: [Macromedia][SQLServer JDBC Driver]Error establishing socket. Connection refused: connect

Or if you run code on CFMX 7 that tries to use such a datasource and get:

Error Executing Database Query. Macromedia][SQLServer JDBC Driver]Error establishing socket. Connection refused: connect

In CFMX 6.1 you get this slightly different error when adding or editing the DSN in the admin:

Connection verification failed for data source: blogcfcsql []java.sql.SQLException: [Macromedia][SQLServer JDBC Driver]Error establishing socket. Connection refused: connect The root cause was that: java.sql.SQLException: [Macromedia][SQLServer JDBC Driver]Error establishing socket. Connection refused: connect

When you simply verify the DSN in CFMX 6.1, it only says "error" in the status column. You need to edit and submit the DSN to see the error above.

And if you run code trying to use such a datasource in CFMX 6.1, you get:

[Macromedia][SQLServer JDBC Driver]Error establishing socket. Connection refused: connect

I don't have CFMX 6.0 installed to see what happens there.

Lastly, in CF5, while most use ODBC datasources there, if you did try to create an OLEDB datasource using the SQLOLEDB provider, in the circumstance I'm describing it would just say "failed" if you try to verify it, and "The connection to the data source failed" if you edit the DSN and submit it.

In code using the DSN you would get:

OLEDB Error Code = 17 [DBNETLIB][ConnectionOpen (Connect()).]SQL Server does not exist or access denied.

Update for CF8

Here's what the error is in CF8 might look like:

Connection verification failed for data source: blogcfcsql java.sql.SQLException: [Macromedia][SQLServer JDBC Driver]Error establishing socket to host and port: [server]:[port]. Reason: Connection refused: connect The root cause was that: java.sql.SQLException: [Macromedia][SQLServer JDBC Driver]Error establishing socket to host and port: [server]:[port]. Reason: Connection refused: connect

So, those are the errors. What's the solution? :-)

Finding the Solution

After some digging around, where folks offered all sorts of remedies (including recommending you just punt and create an ODBC datasource instead), I finally found the CFMX docs, Configuring and Administering ColdFusion MX, in its section, "Connecting to Microsoft SQL Server":

http://livedocs.macromedia.com/coldfusion/7/htmldocs/00001743.htm#1278307

There it offered a few recommendations where "the following situations can cause a Connection Refused error". The one that caught my attention was:

SQL Server does not enable the TCP/IP protocol. This problem can happen when SQL Server is on the same computer as ColdFusion MX.

Well I am running both CFMX and SQL Server on my laptop (development mode for each), so this sounded encouraging. It would also explain why different folks experience different things, if they have SQL running elsewhere and/or in non-development mode. Indeed, further reading (discussed later) shows that there are differences in implementations of SQL 2005 that would influence whether you'd get this error.

Anyway, the CFMX docs go on to say you should make sure that TCP/IP is not listed as a "disabled protocol", which it indicated can happen in a local (development) mode installation of SQL Server.

Unfortunately, the steps they show to check and correct that are not appropriate for SQL Server 2005. (I will add a comment there pointing back to this blog entry.)

But in searching the SQL 2k5 books online, I found an entry that does explain this issue (following is a local link if you have BOL installed yourself):

ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/udb9/html/ad98873f-8478-4bce-8abf-c24f5d111144.htm

It explains how things will be set for different versions of SQL Server 2005 (Enterprise, Standard, Workgroup, Developer, Evaluation, Express) and whether you have a previous SQL Server version that you are or are not upgrading. Each of these has different defaults for if TCP/IP is enabled.

The Solution in SQL Server 2005

BOL goes on to indicate that we need to go into the SQL Server Configuration Manager to configure the network protocols. There are 2 ways to do this: either right-click on the server in Mgt Studio and choose "SQL Server Configuration Manager" or use Start>Programs>Microsoft SQL Server 2005>Configuration Tools>SQL Server Configuration Manager.

From the window that opens, select the "SQL Server 2005 Network Configuration" node, and its "Protocols For [yourserver]". That lists the various protocols (including TCP/IP and named pipes), and sure enough, for me it showed disabled.

I right-clicked the TCP/IP protocol name and andose "enable", which reminded that I needed to restart the SQL Server for the change to take effect (right-click on the server in SQL Management Studio and choose "restart"). I did that, and then reran my code and all was well (and the datasource would now verify in the Admin console.)

Phew! Hope this helps someone else.

Update

I was working on another machine where I had this error, and this time, I had to take one more step in enabling the port, as the step above just wasn't enough.

On that TCP/IP protocol, where I right-clicked to enable, that right-click also offers a Properties option (which is another way you could choose to enable/disable the protocol). But the window that pops up also has a IP Addresses tab. Choosing that, I saw several various indications of IP addresses, with settings to both make them active and enabled, as well as indicate the port they should use. I tried changing various ones, but in the end found that the only solution was to change the last one, IPAll, and add my SQL Server port (1433) in its TCP Port field. After doing that, and restarting, all was well.

There may be (indeed, likely are) important security considerations that should go into enabling that option. Since this is just my local development box, in a firewall, I won't bother to investigate the, but others here should.

Some Closing Additional Thoughts

Again, some posts I'd found suggested that the solution was to use an ODBC connection instead. That surely works, but most regard an ODBC connection as less performant, and involving more communications overhead per transaction. Also, I have seen code that failed using an ODBC connection that worked fine with a SQL Server connection.

I should also note that you can change this another way, using the new SQL 2005 "Surface Area Configuration" wizard, and choosing "Configure Surface Area for [your server]", then choose "Surface Area Configuration for Services and connections" and then in its window choose "[your server]>Database Engine>Remote Connections and changing to "Local and Remote Connections" to enable TCP/IP.

I'll note that for some reason, BlueDragon's SQL Server driver (based on New Atlanta's JTurbo Type 4 JDBC driver for SQL Server) didn't have this problem. Perhaps someone with more experience in the underlying implementation can explain why the two are so different.

Finally, one other comment: if the error you're getting is instead:

Error Executing Database Query. [Macromedia][SQLServefr JDBC Driver]Error establishing socket. Unknown host: (local)

That is a different problem. The online help in the CFMX Admin suggests that you can name the server as "(local)", but I find I get the aforementioned error. Changing it to the server name or IP, such as 127.0.0.1 for my localhost, solved the problem. (The help also indicates this an option, so I'm just speaking to those who try the "(local)" value.)

One other thing: another problem could be that SQL Server is listening on a port other than what you think. Don't presume it's the standard 1433. Someone could have changed it on install, for security reasons, or some other config feature could have changed it. You can see what the port is by using that same SQL Server Configuration Manager, but rather than use the "SQL Server 2005 Network Configuration" option in the left tree, choose the "Client Protocols", then right-click on TCP, and choose Properties. It will list the "Default Port". Hope all that's helpful to some.