2

We have several "reporting" tables in a SQL Server 2012 SP1 instance. Every hour, a job runs procedures that delete all the rows from these tables and then insert current data (involving lots of recursion and aggregations) into them. Each contains thousands of rows. Each such table is queried many times per day. The statistics on both the clustered and non-clustered indexes on these tables show outrageous things like 94,000% of rows updated, with the last statistics update being several days ago. Auto Update of statistics is set to true.

I would have thought that the statistics would get updated every time the table was queried for the first time after the contents were last replaced, but that apparently is not happening. Why?

Mark Freeman
  • 2,181
  • 5
  • 30
  • 53
  • 3
    Statistics for what object? Clustered index? Non-clustered index? Are you sure the query actually used the index you're checking? – Aaron Bertrand Mar 04 '15 at 21:37
  • are you using DBCC SHOW_STATISTICS(<tablename>,<statsname>) to look at the details, or are you using GUI? – Hannah Vernon Mar 05 '15 at 00:22
  • Max: I am using a query that joins dm_db_partition_stats, sys.objects, sys.stats, and sysindexes. DBCC SHOW_STATISTICS shows the same rowcount and updated date as my query. – Mark Freeman Mar 05 '15 at 15:21
  • Aaron: I edited my post to show that I'm referring to both clustered and nonclustered indexes. I am using a query that uses sys.dm_exec_cached_plans with a CROSS APPLY on sys.dm_exec_query_plan and query_plan.nodes to verify that the indexes are being used. I've even run some of the queries myself just to ensure they were run recently, but the stats don't update as a result. – Mark Freeman Mar 05 '15 at 15:24
  • Stats updates are based on the total number of changes. Your delete/insert may not be exceeding that threshold. – Robert L Davis Mar 12 '15 at 18:49
  • If my previous comment is true, then you may want to either update stats manually after you reload the table or consider using trace flag 2371 to trigger it sooner. http://support.microsoft.com/en-us/kb/2754171 – Robert L Davis Mar 12 '15 at 18:52
  • Robert: One of the indexes has 104,597 rows, of which 22,801,507 have been modified (21,799% modified). Trace flag 2371 is in effect. The table content is completely replaced hourly, and the stats were last updated 4 days ago. It's a mystery to me why Auto Update is missing it. I may just have to take your suggestion of modifying the stats as part of the proc that replaces the rows. – Mark Freeman Mar 12 '15 at 20:26
  • @Mark just wondering if you've found resolution on this yet – swasheck Aug 11 '15 at 22:02
  • @swasheck: No. I've either modified the procs to update the stats after repopulating the table or added the update statistics to a nightly job, depending on the frequency of usage. – Mark Freeman Aug 31 '15 at 19:23
  • @Mark how are you calculating the modifications? – swasheck Aug 31 '15 at 19:41
  • I get the numbers from a "SQL Server Ineffective Statistics" alert from SolarWinds' Database Performance Analyzer (formerly Confio Ignite). – Mark Freeman Sep 01 '15 at 14:02
  • @MarkFreeman is it possible that there is a delay in when Confio is getting its dataset? Have you verified SolarWinds' alert against what's actually occurring on the server? – swasheck Jul 13 '16 at 16:37

1 Answers1

2

Unfortunately after searching SolarWinds' KB for about 10 minutes I cannot find what "Ineffective Statistics" measures. That information would be helpful. However, based on your question I'll infer that it means that the Statistics objects may be out of date or have taken quite a few modifications and Database Performance Analyzer has not detected an update of some sort. Without more information from the product vendor, it's hard to tell.

Having said that, there are a few things that you should know.

First, you can verify what's going on with your server right now (e.g. when you get the alert) and compare it against your monitoring tool. Here's a script that I use:

use [mydatabasename];
go
select 
    [schema_name] = sh.name, 
    [table_name] = t.name, 
    [stat_name] = s.name,
    [column_names] = stuff((
                              select ','+index_col(t.name,sc.stats_id,sc.stats_column_id)
                              from sys.stats_columns sc
                              where sc.object_id = s.object_id
                                and sc.stats_id = s.stats_id    
                               order by sc.stats_column_id        
                              for xml path('')), 1, 1, '' 
                            ) ,
    s.stats_id,     
    sp.last_updated, 
    sp.rows, 
    sp.rows_sampled, 
    sp.unfiltered_rows, 
    [sample_rate] = (100.*sp.rows_sampled)/sp.unfiltered_rows,
    modification_counter,
    [default_mod_threshold] = case when sp.unfiltered_rows < 500
                                                   then 500
                                                   else ( .20
                                                          * sp.unfiltered_rows
                                                          + 500 )
                                              end ,
    [2371_mod_threshold] = sqrt(1000 * sp.unfiltered_rows),
    [default_mod_hit] = case when sp.modification_counter > case
                                                              when sp.unfiltered_rows < 500
                                                              then 500
                                                              else ( .20
                                                              * sp.unfiltered_rows
                                                              + 500 )
                                                              end then 1
                                           else 0
                                      end ,
    [2371_mod_hit] = case when sp.modification_counter > sqrt(1000
                                                              * sp.unfiltered_rows)
                                          then 1
                    end

from sys.stats s 
join sys.tables t 
    on s.object_id = t.object_id
join sys.schemas sh
    on t.schema_id = sh.schema_id
cross apply sys.dm_db_stats_properties(s.object_id,s.stats_id) sp
order by t.name, s.name;

Most of it is self-explanatory. However, I added the modification threshold for a statistics object at default and with TF 2371 included. The default threshold is an interesting calculation, while TF 2371 is the square root of 1000 * Table Cardinality. I included a calculation to include whether or not each of these thresholds had been hit yet. So run that inside the alerted database and see if what SolarWinds is seeing comports with what is occurring on the server right now. If it isn't then there's probably a timing issue in there somewhere such that the alert was generated before a stats update but after a massive amount of data change.

Having said that, it's important to know how AUTO_UPDATE of statistics actually works. There isn't a clock-like sweep that goes through and constantly updates statistics. Instead, the statistics are updated when the statistic object is loaded to compile a query plan. If the statistic is rarely/never used then it won't be updated unless you manually update it or if it happens to get caught up in an index rebuild as a dependent object for the index. However, if you have an index that's never getting used then that's a different situation altogether.

So in summary:

  1. Verify what the alert is telling you
  2. Know the factors that lead to a stats update
swasheck
  • 10,665
  • 4
  • 47
  • 88