So we're running into an issues where TempDB's log is currently locked up. After checking log_reuse_wait_desc, we see that it is locked from an active transaction. Now the weird part is that when we run DBCC OPENTRAN to get information on the query causing the lock, the command returns "No active open transactions". We've killed every spid connecting to TempDB and we're at the point where we'll be cycling the instance to resolve the issue. Does anyone understand what's happening here or if there's a better way to deal with this?
Asked
Active
Viewed 2,069 times
2
2 Answers
2
So we figured it out:
We rebooted the server and when it came back up, we were still having issues. We started re-considering whether it was just an issue with TempDB and turned to maybe trying to catch the issues with extended events. While looking into it, we stumbled across an extended event that nobody knew about called "Testing." We disabled that XE and, go figure, everything started working fine again. I'm not sure WHY this locked up everything in TempDB and we're looking into understanding the issue, but for now - the issue is resolved.
8bit
- 388
- 1
- 11
-
we stumbled across an extended event that nobody knew about called "Testing."- Testing should not be on PROD :-) – Kin Shah Dec 03 '15 at 22:54 -
Oh, things I know @Kin. If only there were a way to track who created it... – 8bit Dec 03 '15 at 22:56
0
Can you see what's running in tempdb?
;WITH task_space_usage AS (
-- SUM alloc/delloc pages
SELECT session_id,
request_id,
SUM(internal_objects_alloc_page_count) AS alloc_pages,
SUM(internal_objects_dealloc_page_count) AS dealloc_pages
FROM sys.dm_db_task_space_usage WITH (NOLOCK)
WHERE session_id <> @@SPID
GROUP BY session_id, request_id
)
SELECT TSU.session_id,
TSU.alloc_pages * 1.0 / 128 AS [internal object MB space],
TSU.dealloc_pages * 1.0 / 128 AS [internal object dealloc MB space],
EST.text,
-- Extract statement from sql text
ISNULL(
NULLIF(
SUBSTRING(
EST.text,
ERQ.statement_start_offset / 2,
CASE WHEN ERQ.statement_end_offset < ERQ.statement_start_offset
THEN 0
ELSE( ERQ.statement_end_offset - ERQ.statement_start_offset ) / 2 END
), ''
), EST.text
) AS [statement text],
EQP.query_plan
FROM task_space_usage AS TSU
INNER JOIN sys.dm_exec_requests ERQ WITH (NOLOCK)
ON TSU.session_id = ERQ.session_id
AND TSU.request_id = ERQ.request_id
OUTER APPLY sys.dm_exec_sql_text(ERQ.sql_handle) AS EST
OUTER APPLY sys.dm_exec_query_plan(ERQ.plan_handle) AS EQP
WHERE EST.text IS NOT NULL OR EQP.query_plan IS NOT NULL
ORDER BY 3 DESC;
Nicolas de Fontenay
- 1,873
- 2
- 18
- 36
-
2
-
2This will only return active requests, not those that are idle/sleeping. This could be a
BEGIN TRANSACTION; SELECT ...;that "finished" last Tuesday, and it won't show up insys.dm_exec_requests... – Aaron Bertrand Dec 02 '15 at 23:50 -
Like mentioned above, all this returns are some of the active transactions we can track - none of which seem to be causing the lock (Just a few spids that have kicked off in the past hour or so) – 8bit Dec 02 '15 at 23:58
-
1
-
Yes I had found it here. Is that yours spaghettidba? I have seen a few good things from you peppered across dba.stackexchange. I should read all comments first. @AaronBertrand I had to use it a lot when we struggled with ASP State database causing pain in tempdb. – Nicolas de Fontenay Dec 03 '15 at 18:40
-
What @spaghettidba was gently trying to tell you is you should have stated where you got the SQL. Notice how Aaron provided an attribution in the original post. – Erik Dec 04 '15 at 03:08
-
No problem, I'm fine with anybody using my code, as long as they find it useful. – spaghettidba Dec 04 '15 at 09:19
sys.dm_tran_active_transactions? Just becauseDBCC OPENTRANreturns 0 rows doesn't mean no transactions exist. Try this:USE msdb; BEGIN TRANSACTION; SELECT * INTO #foo FROM dbo.sysjobs;Now tryDBCC OPENTRAN... – Aaron Bertrand Dec 02 '15 at 23:37DBCC FREEPROCCACHE -- clean cache DBCC DROPCLEANBUFFERS -- clean buffers DBCC FREESYSTEMCACHE ('ALL') -- clean system cache DBCC FREESESSIONCACHE -- clean session cache. Also, check sessions using sys.dm_exec* dmv. – Kin Shah Dec 02 '15 at 23:40sys.dm_tran_active_snapshot_database_transactionsand see if you have any orphaned transaction -- or - you can download sp_whoisactive and run it with @show_sleeping_spid = 1(run with @help =1 to show other help parameters … since I am not sure if it is @show_sleeping_spid .. I can check later). – Kin Shah Dec 03 '15 at 04:21