Skip to content

Commit 82f5241

Browse files
uc-msftuc-msft
authored
Merge pull request #653 from bluefooted/in-memory-database
Adding TempDB Demo Files
2 parents 9025f1b + a831428 commit 82f5241

7 files changed

Lines changed: 828 additions & 0 deletions

‎samples/features/in-memory-database/memory-optimized-tempdb-metadata/00a-AdventureWorks_Setup.sql‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
-- SQL 2019
2+
3+
-- Restore AdventureWorks
4+
RESTORE DATABASE [AdventureWorks]
5+
FROM DISK = N'F:\MSSQL\MSSQL15.SQL2019\MSSQL\Backup\AdventureWorks2016_EXT.bak'
6+
WITH FILE = 1
7+
, MOVE N'AdventureWorks2016_EXT_Data' TO N'F:\MSSQL\MSSQL15.SQL2019\MSSQL\DATA\AdventureWorks_Data.mdf'
8+
, MOVE N'AdventureWorks2016_EXT_Log' TO N'F:\MSSQL\MSSQL15.SQL2019\MSSQL\DATA\AdventureWorks_Log.ldf'
9+
, MOVE N'AdventureWorks2016_EXT_mod' TO N'F:\MSSQL\MSSQL15.SQL2019\MSSQL\DATA\AdventureWorks_mod'
10+
, NOUNLOAD, STATS = 5
11+
12+
13+
-- Turn off auto stats
14+
USE [master]
15+
GO
16+
ALTER DATABASE [AdventureWorks] SET AUTO_CREATE_STATISTICS OFF
17+
GO
18+
ALTER DATABASE [AdventureWorks] SET AUTO_UPDATE_STATISTICS OFF WITH NO_WAIT
19+
GO
20+
21+
-- Upgrade compatibility
22+
ALTER DATABASE AdventureWorks
23+
SET COMPATIBILITY_LEVEL = 150;
24+
GO

‎samples/features/in-memory-database/memory-optimized-tempdb-metadata/01b-Create_EmployeeBirthdayList.sql‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
USE AdventureWorks
2+
GO
3+
4+
CREATE OR ALTER PROCEDURE usp_EmployeeBirthdayList @month int AS
5+
BEGIN
6+
7+
IF OBJECT_ID('tempdb..#Birthdays') IS NOT NULL DROP TABLE #Birthdays;
8+
9+
CREATE TABLE #Birthdays (BusinessEntityID int NOT NULL PRIMARY KEY);
10+
11+
INSERT #Birthdays (BusinessEntityID)
12+
SELECT BusinessEntityID
13+
FROM HumanResources.Employee
14+
WHERE MONTH(BirthDate) = @month
15+
16+
SELECT p.FirstName, p.LastName, a.AddressLine1, a.AddressLine2, a.City, sp.StateProvinceCode, a.PostalCode
17+
FROM #Birthdays b
18+
INNER JOIN Person.Person p ON b.BusinessEntityID = p.BusinessEntityID
19+
INNER JOIN Person.BusinessEntityAddress bea ON p.BusinessEntityID = bea.BusinessEntityID
20+
INNER JOIN Person.Address a ON bea.AddressID = a.AddressID
21+
INNER JOIN Person.StateProvince sp ON a.StateProvinceID = sp.StateProvinceID
22+
INNER JOIN Person.AddressType at ON at.AddressTypeID = bea.AddressTypeID
23+
WHERE at.Name = N'Home'
24+
25+
END;

‎samples/features/in-memory-database/memory-optimized-tempdb-metadata/02-get object info from page resource sql 2019.sql‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
-- RESOLVE WAIT RESOURCE TO OBJECT INFORMATION
2+
-- NEW FUNCTIONS AVAILABLE IN SQL SERVER 2019
3+
USE master
4+
GO
5+
SELECT
6+
er.session_id, er.wait_type, er.wait_resource,
7+
OBJECT_NAME(page_info.[object_id],page_info.database_id) as [object_name],
8+
er.blocking_session_id,er.command,
9+
SUBSTRING(st.text, (er.statement_start_offset/2)+1,
10+
((CASE er.statement_end_offset
11+
WHEN -1 THEN DATALENGTH(st.text)
12+
ELSE er.statement_end_offset
13+
END - er.statement_start_offset)/2) + 1) AS statement_text,
14+
page_info.database_id,page_info.[file_id], page_info.page_id, page_info.[object_id],
15+
page_info.index_id, page_info.page_type_desc
16+
FROM sys.dm_exec_requests AS er
17+
CROSS APPLY sys.dm_exec_sql_text(er.sql_handle) AS st
18+
CROSS APPLY sys.fn_PageResCracker (er.page_resource) AS r
19+
CROSS APPLY sys.dm_db_page_info(r.[db_id], r.[file_id], r.page_id, 'DETAILED') AS page_info
20+
WHERE er.wait_type like '%page%'
21+
GO

‎samples/features/in-memory-database/memory-optimized-tempdb-metadata/MemoryOptimizedTempDBMetadata-TSQL.ipynb‎

Lines changed: 490 additions & 0 deletions
Large diffs are not rendered by default.

‎samples/features/in-memory-database/memory-optimized-tempdb-metadata/MemoryOptmizedTempDBMetadata-Python.ipynb‎

Lines changed: 263 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,263 @@
1+
{
2+
"metadata": {
3+
"kernelspec": {
4+
"name": "python3",
5+
"display_name": "Python 3"
6+
},
7+
"language_info": {
8+
"name": "python",
9+
"version": "3.6.5",
10+
"mimetype": "text/x-python",
11+
"codemirror_mode": {
12+
"name": "ipython",
13+
"version": 3
14+
},
15+
"pygments_lexer": "ipython3",
16+
"nbconvert_exporter": "python",
17+
"file_extension": ".py"
18+
}
19+
},
20+
"nbformat_minor": 2,
21+
"nbformat": 4,
22+
"cells": [
23+
{
24+
"cell_type": "markdown",
25+
"source": [
26+
"# Exploring Memory-Optmized TempDB Metadata\r\n",
27+
"\r\n",
28+
"TempDB metadata contention has historically been a bottleneck to scalability for many workloads running on SQL Server. SQL Server 2019 introduces a new feature that is part of the [In-Memory Database](https://docs.microsoft.com/sql/relational-databases/in-memory-database) feature family, memory-optimized tempdb metadata, which effectively removes this bottleneck and unlocks a new level of scalability for tempdb-heavy workloads. In SQL Server 2019, the system tables involved in managing temp table metadata can be moved into latch-free non-durable memory-optimized tables.\r\n",
29+
"\r\n",
30+
"To learn more about tempdb metadata contention, along with other types of tempdb contention, check out the blog article [TEMPDB - Files and Trace Flags and Updates, Oh My!](https://techcommunity.microsoft.com/t5/SQL-Server/TEMPDB-Files-and-Trace-Flags-and-Updates-Oh-My/ba-p/385937). Keep reading to explore the new memory-optimized tempdb metadata feature."
31+
],
32+
"metadata": {
33+
"azdata_cell_guid": "491417cb-b92b-43bc-b87b-7e67bcae5589"
34+
}
35+
},
36+
{
37+
"cell_type": "markdown",
38+
"source": [
39+
"## Configure your environment\r\n",
40+
"\r\n",
41+
"Contention in tempdb happens when a large number of concurrent threads are attemping to create, modify or drop temp tables. In order to simulate this situation, you'll need to have a SQL Server instance that has multiple cores (4 or more is recommended), and a way to simulate multiple concurrent threads. For this example, we'll be using the ostress.exe tool to generate multiple concurrent threads. If you have an existing multi-core SQL Server instance, you can follow the T-SQL instructions to set up the demo, otherwise you can try the docker container steps instead. \r\n",
42+
"\r\n",
43+
"First, download the demo files to your local computer.\r\n",
44+
"\r\n",
45+
"### Docker Container Setup\r\n",
46+
"Note that the docker commands may take some time to execute, but you will not see progress here until they are complete.\r\n",
47+
"\r\n",
48+
"1. Make sure you have your docker environment configured, more information [here](https://docs.docker.com/get-started/). \r\n",
49+
"> NOTE\r\n",
50+
"> <br>If you are using Docker Desktop for Windows or Mac, the default configuration will limit your containers to 2 cores, regardless of the number of cores on your computer. Be sure to configure docker to allow at least 4 cores and 4GB of RAM for this demo to run properly. To do this, right click on the Docker Desktop icon in the status bar and choose Settings -> Advanced.\r\n",
51+
"2. Pull the demo container with the following command:"
52+
],
53+
"metadata": {
54+
"azdata_cell_guid": "c740199e-107b-484a-9994-6883680db75e"
55+
}
56+
},
57+
{
58+
"cell_type": "code",
59+
"source": [
60+
"! docker pull bluefooted/sql2019tempdbdemo"
61+
],
62+
"metadata": {
63+
"azdata_cell_guid": "1c435674-3a29-43db-af14-3bbaeb69cba8"
64+
},
65+
"outputs": [
66+
{
67+
"name": "stdout",
68+
"text": "Using default tag: latest\nlatest: Pulling from bluefooted/sql2019tempdbdemo\nDigest: sha256:035a1bda5539bfe68ad1b2f032a6e389cea91a0cd880e75b83ef186c46b2e34f\nStatus: Image is up to date for bluefooted/sql2019tempdbdemo:latest\ndocker.io/bluefooted/sql2019tempdbdemo:latest\n",
69+
"output_type": "stream"
70+
}
71+
],
72+
"execution_count": 3
73+
},
74+
{
75+
"cell_type": "markdown",
76+
"source": [
77+
"3. Start the demo container with the following command:"
78+
],
79+
"metadata": {
80+
"azdata_cell_guid": "94a1d4c5-806b-4823-bbcc-527a46dcac53"
81+
}
82+
},
83+
{
84+
"cell_type": "code",
85+
"source": [
86+
"! docker run -e \"ACCEPT_EULA=Y\" -e \"SA_PASSWORD=P@ssw0rd!\" -p 1455:1433 --name sql2019tempdbdemo -d bluefooted/sql2019tempdbdemo"
87+
],
88+
"metadata": {
89+
"azdata_cell_guid": "50d697be-9130-4bf8-8071-a670fce06a0c"
90+
},
91+
"outputs": [
92+
{
93+
"name": "stdout",
94+
"text": "2a3dbfff94ce2bd277778e36ac268b4495afb77f1b6c5417478b10a9c545cca6\n",
95+
"output_type": "stream"
96+
}
97+
],
98+
"execution_count": 2
99+
},
100+
{
101+
"cell_type": "markdown",
102+
"source": [
103+
"> NOTE:\r\n",
104+
"> <br> If you see the following error, you may already have run the docker run command with this image:\r\n",
105+
"<br> <br> *docker: Error response from daemon: Conflict. The container name \"/sql2019tempdbdemo\" is already in use by container \"3f662e0fd9b8cbdc1013e874722e066aa8e81ec3a07423fc3ab95cb75e640af9\". You have to remove (or rename) that container to be able to reuse that name.\r\n",
106+
"See 'docker run --help'.*\r\n",
107+
"\r\n",
108+
"If you see this message, you can start the container instead with the following command:"
109+
],
110+
"metadata": {
111+
"azdata_cell_guid": "b713d8ad-99df-4a32-a65e-ab0779575f3b"
112+
}
113+
},
114+
{
115+
"cell_type": "code",
116+
"source": [
117+
"! docker start sql2019tempdbdemo"
118+
],
119+
"metadata": {
120+
"azdata_cell_guid": "e0d6ed1d-c6bc-4f08-a041-b6b70ea2054e"
121+
},
122+
"outputs": [],
123+
"execution_count": 0
124+
},
125+
{
126+
"cell_type": "markdown",
127+
"source": [
128+
"4. Connect to the demo SQL Server instance using Azure Data Studio or SQL Server Management Studio using the following information:\r\n",
129+
" \r\n",
130+
" **Server Name**: localhost,1455<br>\r\n",
131+
" **Username**: sa<br>\r\n",
132+
" **Password**: P@ssw0rd!"
133+
],
134+
"metadata": {
135+
"azdata_cell_guid": "895b3f6a-1ba1-4480-9b06-9cfb2b3c246d"
136+
}
137+
},
138+
{
139+
"cell_type": "markdown",
140+
"source": [
141+
"### Existing SQL Server Instance Setup (skip if you are using the demo container)\r\n",
142+
"\r\n",
143+
"If you already have a SQL Server instance with a minimum of 4 cores, you can download and restore the AdventureWorks database and use the scripts in this repo to configure the database. Follow the steps in the T-SQL notebook to complete this setup.\r\n",
144+
""
145+
],
146+
"metadata": {
147+
"azdata_cell_guid": "0302cf7c-5ee1-426c-b1a5-17fb492a0a8a"
148+
}
149+
},
150+
{
151+
"cell_type": "markdown",
152+
"source": [
153+
"## Detecting TempDB Metadata Contention\r\n",
154+
"\r\n",
155+
"The first thing to figure out before you turn on this new feature is whether or not you are experiencing TempDB metadata contention. The main symptom of this contention is a number of sessions in `Suspended` state with a wait type of `PAGELATCH_xx` and a wait resource of a page that hosts a TempDB system table, such as `2:1:118`. In order to know whether or not the page is part of a TempDB system table, you can use the new dynamic management function `sys.dm_db_page_info()` in SQL Server 2019 or later, or the older `DBCC PAGE` command in older versions. For this demo, let's focus on `sys.dm_db_page_info()`. Note that this command will take some time to complete, you'll want to proceed to the next step before it completes.\r\n",
156+
"\r\n",
157+
"First, start the workload using the `ostress.exe` tool that is included in the downloads. Note that if you are not using the demo container, you will need to change the server name and login information."
158+
],
159+
"metadata": {
160+
"azdata_cell_guid": "7ac2df51-c609-45fc-9e97-996c7dc6b505"
161+
}
162+
},
163+
{
164+
"cell_type": "code",
165+
"source": [
166+
"! ostress.exe -Slocalhost,1455 -Usa -PP@ssw0rd! -dAdventureWorks -Q\"EXEC dbo.usp_EmployeeBirthdayList 4\" -mstress -quiet -n16 -r120 | FINDSTR \"QEXEC Starting Creating elapsed\""
167+
],
168+
"metadata": {
169+
"azdata_cell_guid": "b738ca65-ec24-4762-a773-d37674b41884"
170+
},
171+
"outputs": [
172+
{
173+
"name": "stdout",
174+
"text": "10/08/19 15:39:54.739 [0x00007A74] -QEXEC dbo.usp_EmployeeBirthdayList 4\n10/08/19 15:39:54.779 [0x00007A74] Starting query execution...\n10/08/19 15:39:54.783 [0x00007A74] Creating 16 thread(s) to process queries\n10/08/19 15:40:30.158 [0x00007A74] OSTRESS exiting normally, elapsed time: 00:00:35.419\n",
175+
"output_type": "stream"
176+
}
177+
],
178+
"execution_count": 11
179+
},
180+
{
181+
"cell_type": "markdown",
182+
"source": [
183+
"While the above script is running, switch over to the T-SQL notebook and run the script to monitor your workload for page contention. You should see several sessions with a `wait_type` of `PAGELATCH_EX` or `PAGELATCH_SH`, often with an `object_name` of `sysschobjs`.\r\n",
184+
"\r\n",
185+
"> NOTE\r\n",
186+
"> <br>If this query does not return any results, make sure the command above is still running. If it is not running, start it and try the query again. If you still do not see any sessions waiting, you may need to increase the number of CPUs available to your server, and/or increase the number of concurrent threads by increasing the `-n` parameter in the command. This demo was tested with 4 cores and 16 concurrent sessions, which should yield the expected results. If you would like more time to examine the contention, you can increase the `-r` parameter, which will increase the number of iterations."
187+
],
188+
"metadata": {
189+
"azdata_cell_guid": "a3421322-4a07-4348-88f0-2e870368291b"
190+
}
191+
},
192+
{
193+
"cell_type": "markdown",
194+
"source": [
195+
"## Improve performance with Memory-Optimized TempDB Metadata\r\n",
196+
"\r\n",
197+
"Now that you have observed TempDB metadata contention, let's see how SQL Server 2019 addresses this contention. Switch over to the T-SQL notebook to review and run the script to enable Memory-Optimized TempDB Metadata.\r\n",
198+
"\r\n",
199+
"Once you have run the T-SQL command, you will need to restart the service. If you are using the demo container, you can do so with the following command:"
200+
],
201+
"metadata": {
202+
"azdata_cell_guid": "aa10efd1-a1f1-4fbc-9e20-5ab94929d9ff"
203+
}
204+
},
205+
{
206+
"cell_type": "code",
207+
"source": [
208+
"! docker restart sql2019tempdbdemo"
209+
],
210+
"metadata": {
211+
"azdata_cell_guid": "65567cd7-bdda-4501-b04a-3e7c3579252c"
212+
},
213+
"outputs": [
214+
{
215+
"name": "stdout",
216+
"text": "sql2019tempdbdemo\n",
217+
"output_type": "stream"
218+
}
219+
],
220+
"execution_count": 10
221+
},
222+
{
223+
"cell_type": "markdown",
224+
"source": [
225+
"Once the server is restarted, you can use queries in the T-SQL notebook to verify that the feature has been enabled.\r\n",
226+
"\r\n",
227+
"> NOTE\r\n",
228+
"> <br> It's a good idea to run a few T-SQL queries after the restart to make sure the server is up and running before you attempt the scenario again.\r\n",
229+
"\r\n",
230+
"Now that we have enabled Memory-Optimized TempDB Metadata, let's try running the workload again:"
231+
],
232+
"metadata": {
233+
"azdata_cell_guid": "9624ed6d-06b4-49b2-bb5e-5a0c2b6b457c"
234+
}
235+
},
236+
{
237+
"cell_type": "code",
238+
"source": [
239+
"! ostress.exe -Slocalhost,1455 -Usa -PP@ssw0rd! -dAdventureWorks -Q\"EXEC dbo.usp_EmployeeBirthdayList 4\" -mstress -quiet -n16 -r120 | FINDSTR \"QEXEC Starting Creating elapsed\""
240+
],
241+
"metadata": {
242+
"azdata_cell_guid": "58c2f587-a9a5-4d6c-8cd3-93e555f08dee"
243+
},
244+
"outputs": [
245+
{
246+
"name": "stdout",
247+
"text": "10/08/19 15:38:27.261 [0x00009AA0] -QEXEC dbo.usp_EmployeeBirthdayList 4\n10/08/19 15:38:27.316 [0x00009AA0] Starting query execution...\n10/08/19 15:38:27.321 [0x00009AA0] Creating 16 thread(s) to process queries\n10/08/19 15:38:54.045 [0x00009AA0] OSTRESS exiting normally, elapsed time: 00:00:26.784\n",
248+
"output_type": "stream"
249+
}
250+
],
251+
"execution_count": 9
252+
},
253+
{
254+
"cell_type": "markdown",
255+
"source": [
256+
"While this is running, switch over to the T-SQL notebook to run the monitoring script again. This time, you should not see any sessions waiting on `PAGELATCH_EX` or `PAGELATCH_SH`. Also, the script should complete faster than before the change was made."
257+
],
258+
"metadata": {
259+
"azdata_cell_guid": "da79d28b-80d6-4644-98e5-23f250ec9c19"
260+
}
261+
}
262+
]
263+
}

‎samples/features/in-memory-database/memory-optimized-tempdb-metadata/SQL2019_Perfmon.htm‎

26.6 KB
Binary file not shown.

‎samples/features/in-memory-database/memory-optimized-tempdb-metadata/SQL2019_Run_EmployeeBirthdayList.bat‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
ECHO OFF
2+
rd /s /q %temp%\output
3+
"ostress.exe" -E -S.\SQL2019 -dAdventureWorks -Q"EXEC dbo.usp_EmployeeBirthdayList 4" -mstress -quiet -n1 -r1 | FINDSTR "Cantfindthisstring"
4+
rd /s /q %temp%\output
5+
"ostress.exe" -E -S.\SQL2019 -dAdventureWorks -Q"EXEC dbo.usp_EmployeeBirthdayList 4" -mstress -quiet -n100 -r300 | FINDSTR "QEXEC Starting Creating elapsed"

0 commit comments

Comments
 (0)