Skip to content

Commit 272ced8

Browse files
CarlRabelerCarlRabeler
committed
readme
1 parent 5301a10 commit 272ced8

2 files changed

Lines changed: 144 additions & 36 deletions

File tree

‎samples/manage/azure-sql-db-elastic-pools/Pool Telemetry PS User Guide.docx‎

-31.1 KB
Binary file not shown.

‎samples/manage/azure-sql-db-elastic-pools/readme.md‎

Lines changed: 144 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -1,72 +1,172 @@
1-
# Sample name
1+
# Elastic Pool Telemetry using PowerShell
22

3-
Briefly describe what the sample does. Describe the benefit of running this sample.
3+
This sample provides a set of PowerShell scripts for off-loading elastic pool and elastic database telemetry data into a separate telemetry database.
44

55
<!-- Add a diagram if you have it -->
66

7-
![Alt text](/media/image-name.png "<Friendly name>")
8-
7+
This readme applies to the PowerShell scripts: PoolTelemetryJobRunner.ps1 and PoolTelemetry.ps1.
98

109
### Contents
1110

12-
[About this sample](#about-this-sample)<br/>
13-
[Before you begin](#before-you-begin)<br/>
14-
[Run this sample](#run-this-sample)<br/>
15-
[Sample details](#sample-details)<br/>
11+
[What do the PowerShell scripts do?](#what-do-the-powershell-scripts-do?)<br/>
12+
[Installing the scripts](#installing-the-scripts)<br/>
13+
[Customizing the PoolTelemetryJobRunner script](#customizing-the-pooltelemetryjobrunner-script)<br/>
14+
[Executing the runner script](#executing-the-runner-script)<br/>
15+
[Inspecting the telemetry that is collected](#inspecting-the-telemetry-that-is-collected)<br/>
1616
[Disclaimers](#disclaimers)<br/>
1717
[Related links](#related-links)<br/>
1818

1919

20-
<a name=about-this-sample></a>
20+
<a name=what-do-the-powershell-scripts-do?></a>
21+
22+
## What do the PowerShell scripts do?
23+
24+
The scripts are used to extract telemetry data associated with SQL Database elastic database pools and elastic databases and upload it to a separate telemetry database.
25+
26+
There is a runner script, PoolTelemetryRunner.ps1, which needs to be modified for your environment to identify one or more servers on which elastic pools and databases are hosted and a telemetry database in which telemetry data is to be gathered. The runner script executes a function in the data collection script, PoolTelemetry.ps1 as a PowerShell job for each server.
27+
28+
Each data collection job executes in the background on a pre-determined schedule and will on first execution create the required schema in the telemetry database. It then connects to the master database on the server and retrieves elastic pool telemetry data and loads that to the telemetry database. It can optionally look back 14 days on first execution to get all available telemetry.
29+
30+
It then optionally queries the master database to determine the current elastic databases on the server, resident in each of the pools identified in the prior step. It then connects to each database in turn and retrieves and loads telemetry data for that database. It then sleeps for a period before waking up and repeating the data collection cycle.
31+
32+
<a name=installing-the-scripts></a>
33+
34+
## Installing the scripts
35+
36+
Both scripts, PoolTelemetry.ps1 and PooltelemetryRunner.ps1, should be copied to the same directory.
37+
38+
You must have installed and imported the latest (1.x) Azure PowerShell modules and SQL PowerShell (sqlps) modules. The cmdlet Invoke-SQLCmd is used to execute SQL scripts.
39+
40+
<a name=customizing-the-pooltelemetryjobrunner-script></a>
41+
42+
## Customizing the PoolTelemetryJobRunner script
43+
44+
The runner script should be customized to provide information about the servers from which pool data will be retrieved, as well as to customize frequency and overall duration of data collection.
45+
46+
### Azure log in
47+
48+
The script requires you to log on to Azure with a Microsoft Id, either a personal Id or a work or school Id. The Id used must have read access to the servers in the subscription under which the pools and databases have been created (the telemetry database can be created under a different subscription).
49+
50+
Set the SubscriptionName. If the Microsoft Id used to login has access to multiple Azure subscriptions this allows you to select the subscription under which the server(s) to be monitored were created. All servers to be reported on must be created under the same subscription.
51+
52+
```$AzureSubscriptionName = ‘<subscription name>' ```
53+
54+
### SQL user names for source servers and the telemetry server
55+
56+
While the runner script uses ARM PowerShell cmdlets to gather information about resource groups and servers, the data collection script uses SQL queries to retrieve data. SQL user names and passwords must be provided at script run time and will be passed to each data collection script job to access the source servers and databases using SQL DMVs. The scripts assume the same SQL user name and password are used for all source servers. The telemetry server user name and password are provided separately and can be different.
57+
58+
User credentials are gathered via dialog boxes at run time to avoid storing passwords in the script. You can customize the script to add the user names for each dialog so that these do not need to be entered each time the script is run. To do this add a –UserName parameter to each of the two credentials.
59+
60+
```$sourceCred = Get-Credential -Message 'User name and password for source server' –UserName '<user name>' ```
61+
62+
```$outputServerCred = Get-Credential -Message 'User name and password for telemetry database’ –UserName '<user name>' ```
63+
64+
### Source resource group and server
65+
66+
Provide the resource group if data is to be gathered from all servers in a specific resource group or a specific server.
67+
68+
``` $resourceGroupName = '<resource group name>' ```
69+
70+
Provide the server name if data is to be gathered from all a specific server.
71+
72+
``` $serverName = '<server name>' ```
73+
74+
### Telemetry server and database
75+
76+
It is assumed that telemetry is to be loaded to an Azure SQL Database.
2177

22-
## About this sample
78+
Provide the telemetry database server name.
2379

24-
<!-- Delete the ones that don't apply -->
25-
1. **Applies to:** SQL Server 2016 (or higher), Azure SQL Database, Azure SQL Data Warehouse, Parallel Data Warehouse
26-
1. **Key features:*Elastic pools*
27-
1. **Workload:**
28-
1. **Programming Language:*PowerShell*
29-
1. **Authors:*sriniachyara*
30-
1. **Update history:*5/6/16*
80+
``` $outputServerName = '<telemetry server name>’ ```
3181

32-
<a name=before-you-begin></a>
82+
Provide the telemetry database name.
3383

34-
## Before you begin
84+
``` $outputDatabaseName = '<telemetry database name>' ```
3585

36-
To run this sample, you need the following prerequisites.
86+
### Define if the server to be monitored will change during the monitoring period
3787

38-
**Software prerequisites:**
88+
If the set of servers being monitored may change during the overall monitoring period then set $staticServerList to $false to cause server evaluation to be repeated periodically. If this is set to false, the runner script will run for the same duration as the job scripts, and will start additional jobs if new servers are added and stop jobs if servers are removed from the query scope. Otherwise if set to $true, the runner script will complete as soon as the jobs have been spawned.
3989

40-
<!-- Examples -->
41-
1. Azure SQL Database
42-
2. Azure PowerShell
90+
``` $staticServerList = $true ```
4391

44-
**Azure prerequisites:**
92+
### Collection interval, lag-time and job duration
4593

46-
<!-- Examples -->
47-
1. Permission to create an Azure SQL Database
94+
Provide the interval in minutes. This defines both how far back the data collection will look on each execution and the interval between executions. A value between 15-30 minutes is probably most appropriate. Note that fine-grained database telemetry (15 second averages) is only retained in each database for 60 minutes, beyond that it based on 5 minute averages. Pool telemetry in the master database is always based on 5 minute averages. Pool telemetry is not available immediately. A lag time of 30 minutes is programmed in the collection script. It is not recommended to change this lag setting. The effect of this is that the look-back window for pool data is pushed back, by this lag time so if gathering data for 15 interval the query window is -45 minutes to -30 minutes on each execution. Note that the lag time setting does not affect gathering 15s averaged telemetry from each database, which is available immediately. 5 minute averaged data is retained for 14 days.
4895

49-
<a name=run-this-sample></a>
96+
``` $intervalMinutes = 15 ```
5097

51-
## Run this sample
98+
Provide the job duration in minutes. This defines how long the job will execute for in the background. A value of zero will cause the job to execute once only. The value is best
5299

53-
<!-- Step by step instructions. Here's a few examples -->
100+
``` $durationMinutes = 600 ```
101+
102+
### Load all available pool telemetry
54103

55-
1. Open the ps in PowerShell.
104+
In normal execution the spawned jobs look back 'window' is based on the interval and lag settings. For pool telemetry which is available for 14 days, the data collection script can be configured to look back 15 days on its first execution to ensue it gathers all available telemetry for each pool. Be careful if you stop the runner script and restart it on the same servers within this 15 day period as it may gather and load duplicate data entries. Using this option with many pools may load a large amount of data.
56105

57-
2. Fill in the values for Resource Manager name, server name, user name, password.
106+
``` $loadAllAvailablePoolTelemetry = $true ```
58107

108+
### Specify the source server(s) to use
59109

60-
<a name=sample-details></a>
110+
The script allows either a single server to be specified or multiple. Several sample PowerShell scripted queries are provided but in general only one should be used, the others should be commented out. The script requires that the $servers variable is populated as input to the job execution. Either uncomment and use one of the queries that populates $servers or use one of the queries that populates $resourceList and then uncomment the section in the script that uses the $resourceList to populate $servers. If not using $resourceList leave this translation section commented out.
61111

62-
## Sample details
112+
<a name=executing-the-runner-script></a>
63113

64-
Collects data from the listed server.
114+
## Executing the runner script
115+
116+
The runner script PoolTelemetryRunner.ps1 should be executed from within an Azure PowerShell context.
117+
118+
The script will prompt for Azure login and the user name and password for the source servers and the user name and password for the telemetry server. It will then spawn a PowerShell job for each server that has been identified within the script. Each job will run in the background for the time specified in $durationMinutes.
119+
120+
It will gather data for the most recent period defined by the interval value and load this, then sleep until the next data gathering point, wake up, gather and load more data and then sleep again, etc.
121+
122+
To see jobs in progress, use:
123+
124+
``` Get-Job ```
125+
126+
To see the current console output from a specific job, use:
127+
128+
``` Receive-Job <job id> -Keep ```
129+
130+
If you don’t use *** –Keep***, the output is not retained (but doesn’t affect data collection)
131+
132+
To stop all jobs, use:
133+
134+
``` Stop-Job * ```
135+
136+
Provide a job id to stop a specific job.
137+
138+
To remove all jobs, use:
139+
140+
``` Remove-Job * ```
141+
142+
Provide a job id to remove a specific job.
143+
144+
<a name=inspecting-the-telemetry-that-is-collected></a>
145+
146+
## Inspecting the telemetry that is collected
147+
148+
Use SSMS or other tools such as PowerBI to inspect and query the telemetry database. Data is gathered in two tables which are based on the equvalent DMVs:
149+
150+
- ***dbo.pool_resource_stats*** has the resource usage data for all the elastic pools in the specified servers for the specified duration.
151+
152+
- ***dbo.db_resource_stats*** has the resource usage data for all elastic databases in the elastic pools for the specified duration.
153+
154+
There is also a pre-defined Table-valued function to help querying the aggregate resource usage data for a specified time interval. @start and @end are the time interval of interest for querying.
155+
156+
``` [dbo].[get_aggregated_pool_metrics](@start datetime, @end datetime) ```
157+
158+
For example, once the telemetry is being collected, this TVF can be called with the required time interval to get the top 10 elastic pools with highest average eDTU consumption in the specified time period.
159+
160+
``` select top 10 * from [dbo].[get_aggregated_pool_metrics]('04/29/2016 21:00:00', '04/29/2016 23:00:00') order by avg_DTU_percent desc ```
161+
162+
Data can be queried while data collection is in progress.
163+
164+
> [AZURE.NOTE] If the scripts are stopped and started again within a short period they may add duplicate rows to the telemetry tables.
65165
66166
<a name=disclaimers></a>
67167

68168
## Disclaimers
69-
The code included in this sample is not intended to be ...
169+
The scripts and this guide are copyright Microsoft Corporations and are provided as samples. They are not part of any Azure service and are not covered by any SLA or other Azure-related agreements. They are provided as-is with no warranties express or implied. Microsoft takes no responsibility for the use of the scripts or the accuracy of this document. Familiarize yourself with the scripts before using them.
70170

71171
<a name=related-links></a>
72172

@@ -75,3 +175,11 @@ The code included in this sample is not intended to be ...
75175

76176
For more information, see these articles:
77177

178+
- [Monitor and manage an elastic database pool with PowerShell](https://azure.microsoft.com/documentation/articles/sql-database-elastic-pool-manage-powershell/)
179+
180+
- [Monitor and manage an elastic database pool with C#](https://azure.microsoft.com/documentation/articles/sql-database-elastic-pool-manage-csharp/)
181+
182+
- [Monitor and manage an elastic database pool with Transact-SQL](https://azure.microsoft.com/documentation/articles/sql-database-elastic-pool-manage-tsql/)
183+
184+
- [Monitor and manage an elastic database pool with the Azure portal](https://azure.microsoft.com/documentation/articles/sql-database-elastic-pool-manage-portal/)
185+

0 commit comments

Comments
 (0)