Skip to content

Commit 13327f2

Browse files
author
Mihaela Blendea
committed
Added aks sample script
1 parent edbe2f5 commit 13327f2

4 files changed

Lines changed: 202 additions & 46 deletions

File tree

‎samples/features/sql-big-data-cluster/deployment/README.md‎

Lines changed: 5 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -1,53 +1,12 @@
1+
12
# Creating a Kubernetes cluster for SQL Server 2019 big data cluster
2-
SQL Server 2019 big data cluster is deployed as docker containers on a Kubernetes cluster. These samples provide scripts that can be used to provision a Kubernetes cluster using different methods.
33

4+
SQL Server 2019 big data cluster is deployed as docker containers on a Kubernetes cluster. These samples provide scripts that can be used to provision a Kubernetes clusters using different environments.
45

56
## Create a Kubernetes cluster using Kubeadm on Ubuntu 16.04 LTS or 18.04 LTS
6-
In this example, we will deploy Kubernetes over multiple Linux machines (physical or virtualized) using kubeadm utility. These instructions have been tested primarily with Ubuntu 16.04 LTS version. If you are using Ubuntu 18.04 LTS then install of docker.io may fail with message below depending on your configuration.
7-
8-
Package docker.io is not available, but is referred to by another package.
9-
This may mean that the package is missing, has been obsoleted, or
10-
is only available from another source
11-
12-
E: Package 'docker.io' has no installation candidate
13-
14-
To install docker, you can follow the steps below:
15-
16-
#!/usr/bin/env bash
17-
18-
sudo apt update
19-
20-
sudo apt --yes install \
21-
software-properties-common \
22-
apt-transport-https \
23-
ca-certificates \
24-
curl
25-
26-
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
27-
28-
sudo add-apt-repository \
29-
"deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
30-
31-
sudo apt update
32-
33-
sudo apt --yes install docker-ce
34-
35-
sudo usermod --append --groups docker $USER
36-
37-
38-
### Pre-requisites
39-
1. Multiple Linux machines or virtual machines. Recommended configuration is 8 CPUs, 32 GB memory each and at least 100 GB storage for each machine. Minimum number of machines required is three machines
40-
1. Designate one machine as the Kubernetes master
41-
1. Rest of the machines will be used as the Kubernetes agents
427

43-
#### Useful resources
44-
[Deploy SQL Server 2019 big data cluster on Kubernetes](https://docs.microsoft.com/en-us/sql/big-data-cluster/deployment-guidance?view=sqlallproducts-allversions)
8+
Use the scripts in the **kubeadm** folder to deploy Kubernetes over multiple Linux machines (physical or virtualized) using `kubeadm` utility.
459

46-
[Creating a cluster using kubeadm](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/)
47-
[Troubleshooting kubeadm](https://kubernetes.io/docs/setup/independent/troubleshooting-kubeadm/)
10+
## Deploy a SQL Server big data cluster on Azure Kubernetes Service (AKS)
4811

49-
#### Instructions
50-
1. Execute [kubeadm/ubuntu-setup-k8s-prereqs.sh](kubeadm/ubuntu-setup-k8s-prereqs.sh/) script on each machine
51-
1. Execute [kubeadm/ubuntu-setup-k8s-master.sh](kubeadm/ubuntu-setup-k8s-master.sh/) script on the machine designated as Kubernetes master
52-
1. After successful initialization of the Kubernetes master, follow the kubeadm join commands output by the script on each agent machine
53-
1. Now, you can deploy SQL Server 2019 big data cluster using instructions [here](https://docs.microsoft.com/en-us/sql/big-data-cluster/deployment-guidance?view=sqlallproducts-allversions)
12+
Using the sample Python script in **aks** folder, you will deploy a Kubernetes cluster in Azure using AKS and a SQL Server big data cluster using on top of it.

‎samples/features/sql-big-data-cluster/deployment/aks/README.md‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
2+
# Deploy a SQL Server big data cluster on Azure Kubernetes Service (AKS)
3+
4+
Using this sample Python script, you will deploy a Kubernetes cluster in Azure using AKS and a SQL Server big data cluster using this AKS cluster as its environment. The script can be run from any client OS.
5+
6+
7+
## Pre-requisites
8+
9+
1. Running the script will require: [python minimum version 3.0](https://www.python.org/downloads)
10+
1. Ensure you have installed `mssqlctl` CLI and its prerequisites:
11+
- Install [pip3](https://pip.pypa.io/en/stable/installing/).
12+
- Install/update requests package. Run the command below using elevated priviledges (sudo or admin cmd window):
13+
```
14+
python -m pip install requests
15+
python -m pip install requests --upgrade
16+
```
17+
- Install mssqlctl CLI latest version using . Run the command below using elevated priviledges (sudo or admin cmd window):
18+
```
19+
pip3 install --index-url https://private-repo.microsoft.com/python/ctp-2.0 mssqlctl
20+
```
21+
1. Login into your Azure account. Run this command:
22+
```
23+
az login
24+
```
25+
26+
## Instructions
27+
28+
Run the script using:
29+
```
30+
python deploy-sql-big-data-aks.py
31+
```
32+
33+
>[!Note]
34+
>
35+
>If you have both pyhon3 and pyhon2 on your client machine and in the path, you will have to run the command using pyhon3:
36+
>```
37+
>python3 deploy-sql-big-data-aks.py
38+
>```
39+
40+
41+
When prompted, provide your input for Azure subscription ID, Azure resource group to create the resources in, and Docker credentials. Optionally, you can also provide your input for below configurations or use the defaults provided:
42+
- azure_region
43+
- vm_size - we recommend to use a VM size to accommodate your workload. Default `Standard_DS3_v2` is minimum for running just basic validations
44+
- aks_node_count - this is the number of the worker nodes for the AKS cluster, excluding master node
45+
- cluster_name - this value is used for both AKS cluster and SQL big data cluster created on top of AKS. Note that the name of the SQL big data cluster is going to be a Kubernetes namespace
46+
- password - same value is going to be used for all accounts that require user password input: SQL Server master instance SA account, controller user and Knox user
47+
- controller_username - this is the username for the cluster admin account

‎samples/features/sql-big-data-cluster/deployment/aks/deploy-sql-big-data-aks.py‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
#
2+
# Prerequisites:
3+
#
4+
# Azure CLI (https://docs.microsoft.com/en-us/cli/azure/install-azure-cli), python3 (https://www.python.org/downloads), mssqlctl CLI (pip3 install --index-url https://private-repo.microsoft.com/python/ctp-2.0 mssqlctl )
5+
#
6+
# Run `az login` at least once BEFORE running this script
7+
#
8+
9+
from subprocess import check_output, CalledProcessError, STDOUT, Popen, PIPE
10+
import os
11+
12+
def executeCmd (cmd):
13+
if os.name=="nt":
14+
process = Popen(cmd.split(),stdin=PIPE, shell=True)
15+
else:
16+
process = Popen(cmd.split(),stdin=PIPE)
17+
stdout, stderr = process.communicate()
18+
if (stderr is not None):
19+
raise Exception(stderr)
20+
21+
#
22+
# MUST INPUT THESE VALUES!!!!!
23+
#
24+
SUBSCRIPTION_ID = input("Provide your Azure subscription ID:")
25+
GROUP_NAME = input("Provide Azure resource group name to be created:")
26+
DOCKER_USERNAME = input("Provide your Docker username:")
27+
DOCKER_PASSWORD = input("Provide your Docker password:")
28+
29+
#
30+
# Optionally change these configuration settings
31+
#
32+
AZURE_REGION=input("Provide Azure region - Press ENTER for using `westus`:") or "westus"
33+
VM_SIZE=input("Provide VM size for the AKS cluster - Press ENTER for using `Standard_DS3_v2`:") or "Standard_DS3_v2"
34+
AKS_NODE_COUNT=input("Provide number of worker nodes for AKS cluster - Press ENTER for using `2`:") or "2"
35+
#This is both Kubernetes cluster name and SQL Big Data cluster name
36+
CLUSTER_NAME=input("Provide name of AKS cluster and SQL big data cluster - Press ENTER for using `sqlbigdata`:") or "sqlbigdata"
37+
#This password will be use for Controller user, Knox user and SQL Server Master SA accounts
38+
PASSWORD=input("Provide password to be used for Controller user, Knox user and SQL Server Master SA accounts - Press ENTER for using `MySQLBigData2019`:") or "MySQLBigData2019"
39+
CONTROLLER_USERNAME=input("Provide username to be used for Controller user - Press ENTER for using `admin`:") or "admin"
40+
41+
#
42+
DOCKER_REGISTRY="private-repo.microsoft.com"
43+
DOCKER_REPOSITORY="mssql-private-preview"
44+
DOCKER_IMAGE_TAG="latest"
45+
46+
print ('Setting environment variables')
47+
os.environ['MSSQL_SA_PASSWORD'] = PASSWORD
48+
os.environ['CONTROLLER_USERNAME'] = CONTROLLER_USERNAME
49+
os.environ['CONTROLLER_PASSWORD'] = PASSWORD
50+
os.environ['KNOX_PASSWORD'] = PASSWORD
51+
os.environ['DOCKER_REGISTRY'] = DOCKER_REGISTRY
52+
os.environ['DOCKER_REPOSITORY'] = DOCKER_REPOSITORY
53+
os.environ['DOCKER_USERNAME']=DOCKER_USERNAME
54+
os.environ['DOCKER_PASSWORD']=DOCKER_PASSWORD
55+
os.environ['DOCKER_EMAIL']=DOCKER_USERNAME
56+
os.environ['DOCKER_IMAGE_TAG']=DOCKER_IMAGE_TAG
57+
os.environ['DOCKER_IMAGE_POLICY']="IfNotPresent"
58+
os.environ['DOCKER_PRIVATE_REGISTRY']="1"
59+
os.environ['CLUSTER_PLATFORM']="aks"
60+
os.environ['ACCEPT_EULA']="Y"
61+
62+
print ("Set azure context to subcription: "+SUBSCRIPTION_ID)
63+
command = "az account set -s "+ SUBSCRIPTION_ID
64+
executeCmd (command)
65+
66+
print ("Creating azure resource group: "+GROUP_NAME)
67+
command="az group create --name "+GROUP_NAME+" --location "+AZURE_REGION
68+
executeCmd (command)
69+
70+
print("Creating AKS cluster: "+CLUSTER_NAME)
71+
command = "az aks create --name "+CLUSTER_NAME+" --resource-group "+GROUP_NAME+" --generate-ssh-keys --node-vm-size "+VM_SIZE+" --node-count "+AKS_NODE_COUNT+" --kubernetes-version 1.10.7"
72+
executeCmd (command)
73+
74+
command = "az aks get-credentials --name "+CLUSTER_NAME+" --resource-group "+GROUP_NAME+" --admin"
75+
executeCmd (command)
76+
77+
print("Creating SQL Big Data cluster:" +CLUSTER_NAME)
78+
command="mssqlctl create cluster "+CLUSTER_NAME
79+
executeCmd (command)
80+
81+
print("")
82+
print("SQL Server big data cluster connection endpoints: ")
83+
print("SQL Server master instance:")
84+
command="kubectl get service service-master-pool-lb -o=custom-columns=""IP:.status.loadBalancer.ingress[0].ip,PORT:.spec.ports[0].port"" -n "+CLUSTER_NAME
85+
executeCmd(command)
86+
print("")
87+
print("HDFS/KNOX:")
88+
command="kubectl get service service-security-lb -o=custom-columns=""IP:status.loadBalancer.ingress[0].ip,PORT:.spec.ports[0].port"" -n "+CLUSTER_NAME
89+
executeCmd(command)
90+
print("")
91+
print("Cluster administration portal (https://<ip>:<port>):")
92+
command="kubectl get service service-proxy-lb -o=custom-columns=""IP:status.loadBalancer.ingress[0].ip,PORT:.spec.ports[0].port"" -n "+CLUSTER_NAME
93+
executeCmd(command)
94+
print("")

‎samples/features/sql-big-data-cluster/deployment/kubeadm/README.md‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
2+
3+
# Create a Kubernetes cluster using Kubeadm on Ubuntu 16.04 LTS or 18.04 LTS
4+
5+
In this example, we will deploy Kubernetes over multiple Linux machines (physical or virtualized) using kubeadm utility. These instructions have been tested primarily with Ubuntu 16.04 LTS version. If you are using Ubuntu 18.04 LTS then install of docker.io may fail with message below depending on your configuration.
6+
7+
Package docker.io is not available, but is referred to by another package.
8+
This may mean that the package is missing, has been obsoleted, or
9+
is only available from another source
10+
11+
E: Package 'docker.io' has no installation candidate
12+
13+
To install docker, you can follow the steps below:
14+
15+
#!/usr/bin/env bash
16+
17+
sudo apt update
18+
19+
sudo apt --yes install \
20+
software-properties-common \
21+
apt-transport-https \
22+
ca-certificates \
23+
curl
24+
25+
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
26+
27+
sudo add-apt-repository \
28+
"deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
29+
30+
sudo apt update
31+
32+
sudo apt --yes install docker-ce
33+
34+
sudo usermod --append --groups docker $USER
35+
36+
37+
## Pre-requisites
38+
39+
1. Multiple Linux machines or virtual machines. Recommended configuration is 8 CPUs, 32 GB memory each and at least 100 GB storage for each machine. Minimum number of machines required is three machines
40+
1. Designate one machine as the Kubernetes master
41+
1. Rest of the machines will be used as the Kubernetes agents
42+
43+
### Useful resources
44+
45+
[Deploy SQL Server 2019 big data cluster on Kubernetes](https://docs.microsoft.com/en-us/sql/big-data-cluster/deployment-guidance?view=sqlallproducts-allversions)
46+
47+
[Creating a cluster using kubeadm](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/)
48+
49+
[Troubleshooting kubeadm](https://kubernetes.io/docs/setup/independent/troubleshooting-kubeadm/)
50+
51+
### Instructions
52+
53+
1. Execute [kubeadm/ubuntu-setup-k8s-prereqs.sh](kubeadm/ubuntu-setup-k8s-prereqs.sh/) script on each machine
54+
1. Execute [kubeadm/ubuntu-setup-k8s-master.sh](kubeadm/ubuntu-setup-k8s-master.sh/) script on the machine designated as Kubernetes master
55+
1. After successful initialization of the Kubernetes master, follow the kubeadm join commands output by the script on each agent machine
56+
1. Now, you can deploy SQL Server 2019 big data cluster using instructions [here](https://docs.microsoft.com/en-us/sql/big-data-cluster/deployment-guidance?view=sqlallproducts-allversions)

0 commit comments

Comments
 (0)