Merge remote-tracking branch 'upstream/master'

Author: inchiosa

samples/features/sql-big-data-cluster/deployment/kubeadm/ubuntu/setup-k8s-master.sh

@@ -10,5 +10,5 @@ sudo chown $(id -u):$(id -g) $HOME/.kube/config
 kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
 helm init
 kubectl apply -f rbac.yaml
-kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml
+kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml
 kubectl create clusterrolebinding kubernetes-dashboard --clusterrole=cluster-admin --serviceaccount=kube-system:kubernetes-dashboard

samples/features/sql-big-data-cluster/spark/README.md

@@ -1,29 +1,27 @@
 # SQL Server big data clusters
 
-The new built-in notebooks in Azure Data Studio enables data scientists and data engineers to run Python, R, Scala, or Spark SQL code against the cluster.
+SQL Server Big Data cluster bundles Spark and HDFS together with SQL server. Azure Data Studio IDE provides built in notebooks that enables data scientists and data engineers to run Spark notebooks and job in Python, R, or Scala code against the Big Data Cluster. This folder contains spark sample notebook on using Spark in SQL server Big data cluster
 
-## Instructions to open a notebook from Azure Data Studio and execute the commands
+## Folder contents
 
-1. Connect to the SQL Server Master instance in a big data cluster
+[PySpark Hello World](dataloading/hello_PySpark.ipynb)
 
-1. Right-click on the server name, select **Manage**, switch to **SQL Server Big Data Cluster** tab, and use open Notebook.
+[Scala Hello World ](dataloading/hello_Scala.ipynb)
 
-1. Open the notebook in Azure Data Studio, wait for the “Kernel” and the target context (“Attach to”) to be populated.
+[SparkR Hello World ](dataloading/hello_sparkR.ipynb)
 
-1. Run each cell in the Notebook sequentially.
+[DataLoading   - Transforming CSV to Parquet](dataloading/transform-csv-files.ipynb/)
 
-## __[data-loading](data-loading/)__
+[Data Transfer - Spark to SQL using Spark JDBC connector](data-virtualization/spark_to_sql_jdbc.ipynb/)
 
-This folder contains samples that show how to load data using Spark and query them using SQL statements.
+[Data Transfer - Spark to SQL using MSSQL Spark connector](spark_to_sql/mssql_spark_connector.ipynb/)
 
-[data-loading/transform-csv-files.ipynb](dataloading/transform-csv-files.ipynb/)
-
-This samnple notebook shows how to transform CSV files in HDFS to parquet files.
+## Instructions on how to run in Azure Data Studio
 
-[dataloading/spark-sql.ipynb](dataloading/spark-sql.ipynb/)
+[data-loading/transform-csv-files.ipynb](dataloading/transform-csv-files.ipynb/)
 
-This samnple notebook shows how to query hive tables created from Spark.
+2. From Azure Data Studio Connect to the SQL Server Master instance in a big data cluster. 
 
-## __[data-virtualization](data-virtualization/)__
+3. Right-click on the server name, select **Manage**, switch to **SQL Server Big Data Cluster** tab, and open the notebook in Azure Data Studio.  Wait for the “Kernel” and the target context (“Attach to”) to be populated. If required set the relevant “Kernel” ( e.g **PySpark3** )  and **Attach to** needs to be the IP address of your big data cluster endpoint.
 
-This folder contains samples that show how to integrate Spark with other data sources.
+4. Run each cell in the Notebook sequentially.