Blogs
What's New Trending
-
- February 28, 2022
- Entertainment
- (0)
- 02 mins
How to start your databricks first time in Azure?
Do you like to watch long videos or read long documents?
Do you want to learn data bricks within 2 to 5 minutes?
Coming straight to the point, the following step-by-step sections describe the flow of action. For detail, please visit the following websites
Microsoft azure databricks document
Step 1: Create Azure Databricks Resource (Workspace and Cluster)
Create Workspace:
# Login to azure portal > Create a resource > Search Azure databrick > Create Azure Databricks > Provide following detail
- Subscription
- Resource group: for example ABCD_RG
- Workspace name: ABCP_WS
- Region
- Pricing tier
# Next Networking > Advanced > Tag > Review + Create > Validating > Create > Deployment is in progress > Your deployment is completed > Go to resource
# Launch workspace > a separate window will be poped up > Azure Databrick UI > compute (left side within create) >
Create Cluster:
Create cluster >
- Cluster name : ABCD_CN
- Cluster mode: Single node
- Pool : None
- Databricks runtime version: ML > 10.2 ML Scala 2.12 Spark 3.2.0
- Terminate : 120 min
- Node Type : Standard_DS3_v2
- 14GB Memory, 4 Cores
- Advanced : default
Step 2: Upload your data
Data (left side) > Create table > Upload file > File — Drop files to upload/click browse > Create table with UI > Select a cluster to preview the table — AzureADClusterTestMS > Spacify table attributes
- Table Name : nyc_taxi
- Create a database : Default
- File type : CSV
- Column Delimiter: Checked
- Infer schema: checked
- Multi-line: unchekced
Set data type:
Set the approprate data type for each column: Locate the passengerCount column. In the drop-down menu, set the column to INT.
- passengerCount: INT
- tripDistance: DOUBLE
- hour_of_day: INT
- day_of_week: INT
- month_num: INT
- isPaidTimeOff: BOOLEAN
- snowDepth: DOUBLE
- precipTime: DOUBLE
- precipDepth: DOUBLE
- temperature: DOUBLE
- totalAmount: DOUBLE
Step 3: Now you can import your databrick notebook
- Workspace (left side) > Users > Select your email id/name > click dropdown > Import > Url > ….
Please note the databrick notebook is different from jupyter/livy notebook. If you would like to use your previous notebook, you should first convert it into databrick notebook format. You can pull data from some example dataset provided by databrick.
Step 4: Explore notebooks
- 01 — Introduction to Azure Databricks > Getting Started with Azure Databricks
- Click dropdown (top left) > Select your cluster > Run notebook
Step 5: Terminate the cluster
compute (leftside) > select your cluster > terminate (shutdown)
Step 6: (Re)Start the cluster
compute (leftside) > select your cluster > start
workspace> users/usernamei>/01 — introduction….> Working with data in Azure Databrick > Make sure to attach your cluster (Click dropdown (top left) > Select your cluster) > Run
References
- https://github.com/MicrosoftLearning/dp-090-databricks-ml/raw/master/01%20-%20Introduction%20to%20Azure%20Databricks.dbc
- https://github.com/MicrosoftLearning/dp-090-databricks-ml/raw/master/02%20-%20Training%20and%20Evaluating%20Machine%20Learning%20Models.dbc
- https://github.com/MicrosoftLearning/dp-090-databricks-ml/raw/master/03%20-%20Managing%20Experiments%20and%20Models.dbc
- https://github.com/MicrosoftLearning/dp-090-databricks-ml/raw/master/04%20-%20Integrating%20Azure%20Databricks%20and%20Azure%20Machine%20Learning.dbc
- https://azure.microsoft.com/en-us/features/azure-portal/
- https://docs.databricks.com/
Please leave a comment.
manish.saraf
DSC, PhD, MBA | AI Expert/Advisor | Entrepreneur | Business Coach | Mentor | Artificial intelligence, deep learning, NLP, Chatbot, Data Brick, K8, Azure | Owner of OGM World Wide, OGM-Lands, OGM-Finance
Leave a Reply