Spark jobs not running in a notebook

Sasha Yang 1

I am currently running the "1.Reading Data - CSV" notebook from the "Read and write data in Azure Databricks" module on Microsoft learn. When I tried to run the cell "# A reference to our tab-separated-file", the Spark jobs seems to be stuck while the command just continues running. The cluster attached to the notebook seems fine and running. Below are the screenshots of the notebook and the job description.
I have tried restarting the cluster and then detaching and re-attaching the cluster to the notebook, but the issue persists.

1 answer

PRADEEPCHEEKATLA-MSFT 84,381 Reputation points Microsoft Employee

2020-06-17T08:08:38.577+00:00

@sashayang-8285, Welcome to the Microsoft Q&A platform.

Could you please share the Databricks Runtime version which you are using?

I’m able to run the “1. Reading Data - CSV” from MS Learn, without any issue.

Used: Azure Databricks Runtime version: 6.6 (includes Apache Spark 2.4.5, Scala 2.11)

Hope this helps. Do let us know if you any further queries.

Do click on "Accept Answer" and Upvote on the post that helps you, this can be beneficial to other community members.
Please sign in to rate this answer.
Sasha Yang 1 Reputation point

2020-06-17T19:23:38.893+00:00

My runtime version is 6.5 (includes Apache Spark 2.4.5, Scala 2.11)

Sasha Yang 1 Reputation point

2020-06-17T20:19:04.297+00:00

I just read (from other discussion threads) that my free trial license has a quota of 4 cores, which technically speaking is not enough for me to launch a cluster. But for some unknown reason the cluster I created actually ran yesterday (a glitch I guess?). Maybe this is why the jobs are not running?

PRADEEPCHEEKATLA-MSFT 84,381 Reputation points Microsoft Employee

2020-06-18T08:53:30.047+00:00

@Sasha Yang , Yes, you are correct Azure Free trial subscription are not allowed to create Spark cluster.

You may checkout similar Q&A thread here: https://video2.skills-academy.com/en-us/answers/questions/1544/azure-databricks.html

I have tried it on Databricks Runtime version 6.5 and able to run the query without any issue.

I would suggest you to create a new cluster and check if the issue persists.

Hope this helps. Do let us know if you any further queries.

-----------------------------------------

Do click on "Accept Answer" and Upvote on the post that helps you, this can be beneficial to other community members.

PRADEEPCHEEKATLA-MSFT 84,381 Reputation points Microsoft Employee

2020-07-10T07:21:52.467+00:00

Hello @Sasha Yang ,

Just checking in to see if the above answer helped. If this answers your query, do click “Accept Answer” and Up-Vote for the same. And, if you have any further query do let us know.

PRADEEPCHEEKATLA-MSFT 84,381 Reputation points Microsoft Employee

2020-07-20T11:30:56.053+00:00

Hello @Sasha Yang ,

Just checking in to see if the above answer helped. If this answers your query, do click “Accept Answer” and Up-Vote for the same. And, if you have any further query do let us know.
Sign in to comment

Share via

Spark jobs not running in a notebook

1 answer