creating conda environment on azure hdinsight spark cluster taking hours
kunal kapoor
1
Reputation point
trying to follow the code in this tutorial , https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/training/train-in-spark/train-in-spark.ipynb
when i send the run to run to azure ml, the run essentially stalls on the step creating the conda environment in the hdinsight cluster I am looking to run the pyspark code on.
I am wondering if anyone has run into the same problem where creating a conda environment on a hdinsight cluster for the first time is taking hours, already been 2 hours for me.
Thanks
Sign in to answer