Non-interactive login to registered dataset

Adam Stewart 51 Reputation points
2020-07-21T15:52:44.937+00:00

I'm trying to tune hyperparameters similar to the following guide: https://video2.skills-academy.com/en-us/azure/machine-learning/how-to-tune-hyperparameters

My PyTorch Dataset in train.py contains: 

ws = Workspace.from_config()  
ds = Dataset.get_by_name(ws, 'train')  
df = ds.to_pandas_dataframe()

This code works fine when run from the command-line, but when I submit a hyperparam tuning job to each node of a training cluster, I get the following error:

We could not find config.json in: /mnt/batch/tasks/shared/LS_root/jobs/adamml/azureml/hd_ba15bb39-f0fe-47a7-afbc-d2f9968e9687_3/mounts/workspaceblobstore/azureml/HD_ba15bb39-f0fe-47a7-afbc-d2f9968e9687_3 or in its parent directories. Please provide the full path to the config file or ensure that config.json exists in the parent directories.

If I manually pass my subscription id, resource group, and workspace id, I don't get this error, but now every single hyperparam tuning experiment requires me to log in through the web portal. Is there a way to do a non-interactive login?

Azure Machine Learning
Azure Machine Learning
An Azure machine learning service for building and deploying models.
2,848 questions
Accepted answer
  1. Shuyu Cao 86 Reputation points
    2020-07-23T20:05:42.24+00:00

    If I read the post correctly, you were trying to get an registered dataset within a submitted run. There, Workspace.from_config() won't work since there is no config.json file as the error suggested.

    And when you created an auth object which is InteractiveLoginAuth, it is expected to perform interactive login.

    Within a run the recommended way to connect to current workspace it via: 

       from azureml.core import Run  
   run = Run.get_context().experiment.workspace

    Meanwhile, there is way to pass in an dataset object to a run without involving register and workspace signin. If that fit your scenario better, please refer to the example in this document https://video2.skills-academy.com/en-us/azure/machine-learning/how-to-train-with-datasets#access-and-explore-input-datasets 

       from azureml.core import Dataset, Run  
     
   run = Run.get_context()  
   # get the input dataset by name  
   dataset = run.input_datasets['titanic']  
     
   # load the TabularDataset to pandas DataFrame  
   df = dataset.to_pandas_dataframe()
    2 people found this answer helpful.

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.