Run.get_context给出相同的运行ID 选项1：在运行内创建子运行选项2从控制平面创建行程选项3 Hyperdrive建议使用IMHO方法

问题描述

我正在通过脚本文件提交培训。以下是train.py脚本的内容。由于Run.get_context()返回相同的运行ID，Azure ML将所有这些都视为一次运行（而不是如下所示的每个alpha值运行）。

train.py

from azureml.opendatasets import Diabetes
from azureml.core import Run

from sklearn.model_selection import train_test_split
from sklearn.linear_model import Ridge
from sklearn.metrics import mean_squared_error
from sklearn.externals import joblib

import math
import os
import logging

# Load dataset
dataset = Diabetes.get_tabular_dataset()
print(dataset.take(1))

df = dataset.to_pandas_dataframe()
df.describe()

# Split X (independent variables) & Y (target variable)
x_df = df.dropna()      # Remove rows that have missing values
y_df = x_df.pop("Y")    # Y is the label/target variable

x_train,x_test,y_train,y_test = train_test_split(x_df,y_df,test_size=0.2,random_state=66)
print('Original dataset size:',df.size)
print("Size after dropping 'na':",x_df.size)
print("Training split size: ",x_train.size)
print("Test split size: ",x_test.size)

# Training
alphas = [0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1.0] # Define hyperparameters

# Create and log interactive runs

output_dir = os.path.join(os.getcwd(),'outputs')

for hyperparam_alpha in alphas:
    # Get the experiment run context
    run = Run.get_context()
    print("Started run: ",run.id)
    run.log("train_split_size",x_train.size)
    run.log("test_split_size",x_train.size)
    run.log("alpha_value",hyperparam_alpha)

    # Train
    print("Train ...")
    model = Ridge(hyperparam_alpha)
    model.fit(X = x_train,y = y_train)
    
    # Predict
    print("Predict ...")
    y_pred = model.predict(X = x_test)

    # Calculate & log error
    rmse = math.sqrt(mean_squared_error(y_true = y_test,y_pred = y_pred))
    run.log("rmse",rmse)
    print("rmse",rmse)

    # Serialize the model to local directory
    if not os.path.isdir(output_dir):
        os.makedirs(output_dir,exist_ok=True) 

    print("Save model ...")
    model_name = "model_alpha_" + str(hyperparam_alpha) + ".pkl" # Pickle file
    file_path = os.path.join(output_dir,model_name)
    joblib.dump(value = model,filename = file_path)

    # Upload the model
    run.upload_file(name = model_name,path_or_stream = file_path)

    # Complete the run
    run.complete()

实验视图

创作代码（即控制平面）

import os
from azureml.core import Workspace,Experiment,RunConfiguration,ScriptRunConfig,VERSION,Run

ws = Workspace.from_config()
exp = Experiment(workspace = ws,name = "diabetes-local-script-file")

# Create new run config obj
run_local_config = RunConfiguration()

# This means that when we run locally,all dependencies are already provided.
run_local_config.environment.python.user_managed_dependencies = True

# Create new script config
script_run_cfg = ScriptRunConfig(
    source_directory =  os.path.join(os.getcwd(),'code'),script = 'train.py',run_config = run_local_config) 

run = exp.submit(script_run_cfg)
run.wait_for_completion(show_output=True)

解决方法

简短答案

选项1：在运行内创建子运行

run = Run.get_context()将您当前正在运行的运行对象分配给run。因此，在超参数搜索的每次迭代中，您都将登录到同一运行。要解决此问题，您需要为每个超参数值创建子（或子）运行。您可以使用run.child_run()进行此操作。以下是实现此目标的模板。

run = Run.get_context()

for hyperparam_alpha in alphas:
    # Get the experiment run context
    run_child = run.child_run()
    print("Started run: ",run_child.id)
    run_child.log("train_split_size",x_train.size)

在diabetes-local-script-file实验页面上，如果您单击“包含子运行”页面，则可以看到运行9是父运行，运行10-19是子运行。 “运行9”详细信息页面上还有一个“儿童跑步”标签。

长答案

我强烈建议抽象化超参数搜索，使其远离数据平面（即train.py）并进入控制平面（即“创作代码”）。随着训练时间的增加，这变得特别有价值，并且您可以使用Azure ML的Hyperdrive任意地并行化，也可以更智能地选择Hyperparameters。

选项2从控制平面创建行程

从代码中删除循环，添加如下代码（full data and control here）

import argparse
from pprint import pprint

parser = argparse.ArgumentParser()
parser.add_argument('--alpha',type=float,default=0.5)
args = parser.parse_args()
print("all args:")
pprint(vars(args))

# use the variable like this
model = Ridge(args.alpha)

下面的

是如何使用脚本参数提交单次运行。要提交多个运行，只需在控制平面中使用循环即可。

alphas = [0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1.0] # Define hyperparameters

list_rcs = [ScriptRunConfig(
    source_directory =  os.path.join(os.getcwd(),'code'),script = 'train.py',arguments=['--alpha',a],run_config = run_local_config) for a in alphas]

list_runs = [exp.submit(rc) for rc in list_rcs]

选项3 Hyperdrive（建议使用IMHO方法）

通过这种方式，您可以将超参数源外包给Hyperdrive。用户界面还将根据您的需要准确报告结果，并且通过API，您可以轻松下载最佳模型。请注意，您不能再在本地使用此功能，而必须使用AMLCompute，但对我来说这是一个值得权衡的选择。This is a great overview。以下摘录（full code here）

param_sampling = GridParameterSampling( {
        "alpha": choice(0.1,1.0)
    }
)

estimator = Estimator(
    source_directory =  os.path.join(os.getcwd(),entry_script = 'train.py',compute_target=cpu_cluster,environment_definition=Environment.get(workspace=ws,name="AzureML-Tutorial")
)

hyperdrive_run_config = HyperDriveConfig(estimator=estimator,hyperparameter_sampling=param_sampling,policy=None,primary_metric_name="rmse",primary_metric_goal=PrimaryMetricGoal.MAXIMIZE,max_total_runs=10,max_concurrent_runs=4)

run = exp.submit(hyperdrive_run_config)
run.wait_for_completion(show_output=True)