问题描述
我一直在尝试修改有关图像分类的ML.NET教程，并且遇到了模式不匹配的问题，其中模型期望使用VarVector，并且该模型正在通过Vector。
原始示例创建一个IDataView，该列的名称为“ ImagePath”，其中填充了图像路径，然后使用MLContext.Transforms.LoadRawImageBytes（）生成字节数据的“ Image”列。然后将生成的IDataView馈送到Multiclassification.Trainers.ImageClassification（）对象。非常简单。
我希望将其概括化以包含更多的图像预处理。我希望预处理管道降低图像分辨率，然后在将图像提供给ImageClassification（）训练器之前将图像裁剪为正确的尺寸。它遵循与示例相同的流程，但更改了处理管道，以在图像处理管道中使用LoadImages（），ResizeImage（）和ExtractPixels（）估计器。下面是我正在使用的代码片段。
// Fill an IDataView with the Label and ImagePath information:
// The ImageData class is defined for this purpose and it is just:
//
// public class ImageData {
// public string ImagePath { get; set; }
// public string Label { get; set; }
// }
//
// Create a bunch of image names and their labels into a list of ImageData objects.
IEnumerable<ImageData> images = LoadImagesFromDirectory(folder: assetsRelativePath,useFolderNameAsLabel: true);
IDataView imageData = mlContext.Data.LoadFromEnumerable(images);
// Define the preprocessing pipeline to:
// 1. Map the Label to a unique key
// 2. Load the image data using the ImagePath column
// 3. Reduce the resolution of the image data
// 4. Crop the reduced resolution image data
// 5. Transform the image data from type "Image" to a byte array.
var preprocessingPipeline = mlContext.Transforms.Conversion.MapValuetoKey(
inputColumnName: "Label",outputColumnName: "LabelAsKey")
.Append(mlContext.Transforms.LoadImages(
inputColumnName: "ImagePath",outputColumnName: "RawImage",imageFolder: assetsRelativePath))
.Append(mlContext.Transforms.ResizeImages(
inputColumnName: "RawImage",outputColumnName: "ResReduxImage",imageWidth: 512,imageHeight: 459,resizing: microsoft.ml.Transforms.Image.ImageResizingEstimator.ResizingKind.Fill))
.Append(mlContext.Transforms.ResizeImages(
inputColumnName: "ResReduxImage",outputColumnName: "CroppedImage",resizing: microsoft.ml.Transforms.Image.ImageResizingEstimator.ResizingKind.IsoCrop,cropAnchor: microsoft.ml.Transforms.Image.ImageResizingEstimator.Anchor.Center))
.Append(mlContext.Transforms.ExtractPixels(
inputColumnName: "CroppedImage",outputColumnName: "Image",outputAsFloatArray: false,colorsToExtract: ImagePixelExtractingEstimator.ColorBits.Red));
// Transform the raw ImageData into the TransformedData that the Model will train on
IDataView preProcessedData = preprocessingPipeline.Fit(imageData).Transform(imageData);
// Partition the data set for training and validation
TrainTestData trainSplit = mlContext.Data.TrainTestSplit(data: preProcessedData,testFraction: 0.4);
TrainTestData validationTestSplit = mlContext.Data.TrainTestSplit(trainSplit.TestSet);
IDataView trainSet = trainSplit.TrainSet;
IDataView validationSet = validationTestSplit.TrainSet;
IDataView testSet = validationTestSplit.TestSet;
var classifierOptions = new ImageClassificationTrainer.Options()
{
FeatureColumnName = "Image",LabelColumnName = "LabelAsKey",ValidationSet = validationSet,Arch = ImageClassificationTrainer.Architecture.resnetV2101,MetricsCallback = (metrics) => Console.WriteLine(metrics),TestOnTrainSet = false,ReuseTrainSetBottleneckCachedValues = true,ReuseValidationSetBottleneckCachedValues = true,WorkspacePath=workspaceRelativePath
};
var trainingPipeline =
mlContext.MulticlassClassification.Trainers.ImageClassification(classifierOptions)
.Append(mlContext.Transforms.Conversion.MapKeyTovalue("PredictedLabel"));
// When I call Fit() in the next line of code,the following runtime error occurs:
// System.ArgumentOutOfRangeException: 'Schema mismatch for feature column 'Image':
// expected VarVector<Byte>,got Vector<Byte> Parameter name: inputSchema'
ITransformer trainedModel = trainingPipeline.Fit(trainSet);
我为VarVector和Vector不匹配而感到困惑。在过去的两天内，我尝试了以下方法，但均未成功：
任何人都可以提供一些有关如何使自己摆脱困境的指导吗？我真的想避免读入图像，对其进行预处理，将其写入文件，然后调用原始的LoadRawImageBytes（）函数。我不希望额外的文件系统工作。
解决方法
我弄清楚了如何进行这项工作。这里的问题是基础类型正确，但架构错误。我可以使用CustomMapping和一些属性魔术解决问题。有关CustomMapping的工作原理，请参见CustomMapping documentation。
更正后的代码段如下。
// SOMEWHERE OUTSIDE OF THE FUNCTION DEFINE:
// Define a class to represent the input column type for the custom transform
class InputData
{
[VectorType(1)] // attribute specifies vector type of known length
public VBuffer<Byte> Image1; // the VBuffer<> type actually represents the data
}
// Define a class to represent the output column type for the custom transform
class OutputData
{
// THE MAGICAL FIX: attribute specifies vector type of unknown length (i.e. VarVector)
[VectorType()]
public VBuffer<Byte> Image; // the VBuffer<> type actually represents the data
}
// ------------------------------------------------------------------
// INSIDE THE FUNCTION THAT WILL DO THE TRAINING
// ------------------------------------------------------------------
// Fill an IDataView with the Label and ImagePath information:
// The ImageData class is defined for this purpose and it is just:
//
// public class ImageData {
// public string ImagePath { get; set; }
// public string Label { get; set; }
// }
//
// Create a bunch of image names and their labels into a list of ImageData objects.
IEnumerable<ImageData> images = LoadImagesFromDirectory(folder: assetsRelativePath,useFolderNameAsLabel: true);
IDataView imageData = mlContext.Data.LoadFromEnumerable(images);
// Define the preprocessing pipeline to:
// 1. Map the Label to a unique key
// 2. Load the image data using the ImagePath column
// 3. Reduce the resolution of the image data
// 4. Crop the reduced resolution image data
// 5. Transform the image data from type "Image" to a byte array.
var preprocessingPipeline = mlContext.Transforms.Conversion.MapValueToKey(
inputColumnName: "Label",outputColumnName: "LabelAsKey")
.Append(mlContext.Transforms.LoadImages(
inputColumnName: "ImagePath",outputColumnName: "RawImage",imageFolder: assetsRelativePath))
.Append(mlContext.Transforms.ResizeImages(
inputColumnName: "RawImage",outputColumnName: "ResReduxImage",imageWidth: 512,imageHeight: 459,resizing: Microsoft.ML.Transforms.Image.ImageResizingEstimator.ResizingKind.Fill))
.Append(mlContext.Transforms.ResizeImages(
inputColumnName: "ResReduxImage",outputColumnName: "CroppedImage",resizing: Microsoft.ML.Transforms.Image.ImageResizingEstimator.ResizingKind.IsoCrop,cropAnchor: Microsoft.ML.Transforms.Image.ImageResizingEstimator.Anchor.Center))
.Append(mlContext.Transforms.ExtractPixels(
inputColumnName: "CroppedImage",outputColumnName: "Image1",outputAsFloatArray: false,colorsToExtract: ImagePixelExtractingEstimator.ColorBits.Red));
// Transform the raw ImageData into the TransformedData that the Model will train on
IDataView preProcessedData =
preprocessingPipeline.Fit(imageData).Transform(imageData);
// Create an action representing the custom transform...
// The data itself does not need to be changed at all,so this is just an
// identity transform
Action<InputData,OutputData> convertVecType
= (input,output) => output.Image = input.Image1;
var convertTypePipeline = mlContext.Transforms.CustomMapping(convertVecType,"convertVecType");
preProcessedData =
convertTypePipeline.Fit(preProcessedData).Transform(preProcessedData);
// Partition the data set for training and validation
TrainTestData trainSplit = mlContext.Data.TrainTestSplit(data: preProcessedData,testFraction: 0.4);
TrainTestData validationTestSplit = mlContext.Data.TrainTestSplit(trainSplit.TestSet);
IDataView trainSet = trainSplit.TrainSet;
IDataView validationSet = validationTestSplit.TrainSet;
IDataView testSet = validationTestSplit.TestSet;
var classifierOptions = new ImageClassificationTrainer.Options()
{
FeatureColumnName = "Image",LabelColumnName = "LabelAsKey",ValidationSet = validationSet,Arch = ImageClassificationTrainer.Architecture.ResnetV2101,MetricsCallback = (metrics) => Console.WriteLine(metrics),TestOnTrainSet = false,ReuseTrainSetBottleneckCachedValues = true,ReuseValidationSetBottleneckCachedValues = true,WorkspacePath=workspaceRelativePath
};
var trainingPipeline =
mlContext.MulticlassClassification.Trainers.ImageClassification(classifierOptions)
.Append(mlContext.Transforms.Conversion.MapKeyToValue("PredictedLabel"));
// When I call Fit() in the next line of code,the following runtime error occurs:
// System.ArgumentOutOfRangeException: 'Schema mismatch for feature column 'Image':
// expected VarVector<Byte>,got Vector<Byte> Parameter name: inputSchema'
ITransformer trainedModel = trainingPipeline.Fit(trainSet);