Android：如何从相机获取原始图像数据

问题描述

目前，我计划使用 Android 设备（像素 2）中的相机提供的 RAW 数据开发一个应用。

我使用 ARCore 获取帧数据如下：

final Frame frame = session.update();
final Camera camera = frame.getCamera();
Image image = frame.acquireCameraImage();

这似乎不是我想要的原始数据。

我如何通过 ARCore 获取原始图像数据？ 或者有其他方法可以做到这一点？

---

我之前得到了位图。 但它不是原始图像。 我的意思是“原始”是未经任何处理的CMOS原始数据（RGB RAW DATA），如Gamma校正、AWB、AE或任何姿势处理。

解决方法

如果您只想获取相机图像的位图，则有多种方法可以做到这一点，而且您不需要使用 ARCore。

例如，您可以使用标准的 CameraX 解决方案在捕获图像时获取位图 - 请参阅此处：https://stackoverflow.com/a/63823500/334402

您也可以使用 OpenCV，如果您打算进行任何图像分析，这可能会很好。请注意，将其包含在项目中可能会有些棘手，而且在撰写本文时，大部分文档仍参考 Eclipse 而不是 Android Studio。

,

将android Image对象转换为字节数组

Image image = null;
try {
    image = frame.acquireCameraImage();
} catch (NotYetAvailableException e) {
    e.printStackTrace();
}

int h = image.getHeight();
int w = image.getWidth();
Bitmap outputBitmap = Bitmap.createBitmap(w,h,Bitmap.Config.ARGB_8888);
int pixelSizeBits = ImageFormat.getBitsPerPixel(ImageFormat.YUV_420_888);
byte[] yuvByteArray = new byte[(int)((h * w) * pixelSizeBits / 8)];
YuvToByteArray yuvToByteArray = new YuvToByteArray();
yuvToByteArray.setPixelCount(h * w);
yuvToByteArray.imageToByteArray(image,yuvByteArray);
image.close();

YuvToByteArray.kt

/*
 * Copyright 2020 The Android Open Source Project
 *
 * Licensed under the Apache License,Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     https://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing,software
 * distributed under the License is distributed on an "AS IS" BASIS,* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package com.memex.Helper

import android.graphics.ImageFormat
import android.graphics.Rect
import android.media.Image

class YuvToByteArray {

    var pixelCount: Int = -1

    fun imageToByteArray(image: Image,outputBuffer: ByteArray) {
        assert(image.format == ImageFormat.YUV_420_888)

        val imageCrop = Rect(0,image.width,image.height)
        val imagePlanes = image.planes

        imagePlanes.forEachIndexed { planeIndex,plane ->
            // How many values are read in input for each output value written
            // Only the Y plane has a value for every pixel,U and V have half the resolution i.e.
            //
            // Y Plane            U Plane    V Plane
            // ===============    =======    =======
            // Y Y Y Y Y Y Y Y    U U U U    V V V V
            // Y Y Y Y Y Y Y Y    U U U U    V V V V
            // Y Y Y Y Y Y Y Y    U U U U    V V V V
            // Y Y Y Y Y Y Y Y    U U U U    V V V V
            // Y Y Y Y Y Y Y Y
            // Y Y Y Y Y Y Y Y
            // Y Y Y Y Y Y Y Y
            val outputStride: Int

            // The index in the output buffer the next value will be written at
            // For Y it's zero,for U and V we start at the end of Y and interleave them i.e.
            //
            // First chunk        Second chunk
            // ===============    ===============
            // Y Y Y Y Y Y Y Y    V U V U V U V U
            // Y Y Y Y Y Y Y Y    V U V U V U V U
            // Y Y Y Y Y Y Y Y    V U V U V U V U
            // Y Y Y Y Y Y Y Y    V U V U V U V U
            // Y Y Y Y Y Y Y Y
            // Y Y Y Y Y Y Y Y
            // Y Y Y Y Y Y Y Y
            var outputOffset: Int

            when (planeIndex) {
                0 -> {
                    outputStride = 1
                    outputOffset = 0
                }
                1 -> {
                    outputStride = 2
                    // For NV21 format,U is in odd-numbered indices
                    outputOffset = pixelCount + 1
                }
                2 -> {
                    outputStride = 2
                    // For NV21 format,V is in even-numbered indices
                    outputOffset = pixelCount
                }
                else -> {
                    // Image contains more than 3 planes,something strange is going on
                    return@forEachIndexed
                }
            }

            val planeBuffer = plane.buffer
            val rowStride = plane.rowStride
            val pixelStride = plane.pixelStride

            // We have to divide the width and height by two if it's not the Y plane
            val planeCrop = if (planeIndex == 0) {
                imageCrop
            } else {
                Rect(
                    imageCrop.left / 2,imageCrop.top / 2,imageCrop.right / 2,imageCrop.bottom / 2
                )
            }

            val planeWidth = planeCrop.width()
            val planeHeight = planeCrop.height()

            // Intermediate buffer used to store the bytes of each row
            val rowBuffer = ByteArray(plane.rowStride)

            // Size of each row in bytes
            val rowLength = if (pixelStride == 1 && outputStride == 1) {
                planeWidth
            } else {
                // Take into account that the stride may include data from pixels other than this
                // particular plane and row,and that could be between pixels and not after every
                // pixel:
                //
                // |---- Pixel stride ----|                    Row ends here --> |
                // | Pixel 1 | Other Data | Pixel 2 | Other Data | ... | Pixel N |
                //
                // We need to get (N-1) * (pixel stride bytes) per row + 1 byte for the last pixel
                (planeWidth - 1) * pixelStride + 1
            }

            for (row in 0 until planeHeight) {
                // Move buffer position to the beginning of this row
                planeBuffer.position(
                    (row + planeCrop.top) * rowStride + planeCrop.left * pixelStride
                )

                if (pixelStride == 1 && outputStride == 1) {
                    // When there is a single stride value for pixel and output,we can just copy
                    // the entire row in a single step
                    planeBuffer.get(outputBuffer,outputOffset,rowLength)
                    outputOffset += rowLength
                } else {
                    // When either pixel or output have a stride > 1 we must copy pixel by pixel
                    planeBuffer.get(rowBuffer,rowLength)
                    for (col in 0 until planeWidth) {
                        outputBuffer[outputOffset] = rowBuffer[col * pixelStride]
                        outputOffset += outputStride
                    }
                }
            }
        }
    }
}

从 yuv 字节数组中获取 ARGB 位图的函数：

public Bitmap createImageBitmap(byte[] bytes,Bitmap outputBitmap,RenderScript renderScript,ScriptIntrinsicYuvToRGB intrinsicYuvToRGB,Type typeYUV) {
        Allocation inputAllocation = Allocation.createSized(renderScript,typeYUV.getElement(),bytes.length);
        Allocation outputAllocation = Allocation.createFromBitmap(renderScript,outputBitmap);
        inputAllocation.copyFrom(bytes);
        intrinsicYuvToRGB.setInput(inputAllocation);
        intrinsicYuvToRGB.forEach(outputAllocation);
        outputAllocation.copyTo(outputBitmap);
        return outputBitmap;
    }

如何使用：

// yuvByteArray and outputBitmap from the first code segment
// renderScript,intrinsicYuvToRGB,and typeYUV are provided here: https://github.com/android/renderscript-intrinsics-replacement-toolkit
Bitmap rgbBitmap = createImageBitmap(yuvByteArray,outputBitmap,renderScript,typeYUV);