问题描述
iPhone 12/12 pro支持以10位而不是8位格式记录杜比视觉HDR视频，但是如果AVCaptureVideoDataOutput支持提供10位样本缓冲区的交付（使用AVAssetWriter可以附加到视频文件中），则从iOS 14.1 SDK尚不清楚。有没有人知道是否可以在SDK中使用？
编辑：Apple的Clips应用程序等许多应用程序已开始支持Dolby Vision 10位视频录制。但是我尝试了所有可用的API，包括videoHDREnabled，但无法正常工作。因此，明确的问题是如何使用AVFoundation API录制HDR（杜比视觉）视频？
EDIT2：我能够找出支持10位像素缓冲区格式的设备格式（即“ x420”，而不是将420v或420f作为媒体子类型的格式）。在iPhone 12 mini上，4种设备格式支持 kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange 中的10位像素缓冲区传送，即使AVFoundation文档说这不是受支持的像素格式（在iOS上，引用为-“ ，唯一受支持的密钥是kCVPixelBufferPixelFormatTypeKey。支持的像素格式是kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange，kCVPixelFormatType_420YpCbCr8BiPlanarFullRange和kCVPixelFormatType_32BGRA 。”）。下一步是确定用于录制的HDR格式是否可以手动选择为Dolby Vision，HLG或HDR10。
解决方法
11月26日更新。
正如@Deepak在他自己的答案和评论中所述，带有'x420'标签的格式可使相机在HLG模式下工作。 IP12 Pro中所有可用的支持HLG的格式已在下面更新。
原始答案
对于iOS 14.2，我可以从AVCaptureDevice实例中转储所有可用的格式，似乎日志输出很能说明问题。如下所述，希望将AVCaptureDevice.activeFormat设置为HDR +宽彩色格式之一即可。
<AVCaptureDeviceFormat: 0x282d8daf0 'vide'/'x420' 1280x 720,{ 1- 30 fps},HRSI:4096x2304,fov:68.161,supports vis,max zoom:120.00 (upscales @2.91),AF System:2,ISO:34.0-3264.0,SS:0.000014-1.000000,supports wide color,supports depth>
<AVCaptureDeviceFormat: 0x282d8dac0 'vide'/'x420' 1280x 720,{ 1- 60 fps},supports depth>
<AVCaptureDeviceFormat: 0x282d8da50 'vide'/'x420' 1920x1080,max zoom:120.00 (upscales @1.94),supports depth,supports multicam>
<AVCaptureDeviceFormat: 0x282d8da30 'vide'/'x420' 1920x1080,supports multicam>
<AVCaptureDeviceFormat: 0x282d8d9e0 'vide'/'x420' 1920x1440,HRSI:4032x3024,fov:67.096,max zoom:189.00 (upscales @2.10),supports multicam>
<AVCaptureDeviceFormat: 0x282d8d950 'vide'/'x420' 3840x2160,max zoom:125.25 (upscales @1.00),supports multicam>
截至11月23日，这仍在进行中，我认为需要一些共同的努力，或者一些Apple工程师可以对此进行研究。
我相信我已经观看了有关此主题的所有可用WWDC17 / 18/19/20会议，并且在新版iPhone 12上，这里有一些发现。
仅在iPhone 12及更高版本上才能从相机捕获HDR并直接保存为10位HLG视频。这就是它在产品发布中所声称的，并且我从朋友的新手机中获得了示例视频，它按预期运行。
在WWDC2020中，Export HDR media in your app with AVFoundation声称：
在这一点上，我想简要介绍一下哪些Apple平台可以支持HDR导出。
iOS在装有Apple A10 Fusion芯片或更高版本的设备上支持HEVC硬件编码。
幸运的是A10设备已经存在了一段时间，其历史可以追溯到iPhone 7、2018年发布的iPad和2019年的iPod touch。
对于Mac，HEVC和Apple ProRes软件编码器在所有Mac上均可用。
HEVC硬件编码通常在2017年和运行新macOS的更新Mac上可用。
硬件编码将大大加快导出速度。
在此视频中，它也声明了HDR export only works 10bit HEVC encoding，因此A10 + SoC应该具有10位HEVC编码功能。这是一个猜测，我可以在iPhone 11和SE2上的官方Photo应用程序中编辑iPhone12 HLG视频，并且书写性能（4k @ 60p，HLG）相当不错，这是一个很好的指标。但是，我没有运气用代码实现此功能，视频中列出的示例代码可能无法完整显示，而且我还很难找到有效的演示。从理论上讲，较旧的设备也应具有记录10位HLG的能力，否则摄像机的热/功率预算将受到限制。
但是，其中唯一相关的HDR密钥是VideoProfileLevelKey，在使用HEVC编解码器导出HDR时，必须将其设置为HEVC_Main10_AutoLevel。
请注意，不支持8位HEVC HDR，并且此密钥不适用于ProRes导出。
好吧，让我们现在花些时间来总结一下如何配置当我输出为两种常见HDR格式（HLG和HDR10）的键。下表显示了用于导出HLG文件的相关HDR设置。
另一个值得一再观看的视频：Edit and play back HDR video with AVFoundation
在测试过程中，我确实获得了启用HDR的CVPixelBuffer（format：kCVPixelFormatType_420YpCbCr10BiPlanarFullRange），并且可以从示例HLG视频中正确进行颜色管理。这是从我的控制台日志中转储的，并且可以在任何支持iOS 14的设备上运行，即使使用的是相当老的iPhone6s（A9），因为此处仅涉及10位HEVC解码。
_displayLinkDidRefresh():121 - Optional(<CVPixelBuffer 0x281300500 width=3840 height=2160 pixelFormat=xf20 iosurface=0x282008050 planes=2 poolName=450:decode_1>
<Plane 0 width=3840 height=2160 bytesPerRow=7680>
<Plane 1 width=1920 height=1080 bytesPerRow=7680>
<attributes={
PixelFormatDescription = {
BitsPerComponent = 10;
CGBitmapContextCompatibility = 0;
CGImageCompatibility = 0;
ComponentRange = FullRange;
ContainsAlpha = 0;
ContainsGrayscale = 0;
ContainsRGB = 0;
ContainsYCbCr = 1;
FillExtendedPixelsCallback = {length = 24,bytes = 0x0000000000000000b48ab8a1010000000000000000000000};
IOSurfaceCoreAnimationCompatibility = 1;
IOSurfaceCoreAnimationCompatibilityHTPCOK = 1;
IOSurfaceOpenGLESTextureCompatibility = 1;
OpenGLESCompatibility = 1;
PixelFormat = 2019963440;
Planes = (
{
BitsPerBlock = 16;
HorizontalSubsampling = 1;
VerticalSubsampling = 1;
},{
BitsPerBlock = 32;
BlackBlock = {length = 4,bytes = 0x00800080};
HorizontalSubsampling = 2;
VerticalSubsampling = 2;
}
);
};
} propagatedAttachments={
CVFieldCount = 1;
CVImageBufferChromaLocationBottomField = Left;
CVImageBufferChromaLocationTopField = Left;
CVImageBufferColorPrimaries = "ITU_R_2020";
CVImageBufferTransferFunction = "ITU_R_2100_HLG";
CVImageBufferYCbCrMatrix = "ITU_R_2020";
QTMovieTime = {
TimeScale = 600;
TimeValue = 12090;
};
} nonPropagatedAttachments={
}>)
好吧，给出的答案均不正确，因此我手持iPhone 12 mini进行了研究，这就是我发现的结果。
AVFoundation文档是静默的，有时甚至是不正确的。人们可以从文档中推断出不可能获得10位HDR样本缓冲区，特别是如果人们阅读了 AVCaptureVideoDataOutput 的 videoSettings 属性的文档：
On iOS,the only supported key is kCVPixelBufferPixelFormatTypeKey.
Supported pixel formats are kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange,kCVPixelFormatType_420YpCbCr8BiPlanarFullRange and kCVPixelFormatType_32BGRA
从文档中可以看出，永远无法获得10位帧。但是在探查-[AVCaptureDevice formats]时，可以找到4种不同的格式，并且将mediaSubtype作为'x420'，即kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange，这是一种10位格式。将-[AVCaptureDevice activeFormat]设置为这4种格式之一之后，AVCaptureVideoDataOutput将样本缓冲区格式更改为kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange！ AVCaptureDevice的活动颜色空间也更改为AVCaptureColorSpace_HLG_BT2020。
很难说没有手持设备，但是我想假设iPhone 12中的AVCaptureDevice（部分）将支持支持HDR交付的format（{{1} }。
相应的isVideoHDRSupported的{{1}}可能会列出AVCaptureVideoDataOutput和类似类型作为选项。
万一有人在寻找更多信息，我在iOS AVFoundation中找到了以下列表，请注意，它声明它在CoreVideo中不支持所有这些功能。
/*
CoreVideo pixel format type constants.
CoreVideo does not provide support for all of these formats; this list just defines their names.
*/
public var kCVPixelFormatType_1Monochrome: OSType { get } /* 1 bit indexed */
public var kCVPixelFormatType_2Indexed: OSType { get } /* 2 bit indexed */
public var kCVPixelFormatType_4Indexed: OSType { get } /* 4 bit indexed */
public var kCVPixelFormatType_8Indexed: OSType { get } /* 8 bit indexed */
public var kCVPixelFormatType_1IndexedGray_WhiteIsZero: OSType { get } /* 1 bit indexed gray,white is zero */
public var kCVPixelFormatType_2IndexedGray_WhiteIsZero: OSType { get } /* 2 bit indexed gray,white is zero */
public var kCVPixelFormatType_4IndexedGray_WhiteIsZero: OSType { get } /* 4 bit indexed gray,white is zero */
public var kCVPixelFormatType_8IndexedGray_WhiteIsZero: OSType { get } /* 8 bit indexed gray,white is zero */
public var kCVPixelFormatType_16BE555: OSType { get } /* 16 bit BE RGB 555 */
public var kCVPixelFormatType_16LE555: OSType { get } /* 16 bit LE RGB 555 */
public var kCVPixelFormatType_16LE5551: OSType { get } /* 16 bit LE RGB 5551 */
public var kCVPixelFormatType_16BE565: OSType { get } /* 16 bit BE RGB 565 */
public var kCVPixelFormatType_16LE565: OSType { get } /* 16 bit LE RGB 565 */
public var kCVPixelFormatType_24RGB: OSType { get } /* 24 bit RGB */
public var kCVPixelFormatType_24BGR: OSType { get } /* 24 bit BGR */
public var kCVPixelFormatType_32ARGB: OSType { get } /* 32 bit ARGB */
public var kCVPixelFormatType_32BGRA: OSType { get } /* 32 bit BGRA */
public var kCVPixelFormatType_32ABGR: OSType { get } /* 32 bit ABGR */
public var kCVPixelFormatType_32RGBA: OSType { get } /* 32 bit RGBA */
public var kCVPixelFormatType_64ARGB: OSType { get } /* 64 bit ARGB,16-bit big-endian samples */
public var kCVPixelFormatType_64RGBALE: OSType { get } /* 64 bit RGBA,16-bit little-endian full-range (0-65535) samples */
public var kCVPixelFormatType_48RGB: OSType { get } /* 48 bit RGB,16-bit big-endian samples */
public var kCVPixelFormatType_32AlphaGray: OSType { get } /* 32 bit AlphaGray,16-bit big-endian samples,black is zero */
public var kCVPixelFormatType_16Gray: OSType { get } /* 16 bit Grayscale,black is zero */
public var kCVPixelFormatType_30RGB: OSType { get } /* 30 bit RGB,10-bit big-endian samples,2 unused padding bits (at least significant end). */
public var kCVPixelFormatType_422YpCbCr8: OSType { get } /* Component Y'CbCr 8-bit 4:2:2,ordered Cb Y'0 Cr Y'1 */
public var kCVPixelFormatType_4444YpCbCrA8: OSType { get } /* Component Y'CbCrA 8-bit 4:4:4:4,ordered Cb Y' Cr A */
public var kCVPixelFormatType_4444YpCbCrA8R: OSType { get } /* Component Y'CbCrA 8-bit 4:4:4:4,rendering format. full range alpha,zero biased YUV,ordered A Y' Cb Cr */
public var kCVPixelFormatType_4444AYpCbCr8: OSType { get } /* Component Y'CbCrA 8-bit 4:4:4:4,ordered A Y' Cb Cr,full range alpha,video range Y'CbCr. */
public var kCVPixelFormatType_4444AYpCbCr16: OSType { get } /* Component Y'CbCrA 16-bit 4:4:4:4,video range Y'CbCr,16-bit little-endian samples. */
public var kCVPixelFormatType_444YpCbCr8: OSType { get } /* Component Y'CbCr 8-bit 4:4:4 */
public var kCVPixelFormatType_422YpCbCr16: OSType { get } /* Component Y'CbCr 10,12,14,16-bit 4:2:2 */
public var kCVPixelFormatType_422YpCbCr10: OSType { get } /* Component Y'CbCr 10-bit 4:2:2 */
public var kCVPixelFormatType_444YpCbCr10: OSType { get } /* Component Y'CbCr 10-bit 4:4:4 */
public var kCVPixelFormatType_420YpCbCr8Planar: OSType { get } /* Planar Component Y'CbCr 8-bit 4:2:0. baseAddr points to a big-endian CVPlanarPixelBufferInfo_YCbCrPlanar struct */
public var kCVPixelFormatType_420YpCbCr8PlanarFullRange: OSType { get } /* Planar Component Y'CbCr 8-bit 4:2:0,full range. baseAddr points to a big-endian CVPlanarPixelBufferInfo_YCbCrPlanar struct */
public var kCVPixelFormatType_422YpCbCr_4A_8BiPlanar: OSType { get } /* First plane: Video-range Component Y'CbCr 8-bit 4:2:2,ordered Cb Y'0 Cr Y'1; second plane: alpha 8-bit 0-255 */
public var kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange: OSType { get } /* Bi-Planar Component Y'CbCr 8-bit 4:2:0,video-range (luma=[16,235] chroma=[16,240]). baseAddr points to a big-endian CVPlanarPixelBufferInfo_YCbCrBiPlanar struct */
public var kCVPixelFormatType_420YpCbCr8BiPlanarFullRange: OSType { get } /* Bi-Planar Component Y'CbCr 8-bit 4:2:0,full-range (luma=[0,255] chroma=[1,255]). baseAddr points to a big-endian CVPlanarPixelBufferInfo_YCbCrBiPlanar struct */
public var kCVPixelFormatType_422YpCbCr8BiPlanarVideoRange: OSType { get } /* Bi-Planar Component Y'CbCr 8-bit 4:2:2,240]). baseAddr points to a big-endian CVPlanarPixelBufferInfo_YCbCrBiPlanar struct */
public var kCVPixelFormatType_422YpCbCr8BiPlanarFullRange: OSType { get } /* Bi-Planar Component Y'CbCr 8-bit 4:2:2,255]). baseAddr points to a big-endian CVPlanarPixelBufferInfo_YCbCrBiPlanar struct */
public var kCVPixelFormatType_444YpCbCr8BiPlanarVideoRange: OSType { get } /* Bi-Planar Component Y'CbCr 8-bit 4:4:4,240]). baseAddr points to a big-endian CVPlanarPixelBufferInfo_YCbCrBiPlanar struct */
public var kCVPixelFormatType_444YpCbCr8BiPlanarFullRange: OSType { get } /* Bi-Planar Component Y'CbCr 8-bit 4:4:4,255]). baseAddr points to a big-endian CVPlanarPixelBufferInfo_YCbCrBiPlanar struct */
public var kCVPixelFormatType_422YpCbCr8_yuvs: OSType { get } /* Component Y'CbCr 8-bit 4:2:2,ordered Y'0 Cb Y'1 Cr */
public var kCVPixelFormatType_422YpCbCr8FullRange: OSType { get } /* Component Y'CbCr 8-bit 4:2:2,full range,ordered Y'0 Cb Y'1 Cr */
public var kCVPixelFormatType_OneComponent8: OSType { get } /* 8 bit one component,black is zero */
public var kCVPixelFormatType_TwoComponent8: OSType { get } /* 8 bit two component,black is zero */
public var kCVPixelFormatType_30RGBLEPackedWideGamut: OSType { get } /* little-endian RGB101010,2 MSB are zero,wide-gamut (384-895) */
public var kCVPixelFormatType_ARGB2101010LEPacked: OSType { get } /* little-endian ARGB2101010 full-range ARGB */
public var kCVPixelFormatType_OneComponent10: OSType { get } /* 10 bit little-endian one component,stored as 10 MSBs of 16 bits,black is zero */
public var kCVPixelFormatType_OneComponent12: OSType { get } /* 12 bit little-endian one component,stored as 12 MSBs of 16 bits,black is zero */
public var kCVPixelFormatType_OneComponent16: OSType { get } /* 16 bit little-endian one component,black is zero */
public var kCVPixelFormatType_TwoComponent16: OSType { get } /* 16 bit little-endian two component,black is zero */
public var kCVPixelFormatType_OneComponent16Half: OSType { get } /* 16 bit one component IEEE half-precision float,16-bit little-endian samples */
public var kCVPixelFormatType_OneComponent32Float: OSType { get } /* 32 bit one component IEEE float,32-bit little-endian samples */
public var kCVPixelFormatType_TwoComponent16Half: OSType { get } /* 16 bit two component IEEE half-precision float,16-bit little-endian samples */
public var kCVPixelFormatType_TwoComponent32Float: OSType { get } /* 32 bit two component IEEE float,32-bit little-endian samples */
public var kCVPixelFormatType_64RGBAHalf: OSType { get } /* 64 bit RGBA IEEE half-precision float,16-bit little-endian samples */
public var kCVPixelFormatType_128RGBAFloat: OSType { get } /* 128 bit RGBA IEEE float,32-bit little-endian samples */
public var kCVPixelFormatType_14Bayer_GRBG: OSType { get } /* Bayer 14-bit Little-Endian,packed in 16-bits,ordered G R G R... alternating with B G B G... */
public var kCVPixelFormatType_14Bayer_RGGB: OSType { get } /* Bayer 14-bit Little-Endian,ordered R G R G... alternating with G B G B... */
public var kCVPixelFormatType_14Bayer_BGGR: OSType { get } /* Bayer 14-bit Little-Endian,ordered B G B G... alternating with G R G R... */
public var kCVPixelFormatType_14Bayer_GBRG: OSType { get } /* Bayer 14-bit Little-Endian,ordered G B G B... alternating with R G R G... */
public var kCVPixelFormatType_DisparityFloat16: OSType { get } /* IEEE754-2008 binary16 (half float),describing the normalized shift when comparing two images. Units are 1/meters: ( pixelShift / (pixelFocalLength * baselineInMeters) ) */
public var kCVPixelFormatType_DisparityFloat32: OSType { get } /* IEEE754-2008 binary32 float,describing the normalized shift when comparing two images. Units are 1/meters: ( pixelShift / (pixelFocalLength * baselineInMeters) ) */
public var kCVPixelFormatType_DepthFloat16: OSType { get } /* IEEE754-2008 binary16 (half float),describing the depth (distance to an object) in meters */
public var kCVPixelFormatType_DepthFloat32: OSType { get } /* IEEE754-2008 binary32 float,describing the depth (distance to an object) in meters */
public var kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange: OSType { get } /* 2 plane YCbCr10 4:2:0,each 10 bits in the MSBs of 16bits,video-range (luma=[64,940] chroma=[64,960]) */
public var kCVPixelFormatType_422YpCbCr10BiPlanarVideoRange: OSType { get } /* 2 plane YCbCr10 4:2:2,960]) */
public var kCVPixelFormatType_444YpCbCr10BiPlanarVideoRange: OSType { get } /* 2 plane YCbCr10 4:4:4,960]) */
public var kCVPixelFormatType_420YpCbCr10BiPlanarFullRange: OSType { get } /* 2 plane YCbCr10 4:2:0,full-range (Y range 0-1023) */
public var kCVPixelFormatType_422YpCbCr10BiPlanarFullRange: OSType { get } /* 2 plane YCbCr10 4:2:2,full-range (Y range 0-1023) */
public var kCVPixelFormatType_444YpCbCr10BiPlanarFullRange: OSType { get } /* 2 plane YCbCr10 4:4:4,full-range (Y range 0-1023) */
public var kCVPixelFormatType_420YpCbCr8VideoRange_8A_TriPlanar: OSType { get } /* first and second planes as per 420YpCbCr8BiPlanarVideoRange (420v),alpha 8 bits in third plane full-range. No CVPlanarPixelBufferInfo struct. */
public var kCVPixelFormatType_16VersatileBayer: OSType { get } /* Single plane Bayer 16-bit little-endian sensor element ("sensel") samples from full-size decoding of ProRes RAW images; Bayer pattern (sensel ordering) and other raw conversion information is described via buffer attachments */
public var kCVPixelFormatType_64RGBA_DownscaledProResRAW: OSType { get } /* Single plane 64-bit RGBA (16-bit little-endian samples) from downscaled decoding of ProRes RAW images; components--which may not be co-sited with one another--are sensel values and require raw conversion,information for which is described via buffer attachments */
强制使用 BT2020 格式是确保您以杜比视界拍摄的正确方法。执行此操作需要 iOS 14.1 或更高版本。这是我如何执行此操作的简短片段：
// Setup your session
session.beginConfiguration()
session.sessionPreset = .hd1280x720
// Add your camera to the session
let camera = AVCaptureDevice.default(.builtInWideAngleCamera,for: .video,position: .front)
let cameraInput = try AVCaptureDeviceInput(device: camera)
session.addInput(cameraInput)
// Important! Commit the session configuration before configuring your camera
session.commitConfiguration()
// Configure camera
try camera.lockForConfiguration()
// Force HDR on
camera.automaticallyAdjustsVideoHDREnabled = false
camera.isVideoHDREnabled = true
// Find the first 720p format that supports the correct colorspace
let desiredColorSpace = AVCaptureColorSpace.HLG_BT2020
let desiredFormat = camera.formats.first { format in
// You could of course choose a different resolution if desired
format.formatDescription.dimensions == CMVideoDimensions(width: 1280,height: 720) &&
format.supportedColorSpaces.contains(desiredColorSpace)
}
// Set the HDR format
if let format = desiredFormat {
camera.activeFormat = format
camera.activeColorSpace = desiredColorSpace
} else {
assertionFailure("Counldn't find HDR camera format")
}
camera.unlockForConfiguration()