Pixel Formats Map
Mapping VisionCamera PixelFormats to native iOS/Android Pixel Formats
Requesting a Pixel Format
In a CameraFrameOutput, you have three options that affects your Frame's pixelFormat:
'native': Uses theCameraSessionConfig'snativePixelFormat- this requires zero conversion and is generally the most efficient.'yuv': Uses the platform default YUV format - oftenyuv-420-8-bit-full. In most cases, this requires little to no conversion and is very efficient.'rgb': Uses the platform default RGB format - oftenrgb-rgba-8-bitorrgb-bgra-8-bit. This always requires conversion as the Camera operates in YUV (or BayerRAW), and uses ~2.6x more memory.
Tip
It is recommended to use 'yuv' if possible, as it is the most efficient CPU-accessible format.
Only fall-back to 'rgb' if your Native Frame Processor Plugins do not natively support YUV buffers.
If your pipeline is entirely GPU-based, use 'native', as this requires zero conversions and keeps buffers entirely on the GPU.
Warning
Use 'native' with caution.
Most camera configurations stream in YUV formats like 'yuv-420-8-bit-full', but on some devices the native format could also be a vendor-specific format ('private') which is not readable by the CPU. You can still pass 'private' Frames to consumers like a Media Encoder or a Surface, but you cannot access its pixel data.
Some configurations natively stream in RAW pixel formats like raw-bayer-packed96-12-bit. Make sure to be prepared for this if you use 'native'.
Inspecting a Pixel Format
When you receive a Frame, you can inspect its pixelFormat to find out the precise Pixel Format it uses - here are some examples that map to native pixel formats: