Resizer

interface Resizer extends HybridObject

Represents a GPU-accelerated Frame resizer and converter.

You can create instances of the Resizer via ResizerFactory.createResizer(...).

Discussion

The Resizer internally allocates a memory pool, which can grow quite large, depending on texture size. When you are done with the Resizer, it is recommended to dispose() it to free up resources.

Methods

resize()

resize(frame: Frame): GPUFrame

Resize the given frame using the options this Resizer was configured with.

Returns

A GPUFrame of the converted and resized pixel data. The returned GPUFrame must be disposed (via dispose()) to ensure the pipeline can run continuously.

Discussion

• On iOS, the input Frame's pixelFormat has to be 'yuv-420-8-bit-full', which is configured on the CameraFrameOutput by setting FrameOutputOptions.pixelFormat to 'yuv'.
• On Android, the input Frame's pixelFormat can be any pixel format - even 'private'.

For maximum performance, you should find a CameraFormat whos' nativePixelFormat is 'yuv-420-8-bit-full' on iOS, and 'private' on Android. Then, set FrameOutputOptions.pixelFormat to 'native' on Android, and 'native' if the Format's nativePixelFormat really is 'yuv-420-8-bit-full' (or 'yuv' otherwise) for iOS.

For ML models that expect CHW / NCHW tensors, configure the Resizer with pixelLayout: 'planar'.

Examples

Simple resize

const resizer = ...
const frameOutput = useFrameOutput({
  pixelFormat: 'yuv',
  onFrame(frame) {
    'worklet'
    if (resizer != null) {
      const resized = resizer.resize(frame)
      const buffer = resized.getPixelBuffer()
      resized.dispose()
    }
    frame.dispose()
  }
})

Maximum performance resize

const targetPixelFormat = Platform.select({
  ios: 'yuv-420-8-bit-full',
  android: 'private'
})
const device = ...
const format = device.formats.find((f) =>
  f.nativePixelFormat === targetPixelFormat
)
const resizer = ...
const frameOutput = useFrameOutput({
  pixelFormat: format?.nativePixelFormat === targetPixelFormat ? 'native' : 'yuv',
  onFrame(frame) {
    'worklet'
    if (resizer != null) {
      const resized = resizer.resize(frame)
      const buffer = resized.getPixelBuffer()
      resized.dispose()
    }
    frame.dispose()
  }
})