CMSampleBuffer is a fundamental data structure in Apple's Core Media framework, used extensively in media processing applications. It encapsulates media sample data, such as audio or video frames, and provides metadata about the sample, including timing information and format descriptions. Understanding how to work with CMSampleBuffer is crucial for developers involved in media processing, as it allows for efficient handling and manipulation of media data.

In this article, we will explore how to create and manipulate CMSampleBuffer objects in an Apple development environment. We will provide practical examples using Swift, Apple's preferred programming language for iOS and macOS development.

Examples:

Creating a CMSampleBuffer from a CVPixelBuffer

To create a CMSampleBuffer from a CVPixelBuffer, you need to follow these steps:

1. Create a CVPixelBuffer.
2. Create a CMVideoFormatDescription.
3. Create a CMSampleBuffer using the CVPixelBuffer and CMVideoFormatDescription.

Here is a complete example in Swift:

import CoreMedia
import CoreVideo

func createSampleBuffer() -> CMSampleBuffer? {
    // Step 1: Create a CVPixelBuffer
    var pixelBuffer: CVPixelBuffer?
    let pixelBufferAttributes: [String: Any] = [
        kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_32ARGB,
        kCVPixelBufferWidthKey as String: 1920,
        kCVPixelBufferHeightKey as String: 1080
    ]
    CVPixelBufferCreate(kCFAllocatorDefault, 1920, 1080, kCVPixelFormatType_32ARGB, pixelBufferAttributes as CFDictionary, &pixelBuffer)

    guard let buffer = pixelBuffer else {
        print("Failed to create CVPixelBuffer")
        return nil
    }

    // Step 2: Create a CMVideoFormatDescription
    var videoInfo: CMVideoFormatDescription?
    CMVideoFormatDescriptionCreateForImageBuffer(allocator: kCFAllocatorDefault, imageBuffer: buffer, formatDescriptionOut: &videoInfo)

    guard let formatDescription = videoInfo else {
        print("Failed to create CMVideoFormatDescription")
        return nil
    }

    // Step 3: Create a CMSampleBuffer
    var sampleBuffer: CMSampleBuffer?
    var timingInfo = CMSampleTimingInfo()
    CMSampleBufferCreateForImageBuffer(allocator: kCFAllocatorDefault, imageBuffer: buffer, dataReady: true, makeDataReadyCallback: nil, refcon: nil, formatDescription: formatDescription, sampleTiming: &timingInfo, sampleBufferOut: &sampleBuffer)

    return sampleBuffer
}

if let sampleBuffer = createSampleBuffer() {
    print("Successfully created CMSampleBuffer")
} else {
    print("Failed to create CMSampleBuffer")
}

Extracting a CVPixelBuffer from a CMSampleBuffer

Once you have a CMSampleBuffer, you might want to extract the CVPixelBuffer for further processing. Here’s how you can do it:

import CoreMedia

func extractPixelBuffer(from sampleBuffer: CMSampleBuffer) -> CVPixelBuffer? {
    guard let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else {
        print("Failed to get image buffer from sample buffer")
        return nil
    }

    return imageBuffer
}

// Assuming `sampleBuffer` is a valid CMSampleBuffer
if let sampleBuffer = createSampleBuffer() {
    if let pixelBuffer = extractPixelBuffer(from: sampleBuffer) {
        print("Successfully extracted CVPixelBuffer from CMSampleBuffer")
    } else {
        print("Failed to extract CVPixelBuffer from CMSampleBuffer")
    }
}