reachy_mini/pr_1069/en/API/media.md

# Media

## Media Manager[[reachy_mini.media.media_manager.MediaManager]]

#### reachy_mini.media.media_manager.MediaManager[[reachy_mini.media.media_manager.MediaManager]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_manager.py#L103)

Media Manager for handling camera and audio devices.

This class provides a unified interface for managing both camera and audio
devices across different backends.  It handles initialization,
configuration, and cleanup of media resources.

closereachy_mini.media.media_manager.MediaManager.closehttps://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_manager.py#L174[]
Close the media manager and release resources.

**Parameters:**

logger : Logger instance for media-related messages.

backend : The selected media backend (after deprecation resolution).

camera : Camera device instance, or `None`.

audio : Audio device instance, or `None`.
#### get_DoA[[reachy_mini.media.media_manager.MediaManager.get_DoA]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_manager.py#L375)

Get the Direction of Arrival (DoA) from the microphone array.

**Returns:**

A tuple `(angle_radians, speech_detected)`, or `None` if the
audio system is not available.
#### get_audio_sample[[reachy_mini.media.media_manager.MediaManager.get_audio_sample]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_manager.py#L275)

Get an audio sample from the audio device.

**Returns:**

The recorded audio sample, or `None` if no data is available.
#### get_frame[[reachy_mini.media.media_manager.MediaManager.get_frame]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_manager.py#L238)

Get a frame from the camera.

**Returns:**

The captured BGR frame as a numpy array with shape
`(height, width, 3)`, or `None` if the camera is not
available.
#### get_input_audio_samplerate[[reachy_mini.media.media_manager.MediaManager.get_input_audio_samplerate]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_manager.py#L287)

Get the input samplerate of the audio device.
#### get_input_channels[[reachy_mini.media.media_manager.MediaManager.get_input_channels]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_manager.py#L301)

Get the number of input channels of the audio device.
#### get_output_audio_samplerate[[reachy_mini.media.media_manager.MediaManager.get_output_audio_samplerate]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_manager.py#L294)

Get the output samplerate of the audio device.
#### get_output_channels[[reachy_mini.media.media_manager.MediaManager.get_output_channels]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_manager.py#L308)

Get the number of output channels of the audio device.
#### play_sound[[reachy_mini.media.media_manager.MediaManager.play_sound]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_manager.py#L252)

Play a sound file.

Note:
If the audio backend is not initialised, a warning is logged
and the call is silently ignored.

**Parameters:**

sound_file : Path to the sound file to play.
#### push_audio_sample[[reachy_mini.media.media_manager.MediaManager.push_audio_sample]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_manager.py#L329)

Push audio data to the output device.

**Parameters:**

data : Audio samples as a float32 array.  Shape should be `(num_samples,)` for mono or `(num_samples, channels)` for multi-channel.  The manager adapts the data to match the output device's channel count before forwarding it.
#### start_playing[[reachy_mini.media.media_manager.MediaManager.start_playing]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_manager.py#L322)

Start playing audio.
#### start_recording[[reachy_mini.media.media_manager.MediaManager.start_recording]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_manager.py#L268)

Start recording audio.
#### stop_playing[[reachy_mini.media.media_manager.MediaManager.stop_playing]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_manager.py#L368)

Stop playing audio.
#### stop_recording[[reachy_mini.media.media_manager.MediaManager.stop_recording]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_manager.py#L315)

Stop recording audio.

## Audio[[reachy_mini.media.audio_base.AudioBase]]

#### reachy_mini.media.audio_base.AudioBase[[reachy_mini.media.audio_base.AudioBase]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_base.py#L34)

Abstract audio backend.

apply_audio_configreachy_mini.media.audio_base.AudioBase.apply_audio_confighttps://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_base.py#L110[{"name": "config", "val": ": collections.abc.Sequence[tuple[str, collections.abc.Sequence[float | int]]]"}, {"name": "verify", "val": ": bool = True"}, {"name": "write_settle_seconds", "val": ": float = 0.1"}]- **config** -- Sequence of `(parameter_name, values)` pairs to write.
- **verify** -- When true, read each parameter back after writing it.
- **write_settle_seconds** -- Delay after each write before readback.0True when all parameters were written and verified successfully.
False when the ReSpeaker audio board is unavailable or a parameter
write/readback fails.
Apply caller-provided audio control parameters to the ReSpeaker.

This opens a short-lived ReSpeaker USB handle, writes each parameter in
`config`, and optionally verifies the written values. The SDK does
not provide default values for these parameters; callers should pass the
values tuned for their own app.

**Parameters:**

SAMPLE_RATE : Default sample rate (16 000 Hz — ReSpeaker hardware).

CHANNELS : Number of audio channels (2 — stereo).

**Returns:**

True when all parameters were written and verified successfully.
False when the ReSpeaker audio board is unavailable or a parameter
write/readback fails.
#### cleanup[[reachy_mini.media.audio_base.AudioBase.cleanup]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_base.py#L148)

Release shared resources (DoA USB device).
#### get_DoA[[reachy_mini.media.audio_base.AudioBase.get_DoA]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_base.py#L100)

Get the Direction of Arrival (DoA) from the ReSpeaker.

**Returns:**

A tuple `(angle_radians, speech_detected)` or `None`
if the device is unavailable.
#### get_audio_sample[[reachy_mini.media.audio_base.AudioBase.get_audio_sample]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_base.py#L52)

Pull the next recorded audio chunk.

**Returns:**

A float32 array of shape `(num_samples, 2)` (stereo), or
`None` if no data is available yet.
#### get_input_audio_samplerate[[reachy_mini.media.audio_base.AudioBase.get_input_audio_samplerate]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_base.py#L68)

Input sample rate in Hz (16 000).
#### get_input_channels[[reachy_mini.media.audio_base.AudioBase.get_input_channels]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_base.py#L76)

Return the number of input channels (2 — stereo).
#### get_output_audio_samplerate[[reachy_mini.media.audio_base.AudioBase.get_output_audio_samplerate]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_base.py#L72)

Output sample rate in Hz (16 000).
#### get_output_channels[[reachy_mini.media.audio_base.AudioBase.get_output_channels]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_base.py#L80)

Return the number of output channels (2 — stereo).
#### play_sound[[reachy_mini.media.audio_base.AudioBase.play_sound]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_base.py#L177)

Play a sound file.
#### push_audio_sample[[reachy_mini.media.audio_base.AudioBase.push_audio_sample]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_base.py#L172)

Push audio data to the output.
#### set_max_output_buffers[[reachy_mini.media.audio_base.AudioBase.set_max_output_buffers]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_base.py#L84)

Limit the number of queued playback buffers.

**Parameters:**

max_buffers : Maximum number of buffers to queue.
#### start_playing[[reachy_mini.media.audio_base.AudioBase.start_playing]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_base.py#L162)

Start the playback pipeline.
#### start_recording[[reachy_mini.media.audio_base.AudioBase.start_recording]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_base.py#L152)

Start capturing audio from the microphone.
#### stop_playing[[reachy_mini.media.audio_base.AudioBase.stop_playing]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_base.py#L167)

Stop the playback pipeline.
#### stop_recording[[reachy_mini.media.audio_base.AudioBase.stop_recording]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_base.py#L157)

Stop the recording pipeline.

#### reachy_mini.media.audio_gstreamer.GStreamerAudio[[reachy_mini.media.audio_gstreamer.GStreamerAudio]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_gstreamer.py#L81)

Audio implementation using GStreamer.

Extends `AudioBase` with two GStreamer-specific helpers:

- `clear_output_buffer()`: flush queued playback data without stopping
  the pipeline (no-op by default; useful before refilling the buffer).
- `clear_player()`: flush the playback appsrc immediately via GStreamer
  flush events, dropping any queued audio.

cleanupreachy_mini.media.audio_gstreamer.GStreamerAudio.cleanuphttps://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_gstreamer.py#L469[]
Release all resources (pipelines, USB devices).
#### clear_output_buffer[[reachy_mini.media.audio_gstreamer.GStreamerAudio.clear_output_buffer]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_gstreamer.py#L334)

Flush queued playback data so it is not played.

A low `set_max_output_buffers` value may make this unnecessary
for most use-cases.
#### clear_player[[reachy_mini.media.audio_gstreamer.GStreamerAudio.clear_player]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_gstreamer.py#L343)

Flush the player's appsrc to drop any queued audio immediately.
#### delete_sound[[reachy_mini.media.audio_gstreamer.GStreamerAudio.delete_sound]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_gstreamer.py#L450)

No-op for the local backend.

**Returns:**

Always `False`.
#### list_sounds[[reachy_mini.media.audio_gstreamer.GStreamerAudio.list_sounds]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_gstreamer.py#L441)

No-op for the local backend.

**Returns:**

An empty list.
#### play_sound[[reachy_mini.media.audio_gstreamer.GStreamerAudio.play_sound]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_gstreamer.py#L357)

Play a sound file through the Reachy Mini Audio card.

The file is played via a GStreamer `playbin` routed to the same
audio sink used by the push-based playback pipeline.

**Parameters:**

sound_file : Absolute path **or** filename relative to the built-in assets directory.
#### push_audio_sample[[reachy_mini.media.audio_gstreamer.GStreamerAudio.push_audio_sample]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_gstreamer.py#L299)

Push audio data to the speaker.

**Parameters:**

data : Audio samples as a float32 array.  Shape should be `(num_samples, 2)` for stereo or `(num_samples,)` for mono (the caller is responsible for channel adaptation).
#### start_playing[[reachy_mini.media.audio_gstreamer.GStreamerAudio.start_playing]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_gstreamer.py#L293)

Start the playback pipeline so `push_audio_sample` can feed data.
#### upload_sound[[reachy_mini.media.audio_gstreamer.GStreamerAudio.upload_sound]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_gstreamer.py#L432)

No-op for the local backend — the file is already accessible.

**Returns:**

The unchanged *sound_file* path.

### Audio Utils Functions[[reachy_mini.media.audio_utils.get_respeaker_card_number]]

#### reachy_mini.media.audio_utils.get_respeaker_card_number[[reachy_mini.media.audio_utils.get_respeaker_card_number]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_utils.py#L75)

Return the card number of the ReSpeaker sound card, or 0 if not found.

Note:
This function runs 'arecord -l' to list available audio capture devices
and processes the output to find Reachy Mini Audio or ReSpeaker devices.
It's primarily used on Linux systems with ALSA audio configuration.

The function returns:
- Positive integer: Card number of detected Reachy Mini Audio device
- 0: No Reachy Mini Audio device found, using default sound card
- -1: Error occurred while trying to detect audio devices

Example:
```python
card_num = get_respeaker_card_number()
if card_num > 0:
    print(f"Using Reachy Mini Audio card {card_num}")
elif card_num == 0:
    print("Using default sound card")
else:
    print("Error detecting audio devices")
```

**Returns:**

`int`

The card number of the detected ReSpeaker/Reachy Mini Audio device.
Returns 0 if no specific device is found (uses default sound card),
or -1 if there's an error running the detection command.

#### reachy_mini.media.audio_utils.has_reachymini_asoundrc[[reachy_mini.media.audio_utils.has_reachymini_asoundrc]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_utils.py#L118)

Check if ~/.asoundrc exists and contains both reachymini_audio_sink and reachymini_audio_src.

Note:
This function checks for the presence of the ALSA configuration file
~/.asoundrc and verifies that it contains the necessary configuration
entries for Reachy Mini audio devices (reachymini_audio_sink and
reachymini_audio_src). These entries are required for proper audio
routing and device management.

Example:
```python
if has_reachymini_asoundrc():
    print("Reachy Mini audio configuration is properly set up")
else:
    print("Need to configure Reachy Mini audio devices")
    write_asoundrc_to_home()  # Create the configuration
```

**Returns:**

`bool`

True if ~/.asoundrc exists and contains the required Reachy Mini
audio configuration entries, False otherwise.

#### reachy_mini.media.audio_utils.check_reachymini_asoundrc[[reachy_mini.media.audio_utils.check_reachymini_asoundrc]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_utils.py#L149)

Check if ~/.asoundrc exists and is correctly configured for Reachy Mini Audio.

#### reachy_mini.media.audio_utils.write_asoundrc_to_home[[reachy_mini.media.audio_utils.write_asoundrc_to_home]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_utils.py#L168)

Write the .asoundrc file with Reachy Mini audio configuration to the user's home directory.

This function creates an ALSA configuration file (.asoundrc) in the user's home directory
that configures the ReSpeaker sound card for proper audio routing and multi-client support.
The configuration enables simultaneous audio input and output access, which is essential
for the Reachy Mini Wireless version's audio functionality.

The generated configuration includes:
- Default audio device settings pointing to the ReSpeaker sound card
- dmix plugin for multi-client audio output (reachymini_audio_sink)
- dsnoop plugin for multi-client audio input (reachymini_audio_src)
- Proper buffer and sample rate settings for optimal performance

Note:
This function automatically detects the ReSpeaker card number and creates a configuration
tailored to the detected hardware. It is primarily used for the Reachy Mini Wireless version.

The configuration file will be created at ~/.asoundrc and will overwrite any existing file
with the same name. Existing audio configurations should be backed up before calling this function.

### Audio Control Utils Functions[[reachy_mini.media.audio_control_utils.ReSpeaker]]

#### reachy_mini.media.audio_control_utils.ReSpeaker[[reachy_mini.media.audio_control_utils.ReSpeaker]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_control_utils.py#L181)

Class to interface with the ReSpeaker XVF3800 USB device.

apply_audio_configreachy_mini.media.audio_control_utils.ReSpeaker.apply_audio_confighttps://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_control_utils.py#L331[{"name": "config", "val": ": collections.abc.Sequence[tuple[str, collections.abc.Sequence[float | int]]]"}, {"name": "verify", "val": ": bool = True"}, {"name": "write_settle_seconds", "val": ": float = 0.1"}]- **config** -- Parameter names and values to write.
- **verify** -- When true, read each parameter back after writing it.
- **write_settle_seconds** -- Delay after each write before readback.0True when all parameters were written and verified successfully.
Apply a set of audio control parameters to the ReSpeaker.

**Parameters:**

config : Parameter names and values to write.

verify : When true, read each parameter back after writing it.

write_settle_seconds : Delay after each write before readback.

**Returns:**

True when all parameters were written and verified successfully.
#### close[[reachy_mini.media.audio_control_utils.ReSpeaker.close]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_control_utils.py#L459)

Close the interface.
#### read[[reachy_mini.media.audio_control_utils.ReSpeaker.read]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_control_utils.py#L249)

Read data from a specified parameter on the ReSpeaker device.
#### read_values[[reachy_mini.media.audio_control_utils.ReSpeaker.read_values]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_control_utils.py#L326)

Read a parameter and decode it into numeric values.
#### write[[reachy_mini.media.audio_control_utils.ReSpeaker.write]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_control_utils.py#L190)

Write data to a specified parameter on the ReSpeaker device.

#### reachy_mini.media.audio_control_utils.find[[reachy_mini.media.audio_control_utils.find]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_control_utils.py#L464)

Find and return the ReSpeaker USB device with the given Vendor ID and Product ID.

Note:
This function searches for USB devices with the specified Vendor ID
and Product ID using libusb backend. The default values target
XMOS XVF3800 devices used in ReSpeaker microphone arrays.

Example:
```python
from reachy_mini.media.audio_control_utils import find

# Find default ReSpeaker device
respeaker = find()
if respeaker is not None:
    print("Found ReSpeaker device")
    respeaker.close()

# Find specific device
custom_device = find(vid=0x1234, pid=0x5678)
```

**Parameters:**

vid (int) : USB Vendor ID to search for. Default: 0x2886 (XMOS).

pid (int) : USB Product ID to search for. Default: 0x001A (XMOS XVF3800).

**Returns:**

`ReSpeaker | None`

A ReSpeaker object if the device is found,
None otherwise.

#### reachy_mini.media.audio_control_utils.init_respeaker_usb[[reachy_mini.media.audio_control_utils.init_respeaker_usb]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_control_utils.py#L502)

Initialize the ReSpeaker USB device. Looks for both new and beta device IDs.

Note:
This function attempts to initialize a ReSpeaker microphone array by
searching for USB devices with known Vendor and Product IDs. It tries:
1. New Reachy Mini Audio firmware (0x38FB:0x1001) - preferred
2. Old ReSpeaker firmware (0x2886:0x001A) - with warning to update

The function handles USB backend errors gracefully and returns
None if no compatible device is found or if initialization fails.

Example:
```python
from reachy_mini.media.audio_control_utils import init_respeaker_usb

# Initialize ReSpeaker device
respeaker = init_respeaker_usb()
if respeaker is not None:
    print("ReSpeaker initialized successfully")
    # Use the device...
    doa = respeaker.read("DOA_VALUE_RADIANS")
    respeaker.close()
else:
    print("No ReSpeaker device found")
```

**Returns:**

`Optional[ReSpeaker]`

A ReSpeaker object if a compatible device is found,
None otherwise.

## Camera[[reachy_mini.media.camera_gstreamer.GStreamerCamera]]

#### reachy_mini.media.camera_gstreamer.GStreamerCamera[[reachy_mini.media.camera_gstreamer.GStreamerCamera]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/camera_gstreamer.py#L76)

Camera that reads BGR frames from the daemon's local IPC endpoint.

The WebRTC daemon exposes BGR camera frames via a local IPC mechanism:

- Linux / macOS: `unixfdsink` / `unixfdsrc` (Unix domain socket)
- Windows: `win32ipcvideosink` / `win32ipcvideosrc` (shared memory)

Since the daemon's IPC branch already converts to BGR, the reader
pipeline is simply `source → queue → appsink` with no extra
conversion.

closereachy_mini.media.camera_gstreamer.GStreamerCamera.closehttps://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/camera_gstreamer.py#L274[]
Stop the pipeline and release resources.

**Parameters:**

camera_specs : Camera specifications (resolutions, intrinsics, …).
#### open[[reachy_mini.media.camera_gstreamer.GStreamerCamera.open]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/camera_gstreamer.py#L243)

Start the GStreamer pipeline and begin receiving frames.
#### read[[reachy_mini.media.camera_gstreamer.GStreamerCamera.read]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/camera_gstreamer.py#L259)

Pull the latest BGR frame from the IPC endpoint.

**Returns:**

A NumPy array of shape `(height, width, 3)` in BGR order,
or `None` if no frame is available within the timeout.

### Camera Utils Functions[[reachy_mini.media.camera_utils.undistort_points]]

#### reachy_mini.media.camera_utils.undistort_points[[reachy_mini.media.camera_utils.undistort_points]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/camera_utils.py#L46)

Undistort a single pixel coordinate to normalized camera coordinates.

Pure numpy equivalent of cv2.undistortPoints(). Supports the OpenCV distortion
model with up to 12 coefficients (rational model + thin prism):
D = (k1, k2, p1, p2, k3, k4, k5, k6, s1, s2, s3, s4)

Also works with 5-coefficient models (k1, k2, p1, p2, k3) and zero-distortion.

The algorithm matches OpenCV's cvUndistortPointsInternal:
1. Remove camera intrinsics to get normalized distorted coordinates.
2. Iteratively solve for undistorted coordinates using a damped
   fixed-point method with adaptive step size.

Reference:
OpenCV distortion model and undistortPoints algorithm:
https://docs.opencv.org/4.x/d9/d0c/group__calib3d.html
https://github.com/opencv/opencv/blob/4.x/modules/calib3d/src/undistort.dispatch.cpp

**Parameters:**

u : Horizontal pixel coordinate.

v : Vertical pixel coordinate.

K : 3x3 camera intrinsic matrix [[fx, 0, cx], [0, fy, cy], [0, 0, 1]].

D : Distortion coefficients array. Supports lengths 0, 4, 5, 8, 12, or 14. Unused positions default to 0.

max_iterations : Maximum number of iterations (default 20).

epsilon : Convergence threshold in pixel reprojection error (default 0.01).

**Returns:**

`Tuple (x_n, y_n)`

Normalized undistorted coordinates (on the z=1 plane).

#### reachy_mini.media.camera_utils.scale_intrinsics[[reachy_mini.media.camera_utils.scale_intrinsics]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/camera_utils.py#L180)

Scale camera intrinsics for a different resolution with cropping.

**Parameters:**

K_original : Original 3x3 camera matrix

original_size : (width, height) of original calibration

target_size : (width, height) of target resolution

crop_scale : Scale factor due to digital zoom/crop (>1 means more zoomed in)

**Returns:**

`K_scaled`

Adjusted camera matrix for target resolution

### Camera Constants[[reachy_mini.media.camera_constants.CameraResolution]]

#### reachy_mini.media.camera_constants.CameraResolution[[reachy_mini.media.camera_constants.CameraResolution]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/camera_constants.py#L37)

Base class for camera resolutions.

Enumeration of standardized camera resolutions and frame rates supported
by Reachy Mini cameras. Each enum value contains a tuple of (width, height, fps).

Note:
The enum values are tuples containing (width, height, frames_per_second, crop_factor).
Not all resolutions are supported by all camera models - check the specific
camera specifications for available resolutions.

Example:
```python
from reachy_mini.media.camera_constants import CameraResolution

# Get resolution information
res = CameraResolution.R1280x720at30fps
width, height, fps, crop_factor = res.value
print(f"Resolution: {width}x{height}@{fps}fps")

# Check if a resolution is supported by a camera
from reachy_mini.media.camera_constants import ReachyMiniLiteCamSpecs
res = CameraResolution.R1920x1080at60fps
if res in ReachyMiniLiteCamSpecs.available_resolutions:
    print("This resolution is supported")
```

**Parameters:**

R1536x864at40fps : 1536x864 resolution at 40 fps

R1280x720at60fps : 1280x720 resolution at 60 fps (HD)

R1280x720at30fps : 1280x720 resolution at 30 fps (HD)

R1920x1080at30fps : 1920x1080 resolution at 30 fps (Full HD)

R1920x1080at60fps : 1920x1080 resolution at 60 fps (Full HD)

R2304x1296at30fps : 2304x1296 resolution at 30 fps

R1600x1200at30fps : 1600x1200 resolution at 30 fps

R3264x2448at30fps : 3264x2448 resolution at 30 fps

R3264x2448at10fps : 3264x2448 resolution at 10 fps

R3840x2592at30fps : 3840x2592 resolution at 30 fps

R3840x2592at10fps : 3840x2592 resolution at 10 fps

R3840x2160at30fps : 3840x2160 resolution at 30 fps (4K UHD)

R3840x2160at10fps : 3840x2160 resolution at 10 fps (4K UHD)

R3072x1728at10fps : 3072x1728 resolution at 10 fps

R4608x2592at10fps : 4608x2592 resolution at 10 fps

#### reachy_mini.media.camera_constants.CameraSpecs[[reachy_mini.media.camera_constants.CameraSpecs]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/camera_constants.py#L110)

Base camera specifications.

Dataclass containing specifications for a camera model, including supported
resolutions, calibration parameters, and USB identification information.

Note:
The intrinsic matrix K has the format:
[[fx,  0, cx],
[ 0, fy, cy],
[ 0,  0,  1]]

Where fx, fy are focal lengths in pixels, and cx, cy are the principal
point coordinates (typically near the image center).

Example:
```python
from reachy_mini.media.camera_constants import CameraSpecs

# Create a custom camera specification
custom_specs = CameraSpecs(
    name="custom_camera",
    available_resolutions=[CameraResolution.R1280x720at30fps],
    default_resolution=CameraResolution.R1280x720at30fps,
    vid=0x1234,
    pid=0x5678,
    K=np.array([[800, 0, 640], [0, 800, 360], [0, 0, 1]]),
    D=np.zeros(5)
)
```

**Parameters:**

name (str) : Human-readable name of the camera model.

available_resolutions (List[CameraResolution]) : List of supported resolutions and frame rates for this camera model.

default_resolution (CameraResolution) : Default resolution used when the camera is initialized.

vid (int) : USB Vendor ID for identifying this camera model.

pid (int) : USB Product ID for identifying this camera model.

K (npt.NDArray[np.float64]) : 3x3 camera intrinsic matrix containing focal lengths and principal point coordinates.

D (npt.NDArray[np.float64]) : 5-element array containing distortion coefficients (k1, k2, p1, p2, k3) for radial and tangential distortion.

#### reachy_mini.media.camera_constants.ArducamSpecs[[reachy_mini.media.camera_constants.ArducamSpecs]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/camera_constants.py#L180)

Arducam camera specifications.

#### reachy_mini.media.camera_constants.ReachyMiniLiteCamSpecs[[reachy_mini.media.camera_constants.ReachyMiniLiteCamSpecs]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/camera_constants.py#L199)

Reachy Mini Lite camera specifications.

#### reachy_mini.media.camera_constants.ReachyMiniWirelessCamSpecs[[reachy_mini.media.camera_constants.ReachyMiniWirelessCamSpecs]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/camera_constants.py#L246)

Reachy Mini Wireless camera specifications.

#### reachy_mini.media.camera_constants.OlderRPiCamSpecs[[reachy_mini.media.camera_constants.OlderRPiCamSpecs]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/camera_constants.py#L264)

Older Raspberry Pi camera specifications. Keeping for compatibility.

#### reachy_mini.media.camera_constants.MujocoCameraSpecs[[reachy_mini.media.camera_constants.MujocoCameraSpecs]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/camera_constants.py#L273)

Mujoco simulated camera specifications.

#### reachy_mini.media.camera_constants.GenericWebcamSpecs[[reachy_mini.media.camera_constants.GenericWebcamSpecs]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/camera_constants.py#L301)

Generic webcam specifications (fallback for any webcam).

## WebRTC[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient]]

#### reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/webrtc_client_gstreamer.py#L68)

WebRTC client that provides both camera frames and audio.

Implements the same public API surface as `GStreamerCamera` (for
video) and `GStreamerAudio` (for audio) so that `MediaManager`
can assign the same instance to both its `camera` and `audio`
slots.

cleanupreachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.cleanuphttps://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/webrtc_client_gstreamer.py#L616[]
Release all resources.
#### clear_output_buffer[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.clear_output_buffer]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/webrtc_client_gstreamer.py#L467)

No-op (WebRTC send chain does not buffer significantly).
#### close[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.close]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/webrtc_client_gstreamer.py#L352)

Stop the WebRTC pipeline.
#### delete_sound[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.delete_sound]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/webrtc_client_gstreamer.py#L584)

Delete a sound file from the daemon's temporary sound directory.

**Parameters:**

filename : Name of the file to delete (not a full path).

**Returns:**

`True` if the file was deleted, `False` otherwise.
#### get_DoA[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.get_DoA]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/webrtc_client_gstreamer.py#L607)

Get the Direction of Arrival from the ReSpeaker.

**Returns:**

A tuple `(angle_radians, speech_detected)` or `None`.
#### list_sounds[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.list_sounds]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/webrtc_client_gstreamer.py#L562)

List sound files in the daemon's temporary sound directory.

**Returns:**

A list of filenames, or an empty list on error.
#### open[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.open]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/webrtc_client_gstreamer.py#L334)

Start the WebRTC pipeline (both video and audio).
#### play_sound[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.play_sound]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/webrtc_client_gstreamer.py#L491)

Play a sound file on the robot's speaker via the daemon REST API.

If *sound_file* is a local path that exists on this machine the
file is automatically uploaded to the daemon's temporary sound
directory (skipping the upload when a file with the same name is
already present).  Otherwise the filename is sent as-is and the
daemon resolves it from its built-in assets or filesystem.

**Parameters:**

sound_file : Absolute local path **or** asset filename (e.g. `"wake_up.wav"`).
#### push_audio_sample[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.push_audio_sample]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/webrtc_client_gstreamer.py#L471)

Push audio data to the remote peer via WebRTC.

**Parameters:**

data : Float32 audio samples.
#### read[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.read]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/webrtc_client_gstreamer.py#L338)

Pull the latest BGR video frame.

**Returns:**

A NumPy array of shape `(height, width, 3)` or `None`.
#### start_playing[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.start_playing]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/webrtc_client_gstreamer.py#L450)

No-op — audio send chain is set up automatically on WebRTC connection.
#### start_recording[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.start_recording]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/webrtc_client_gstreamer.py#L356)

No-op — recording starts automatically with `open()`.
#### stop_playing[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.stop_playing]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/webrtc_client_gstreamer.py#L454)

Reset the PTS counter for the send chain and stop daemon-side sound.
#### stop_recording[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.stop_recording]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/webrtc_client_gstreamer.py#L360)

No-op — managed by `close()`.
#### upload_sound[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.upload_sound]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/webrtc_client_gstreamer.py#L532)

Upload a local sound file to the daemon's temporary directory.

**Parameters:**

sound_file : Local path to the sound file.

**Returns:**

The absolute path of the file on the daemon.

#### reachy_mini.media.media_server.GstMediaServer[[reachy_mini.media.media_server.GstMediaServer]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_server.py#L56)

Daemon-side GStreamer media server.

Owns the camera and audio hardware and distributes media to consumers:

- **IPC branch** — raw BGR frames via `unixfdsink` / `win32ipcvideosink`
  for on-device applications (`GStreamerCamera` reads from this).
- **WebRTC branch** — encoded video + audio via `webrtcsink` for remote
  clients (`GstWebRTCClient` connects to this).
- **Sound playback** — `playbin` for playing WAV files on the speaker.

closereachy_mini.media.media_server.GstMediaServer.closehttps://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_server.py#L147[]
Release GStreamer resources (MainLoop, bus watch).

**Parameters:**

camera_specs (CameraSpecs) : Specifications of the detected camera.

resized_K (npt.NDArray[np.float64]) : Camera intrinsic matrix for current resolution.
#### play_sound[[reachy_mini.media.media_server.GstMediaServer.play_sound]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_server.py#L890)

Play a sound file on the robot's speaker.

Uses GStreamer's playbin element with a platform-aware audio sink.
This is used for daemon-side sounds (wake-up, sleep, etc.).

**Parameters:**

sound_file : Path to the sound file to play. If the file is not found at the given path, it is looked up in the assets directory.
#### send_data_message[[reachy_mini.media.media_server.GstMediaServer.send_data_message]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_server.py#L1001)

Send a message to connected peers via data channel.

**Parameters:**

message : The string message to send

peer_id : If specified, send only to this peer. Otherwise broadcast to all.
#### set_message_handler[[reachy_mini.media.media_server.GstMediaServer.set_message_handler]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_server.py#L989)

Set a callback for incoming data channel messages.

**Parameters:**

handler : Callback function that receives (peer_id, message)
#### start[[reachy_mini.media.media_server.GstMediaServer.start]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_server.py#L875)

Rebuild the pipeline from scratch and start it.

Rebuilding ensures a clean state after stop() released all hardware.
#### stop[[reachy_mini.media.media_server.GstMediaServer.stop]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_server.py#L885)

Stop the pipeline and release all hardware (camera, audio).
#### stop_sound[[reachy_mini.media.media_server.GstMediaServer.stop_sound]]

[Source](https://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_server.py#L940)

Stop the currently playing sound file.

If no sound is currently playing this is a no-op.