reachy_mini/pr_1069/en/API/media.md

Media

Media Manager[[reachy_mini.media.media_manager.MediaManager]]

reachy_mini.media.media_manager.MediaManager[[reachy_mini.media.media_manager.MediaManager]]

Source

Media Manager for handling camera and audio devices.

This class provides a unified interface for managing both camera and audio devices across different backends. It handles initialization, configuration, and cleanup of media resources.

closereachy_mini.media.media_manager.MediaManager.closehttps://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_manager.py#L174[] Close the media manager and release resources.

Parameters:

logger : Logger instance for media-related messages.

backend : The selected media backend (after deprecation resolution).

camera : Camera device instance, or None.

audio : Audio device instance, or None.

get_DoA[[reachy_mini.media.media_manager.MediaManager.get_DoA]]

Source

Get the Direction of Arrival (DoA) from the microphone array.

Returns:

A tuple (angle_radians, speech_detected), or None if the audio system is not available.

get_audio_sample[[reachy_mini.media.media_manager.MediaManager.get_audio_sample]]

Source

Get an audio sample from the audio device.

Returns:

The recorded audio sample, or None if no data is available.

get_frame[[reachy_mini.media.media_manager.MediaManager.get_frame]]

Source

Get a frame from the camera.

Returns:

The captured BGR frame as a numpy array with shape (height, width, 3), or None if the camera is not available.

get_input_audio_samplerate[[reachy_mini.media.media_manager.MediaManager.get_input_audio_samplerate]]

Source

Get the input samplerate of the audio device.

get_input_channels[[reachy_mini.media.media_manager.MediaManager.get_input_channels]]

Source

Get the number of input channels of the audio device.

get_output_audio_samplerate[[reachy_mini.media.media_manager.MediaManager.get_output_audio_samplerate]]

Source

Get the output samplerate of the audio device.

get_output_channels[[reachy_mini.media.media_manager.MediaManager.get_output_channels]]

Source

Get the number of output channels of the audio device.

play_sound[[reachy_mini.media.media_manager.MediaManager.play_sound]]

Source

Play a sound file.

Note: If the audio backend is not initialised, a warning is logged and the call is silently ignored.

Parameters:

sound_file : Path to the sound file to play.

push_audio_sample[[reachy_mini.media.media_manager.MediaManager.push_audio_sample]]

Source

Push audio data to the output device.

Parameters:

data : Audio samples as a float32 array. Shape should be (num_samples,) for mono or (num_samples, channels) for multi-channel. The manager adapts the data to match the output device's channel count before forwarding it.

start_playing[[reachy_mini.media.media_manager.MediaManager.start_playing]]

Source

Start playing audio.

start_recording[[reachy_mini.media.media_manager.MediaManager.start_recording]]

Source

Start recording audio.

stop_playing[[reachy_mini.media.media_manager.MediaManager.stop_playing]]

Source

Stop playing audio.

stop_recording[[reachy_mini.media.media_manager.MediaManager.stop_recording]]

Source

Stop recording audio.

Audio[[reachy_mini.media.audio_base.AudioBase]]

reachy_mini.media.audio_base.AudioBase[[reachy_mini.media.audio_base.AudioBase]]

Source

Abstract audio backend.

apply_audio_configreachy_mini.media.audio_base.AudioBase.apply_audio_confighttps://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_base.py#L110[{"name": "config", "val": ": collections.abc.Sequence[tuple[str, collections.abc.Sequence[float | int]]]"}, {"name": "verify", "val": ": bool = True"}, {"name": "write_settle_seconds", "val": ": float = 0.1"}]- config -- Sequence of (parameter_name, values) pairs to write.

• verify -- When true, read each parameter back after writing it.
• write_settle_seconds -- Delay after each write before readback.0True when all parameters were written and verified successfully. False when the ReSpeaker audio board is unavailable or a parameter write/readback fails. Apply caller-provided audio control parameters to the ReSpeaker.

This opens a short-lived ReSpeaker USB handle, writes each parameter in config, and optionally verifies the written values. The SDK does not provide default values for these parameters; callers should pass the values tuned for their own app.

Parameters:

SAMPLE_RATE : Default sample rate (16 000 Hz — ReSpeaker hardware).

CHANNELS : Number of audio channels (2 — stereo).

Returns:

True when all parameters were written and verified successfully. False when the ReSpeaker audio board is unavailable or a parameter write/readback fails.

cleanup[[reachy_mini.media.audio_base.AudioBase.cleanup]]

Source

Release shared resources (DoA USB device).

get_DoA[[reachy_mini.media.audio_base.AudioBase.get_DoA]]

Source

Get the Direction of Arrival (DoA) from the ReSpeaker.

Returns:

A tuple (angle_radians, speech_detected) or None if the device is unavailable.

get_audio_sample[[reachy_mini.media.audio_base.AudioBase.get_audio_sample]]

Source

Pull the next recorded audio chunk.

Returns:

A float32 array of shape (num_samples, 2) (stereo), or None if no data is available yet.

get_input_audio_samplerate[[reachy_mini.media.audio_base.AudioBase.get_input_audio_samplerate]]

Source

Input sample rate in Hz (16 000).

get_input_channels[[reachy_mini.media.audio_base.AudioBase.get_input_channels]]

Source

Return the number of input channels (2 — stereo).

get_output_audio_samplerate[[reachy_mini.media.audio_base.AudioBase.get_output_audio_samplerate]]

Source

Output sample rate in Hz (16 000).

get_output_channels[[reachy_mini.media.audio_base.AudioBase.get_output_channels]]

Source

Return the number of output channels (2 — stereo).

play_sound[[reachy_mini.media.audio_base.AudioBase.play_sound]]

Source

Play a sound file.

push_audio_sample[[reachy_mini.media.audio_base.AudioBase.push_audio_sample]]

Source

Push audio data to the output.

set_max_output_buffers[[reachy_mini.media.audio_base.AudioBase.set_max_output_buffers]]

Source

Limit the number of queued playback buffers.

Parameters:

max_buffers : Maximum number of buffers to queue.

start_playing[[reachy_mini.media.audio_base.AudioBase.start_playing]]

Source

Start the playback pipeline.

start_recording[[reachy_mini.media.audio_base.AudioBase.start_recording]]

Source

Start capturing audio from the microphone.

stop_playing[[reachy_mini.media.audio_base.AudioBase.stop_playing]]

Source

Stop the playback pipeline.

stop_recording[[reachy_mini.media.audio_base.AudioBase.stop_recording]]

Source

Stop the recording pipeline.

reachy_mini.media.audio_gstreamer.GStreamerAudio[[reachy_mini.media.audio_gstreamer.GStreamerAudio]]

Source

Audio implementation using GStreamer.

Extends AudioBase with two GStreamer-specific helpers:

• clear_output_buffer(): flush queued playback data without stopping the pipeline (no-op by default; useful before refilling the buffer).
• clear_player(): flush the playback appsrc immediately via GStreamer flush events, dropping any queued audio.

cleanupreachy_mini.media.audio_gstreamer.GStreamerAudio.cleanuphttps://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_gstreamer.py#L469[] Release all resources (pipelines, USB devices).

clear_output_buffer[[reachy_mini.media.audio_gstreamer.GStreamerAudio.clear_output_buffer]]

Source

Flush queued playback data so it is not played.

A low set_max_output_buffers value may make this unnecessary for most use-cases.

clear_player[[reachy_mini.media.audio_gstreamer.GStreamerAudio.clear_player]]

Source

Flush the player's appsrc to drop any queued audio immediately.

delete_sound[[reachy_mini.media.audio_gstreamer.GStreamerAudio.delete_sound]]

Source

No-op for the local backend.

Returns:

Always False.

list_sounds[[reachy_mini.media.audio_gstreamer.GStreamerAudio.list_sounds]]

Source

No-op for the local backend.

Returns:

An empty list.

play_sound[[reachy_mini.media.audio_gstreamer.GStreamerAudio.play_sound]]

Source

Play a sound file through the Reachy Mini Audio card.

The file is played via a GStreamer playbin routed to the same audio sink used by the push-based playback pipeline.

Parameters:

sound_file : Absolute path or filename relative to the built-in assets directory.

push_audio_sample[[reachy_mini.media.audio_gstreamer.GStreamerAudio.push_audio_sample]]

Source

Push audio data to the speaker.

Parameters:

data : Audio samples as a float32 array. Shape should be (num_samples, 2) for stereo or (num_samples,) for mono (the caller is responsible for channel adaptation).

start_playing[[reachy_mini.media.audio_gstreamer.GStreamerAudio.start_playing]]

Source

Start the playback pipeline so push_audio_sample can feed data.

upload_sound[[reachy_mini.media.audio_gstreamer.GStreamerAudio.upload_sound]]

Source

No-op for the local backend — the file is already accessible.

Returns:

The unchanged sound_file path.

Audio Utils Functions[[reachy_mini.media.audio_utils.get_respeaker_card_number]]

reachy_mini.media.audio_utils.get_respeaker_card_number[[reachy_mini.media.audio_utils.get_respeaker_card_number]]

Source

Return the card number of the ReSpeaker sound card, or 0 if not found.

Note: This function runs 'arecord -l' to list available audio capture devices and processes the output to find Reachy Mini Audio or ReSpeaker devices. It's primarily used on Linux systems with ALSA audio configuration.

The function returns:

• Positive integer: Card number of detected Reachy Mini Audio device
• 0: No Reachy Mini Audio device found, using default sound card
• -1: Error occurred while trying to detect audio devices

Example:

card_num = get_respeaker_card_number()
if card_num > 0:
    print(f"Using Reachy Mini Audio card {card_num}")
elif card_num == 0:
    print("Using default sound card")
else:
    print("Error detecting audio devices")

Returns:

int

The card number of the detected ReSpeaker/Reachy Mini Audio device. Returns 0 if no specific device is found (uses default sound card), or -1 if there's an error running the detection command.

reachy_mini.media.audio_utils.has_reachymini_asoundrc[[reachy_mini.media.audio_utils.has_reachymini_asoundrc]]

Source

Check if ~/.asoundrc exists and contains both reachymini_audio_sink and reachymini_audio_src.

Note: This function checks for the presence of the ALSA configuration file ~/.asoundrc and verifies that it contains the necessary configuration entries for Reachy Mini audio devices (reachymini_audio_sink and reachymini_audio_src). These entries are required for proper audio routing and device management.

Example:

if has_reachymini_asoundrc():
    print("Reachy Mini audio configuration is properly set up")
else:
    print("Need to configure Reachy Mini audio devices")
    write_asoundrc_to_home()  # Create the configuration

Returns:

bool

True if ~/.asoundrc exists and contains the required Reachy Mini audio configuration entries, False otherwise.

reachy_mini.media.audio_utils.check_reachymini_asoundrc[[reachy_mini.media.audio_utils.check_reachymini_asoundrc]]

Source

Check if ~/.asoundrc exists and is correctly configured for Reachy Mini Audio.

reachy_mini.media.audio_utils.write_asoundrc_to_home[[reachy_mini.media.audio_utils.write_asoundrc_to_home]]

Source

Write the .asoundrc file with Reachy Mini audio configuration to the user's home directory.

This function creates an ALSA configuration file (.asoundrc) in the user's home directory that configures the ReSpeaker sound card for proper audio routing and multi-client support. The configuration enables simultaneous audio input and output access, which is essential for the Reachy Mini Wireless version's audio functionality.

The generated configuration includes:

• Default audio device settings pointing to the ReSpeaker sound card
• dmix plugin for multi-client audio output (reachymini_audio_sink)
• dsnoop plugin for multi-client audio input (reachymini_audio_src)
• Proper buffer and sample rate settings for optimal performance

Note: This function automatically detects the ReSpeaker card number and creates a configuration tailored to the detected hardware. It is primarily used for the Reachy Mini Wireless version.

The configuration file will be created at ~/.asoundrc and will overwrite any existing file with the same name. Existing audio configurations should be backed up before calling this function.

Audio Control Utils Functions[[reachy_mini.media.audio_control_utils.ReSpeaker]]

reachy_mini.media.audio_control_utils.ReSpeaker[[reachy_mini.media.audio_control_utils.ReSpeaker]]

Source

Class to interface with the ReSpeaker XVF3800 USB device.

apply_audio_configreachy_mini.media.audio_control_utils.ReSpeaker.apply_audio_confighttps://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/audio_control_utils.py#L331[{"name": "config", "val": ": collections.abc.Sequence[tuple[str, collections.abc.Sequence[float | int]]]"}, {"name": "verify", "val": ": bool = True"}, {"name": "write_settle_seconds", "val": ": float = 0.1"}]- config -- Parameter names and values to write.

• verify -- When true, read each parameter back after writing it.
• write_settle_seconds -- Delay after each write before readback.0True when all parameters were written and verified successfully. Apply a set of audio control parameters to the ReSpeaker.

Parameters:

config : Parameter names and values to write.

verify : When true, read each parameter back after writing it.

write_settle_seconds : Delay after each write before readback.

Returns:

True when all parameters were written and verified successfully.

close[[reachy_mini.media.audio_control_utils.ReSpeaker.close]]

Source

Close the interface.

read[[reachy_mini.media.audio_control_utils.ReSpeaker.read]]

Source

Read data from a specified parameter on the ReSpeaker device.

read_values[[reachy_mini.media.audio_control_utils.ReSpeaker.read_values]]

Source

Read a parameter and decode it into numeric values.

write[[reachy_mini.media.audio_control_utils.ReSpeaker.write]]

Source

Write data to a specified parameter on the ReSpeaker device.

reachy_mini.media.audio_control_utils.find[[reachy_mini.media.audio_control_utils.find]]

Source

Find and return the ReSpeaker USB device with the given Vendor ID and Product ID.

Note: This function searches for USB devices with the specified Vendor ID and Product ID using libusb backend. The default values target XMOS XVF3800 devices used in ReSpeaker microphone arrays.

Example:

from reachy_mini.media.audio_control_utils import find

# Find default ReSpeaker device
respeaker = find()
if respeaker is not None:
    print("Found ReSpeaker device")
    respeaker.close()

# Find specific device
custom_device = find(vid=0x1234, pid=0x5678)

Parameters:

vid (int) : USB Vendor ID to search for. Default: 0x2886 (XMOS).

pid (int) : USB Product ID to search for. Default: 0x001A (XMOS XVF3800).

Returns:

ReSpeaker | None

A ReSpeaker object if the device is found, None otherwise.

reachy_mini.media.audio_control_utils.init_respeaker_usb[[reachy_mini.media.audio_control_utils.init_respeaker_usb]]

Source

Initialize the ReSpeaker USB device. Looks for both new and beta device IDs.

Note: This function attempts to initialize a ReSpeaker microphone array by searching for USB devices with known Vendor and Product IDs. It tries:

1. New Reachy Mini Audio firmware (0x38FB:0x1001) - preferred
2. Old ReSpeaker firmware (0x2886:0x001A) - with warning to update

The function handles USB backend errors gracefully and returns None if no compatible device is found or if initialization fails.

Example:

from reachy_mini.media.audio_control_utils import init_respeaker_usb

# Initialize ReSpeaker device
respeaker = init_respeaker_usb()
if respeaker is not None:
    print("ReSpeaker initialized successfully")
    # Use the device...
    doa = respeaker.read("DOA_VALUE_RADIANS")
    respeaker.close()
else:
    print("No ReSpeaker device found")

Returns:

Optional[ReSpeaker]

A ReSpeaker object if a compatible device is found, None otherwise.

Camera[[reachy_mini.media.camera_gstreamer.GStreamerCamera]]

reachy_mini.media.camera_gstreamer.GStreamerCamera[[reachy_mini.media.camera_gstreamer.GStreamerCamera]]

Source

Camera that reads BGR frames from the daemon's local IPC endpoint.

The WebRTC daemon exposes BGR camera frames via a local IPC mechanism:

• Linux / macOS: unixfdsink / unixfdsrc (Unix domain socket)
• Windows: win32ipcvideosink / win32ipcvideosrc (shared memory)

Since the daemon's IPC branch already converts to BGR, the reader pipeline is simply source → queue → appsink with no extra conversion.

closereachy_mini.media.camera_gstreamer.GStreamerCamera.closehttps://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/camera_gstreamer.py#L274[] Stop the pipeline and release resources.

Parameters:

camera_specs : Camera specifications (resolutions, intrinsics, …).

open[[reachy_mini.media.camera_gstreamer.GStreamerCamera.open]]

Source

Start the GStreamer pipeline and begin receiving frames.

read[[reachy_mini.media.camera_gstreamer.GStreamerCamera.read]]

Source

Pull the latest BGR frame from the IPC endpoint.

Returns:

A NumPy array of shape (height, width, 3) in BGR order, or None if no frame is available within the timeout.

Camera Utils Functions[[reachy_mini.media.camera_utils.undistort_points]]

reachy_mini.media.camera_utils.undistort_points[[reachy_mini.media.camera_utils.undistort_points]]

Source

Undistort a single pixel coordinate to normalized camera coordinates.

Pure numpy equivalent of cv2.undistortPoints(). Supports the OpenCV distortion model with up to 12 coefficients (rational model + thin prism): D = (k1, k2, p1, p2, k3, k4, k5, k6, s1, s2, s3, s4)

Also works with 5-coefficient models (k1, k2, p1, p2, k3) and zero-distortion.

The algorithm matches OpenCV's cvUndistortPointsInternal:

1. Remove camera intrinsics to get normalized distorted coordinates.
2. Iteratively solve for undistorted coordinates using a damped fixed-point method with adaptive step size.

Reference: OpenCV distortion model and undistortPoints algorithm: https://docs.opencv.org/4.x/d9/d0c/group__calib3d.html https://github.com/opencv/opencv/blob/4.x/modules/calib3d/src/undistort.dispatch.cpp

Parameters:

u : Horizontal pixel coordinate.

v : Vertical pixel coordinate.

K : 3x3 camera intrinsic matrix [[fx, 0, cx], [0, fy, cy], [0, 0, 1]].

D : Distortion coefficients array. Supports lengths 0, 4, 5, 8, 12, or 14. Unused positions default to 0.

max_iterations : Maximum number of iterations (default 20).

epsilon : Convergence threshold in pixel reprojection error (default 0.01).

Returns:

Tuple (x_n, y_n)

Normalized undistorted coordinates (on the z=1 plane).

reachy_mini.media.camera_utils.scale_intrinsics[[reachy_mini.media.camera_utils.scale_intrinsics]]

Source

Scale camera intrinsics for a different resolution with cropping.

Parameters:

K_original : Original 3x3 camera matrix

original_size : (width, height) of original calibration

target_size : (width, height) of target resolution

crop_scale : Scale factor due to digital zoom/crop (>1 means more zoomed in)

Returns:

K_scaled

Adjusted camera matrix for target resolution

Camera Constants[[reachy_mini.media.camera_constants.CameraResolution]]

reachy_mini.media.camera_constants.CameraResolution[[reachy_mini.media.camera_constants.CameraResolution]]

Source

Base class for camera resolutions.

Enumeration of standardized camera resolutions and frame rates supported by Reachy Mini cameras. Each enum value contains a tuple of (width, height, fps).

Note: The enum values are tuples containing (width, height, frames_per_second, crop_factor). Not all resolutions are supported by all camera models - check the specific camera specifications for available resolutions.

Example:

from reachy_mini.media.camera_constants import CameraResolution

# Get resolution information
res = CameraResolution.R1280x720at30fps
width, height, fps, crop_factor = res.value
print(f"Resolution: {width}x{height}@{fps}fps")

# Check if a resolution is supported by a camera
from reachy_mini.media.camera_constants import ReachyMiniLiteCamSpecs
res = CameraResolution.R1920x1080at60fps
if res in ReachyMiniLiteCamSpecs.available_resolutions:
    print("This resolution is supported")

Parameters:

R1536x864at40fps : 1536x864 resolution at 40 fps

R1280x720at60fps : 1280x720 resolution at 60 fps (HD)

R1280x720at30fps : 1280x720 resolution at 30 fps (HD)

R1920x1080at30fps : 1920x1080 resolution at 30 fps (Full HD)

R1920x1080at60fps : 1920x1080 resolution at 60 fps (Full HD)

R2304x1296at30fps : 2304x1296 resolution at 30 fps

R1600x1200at30fps : 1600x1200 resolution at 30 fps

R3264x2448at30fps : 3264x2448 resolution at 30 fps

R3264x2448at10fps : 3264x2448 resolution at 10 fps

R3840x2592at30fps : 3840x2592 resolution at 30 fps

R3840x2592at10fps : 3840x2592 resolution at 10 fps

R3840x2160at30fps : 3840x2160 resolution at 30 fps (4K UHD)

R3840x2160at10fps : 3840x2160 resolution at 10 fps (4K UHD)

R3072x1728at10fps : 3072x1728 resolution at 10 fps

R4608x2592at10fps : 4608x2592 resolution at 10 fps

reachy_mini.media.camera_constants.CameraSpecs[[reachy_mini.media.camera_constants.CameraSpecs]]

Source

Base camera specifications.

Dataclass containing specifications for a camera model, including supported resolutions, calibration parameters, and USB identification information.

Note: The intrinsic matrix K has the format: [[fx, 0, cx], [ 0, fy, cy], [ 0, 0, 1]]

Where fx, fy are focal lengths in pixels, and cx, cy are the principal point coordinates (typically near the image center).

Example:

from reachy_mini.media.camera_constants import CameraSpecs

# Create a custom camera specification
custom_specs = CameraSpecs(
    name="custom_camera",
    available_resolutions=[CameraResolution.R1280x720at30fps],
    default_resolution=CameraResolution.R1280x720at30fps,
    vid=0x1234,
    pid=0x5678,
    K=np.array([[800, 0, 640], [0, 800, 360], [0, 0, 1]]),
    D=np.zeros(5)
)

Parameters:

name (str) : Human-readable name of the camera model.

available_resolutions (List[CameraResolution]) : List of supported resolutions and frame rates for this camera model.

default_resolution (CameraResolution) : Default resolution used when the camera is initialized.

vid (int) : USB Vendor ID for identifying this camera model.

pid (int) : USB Product ID for identifying this camera model.

K (npt.NDArray[np.float64]) : 3x3 camera intrinsic matrix containing focal lengths and principal point coordinates.

D (npt.NDArray[np.float64]) : 5-element array containing distortion coefficients (k1, k2, p1, p2, k3) for radial and tangential distortion.

reachy_mini.media.camera_constants.ArducamSpecs[[reachy_mini.media.camera_constants.ArducamSpecs]]

Source

Arducam camera specifications.

reachy_mini.media.camera_constants.ReachyMiniLiteCamSpecs[[reachy_mini.media.camera_constants.ReachyMiniLiteCamSpecs]]

Source

Reachy Mini Lite camera specifications.

reachy_mini.media.camera_constants.ReachyMiniWirelessCamSpecs[[reachy_mini.media.camera_constants.ReachyMiniWirelessCamSpecs]]

Source

Reachy Mini Wireless camera specifications.

reachy_mini.media.camera_constants.OlderRPiCamSpecs[[reachy_mini.media.camera_constants.OlderRPiCamSpecs]]

Source

Older Raspberry Pi camera specifications. Keeping for compatibility.

reachy_mini.media.camera_constants.MujocoCameraSpecs[[reachy_mini.media.camera_constants.MujocoCameraSpecs]]

Source

Mujoco simulated camera specifications.

reachy_mini.media.camera_constants.GenericWebcamSpecs[[reachy_mini.media.camera_constants.GenericWebcamSpecs]]

Source

Generic webcam specifications (fallback for any webcam).

WebRTC[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient]]

reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient]]

Source

WebRTC client that provides both camera frames and audio.

Implements the same public API surface as GStreamerCamera (for video) and GStreamerAudio (for audio) so that MediaManager can assign the same instance to both its camera and audio slots.

cleanupreachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.cleanuphttps://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/webrtc_client_gstreamer.py#L616[] Release all resources.

clear_output_buffer[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.clear_output_buffer]]

Source

No-op (WebRTC send chain does not buffer significantly).

close[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.close]]

Source

Stop the WebRTC pipeline.

delete_sound[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.delete_sound]]

Source

Delete a sound file from the daemon's temporary sound directory.

Parameters:

filename : Name of the file to delete (not a full path).

Returns:

True if the file was deleted, False otherwise.

get_DoA[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.get_DoA]]

Source

Get the Direction of Arrival from the ReSpeaker.

Returns:

A tuple (angle_radians, speech_detected) or None.

list_sounds[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.list_sounds]]

Source

List sound files in the daemon's temporary sound directory.

Returns:

A list of filenames, or an empty list on error.

open[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.open]]

Source

Start the WebRTC pipeline (both video and audio).

play_sound[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.play_sound]]

Source

Play a sound file on the robot's speaker via the daemon REST API.

If sound_file is a local path that exists on this machine the file is automatically uploaded to the daemon's temporary sound directory (skipping the upload when a file with the same name is already present). Otherwise the filename is sent as-is and the daemon resolves it from its built-in assets or filesystem.

Parameters:

sound_file : Absolute local path or asset filename (e.g. "wake_up.wav").

push_audio_sample[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.push_audio_sample]]

Source

Push audio data to the remote peer via WebRTC.

Parameters:

data : Float32 audio samples.

read[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.read]]

Source

Pull the latest BGR video frame.

Returns:

A NumPy array of shape (height, width, 3) or None.

start_playing[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.start_playing]]

Source

No-op — audio send chain is set up automatically on WebRTC connection.

start_recording[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.start_recording]]

Source

No-op — recording starts automatically with open().

stop_playing[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.stop_playing]]

Source

Reset the PTS counter for the send chain and stop daemon-side sound.

stop_recording[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.stop_recording]]

Source

No-op — managed by close().

upload_sound[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.upload_sound]]

Source

Upload a local sound file to the daemon's temporary directory.

Parameters:

sound_file : Local path to the sound file.

Returns:

The absolute path of the file on the daemon.

reachy_mini.media.media_server.GstMediaServer[[reachy_mini.media.media_server.GstMediaServer]]

Source

Daemon-side GStreamer media server.

Owns the camera and audio hardware and distributes media to consumers:

• IPC branch — raw BGR frames via unixfdsink / win32ipcvideosink for on-device applications (GStreamerCamera reads from this).
• WebRTC branch — encoded video + audio via webrtcsink for remote clients (GstWebRTCClient connects to this).
• Sound playback — playbin for playing WAV files on the speaker.

closereachy_mini.media.media_server.GstMediaServer.closehttps://github.com/pollen-robotics/reachy_mini/blob/vr_1069/src/reachy_mini/media/media_server.py#L147[] Release GStreamer resources (MainLoop, bus watch).

Parameters:

camera_specs (CameraSpecs) : Specifications of the detected camera.

resized_K (npt.NDArray[np.float64]) : Camera intrinsic matrix for current resolution.

play_sound[[reachy_mini.media.media_server.GstMediaServer.play_sound]]

Source

Play a sound file on the robot's speaker.

Uses GStreamer's playbin element with a platform-aware audio sink. This is used for daemon-side sounds (wake-up, sleep, etc.).

Parameters:

sound_file : Path to the sound file to play. If the file is not found at the given path, it is looked up in the assets directory.

send_data_message[[reachy_mini.media.media_server.GstMediaServer.send_data_message]]

Source

Send a message to connected peers via data channel.

Parameters:

message : The string message to send

peer_id : If specified, send only to this peer. Otherwise broadcast to all.

set_message_handler[[reachy_mini.media.media_server.GstMediaServer.set_message_handler]]

Source

Set a callback for incoming data channel messages.

Parameters:

handler : Callback function that receives (peer_id, message)

start[[reachy_mini.media.media_server.GstMediaServer.start]]

Source

Rebuild the pipeline from scratch and start it.

Rebuilding ensures a clean state after stop() released all hardware.

stop[[reachy_mini.media.media_server.GstMediaServer.stop]]

Source

Stop the pipeline and release all hardware (camera, audio).

stop_sound[[reachy_mini.media.media_server.GstMediaServer.stop_sound]]

Source

Stop the currently playing sound file.

If no sound is currently playing this is a no-op.