reachy_mini/pr_913/en/API/media.md

Media

Media Manager[[reachy_mini.media.media_manager.MediaManager]]

reachy_mini.media.media_manager.MediaManager[[reachy_mini.media.media_manager.MediaManager]]

Source

Media Manager for handling camera and audio devices.

This class provides a unified interface for managing both camera and audio devices across different backends. It handles initialization, configuration, and cleanup of media resources.

closereachy_mini.media.media_manager.MediaManager.closehttps://github.com/pollen-robotics/reachy_mini/blob/vr_913/src/reachy_mini/media/media_manager.py#L153[] Close the media manager and release resources.

This method should be called when the media manager is no longer needed to properly clean up and release all media resources. It stops any ongoing audio recording/playback and closes the camera device.

Note: After calling this method, the media manager can be reused by calling the appropriate initialization methods again, but it's generally recommended to create a new MediaManager instance if needed.

Example:

media = MediaManager()
try:
    # Use media devices
    frame = media.get_frame()
finally:
    media.close()

Parameters:

logger (logging.Logger) : Logger instance for media-related messages.

backend (MediaBackend) : The selected media backend.

camera (Optional[CameraBase]) : Camera device instance.

audio (Optional[AudioBase]) : Audio device instance.

get_DoA[[reachy_mini.media.media_manager.MediaManager.get_DoA]]

Source

Get the Direction of Arrival (DoA) from the microphone array.

Returns:

tuple[float, bool] | None

A tuple (angle_radians, speech_detected), or None if the audio system is not available.

get_audio_sample[[reachy_mini.media.media_manager.MediaManager.get_audio_sample]]

Source

Get an audio sample from the audio device.

Returns:

Optional[np.ndarray]

The recorded audio sample, or None if no data is available.

get_frame[[reachy_mini.media.media_manager.MediaManager.get_frame]]

Source

Get a frame from the camera.

Note: This method returns None if the camera is not initialized or if there's an error capturing the frame. Always check the return value before using the frame.

Example:

frame = media.get_frame()
if frame is not None:
    # Process the frame
    cv2.imshow("Camera", frame)
    cv2.waitKey(1)

    # Convert to RGB if needed
    rgb_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)

Returns:

Optional[npt.NDArray[np.uint8]]

The captured BGR frame as a numpy array with shape (height, width, 3), or None if the camera is not available or an error occurred.

The image is in BGR format (OpenCV convention) and can be directly used with OpenCV functions or converted to RGB if needed.

get_input_audio_samplerate[[reachy_mini.media.media_manager.MediaManager.get_input_audio_samplerate]]

Source

Get the input samplerate of the audio device.

get_input_channels[[reachy_mini.media.media_manager.MediaManager.get_input_channels]]

Source

Get the number of input channels of the audio device.

get_output_audio_samplerate[[reachy_mini.media.media_manager.MediaManager.get_output_audio_samplerate]]

Source

Get the output samplerate of the audio device.

get_output_channels[[reachy_mini.media.media_manager.MediaManager.get_output_channels]]

Source

Get the number of output channels of the audio device.

play_sound[[reachy_mini.media.media_manager.MediaManager.play_sound]]

Source

Play a sound file.

Parameters:

sound_file (str) : Path to the sound file to play.

push_audio_sample[[reachy_mini.media.media_manager.MediaManager.push_audio_sample]]

Source

Push audio data to the output device.

Parameters:

data (npt.NDArray[np.float32]) : The audio data to push to the output device (mono format).

start_playing[[reachy_mini.media.media_manager.MediaManager.start_playing]]

Source

Start playing audio.

start_recording[[reachy_mini.media.media_manager.MediaManager.start_recording]]

Source

Start recording audio.

stop_playing[[reachy_mini.media.media_manager.MediaManager.stop_playing]]

Source

Stop playing audio.

stop_recording[[reachy_mini.media.media_manager.MediaManager.stop_recording]]

Source

Stop recording audio.

Audio[[reachy_mini.media.audio_base.AudioBase]]

reachy_mini.media.audio_base.AudioBase[[reachy_mini.media.audio_base.AudioBase]]

Source

Abstract class for opening and managing audio devices.

This class defines the interface that all audio implementations must follow. It provides common audio parameters and methods for managing audio devices, including microphone input and speaker output functionality.

cleanupreachy_mini.media.audio_base.AudioBase.cleanuphttps://github.com/pollen-robotics/reachy_mini/blob/vr_913/src/reachy_mini/media/audio_base.py#L64[] Cleanup resources before destruction.

This method should be called to release any resources held by the audio implementation before the object is destroyed.

Parameters:

SAMPLE_RATE (int) : Default sample rate for audio operations (16000 Hz).

CHANNELS (int) : Default number of audio channels (2 for stereo).

logger (logging.Logger) : Logger instance for audio-related messages.

_respeaker (Optional[ReSpeaker]) : ReSpeaker microphone array device handler.

clear_output_buffer[[reachy_mini.media.audio_base.AudioBase.clear_output_buffer]]

Source

Clear the output buffer.

This method flushes the output buffer to prevent push samples from being played. Overwrite if necessary. It seems that set_max_output_buffers with a low value may be enough for gstreamer backend.

get_DoA[[reachy_mini.media.audio_base.AudioBase.get_DoA]]

Source

Get the Direction of Arrival (DoA) value from the ReSpeaker device.

The spatial angle is given in radians: 0 radians is left, π/2 radians is front/back, π radians is right.

Note: The microphone array requires firmware version 2.1.0 or higher to support this feature. The firmware is located in src/reachy_mini/assets/firmware/*.bin. Refer to https://wiki.seeedstudio.com/respeaker_xvf3800_introduction/#update-firmware for the upgrade process.

Returns:

tuple

A tuple containing the DoA value as a float (radians) and the speech detection as a bool, or None if the device is not found.

get_audio_sample[[reachy_mini.media.audio_base.AudioBase.get_audio_sample]]

Source

Read audio data from the device. Returns the data or None if error.

Note: This method should be called after start_recording() has been called. The sample rate and number of channels can be obtained via get_input_audio_samplerate() and get_input_channels() respectively.

Example:

audio.start_recording()
samples = audio.get_audio_sample()
if samples is not None:
    print(f"Got {len(samples)} audio samples")

Returns:

Optional[npt.NDArray[np.float32]]

A numpy array containing audio samples in float32 format, or None if no data is available or an error occurred.

The array shape is typically (num_samples,) for mono or (num_samples, num_channels) for multi-channel audio.

get_input_audio_samplerate[[reachy_mini.media.audio_base.AudioBase.get_input_audio_samplerate]]

Source

Get the input samplerate of the audio device.

Note: This value represents the number of audio samples captured per second for each channel.

Returns:

int

The sample rate in Hz at which audio is being captured. Default is 16000 Hz.

get_input_channels[[reachy_mini.media.audio_base.AudioBase.get_input_channels]]

Source

Get the number of input channels of the audio device.

Note: For the ReSpeaker microphone array, this typically returns 2 channels representing the stereo microphone configuration.

Returns:

int

The number of audio input channels (e.g., 1 for mono, 2 for stereo). Default is 2 channels.

get_output_audio_samplerate[[reachy_mini.media.audio_base.AudioBase.get_output_audio_samplerate]]

Source

Get the output samplerate of the audio device.

Note: This value represents the number of audio samples played per second for each channel.

Returns:

int

The sample rate in Hz at which audio is being played back. Default is 16000 Hz.

get_output_channels[[reachy_mini.media.audio_base.AudioBase.get_output_channels]]

Source

Get the number of output channels of the audio device.

Note: This determines how audio data should be formatted when passed to push_audio_sample() method.

Returns:

int

The number of audio output channels (e.g., 1 for mono, 2 for stereo). Default is 2 channels.

play_sound[[reachy_mini.media.audio_base.AudioBase.play_sound]]

Source

Play a sound file.

Note: This is a convenience method that handles the complete playback of a sound file from start to finish. For more control over audio playback, use start_playing(), push_audio_sample(), and stop_playing() methods.

Example:

audio.play_sound("/path/to/sound.wav")

Parameters:

sound_file (str) : Path to the sound file to play. Supported formats depend on the specific implementation.

push_audio_sample[[reachy_mini.media.audio_base.AudioBase.push_audio_sample]]

Source

Push audio data to the output device.

Note: This method should be called after start_playing() has been called. The audio data will be played at the sample rate returned by get_output_audio_samplerate().

Parameters:

data (npt.NDArray[np.float32]) : Audio samples to be played. The array should contain float32 values typically in the range [-1.0, 1.0]. For mono audio: shape should be (num_samples,) For stereo audio: shape should be (num_samples, 2)

set_max_output_buffers[[reachy_mini.media.audio_base.AudioBase.set_max_output_buffers]]

Source

Set the maximum number of output buffers to queue in the player.

Parameters:

max_buffers (int) : Maximum number of buffers to queue.

start_playing[[reachy_mini.media.audio_base.AudioBase.start_playing]]

Source

Start playing audio.

This method should initialize the audio playback system and prepare it to receive audio data via push_audio_sample().

Note: Implementations should handle any necessary resource allocation and error checking. If playback cannot be started, implementations should log appropriate error messages.

start_recording[[reachy_mini.media.audio_base.AudioBase.start_recording]]

Source

Start recording audio.

This method should initialize the audio recording system and prepare it to capture audio data. After calling this method, get_audio_sample() should be able to retrieve recorded audio data.

Note: Implementations should handle any necessary resource allocation and error checking. If recording cannot be started, implementations should log appropriate error messages.

stop_playing[[reachy_mini.media.audio_base.AudioBase.stop_playing]]

Source

Stop playing audio and release resources.

This method should stop any ongoing audio playback and release all associated resources. After calling this method, push_audio_sample() calls will have no effect until start_playing() is called again.

Note: Implementations should ensure proper cleanup to prevent resource leaks.

stop_recording[[reachy_mini.media.audio_base.AudioBase.stop_recording]]

Source

Close the audio device and release resources.

This method should stop any ongoing audio recording and release all associated resources. After calling this method, get_audio_sample() should return None until start_recording() is called again.

Note: Implementations should ensure proper cleanup to prevent resource leaks.

reachy_mini.media.audio_gstreamer.GStreamerAudio[[reachy_mini.media.audio_gstreamer.GStreamerAudio]]

Source

Audio implementation using GStreamer.

clear_playerreachy_mini.media.audio_gstreamer.GStreamerAudio.clear_playerhttps://github.com/pollen-robotics/reachy_mini/blob/vr_913/src/reachy_mini/media/audio_gstreamer.py#L398[] Flush the player's appsrc to drop any queued audio immediately.

get_audio_sample[[reachy_mini.media.audio_gstreamer.GStreamerAudio.get_audio_sample]]

Source

Read a sample from the audio card. Returns the sample or None if error.

See AudioBase.get_audio_sample() for complete documentation.

Returns:

Optional[npt.NDArray[np.float32]]

The captured sample in raw format, or None if error.

get_input_audio_samplerate[[reachy_mini.media.audio_gstreamer.GStreamerAudio.get_input_audio_samplerate]]

Source

Get the input samplerate of the audio device.

See AudioBase.get_input_audio_samplerate() for complete documentation.

get_input_channels[[reachy_mini.media.audio_gstreamer.GStreamerAudio.get_input_channels]]

Source

Get the number of input channels of the audio device.

See AudioBase.get_input_channels() for complete documentation.

get_output_audio_samplerate[[reachy_mini.media.audio_gstreamer.GStreamerAudio.get_output_audio_samplerate]]

Source

Get the output samplerate of the audio device.

See AudioBase.get_output_audio_samplerate() for complete documentation.

get_output_channels[[reachy_mini.media.audio_gstreamer.GStreamerAudio.get_output_channels]]

Source

Get the number of output channels of the audio device.

See AudioBase.get_output_channels() for complete documentation.

play_sound[[reachy_mini.media.audio_gstreamer.GStreamerAudio.play_sound]]

Source

Play a sound file.

See AudioBase.play_sound() for complete documentation.

Todo: for now this function is mean to be used on the wireless version.

Parameters:

sound_file (str) : Path to the sound file to play.

push_audio_sample[[reachy_mini.media.audio_gstreamer.GStreamerAudio.push_audio_sample]]

Source

Push audio data to the output device.

See AudioBase.push_audio_sample() for complete documentation.

start_playing[[reachy_mini.media.audio_gstreamer.GStreamerAudio.start_playing]]

Source

Open the audio output using GStreamer.

See AudioBase.start_playing() for complete documentation.

start_recording[[reachy_mini.media.audio_gstreamer.GStreamerAudio.start_recording]]

Source

Open the audio card using GStreamer.

See AudioBase.start_recording() for complete documentation.

stop_playing[[reachy_mini.media.audio_gstreamer.GStreamerAudio.stop_playing]]

Source

Stop playing audio and release resources.

See AudioBase.stop_playing() for complete documentation.

stop_recording[[reachy_mini.media.audio_gstreamer.GStreamerAudio.stop_recording]]

Source

Release the camera resource.

See AudioBase.stop_recording() for complete documentation.

Audio Utils Functions[[reachy_mini.media.audio_utils.get_respeaker_card_number]]

reachy_mini.media.audio_utils.get_respeaker_card_number[[reachy_mini.media.audio_utils.get_respeaker_card_number]]

Source

Return the card number of the ReSpeaker sound card, or 0 if not found.

Note: This function runs 'arecord -l' to list available audio capture devices and processes the output to find Reachy Mini Audio or ReSpeaker devices. It's primarily used on Linux systems with ALSA audio configuration.

The function returns:

• Positive integer: Card number of detected Reachy Mini Audio device
• 0: No Reachy Mini Audio device found, using default sound card
• -1: Error occurred while trying to detect audio devices

Example:

card_num = get_respeaker_card_number()
if card_num > 0:
    print(f"Using Reachy Mini Audio card {card_num}")
elif card_num == 0:
    print("Using default sound card")
else:
    print("Error detecting audio devices")

Returns:

int

The card number of the detected ReSpeaker/Reachy Mini Audio device. Returns 0 if no specific device is found (uses default sound card), or -1 if there's an error running the detection command.

reachy_mini.media.audio_utils.has_reachymini_asoundrc[[reachy_mini.media.audio_utils.has_reachymini_asoundrc]]

Source

Check if ~/.asoundrc exists and contains both reachymini_audio_sink and reachymini_audio_src.

Note: This function checks for the presence of the ALSA configuration file ~/.asoundrc and verifies that it contains the necessary configuration entries for Reachy Mini audio devices (reachymini_audio_sink and reachymini_audio_src). These entries are required for proper audio routing and device management.

Example:

if has_reachymini_asoundrc():
    print("Reachy Mini audio configuration is properly set up")
else:
    print("Need to configure Reachy Mini audio devices")
    write_asoundrc_to_home()  # Create the configuration

Returns:

bool

True if ~/.asoundrc exists and contains the required Reachy Mini audio configuration entries, False otherwise.

reachy_mini.media.audio_utils.check_reachymini_asoundrc[[reachy_mini.media.audio_utils.check_reachymini_asoundrc]]

Source

Check if ~/.asoundrc exists and is correctly configured for Reachy Mini Audio.

reachy_mini.media.audio_utils.write_asoundrc_to_home[[reachy_mini.media.audio_utils.write_asoundrc_to_home]]

Source

Write the .asoundrc file with Reachy Mini audio configuration to the user's home directory.

This function creates an ALSA configuration file (.asoundrc) in the user's home directory that configures the ReSpeaker sound card for proper audio routing and multi-client support. The configuration enables simultaneous audio input and output access, which is essential for the Reachy Mini Wireless version's audio functionality.

The generated configuration includes:

• Default audio device settings pointing to the ReSpeaker sound card
• dmix plugin for multi-client audio output (reachymini_audio_sink)
• dsnoop plugin for multi-client audio input (reachymini_audio_src)
• Proper buffer and sample rate settings for optimal performance

Note: This function automatically detects the ReSpeaker card number and creates a configuration tailored to the detected hardware. It is primarily used for the Reachy Mini Wireless version.

The configuration file will be created at ~/.asoundrc and will overwrite any existing file with the same name. Existing audio configurations should be backed up before calling this function.

Audio Control Utils Functions[[reachy_mini.media.audio_control_utils.ReSpeaker]]

reachy_mini.media.audio_control_utils.ReSpeaker[[reachy_mini.media.audio_control_utils.ReSpeaker]]

Source

Class to interface with the ReSpeaker XVF3800 USB device.

closereachy_mini.media.audio_control_utils.ReSpeaker.closehttps://github.com/pollen-robotics/reachy_mini/blob/vr_913/src/reachy_mini/media/audio_control_utils.py#L317[] Close the interface.

read[[reachy_mini.media.audio_control_utils.ReSpeaker.read]]

Source

Read data from a specified parameter on the ReSpeaker device.

write[[reachy_mini.media.audio_control_utils.ReSpeaker.write]]

Source

Write data to a specified parameter on the ReSpeaker device.

reachy_mini.media.audio_control_utils.find[[reachy_mini.media.audio_control_utils.find]]

Source

Find and return the ReSpeaker USB device with the given Vendor ID and Product ID.

Note: This function searches for USB devices with the specified Vendor ID and Product ID using libusb backend. The default values target XMOS XVF3800 devices used in ReSpeaker microphone arrays.

Example:

from reachy_mini.media.audio_control_utils import find

# Find default ReSpeaker device
respeaker = find()
if respeaker is not None:
    print("Found ReSpeaker device")
    respeaker.close()

# Find specific device
custom_device = find(vid=0x1234, pid=0x5678)

Parameters:

vid (int) : USB Vendor ID to search for. Default: 0x2886 (XMOS).

pid (int) : USB Product ID to search for. Default: 0x001A (XMOS XVF3800).

Returns:

ReSpeaker | None

A ReSpeaker object if the device is found, None otherwise.

reachy_mini.media.audio_control_utils.init_respeaker_usb[[reachy_mini.media.audio_control_utils.init_respeaker_usb]]

Source

Initialize the ReSpeaker USB device. Looks for both new and beta device IDs.

Note: This function attempts to initialize a ReSpeaker microphone array by searching for USB devices with known Vendor and Product IDs. It tries:

1. New Reachy Mini Audio firmware (0x38FB:0x1001) - preferred
2. Old ReSpeaker firmware (0x2886:0x001A) - with warning to update

The function handles USB backend errors gracefully and returns None if no compatible device is found or if initialization fails.

Example:

from reachy_mini.media.audio_control_utils import init_respeaker_usb

# Initialize ReSpeaker device
respeaker = init_respeaker_usb()
if respeaker is not None:
    print("ReSpeaker initialized successfully")
    # Use the device...
    doa = respeaker.read("DOA_VALUE_RADIANS")
    respeaker.close()
else:
    print("No ReSpeaker device found")

Returns:

Optional[ReSpeaker]

A ReSpeaker object if a compatible device is found, None otherwise.

Camera[[reachy_mini.media.camera_base.CameraBase]]

reachy_mini.media.camera_base.CameraBase[[reachy_mini.media.camera_base.CameraBase]]

Source

Abstract class for opening and managing a camera.

This class defines the interface that all camera implementations must follow. It provides common camera parameters and methods for managing camera devices, including image capture, resolution management, and camera calibration.

closereachy_mini.media.camera_base.CameraBase.closehttps://github.com/pollen-robotics/reachy_mini/blob/vr_913/src/reachy_mini/media/camera_base.py#L275[] Close the camera and release resources.

This method should stop any ongoing image capture and release all associated resources. After calling this method, read() should return None until open() is called again.

Note: Implementations should ensure proper cleanup to prevent resource leaks.

Parameters:

logger (logging.Logger) : Logger instance for camera-related messages.

_resolution (Optional[CameraResolution]) : Current camera resolution setting.

camera_specs (Optional[CameraSpecs]) : Camera specifications including supported resolutions and calibration parameters.

resized_K (Optional[npt.NDArray[np.float64]]) : Camera intrinsic matrix resized to match the current resolution.

open[[reachy_mini.media.camera_base.CameraBase.open]]

Source

Open the camera.

This method should initialize the camera device and prepare it for capturing images. After calling this method, read() should be able to retrieve camera frames.

Note: Implementations should handle any necessary resource allocation, camera configuration, and error checking. If the camera cannot be opened, implementations should log appropriate error messages.

read[[reachy_mini.media.camera_base.CameraBase.read]]

Source

Read an image from the camera. Returns the image or None if error.

Note: This method should be called after open() has been called. The image resolution can be obtained via the resolution property.

Example:

camera.open()
frame = camera.read()
if frame is not None:
    cv2.imshow("Camera Frame", frame)
    cv2.waitKey(1)

Returns:

Optional[npt.NDArray[np.uint8]]

A numpy array containing the captured image in BGR format (OpenCV convention), or None if no image is available or an error occurred.

The array shape is (height, width, 3) where the last dimension represents the BGR color channels.

set_resolution[[reachy_mini.media.camera_base.CameraBase.set_resolution]]

Source

Set the camera resolution.

Note: This method updates the camera's resolution and automatically rescales the camera intrinsic matrix (K) to match the new resolution. The rescaling preserves the camera's field of view and principal point position relative to the image dimensions.

Example:

from reachy_mini.media.camera_constants import CameraResolution
camera.set_resolution(CameraResolution.R1280x720at30fps)

Parameters:

resolution (CameraResolution) : The desired camera resolution from the CameraResolution enum.

reachy_mini.media.camera_opencv.OpenCVCamera[[reachy_mini.media.camera_opencv.OpenCVCamera]]

Source

Camera implementation using OpenCV.

This class implements the CameraBase interface using OpenCV, providing cross-platform camera support for Reachy Mini robots. It automatically detects and configures supported camera models.

closereachy_mini.media.camera_opencv.OpenCVCamera.closehttps://github.com/pollen-robotics/reachy_mini/blob/vr_913/src/reachy_mini/media/camera_opencv.py#L173[] Release the camera resource.

See CameraBase.close() for complete documentation.

Parameters:

Inherits all attributes from CameraBase. --

Additionally manages OpenCV VideoCapture objects and camera connections. --

open[[reachy_mini.media.camera_opencv.OpenCVCamera.open]]

Source

Open the camera using OpenCV VideoCapture.

See CameraBase.open() for complete documentation.

read[[reachy_mini.media.camera_opencv.OpenCVCamera.read]]

Source

Read a frame from the camera.

See CameraBase.read() for complete documentation.

Returns:

The frame as a uint8 numpy array, or None if no frame could be read.

set_resolution[[reachy_mini.media.camera_opencv.OpenCVCamera.set_resolution]]

Source

Set the camera resolution.

reachy_mini.media.camera_gstreamer.GStreamerCamera[[reachy_mini.media.camera_gstreamer.GStreamerCamera]]

Source

Camera implementation using GStreamer.

closereachy_mini.media.camera_gstreamer.GStreamerCamera.closehttps://github.com/pollen-robotics/reachy_mini/blob/vr_913/src/reachy_mini/media/camera_gstreamer.py#L292[] Release the camera resource.

get_video_device[[reachy_mini.media.camera_gstreamer.GStreamerCamera.get_video_device]]

Source

Use Gst.DeviceMonitor to find the unix camera path /dev/videoX.

Returns the device path (e.g., '/dev/video2'), or '' if not found.

open[[reachy_mini.media.camera_gstreamer.GStreamerCamera.open]]

Source

Open the camera using GStreamer.

read[[reachy_mini.media.camera_gstreamer.GStreamerCamera.read]]

Source

Read a frame from the camera. Returns the frame or None if error.

Returns:

Optional[npt.NDArray[np.uint8]]

The captured BGR frame as a NumPy array, or None if error.

set_resolution[[reachy_mini.media.camera_gstreamer.GStreamerCamera.set_resolution]]

Source

Set the camera resolution.

Camera Utils Functions[[reachy_mini.media.camera_utils.find_camera]]

reachy_mini.media.camera_utils.find_camera[[reachy_mini.media.camera_utils.find_camera]]

Source

Find and return the Reachy Mini camera.

Looks for the Reachy Mini camera first, then Arducam, then older Raspberry Pi Camera. Returns None if no camera is found. Falls back to generic webcam if no specific camera is detected.

Note: This function tries to detect cameras in the following order:

1. Reachy Mini Lite Camera (preferred)
2. Older Raspberry Pi Camera
3. Arducam
4. Generic Webcam (fallback)

The function automatically sets the appropriate video codec (MJPG) for Reachy Mini and Raspberry Pi cameras to ensure compatibility.

Example:

cap, specs = find_camera()
if cap is not None:
    print(f"Found {specs.name} camera")
    # Set resolution
    cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1280)
    cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 720)
    # Capture a frame
    ret, frame = cap.read()
    cap.release()
else:
    print("No camera found")

Parameters:

apiPreference (int) : Preferred API backend for the camera. Default is cv2.CAP_ANY. Options include cv2.CAP_V4L2 (Linux), cv2.CAP_DSHOW (Windows), cv2.CAP_MSMF (Windows), etc.

no_cap (bool) : If True, close the camera after finding it. Useful for testing camera detection without keeping the camera open. Default is False.

Returns:

Tuple[Optional[cv2.VideoCapture], Optional[CameraSpecs]]

A tuple containing:

• cv2.VideoCapture: A VideoCapture object if the camera is found and opened successfully, otherwise None.
• CameraSpecs: The camera specifications for the detected camera, or None if no camera was found.

reachy_mini.media.camera_utils.find_camera_by_vid_pid[[reachy_mini.media.camera_utils.find_camera_by_vid_pid]]

Source

Find and return a camera with the specified VID and PID.

Note: This function uses the cv2_enumerate_cameras package to enumerate available cameras and find one with the specified USB Vendor ID and Product ID. This is useful for selecting specific camera models when multiple cameras are connected to the system.

The Arducam camera creates two /dev/videoX devices that enumerate_cameras cannot differentiate, so this function tries to open each potential device until it finds a working one.

Example:

# Find Reachy Mini Lite Camera by its default VID/PID
cap = find_camera_by_vid_pid()
if cap is not None:
    print("Found Reachy Mini Lite Camera")
    cap.release()

# Find a specific camera by custom VID/PID
cap = find_camera_by_vid_pid(vid=0x0C45, pid=0x636D)  # Arducam
if cap is not None:
    print("Found Arducam")

... cap.release()

Parameters:

vid (int) : Vendor ID of the camera. Default is ReachyMiniLiteCamSpecs.vid (0x38FB).

pid (int) : Product ID of the camera. Default is ReachyMiniLiteCamSpecs.pid (0x1002).

apiPreference (int) : Preferred API backend for the camera. Default is cv2.CAP_ANY. On Linux, this automatically uses cv2.CAP_V4L2 for better compatibility.

Returns:

cv2.VideoCapture | None

A VideoCapture object if the camera with matching VID/PID is found and opened successfully, otherwise None.

Camera Constants[[reachy_mini.media.camera_constants.CameraResolution]]

reachy_mini.media.camera_constants.CameraResolution[[reachy_mini.media.camera_constants.CameraResolution]]

Source

Base class for camera resolutions.

Enumeration of standardized camera resolutions and frame rates supported by Reachy Mini cameras. Each enum value contains a tuple of (width, height, fps).

Note: The enum values are tuples containing (width, height, frames_per_second). Not all resolutions are supported by all camera models - check the specific camera specifications for available resolutions.

Example:

from reachy_mini.media.camera_constants import CameraResolution

# Get resolution information
res = CameraResolution.R1280x720at30fps
width, height, fps = res.value
print(f"Resolution: {width}x{height}@{fps}fps")

# Check if a resolution is supported by a camera
from reachy_mini.media.camera_constants import ReachyMiniLiteCamSpecs
res = CameraResolution.R1920x1080at60fps
if res in ReachyMiniLiteCamSpecs.available_resolutions:
    print("This resolution is supported")

Parameters:

R1536x864at40fps : 1536x864 resolution at 40 fps

R1280x720at60fps : 1280x720 resolution at 60 fps (HD)

R1280x720at30fps : 1280x720 resolution at 30 fps (HD)

R1920x1080at30fps : 1920x1080 resolution at 30 fps (Full HD)

R1920x1080at60fps : 1920x1080 resolution at 60 fps (Full HD)

R2304x1296at30fps : 2304x1296 resolution at 30 fps

R1600x1200at30fps : 1600x1200 resolution at 30 fps

R3264x2448at30fps : 3264x2448 resolution at 30 fps

R3264x2448at10fps : 3264x2448 resolution at 10 fps

R3840x2592at30fps : 3840x2592 resolution at 30 fps

R3840x2592at10fps : 3840x2592 resolution at 10 fps

R3840x2160at30fps : 3840x2160 resolution at 30 fps (4K UHD)

R3840x2160at10fps : 3840x2160 resolution at 10 fps (4K UHD)

R3072x1728at10fps : 3072x1728 resolution at 10 fps

R4608x2592at10fps : 4608x2592 resolution at 10 fps

reachy_mini.media.camera_constants.CameraSpecs[[reachy_mini.media.camera_constants.CameraSpecs]]

Source

Base camera specifications.

Dataclass containing specifications for a camera model, including supported resolutions, calibration parameters, and USB identification information.

Note: The intrinsic matrix K has the format: [[fx, 0, cx], [ 0, fy, cy], [ 0, 0, 1]]

Where fx, fy are focal lengths in pixels, and cx, cy are the principal point coordinates (typically near the image center).

Example:

from reachy_mini.media.camera_constants import CameraSpecs

# Create a custom camera specification
custom_specs = CameraSpecs(
    name="custom_camera",
    available_resolutions=[CameraResolution.R1280x720at30fps],
    default_resolution=CameraResolution.R1280x720at30fps,
    vid=0x1234,
    pid=0x5678,
    K=np.array([[800, 0, 640], [0, 800, 360], [0, 0, 1]]),
    D=np.zeros(5)
)

Parameters:

name (str) : Human-readable name of the camera model.

available_resolutions (List[CameraResolution]) : List of supported resolutions and frame rates for this camera model.

default_resolution (CameraResolution) : Default resolution used when the camera is initialized.

vid (int) : USB Vendor ID for identifying this camera model.

pid (int) : USB Product ID for identifying this camera model.

K (npt.NDArray[np.float64]) : 3x3 camera intrinsic matrix containing focal lengths and principal point coordinates.

D (npt.NDArray[np.float64]) : 5-element array containing distortion coefficients (k1, k2, p1, p2, k3) for radial and tangential distortion.

reachy_mini.media.camera_constants.ArducamSpecs[[reachy_mini.media.camera_constants.ArducamSpecs]]

Source

Arducam camera specifications.

reachy_mini.media.camera_constants.ReachyMiniLiteCamSpecs[[reachy_mini.media.camera_constants.ReachyMiniLiteCamSpecs]]

Source

Reachy Mini Lite camera specifications.

reachy_mini.media.camera_constants.ReachyMiniWirelessCamSpecs[[reachy_mini.media.camera_constants.ReachyMiniWirelessCamSpecs]]

Source

Reachy Mini Wireless camera specifications.

reachy_mini.media.camera_constants.OlderRPiCamSpecs[[reachy_mini.media.camera_constants.OlderRPiCamSpecs]]

Source

Older Raspberry Pi camera specifications. Keeping for compatibility.

reachy_mini.media.camera_constants.MujocoCameraSpecs[[reachy_mini.media.camera_constants.MujocoCameraSpecs]]

Source

Mujoco simulated camera specifications.

reachy_mini.media.camera_constants.GenericWebcamSpecs[[reachy_mini.media.camera_constants.GenericWebcamSpecs]]

Source

Generic webcam specifications (fallback for any webcam).

WebRTC[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient]]

reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient]]

Source

GStreamer WebRTC client implementation.

This class implements a WebRTC client using GStreamer that can connect to a WebRTC server (such as the one hosted on Reachy Mini Wireless) to stream audio and video in real-time. It implements both CameraBase and AudioBase interfaces, allowing seamless integration with the media system.

closereachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.closehttps://github.com/pollen-robotics/reachy_mini/blob/vr_913/src/reachy_mini/media/webrtc_client_gstreamer.py#L365[] Stop the pipeline.

See CameraBase.close() for complete documentation.

Parameters:

Inherits all attributes from CameraBase and AudioBase. --

Additionally manages GStreamer pipelines for WebRTC communication. --

get_audio_sample[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.get_audio_sample]]

Source

Read a sample from the audio card. Returns the sample or None if error.

See AudioBase.get_audio_sample() for complete documentation.

Returns:

Optional[npt.NDArray[np.float32]]

The captured sample in raw format, or None if error.

open[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.open]]

Source

Open the video stream.

See CameraBase.open() for complete documentation.

play_sound[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.play_sound]]

Source

Play a sound file.

See AudioBase.play_sound() for complete documentation.

Parameters:

sound_file (str) : Path to the sound file to play.

push_audio_sample[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.push_audio_sample]]

Source

Push audio data to the output device via the bidirectional WebRTC connection.

read[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.read]]

Source

Read a frame from the camera. Returns the frame or None if error.

See CameraBase.read() for complete documentation.

Returns:

Optional[npt.NDArray[np.uint8]]

The captured frame in BGR format, or None if error.

set_resolution[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.set_resolution]]

Source

Set the camera resolution.

start_playing[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.start_playing]]

Source

Open the audio output using GStreamer.

See AudioBase.start_playing() for complete documentation.

start_recording[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.start_recording]]

Source

Open the audio card using GStreamer.

See AudioBase.start_recording() for complete documentation.

stop_playing[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.stop_playing]]

Source

Stop playing audio and release resources.

See AudioBase.stop_playing() for complete documentation.

stop_recording[[reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient.stop_recording]]

Source

Release the camera resource.

See AudioBase.stop_recording() for complete documentation.

reachy_mini.media.webrtc_daemon.GstWebRTC[[reachy_mini.media.webrtc_daemon.GstWebRTC]]

Source

WebRTC pipeline using GStreamer.

This class implements a WebRTC server using GStreamer that streams video and audio from the Reachy Mini robot to connected WebRTC clients. It's designed to run as a daemon process and handle the complete WebRTC signaling and media streaming pipeline.

pausereachy_mini.media.webrtc_daemon.GstWebRTC.pausehttps://github.com/pollen-robotics/reachy_mini/blob/vr_913/src/reachy_mini/media/webrtc_daemon.py#L510[] Pause the WebRTC pipeline.

Parameters:

_logger (logging.Logger) : Logger instance for WebRTC daemon operations.

_loop (GLib.MainLoop) : GLib main loop for handling GStreamer events.

camera_specs (CameraSpecs) : Specifications of the detected camera.

_resolution (CameraResolution) : Current streaming resolution.

resized_K (npt.NDArray[np.float64]) : Camera intrinsic matrix for current resolution.

send_data_message[[reachy_mini.media.webrtc_daemon.GstWebRTC.send_data_message]]

Source

Send a message to connected peers via data channel.

Parameters:

message : The string message to send

peer_id : If specified, send only to this peer. Otherwise broadcast to all.

set_message_handler[[reachy_mini.media.webrtc_daemon.GstWebRTC.set_message_handler]]

Source

Set a callback for incoming data channel messages.

Parameters:

handler : Callback function that receives (peer_id, message)

start[[reachy_mini.media.webrtc_daemon.GstWebRTC.start]]

Source

Start the WebRTC pipeline.

stop[[reachy_mini.media.webrtc_daemon.GstWebRTC.stop]]

Source

Stop the WebRTC pipeline.