README.md

metadata

title: Linguini
emoji: 🔊
colorFrom: purple
colorTo: gray
sdk: gradio
sdk_version: 4.37.2
app_file: stream_app.py
pinned: false

Audio Stream Project

This project is a Gradio-based application that allows users to interact with an AI-powered audio streaming service. It utilizes OpenAI's language models for various tasks, such as speech recognition, text generation, language understanding, and speech generation.

Table of Contents

• Install
• Usage
• Features
• Configuration
• Deployment
• License

Install

Installation and environment setup is currently done locally, but with a little effort, we could make a docker image.

1. Install miniconda or anaconda here
2. Create a new environment with the following command:

   conda create -n audio-stream python=3.11
   

3. Activate the environment:

   conda activate audio-stream
   

4. Install the required packages:

   pip install -r requirements.txt
   

5. Set the required environment variables or create a .env file (exclude the word export for .env)

   export OPENAI_API_KEY=<your_openai_api_key>
   

6. Run the Gradio application:

   python stream_app.py
   

General Requirements

• Python 3.9 or higher (recommend 3.11)
• Gradio library
• OpenAI API key
• Whisper library (for speech recognition)

Running local Ollama

Ollama is great for running local models that are tuned or high performance versus those that are running online. Generally, there is a three step process of getting ollama running, downloading the right llm model to use, and locally launching the model with litellm. The library litellm provides the glue between ollama and the a programmatic interface for you to access locally.

1. Download ollama
2. Find the model you want to use and install it via the commandn line ollama pull <model>
3. Run it locally with the command ollama serve

• If you run the local application (a lamma appears in your menu/run items), you may not need to explicitly run the serve command.
• After launching, you can confirm that ollama is running on at this endpoint 127.0.0.1:11434

As of 5/25/24, some models to consider are lamma3 for general conversations and dolphin-llama3 for coding tasks. Runner up mentions are Microsoft's wizard2 and llava-llama3

Microphone access error

Your browser might prevent you from access the microphone when running locally (http). To update this in chrome: Update chrome flags (chrome://flags) and allow local (http://127.0.0.1:7860) to be treated as secure (Insecure origins treated as secure)

Usage

1. Set the required environment variables:

• OPENAI_API_KEY: Your OpenAI API key.

2. Run the Gradio application:
3. Access the application in your web browser at the provided URL.

Features

• Speech-to-Text Transcription: Users can upload or record audio files, and the application will transcribe the speech using the Whisper model.
• Text-to-Speech Generation: Users can input text, and the application will generate audio output using OpenAI's language models.
• Language Understanding: The application can analyze the user's input and provide insights, such as sentiment analysis, topic detection, and entity recognition.
• Conversational AI: Users can engage in interactive conversations with the AI assistant, which can respond to queries, provide information, and assist with various tasks.

Configuration

The application can be configured using command-line arguments or environment variables. Run the main command with the option --help to get a full list of available options.

Deployment

The application can be deployed to various platforms, such as:

• Local Machine: Run the application on your local machine using the instructions in the Usage section.
• Docker: Package the application in a Docker container for easy deployment and scaling.
• Cloud Platform: Deploy the application to a cloud platform like AWS, Google Cloud, or Azure. At first blush a deployment strategy like this one seems like a good follow-up for automated deployment.

License

This project is licensed under the MIT License.