Text Generation
PyTorch
GGUF
English
quantum
quantum-entropy
from-scratch
char-level
cosmic-synapse-theory
custom-architecture
llama-cpp
continual-learning
reproducible-seed
open-science
null-results
Instructions to use phera-ra/QC67_cosmo with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use phera-ra/QC67_cosmo with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf phera-ra/QC67_cosmo # Run inference directly in the terminal: llama cli -hf phera-ra/QC67_cosmo
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf phera-ra/QC67_cosmo # Run inference directly in the terminal: llama cli -hf phera-ra/QC67_cosmo
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf phera-ra/QC67_cosmo # Run inference directly in the terminal: ./llama-cli -hf phera-ra/QC67_cosmo
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf phera-ra/QC67_cosmo # Run inference directly in the terminal: ./build/bin/llama-cli -hf phera-ra/QC67_cosmo
Use Docker
docker model run hf.co/phera-ra/QC67_cosmo
- LM Studio
- Jan
- vLLM
How to use phera-ra/QC67_cosmo with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "phera-ra/QC67_cosmo" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "phera-ra/QC67_cosmo", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/phera-ra/QC67_cosmo
- Ollama
How to use phera-ra/QC67_cosmo with Ollama:
ollama run hf.co/phera-ra/QC67_cosmo
- Unsloth Studio
How to use phera-ra/QC67_cosmo with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for phera-ra/QC67_cosmo to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for phera-ra/QC67_cosmo to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for phera-ra/QC67_cosmo to start chatting
- Docker Model Runner
How to use phera-ra/QC67_cosmo with Docker Model Runner:
docker model run hf.co/phera-ra/QC67_cosmo
- Lemonade
How to use phera-ra/QC67_cosmo with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull phera-ra/QC67_cosmo
Run and chat with the model
lemonade run user.QC67_cosmo-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
| """ | |
| COSMOS Cloud Endpoint API | |
| REST API for cloud service integration and configuration. | |
| Allows users to input API credentials and route requests to their cloud providers. | |
| """ | |
| from flask import Flask, request, jsonify, render_template | |
| from cloud_router import CloudConfig, CloudRouter | |
| import os | |
| import json | |
| app = Flask(__name__) | |
| config = CloudConfig() | |
| router = CloudRouter(config) | |
| # ============================================================================ | |
| # CONFIGURATION ENDPOINTS | |
| # ============================================================================ | |
| def get_config(): | |
| """Get current cloud configuration (sanitized, no keys).""" | |
| safe_config = json.loads(json.dumps(config.config)) | |
| # Strip API keys | |
| for provider in safe_config["providers"].values(): | |
| if "api_key" in provider: | |
| provider["api_key"] = "[HIDDEN]" | |
| return jsonify(safe_config) | |
| def get_provider_config(provider): | |
| """Get config for specific provider.""" | |
| cfg = config.get_provider_config(provider) | |
| if not cfg: | |
| return jsonify({"error": f"Unknown provider: {provider}"}), 404 | |
| # Sanitize | |
| if "api_key" in cfg: | |
| cfg["api_key"] = "[HIDDEN]" | |
| return jsonify(cfg) | |
| def update_provider_config(provider): | |
| """Update provider configuration.""" | |
| data = request.json | |
| if provider not in config.config["providers"]: | |
| return jsonify({"error": f"Unknown provider: {provider}"}), 404 | |
| # Update configuration | |
| if "endpoint" in data: | |
| config.set_provider_endpoint(provider, data["endpoint"]) | |
| if "model_name" in data: | |
| model_blob = data.get("model_blob", data["model_name"]) | |
| config.set_provider_model(provider, data["model_name"], model_blob) | |
| if "enabled" in data: | |
| config.enable_provider(provider, data["enabled"]) | |
| # Save (without keys) | |
| config.save() | |
| return jsonify({"status": "updated", "provider": provider}) | |
| def enable_provider(provider): | |
| """Enable a provider.""" | |
| config.enable_provider(provider, True) | |
| config.save() | |
| return jsonify({"status": "enabled", "provider": provider}) | |
| def disable_provider(provider): | |
| """Disable a provider.""" | |
| config.enable_provider(provider, False) | |
| config.save() | |
| return jsonify({"status": "disabled", "provider": provider}) | |
| def get_default_provider(): | |
| """Get default provider.""" | |
| return jsonify({"default_provider": config.get_active_provider()}) | |
| def set_default_provider(): | |
| """Set default provider.""" | |
| data = request.json | |
| provider = data.get("provider") | |
| if provider not in config.config["providers"]: | |
| return jsonify({"error": f"Unknown provider: {provider}"}), 400 | |
| config.set_default_provider(provider) | |
| config.save() | |
| return jsonify({"status": "default_set", "provider": provider}) | |
| # ============================================================================ | |
| # CREDENTIAL ENDPOINTS (ENVIRONMENT VARIABLE BASED) | |
| # ============================================================================ | |
| def set_azure_credentials(): | |
| """Set Azure credentials via environment variable.""" | |
| data = request.json | |
| if "api_key" not in data: | |
| return jsonify({"error": "Missing api_key"}), 400 | |
| # Set environment variable (in memory only, not persisted) | |
| os.environ["COSMOS_AZURE_KEY"] = data["api_key"] | |
| config.load_credentials_from_env() | |
| return jsonify({"status": "credentials_set", "provider": "azure"}) | |
| def set_ibm_credentials(): | |
| """Set IBM credentials via environment variable.""" | |
| data = request.json | |
| if "api_key" not in data: | |
| return jsonify({"error": "Missing api_key"}), 400 | |
| # Set environment variable (in memory only, not persisted) | |
| os.environ["COSMOS_IBM_KEY"] = data["api_key"] | |
| config.load_credentials_from_env() | |
| return jsonify({"status": "credentials_set", "provider": "ibm"}) | |
| def test_credentials(provider): | |
| """Test provider credentials and connectivity.""" | |
| try: | |
| response = router.generate("Hello", provider=provider) | |
| return jsonify({"status": "success", "provider": provider, "sample": response[:100]}) | |
| except Exception as e: | |
| return jsonify({"status": "error", "provider": provider, "error": str(e)}), 400 | |
| # ============================================================================ | |
| # GENERATION ENDPOINTS | |
| # ============================================================================ | |
| def generate(): | |
| """Generate response from configured provider.""" | |
| data = request.json | |
| prompt = data.get("prompt") | |
| provider = data.get("provider") # Optional, uses default if not provided | |
| if not prompt: | |
| return jsonify({"error": "Missing prompt"}), 400 | |
| try: | |
| response = router.generate(prompt, provider=provider) | |
| return jsonify({"response": response, "provider": provider or config.get_active_provider()}) | |
| except Exception as e: | |
| return jsonify({"error": str(e)}), 400 | |
| def vision(): | |
| """Analyze image with vision model.""" | |
| # Expects multipart form with 'image' file and 'prompt' field | |
| if "image" not in request.files: | |
| return jsonify({"error": "Missing image file"}), 400 | |
| prompt = request.form.get("prompt", "Describe this image.") | |
| provider = request.form.get("provider") | |
| image_file = request.files["image"] | |
| image_path = f"/tmp/{image_file.filename}" | |
| image_file.save(image_path) | |
| try: | |
| response = router.vision(image_path, prompt, provider=provider) | |
| return jsonify({"response": response, "provider": provider or config.get_active_provider()}) | |
| except Exception as e: | |
| return jsonify({"error": str(e)}), 400 | |
| finally: | |
| if os.path.exists(image_path): | |
| os.remove(image_path) | |
| # ============================================================================ | |
| # STATUS & HEALTH ENDPOINTS | |
| # ============================================================================ | |
| def status(): | |
| """Get system status.""" | |
| providers = config.config["providers"] | |
| status_info = { | |
| "default_provider": config.get_active_provider(), | |
| "providers": { | |
| name: { | |
| "enabled": cfg.get("enabled", False), | |
| "endpoint": cfg.get("api_endpoint", "N/A"), | |
| "model": cfg.get("model_name", "N/A"), | |
| "has_credentials": bool(cfg.get("api_key")) | |
| } | |
| for name, cfg in providers.items() | |
| } | |
| } | |
| return jsonify(status_info) | |
| def health(): | |
| """Health check.""" | |
| return jsonify({"status": "healthy", "service": "COSMOS Cloud Router"}) | |
| # ============================================================================ | |
| # UI ENDPOINTS | |
| # ============================================================================ | |
| def dashboard(): | |
| """Configuration dashboard (simple HTML form).""" | |
| return render_template("dashboard.html") | |
| def docs(): | |
| """API documentation.""" | |
| return render_template("api_docs.html") | |
| # ============================================================================ | |
| # ERROR HANDLERS | |
| # ============================================================================ | |
| def not_found(error): | |
| return jsonify({"error": "Endpoint not found"}), 404 | |
| def internal_error(error): | |
| return jsonify({"error": "Internal server error"}), 500 | |
| # ============================================================================ | |
| # MAIN | |
| # ============================================================================ | |
| if __name__ == "__main__": | |
| print("COSMOS Cloud Endpoint Starting...") | |
| print(f"Default provider: {config.get_active_provider()}") | |
| print(f"Enabled providers: {[name for name, cfg in config.config['providers'].items() if cfg.get('enabled')]}") | |
| print("\nAPI Documentation: http://localhost:5000/docs") | |
| print("Configuration Dashboard: http://localhost:5000/") | |
| print("\nEnvironment variables for credentials:") | |
| print(" COSMOS_AZURE_KEY - Azure OpenAI API key") | |
| print(" COSMOS_IBM_KEY - IBM Watsonx API key") | |
| app.run(debug=True, host="0.0.0.0", port=5000) | |