Red (HF)

redaihf

·

AI & ML interests

(SO)MPOA decensoring, residual refusals, contextual ethical realignment

Recent Activity

new activity about 6 hours ago

SubMaroon/Boulesis-v2.1-26B-A4B-HeadTune:Hm 🤔

new activity about 6 hours ago

MuXodious/gpt-oss-20b-RichardErkhov-heresy:Writing Model Requests

reacted to OppaAI's post with 🧠 about 9 hours ago

Benchmark test: Jev vs. Laya-ONNX (multilingual) vs. Harrier OSS 270M embedder 🔬 My AI wAIfu (Jetson Orin Nano 8GB) uses Harrier OSS 270M for semantic routing in 2 places. It reads vectors of router prompts (English only) and calculates cosine similarity: - Quaternary routing: greeting, local chat (no websearch), web chat (needs websearch), or agentic chat - Agentic routing: which tools in my AI's capability list to use Benchmarked the 2 most hyped decision models — Jev and Laya (ONNX, multilingual) — against Harrier OSS 270M. Setup: 221 quaternary + 58 capability-trigger examples, leave-one-out eval, argmax, no thresholds. Results: → Harrier-270M (local, cosine): 94.6% / 93.1% accuracy, 17ms P50 ⚡ → Jev API (hosted): 82.4% / 94.8% accuracy, ~195ms P50 → Laya-ONNX multilingual (fp16, local): 48.0% / 20.7% accuracy, 25-40ms P50 Conclusion: 🚫 Laya is out of the question. 4 of 7 capability categories at 0.0% accuracy while reporting 80-90% confidence means it needs real training before it's practical. ☁️ Jev is a cloud API, not sure it can be trained further. Accuracy is high but not improvable on my end. Latency is ~10x my local embedder (network latency). Input token cost, though small, is still more than $0. Not fully sure about privacy implications either. ✅ Embedding is only semantic cosine similarity, not real reasoning. But it's already doing double duty for memory extraction and RAG — no extra RAM or token cost. Latency is 17ms, accuracy in the 90s%. Even tried Japanese/Chinese prompts, still got high accuracy with only English exemplars. Bigger advantage: I just add exemplars to boost accuracy. When I add/modify/remove tools — often — no retraining needed, just update exemplars, vectors recompute once. Turns out my self-invented routing method, built ~6 months ago, already solved what these now hyped up models — beating Jev and Laya on latency and convenience, matching/beating on accuracy. 🎯

View all activity

Organizations

redaihf 's collections 4