Fine-Tuning xLSTM-7B on a budget: An Experimental Journey with Chat Templates
β’
1
AI & ML interests
Building practical, ethical and sustainable AI/ML projects.
Recent Activity
Articles
mrs83Β
updated
a
model
11 days ago
mrs83Β
published
a
dataset
28 days ago
mrs83Β
published
a
model
about 1 month ago
mrs83Β
updated
2
models
3 months ago
mrs83Β
published
2
models
3 months ago
mrs83Β
updated
2
models
3 months ago
mrs83Β
updated
a
Space
3 months ago
Post
235
Have you ever had a real conversation with a local LLM?
Or even taken a VoIP (SIP) phone call with one?
Check out Kurtis E1: A Fully On-Device MLX Voice Agent.
- GitHub Repo: https://github.com/ethicalabs-ai/Kurtis-E1-MLX-Voice-Agent
- Demo Video 1: https://www.youtube.com/watch?v=k6BbF0262I8
- Demo Video 2: https://www.youtube.com/shorts/w-YSCyKTR78
The entire stack runs on-device, leveraging MLX-LM for Apple Silicon.
- Whisper for real-time STT
- WebRTC VAD for voice activity detection
- A custom-tuned Qwen #LLM (Kurtis E1) ethicalabs/Kurtis-E1.1-Qwen2.5-3B-Instruct
- Coqui XTTSv2 for natural speech
This showcases the power of local AI/ML. I am also actively developing the SIP #VoIP integration (now in testing). The goal? To let you take a phone call and talk directly with your private agent, even without a computer or internet connection.
While Kurtis isn't built for math/coding, it shows a valuable path forward for on-device workflows.
We are actively looking for partners and clients to build out these POCs into real-world use cases.
https://www.ethicalabs.ai/ isn't a startup. We are not looking for VCs, equity deals, or grants: we're an open-source project.
If you like the R&D, you can support the R&D directly: https://github.com/sponsors/ethicalabs-ai?frequency=one-time
Or even taken a VoIP (SIP) phone call with one?
Check out Kurtis E1: A Fully On-Device MLX Voice Agent.
- GitHub Repo: https://github.com/ethicalabs-ai/Kurtis-E1-MLX-Voice-Agent
- Demo Video 1: https://www.youtube.com/watch?v=k6BbF0262I8
- Demo Video 2: https://www.youtube.com/shorts/w-YSCyKTR78
The entire stack runs on-device, leveraging MLX-LM for Apple Silicon.
- Whisper for real-time STT
- WebRTC VAD for voice activity detection
- A custom-tuned Qwen #LLM (Kurtis E1) ethicalabs/Kurtis-E1.1-Qwen2.5-3B-Instruct
- Coqui XTTSv2 for natural speech
This showcases the power of local AI/ML. I am also actively developing the SIP #VoIP integration (now in testing). The goal? To let you take a phone call and talk directly with your private agent, even without a computer or internet connection.
While Kurtis isn't built for math/coding, it shows a valuable path forward for on-device workflows.
We are actively looking for partners and clients to build out these POCs into real-world use cases.
https://www.ethicalabs.ai/ isn't a startup. We are not looking for VCs, equity deals, or grants: we're an open-source project.
If you like the R&D, you can support the R&D directly: https://github.com/sponsors/ethicalabs-ai?frequency=one-time
Post
344
hallbayes https://github.com/leochlon/hallbayes is an interesting project by Leon Chlon (Hassana Labs) for checking for hallucination risk before text generation and it uses a powerful approach to decide if an LLM is confident enough to answer (or not).
https://arxiv.org/html/2509.11208v1
Predictable Compression Failures: Why Language Models Actually Hallucinate (2509.11208)
I've just integrated the hallbayes library into my
Ran a small test on 10 samples on google/boolq with a 4B Qwen Instruct model Qwen/Qwen3-4B-Instruct-2507.
The output dataset now contains a
Test w/ hallucination flags: ethicalabs/google-boolq-hallbayes-test-qwen3-4b-2507
Implementation MRs:
https://github.com/leochlon/hallbayes/pull/16
https://github.com/ethicalabs-ai/completionist/pull/11
https://arxiv.org/html/2509.11208v1
Predictable Compression Failures: Why Language Models Actually Hallucinate (2509.11208)
I've just integrated the hallbayes library into my
completionist (synthetic dataset generation CLI tool) project to do exactly that, adding a new quality control layer to synthetic data generation.Ran a small test on 10 samples on google/boolq with a 4B Qwen Instruct model Qwen/Qwen3-4B-Instruct-2507.
The output dataset now contains a
hallucination_info column, flagging each sample with detailed metrics. The inference server is LM Studio, running on a Macbook Air M4 16GBTest w/ hallucination flags: ethicalabs/google-boolq-hallbayes-test-qwen3-4b-2507
Implementation MRs:
https://github.com/leochlon/hallbayes/pull/16
https://github.com/ethicalabs-ai/completionist/pull/11
Post
366
Hello HF community, I'm happy to share a project I've been working on that combines mlx-lm with Flower, to enable federated fine-tuning of SLMs (Small Language Models) on MacOS devices ο£Ώ
GitHub Repo: https://github.com/ethicalabs-ai/BlossomTuneLLM-MLX
By combining mlx-lm with a federated learning framework like Flower (https://flower.ai/), we can leverage the hardware people already own and reduce the reliance on expensive GPUs, enabling collaborative model training.
This project is the MLX-native evolution of an earlier codebase for FlowerTune LLM:
https://arxiv.org/abs/2506.02961
https://flower.ai/blog/2024-10-16-flowertune-llm-leaderboard
https://github.com/ethicalabs-ai/BlossomTuneLLM
How it works:
Flower handles all the federated learning logic.
A central server (superlink) coordinates the training rounds, client selection, and parameter aggregation.
Each participant in the network runs a Flower client (supernode) on their Mac. In each round, the client:
- Receives the global LoRA/DoRA adapter weights from the server.
- Loads its local data partition.
- It makes use of the mlx-lm programmatic API (mlx_lm.tuner.train) to perform LoRA/DoRA fine-tuning.
- Sends only the updated adapter weights back to the server.
The server only ever sees the aggregated model updates and private data never leaves the device.
Flower made it easy to run a full simulation (with a centralized HF dataset, partitioned using flower-datasets) on a single machine or multiple machines, to test the whole process in action and experiment further.
All you need is a single or multiple Mac machines with Apple Silicon ο£Ώ
GitHub Repo: https://github.com/ethicalabs-ai/BlossomTuneLLM-MLX
By combining mlx-lm with a federated learning framework like Flower (https://flower.ai/), we can leverage the hardware people already own and reduce the reliance on expensive GPUs, enabling collaborative model training.
This project is the MLX-native evolution of an earlier codebase for FlowerTune LLM:
https://arxiv.org/abs/2506.02961
https://flower.ai/blog/2024-10-16-flowertune-llm-leaderboard
https://github.com/ethicalabs-ai/BlossomTuneLLM
How it works:
Flower handles all the federated learning logic.
A central server (superlink) coordinates the training rounds, client selection, and parameter aggregation.
Each participant in the network runs a Flower client (supernode) on their Mac. In each round, the client:
- Receives the global LoRA/DoRA adapter weights from the server.
- Loads its local data partition.
- It makes use of the mlx-lm programmatic API (mlx_lm.tuner.train) to perform LoRA/DoRA fine-tuning.
- Sends only the updated adapter weights back to the server.
The server only ever sees the aggregated model updates and private data never leaves the device.
Flower made it easy to run a full simulation (with a centralized HF dataset, partitioned using flower-datasets) on a single machine or multiple machines, to test the whole process in action and experiment further.
All you need is a single or multiple Mac machines with Apple Silicon ο£Ώ
Post
340
While LLMs are getting a lot of attention, I believe in the power of narrow AI/ML to solve everyday problems.
That's why I've created "Obesity Risk Predictor", a tool designed to be a preventive measure, helping to identify health risks based on lifestyle habits.
Itβs a clear example of AI/ML built for a specific and impactful task.
The gradio app lets you compare the performance of three different models (Random Forest, LightGBM, and XGBoost) on the same dataset.
Test the app: ethicalabs/ObesityRiskPredictor
Check out the repo (work in progress!): https://github.com/ethicalabs-ai/ObesityRiskPredictor
Please donate to support ethicalabs.ai projects!
- GitHub: https://github.com/sponsors/ethicalabs-ai
- OpenCollective: https://opencollective.com/ethicalabs-ai
That's why I've created "Obesity Risk Predictor", a tool designed to be a preventive measure, helping to identify health risks based on lifestyle habits.
Itβs a clear example of AI/ML built for a specific and impactful task.
The gradio app lets you compare the performance of three different models (Random Forest, LightGBM, and XGBoost) on the same dataset.
Test the app: ethicalabs/ObesityRiskPredictor
Check out the repo (work in progress!): https://github.com/ethicalabs-ai/ObesityRiskPredictor
Please donate to support ethicalabs.ai projects!
- GitHub: https://github.com/sponsors/ethicalabs-ai
- OpenCollective: https://opencollective.com/ethicalabs-ai
Post
2863
Introducing the Computer Says No Dataset:
ethicalabs/computer-says-no
An LLM can do almost anything, but should it?
This dataset provides clear examples of when LLMs should decline requests, such as:
- Counting characters (e.g., "number of 'r's in 'raspberry'" β seriously, youβve got this)
- Solving basic equations (like *5.9 = x + 5.11* β please, show that calculator some love)
Inspired by Little Britain's iconic "Computer Says No" sketch, we address a critical issue in AI systems today: the waste of using a rocket launcher to swat flies (aka powerful models for trivial tasks).
Goals:
- Reduce waste by saving compute for tasks that actually need it
- Guide users to better tools
- Spark discussion about ethical AI
This isnβt a training set. Itβs a provocation: if we donβt define AI's limits, who will?
An LLM can do almost anything, but should it?
This dataset provides clear examples of when LLMs should decline requests, such as:
- Counting characters (e.g., "number of 'r's in 'raspberry'" β seriously, youβve got this)
- Solving basic equations (like *5.9 = x + 5.11* β please, show that calculator some love)
Inspired by Little Britain's iconic "Computer Says No" sketch, we address a critical issue in AI systems today: the waste of using a rocket launcher to swat flies (aka powerful models for trivial tasks).
Goals:
- Reduce waste by saving compute for tasks that actually need it
- Guide users to better tools
- Spark discussion about ethical AI
This isnβt a training set. Itβs a provocation: if we donβt define AI's limits, who will?
-
- 9 replies
Post
2760
Introducing Completionist, an open-source command-line tool that automates synthetic dataset generation.
It works by iterating over an existing HF dataset and by using a LLM to create completions.
- Problem: You need a fast way to create custom datasets for fine-tuning or RAG, but you want the flexibility to use different LLM backends or your own infrastructure.
- Solution: Completionist connects with any OpenAI-compatible endpoint, including Ollama and LM Studio, or a Hugging Face inference endpoint.
A simple CLI like Completionist gives you the possibility to take full control of your synthetic data generation workflow.
π Check out Completionist on GitHub: https://github.com/ethicalabs-ai/completionist
Synthetic Dataset Example: https://huggingface.co/datasets/ethicalabs/kurtis-mental-health-v2-sft-reasoning
It works by iterating over an existing HF dataset and by using a LLM to create completions.
- Problem: You need a fast way to create custom datasets for fine-tuning or RAG, but you want the flexibility to use different LLM backends or your own infrastructure.
- Solution: Completionist connects with any OpenAI-compatible endpoint, including Ollama and LM Studio, or a Hugging Face inference endpoint.
A simple CLI like Completionist gives you the possibility to take full control of your synthetic data generation workflow.
π Check out Completionist on GitHub: https://github.com/ethicalabs-ai/completionist
Synthetic Dataset Example: https://huggingface.co/datasets/ethicalabs/kurtis-mental-health-v2-sft-reasoning
-
- 1 reply
Post
1054
Hello Hugging Face Community! I'm excited to share a project I've been working on: SkinCancerViT, a multimodal Vision Transformer model for skin lesion analysis
ethicalabs/SkinCancerViT
I've wrapped it in a Gradio app to make it easy to explore: ethicalabs/SkinCancerViTPredictor
This app is a research demonstration that combines dermatoscopic images with patient age and lesion localization to assist in classifying skin lesions.
You can either upload your own image and patient data for a prediction, or explore how the model performs on random samples from the marmal88/skin_cancer dataset.
I firmly believe that the only final, trustworthy diagnosis comes from medical professionals, and I am actively seeking medical institutions and researchers who might be interested in partnering with me to further explore the usage of this methodology, conducting further training with diverse datasets (ethically sourced and anonymized), performing extensive validation tests, and explore the possibility of running a federated fine-tuning simulation with https://flower.ai/
As a software engineer, I do not possess medical expertise and I am seeking collaboration with medical professionals and AI/ML researchers. You can find the project source code, which includes data preprocessing, model training and testing, at the following url: https://github.com/ethicalabs-ai/SkinCancerViT/tree/main
Thank you for your time and consideration!!!
I've wrapped it in a Gradio app to make it easy to explore: ethicalabs/SkinCancerViTPredictor
This app is a research demonstration that combines dermatoscopic images with patient age and lesion localization to assist in classifying skin lesions.
You can either upload your own image and patient data for a prediction, or explore how the model performs on random samples from the marmal88/skin_cancer dataset.
I firmly believe that the only final, trustworthy diagnosis comes from medical professionals, and I am actively seeking medical institutions and researchers who might be interested in partnering with me to further explore the usage of this methodology, conducting further training with diverse datasets (ethically sourced and anonymized), performing extensive validation tests, and explore the possibility of running a federated fine-tuning simulation with https://flower.ai/
As a software engineer, I do not possess medical expertise and I am seeking collaboration with medical professionals and AI/ML researchers. You can find the project source code, which includes data preprocessing, model training and testing, at the following url: https://github.com/ethicalabs-ai/SkinCancerViT/tree/main
Thank you for your time and consideration!!!
-
- 3 replies
Post
1764
To developers: Build opt-in systems.
To policymakers: Legislate data transparency.
To artists: Unionize.
To users: Demand ethical tools.
To policymakers: Legislate data transparency.
To artists: Unionize.
To users: Demand ethical tools.
-
- 1 reply
Post
3234
π Just released a PoC: Kurtis-E1 MLX Voice Agent
An offline, privacy-first voice assistant built for macOS (Apple Silicon), designed for empathetic, short-form interactions.
π§ Powered by:
- Whisper (via MLX) for speech-to-text: https://pypi.org/project/mlx-whisper/
- Kurtis-E1 (a custom SmolLM2 LLM) via Ollama
- Coqui-TTS XTTSv2 for multilingual TTS
- Optional translation layer via TowerInstruct-13B-v0.1 for non-English voice input/output: Unbabel/TowerInstruct-13B-v0.1
π§ Everything runs entirely on-device (Mac Mini M4 Max - 24gb) β no cloud, no remote API calls, no data leakage.
π‘ Code is fully handcrafted (no AI-generated code), and designed to showcase whatβs possible with local models, even on laptops.
π οΈ Open to contributions, ideas (e.g., LM Studio for MLX inference, MLX worker subprocess, optimize for latency and VRAM usage).
π Video demo (Italian): https://www.youtube.com/watch?v=8-1PcmUStaI
PoC: https://github.com/ethicalabs-ai/Kurtis-E1-MLX-Voice-Agent
Kurtis-E1: ethicalabs/kurtis-e1-67a9148e0836885c44c7902c
Kurtis-E1 WebGPU: ethicalabs/Kurtis-E1-WebGPU
An offline, privacy-first voice assistant built for macOS (Apple Silicon), designed for empathetic, short-form interactions.
π§ Powered by:
- Whisper (via MLX) for speech-to-text: https://pypi.org/project/mlx-whisper/
- Kurtis-E1 (a custom SmolLM2 LLM) via Ollama
- Coqui-TTS XTTSv2 for multilingual TTS
- Optional translation layer via TowerInstruct-13B-v0.1 for non-English voice input/output: Unbabel/TowerInstruct-13B-v0.1
π§ Everything runs entirely on-device (Mac Mini M4 Max - 24gb) β no cloud, no remote API calls, no data leakage.
π‘ Code is fully handcrafted (no AI-generated code), and designed to showcase whatβs possible with local models, even on laptops.
π οΈ Open to contributions, ideas (e.g., LM Studio for MLX inference, MLX worker subprocess, optimize for latency and VRAM usage).
π Video demo (Italian): https://www.youtube.com/watch?v=8-1PcmUStaI
PoC: https://github.com/ethicalabs-ai/Kurtis-E1-MLX-Voice-Agent
Kurtis-E1: ethicalabs/kurtis-e1-67a9148e0836885c44c7902c
Kurtis-E1 WebGPU: ethicalabs/Kurtis-E1-WebGPU
-
- 3 replies
mrs83Β
updated
a
dataset
10 months ago