Falcon-E-3B-FL-BitLoRA

Falcon-E-3B-FL-BitLoRA is a fine-tuned variant of tiiuae/Falcon-E-3B-Instruct trained with Federated Learning (FL) using BitLoRA adapters.

Key idea

• Base model: tiiuae/Falcon-E-3B-Instruct
• Training paradigm: Federated Learning (FedAvg)
• Parameter-efficient tuning: BitLoRA (adapter-only updates)
• Efficiency goal: reduce memory + communication cost while maintaining/boosting task performance

Training setup (summary)

• #Clients: 10
• Federated rounds: 10
• Local epochs per round: 3
• Base quantization: 1.58-bit
• LoRA config: r=8, α=16
• Dataset: medalpaca/medical_meadow_medqa

This repository focuses on the resulting fine-tuned checkpoints.