rahulshetty/gliner2-log-analysis-v1

GLiNER2 Log Analysis — Fine-Tuning Kit

What was built

1. Synthetic log dataset (train.jsonl / val.jsonl) in the exact GLiNER2 JSONL format with:

   • Entity extraction: timestamp, log_level, service, source_ip, port, user, file_path, domain, process_id, error_message
   • Classification: log_category → error, network, security, info, warning, application
   • 3,000 train / 300 validation samples

2. Training script (train_log_gliner.py) that supports both LoRA and full fine-tuning using the official GLiNER2Trainer + TrainingConfig API.

3. Inference script (inference.py) that loads the base model + adapter and extracts entities + classification from raw log lines.

4. HF Jobs script (hf_train_job.py) ready for cloud GPU training with automatic Hub push.

Dataset structure (GLiNER2 JSONL)

Each line:

{
  "input": "2024-05-15T14:18:27Z nginx [INFO] New connection from 183.80.103.255:45043",
  "output": {
    "entities": {
      "timestamp": ["2024-05-15T14:18:27Z"],
      "log_level": ["INFO"],
      "service": ["nginx"],
      "source_ip": ["183.80.103.255"],
      "port": ["45043"]
    },
    "classifications": [
      {
        "task": "log_category",
        "labels": ["error","network","security","info","warning","application"],
        "true_label": ["network"]
      }
    ]
  }
}

Running locally (CPU)

# Install dependencies
pip install gliner2 torch transformers peft accelerate urllib3 requests

# Quick training test (2 epochs, 50 samples)
python train_log_gliner.py \
  --base_model fastino/gliner2-base-v1 \
  --train_data train.jsonl \
  --val_data val.jsonl \
  --output_dir ./test_output \
  --num_epochs 2 \
  --batch_size 4 \
  --max_train_samples 50

# Inference using trained adapter
python inference.py \
  --model_path ./test_output/best \
  --text "2024-06-15T10:23:45Z nginx [ERROR] Connection refused to backend 10.0.1.5:8080"

Running full training on GPU (recommended)

Use the HF Jobs script (or any machine with a GPU):

python hf_train_job.py

Hyperparameters tuned for a 205M base model with LoRA:

• num_epochs=15
• batch_size=16 + gradient_accumulation_steps=4 (effective batch = 64)
• encoder_lr=5e-6, task_lr=1e-4
• lora_r=16, lora_alpha=32
• fp16=True

Using a different encoder (e.g. Qwen3)

The GLiNER2 Extractor class loads its transformer encoder via:

AutoModel.from_pretrained(model_name, trust_remote_code=True)

(from gliner2/model.py line ~189)

So any HuggingFace model that is loadable with AutoModel (DeBERTa, BERT, RoBERTa, Qwen2, Qwen3, etc.) can be used as the backbone as long as you train from scratch or convert the GLiNER2 head weights accordingly. However, the released GLiNER2 checkpoints (fastino/gliner2-base-v1, fastino/gliner2-large-v1) are tied to microsoft/deberta-v3-base and microsoft/deberta-v3-large. You cannot just swap the encoder and load these pretrained checkpoints.

To use Qwen3 (or any other encoder)

1. Pick a small GLiNER2-compatible config or initialize a fresh Extractor with model_name="Qwen/Qwen3-0.6B".
2. Train the full model from scratch (no LoRA on released checkpoints).
3. Note: Qwen3 is a decoder-only model; GLiNER2 uses an encoder-style pooling (first token / span markers). You may need to adapt the SpanRepLayer or token-pooling if the hidden dimension / architecture differs significantly.

For most practical fine-tuning, the released deberta-v3 checkpoints are the fastest starting point. If you need a different encoder for embedding quality, retrain the full GLiNER2 head from scratch with that encoder.

Artifacts on HuggingFace Hub

• Dataset: https://huggingface.co/datasets/rahulshetty/log-gliner2-dataset
• Model repo (adapter): https://huggingface.co/rahulshetty/gliner2-log-analysis-v1