| --- |
| base_model: Qwen/Qwen3-VL-8B-Instruct |
| library_name: peft |
| pipeline_tag: image-text-to-text |
| tags: |
| - qwen3-vl |
| - lora |
| - causal-inference |
| - mm-vet |
| --- |
| |
| # CausalLens (Qwen3-VL-8B) |
|
|
| Weights for **CausalLens**: a Qwen3-VL-8B model with a LoRA adapter on the language model and a |
| *causal residual Q-Former* on the visual tokens. The Q-Former applies a backdoor-adjustment style |
| intervention over a fixed confounder dictionary and adds a small, gated residual to every visual |
| token before it enters the language model. |
|
|
| Code and usage: https://github.com/litcoderr/CausalLens |
|
|
| ## Files |
|
|
| | Path | Content | |
| | --- | --- | |
| | `adapter_config.json`, `adapter_model.safetensors` | LoRA adapter (r=16, alpha=32; q/k/v/o/gate/up/down projections of the LLM) | |
| | `causal_qformer_adapter.pt`, `causal_qformer_config.json` | Causal residual Q-Former (weights + confounder dictionary 16384x256 + prior) | |
| | tokenizer / `preprocessor_config.json` / `chat_template.jinja` | Processor used for evaluation | |
| | `base/` | Qwen3-VL-8B base checkpoint that the adapters were trained on (fine-tuned from `Qwen/Qwen3-VL-8B-Instruct`, weights merged) | |
|
|
| ## Usage |
|
|
| ```bash |
| git clone https://github.com/litcoderr/CausalLens.git && cd CausalLens |
| ./setup_env.sh # Python 3.11 venv with pinned dependencies |
| ./download.sh # this repository -> checkpoints/CausalLens (base -> checkpoints/CausalLens/base) |
| ./run_mmvet.sh # MM-Vet inference (data setup: see the GitHub README) |
| ``` |
|
|
| From Python (inside the CausalLens repository): |
|
|
| ```python |
| from causallens import load_model_and_processor |
| |
| model, processor = load_model_and_processor( |
| base_model="checkpoints/CausalLens/base", |
| checkpoint="checkpoints/CausalLens", |
| ) |
| ``` |
|
|
