kashif HF Staff commited on
Commit
094fb45
·
verified ·
1 Parent(s): e226c73

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +50 -1
README.md CHANGED
@@ -4,4 +4,53 @@ library_name: transformers
4
  tags:
5
  - custom_generate
6
  - sampling
7
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
  tags:
5
  - custom_generate
6
  - sampling
7
+ ---
8
+
9
+
10
+ # DeepCONF Custom Generation Strategy
11
+
12
+ This repository implements the DeepCONF (Deep Confidence-based Early Stopping) generation strategy for Hugging Face Transformers models, following the [Deep Think with Confidence](https://jiaweizzhao.github.io/deepconf/) approach from the paper [Deep Think with Confidence](https://huggingface.co/papers/2508.15260).
13
+
14
+ ## Overview
15
+
16
+ DeepCONF monitors the confidence of generated tokens and stops generation when confidence falls below a threshold.
17
+
18
+ ## Parameters
19
+
20
+ - `enable_conf` (bool): Whether to enable the DeepCONF strategy. Defaults to `False`.
21
+ - `window_size` (int): Size of the sliding window for confidence calculation. Defaults to `2048`.
22
+ - `threshold` (float): Confidence threshold for early stopping. Defaults to `17.0`.
23
+ - `output_confidences` (bool): If `True` and `return_dict_in_generate=True`, returns a per-step confidence tensor alongside generated sequences for debugging/visualization.
24
+
25
+ ## Usage
26
+
27
+ To use this custom generation strategy, you can pass it directly to the `generate` method:
28
+
29
+ ```python
30
+ from transformers import AutoModelForCausalLM, AutoTokenizer
31
+
32
+ model = AutoModelForCausalLM.from_pretrained("your-model")
33
+ tokenizer = AutoTokenizer.from_pretrained("your-model")
34
+
35
+ inputs = tokenizer("Hello, world!", return_tensors="pt")
36
+
37
+ # Generate with DeepCONF (Hub repo)
38
+ outputs = model.generate(
39
+ **inputs,
40
+ enable_conf=True,
41
+ window_size=2048,
42
+ threshold=17.0,
43
+ output_confidences=True, # request confidences
44
+ return_dict_in_generate=True, # required to get tensors
45
+ max_new_tokens=100,
46
+ custom_generate="kashif/DeepConf", # Hugging Face Hub repo
47
+ trust_remote_code=True
48
+ )
49
+
50
+ ```
51
+
52
+ ## Requirements
53
+
54
+ - PyTorch >= 1.13.0
55
+ - Transformers >= 4.35.0
56
+