Update README.md

README.md

@@ -3,8 +3,6 @@ library_name: transformers
 license: apache-2.0
 language:
 - en
-datasets:
-- HuggingFaceTB/smollm-corpus
 ---
 
 
@@ -14,13 +12,6 @@ datasets:
     <img src="https://huggingface.co/datasets/HuggingFaceTB/images/resolve/main/banner_smol.png" alt="SmolLM" width="1100" height="600">
 </center>
 
-##  Table of Contents
-
-1. [Model Summary](##model-summary)
-2. [Limitations](##limitations)
-3. [Training](##training)
-4. [License](##license)
-5. [Citation](##citation)
 
 ## Model Summary
 
@@ -36,65 +27,23 @@ This is the SmolLM-135M-Instruct.
 pip install transformers
 ```
 
-#### Running the model on CPU/GPU/multi GPU
-* _Using full precision_
 ```python
 # pip install git+https://github.com/huggingface/transformers.git # TODO: merge PR to main
 from transformers import AutoModelForCausalLM, AutoTokenizer
-checkpoint = "HuggingFaceTB/SmolLM-135M"
+checkpoint = "HuggingFaceTB/SmolLM-135M-Instruct"
+
 device = "cuda" # for GPU usage or "cpu" for CPU usage
 tokenizer = AutoTokenizer.from_pretrained(checkpoint)
 # for multiple GPUs install accelerate and do `model = AutoModelForCausalLM.from_pretrained(checkpoint, device_map="auto")`
 model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)
-inputs = tokenizer.encode("def print_hello_world():", return_tensors="pt").to(device)
-outputs = model.generate(inputs)
-print(tokenizer.decode(outputs[0]))
-```
-```bash
->>> print(f"Memory footprint: {model.get_memory_footprint() / 1e6:.2f} MB")
-Memory footprint: 12624.81 MB
-```
-* _Using `torch.bfloat16`_
-```python
-# pip install accelerate
-import torch
-from transformers import AutoTokenizer, AutoModelForCausalLM
-checkpoint = "HuggingFaceTB/SmolLM-135M"
-tokenizer = AutoTokenizer.from_pretrained(checkpoint)
-# for fp16 use `torch_dtype=torch.float16` instead
-model = AutoModelForCausalLM.from_pretrained(checkpoint, device_map="auto", torch_dtype=torch.bfloat16)
-inputs = tokenizer.encode("def print_hello_world():", return_tensors="pt").to("cuda")
-outputs = model.generate(inputs)
-print(tokenizer.decode(outputs[0]))
-```
-```bash
->>> print(f"Memory footprint: {model.get_memory_footprint() / 1e6:.2f} MB")
-Memory footprint: 269.03 MB
-```
-
-#### Quantized Versions through `bitsandbytes`
-* _Using 8-bit precision (int8)_
 
-```python
-# pip install bitsandbytes accelerate
-from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
-# to use 4bit use `load_in_4bit=True` instead
-quantization_config = BitsAndBytesConfig(load_in_8bit=True)
-checkpoint = "HuggingFaceTB/SmolLM-135M"
-tokenizer = AutoTokenizer.from_pretrained(checkpoint)
-model = AutoModelForCausalLM.from_pretrained(checkpoint, quantization_config=quantization_config)
-inputs = tokenizer.encode("def print_hello_world():", return_tensors="pt").to("cuda")
-outputs = model.generate(inputs)
+messages = [{"role": "user", "content": "List the steps to bake a chocolate cake from scratch."}]
+input_text=tokenizer.apply_chat_template(messages, tokenize=False)
+print(input_text)
+inputs = tokenizer.encode(input_text, return_tensors="pt").to("cuda")
+outputs = model.generate(inputs, max_new_tokens=100, temperature=0.6, top_p=0.92, do_sample=True)
 print(tokenizer.decode(outputs[0]))
 ```
-```bash
->>> print(f"Memory footprint: {model.get_memory_footprint() / 1e6:.2f} MB")
-# load_in_8bit
-Memory footprint: 162.87 MB
-# load_in_4bit
->>> print(f"Memory footprint: {model.get_memory_footprint() / 1e6:.2f} MB")
-Memory footprint: 109.78 MB
-```
 
 # Limitations