Update README.md

README.md

@@ -13,7 +13,7 @@ tags:
 - legal
 ---
 
-# Adapting Large Language Models to Domains (ICLR 2024)
+# Adapting LLMs to Domains via Continual Pre-Training (ICLR 2024)
 This repo contains the domain-specific base model developed from **LLaMA-1-13B**, using the method in our paper [Adapting Large Language Models via Reading Comprehension](https://huggingface.co/papers/2309.09530).
 
 We explore **continued pre-training on domain-specific corpora** for large language models. While this approach enriches LLMs with domain knowledge, it significantly hurts their prompting ability for question answering. Inspired by human learning via reading comprehension, we propose a simple method to **transform large-scale pre-training corpora into reading comprehension texts**, consistently improving prompting performance across tasks in biomedicine, finance, and law domains. **Our 7B model competes with much larger domain-specific models like BloombergGPT-50B**. 
@@ -71,7 +71,7 @@ outputs = model.generate(input_ids=inputs, max_length=2048)[0]
 answer_start = int(inputs.shape[-1])
 pred = tokenizer.decode(outputs[answer_start:], skip_special_tokens=True)
 
-print(f'### User Input:\n{user_input}\n\n### Assistant Output:\n{pred}')
+print(pred)
 ```
 
 ## 2. Domain-Specific Tasks
@@ -97,7 +97,7 @@ You can use the following scripts to reproduce our results and evaluate any othe
    DOMAIN='law'
   
    # Specify any Huggingface model name (Not applicable to chat models)
-   MODEL='AdaptLLM/law-LLM'
+   MODEL='AdaptLLM/law-LLM-13B'
   
    # Model parallelization:
    # - Set MODEL_PARALLEL=False if the model fits on a single GPU.