fumiyau/exp5_a_1_pt_123145

ロード方法

1. meta linguaの公式レポジトリをクローン
2. 所定の位置にtokenizerを配置する
   fumiyau/cryptollm_tokenizerにあるトークナイザを、{lingua_repo_path}/data/cryptollm_exp5/以下に置く。(こちらを参照)
3. apps/main/generate.pyを参考にgeneratorを呼び出す

   def main():
    # Load CLI arguments (overrides) and combine with a YAML config
    cfg = OmegaConf.from_cli()
    gen_cfg = dataclass_from_dict(
        PackedCausalTransformerGeneratorArgs, cfg, strict=False
    )
    print(cfg)
   
    model, tokenizer, _ = load_consolidated_model_and_tokenizer(cfg.ckpt)
   
    generator = PackedCausalTransformerGenerator(gen_cfg, model, tokenizer)
   
    # Allow multiple prompts
    prompts = []
    while True:
        prompt = input("Enter a prompt (or press enter to finish): ")
        if not prompt:
            break
        prompts.append(prompt)
   
    # Start generation
    start_time = time.time()
    generation, loglikelihood, greedy = generator.generate(prompts)
    end_time = time.time()
   
    # Calculate tokens per second
    total_tokens = sum(len(tokenizer.encode(gen, False, False)) for gen in generation)
    tokens_per_second = total_tokens / (end_time - start_time)
   
    # Display the results
    for i, gen in enumerate(generation):
        print(f"\nPrompt {i+1}: {prompts[i]}")
        print(f"Generated Text: {gen}")
   
    print(f"\nTokens per second: {tokens_per_second:.2f}")
   

   上記generate.pyは以下のコマンドで実行できる

   python -m apps.main.generate ckpt={model_repo_path}/consolidated