BlackSamorez

/

llama-13b-hf-tensor-parallel

like 2

Hi @milyiyo !
It's a model for tensor_parallel library and it' supposed to be ran on 2 GPUs
Here's the demo: https://www.kaggle.com/code/blacksamorez/tensor-parallel-llama-13b/

@BlackSamorez How do you convert a llama hf model into a tensor parallel model? Thanks. Asking since I am trying to test the 30b model.

@diegomontoya
It's detailed in the project's readme.