silviachen46
/

llama-test-model

Text Generation

text-generation-inference

Model card Files Files and versions

llama-test-model / serving.properties

silviachen46's picture

Upload 17 files

38020f8 about 2 years ago

history blame contribute delete

205 Bytes

	engine=DeepSpeed
	option.entryPoint=inference.py
	option.tensor_parallel_degree=1
	option.max_tokens=4096
	option.task=text-generation
	option.dtype=bf16
	option.model_type=base
	option.model_loading_timeout=3600