Instructions to use ZhihaiLLM/wisdomInterrogatory with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ZhihaiLLM/wisdomInterrogatory with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ZhihaiLLM/wisdomInterrogatory", trust_remote_code=True)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("ZhihaiLLM/wisdomInterrogatory", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use ZhihaiLLM/wisdomInterrogatory with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ZhihaiLLM/wisdomInterrogatory"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ZhihaiLLM/wisdomInterrogatory",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/ZhihaiLLM/wisdomInterrogatory

SGLang

How to use ZhihaiLLM/wisdomInterrogatory with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ZhihaiLLM/wisdomInterrogatory" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ZhihaiLLM/wisdomInterrogatory",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ZhihaiLLM/wisdomInterrogatory" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ZhihaiLLM/wisdomInterrogatory",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use ZhihaiLLM/wisdomInterrogatory with Docker Model Runner:
```
docker model run hf.co/ZhihaiLLM/wisdomInterrogatory
```

智海-录问

项目背景

智海-录问(wisdomInterrogatory)是由浙江大学、阿里巴巴达摩院以及华院计算三家单位共同设计研发的法律大模型。核心思想：以“普法共享和司法效能提升”为目标，从推动法律智能化体系入司法实践、数字化案例建设、虚拟法律咨询服务赋能等方面提供支持，形成数字化和智能化的司法基座能力。

模型训练

我们的模型基座是Baichuan-7B，在此基础上，进行了二次预训练以及指令微调训练。

二次预训练

二次预训练的目的是给通用的大模型注入法律领域的知识。预训练的数据包括法律文书、司法案例以及法律问答数据，共40G。

指令微调训练

经过了二次预训练之后，在指令微调阶段，我们使用了100k的指微调训练，其目的是让大模型具备问答的能力，能够直接与用户进行交流。

推理代码

推理环境安装

transformers>=4.27.1
accelerate>=0.20.1
torch>=2.0.1
modelscope>=1.8.3
sentencepiece==0.1.99

推理代码调用

import os
os.environ["CUDA_VISIBLE_DEVICES"] = "0"
from modelscope import AutoModelForCausalLM, AutoTokenizer, snapshot_download
import torch


model_id = "wisdomOcean/wisdomInterrogatory"
revision = 'v1.0.0'
model_dir = snapshot_download(model_id, revision)

def generate_response(prompt: str) -> str:
    inputs = tokenizer(f'</s>Human:{prompt} </s>Assistant: ', return_tensors='pt')
    inputs = inputs.to('cuda')
    pred = model.generate(**inputs, max_new_tokens=800, 
                          repetition_penalty=1.2)
    response = tokenizer.decode(pred.cpu()[0], skip_special_tokens=True)
    return response.split("Assistant: ")[1]

tokenizer = AutoTokenizer.from_pretrained(model_dir, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_dir, device_map="auto", 
                                             torch_dtype=torch.float16,
                                             trust_remote_code=True)
prompt = "如果喝了两斤白酒后开车，会有什么后果？"
resp = generate_response(prompt)
print(resp)

免责声明

本模型仅供学术研究之目的而提供，不保证结果的准确性、完整性或适用性。在使用模型生成的内容时，您应自行判断其适用性，并自担风险。

Downloads last month: 14