Instructions to use jialinyyzz/humanizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jialinyyzz/humanizer with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="jialinyyzz/humanizer")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("jialinyyzz/humanizer")
model = AutoModelForMultimodalLM.from_pretrained("jialinyyzz/humanizer", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use jialinyyzz/humanizer with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf jialinyyzz/humanizer:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf jialinyyzz/humanizer:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf jialinyyzz/humanizer:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf jialinyyzz/humanizer:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf jialinyyzz/humanizer:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf jialinyyzz/humanizer:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf jialinyyzz/humanizer:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf jialinyyzz/humanizer:Q4_K_M

Use Docker

docker model run hf.co/jialinyyzz/humanizer:Q4_K_M

LM Studio
Jan

vLLM

How to use jialinyyzz/humanizer with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jialinyyzz/humanizer"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jialinyyzz/humanizer",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/jialinyyzz/humanizer:Q4_K_M

SGLang

How to use jialinyyzz/humanizer with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "jialinyyzz/humanizer" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jialinyyzz/humanizer",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "jialinyyzz/humanizer" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jialinyyzz/humanizer",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use jialinyyzz/humanizer with Ollama:
```
ollama run hf.co/jialinyyzz/humanizer:Q4_K_M
```
Unsloth Desktop
Docker Model Runner
How to use jialinyyzz/humanizer with Docker Model Runner:
```
docker model run hf.co/jialinyyzz/humanizer:Q4_K_M
```

Lemonade

How to use jialinyyzz/humanizer with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull jialinyyzz/humanizer:Q4_K_M

Run and chat with the model

lemonade run user.humanizer-Q4_K_M

List all available models

lemonade list

Atomic Chat

Overall really good

by Grio43 - opened 3 days ago

Discussion

Grio43

3 days ago

Overall this is very good.

It can loose things like the subject within the paragraph and shift things so they don't make sense. Overall likely the strongest model for the current use case.

Data does inject spelling errors and grammar errors.
Overall fairly good.

jialinyyzz

Owner 3 days ago

Thanks so much for trying this out and for the write up!

Those are both good points. The spelling/grammar issues are actually due to the training data - the 'human' side was actual human writing (with typos, etc) and the model kind of picked up on that. And yes, it dropping the subject or rearranging stuff to the point where it makes no sense is something I'm still training it out of (ie - meaning drift is something the fact judge penalizes)

In the meantime, you could do the rewrite then have an AI assistant do a comparison to your draft and just correct the problems (ie - subject drop, fact change, typo) but not everything else so you still have the rewrite style but without the errors. (Agents could call the model and do this step automatically)

Thanks you so much again

Grio43

3 days ago

I look forward to the future releases

jialinyyzz

Owner 3 days ago

Thanks, Im currently making some high quality quantizations for edge devices(it's much more better than the regular/popular quantization method). I got a little bit stuck on improving the rewrites quality, but, will do!

Grio43

2 days ago

•

edited 2 days ago

I don't really think a more powerful model or quant is currently needed.

I would suspect a supervisor that punishes for subject shift would help. Possibly a larger dataset of scholar articles, papers, and news prior to 2015 to avoid data poisoning. That would likely be a primary focus so it has a larger example set to not bake in a pitcular way of writing, and has more varried gradients.

Its hard to say currently if the best use is for multiple paragraphs or a single. Multi paragraphs, facts start getting lost. Single paragraphs things won't make overall sense sometimes in the big picture.

jialinyyzz

Owner 2 days ago

Yes, I totally agree with it. But it seems I Encounter some problem with it, like there's a ceiling of the accuracy rates and I didn't really came out a good method to break it. Maybe adding chain of thought is a good idea, but crafting this kind of data for chain of thought make me flagged by the AI company for distillation.

Grio43

2 days ago

Right now using an agent to do a before And after compression is doing the job fairly well. Ensure it isn't turning a paragraph that is supporting a subject to advocating against it. Agent also repairs facts. Even with the repairs+ grammerly grammer repair. The output is still 90% human on gptzero..panagram is still a failure..

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment