Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,75 @@
+---
+license: other
+language:
+- en
+pipeline_tag: text-generation
+inference: false
+tags:
+- transformers
+- gguf
+- imatrix
+- codegeex4-all-9b
+---
+Quantizations of https://huggingface.co/THUDM/codegeex4-all-9b
+
+
+### Inference Clients/UIs
+* [llama.cpp](https://github.com/ggerganov/llama.cpp)
+* [JanAI](https://github.com/janhq/jan)
+* [KoboldCPP](https://github.com/LostRuins/koboldcpp)
+* [text-generation-webui](https://github.com/oobabooga/text-generation-webui)
+* [ollama](https://github.com/ollama/ollama)
+* [GPT4All](https://github.com/nomic-ai/gpt4all)
+
+---
+
+# From original readme
+
+## Get Started
+
+Use `4.39.0<=transformers<=4.40.2` to quickly launch [codegeex4-all-9b](https://huggingface.co/THUDM/codegeex4-all-9b)：
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+device = "cuda" if torch.cuda.is_available() else "cpu"
+tokenizer = AutoTokenizer.from_pretrained("THUDM/codegeex4-all-9b", trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(
+    "THUDM/codegeex4-all-9b",
+    torch_dtype=torch.bfloat16,
+    low_cpu_mem_usage=True,
+    trust_remote_code=True
+).to(device).eval()
+inputs = tokenizer.apply_chat_template([{"role": "user", "content": "write a quick sort"}], add_generation_prompt=True, tokenize=True, return_tensors="pt", return_dict=True ).to(device)
+with torch.no_grad():
+    outputs = model.generate(**inputs, max_length=256)
+    outputs = outputs[:, inputs['input_ids'].shape[1]:]
+    print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+If you want to build the **chat** prompt manually, please make sure it follows the following format:
+```
+f"<|system|>\n{system_prompt}\n<|user|>\n{prompt}\n<|assistant|>\n"
+```
+Default system_prompt:
+```
+你是一位智能编程助手，你叫CodeGeeX。你会为用户回答关于编程、代码、计算机方面的任何问题，并提供格式规范、可以执行、准确安全的代码，并在必要时提供详细的解释。
+```
+The English version:
+```
+You are an intelligent programming assistant named CodeGeeX. You will answer any questions users have about programming, coding, and computers, and provide code that is formatted correctly.
+```
+
+For **infilling** ability, please use (without system prompt):
+```
+f"<|user|>\n<|code_suffix|>{suffix}<|code_prefix|>{prefix}<|code_middle|><|assistant|>\n"
+```
+Additional infos (like file path, programming language, mode) can be added. Example:
+```
+<|user|>
+###PATH:src/example.py
+###LANGUAGE:Python
+###MODE:BLOCK
+<|code_suffix|>{suffix}<|code_prefix|>{prefix}<|code_middle|><|assistant|>
+```