Create README.md

README.md

@@ -0,0 +1,95 @@
+---
+license: creativeml-openrail-m
+language:
+- en
+tags:
+- LLM
+- tensorRT
+- chatGLM
+---
+## Model Card for lyraChatGLM
+
+lyraChatGLM is currently the **fastest chatGLM-6B** available, as far as we know, it is also the **fisrt accelerated version of chatGLM-6B**.
+
+The inference speed of lyraChatGLM is **10x** faster than the original version, and we're still working to improve the performance.
+
+Among its main features are:
+
+- weights: original ChatGLM-6B weights released by THUDM.
+- device: lyraChatGLM is mainly based on FasterTransformer compiled for SM=80 (A100, for example).
+
+## Speed
+
+### test environment
+
+- device: Nvidia A100 40G
+- img size: 512x512
+- percision:fp16
+- steps: 30
+- solver: LMSD
+
+### text2img
+
+
+
+
+## Model Sources
+
+- **Repository:** [https://huggingface.co/THUDM/chatglm-6b]
+
+## Uses
+
+```python
+from faster_chat_glm import GLM6B, FasterChatGLM
+
+# kernel for chat model.
+kernel = GLM6B(plan_path=plan_path,
+               batch_size=BATCH_SIZE,
+               num_beams=1,
+               use_cache=USE_CACHE,
+               num_heads=32,
+               emb_size_per_heads=128,
+               decoder_layers=28,
+               vocab_size=150528,
+               max_seq_len=MAX_OUT_LEN)
+
+chat = FasterChatGLM(model_dir=chatglm6b_dir, kernel=kernel).half().cuda()
+
+# generate
+sample_output = chat.generate(inputs=input_ids, max_length=MAX_OUT_LEN)
+
+```
+## Demo output
+
+### text2img
+![text2img_demo](./output/text2img_demo.jpg)
+
+### img2img
+
+![text2img_demo](./output/img2img_input.jpg)
+
+![text2img_demo](./output/img2img_demo.jpg)
+
+
+
+## Environment
+
+- hardware: Nvidia Ampere architecture (A100) or compatable
+- docker image avaible: https://hub.docker.com/r/bigmoyan/lyra_aigc/tags
+```
+docker pull bigmoyan/lyra_aigc:v0.1
+```
+
+## Citation
+``` bibtex
+@Misc{lyraChatGLM2023,
+  author =       {Kangjian Wu, Zhengtao Wang, Bin Wu},
+  title =        {lyaraChatGLM: Accelerating chatGLM by 10x+},
+  howpublished = {\url{https://huggingface.co/TMElyralab/lyraChatGLM}},
+  year =         {2023}
+}
+```
+
+## Report bug
+- start a discussion to report any bugs!--> https://huggingface.co/TMElyralab/lyraChatGLM/discussions
+- report bug with a `[bug]` mark in the title.