librepower commited on
Commit
9f3b322
·
verified ·
1 Parent(s): 15e27d6

Add README documentation

Browse files
Files changed (1) hide show
  1. README.md +81 -0
README.md ADDED
@@ -0,0 +1,81 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - llama
5
+ - gguf
6
+ - big-endian
7
+ - aix
8
+ - power
9
+ - librepower
10
+ library_name: llama.cpp
11
+ ---
12
+
13
+ # TinyLlama 1.1B Chat - Big-Endian GGUF
14
+
15
+ This is a **big-endian** version of [TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0) in GGUF format, optimized for **IBM AIX on POWER architecture**.
16
+
17
+ ## Model Details
18
+
19
+ - **Base Model**: TinyLlama 1.1B Chat v1.0
20
+ - **Format**: GGUF (Q4_K_M quantization)
21
+ - **Endianness**: Big-endian (IBM Power Systems)
22
+ - **Size**: 638 MB
23
+ - **License**: Apache 2.0
24
+
25
+ ## Usage
26
+
27
+ This model is designed for use with [llama-aix](https://gitlab.com/librepower/llama-aix), a port of llama.cpp for IBM AIX.
28
+
29
+ ```bash
30
+ # Download model
31
+ wget https://huggingface.co/librepower/tinyllama-1.1b-chat-be/resolve/main/tinyllama-1.1b-q4_k_m-be.gguf
32
+
33
+ # Run inference on AIX
34
+ ./llama-simple -m tinyllama-1.1b-q4_k_m-be.gguf -n 128 -p "Hello, world!"
35
+ ```
36
+
37
+ ## Performance
38
+
39
+ On IBM POWER9 (16 cores, 128GB RAM):
40
+ - **Speed**: ~18 tokens/second
41
+ - **Memory**: ~800 MB RAM
42
+
43
+ ## Why Big-Endian?
44
+
45
+ IBM Power Systems use big-endian byte order, while most modern systems use little-endian. This model has been converted using llama.cpp's endianness conversion tool to run natively on AIX without runtime conversion overhead.
46
+
47
+ ## Conversion
48
+
49
+ This model was converted from the original little-endian GGUF using:
50
+
51
+ ```bash
52
+ ./llama-gguf-split --convert-be model.gguf model-be.gguf
53
+ ```
54
+
55
+ ## About LibrePower
56
+
57
+ **Unlocking Power Systems through open source.**
58
+
59
+ LibrePower brings modern AI and open-source tools to IBM Power Systems, extending the life and capabilities of enterprise infrastructure.
60
+
61
+ - Web: https://librepower.org
62
+ - GitLab: https://gitlab.com/librepower
63
+ - Newsletter: https://librepower.substack.com
64
+
65
+ ## Related Projects
66
+
67
+ - [llama-aix](https://gitlab.com/librepower/llama-aix) - llama.cpp for IBM AIX
68
+ - [redbook-explorer](https://gitlab.com/librepower/redbook-explorer) - RAG application for IBM Redbooks
69
+
70
+ ## Citation
71
+
72
+ Original model by Zhang et al. (TinyLlama team):
73
+
74
+ ```bibtex
75
+ @article{tinyllama,
76
+ title={TinyLlama: An Open-Source Small Language Model},
77
+ author={Zhang, Peiyuan and Guangtao, Zeng and Wang, Tianduo and Lu, Wei},
78
+ journal={arXiv preprint arXiv:2401.02385},
79
+ year={2024}
80
+ }
81
+ ```