Update README.md

#6
by Myric - opened
Files changed (1) hide show
  1. README.md +24 -0
README.md CHANGED
@@ -77,3 +77,27 @@ cd llama.cpp && cmake -B build && cmake --build build -j
77
  > contract this draft model needs, and upstream PR
78
  > [ggml-org/llama.cpp#25165](https://github.com/ggml-org/llama.cpp/pull/25165)
79
  > covers the target architecture only.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
77
  > contract this draft model needs, and upstream PR
78
  > [ggml-org/llama.cpp#25165](https://github.com/ggml-org/llama.cpp/pull/25165)
79
  > covers the target architecture only.
80
+
81
+ ### Production experience
82
+ I ran the i-quality version of this on my dgx-spark with a llama.cpp backend and a nice large context and
83
+ hooked it up to Opencode. I had it read the source of a moderately complex (19M of text) golang project I've
84
+ been working on. It took a while to read and summarize the entire project into context. I'd say it meandered
85
+ a little bit as I watched the reasoning process, but it inferred some pretty subtle details.
86
+
87
+ I then gave it a task to create a pair of systemd --user unit files for a pair of new models I had generated.
88
+ It needed to create the files in the proper format to be discovered by my parser, which was buried in the code.
89
+ It had to decide proper context sizes and model tuning parameters based on the model size and my available
90
+ system parameters. It figured out my system architecture autonomously, including the unified memory and cuda
91
+ architecture. It made educated guesses about the drafter settings (and got them right).
92
+
93
+ It determined that naming conventions and picked sane defaults for both models. It crafted and placed the
94
+ two units, ran the proper system tools to reread the units, ran the systemd verify command to check that the
95
+ format was correctly parsed by systemd.
96
+
97
+ It then wrote a little parser to double-check that the systemd units it had written parsed correctly and fixed
98
+ a bug it found in its parsing string. Once all that was done, it summarized all this for me and reported success.
99
+ If anything I would have preferred it asked for more clarification, but it completed the entire task autonomously.
100
+ Everything worked on the first try.
101
+
102
+ This is a capable large-context model natively trained to 1M tokens of context. I think this is the model I'll
103
+ use when I want to do something hard on my local machine that the fast little models get stuck.