Upload edit\takes_packed.md with huggingface_hub

edit//takes_packed.md

@@ -0,0 +1,16 @@
+# Packed transcripts
+
+Phrase-level, grouped on silences ≥ 0.5s or speaker change.
+Use `[start-end]` ranges to address cuts in the EDL.
+
+## LMSTUDIO-MTP1  (duration: 6m 33.5s, 10 phrases)
+  [000.16-192.22] S0 If you use LM Studio, you're gonna love this. So, there is a concept known as MTP, and you can see without MTP, we have about 62 tokens per second, and with MTP, we can see that we have this increase of 78 tokens per second. Now, I'm going to show you how to do this, how to switch that setting to unlock that full speed. But before that, let's go ahead and understand what we're talking about. So, we are talking about this concept known as multi-token prediction, MTP. Now, I have created two videos on MTP using LLaMA CPP. You can go ahead and check this out. I will create more such videos on MTP. But basically, MTP unlocks that speed in LLaMA CPP. And as we know, LM Studio has been made on top of LLaMA CPP. Therefore, LM Studio has that unlock feature as well. So, what happens on the left, we see the standard autoregression, is that when we put the context to the 27 billion models and it predicts the token "the", that is pass one. Then we send the context again, plus this word, or token "the" through the 27 billion model again, and then we get this model as prediction, token model as prediction. Then in the pass three, we get "runs" as prediction. So, three tokens, three full passes, and every token pays the full 27 billion cost, if we work with a 27 billion model. And we know this is very slow. This cannot be parallelized because, because the input of one depends on the output of the previous one. So, that is what the standard autoregression is, and this is what, this is how an LLM proceeds. But when we have MTP enabled, then what happens is that we create some drafts here, and then we verify, and then we get some increase in speed. So, again, we have the same context, it passes through the 27 billion model. It gives out the word "the" or the, the token "the" here. But while this is running, while the 27 billion model is running, then at the same time we have attached an MTP head, that within the same pass, it gives out two, three, or whatever you choose, number of drafts tokens, which are speculative guesses from the MTP head. And this is really fast because it's all within the same pass. And then we have the verify phase in which the same pass, within the same forward pass, it checks everything. So, one pass, it checks, uh, whether the model is acceptable, the runs is acceptable, and fast is acceptable. So, for example, in this case, these two are accepted, model and runs, and the fast is rejected. Having done this, you can see that in the same pass, which is the draw phase and the verify, within the same pass, in this example, we get three tokens. The first is the "the", then the model, and then the runs, okay? While on the standard we see that we need, uh, three pass. Here, on the same pass we get this. If you average out everything in all the generations, uh, people have seen that we get, uh, a 2.5 token per verified pass versus one in the standard. So, that's roughly two times faster, and, uh, the acceptance rate is about 75% to 80% on Q&3, and we need to give about 1 GB VRAM as cost. And this is how the MTP is able to solve or increase the speed. There are other such techniques to increase the speed, but this is just one of them. Now, you can go ahead and watch these two videos to understand it better, to understand it on LLaMA CPP. But in this video, I'm going to show how to do it on LM Studio. Now, we first need to install LM Studio on our computer. So, go to LM Studio, go to download for Windows. This is going to download the EXE file, and then you are going to install this. So, once we install LM Studio, it will open something like this here. Now, in a normal version, uh, you won't be able to get MTP,
+  [192.74-195.85] S0 at least at this point of time. When you have Stable here,
+  [196.48-199.98] S0 and then you go to check for updates, uh, you will be somewhere on the
+  [200.78-247.36] S0 04.13. And if you want the MTP, you're going to go to the beta here, and then click for, check for updates, and then you will get that 04.14 version, and you install that version, okay? So, in the normal version you won't get, but on these beta releases, or you can go ahead and download this beta release 04.14 and install this. This will do as well. Okay, so let's go ahead and download some models here. So, if you go to this model search tab, uh, you search for MTP-enabled models, and then you select a model which is suitable for your device. Now, for my laptop, which has an 8GB GPU, I'm gonna go with this 4 billion MTP-GTUF, and I've already downloaded this. So, go to Use & New Chat, or I can go to this chat here and go to Select A New Model, and I'm gonna load this model from unsloth, uh,
+  [247.92-268.62] S0 23.54 billion. I'm gonna click here, and then you can see, uh, we have this MTP Speculative Decoding here. I'm gonna switch this off as of now, and then load this model. And when this is loading, as you can see, uh, we are loading this model. While this is loading, I'm gonna put this question of, "What is the capital of India and Japan?" And once this is loaded, I'm gonna fire this up.
+  [270.00-271.18] S0 So, you can see, it's thinking.
+  [273.56-315.30] S0 Thought for 11 seconds, and the output is 17 or about 18 tokens per second, okay? Next, we are going to eject this model, and then take the same model again, but this time, we go down and use MTP Speculative Decoding by using the draft tokens of three, which means, uh, we are putting out three number of draft tokens, as we have seen in this explainer. So, there are three number of draft tokens that is output here. And then the minimum is zero, and the maximum is three, okay? So, we load this model now, and then I'll go to a new chat, copy this same, and go to a new chat. And when this is loaded fully, I'm gonna go ahead and fire this up. So, the earlier was 18 tokens per second. Let's see the token count now.
+  [317.00-335.39] S0 So, you can see, 20 tokens per second. That's good, which is about 11% increase in the speed. Now, uh, some of the GPU of my device has been used by the recording instrument as well. Uh, I was trying out some examples earlier. I was trying this. You can see, uh, "Write me a 500-word essay on AI,"
+  [336.04-354.22] S0 and you can see the speed that I got was 62 tokens per second, okay? And, uh, w- this is without MTP. And with MTP, what I obtained was 78 tokens per second, so 62 and 78, and that's, again, roughly 78 by 62. That's, again, roughly a 25% increase in the speed.
+  [354.72-393.69] S0 So, go ahead and try this. You can expect, uh, at least for me, something in the range of 10 to 25% increase in your local LLM speed, which is really great, again. So, go ahead and use this. And if you're looking for good GPUs, excellent GPUs, you can go ahead and see RunPod. Link's in the description. If you're trying to set up your entire business on your server or try out VPS, you can go ahead and check out Hostinger. So, these two are the tools, basically, I use. GPUs, I use RunPods. CPUs, 247 services. Hosting, uh, I use Hostinger. So, go ahead and check out. Links are in the description, and I will see you in the next one. Subscribe to my channel.