Question: What is the alternative to lots of LLM calls in a database tool implementation?

Instead of looking things up in a Python dictionary, you can use SQL Lite as part of the UV project to do an actual database query. You import sqlite3, create a connection, and write SQL queries like SELECT price FROM prices WHERE city = ?, using parameter substitution to avoid SQL injection attacks.

the easiest way to do this without needing you to do lots of configuration and set things up is to use SQL Lite, which is so quick and easy and just part of the UV project, and you can just use it. So we can just import Sqlite3 and we can write some database queries. If you'd rather build something a bit more industrial strength, you should feel free to replace these SQL queries with something like a Postgres lookup or something like that.


Question: What is an end point in the context of API calls?

endpoint
http url
hitting some web address

An endpoint is an HTTP URL which you can call to make an API request by hitting some web address. That web address is known as an endpoint. There's nothing special about it — it's just a way to have an API exposed at a particular URL.

It's one of those words people throw around all the time. But you might not. And for people that don't know about it, you should quickly find out. I explain it all in the Technical Foundations guide. There's nothing special at endpoint. It's an http url which you can call to, to to to make an API request by hitting some web address. And that web address would be known as an endpoint. It's a way that you have an API.


Question: What does the chinchilla scaling concept imply about model performance from diminishing returns?

8 billion parameters
halfway through training
twice as many parameters

If you're training an 8 billion parameter model and find diminishing returns halfway through your training data, with more data to go, that's a sign you need to double the number of parameters. You will need twice as many parameters to absorb twice as much training data effectively.

imagine that you're trying to train a model with, with with 8 billion parameters, and you start to find that you're let's say you're halfway through training your training data and you're seeing that you're getting diminishing returns. Your model's not doing any better. It seems to be sort of stuck at the halfway point in your training data, and you've got like, like double as much to go. Then what you have to think is, okay, that's my sign that I need to double the number of parameters. I will need twice as many parameters in my neural network to be able to take on twice as much training data


Question: What does the alpha hyperparameter do in LoRA?

Alpha in LoRA controls how much you scale up the matrix product of LoRA A and LoRA B before you add it to the base model. The convention is to set alpha as twice the value of R — that's what people typically do.

alpha is once you've done this multiplied Laura A and Laura B, and you're multiplying two matrices, which you may remember from linear algebra, you multiply two matrices like that, you get one bigger one, uh, and as a result, uh, alpha is how much you scale that up by before you add it to the base model. Uh, and it's, it's typically, it's always a thing that you just double R that's what you do.


Question: Why might a higher dropout value take longer to train?

higher values of dropout
longer to train
generalize

Higher values of dropout tend to mean you can generalize better, but they take much longer to train because the whole neural network has to learn to respond to different inputs rather than letting localized parts of the network focus on specific input types. The randomness forces more general learning.

And higher values of dropout tend to mean that you can generalize better. But of course, as you could probably guess, higher values of dropout means it takes much longer to train because you're doing this, this thing of removing bits. So it's got to learn in some way. The whole neural network has got to be able to learn how to respond to different inputs, rather than allowing it just to cheat and just kind of have different parts of the neural network be focused on different types of input.


Question: What is an epoch in machine learning training?

An epoch is a complete pass through all of your data. You often do multiple epochs because at each step you're only taking a small step in the right direction. Within each epoch, data is randomly selected into batches, so different batches are used each time across epochs.

So epochs we covered last week and epoch of course, is a complete pass through all of your data. And you often do multiple epochs because you might think, but why are we putting in the same input data again? Remember in this in this cycle you're only you're finding the gradients and you're only doing a small step in the right direction. So it can make sense to make multiple steps as you get better and better. There's another reason why epochs make sense. Remember that we don't we don't pass data through one data point at a time. We batch up our data into batches and those batches every time we start an epoch, the batches are randomly selected.