Spaces:
Sleeping
Sleeping
Updated my answers in the README
Browse files
README.md
CHANGED
|
@@ -168,6 +168,17 @@ Simply put, this downloads the file as a temp file, we load it in with `TextFile
|
|
| 168 |
|
| 169 |
Why do we want to support streaming? What about streaming is important, or useful?
|
| 170 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 171 |
### On Chat Start:
|
| 172 |
|
| 173 |
The next scope is where "the magic happens". On Chat Start is when a user begins a chat session. This will happen whenever a user opens a new chat window, or refreshes an existing chat window.
|
|
@@ -210,6 +221,19 @@ Now, we'll save that into our user session!
|
|
| 210 |
|
| 211 |
Why are we using User Session here? What about Python makes us need to use this? Why not just store everything in a global variable?
|
| 212 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 213 |
### On Message
|
| 214 |
|
| 215 |
First, we load our chain from the user session:
|
|
|
|
| 168 |
|
| 169 |
Why do we want to support streaming? What about streaming is important, or useful?
|
| 170 |
|
| 171 |
+
|
| 172 |
+
#### 😎 ANSWER #1:
|
| 173 |
+
Streaming enables to :
|
| 174 |
+
1. **Improve user experience**
|
| 175 |
+
Users see the response appear progressively instead of waiting for everything to be generated, making the interaction more natural, like a real conversation.
|
| 176 |
+
2. **Perceive responsiveness**
|
| 177 |
+
Even if the total generation time remains the same, the immediate display of the first tokens gives the impression that the system is faster and more responsive.
|
| 178 |
+
3. **Improve memory efficiency**
|
| 179 |
+
Processing tokens as they are generated can be more memory-efficient, especially for long responses.
|
| 180 |
+
|
| 181 |
+
|
| 182 |
### On Chat Start:
|
| 183 |
|
| 184 |
The next scope is where "the magic happens". On Chat Start is when a user begins a chat session. This will happen whenever a user opens a new chat window, or refreshes an existing chat window.
|
|
|
|
| 221 |
|
| 222 |
Why are we using User Session here? What about Python makes us need to use this? Why not just store everything in a global variable?
|
| 223 |
|
| 224 |
+
#### 😎 ANSWER #2:
|
| 225 |
+
Using 'User Session' instead of global variables is necessary to:
|
| 226 |
+
|
| 227 |
+
1. **Support multi-user interactions**
|
| 228 |
+
In a web application, multiple users can be connected simultaneously. Global variables would be shared among all, causing interference, while 'User Session' isolates each user data.
|
| 229 |
+
2. **Manage concurrency**
|
| 230 |
+
Python handles global variables poorly in a concurrent environment. Web servers typically process multiple requests in parallel, and without proper synchronization, global variables can cause race conditions.
|
| 231 |
+
3. **Manage data lifecycle**
|
| 232 |
+
'User Session' properly manages the initialization and cleanup of resources, which would have to be done manually with global variables.
|
| 233 |
+
4. **Manage security**
|
| 234 |
+
Isolating user data prevents information leaks between users.
|
| 235 |
+
|
| 236 |
+
|
| 237 |
### On Message
|
| 238 |
|
| 239 |
First, we load our chain from the user session:
|