Julia Klee commited on
Commit ·
b699db2
1
Parent(s): 376b4b0
Card: comparison without empty group rows or rows no model can do
Browse files
README.md
CHANGED
|
@@ -316,30 +316,24 @@ What each model can understand and do, side by side:
|
|
| 316 |
|
| 317 |
| | Julia 1 | Julia 1.5 | Julia 2 |
|
| 318 |
|---|---|---|---|
|
| 319 |
-
| **Reading the situation** | | | |
|
| 320 |
| Short text or JSON state | Yes | Yes | Yes |
|
| 321 |
| Long states (thousands to tens of thousands of tokens) | Partly: runtime 8k, tuned on 1k | No: 1,024 tokens | Yes |
|
| 322 |
| A state that keeps growing (live input) | No | No | Yes |
|
| 323 |
| Images in the state | No | No | Yes: zero-shot |
|
| 324 |
| Speech and live calls | No | No | Next checkpoint |
|
| 325 |
-
| **Understanding the question** | | | |
|
| 326 |
| Written rules and policies in the state | No | Yes | Yes |
|
| 327 |
| Negated questions ("is it NOT eligible?") | No | Partly: trained phrasings only | Yes |
|
| 328 |
| Numbers compared correctly (8,143 vs 8,134) | No | No | Yes |
|
| 329 |
| Yes/no answers that depend on the state, not a default | Partly | Partly | Yes |
|
| 330 |
| General knowledge to fill gaps | No | No | Partly |
|
| 331 |
-
| Multi-step arithmetic | No | No | No (compute upstream) |
|
| 332 |
-
| **Understanding the options** | | | |
|
| 333 |
| Option descriptions read as meaning | Partly | Partly | Yes |
|
| 334 |
| New label sets and renamed keys without retraining | Partly | Partly | Yes |
|
| 335 |
| Same answer when options are reordered | No | Partly (fully with runtime views) | Yes: in the weights |
|
| 336 |
| Ignoring an irrelevant option slipped into the list | Partly | Yes | Yes |
|
| 337 |
| More than 20 options in one question | Partly: grouped router | No | Yes: up to 255, more in two rounds |
|
| 338 |
-
| **Answering** | | | |
|
| 339 |
| Choice, score and yes/no | Yes | Yes | Yes |
|
| 340 |
| Several questions on one state | One pass each | one pass each | One reading, short branches |
|
| 341 |
| Calibrated probabilities | Partly | Partly | Yes |
|
| 342 |
-
| Text generation | None | None | None |
|
| 343 |
| Same request and answer format | Yes | Yes | Yes |
|
| 344 |
|
| 345 |
## Live inputs and audio
|
|
|
|
| 316 |
|
| 317 |
| | Julia 1 | Julia 1.5 | Julia 2 |
|
| 318 |
|---|---|---|---|
|
|
|
|
| 319 |
| Short text or JSON state | Yes | Yes | Yes |
|
| 320 |
| Long states (thousands to tens of thousands of tokens) | Partly: runtime 8k, tuned on 1k | No: 1,024 tokens | Yes |
|
| 321 |
| A state that keeps growing (live input) | No | No | Yes |
|
| 322 |
| Images in the state | No | No | Yes: zero-shot |
|
| 323 |
| Speech and live calls | No | No | Next checkpoint |
|
|
|
|
| 324 |
| Written rules and policies in the state | No | Yes | Yes |
|
| 325 |
| Negated questions ("is it NOT eligible?") | No | Partly: trained phrasings only | Yes |
|
| 326 |
| Numbers compared correctly (8,143 vs 8,134) | No | No | Yes |
|
| 327 |
| Yes/no answers that depend on the state, not a default | Partly | Partly | Yes |
|
| 328 |
| General knowledge to fill gaps | No | No | Partly |
|
|
|
|
|
|
|
| 329 |
| Option descriptions read as meaning | Partly | Partly | Yes |
|
| 330 |
| New label sets and renamed keys without retraining | Partly | Partly | Yes |
|
| 331 |
| Same answer when options are reordered | No | Partly (fully with runtime views) | Yes: in the weights |
|
| 332 |
| Ignoring an irrelevant option slipped into the list | Partly | Yes | Yes |
|
| 333 |
| More than 20 options in one question | Partly: grouped router | No | Yes: up to 255, more in two rounds |
|
|
|
|
| 334 |
| Choice, score and yes/no | Yes | Yes | Yes |
|
| 335 |
| Several questions on one state | One pass each | one pass each | One reading, short branches |
|
| 336 |
| Calibrated probabilities | Partly | Partly | Yes |
|
|
|
|
| 337 |
| Same request and answer format | Yes | Yes | Yes |
|
| 338 |
|
| 339 |
## Live inputs and audio
|