Julia Klee commited on
Commit
b699db2
·
1 Parent(s): 376b4b0

Card: comparison without empty group rows or rows no model can do

Browse files
Files changed (1) hide show
  1. README.md +0 -6
README.md CHANGED
@@ -316,30 +316,24 @@ What each model can understand and do, side by side:
316
 
317
  | | Julia 1 | Julia 1.5 | Julia 2 |
318
  |---|---|---|---|
319
- | **Reading the situation** | | | |
320
  | Short text or JSON state | Yes | Yes | Yes |
321
  | Long states (thousands to tens of thousands of tokens) | Partly: runtime 8k, tuned on 1k | No: 1,024 tokens | Yes |
322
  | A state that keeps growing (live input) | No | No | Yes |
323
  | Images in the state | No | No | Yes: zero-shot |
324
  | Speech and live calls | No | No | Next checkpoint |
325
- | **Understanding the question** | | | |
326
  | Written rules and policies in the state | No | Yes | Yes |
327
  | Negated questions ("is it NOT eligible?") | No | Partly: trained phrasings only | Yes |
328
  | Numbers compared correctly (8,143 vs 8,134) | No | No | Yes |
329
  | Yes/no answers that depend on the state, not a default | Partly | Partly | Yes |
330
  | General knowledge to fill gaps | No | No | Partly |
331
- | Multi-step arithmetic | No | No | No (compute upstream) |
332
- | **Understanding the options** | | | |
333
  | Option descriptions read as meaning | Partly | Partly | Yes |
334
  | New label sets and renamed keys without retraining | Partly | Partly | Yes |
335
  | Same answer when options are reordered | No | Partly (fully with runtime views) | Yes: in the weights |
336
  | Ignoring an irrelevant option slipped into the list | Partly | Yes | Yes |
337
  | More than 20 options in one question | Partly: grouped router | No | Yes: up to 255, more in two rounds |
338
- | **Answering** | | | |
339
  | Choice, score and yes/no | Yes | Yes | Yes |
340
  | Several questions on one state | One pass each | one pass each | One reading, short branches |
341
  | Calibrated probabilities | Partly | Partly | Yes |
342
- | Text generation | None | None | None |
343
  | Same request and answer format | Yes | Yes | Yes |
344
 
345
  ## Live inputs and audio
 
316
 
317
  | | Julia 1 | Julia 1.5 | Julia 2 |
318
  |---|---|---|---|
 
319
  | Short text or JSON state | Yes | Yes | Yes |
320
  | Long states (thousands to tens of thousands of tokens) | Partly: runtime 8k, tuned on 1k | No: 1,024 tokens | Yes |
321
  | A state that keeps growing (live input) | No | No | Yes |
322
  | Images in the state | No | No | Yes: zero-shot |
323
  | Speech and live calls | No | No | Next checkpoint |
 
324
  | Written rules and policies in the state | No | Yes | Yes |
325
  | Negated questions ("is it NOT eligible?") | No | Partly: trained phrasings only | Yes |
326
  | Numbers compared correctly (8,143 vs 8,134) | No | No | Yes |
327
  | Yes/no answers that depend on the state, not a default | Partly | Partly | Yes |
328
  | General knowledge to fill gaps | No | No | Partly |
 
 
329
  | Option descriptions read as meaning | Partly | Partly | Yes |
330
  | New label sets and renamed keys without retraining | Partly | Partly | Yes |
331
  | Same answer when options are reordered | No | Partly (fully with runtime views) | Yes: in the weights |
332
  | Ignoring an irrelevant option slipped into the list | Partly | Yes | Yes |
333
  | More than 20 options in one question | Partly: grouped router | No | Yes: up to 255, more in two rounds |
 
334
  | Choice, score and yes/no | Yes | Yes | Yes |
335
  | Several questions on one state | One pass each | one pass each | One reading, short branches |
336
  | Calibrated probabilities | Partly | Partly | Yes |
 
337
  | Same request and answer format | Yes | Yes | Yes |
338
 
339
  ## Live inputs and audio