Update README.md
Browse files
README.md
CHANGED
|
@@ -1,12 +1,14 @@
|
|
| 1 |
## Performance
|
| 2 |
|
|
|
|
|
|
|
| 3 |
This model has reached 75.12\%(*12\% better than previous version*)/74.98\%(*8.5\% better than previous version*) on Q-Bench A1 *dev/test* (multi-choice questions).
|
| 4 |
|
| 5 |
It also outperforms the following close-source models with much larger model capacities:
|
| 6 |
|
| 7 |
| Model | *dev* | *test* |
|
| 8 |
| ---- | ---- | ---- |
|
| 9 |
-
| Co-Instruct-Preview (mPLUG-Owl2) | **75.12\%** | **74.98\%** |
|
| 10 |
| \*GPT-4V-Turbo | 74.41\% | 74.10\% |
|
| 11 |
| \*Qwen-VL-**Max** | 73.63\% | 73.90\% |
|
| 12 |
| \*GPT-4V (Nov. 2023) | 71.78\% | 73.44\% |
|
|
@@ -17,6 +19,14 @@ It also outperforms the following close-source models with much larger model cap
|
|
| 17 |
|
| 18 |
\*: Proprietary Models.
|
| 19 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
We are also constructing multi-image benchmark sets (image pairs, triple-quadruple images), and the results on multi-image benchmarks will be released soon!
|
| 21 |
|
| 22 |
## Load Model
|
|
|
|
| 1 |
## Performance
|
| 2 |
|
| 3 |
+
### Low-level Question-Answering
|
| 4 |
+
|
| 5 |
This model has reached 75.12\%(*12\% better than previous version*)/74.98\%(*8.5\% better than previous version*) on Q-Bench A1 *dev/test* (multi-choice questions).
|
| 6 |
|
| 7 |
It also outperforms the following close-source models with much larger model capacities:
|
| 8 |
|
| 9 |
| Model | *dev* | *test* |
|
| 10 |
| ---- | ---- | ---- |
|
| 11 |
+
| **Co-Instruct-Preview (mPLUG-Owl2) (This Model)** | **75.12\%** | **74.98\%** |
|
| 12 |
| \*GPT-4V-Turbo | 74.41\% | 74.10\% |
|
| 13 |
| \*Qwen-VL-**Max** | 73.63\% | 73.90\% |
|
| 14 |
| \*GPT-4V (Nov. 2023) | 71.78\% | 73.44\% |
|
|
|
|
| 19 |
|
| 20 |
\*: Proprietary Models.
|
| 21 |
|
| 22 |
+
#### Image/Video Quality Assessment
|
| 23 |
+
|
| 24 |
+
| Model | live | agi | livec | test_spaq | csiq | test_kadid | test_koniq | konvid | maxwell_test |
|
| 25 |
+
|--------------------------|--------------|--------------|-------------|-------------|-------------|-------------|-------------|-------------|--------------|
|
| 26 |
+
|**Co-Instruct-Preview (mPLUG-Owl2) (This Model)** | **0.771/0.751** | **0.727/0.749** | **0.861/0.865** | **0.946/0.938** | **0.735/0.748** | **0.782/0.770** | **0.908/0.941** | **0.818/0.790** | **0.735/0.714** |
|
| 27 |
+
| Q-Instruct (mPLUG-Owl2, Nov. 2023) | 0.749/0.747 | 0.710/0.753 | 0.781/0.791 | 0.921/0.917 | 0.693/0.723 | 0.670/0.665 | 0.904/0.921 | 0.766/0.738 | 0.650/0.649 |
|
| 28 |
+
|
| 29 |
+
|
| 30 |
We are also constructing multi-image benchmark sets (image pairs, triple-quadruple images), and the results on multi-image benchmarks will be released soon!
|
| 31 |
|
| 32 |
## Load Model
|