Commit
·
99ee4e5
1
Parent(s):
cbfc8da
Update README.md
Browse files
README.md
CHANGED
|
@@ -51,7 +51,7 @@ However, if you are not satisfied with only using the API, feel free to check ou
|
|
| 51 |
<br><br>
|
| 52 |
|
| 53 |
## Results
|
| 54 |
-
|
| 55 |
<table border="1" width="100%">
|
| 56 |
<tr align="center">
|
| 57 |
<th>Setup</th><th colspan="4">Zero-shot</th><th colspan="4">Finetune</th>
|
|
@@ -59,20 +59,20 @@ However, if you are not satisfied with only using the API, feel free to check ou
|
|
| 59 |
<tr align="center">
|
| 60 |
<td>Metric</td><td>R@1</td><td>R@5</td><td>R@10</td><td>MR</td><td>R@1</td><td>R@5</td><td>R@10</td><td>MR</td>
|
| 61 |
</tr>
|
| 62 |
-
<tr align="center">
|
| 63 |
-
<td>Wukong<sub>ViT-B</sub></td><td>33.4</td><td>59.3</td><td>69.7</td><td>54.1</td><td>39.2</td><td>66.9</td><td>77.4</td><td>61.2</td>
|
| 64 |
-
</tr>
|
| 65 |
<tr align="center">
|
| 66 |
-
<td>
|
| 67 |
-
|
| 68 |
<tr align="center">
|
| 69 |
-
<td>
|
| 70 |
-
|
|
|
|
|
|
|
|
|
|
| 71 |
</table>
|
|
|
|
| 72 |
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
<table border="1" width="100%">
|
| 76 |
<tr align="center">
|
| 77 |
<th>Task</th><th colspan="6">Text-to-Image</th><th colspan="6">Image-to-Text</th>
|
| 78 |
</tr>
|
|
@@ -82,19 +82,19 @@ However, if you are not satisfied with only using the API, feel free to check ou
|
|
| 82 |
<tr align="center">
|
| 83 |
<td>Metric</td><td>R@1</td><td>R@5</td><td>R@10</td><td>R@1</td><td>R@5</td><td>R@10</td><td>R@1</td><td>R@5</td><td>R@10</td><td>R@1</td><td>R@5</td><td>R@10</td>
|
| 84 |
</tr>
|
| 85 |
-
<tr align="center">
|
| 86 |
-
<td>Wukong<sub>ViT-B</sub></td><td>45.7</td><td>73.8</td><td>82.2</td><td>67.6</td><td>89.6</td><td>94.2</td><td>66.2</td><td>88.7</td><td>94.3</td><td>83.9</td><td>97.6</td><td>99.0</td>
|
| 87 |
-
</tr>
|
| 88 |
<tr align="center">
|
| 89 |
-
<td>
|
| 90 |
-
|
|
|
|
|
|
|
|
|
|
| 91 |
<tr align="center">
|
| 92 |
-
<td>CN-CLIP
|
| 93 |
-
|
| 94 |
</table>
|
|
|
|
| 95 |
|
| 96 |
-
|
| 97 |
-
### COCO-CN Retrieval
|
| 98 |
<table border="1" width="100%">
|
| 99 |
<tr align="center">
|
| 100 |
<th>Task</th><th colspan="6">Text-to-Image</th><th colspan="6">Image-to-Text</th>
|
|
@@ -105,18 +105,41 @@ However, if you are not satisfied with only using the API, feel free to check ou
|
|
| 105 |
<tr align="center">
|
| 106 |
<td>Metric</td><td>R@1</td><td>R@5</td><td>R@10</td><td>R@1</td><td>R@5</td><td>R@10</td><td>R@1</td><td>R@5</td><td>R@10</td><td>R@1</td><td>R@5</td><td>R@10</td>
|
| 107 |
</tr>
|
| 108 |
-
<tr align="center">
|
| 109 |
-
<td>Wukong<sub>ViT-B</sub></td><td>49.2</td><td>79.4</td><td>87.9</td><td>67.0</td><td>91.4</td><td>96.7</td><td>48.3</td><td>77.8</td><td>88.8</td><td>65.8</td><td>90.3</td><td>96.6</td>
|
| 110 |
-
</tr>
|
| 111 |
<tr align="center">
|
| 112 |
-
<td>
|
| 113 |
-
|
| 114 |
<tr align="center">
|
| 115 |
-
<td>
|
| 116 |
-
|
|
|
|
|
|
|
|
|
|
| 117 |
</table>
|
| 118 |
<br>
|
| 119 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 120 |
## Citation
|
| 121 |
If you find Chinese CLIP helpful, feel free to cite our paper. Thanks for your support!
|
| 122 |
|
|
|
|
| 51 |
<br><br>
|
| 52 |
|
| 53 |
## Results
|
| 54 |
+
**MUGE Text-to-Image Retrieval**:
|
| 55 |
<table border="1" width="100%">
|
| 56 |
<tr align="center">
|
| 57 |
<th>Setup</th><th colspan="4">Zero-shot</th><th colspan="4">Finetune</th>
|
|
|
|
| 59 |
<tr align="center">
|
| 60 |
<td>Metric</td><td>R@1</td><td>R@5</td><td>R@10</td><td>MR</td><td>R@1</td><td>R@5</td><td>R@10</td><td>MR</td>
|
| 61 |
</tr>
|
|
|
|
|
|
|
|
|
|
| 62 |
<tr align="center">
|
| 63 |
+
<td width="120%">Wukong</td><td>42.7</td><td>69.0</td><td>78.0</td><td>63.2</td><td>52.7</td><td>77.9</td><td>85.6</td><td>72.1</td>
|
| 64 |
+
</tr>
|
| 65 |
<tr align="center">
|
| 66 |
+
<td width="120%">R2D2</td><td>49.5</td><td>75.7</td><td>83.2</td><td>69.5</td><td>60.1</td><td>82.9</td><td>89.4</td><td>77.5</td>
|
| 67 |
+
</tr>
|
| 68 |
+
<tr align="center">
|
| 69 |
+
<td width="120%">CN-CLIP</td><td>63.0</td><td>84.1</td><td>89.2</td><td>78.8</td><td>68.9</td><td>88.7</td><td>93.1</td><td>83.6</td>
|
| 70 |
+
</tr>
|
| 71 |
</table>
|
| 72 |
+
<br>
|
| 73 |
|
| 74 |
+
**Flickr30K-CN Retrieval**:
|
| 75 |
+
<table border="1" width="120%">
|
|
|
|
| 76 |
<tr align="center">
|
| 77 |
<th>Task</th><th colspan="6">Text-to-Image</th><th colspan="6">Image-to-Text</th>
|
| 78 |
</tr>
|
|
|
|
| 82 |
<tr align="center">
|
| 83 |
<td>Metric</td><td>R@1</td><td>R@5</td><td>R@10</td><td>R@1</td><td>R@5</td><td>R@10</td><td>R@1</td><td>R@5</td><td>R@10</td><td>R@1</td><td>R@5</td><td>R@10</td>
|
| 84 |
</tr>
|
|
|
|
|
|
|
|
|
|
| 85 |
<tr align="center">
|
| 86 |
+
<td width="120%">Wukong</td><td>51.7</td><td>78.9</td><td>86.3</td><td>77.4</td><td>94.5</td><td>97.0</td><td>76.1</td><td>94.8</td><td>97.5</td><td>92.7</td><td>99.1</td><td>99.6</td>
|
| 87 |
+
</tr>
|
| 88 |
+
<tr align="center">
|
| 89 |
+
<td width="120%">R2D2</td><td>60.9</td><td>86.8</td><td>92.7</td><td>84.4</td><td>96.7</td><td>98.4</td><td>77.6</td><td>96.7</td><td>98.9</td><td>95.6</td><td>99.8</td><td>100.0</td>
|
| 90 |
+
</tr>
|
| 91 |
<tr align="center">
|
| 92 |
+
<td width="120%">CN-CLIP</td><td>71.2</td><td>91.4</td><td>95.5</td><td>83.8</td><td>96.9</td><td>98.6</td><td>81.6</td><td>97.5</td><td>98.8</td><td>95.3</td><td>99.7</td><td>100.0</td>
|
| 93 |
+
</tr>
|
| 94 |
</table>
|
| 95 |
+
<br>
|
| 96 |
|
| 97 |
+
**COCO-CN Retrieval**:
|
|
|
|
| 98 |
<table border="1" width="100%">
|
| 99 |
<tr align="center">
|
| 100 |
<th>Task</th><th colspan="6">Text-to-Image</th><th colspan="6">Image-to-Text</th>
|
|
|
|
| 105 |
<tr align="center">
|
| 106 |
<td>Metric</td><td>R@1</td><td>R@5</td><td>R@10</td><td>R@1</td><td>R@5</td><td>R@10</td><td>R@1</td><td>R@5</td><td>R@10</td><td>R@1</td><td>R@5</td><td>R@10</td>
|
| 107 |
</tr>
|
|
|
|
|
|
|
|
|
|
| 108 |
<tr align="center">
|
| 109 |
+
<td width="120%">Wukong</td><td>53.4</td><td>80.2</td><td>90.1</td><td>74.0</td><td>94.4</td><td>98.1</td><td>55.2</td><td>81.0</td><td>90.6</td><td>73.3</td><td>94.0</td><td>98.0</td>
|
| 110 |
+
</tr>
|
| 111 |
<tr align="center">
|
| 112 |
+
<td width="120%">R2D2</td><td>56.4</td><td>85.0</td><td>93.1</td><td>79.1</td><td>96.5</td><td>98.9</td><td>63.3</td><td>89.3</td><td>95.7</td><td>79.3</td><td>97.1</td><td>98.7</td>
|
| 113 |
+
</tr>
|
| 114 |
+
<tr align="center">
|
| 115 |
+
<td width="120%">CN-CLIP</td><td>69.2</td><td>89.9</td><td>96.1</td><td>81.5</td><td>96.9</td><td>99.1</td><td>63.0</td><td>86.6</td><td>92.9</td><td>83.5</td><td>97.3</td><td>99.2</td>
|
| 116 |
+
</tr>
|
| 117 |
</table>
|
| 118 |
<br>
|
| 119 |
|
| 120 |
+
**Zero-shot Image Classification**:
|
| 121 |
+
<table border="1" width="100%">
|
| 122 |
+
<tr align="center">
|
| 123 |
+
<th>Task</th><th>CIFAR10</th><th>CIFAR100</th><th>DTD</th><th>EuroSAT</th><th>FER</th><th>FGVC</th><th>KITTI</th><th>MNIST</th><th>PC</th><th>VOC</th>
|
| 124 |
+
</tr>
|
| 125 |
+
<tr align="center">
|
| 126 |
+
<td width="150%">GIT</td><td>88.5</td><td>61.1</td><td>42.9</td><td>43.4</td><td>41.4</td><td>6.7</td><td>22.1</td><td>68.9</td><td>50.0</td><td>80.2</td>
|
| 127 |
+
</tr>
|
| 128 |
+
<tr align="center">
|
| 129 |
+
<td width="150%">ALIGN</td><td>94.9</td><td>76.8</td><td>66.1</td><td>52.1</td><td>50.8</td><td>25.0</td><td>41.2</td><td>74.0</td><td>55.2</td><td>83.0</td>
|
| 130 |
+
</tr>
|
| 131 |
+
<tr align="center">
|
| 132 |
+
<td width="150%">CLIP</td><td>94.9</td><td>77.0</td><td>56.0</td><td>63.0</td><td>48.3</td><td>33.3</td><td>11.5</td><td>79.0</td><td>62.3</td><td>84.0</td>
|
| 133 |
+
</tr>
|
| 134 |
+
<tr align="center">
|
| 135 |
+
<td width="150%">Wukong</td><td>95.4</td><td>77.1</td><td>40.9</td><td>50.3</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td>
|
| 136 |
+
</tr>
|
| 137 |
+
<tr align="center">
|
| 138 |
+
<td width="150%">CN-CLIP</td><td>96.0</td><td>79.7</td><td>51.2</td><td>52.0</td><td>55.1</td><td>26.2</td><td>49.9</td><td>79.4</td><td>63.5</td><td>84.9</td>
|
| 139 |
+
</tr>
|
| 140 |
+
</table>
|
| 141 |
+
<br><br>
|
| 142 |
+
|
| 143 |
## Citation
|
| 144 |
If you find Chinese CLIP helpful, feel free to cite our paper. Thanks for your support!
|
| 145 |
|