PCIResearch commited on
Commit
3e0cec4
·
1 Parent(s): e885ab3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +50 -1
README.md CHANGED
@@ -1,4 +1,53 @@
1
  ---
2
  language:
3
  - en
4
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  language:
3
  - en
4
+ ---
5
+
6
+ # PC_TransCore_MM_Vis_Infer
7
+
8
+ Details and codes will be released soon. Stay tuned!
9
+
10
+ ![](./figure/Picture1.png)
11
+
12
+ <center>TransCoreM model architecture</center>
13
+
14
+
15
+ # Overview
16
+ PCI Trans-Core-M is a multimodal model, it's composed of a visual encoder and a language model. It enables content comprehension, recognition and multi-round conversations about pictures.
17
+
18
+ - **Visual Encoder:** CLIP ViT-L/14
19
+ - **Pre-trained LLM:** TansCore-M
20
+
21
+ # Install
22
+ Our install environment:NVIDIA A800-SXM4-80GB,CUDA Version: 11.7
23
+
24
+ Pull project and install some library:
25
+ ```shell
26
+ git clone http://10.200.103.133/llm2/trans_core-llm/trans-core-mllm/pc_transcore_mm_vis_infer.git
27
+ cd pc_transcore_mm_vis_infer
28
+ conda create -n transcorem python=3.10 -y
29
+ conda activate transcorem
30
+
31
+ pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
32
+ ```
33
+
34
+ # Inference
35
+ ```shell
36
+ cd PC_TransCore_MM_Vis_Infer
37
+ CUDA_VISIBLE_DEVICES=0 python inference.py --model-path ./weights/transcorem_pci-v1.1_20231112 --vision-path ./weights/clip-vit-large-patch14-336
38
+
39
+ Note:
40
+ The default setting in our code for the vision encoder model is to actively download openai/clip-vit-large-patch14-336.
41
+ But due to the slow network speed to download vision encoder model by default, you can download the vision encoder in advance and then specify the path to the clip model in the terminal where the python script is started.
42
+ Our vision encoder model: clip-vit-large-patch14-336, you can download from https://huggingface.co/openai/clip-vit-large-patch14-336
43
+ ```
44
+
45
+ # Demo
46
+ demo: [demo_link](http://10.210.21.107:7860/)
47
+
48
+
49
+ # 声明
50
+ PCI Trans-Core-M 模型旨在提升技术领域的研究和应用效率,我们
51
+ 鼓励和欢迎开发者及社区成员共同遵守相关开源协议,确保技术的正当和安全使用。请广大用户注意,不得将此模型及其衍生代码用于任何可能对国家安全和社会稳定造成威胁的活动,以及未经过严格安全评估和官方备案的服务。
52
+
53
+ PCI Trans-Core-M 模型在设计和训练过程中尽量保证数据的合规性和准确性,但由于其复杂性和概率性的特点,我们无法完全保证输出内容的百分百准确无误。此外,模型输出结果可能受到用户输入内容的影响而产生偏差。因此,我们不对由于使用本模型而引起的任何数据安全问题、舆论风险、误导信息的传播或任何形式的不当使用承担责任。