Upload 2 files
Browse files- data/processed/flattened_docs.jsonl +0 -0
- requirements.txt +12 -0
data/processed/flattened_docs.jsonl
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
requirements.txt
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
torch>=2.0
|
| 2 |
+
transformers>=4.30
|
| 3 |
+
sentence-transformers>=2.2
|
| 4 |
+
faiss-cpu>=1.7.4 # or faiss-gpu if you have CUDA and want GPU speed
|
| 5 |
+
PyPDF2>=3.0.0
|
| 6 |
+
tqdm
|
| 7 |
+
peft
|
| 8 |
+
datasets
|
| 9 |
+
streamlit
|
| 10 |
+
bitsandbytes # for 4-bit quantization (GPU speedup)
|
| 11 |
+
accelerate # for optimized model loading
|
| 12 |
+
pypdf # updated PDF reader
|