Spaces:
Sleeping
Sleeping
File size: 6,839 Bytes
c0f74f5 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 | {
"cells": [
{
"cell_type": "code",
"execution_count": 4,
"id": "f6960caa",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[Document(metadata={'source': 'c:\\\\Users\\\\xyzai\\\\Documents\\\\GraphDB\\\\FinalProject\\\\data\\\\pdfs\\\\RU-MILITARY.pdf', 'creationdate': '2014-04-22T10:23:16+02:00', 'creator': 'Adobe InDesign CS6 (Macintosh)', 'trapped': '/False', 'page_label': '3', 'total_pages': 4, 'moddate': '2014-04-22T10:23:18+02:00', 'page': 2, 'producer': 'Adobe PDF Library 10.0.1'}, page_content='developing the military-industrial complex.\\nThe Kremlin’s massive armaments program \\nand its reform of the military-industrial \\ncomplex also has significance in terms of \\nindustrial and social policy. The military in-\\ndustry employs two million workers; five \\nper cent of the Russian population depend \\non it for their livelihood. In this way, the \\nKremlin is “solving” an issue in Soviet style: \\nFunding for the military is once more tak -\\ning on a central role in society. It is hoped \\nthat this will boost innovation and global \\nRussian Demographics\\nRussia has a security apparatus \\nproportionally more than twice \\nthe size of that of the US.'),\n",
" Document(metadata={'creator': 'Adobe InDesign CS6 (Macintosh)', 'creationdate': '2014-04-22T10:23:16+02:00', 'source': 'c:\\\\Users\\\\xyzai\\\\Documents\\\\GraphDB\\\\FinalProject\\\\data\\\\pdfs\\\\RU-MILITARY.pdf', 'producer': 'Adobe PDF Library 10.0.1', 'moddate': '2014-04-22T10:23:18+02:00', 'total_pages': 4, 'page_label': '3', 'page': 2, 'trapped': '/False'}, page_content='developing the military-industrial complex.\\nThe Kremlin’s massive armaments program \\nand its reform of the military-industrial \\ncomplex also has significance in terms of \\nindustrial and social policy. The military in-\\ndustry employs two million workers; five \\nper cent of the Russian population depend \\non it for their livelihood. In this way, the \\nKremlin is “solving” an issue in Soviet style: \\nFunding for the military is once more tak -\\ning on a central role in society. It is hoped \\nthat this will boost innovation and global \\nRussian Demographics\\nRussia has a security apparatus \\nproportionally more than twice \\nthe size of that of the US.'),\n",
" Document(metadata={'producer': 'Adobe PDF Library 10.0.1', 'source': 'c:\\\\Users\\\\xyzai\\\\Documents\\\\GraphDB\\\\FinalProject\\\\data\\\\pdfs\\\\RU-MILITARY.pdf', 'total_pages': 4, 'trapped': '/False', 'creationdate': '2014-04-22T10:23:16+02:00', 'creator': 'Adobe InDesign CS6 (Macintosh)', 'page_label': '4', 'moddate': '2014-04-22T10:23:18+02:00', 'page': 3}, page_content='there are bottlenecks in production capaci-\\nty, for example in aircraft production and \\nshipbuilding. The plans to enhance military \\ntransport aviation can only be realized if \\ncapacity is expanded rapidly. Russia also \\ndepends on cooperation with Ukraine: So \\nfar, many motors for helicopters and air -\\ncraft as well as rockets have been produced \\nin Ukraine. Russia lacks the know-how for \\nproducing many of the parts required. The \\ncurrent conflict is putting a strain on this \\ncooperation and necessitates import substi-\\ntutes, which entail great cost and delays.\\nThe Effects of Remilitarization\\nUnder Vladimir Putin, the modernization \\nof Russia’s armed forces has become a pri -\\nority for the first time since 1991. For sev-\\neral years, considerable sums have been ex-\\npended on this reform. However, challenges \\nremain when it comes to technology and \\norganizational culture; and demographic \\nproblems are also still an issue. Moreover, \\nthe lagging economic output will exacer -'),\n",
" Document(metadata={'creationdate': '2014-04-22T10:23:16+02:00', 'total_pages': 4, 'moddate': '2014-04-22T10:23:18+02:00', 'trapped': '/False', 'page_label': '4', 'source': 'c:\\\\Users\\\\xyzai\\\\Documents\\\\GraphDB\\\\FinalProject\\\\data\\\\pdfs\\\\RU-MILITARY.pdf', 'creator': 'Adobe InDesign CS6 (Macintosh)', 'page': 3, 'producer': 'Adobe PDF Library 10.0.1'}, page_content='there are bottlenecks in production capaci-\\nty, for example in aircraft production and \\nshipbuilding. The plans to enhance military \\ntransport aviation can only be realized if \\ncapacity is expanded rapidly. Russia also \\ndepends on cooperation with Ukraine: So \\nfar, many motors for helicopters and air -\\ncraft as well as rockets have been produced \\nin Ukraine. Russia lacks the know-how for \\nproducing many of the parts required. The \\ncurrent conflict is putting a strain on this \\ncooperation and necessitates import substi-\\ntutes, which entail great cost and delays.\\nThe Effects of Remilitarization\\nUnder Vladimir Putin, the modernization \\nof Russia’s armed forces has become a pri -\\nority for the first time since 1991. For sev-\\neral years, considerable sums have been ex-\\npended on this reform. However, challenges \\nremain when it comes to technology and \\norganizational culture; and demographic \\nproblems are also still an issue. Moreover, \\nthe lagging economic output will exacer -')]"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from langchain_community.retrievers import WikipediaRetriever\n",
"from langchain_community.vectorstores import Chroma\n",
"from data.dataingestion import load_all_pdfs\n",
"from langchain_text_splitters import RecursiveCharacterTextSplitter\n",
"from langchain_community.vectorstores import Chroma\n",
"from models.embedding import get_embeddings\n",
"embeddings = get_embeddings()\n",
"pdf_data = load_all_pdfs()\n",
"\n",
"text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000,chunk_overlap=270)\n",
"splits = text_splitter.split_documents(pdf_data)\n",
"vectorstore = Chroma.from_documents(documents=splits,embedding=embeddings)\n",
"rag_retriever = vectorstore.as_retriever()\n",
"resp=rag_retriever.invoke(\"russian military\")\n",
"resp \n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "261ab304",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "c7e948d4",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "myenv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.8"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
|