{ "cells": [ { "cell_type": "code", "execution_count": null, "id": "61997615", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "id": "baa55c98", "metadata": {}, "outputs": [], "source": [ "import faiss\n", "\n", "merged_index = faiss.read_index(\"/home/mshahidul/readctrl/data/vector_db/qwen_em/shard_0_en.faiss\")\n", "for i in range(1, NUM_SHARDS):\n", " next_index = faiss.read_index(f\"/home/mshahidul/readctrl/data/vector_db/qwen_em/shard_{i}_en.faiss\")\n", " merged_index.merge_from(next_index)\n", "\n", "faiss.write_index(merged_index, \"full_wikipedia_index.faiss\")" ] }, { "cell_type": "code", "execution_count": 1, "id": "c8c08129", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Successfully loaded. Type: \n" ] } ], "source": [ "import json\n", "\n", "file_path = '/home/mshahidul/readctrl/data/testing_data_gs/multiclinsum_gs_train_en.json'\n", "\n", "try:\n", " with open(file_path, 'r', encoding='utf-8') as file:\n", " data = json.load(file)\n", " \n", " # Success: 'data' is now a Python object\n", " print(f\"Successfully loaded. Type: {type(data)}\")\n", " \n", "except FileNotFoundError:\n", " print(\"Error: The file path was not found.\")\n", "except json.JSONDecodeError:\n", " print(\"Error: Failed to decode JSON. Check if the file is formatted correctly.\")" ] }, { "cell_type": "code", "execution_count": 6, "id": "055e14be", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "summary 1========================================\n", "We present the case of a 20-year-old woman with a 12-year history of idiopathic NS revealed by extensive cerebral venous thrombosis with pulmonary embolism treated with anticoagulation therapy and oral corticosteroid therapy followed by mycophenolate mofetil (MMF). The thrombophilia assessment did not show any abnormalities. The evolution was marked by the occurrence of several NS relapses controlled by oral corticosteroid therapy until 2017. Subsequently, the patient had not presented a relapse of her disease. The anticoagulant treatment and the MMF were therefore stopped. One year later, the patient presented with severe diffuse acute abdominal pain associated with postprandial vomiting and bilateral lower limb edema. Laboratory results confirmed a NS relapse. An abdominal CT scan revealed acute thrombosis of the superior mesenteric artery with acute mesenteric ischemia. Intraoperative exploration showed mesenteric ischemia with extensive necrosis of the small intestine making their resections incompatible with life. The patient died after 48 hours.\n", "summary 2========================================\n", "A 34-year-old pregnant woman presents with seizures and dysarthria and is urgently referred for a cranial MRI. The classic ‘Medusa head’ sign is seen and the diagnosis is made as a venous anomaly of development with peripheral partial thrombosis and proximal slow flow.\n", "\n", "summary 3========================================\n", "A 22-year-old woman came to the Oral Medicine Department with complaints of stomatitis causing pain, eating, and drinking difficulty, which started with fever and pimple-like on the lips. She was an active vape user for one year. Extraoral examination revealed no lesions on other body parts. The serosanguinolent crusts on the lips, an erosive area on the labial commissures and tended to bleed. Intraoral examination revealed white ulcers with yellowish edges and irregular, varying sizes in several parts of the oral mucosa. The anti-HSV-1 IgG laboratory results showed non-reactive, leading to a diagnosis of oral erythema multiforme. Management of oral conditions using 0.9% NaCl compress, dexamethasone mouthwash, and hyaluronic acid, applying 2% miconazole cream on labial commissures and vaseline album cream on the dry lips, and stopping vaping. Oral condition improved in a week of therapy.\n", "summary 4========================================\n", "We are reporting an isolated, asymptomatic fetal intra-cardiac mass (rhabdomyoma) that was discovered at 32 weeks of gestation and was followed as an outpatient until 39 weeks plus one day, at which point a cesarean section was performed. After delivery, the child underwent evaluations at the 1st day, 7th day, 30th day, 7th month, and 12th month of age. Following a checkup, the child's anthropometric and neurobehavioral growth were both healthy. Except for the tumor, which was neither growing nor shrinking in size, none of the clinical diagnostic criteria for tuberous sclerosis complex were met for this child up to the age of one year.\n" ] } ], "source": [ "for i,x in enumerate(data[:4]):\n", " print(f\"summary {i+1}========================================\")\n", " print(x['summary'])" ] }, { "cell_type": "code", "execution_count": 7, "id": "94ad4f5d", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'id': 'multiclinsum_gs_en_27.txt',\n", " 'fulltext': 'A 20-year-old woman was followed up since the age of eight for idiopathic NS inaugurated by cerebral venous thrombosis extended to the right jugular vein with a massive pulmonary embolism. The patient did not have any sequelae. She had no other medical or surgical history. A family history of thrombosis has not been reported. The patient was not biopsied because she had no kidney failure nor gross hematuria, or hypertension at first presentation; added to that, she had no extra renal signs suggestive of a secondary nephrotic syndrome. She was accordingly put on anticoagulant therapy (Oral vitamin K antagonist) and oral corticosteroid therapy with good evolution. Thereafter, the patient received several cures of high-dose corticosteroids for steroid-dependent relapses of NS. She was, hence, put on mycophenolate mofetil (MMF) as a background therapy to avoid corticosteroids and ensure normal growth. An exhaustive assessment of thrombophilia was performed and did not show any abnormality. Homocysteine rate, blood fibrinogen rate, Protein C, protein S, antithrombin III, factor V Leiden mutation, JAK-2 mutation, cryoglobulins, anticardiolipin antibodies, lupus anticoagulant and beta-1-glycoprotein antibodies were normal. The anticoagulant treatment was stopped after nine years. The evolution was enameled by the occurrence of several relapses of her disease controlled by oral corticosteroid therapy. Remission of NS has been noted since 2017, so MMF was gradually stopped in 2019 and the patient remained asymptomatic and without any relapse.\\n\\nOne year later, the patient came up to our emergency department for acute intense diffuse abdominal pain without any particular irradiation associated with postprandial vomiting and bilateral lower limb edema for the last six hours. The physical examination revealed an intense epigastric tenderness with normal vital signs (arterial pressure of 120/70 mm Hg, heart rate of 83 bpm, and oxygen saturation at 100% on room air). The patient was afebrile with normal consciousness. The rest of the physical examination was unremarkable. The urinalysis with labstix revealed proteinuria. The hemogasanalysis results showed metabolic acidosis with respiratory compensation. Further laboratory tests revealed hypoalbuminemia, hypercholesterolemia, a prothrombin time at 90%, high levels of D-dimer, lactate dehydrogenase, and creatine phosphokinase as well as a biological inflammatory syndrome with a CRP of 37 mg/L, and leucocytosis at 26.4 x 103/µL. Renal and liver functions were normal.\\n\\nThe patient was hospitalized in an intensive care unit with close monitoring of vital signs and initiation of resuscitation measures. An abdominal ultrasound was performed urgently showing an intra-abdominal effusion of low to moderate abundance. An abdominal CT scan revealed acute thrombosis of the superior mesenteric artery with acute mesenteric ischemia. The patient was immediately routed to the operating room. Intraoperative exploration confirmed mesenteric ischemia with extensive necrosis of almost entirely of the small bowel making their resections incompatible with life shown in Figure 3. The patient died after 48 hours.',\n", " 'summary': 'We present the case of a 20-year-old woman with a 12-year history of idiopathic NS revealed by extensive cerebral venous thrombosis with pulmonary embolism treated with anticoagulation therapy and oral corticosteroid therapy followed by mycophenolate mofetil (MMF). The thrombophilia assessment did not show any abnormalities. The evolution was marked by the occurrence of several NS relapses controlled by oral corticosteroid therapy until 2017. Subsequently, the patient had not presented a relapse of her disease. The anticoagulant treatment and the MMF were therefore stopped. One year later, the patient presented with severe diffuse acute abdominal pain associated with postprandial vomiting and bilateral lower limb edema. Laboratory results confirmed a NS relapse. An abdominal CT scan revealed acute thrombosis of the superior mesenteric artery with acute mesenteric ischemia. Intraoperative exploration showed mesenteric ischemia with extensive necrosis of the small intestine making their resections incompatible with life. The patient died after 48 hours.'}" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data[0]" ] }, { "cell_type": "code", "execution_count": null, "id": "2b2eead1", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "un", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.14" } }, "nbformat": 4, "nbformat_minor": 5 }