VictorLJZ commited on
Commit
f06bcdb
·
1 Parent(s): 25fe9b8

updated documentation and pyprojects

Browse files
Files changed (2) hide show
  1. README.md +25 -8
  2. pyproject.toml +1 -0
README.md CHANGED
@@ -28,6 +28,7 @@ MedRAX is built on a robust technical foundation:
28
  - **Disease Classification**: Leverages DenseNet-121 from TorchXRayVision for detecting 18 pathology classes
29
  - **X-ray Generation**: Utilizes RoentGen for synthetic CXR generation
30
  - **Web Browser**: Provides web search capabilities and URL content retrieval using Google Custom Search API
 
31
  - **Python Sandbox**: Executes Python code in a secure, stateful sandbox environment using `langchain-sandbox` and Pyodide. Supports custom data analysis, calculations, and dynamic package installations. Pre-configured with medical analysis packages including pandas, numpy, pydicom, SimpleITK, scikit-image, Pillow, scikit-learn, matplotlib, seaborn, and openpyxl. **Requires Deno runtime.**
32
  - **Utilities**: Includes DICOM processing, visualization tools, and custom plotting capabilities
33
  <br><br>
@@ -159,6 +160,7 @@ selected_tools = [
159
  "ChestXRaySegmentationTool",
160
  "PythonSandboxTool", # Python code execution
161
  "WebBrowserTool", # Web search and URL access
 
162
  # Add or remove tools as needed
163
  ]
164
 
@@ -174,17 +176,10 @@ agent, tools_dict = initialize_agent(
174
 
175
  The following tools will automatically download their model weights when initialized:
176
 
177
- ### Classification Tools
178
  ```python
179
  # TorchXRayVision-based classifier (original)
180
  TorchXRayVisionClassifierTool(device=device)
181
-
182
- # ArcPlus SwinTransformer-based classifier (new)
183
- ArcPlusClassifierTool(
184
- model_path="/path/to/Ark6_swinLarge768_ep50.pth.tar", # Optional
185
- num_classes=18, # Default
186
- device=device
187
- )
188
  ```
189
 
190
  ### Segmentation Tool
@@ -263,6 +258,7 @@ No additional model weights required:
263
  ImageVisualizerTool()
264
  DicomProcessorTool(temp_dir=temp_dir)
265
  WebBrowserTool() # Requires Google Search API credentials
 
266
  ```
267
  <br>
268
 
@@ -281,6 +277,25 @@ ChestXRayGeneratorTool(
281
  2. Place weights in `{model_dir}/roentgen`
282
  3. Optional tool, can be excluded if not needed
283
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
284
  ### Knowledge Base Setup (MedicalRAGTool)
285
 
286
  The `MedicalRAGTool` uses a Pinecone vector database to store and retrieve medical knowledge. To use this tool, you need to set up a Pinecone account and a Cohere account.
@@ -383,6 +398,8 @@ If you are running a local LLM using frameworks like [Ollama](https://ollama.com
383
 
384
  **WebBrowserTool**: Requires Google Custom Search API credentials, which can be set in the `.env` file.
385
 
 
 
386
  **PythonSandboxTool**: Requires Deno runtime installation:
387
  ```bash
388
  # Verify Deno is installed
 
28
  - **Disease Classification**: Leverages DenseNet-121 from TorchXRayVision for detecting 18 pathology classes
29
  - **X-ray Generation**: Utilizes RoentGen for synthetic CXR generation
30
  - **Web Browser**: Provides web search capabilities and URL content retrieval using Google Custom Search API
31
+ - **DuckDuckGo Search**: Offers privacy-focused web search capabilities using DuckDuckGo search engine for medical research, fact-checking, and accessing current medical information without API keys
32
  - **Python Sandbox**: Executes Python code in a secure, stateful sandbox environment using `langchain-sandbox` and Pyodide. Supports custom data analysis, calculations, and dynamic package installations. Pre-configured with medical analysis packages including pandas, numpy, pydicom, SimpleITK, scikit-image, Pillow, scikit-learn, matplotlib, seaborn, and openpyxl. **Requires Deno runtime.**
33
  - **Utilities**: Includes DICOM processing, visualization tools, and custom plotting capabilities
34
  <br><br>
 
160
  "ChestXRaySegmentationTool",
161
  "PythonSandboxTool", # Python code execution
162
  "WebBrowserTool", # Web search and URL access
163
+ "DuckDuckGoSearchTool", # Privacy-focused web search
164
  # Add or remove tools as needed
165
  ]
166
 
 
176
 
177
  The following tools will automatically download their model weights when initialized:
178
 
179
+ ### Classification Tool
180
  ```python
181
  # TorchXRayVision-based classifier (original)
182
  TorchXRayVisionClassifierTool(device=device)
 
 
 
 
 
 
 
183
  ```
184
 
185
  ### Segmentation Tool
 
258
  ImageVisualizerTool()
259
  DicomProcessorTool(temp_dir=temp_dir)
260
  WebBrowserTool() # Requires Google Search API credentials
261
+ DuckDuckGoSearchTool() # No API key required, privacy-focused search
262
  ```
263
  <br>
264
 
 
277
  2. Place weights in `{model_dir}/roentgen`
278
  3. Optional tool, can be excluded if not needed
279
 
280
+ ### ArcPlus SwinTransformer-based Classifier
281
+ ```python
282
+ ArcPlusClassifierTool(
283
+ model_path="/path/to/Ark6_swinLarge768_ep50.pth.tar", # Optional
284
+ num_classes=18, # Default
285
+ device=device
286
+ )
287
+ ```
288
+
289
+ The ArcPlus classifier requires manual setup as the pre-trained model is not publicly available for automatic download:
290
+
291
+ 1. **Request Access**: Visit [https://github.com/jlianglab/Ark](https://github.com/jlianglab/Ark) and request the pretrained model through their Google Forms
292
+ 2. **Download Model**: Once approved, download the `Ark6_swinLarge768_ep50.pth.tar` file
293
+ 3. **Place in Directory**: Drag the downloaded file into your `model-weights` directory
294
+ 4. **Initialize Tool**: The tool will automatically look for the model file in the specified `cache_dir`
295
+
296
+ The ArcPlus model provides advanced chest X-ray classification across 6 medical datasets (MIMIC, CheXpert, NIH, RSNA, VinDr, Shenzhen) with 52+ pathology categories.
297
+ ```
298
+
299
  ### Knowledge Base Setup (MedicalRAGTool)
300
 
301
  The `MedicalRAGTool` uses a Pinecone vector database to store and retrieve medical knowledge. To use this tool, you need to set up a Pinecone account and a Cohere account.
 
398
 
399
  **WebBrowserTool**: Requires Google Custom Search API credentials, which can be set in the `.env` file.
400
 
401
+ **DuckDuckGoSearchTool**: No API key required. Uses DuckDuckGo's privacy-focused search engine for medical research and fact-checking.
402
+
403
  **PythonSandboxTool**: Requires Deno runtime installation:
404
  ```bash
405
  # Verify Deno is installed
pyproject.toml CHANGED
@@ -75,6 +75,7 @@ dependencies = [
75
  "seaborn>=0.12.0",
76
  "huggingface_hub>=0.17.0",
77
  "iopath>=0.1.10",
 
78
  ]
79
 
80
  [project.optional-dependencies]
 
75
  "seaborn>=0.12.0",
76
  "huggingface_hub>=0.17.0",
77
  "iopath>=0.1.10",
78
+ "duckduckgo-search>=4.0.0",
79
  ]
80
 
81
  [project.optional-dependencies]