Retrieved Papers
42 resultsAI-Generated Summary
Generated by our fine-tuned BioBERT-RoBERTa hybrid model with RAG
The literature review reveals several key approaches to customizing LLMs for scientific literature review:
(Lewis et al., 2020) introduced Retrieval-Augmented Generation (RAG) models that combine parametric memory (a pre-trained seq2seq model) with non-parametric memory (a dense vector index of Wikipedia). This approach shows promise for knowledge-intensive tasks.
For biomedical applications, (Lee et al., 2020) demonstrated the effectiveness of domain-specific pretraining with BioBERT, achieving state-of-the-art results on biomedical NLP tasks. Their work highlights the importance of domain adaptation.
(Zhang et al., 2022) optimized RAG specifically for scientific literature review, implementing a hybrid retrieval-generation framework with improved efficiency for large document collections.
Recent work by (Wang et al., 2023) addresses multi-domain challenges, proposing adaptation techniques that enable a single model to handle diverse scientific literature with minimal retraining.
Citation Analysis
Citation Accuracy
Citation Distribution
Potential Issues
Citation to "Zhang et al., 2022" may refer to multiple papers - verify context.
"Wang et al., 2023" preprint - consider peer-reviewed alternatives.
Provide Feedback
Help improve our model by rating this summary
Methodology Overview
Data Collection
- PubMed for biomedical research
- arXiv for CS/physics/math preprints
- IEEE Xplore for engineering
- Google Scholar for benchmarking
Model Architecture
- BioBERT for biomedical domains
- RoBERTa for general science
- DPR with FAISS indexing
- End-to-end RAG training
Evaluation Metrics
- ROUGE-L, BLEU for summaries
- Citation accuracy percentage
- User satisfaction surveys
- Comparison with Elicit/Scispace
RAG Training Pipeline
Data Preprocessing
GROBID for PDF parsing, text normalization, SciBERT tokenization
Supervised Fine-Tuning
Train on S2ORC for summary generation with embedded citations
RLHF Refinement
Researcher feedback on citation relevance and clarity
End-to-End Training
Contrastive loss to align retriever-generator outputs