PubMedTM / 01_Introduction.py
Array3022
2024-03-27 Release Fix
ca38c99
Raw History Blame Contribute Delete
4.78 kB
import streamlit as st
st.set_page_config(page_title='The PubMed Topic Modeling Toolkit')
# Introduction
st.title('The PubMed Topic Modeling Toolkit')
st.caption('By [Brice Bowrey](https://bowrey.info)')
st.caption('Developed with support from the NLM Michael E. DeBakey Fellowship in the History of Medicine.')
st.caption('''\nThis is a stable release of a project that may see continued development.
If you have suggestions, advice, or an interest in collaborating/contributing, please contact me at bowrey[at]umd.edu.''')
st.header('Introduction & Tutorial Video')
st.markdown('''This short video will provide an introduction to this site.
If you'd like to learn more about topic modeling or the inner workings of the software,
read the brief descriptions below. If you'd like to run the application locally to make changes
or get better performance, please see the information in the `README.md` file in the Hugging Face Files tab.''')
with open('PubmedTM_Tutorial.mp4', 'rb') as video:
video_bytes = video.read()
st.video(video_bytes)
# About Topic Modeling
st.header('About Topic Modeling')
st.subheader('Vocabulary')
st.markdown('''**Corpus**: a set of documents or texts. In this tool, the corpus consists of a approximately 300,000 abstracts related to biomedical technology. These abstracts were published between 1950 and 2000 and retried from the PubMed index.
**Topic**: a cluster of related documents, as defined by the model.
**Model**: the computational engine that produces topics and information about the documents inside topics.
**Cosine Similarity**: a measure of the similarity between two pieces of text that is independent of the length of the texts. Possible values range between negative one (-1) and positive one (1). Two texts are more similar the closer the cosine similarity is to one (1). In this tool, cosine similarity is often presented to measure how closely returned results match a query.
''')
st.subheader('Topic Modeling')
st.markdown('''Topic modeling uses statistics and machine learning to identify clusters of related
words and texts (e.g., text within the same “topic”) in a corpus. Topic modeling is mostly
“unsupervised,” which means that the researcher does not tell the algorithm how it
should label the data. While this technique leverages quantitative tools, there is still significant humanistic inquiry involved in
producing and interpreting the model. The algorithm identifies related texts,
but the user must determine how and why those texts are related. In other words,
the user has to determine what a topic is “about.”
''')
st.markdown('''Because the meaning of a topic is a matter of interpretation, this tool
identifies topics using numbers, not names. When presented with a topic,
you will see a list of words that are most associated with that topic.
From those words and the documents linked to that topic, you can make your own
judgment about a topic's meaning. Keep in mind that some topics might just
be noise or outliers.
''')
st.markdown('''For more detailed discussion of topic modeling in humanities and social science research, see:
* Mohr, John W., and Petko Bogdanov. “Introduction—Topic Models: What They Are and Why They Matter.” _Poetics_ 41, no. 6 (December 2013): 545–69. [https://doi.org/10.1016/j.poetic.2013.10.001](https://doi.org/10.1016/j.poetic.2013.10.001).
* Goldstone, Andrew, and Ted Underwood. “The Quiet Transformations of Literary Studies: What Thirteen Thousand Scholars Could Tell Us.” _New Literary History_ 45, no. 3 (2014): 359–84. [https://doi.org/10.1353/nlh.2014.0025](https://doi.org/10.1353/nlh.2014.0025).
* Hannigan T.R., Glaser V.L., Wang M.S., Jennings P.D., Haan R.F.J., Vakili K., Tchalian H., and Kaplan S. “Topic Modeling in Management Research: Rendering New Theory from Textual Data.” _Academy of Management Annals_ 13, no. 2 (2019): 586–632. [https://doi.org/10.5465/annals.2017.0099](https://doi.org/10.5465/annals.2017.0099).
''')
st.markdown('''This tool was created using Top2Vec. This algorithm avoids the need
for significant pre-processing of the text. This was advantageous due to the difficulty
of using existing tools to clean and pre-process the unique and
discipline-specific jargon in the corpus. For more on Top2Vec, see:
* Angelov, Dimo. “Top2Vec: Distributed Representations of Topics,” August 19, 2020. [https://doi.org/10.48550/ARXIV.2008.09470](https://doi.org/10.48550/ARXIV.2008.09470).
* Egger, Roman, and Joanne Yu. “A Topic Modeling Comparison Between LDA, NMF, Top2Vec, and BERTopic to Demystify Twitter Posts.” _Frontiers in Sociology_ 7 (May 6, 2022): 886498. [https://doi.org/10.3389/fsoc.2022.886498](https://doi.org/10.3389/fsoc.2022.886498)
''')