kamkol commited on
Commit
02cbcb3
Β·
1 Parent(s): 4ddf811

Remove notebook_version folder from git tracking

Browse files
notebook_version/AB_Testing_RAG_Agent.ipynb DELETED
The diff for this file is too large to render. See raw diff
 
notebook_version/README.md DELETED
@@ -1,94 +0,0 @@
1
-
2
- <p align = "center" draggable=”false” ><img src="https://github.com/AI-Maker-Space/LLM-Dev-101/assets/37101144/d1343317-fa2f-41e1-8af1-1dbb18399719"
3
- width="200px"
4
- height="auto"/>
5
- </p>
6
-
7
- ## <h1 align="center" id="heading">Session 8: Evaluating RAG with Ragas</h1>
8
-
9
- | πŸ€“ Pre-work | πŸ“° Session Sheet | ⏺️ Recording | πŸ–ΌοΈ Slides | πŸ‘¨β€πŸ’» Repo | πŸ“ Homework | πŸ“ Feedback |
10
- |:-----------------|:-----------------|:-----------------|:-----------------|:-----------------|:-----------------|:-----------------|
11
- | [Session 8: Pre-Work](https://www.notion.so/Session-8-RAG-Evaluation-and-Assessment-1c8cd547af3d81d08f7cf5521d0253bb?pvs=4#1c8cd547af3d816583d6c23183b6f87f) | [Session 8: RAG Evaluation and Assessment](https://www.notion.so/Session-8-RAG-Evaluation-and-Assessment-1c8cd547af3d81d08f7cf5521d0253bb) | Coming soon! | [Session 8 Slides](https://www.canva.com/design/DAGjadKGqcw/0Gff9K2EwbOb3lX14un3uw/edit?utm_content=DAGjadKGqcw&utm_campaign=designshare&utm_medium=link2&utm_source=sharebutton) | You are here! | [Session 8: RAG Evaluation and Assessment](https://forms.gle/ujAQLqx2ZHMWTUH79) | [AIE6 Feedback 4/24](https://forms.gle/wA7p89e6svCgjtr58) |
12
-
13
- In today's assignment, we'll be creating Synthetic Data, and using it to benchmark (and improve) a LCEL RAG Chain.
14
-
15
- - 🀝 Breakout Room #1
16
- 1. Task 1: Installing Required Libraries
17
- 2. Task 2: Set Environment Variables
18
- 3. Task 3: Synthetic Dataset Generation for Evaluation using Ragas
19
- 4. Task 4: Evaluating our Pipeline with Ragas
20
- 5. Task 6: Making Adjustments and Re-Evaluating
21
-
22
- The notebook Colab link is located [here](https://colab.research.google.com/drive/1-t4POIFJI-SWF1lmoBOPETZZqgWCTV4Y?usp=sharing)
23
-
24
- - 🀝 Breakout Room #2
25
- 1. Task 1: Building a ReAct Agent with Metal Price Tool
26
- 2. Task 2: Implementing the Agent Graph Structure
27
- 3. Task 3: Converting Agent Messages to Ragas Format
28
- 4. Task 4: Evaluating Agent Performance using Ragas Metrics
29
- - Tool Call Accuracy
30
- - Agent Goal Accuracy
31
- - Topic Adherence
32
-
33
- The notebook Colab link is located [here](https://colab.research.google.com/drive/1KQm7nA_zTaCyjaAeAacjqanMPv03um7T?usp=sharing)
34
-
35
- ## Ship 🚒
36
-
37
- The completed notebook!
38
-
39
- <details>
40
- <summary>🚧 BONUS CHALLENGE 🚧 (OPTIONAL)</summary>
41
-
42
- > NOTE: Completing this challenge will provide full marks on the assignment, regardless of the completion of the notebook. You do not need to complete this in the notebook for full marks.
43
-
44
- ##### **MINIMUM REQUIREMENTS**:
45
-
46
- 1. Baseline `LangGraph RAG` Application using `NAIVE RETRIEVAL`
47
- 2. Baseline Evaluation using `RAGAS METRICS`
48
- - [Faithfulness](https://docs.ragas.io/en/stable/concepts/metrics/faithfulness.html)
49
- - [Answer Relevancy](https://docs.ragas.io/en/stable/concepts/metrics/answer_relevance.html)
50
- - [Context Precision](https://docs.ragas.io/en/stable/concepts/metrics/context_precision.html)
51
- - [Context Recall](https://docs.ragas.io/en/stable/concepts/metrics/context_recall.html)
52
- - [Answer Correctness](https://docs.ragas.io/en/stable/concepts/metrics/answer_correctness.html)
53
- 3. Implement a `SEMANTIC CHUNKING STRATEGY`.
54
- 4. Create an `LangGraph RAG` Application using `SEMANTIC CHUNKING` with `NAIVE RETRIEVAL`.
55
- 5. Compare and contrast results.
56
-
57
- ##### **SEMANTIC CHUNKING REQUIREMENTS**:
58
-
59
- Chunk semantically similar (based on designed threshold) sentences, and then paragraphs, greedily, up to a maximum chunk size. Minimum chunk size is a single sentence.
60
-
61
- Have fun!
62
- </details>
63
-
64
- ### Deliverables
65
-
66
- - A short Loom of the notebook, and a 1min. walkthrough of the application in full
67
-
68
- ## Share πŸš€
69
-
70
- Make a social media post about your final application!
71
-
72
- ### Deliverables
73
-
74
- - Make a post on any social media platform about what you built!
75
-
76
- Here's a template to get you started:
77
-
78
- ```
79
- πŸš€ Exciting News! πŸš€
80
-
81
- I am thrilled to announce that I have just built and shipped Synthetic Data Generation, benchmarking, and iteration with RAGAS & LangChain! πŸŽ‰πŸ€–
82
-
83
- πŸ” Three Key Takeaways:
84
- 1️⃣
85
- 2️⃣
86
- 3️⃣
87
-
88
- Let's continue pushing the boundaries of what's possible in the world of AI and question-answering. Here's to many more innovations! πŸš€
89
- Shout out to @AIMakerspace !
90
-
91
- #LangChain #QuestionAnswering #RetrievalAugmented #Innovation #AI #TechMilestone
92
-
93
- Feel free to reach out if you're curious or would like to collaborate on similar projects! 🀝πŸ”₯
94
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
notebook_version/pyproject.toml DELETED
@@ -1,16 +0,0 @@
1
- [project]
2
- name = "08-evaluating-rag-with-ragas"
3
- version = "0.1.0"
4
- description = "Add your description here"
5
- readme = "README.md"
6
- requires-python = ">=3.13"
7
- dependencies = [
8
- "jupyter>=1.1.1",
9
- "langchain-community==0.3.14",
10
- "langchain-openai==0.2.14",
11
- "langchain-qdrant>=0.2.0",
12
- "langgraph==0.2.61",
13
- "numpy>=2.2.2",
14
- "unstructured>=0.14.8",
15
- "arxiv>=1.4.0",
16
- ]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
notebook_version/uv.lock DELETED
The diff for this file is too large to render. See raw diff