Strategic Navigation or Stochastic Search? How Agents and Humans Reason Over Document Collections
Paper โข 2603.12180 โข Published โข 65
Akim Mousterou
AkimfromParis
AI & ML interests
NLP for French, English, and Japanese - Opinion mining, dataset, evaluation, and quantitative research.
Recent Activity
posted an update 14 days ago
๐ธ ๐๐ฅ๐๐ฃ ๐
๐๐ฅ๐๐ฃ๐๐จ๐ ๐๐๐ ๐๐๐๐๐๐ง๐๐ค๐๐ง๐ ๐2 ๐ค๐ฃ ๐๐ช๐๐๐๐ฃ๐ ๐๐๐๐ ๐ฏ๐ต // ๐ธ ใใฎใณใฐใใงใคใน็ใ ๐ข๐ฝ๐ฒ๐ป ๐๐ฎ๐ฝ๐ฎ๐ป๐ฒ๐๐ฒ ๐๐๐ ๐๐ฒ๐ฎ๐ฑ๐ฒ๐ฟ๐ฏ๐ผ๐ฎ๐ฟ๐ฑ ๐ฉ๐ฎ ใๅ
ฌ้ ๐ฏ๐ต
I am thrilled to announce the launch of version 2 of the ๐๐ฅ๐๐ฃ ๐
๐๐ฅ๐๐ฃ๐๐จ๐ ๐๐๐ ๐๐๐๐๐๐ง๐๐ค๐๐ง๐. This initiative is driven by the "Fine-tuning and Evaluation" team, led by Professor Miyao at the The University of Tokyo, under the Research and Development Center for Large Language Models (LLMC) at Japanโs National Institute of Informatics (NII).
๐๐ฉ๐ง๐๐ฉ๐๐๐๐ ๐๐ฃ๐ ๐ฉ๐๐๐๐ฃ๐๐๐๐ก ๐ช๐ฅ๐๐ง๐๐๐๐จ:
- Our new backend features eight A100 GPUs, enabling the evaluation of open-source models of more than 100B parameters.
- Submissions now require a Hugging Face Hub login to ensure accountability.
- We have added metrics for evaluation time, COโ emissions (thx to Code Carbon ๐ฑ ), alongside reasoning capabilities.
๐ฟ๐๐ฉ๐๐จ๐๐ฉ๐จ ๐๐ฃ๐ ๐๐ซ๐๐ก๐ช๐๐ฉ๐๐ค๐ฃ ๐จ๐ฉ๐๐ฃ๐๐๐ง๐๐จ:
- New datasets cover reasoning, mathematics, exams, and instruction following.
- Math evaluations now span from grade-school levels to expert-tier challenges (GSM8K, PolyMath, AIME).
- While integrating English-heavy and multilingual benchmarks (including Humanityโs Last Exam, GPQA, and BBH in both English and Japanese), we continue to prioritize unique Japanese cultural datasets.
https://huggingface.co/spaces/llm-jp/open-japanese-llm-leaderboard-v2
ใฉใใใ้กใ่ดใใพใ๏ผ๐ upvoted a paper about 2 months ago
Strategic Navigation or Stochastic Search? How Agents and Humans Reason Over Document CollectionsOrganizations
upvoted a paper about 2 months ago
upvoted an article 5 months ago
Article
Continuous batching from first principles
- +1
โข
373
upvoted a paper 6 months ago
upvoted a paper about 1 year ago
upvoted an article over 1 year ago
Article
๐ช๐บโ๏ธ EU AI Act: Systemic Risks in the First CoP Draft Comments โ๏ธ๐ช๐บ
โข
14
upvoted 2
papers over 1 year ago
upvoted an article almost 2 years ago
Article
Let's talk about LLM evaluation
โข
209
upvoted a paper about 2 years ago
upvoted a paper over 2 years ago