clarke / Additional Competition Context.md
yashvshetty's picture
Clarke: NHS clinical documentation system
9636a02

Additional Competition Context β€” MedGemma Impact Challenge

Compiled 2026-02-12 from systematic review of Kaggle competition pages, discussions, rules, and HAI-DEF developer documentation.

Key NEW findings not in Clarke spec:

  1. Submission is a Kaggle Writeup (not a file upload) β€” created via the Writeups tab, with a Submit button. One submission per team, can re-submit.
  2. 3 pages = ~1,500 words (host clarification). With images, aim for 1,000-1,200 words.
  3. 12 named judges β€” all Google Research/Health AI staff. Profiles documented.
  4. Judged as DEMO, not production β€” Yun Liu explicitly said regulatory pathway is NOT the focus. Focus is technical demonstration.
  5. Non-commercial training data is OK β€” Daniel Golden confirmed. Model weights release is BONUS not required.
  6. MedGemma 4B has known instruction-following bugs β€” leaks system prompts, generates meta-commentary. Risk for Clarke's EHR pipeline.
  7. MedGemma 27B deployment is very hard β€” needs ~54GB VRAM, Vertex AI A100 quotas being rejected. Unsloth GGUF quantizations available via Ollama (Q8_0 = 31.8GB).
  8. MedGemma 1.5 4B adds EHR understanding β€” directly relevant to Clarke. Should use 1.5 not 1.0.
  9. MedASR runs in-browser via ONNX/WebGPU β€” someone built it, could strengthen Edge AI track claim.
  10. Only 129 submissions from 5,855 entrants β€” competitive field may be smaller than expected.
  11. Google explicitly suggests agentic orchestration in their MedGemma docs β€” validates Clarke's architecture.

1. Submission Requirements

Submission Format: Kaggle Writeup (NOT a file upload)

3-Page Limit Clarification

Required Links in Writeup

Track Selection

Writeup Template (exact structure required)

### Project name
### Your team [Name members, speciality, role]
### Problem statement [Problem domain + Impact potential criteria]
### Overall solution [Effective use of HAI-DEF models criterion]
### Technical details [Product feasibility criterion]

Private Resources Warning

Submissions Must Be in English


2. Judging Details

Full Judge Panel (12 judges)

Judge Role
Fereshteh Mahvar Staff Medical Software Engineer & Solutions Architect, Google Health AI
Omar Sanseviero Developer Experience Lead, Google DeepMind
Glenn Cameron Sr. PMM, Google
Can "John" Kirmizi Software Engineer, Google Research
Andrew Sellergren Software Engineer, Google Research
Dave Steiner Clinical Research Scientist, Google
Sunny Virmani Group Product Manager, Google Research
Liron Yatziv Research Engineer, Google Research
Daniel Golden Engineering Manager, Google Research
Yun Liu Research Scientist, Google Research
Rebecca Hemenway Health AI Strategic Partnerships, Google Research
Fayaz Jamil Technical Program Manager, Google Research

Source: https://www.kaggle.com/competitions/med-gemma-impact-challenge/overview (Judges section)

Evaluation is Through Demonstration Application Lens

  • Judges evaluate through the lens of a demonstration application, NOT a finished product.
  • Regulatory pathway, HIPAA/GDPR compliance, etc. are not the focus of evaluation criteria β€” though you may include them.
  • Quote from Yun Liu (Competition Host): "The focus of the evaluation criteria is the technical aspects of the demonstration application. Each evaluation criteria will be judged through the lens of a demonstration application and not a finished product."
  • Source: https://www.kaggle.com/competitions/med-gemma-impact-challenge/discussion/668280

Execution & Communication is the Highest-Weighted Category (30%)

  • Judges look for a "cohesive and compelling narrative across all submitted materials" that articulates how you meet the rest of the criteria.
  • Assess: clarity/polish/effectiveness of video demo, completeness/readability of writeup, quality of source code (organization, comments, reusability).
  • Source: https://www.kaggle.com/competitions/med-gemma-impact-challenge/overview (Evaluation Criteria table)

3. Rules & Restrictions

Team Size

One Submission Per Team

Winner License: CC BY 4.0

  • Winning submissions must be licensed under CC BY 4.0 (code and demos).
  • For generally commercially available software you used but don't own, you don't need to grant that license.
  • For input data or pretrained models with incompatible licenses used to generate your winning solution, you don't need to grant open source license for that data/model.
  • Source: https://www.kaggle.com/competitions/med-gemma-impact-challenge/rules (Section 2.5)

Non-Commercial Data is Allowed for Training

  • Using public, research-only / non-commercial external datasets during development is permitted.
  • Participation in a Kaggle challenge is not considered commercial use.
  • Releasing final model weights is a bonus, not a requirement.
  • Daniel Golden (Competition Host) quote: "You are permitted to use data and other code sources during development that are governed under other, potentially more restrictive licenses."
  • Source: https://www.kaggle.com/competitions/med-gemma-impact-challenge/discussion/671596

External Data & Tools

HAI-DEF Terms Key Restrictions

Mandatory HAI-DEF Model Usage

Eligibility

Winner's Obligations

  • Deliver final model's software code + documentation.
  • Code must be capable of generating the winning submission.
  • Must describe resources required to build/run.
  • For hackathons, deliverables are as described on the competition website (may not be software code).
  • Source: https://www.kaggle.com/competitions/med-gemma-impact-challenge/rules (Section 2.8)

No Cloud Credits Provided


4. Winning Patterns & Insights from Discussions

Focus on Demonstration, Not Production

  • The organizers repeatedly emphasize this is about demonstration applications with impact potential, not production-ready systems. Keep the writeup high-level; use the video to convey concepts.
  • "Less is more! You should take advantage of the video to convey most of the concepts and keep the write-up as high level as possible."
  • Source: https://www.kaggle.com/competitions/med-gemma-impact-challenge/overview (Submission Instructions)

Storytelling is Explicitly Scored

Use HAI-DEF Models "to Their Fullest Potential"

Agentic Orchestration is Officially Suggested

  • Google's own MedGemma documentation explicitly suggests agentic orchestration: using MedGemma as a tool within an agentic system, coupled with FHIR generators/interpreters, Gemini Live for bidirectional audio, or Gemini 2.5 Pro for function calling.
  • MedGemma can "parse private health data locally before sending anonymized requests to centralized models" β€” directly supports Clarke's privacy-preserving architecture.
  • Source: https://developers.google.com/health-ai-developer-foundations/medgemma

Competition Scale: Low Submissions So Far

MedGemma 1.5 is the Latest Version


5. Technical Resources

Official Notebooks & Code

MedASR Resources

Resource URL
MedASR model (HuggingFace) https://huggingface.co/google/medasr
MedASR developer docs https://developers.google.com/health-ai-developer-foundations/medasr/
MedASR in-browser (ONNX/WebGPU) https://medasr.ainergiz.com/
MedASR in-browser source https://github.com/ainergiz/medasr-web
MedASR MLX (Apple Silicon) Discussion: https://www.kaggle.com/competitions/med-gemma-impact-challenge/discussion/672879
MedASR ONNX export script In ainergiz/medasr-web repo at /scripts/export_onnx.py

MedGemma 27B GGUF Quantizations (Unsloth)

HuggingFace Collections

Key Blog Posts

Notable Competition Notebooks (for inspiration)

Notebook Theme
MedFlow AI (24 upvotes) Top-voted notebook
MedGemma Navigator: DICOMweb ↔ FHIR (16 upvotes) FHIR/DICOM integration β€” similar to Clarke's EHR focus
MedGemma Medical AI Chatbot + 5 Test Scenarios (14 upvotes) Medical chatbot with test scenarios
MedAssist Edge Offline Medical AI (7 upvotes) Offline/edge deployment
Spasht AI β€” Bridging India's "Last Mile" Health Gap (6 upvotes) Community health focus
RadAssist-MedGemma: AI Radiology Triage Assistant (4 upvotes) Radiology triage
CRSA β€” Clinical Reasoning Stability Auditor Reasoning evaluation

Source: https://www.kaggle.com/competitions/med-gemma-impact-challenge/code

6. Gaps & Risks

MedGemma 4B Instruction Following Issues

  • Multiple competitors report MedGemma 4B (medgemma-1.5-4b-it) leaking system prompts, generating meta-commentary, and outputting chain-of-thought training artifacts (critique responses, constraint checklists, special tokens).
  • One user reports running with Ollama locally works well; the issues may be prompt/framework dependent.
  • Risk for Clarke: If using MedGemma 4B for EHR extraction, we need thorough prompt engineering and output parsing to handle instruction-following failures.
  • Source: https://www.kaggle.com/competitions/med-gemma-impact-challenge/discussion/673091

MedGemma 27B Deployment is Extremely Challenging

  • Requires ~54GB VRAM β€” won't run on Apple M3 Max 64GB (MPS buffer limit), won't fit on 2x L4 GPUs (48GB).
  • Vertex AI A100 quota requests being rejected by Google.
  • Quantized GGUF versions (Unsloth Q8_0 = 31.8GB) can run via Ollama on local hardware.
  • Risk for Clarke: The 24-hour build plan assumes MedGemma 27B for letter generation. If deploying on Kaggle/cloud is blocked by GPU quota, we need a fallback (quantized 27B via Ollama, or enhanced 4B with heavy prompt engineering).
  • Source: https://www.kaggle.com/competitions/med-gemma-impact-challenge/discussion/673091

MedGemma 1.5 Only Available as 4B

No Provided Dataset = You Must Source Your Own

Model Weights Release is Bonus, Not Required

EHR/FHIR Understanding is a New MedGemma 1.5 Capability

Official Discord Exists But Not Monitored by Staff


Pages That Could Not Be Accessed

URL Issue
Kaggle discussion threads via web_fetch JS-rendered, required browser automation
Kaggle Code notebooks (content) Would require login/browser to read notebook code cells
AskCPG concept app details https://discuss.ai.google.dev/t/sharing-our-product-integration-with-medgemma-askcpg/94556 β€” not fetched
MedGemma model card https://developers.google.com/health-ai-developer-foundations/medgemma/model-card β€” not fetched
MedASR developer docs https://developers.google.com/health-ai-developer-foundations/medasr/ β€” not fetched
HAI-DEF Terms of Use (full) https://developers.google.com/health-ai-developer-foundations/terms β€” truncated at Section 3