cortex / benchmark

Commit History

  1. Enhance benchmark and Cortex modules with new training utilities and improved state management. Update README with example output for Llama-3.2-1B and add training CLI for Cortex module tuning. Refactor scoring functions to reset Cortex state between examples and ensure consistent output. Modify task handling to ensure proper formatting of input data.

    0de2901
  2. Refactor device handling in benchmark and cortex modules to use resolve_torch_device function for improved device selection. Update README with benchmark table formatting and additional device options.

    0ac64e3
  3. Add benchmark harness: run_benchmark.py - CLI entry point

    Verified signature
    18adc2c
  4. Add benchmark harness: runner.py - Main comparison orchestrator

    Verified signature
    fed2333
  5. Add benchmark harness: tasks.py - Standard NLP task definitions

    Verified signature
    e014dff
  6. Add benchmark harness: memory_tasks.py - Passkey retrieval and multi-hop memory

    Verified signature
    81ff944
  7. Add benchmark harness: __init__.py

    Verified signature
    7dd817a
  8. Add benchmark harness: scoring.py

    Verified signature
    4c1ba64