Feature Extraction
Transformers
TensorBoard
Safetensors
English
captionbert_v2
sentence-similarity
consensus-distillation
geometric-deep-learning
amoe
custom_code
Instructions to use AbstractPhil/captionbert-8192-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use AbstractPhil/captionbert-8192-v2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="AbstractPhil/captionbert-8192-v2", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("AbstractPhil/captionbert-8192-v2", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Commit History
-
link the published week-in-review article (Geometric Memory FT5)
Verified signatureed468fe -
link the technical companion from the card
Verified signature82cd892 -
references+attribution section, inline citations
Verified signaturebb5440a
-
technical companion (week-in-review docket, verified)
Verified signature1ad21ba
-
remove stray: byte-identical duplicate (sha256 45bcbf0c) of amoe/sts/captionbert-v2-sts-anchor.anchor.pt, which survives. The combo folder ships only its own -combo anchor; the solo anchor sitting here invited loading the wrong artifact
Verified signaturebe530bf -
card: attach path pointed at amoe/sts/ for a file that lives in amoe/sts-combo/ (404); added the measured step-500 results and the trunk/ckpt pairing
Verified signature90a9236 -
card: moe-v2 carries its OWN numbers and the tau sweep (sharper routing is worse, monotone in 2 seeds while separation rises). Load path pointed at amoe/moe/ whose anchors are a DIFFERENT checkpoint (f13ad5ad vs 0029ad69) - fixed to this folder
Verified signaturef6118e4 -
card: this folder is the 3-ANCHOR COLLECTIVE, not the 2-anchor moe run - the moe card had been copied here wholesale (wrong title, wrong anchor table, wrong results, moe-v1 amplitude telemetry). Real 8-task rows, greedy sweep incl. the random capacity control, correct in-folder load paths, masked-vs-solo labelled
Verified signature7c1933b -
modeling: AMOE_REPO resolves from config._name_or_path, not a hardcoded v2 - anchors are TRUNK-BOUND (v2 arms lose 31%% of their gain on the -B trunk, .7287 -> .6863; key realignment recovers 29%%). Dispatch is now the first candidate PRESENT in the loaded repo, with a local-dir hint on miss.
Verified signaturedbc600c -
Upload CaptionBert_V2.ipynb
Verified signature6b44062 -
Create full_benchmark.py
Verified signatureb335dea -
Update README.md
Verified signature48ef031 -
final (config)
Verified signature468ad1b -
final (tensorboard)
Verified signaturee8e7e63 -
final (checkpoints)
Verified signatureffc8b95
-
Update README.md
Verified signature8f4d990 -
Update modeling_captionbert.py
Verified signature177168d -
Update README.md
Verified signature0c8f923 -
final: 2 anchors + dispatch + card
Verified signature22eb16f -
arm random
Verified signature5efad23 -
arm topical
Verified signature3bc0314 -
arm lexical
Verified signature2d26612 -
arm paraphrase
Verified signatureb1902b6 -
arm simplify
Verified signature19c5025 -
arm equiv
Verified signature689da64 -
final: 2 anchors + dispatch + card
Verified signature26d53b2 -
Create train_amoe_v2_2anchor_equiv_simplify.py
Verified signatured777d10 -
anchor simplify
Verified signature8ead4d6 -
anchor equiv
Verified signaturec78bfb8 -
Create early_eval.py
Verified signature327ed1e -
Create pretrain_trainer.py
Verified signature9bdd151 -
Create trainers/train_amoe_2anchor_equi_simplify.py
Verified signature4ef2033 -
final: 2 anchors + dispatch + card
Verified signature9154293 -
anchor simplify
Verified signatureb5b6e0f -
anchor equiv
Verified signatureee040c1 -
AutoModel: safetensors + config + modeling + card
Verified signatured2ba5ec -
AutoModel: safetensors + config + modeling + card
Verified signaturea06aad3 -
Create modeling_captionbert.py
Verified signature7d843f4 -
step 500
Verified signaturea2a2a0f -
final anchor
Verified signature82c8bcc -
step 4000
Verified signature7d213a7 -
step 3500
Verified signature3cbee9e -
step 3000
Verified signature64d984a -
step 2500
Verified signaturebd63d6a -
step 2000
Verified signaturebc48389 -
step 1500
Verified signature359ff55 -
step 1000
Verified signaturee46418d -
step 500
Verified signature2ebd035 -
final (tensorboard)
Verified signature2c1931d -
final (checkpoints)
Verified signature12f2835