gmongaras
/

Cosine_Attention_GPT_1.2B

Feature Extraction

Model card Files Files and versions

This repository contains the GPT-Cos-1.2b model described in Cottention: Linear Transformers With Cosine Attention.

Downloads last month: 1

Collection including gmongaras/Cosine_Attention_GPT_1.2B

Cosine Attention (Cottention)

Models for the paper Cottention: Linear Transformers With Cosine Attention https://arxiv.org/abs/2409.18747 • 6 items • Updated Oct 7, 2024

Paper for gmongaras/Cosine_Attention_GPT_1.2B

Cottention: Linear Transformers With Cosine Attention

Paper • 2409.18747 • Published Sep 27, 2024 • 16