Kurtis-EON1

Kurtis-EON(1): "Infinite" Context (see notes), O(1) Memory, Zero KV-Cache growth, Constant inference cost. Recurrent State.

• "Infinite" Context: Capable of processing input streams of unlimited length by compressing history into a continuously evolving Recurrent State, rather than storing raw tokens in a fixed window.
• Kurtis-EON1 can process streams of unlimited length, maintaining a persistent state that evolves over time without memory explosion.
• A Transformer stores every token in the KV-Cache. If you ask for the 3rd word from 10,000 tokens ago, it has perfect fidelity.
• The Recurrent State has a fixed size (e.g., 1024 dimensions). If you feed it 1 Billion tokens, it physically cannot store 1 Billion distinct facts in a 1024-float vector.

For comparison:

• Transformer: A photographic memory, but it faints after 1 hour.
• Kurtis-EON1: Attempts to mimic human memory.

Infinite Context vs. Lossy Recall:

Think of the model like human memory. You can live for 80 years (Infinite Context), but you don't remember exactly what you ate for breakfast in Berlin on February 2, 2016. Or why you were working on LSTM/RNNs at that time, in an empty flat. Trying to build a chatbot because you felt alone and you... You remember the gist of your life. The model compresses the past into a feeling (State), rather than a recording (Cache).

Work in Progress: This model is currently under active development.

Overview

Kurtis-EON1 is an experimental ~400M parameter language model based on a custom Recurrent State Architecture.

Data & Status

• Architecture: Hybrid (codename: Echo-DSRN)
• Base: Trained from scratch on FineWeb-EDU (sample-10BT).
• Instruct (WIP): Currently fine-tuning on UltraChat, Cosmopedia, and custom synthetic sets.

Weights will be released upon completion of safety alignment.

• Surprise Mechanism: Incorporates a novel surprise-based gating mechanism (inspired by Google Titans).
• Gating: specific gating architecture adjustments (details confidential/WIP).

---

Base Model

Training metrics and logs are available in the logs/ directory.

Training & Validation Metrics

Train Loss	Validation Loss	Extrapolation (1024T)

Avg Train Loss	Avg Gate Activation	Surprise Lambda Grad

Learning Rate	Tokens Seen

GPU Performance

GPU Util %	Memory Alloc %	Read/Write

Power Usage	Power (W)

System Metrics

CPU Util	threads

Proc Memory (MB)	Available Memory	Sys Mem Util

---

Instruct Model

Work in progress!

Supervised Fine-Tuning (SFT)

Following pre-training, we conducted two successive supervised fine-tuning runs using the echo_hf bridge and TRL (Transformer Reinforcement Learning):

• SFT-001 (UltraChat): Initial alignment on conversational data to establish basic instruction-following capability.
• SFT-002 (Cosmopedia/OpenStax): Second iteration focused on educational datasets (Cosmopedia, OpenStax) to enhance logical reasoning.

SFT-002 Benchmark Results (Zero-Shot)

We evaluated the SFT-002 checkpoint on standard zero-shot benchmarks to assess generalization capabilities. Results demonstrate strong performance on commonsense and physical reasoning tasks, with identified areas for improvement in multi-step inference (HellaSwag).

Task	Metric	Value	Stderr
ARC-Easy	acc	0.4987	0.0103
	acc_norm	0.4407	0.0102
HellaSwag	acc	0.2899	0.0045
	acc_norm	0.3202	0.0047
PIQA	acc	0.6284	0.0113
	acc_norm	0.6115	0.0114

Key Observations:

• PIQA (Physical Commonsense): Strong performance (62.8% accuracy) indicates robust "intuition" despite the model's compact size.
• ARC-Easy (Scientific Reasoning): Competitive performance (49.9% accuracy) on grade-school science questions.
• HellaSwag (Narrative Completion): Performance (32.0% normalized accuracy) reveals a current limitation in multi-step narrative reasoning. This is a priority target for next SFT iterations.

SFT-003 Benchmark Results (Zero-Shot)

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
arc_easy	1	none	0	acc	↑	0.5063	±	0.0103
		none	0	acc_norm	↑	0.4423	±	0.0102
hellaswag	1	none	0	acc	↑	0.2938	±	0.0045
		none	0	acc_norm	↑	0.3188	±	0.0047
piqa	1	none	0	acc	↑	0.6376	±	0.0112
		none	0	acc_norm	↑	0.6186	±	0.0113

SFT-003 Text Generations / Failure Analysis

At this stage, the model is ignoring the prompt's topic and force-fitting everything into the only domain it truly understands: Biology/Health (OpenStax).

Setting: Stable (Temp=0.2, Top-P=0.85)

Category: WikiHow_Instruction Prompt: How to make a paper airplane?

Response:

1. Start with the top of your paper, then go down one side and back up the other side. This will give you an idea about how much air is going through it.
2. Now draw two circles on each side so that they are parallel. Then add more circles around the bottom circle. These should be at right angles to each other.
3. Next, use a pencil or pen to mark where the circles meet. You can do this by using a ruler or marker if needed.
4. Finally, place your hand over the center circle and press firmly against the edge of the paper until it touches the inside surface of the paper.
5. Repeat steps 6-8 for all three sides.
7. Once you have completed these steps, repeat step 9 again.
8. When finished, continue adding circles around the outside surface of the paper.
9. If necessary, adjust the size of the paper depending on what type of project you want to complete. For example, if you wanted to create a large house, you could simply fold it in half lengthwise (or even into a square). Alternatively, you could also cut out smaller squares from cardboard instead of cutting them out

---

Category: WikiHow_Instruction Prompt: How to save money as a student?

Response:

1. Get the right amount of sleep each night, and don’t get too tired during study time. Sleeping is essential for learning because it helps you stay alert throughout the day. If you are not getting enough sleep, your brain will be less active which can lead to poor concentration and memory loss.
2. Don’t use electronic devices at home or work; instead, try using them in bedtime. These devices emit blue light that makes us feel drowsy and causes headaches.
3. Avoid screens before going to bed. Studies show that screen-based activities such as reading, watching TV, playing video games, or taking a short nap can cause stress hormones in our bodies which affects our mood and behavior. Instead, choose digital devices like smartphones, tablets, or laptops when possible.
4. Use natural remedies whenever possible: Natural remedies include herbs, spices, fruits, vegetables, whole grains, nuts, seeds, and oils. Try these natural remedies if you have trouble falling asleep or waking up early.
5. Stay hydrated: Drinking plenty of water keeps your body hydr

---

Category: WikiHow_Instruction Prompt: How to improve your focus while studying?

Response:

1. Get a good night’s sleep
Sleep is essential for our brain and body functions, so getting enough quality sleep can help you stay sharp during the examination process. Sleep helps regulate hormones that affect memory, concentration, alertness, and overall well-being. It also improves blood flow throughout the body, which in turn reduces stress levels.
2. Eat healthy foods
Eating nutritious meals regularly will ensure proper digestion, absorption of vitamins, minerals, and antioxidants. Aim for at least 30 minutes before bedtime to promote better sleep habits.
3. Exercise regularly
Regular exercise has been shown to boost metabolism, reduce weight gain, and lower cholesterol levels. Regular physical activity releases endorphins (the feel-good chemical) into the brain, helping us feel more energetic and focused on tasks we need to do daily.
4. Stay hydrated
Water plays an important role in maintaining optimal mental function, especially when it comes to cognitive performance. Drinking plenty of water ensures adequate hydration throughout the day,

---

Category: WikiHow_Instruction Prompt: How to care for a houseplant?

Response:

Houseplants are great companions for people who have limited space. They can be grown in any size, but they need room and sunlight. You should also consider the amount of water you use during transpiration. This will help keep your plants healthy and happy.
You may want to add some compost or manure into the soil around your houseplant so that it doesn’t dry out quickly. Compost helps break down organic matter faster than fertilizer. It also contains nutrients such as nitrogen, phosphorus, potassium, magnesium, copper, zinc, iron, manganese, molybdenum, and selenium. These elements make up about 10% of your garden’s total nutrition value!
Adding more mulch is another way to increase its overall nutritional content. Mulches act like natural drainage systems by absorbing moisture from the ground, helping prevent erosion and keeping weeds at bay. Adding too much mulch can cause excessive heat loss through the plant, which could lead to root rot. Instead, opt for loose-leaf bark or strawberries instead.
Water

---

Category: WikiHow_Instruction Prompt: How to plan a weekend trip?

Response:

A daytrip is an exciting and challenging experience that can be enjoyed by many people. It offers unique opportunities for relaxation, socialization, and learning new skills. However, it also comes with its own set of risks and benefits. Here are some tips on how you can prepare your trip:
- Plan ahead: Before heading off in the morning or after dinner, make sure everyone has their backpack ready beforehand so they don’t miss out on any crucial moments during the journey. This will help ensure safety while traveling and reduce stress levels throughout the entire process.
- Stay hydrated: Drinking plenty of water helps flush out body fluids from the skin and prevents dehydration. Aim for at least 8 glasses per person per day (depending on age), ideally between 12 and 15 cups.
- Wear lightweight clothing: Lightweight clothing reduces pressure points caused by windblown dust particles. Consider wearing loose fitting clothes when hiking or biking around town.
- Practice good posture: Keep your head elevated and lean forward slightly towards your chest as much as possible. Ensure proper

---

Developed by ethicalabs.ai