jacob-valdez commited on
Commit
0cb5921
·
verified ·
1 Parent(s): a454f57

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +42 -16
README.md CHANGED
@@ -5,62 +5,88 @@ tags:
5
  - builderbrain
6
  - compositional-ai
7
  - grammar-constrained
 
8
  - pytorch
9
  - transformers
10
  model-index:
11
- - name: builderbrain-small
12
  results: []
13
  ---
14
 
15
- # BuilderBrain Small Model
16
 
17
  BuilderBrain is a dual-rail compositional AI system that extends pretrained transformers with learned composition blocks, grammar constraints, and executable plans.
18
 
19
  ## Model Description
20
 
21
- This is a small scale BuilderBrain model trained for compositional reasoning tasks.
22
 
23
  ### Architecture
24
 
25
- - **Base Model**: GPT-2 based transformer
26
- - **Builder Rail**: Additional composition layer with discrete program skills
27
  - **Grammar Constraints**: CFG/PEG parsing for structured outputs
28
  - **Plan Validation**: DAG-based plan execution with precondition checking
29
  - **Multi-objective Training**: Lagrangian optimization with constraint satisfaction
 
 
 
 
 
 
 
 
 
30
 
31
  ### Training
32
 
33
- - **Dataset**: Compositional reasoning tasks
34
  - **Loss Functions**: Multi-objective with grammar, plan, and reuse constraints
35
- - **Training Steps**: 50 epochs
 
 
36
 
37
  ## Usage
38
 
39
  ```python
40
  from transformers import AutoTokenizer, AutoModelForCausalLM
41
 
42
- tokenizer = AutoTokenizer.from_pretrained("builderbrain_small_1759327754")
43
- model = AutoModelForCausalLM.from_pretrained("builderbrain_small_1759327754")
44
 
45
  # Grammar-constrained generation
46
- input_text = "Generate a JSON API call"
47
  inputs = tokenizer(input_text, return_tensors="pt")
48
 
49
- # Generate with grammar constraints (implementation specific)
50
- outputs = model.generate(**inputs, max_length=150)
 
 
 
 
 
 
51
  response = tokenizer.decode(outputs[0], skip_special_tokens=True)
52
  ```
53
 
 
 
 
 
 
 
 
 
54
  ## Limitations
55
 
56
- - This is a mock export for demonstration purposes
57
- - In production, models would be trained on domain-specific datasets
58
- - Grammar constraints and plan validation would be fully implemented
59
 
60
  ## Citation
61
 
62
  ```bibtex
63
- @misc{builderbrain_small,
64
  title={BuilderBrain: Dual-Rail Compositional AI System},
65
  author={BuilderBrain Team},
66
  year={2024},
 
5
  - builderbrain
6
  - compositional-ai
7
  - grammar-constrained
8
+ - dual-rail
9
  - pytorch
10
  - transformers
11
  model-index:
12
+ - name: builderbrain-tiny
13
  results: []
14
  ---
15
 
16
+ # BuilderBrain Tiny Model
17
 
18
  BuilderBrain is a dual-rail compositional AI system that extends pretrained transformers with learned composition blocks, grammar constraints, and executable plans.
19
 
20
  ## Model Description
21
 
22
+ This is a tiny scale BuilderBrain model designed for compositional reasoning tasks with formal guarantees.
23
 
24
  ### Architecture
25
 
26
+ - **Base Rail**: Frozen pretrained transformer (gpt2)
27
+ - **Builder Rail**: Additional composition layer with 8 discrete program skills
28
  - **Grammar Constraints**: CFG/PEG parsing for structured outputs
29
  - **Plan Validation**: DAG-based plan execution with precondition checking
30
  - **Multi-objective Training**: Lagrangian optimization with constraint satisfaction
31
+ - **Safety Monitoring**: Risk energy prediction and violation detection
32
+
33
+ ### Model Specifications
34
+
35
+ - **Hidden Size**: 768
36
+ - **Builder Layers**: 4
37
+ - **Program Skills**: 8
38
+ - **Alpha Cap**: 0.05
39
+ - **Grammar Constraints**: 2 active constraints
40
 
41
  ### Training
42
 
43
+ - **Dataset**: Compositional reasoning tasks with structured outputs
44
  - **Loss Functions**: Multi-objective with grammar, plan, and reuse constraints
45
+ - **Training Steps**: 5 epochs
46
+ - **Batch Size**: 2
47
+ - **Learning Rate**: 1e-4
48
 
49
  ## Usage
50
 
51
  ```python
52
  from transformers import AutoTokenizer, AutoModelForCausalLM
53
 
54
+ tokenizer = AutoTokenizer.from_pretrained("builderbrain_tiny_1759367360")
55
+ model = AutoModelForCausalLM.from_pretrained("builderbrain_tiny_1759367360")
56
 
57
  # Grammar-constrained generation
58
+ input_text = "Generate a JSON API call for user registration"
59
  inputs = tokenizer(input_text, return_tensors="pt")
60
 
61
+ # Generate with grammar constraints and safety monitoring
62
+ outputs = model.generate(
63
+ **inputs,
64
+ max_length=150,
65
+ grammar_constraint=True,
66
+ safety_monitoring=True,
67
+ temperature=0.8
68
+ )
69
  response = tokenizer.decode(outputs[0], skip_special_tokens=True)
70
  ```
71
 
72
+ ## Capabilities
73
+
74
+ - **Compositional Reasoning**: Combines discrete skills into complex behaviors
75
+ - **Grammar Compliance**: Generates syntactically correct structured outputs
76
+ - **Safety Awareness**: Monitors and prevents harmful outputs
77
+ - **Planning**: Uses world models for multi-step reasoning
78
+ - **Constraint Satisfaction**: Maintains formal guarantees during generation
79
+
80
  ## Limitations
81
 
82
+ - Requires domain-specific training data for optimal performance
83
+ - Grammar constraints may limit creative outputs in unconstrained domains
84
+ - Safety monitoring adds computational overhead
85
 
86
  ## Citation
87
 
88
  ```bibtex
89
+ @misc{builderbrain_tiny,
90
  title={BuilderBrain: Dual-Rail Compositional AI System},
91
  author={BuilderBrain Team},
92
  year={2024},