Hugging Face's logo

SurweeshSP
/
mathtok

Text Generation
Transformers
English
custom
tokenizer
symbolic-ai
mathematics
llm
reasoning
ast
compiler
nlp
deep-learning
machine-learning
mathematical-reasoning
symbolic-reasoning
tokenization
parser
artificial-intelligence
Eval Results (legacy)
Model card Files
xet
 Community
SurweeshSP commited on
Commit
f2b7e07
·
verified ·
1 Parent(s): 43e39a7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +47 -0
README.md CHANGED
@@ -7,6 +7,8 @@ language:
7
 
8
  license: mit
9
 
 
 
10
  tags:
11
  - tokenizer
12
  - symbolic-ai
@@ -16,8 +18,53 @@ tags:
16
  - ast
17
  - compiler
18
  - nlp
 
 
 
 
 
 
 
 
19
 
20
  pipeline_tag: text-generation
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
21
  ---
22
 
23
  ## Overview
 
7
 
8
  license: mit
9
 
10
+ library_name: custom
11
+
12
  tags:
13
  - tokenizer
14
  - symbolic-ai
 
18
  - ast
19
  - compiler
20
  - nlp
21
+ - deep-learning
22
+ - machine-learning
23
+ - mathematical-reasoning
24
+ - symbolic-reasoning
25
+ - tokenization
26
+ - parser
27
+ - transformers
28
+ - artificial-intelligence
29
 
30
  pipeline_tag: text-generation
31
+
32
+ datasets:
33
+ - custom-mathematical-dataset
34
+
35
+ metrics:
36
+ - semantic-density
37
+ - structural-efficiency
38
+ - symbolic-compression-ratio
39
+
40
+ model-index:
41
+ - name: MathTok
42
+ results:
43
+ - task:
44
+ type: tokenization
45
+ name: Mathematical Tokenization
46
+ dataset:
47
+ name: Custom Mathematical Benchmark
48
+ type: symbolic-math
49
+ metrics:
50
+ - type: semantic-density
51
+ value: Improved
52
+ name: Semantic Density
53
+ - type: structural-efficiency
54
+ value: Optimized
55
+ name: Structural Efficiency
56
+ - type: symbolic-compression-ratio
57
+ value: Enhanced
58
+ name: SCR
59
+
60
+ co2_eq_emissions:
61
+ emissions: 0
62
+
63
+ license_name: mit
64
+
65
+ pretty_name: MathTok
66
+
67
+ thumbnail: assets/mathtok_architecture_improvements.svg
68
  ---
69
 
70
  ## Overview