GiuLeo01 commited on
Commit
48c9e8e
·
verified ·
1 Parent(s): 945c31d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -17,9 +17,9 @@ This model is a prototype of a large language model specifically fine-tuned for
17
  This model was fine-tuned briefly, without any human-labeled data and using only a single consumer GPU. Despite these clear constraints, the training process led to a 400% boost in performance on tasks involving simple to moderately complex fortran program generation (HumanEval-like). Compilation errors dropped as well, and the model now performs close to much larger general-purpose models that weren’t specifically trained for this task.
18
 
19
  ## Evaluation
20
- Due to the lack of existing benchmarks for Fortran code generation, a quick-and-dirty adaptation of the HumanEval benchmark was created for Fortran in order to evaluate the model. This benchmark is currently under review and will be released publicly at a later date.
21
 
22
- According to the current demo version of the FortranHumanEval benchmark:
23
 
24
  | Model | pass@1 | Compile Error Rate |
25
  |----------------------|--------|--------------------|
 
17
  This model was fine-tuned briefly, without any human-labeled data and using only a single consumer GPU. Despite these clear constraints, the training process led to a 400% boost in performance on tasks involving simple to moderately complex fortran program generation (HumanEval-like). Compilation errors dropped as well, and the model now performs close to much larger general-purpose models that weren’t specifically trained for this task.
18
 
19
  ## Evaluation
20
+ Due to the lack of existing benchmarks for Fortran code generation, a quick-and-dirty adaptation of the HumanEval benchmark was created for Fortran in order to evaluate the model. **Fortran HumanEval is available at https://github.com/GiuLeo01/Fortran-HumanEval**.
21
 
22
+ According to the FortranHumanEval benchmark:
23
 
24
  | Model | pass@1 | Compile Error Rate |
25
  |----------------------|--------|--------------------|