caikybaldo999's picture
Full SFT WebCoder-100M-Instruct | 78,226,740 SFT tokens
062ffb1 verified
|
Raw
History Blame Contribute Delete
899 Bytes
metadata
language:
  - code
library_name: transformers
pipeline_tag: text-generation
base_model: caikybaldo999/webcoder-100m-html-css-js
tags:
  - html
  - css
  - javascript
  - instruct
  - sft

WebCoder-100M-Instruct

Instruction-tuned version of caikybaldo999/webcoder-100m-html-css-js specialized in HTML, CSS and JavaScript.

Training

  • Full SFT
  • Parameters: 99,894,528
  • Dataset: iamtarun/code_instructions_120k_alpaca filtered for web-development examples
  • Training examples: 30,032
  • Max sequence length: 1024
  • Tokens processed: 78,226,740
  • Supervised response tokens: 48,809,920

Loss is masked on system/user tokens and computed on assistant response tokens.

Prompt format

<|system|>
You are WebCoder...<|end|>
<|user|>
Create a responsive website...<|end|>
<|assistant|>
...

Limitations

This is a small ~100M parameter model. Review generated code before deployment.