|
|
--- |
|
|
license: apache-2.0 |
|
|
--- |
|
|
|
|
|
# Seq2Seq Transformer for Function Call Generation |
|
|
|
|
|
This repository hosts a custom-trained Seq2Seq Transformer model designed to convert natural language queries into corresponding function call representations. The model leverages an encoder-decoder Transformer architecture built from scratch using PyTorch and supports versioning to facilitate continuous improvements and updates. |
|
|
|
|
|
## Model Description |
|
|
|
|
|
- **Architecture:** |
|
|
A full Transformer-based encoder-decoder model with multi-head attention and feed-forward layers. The model incorporates sinusoidal positional encoding to capture sequential information. |
|
|
|
|
|
- **Tokenization & Vocabulary:** |
|
|
The model uses a custom-built vocabulary derived from training data. Special tokens include: |
|
|
- `<pad>` for padding, |
|
|
- `<bos>` to denote the beginning of a sequence, |
|
|
- `<eos>` to denote the end of a sequence, and |
|
|
- `<unk>` for unknown tokens. |
|
|
|
|
|
- **Training:** |
|
|
Trained on paired examples of natural language inputs and function call outputs using a cross-entropy loss function. The training process supports versioning, where each training run increments the model version, and each version is stored for reproducibility and comparison. |
|
|
|
|
|
- **Inference:** |
|
|
Greedy decoding is used to generate output sequences from an input sequence. Users can specify the model version to load the appropriate model for inference. |
|
|
|
|
|
## Intended Use |
|
|
|
|
|
This model is primarily intended for: |
|
|
- Automated function call generation from natural language instructions. |
|
|
- Enhancing natural language interfaces for code generation or task automation. |
|
|
- Integrating into virtual assistants and chatbots to execute backend function calls. |
|
|
|
|
|
## Limitations |
|
|
|
|
|
- **Data Dependency:** |
|
|
The model's performance relies on the quality and representativeness of the training data. Out-of-distribution inputs may yield suboptimal or erroneous outputs. |
|
|
|
|
|
- **Decoding Strategy:** |
|
|
The current greedy decoding approach may not always produce the most diverse or optimal outputs. Alternative strategies (e.g., beam search) might be explored for improved results. |
|
|
|
|
|
- **Generalization:** |
|
|
While the model works well on data similar to its training examples, its performance may degrade on substantially different domains or complex instructions. |
|
|
|
|
|
## Training Data |
|
|
|
|
|
The model is trained on custom datasets comprising natural language inputs paired with function call outputs. Users are encouraged to fine-tune the model on domain-specific data to maximize its utility in real-world applications. |
|
|
|
|
|
## How to Use |
|
|
|
|
|
1. **Loading a Specific Version:** |
|
|
The system supports multiple versions. Specify the model version when performing inference to load the desired model. |
|
|
|
|
|
2. **Inference:** |
|
|
Provide an input text (e.g., "Book me a flight from London to NYC") and the model will generate the corresponding function call output. |
|
|
|
|
|
3. **Publishing:** |
|
|
The model can be published to the Hugging Face Hub with version-specific details for reproducibility and community sharing. |
|
|
|
