Update README.md
Browse files
README.md
CHANGED
|
@@ -1,39 +1,40 @@
|
|
| 1 |
-
|
| 2 |
license: apache-2.0
|
| 3 |
-
|
|
|
|
| 4 |
# MoME-A2.7B (Multi-Chain Mixture of Experts)
|
| 5 |
|
| 6 |
## Introduction
|
| 7 |
-
**MoME** (Multi-Chain Mixture of Experts) is a specialized large language model
|
| 8 |
|
| 9 |
-
MoME-A2.7B
|
| 10 |
|
| 11 |
## Model Details
|
| 12 |
-
- **Architecture**:
|
| 13 |
-
- **Parameters**:
|
| 14 |
-
- **Performance**:
|
| 15 |
-
- **Training Data**:
|
| 16 |
|
| 17 |
## Requirements
|
| 18 |
-
MoME
|
| 19 |
-
```
|
| 20 |
pip install git+https://github.com/huggingface/transformers
|
| 21 |
```
|
| 22 |
-
This ensures
|
| 23 |
|
| 24 |
## Usage
|
| 25 |
-
While MoME-A2.7B can
|
| 26 |
- **Cross-chain transaction decoding**
|
| 27 |
-
- **Chain-specific
|
| 28 |
-
- **
|
| 29 |
-
- **NFT contract
|
| 30 |
-
- **General blockchain
|
| 31 |
|
| 32 |
### Basic Example
|
| 33 |
```python
|
| 34 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 35 |
|
| 36 |
-
#
|
| 37 |
model_name = "momeaicrypto/mome-a2.7b"
|
| 38 |
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
| 39 |
model = AutoModelForCausalLM.from_pretrained(model_name)
|
|
@@ -45,12 +46,12 @@ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
|
| 45 |
```
|
| 46 |
|
| 47 |
## Limitations & Disclaimer
|
| 48 |
-
1. **Early Release**: MoME
|
| 49 |
-
2. **Chain Expertise Bias**:
|
| 50 |
-
3. **
|
| 51 |
-
4. **Responsible Use**:
|
| 52 |
|
| 53 |
## Citation & Contact
|
| 54 |
-
|
| 55 |
|
| 56 |
-
*We look forward to releasing
|
|
|
|
| 1 |
+
```yaml
|
| 2 |
license: apache-2.0
|
| 3 |
+
```
|
| 4 |
+
|
| 5 |
# MoME-A2.7B (Multi-Chain Mixture of Experts)
|
| 6 |
|
| 7 |
## Introduction
|
| 8 |
+
**MoME** (Multi-Chain Mixture of Experts) is a specialized large language model tailored for multi-chain transaction analysis and cross-chain data workflows. By leveraging a Mixture of Experts (MoE) architecture, MoME delivers chain-specific insights for multiple blockchain networks—such as Aptos, Polkadot, Ripple, and more—all under one inference environment.
|
| 9 |
|
| 10 |
+
**MoME-A2.7B** will be **open-sourced** soon. We will update this card with direct links to the weights and checkpoints when they become publicly available.
|
| 11 |
|
| 12 |
## Model Details
|
| 13 |
+
- **Architecture**: MoE-based, derived from a dense LLM and optimized for multi-chain transaction parsing and domain-focused conversation.
|
| 14 |
+
- **Parameters**: Approximately **14.3B total parameters**, with an average of **2.7B activated** at runtime, enabling efficient inference across multiple “expert” domains.
|
| 15 |
+
- **Performance**: Achieves performance on par with a larger 7B-class multi-chain model while requiring around **25%** fewer computational resources. Early benchmarking shows **1.74×** faster inference compared to more extensive multi-chain models.
|
| 16 |
+
- **Training Data**: Trained on a curated set of chain-centric corpora (e.g., on-chain logs, developer manuals, academic references). This specialized data covers Aptos, Polkadot, Ripple, and beyond.
|
| 17 |
|
| 18 |
## Requirements
|
| 19 |
+
MoME relies on custom modules in the latest `transformers` library from Hugging Face. For best compatibility, install from source:
|
| 20 |
+
```bash
|
| 21 |
pip install git+https://github.com/huggingface/transformers
|
| 22 |
```
|
| 23 |
+
This ensures any custom model classes (e.g., `mome_moe`) are properly registered and loaded.
|
| 24 |
|
| 25 |
## Usage
|
| 26 |
+
While MoME-A2.7B can provide a foundation for multi-chain text generation tasks, **targeted fine-tuning**—such as SFT, RLHF, or extended domain pretraining—is strongly recommended for:
|
| 27 |
- **Cross-chain transaction decoding**
|
| 28 |
+
- **Chain-specific Q&A**
|
| 29 |
+
- **DeFi analytics across multiple blockchains**
|
| 30 |
+
- **NFT contract interpretation**
|
| 31 |
+
- **General blockchain R&D**
|
| 32 |
|
| 33 |
### Basic Example
|
| 34 |
```python
|
| 35 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 36 |
|
| 37 |
+
# Example usage - subject to change once weights are released
|
| 38 |
model_name = "momeaicrypto/mome-a2.7b"
|
| 39 |
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
| 40 |
model = AutoModelForCausalLM.from_pretrained(model_name)
|
|
|
|
| 46 |
```
|
| 47 |
|
| 48 |
## Limitations & Disclaimer
|
| 49 |
+
1. **Early Release**: MoME remains under development, and final weights will be shared pending internal validation.
|
| 50 |
+
2. **Chain Expertise Bias**: Certain blockchains or contract types may be underrepresented in the training data, leading to potentially incomplete or biased outputs.
|
| 51 |
+
3. **Production Readiness**: Further finetuning or adaptation is advised if using this model in production-critical settings.
|
| 52 |
+
4. **Responsible Use**: Comply with relevant legal and ethical guidelines for AI applications in finance and blockchain.
|
| 53 |
|
| 54 |
## Citation & Contact
|
| 55 |
+
Questions or collaboration inquiries can be directed to our forthcoming GitHub repo (link to be provided) or directly to the maintainers. If you integrate MoME into research or production, please cite it once the official white paper becomes available.
|
| 56 |
|
| 57 |
+
*We look forward to releasing MoME-A2.7B and expanding the multi-chain LLM ecosystem.*
|