momeaicrypto commited on
Commit
3e7b6aa
·
verified ·
1 Parent(s): fc939e6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +24 -23
README.md CHANGED
@@ -1,39 +1,40 @@
1
- ---
2
  license: apache-2.0
3
- ---
 
4
  # MoME-A2.7B (Multi-Chain Mixture of Experts)
5
 
6
  ## Introduction
7
- **MoME** (Multi-Chain Mixture of Experts) is a specialized large language model designed to handle multi-chain transaction analysis and cross-chain data interactions. Built on a Mixture of Experts (MoE) architecture, this model aims to provide detailed, chain-specific context for multiple blockchain networks—such as Aptos, Polkadot, Ripple, and more—within a single inference environment.
8
 
9
- MoME-A2.7B is slated for **open-source release** soon. We will update this card with direct download links once the model weights and checkpoints are made publicly available.
10
 
11
  ## Model Details
12
- - **Architecture**: Mixture of Experts (MoE) based, upcycled from an existing dense LLM to specialize in multi-chain transaction parsing and domain-specific dialogue.
13
- - **Parameters**: ~14.3B total parameters, with **2.7B activated parameters** on average at runtime, allowing efficient inference for each chain “expert.
14
- - **Performance**: Targets near-parity performance with a larger 7B-class multi-chain model while using roughly **25% fewer computational resources** during training. Early benchmarks show a **1.74x** speed improvement in inference compared to more extensive 7B-class multi-chain baselines.
15
- - **Training Data**: Mixture of chain-focused textual corpora (including blockchain transaction logs, developer guides, and academic papers). For multi-chain coverage, MoME includes specialized corpora for Aptos, Polkadot, Ripple, and others.
16
 
17
  ## Requirements
18
- MoME leverages custom modules from the latest Hugging Face `transformers` library. To avoid compatibility issues, **install from source**:
19
- ```
20
  pip install git+https://github.com/huggingface/transformers
21
  ```
22
- This ensures that any custom model classes (e.g., `mome_moe`) are recognized and loaded properly.
23
 
24
  ## Usage
25
- While MoME-A2.7B can serve as a foundation for multi-chain text generation tasks, **we recommend specialized fine-tuning** (e.g., SFT, RLHF, or additional domain pretraining) to unlock its full potential for:
26
  - **Cross-chain transaction decoding**
27
- - **Chain-specific question-answering**
28
- - **Multi-chain DeFi monitoring**
29
- - **NFT contract analysis**
30
- - **General blockchain research and development**
31
 
32
  ### Basic Example
33
  ```python
34
  from transformers import AutoTokenizer, AutoModelForCausalLM
35
 
36
- # Hypothetical usage - subject to change when weights become available
37
  model_name = "momeaicrypto/mome-a2.7b"
38
  tokenizer = AutoTokenizer.from_pretrained(model_name)
39
  model = AutoModelForCausalLM.from_pretrained(model_name)
@@ -45,12 +46,12 @@ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
45
  ```
46
 
47
  ## Limitations & Disclaimer
48
- 1. **Early Release**: MoME is an ongoing project. The model weights will be made available once internal validations are complete.
49
- 2. **Chain Expertise Bias**: While MoME has specialized chain experts, certain blockchains or contract frameworks may have less representation in the training corpus, which can lead to biased or incomplete insights.
50
- 3. **Not Production-Ready**: Users should further fine-tune or adapt MoME for production applications, ensuring appropriate risk mitigation and domain validation.
51
- 4. **Responsible Use**: Please abide by your local regulations and best practices when using AI for financial or blockchain applications.
52
 
53
  ## Citation & Contact
54
- For questions or collaborations, see our upcoming GitHub repository (link to be provided) or contact the maintainers. If you use MoME in your research or production, please cite accordingly once the official white paper is released.
55
 
56
- *We look forward to releasing the MoME-A2.7B weights soon and enabling broader community engagement for robust multi-chain use cases.*
 
1
+ ```yaml
2
  license: apache-2.0
3
+ ```
4
+
5
  # MoME-A2.7B (Multi-Chain Mixture of Experts)
6
 
7
  ## Introduction
8
+ **MoME** (Multi-Chain Mixture of Experts) is a specialized large language model tailored for multi-chain transaction analysis and cross-chain data workflows. By leveraging a Mixture of Experts (MoE) architecture, MoME delivers chain-specific insights for multiple blockchain networks—such as Aptos, Polkadot, Ripple, and more—all under one inference environment.
9
 
10
+ **MoME-A2.7B** will be **open-sourced** soon. We will update this card with direct links to the weights and checkpoints when they become publicly available.
11
 
12
  ## Model Details
13
+ - **Architecture**: MoE-based, derived from a dense LLM and optimized for multi-chain transaction parsing and domain-focused conversation.
14
+ - **Parameters**: Approximately **14.3B total parameters**, with an average of **2.7B activated** at runtime, enabling efficient inference across multiple “expert” domains.
15
+ - **Performance**: Achieves performance on par with a larger 7B-class multi-chain model while requiring around **25%** fewer computational resources. Early benchmarking shows **1.74×** faster inference compared to more extensive multi-chain models.
16
+ - **Training Data**: Trained on a curated set of chain-centric corpora (e.g., on-chain logs, developer manuals, academic references). This specialized data covers Aptos, Polkadot, Ripple, and beyond.
17
 
18
  ## Requirements
19
+ MoME relies on custom modules in the latest `transformers` library from Hugging Face. For best compatibility, install from source:
20
+ ```bash
21
  pip install git+https://github.com/huggingface/transformers
22
  ```
23
+ This ensures any custom model classes (e.g., `mome_moe`) are properly registered and loaded.
24
 
25
  ## Usage
26
+ While MoME-A2.7B can provide a foundation for multi-chain text generation tasks, **targeted fine-tuning**—such as SFT, RLHF, or extended domain pretraining—is strongly recommended for:
27
  - **Cross-chain transaction decoding**
28
+ - **Chain-specific Q&A**
29
+ - **DeFi analytics across multiple blockchains**
30
+ - **NFT contract interpretation**
31
+ - **General blockchain R&D**
32
 
33
  ### Basic Example
34
  ```python
35
  from transformers import AutoTokenizer, AutoModelForCausalLM
36
 
37
+ # Example usage - subject to change once weights are released
38
  model_name = "momeaicrypto/mome-a2.7b"
39
  tokenizer = AutoTokenizer.from_pretrained(model_name)
40
  model = AutoModelForCausalLM.from_pretrained(model_name)
 
46
  ```
47
 
48
  ## Limitations & Disclaimer
49
+ 1. **Early Release**: MoME remains under development, and final weights will be shared pending internal validation.
50
+ 2. **Chain Expertise Bias**: Certain blockchains or contract types may be underrepresented in the training data, leading to potentially incomplete or biased outputs.
51
+ 3. **Production Readiness**: Further finetuning or adaptation is advised if using this model in production-critical settings.
52
+ 4. **Responsible Use**: Comply with relevant legal and ethical guidelines for AI applications in finance and blockchain.
53
 
54
  ## Citation & Contact
55
+ Questions or collaboration inquiries can be directed to our forthcoming GitHub repo (link to be provided) or directly to the maintainers. If you integrate MoME into research or production, please cite it once the official white paper becomes available.
56
 
57
+ *We look forward to releasing MoME-A2.7B and expanding the multi-chain LLM ecosystem.*