Bry14 commited on
Commit
f574b68
·
verified ·
1 Parent(s): 7bf817d

Upload folder using huggingface_hub

Browse files
Files changed (7) hide show
  1. MLmodel +27 -0
  2. README.md +98 -0
  3. conda.yaml +26 -0
  4. model.lgb +0 -0
  5. model.onnx +3 -0
  6. python_env.yaml +7 -0
  7. requirements.txt +19 -0
MLmodel ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ artifact_path: /content/mlruns/1/models/m-8a7257ac0857427e9daa6ba9105c4f56/artifacts
2
+ flavors:
3
+ lightgbm:
4
+ code: null
5
+ data: model.lgb
6
+ lgb_version: 4.6.0
7
+ model_class: lightgbm.basic.Booster
8
+ serialization_format: skops
9
+ skops_trusted_types:
10
+ - collections.OrderedDict
11
+ - lightgbm.sklearn.LGBMRegressor
12
+ - lightgbm.sklearn.LGBMClassifier
13
+ - lightgbm.basic.Booster
14
+ python_function:
15
+ data: model.lgb
16
+ env:
17
+ conda: conda.yaml
18
+ virtualenv: python_env.yaml
19
+ loader_module: mlflow.lightgbm
20
+ python_version: 3.12.13
21
+ mlflow_version: 3.14.0
22
+ model_id: m-8a7257ac0857427e9daa6ba9105c4f56
23
+ model_size_bytes: 4584488
24
+ model_uuid: m-8a7257ac0857427e9daa6ba9105c4f56
25
+ prompts: null
26
+ run_id: a23d317e18624b96856fa72e526548bf
27
+ utc_time_created: '2026-06-27 18:10:13.388419'
README.md ADDED
@@ -0,0 +1,98 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ - hi
5
+ - mr
6
+ - bn
7
+ - ta
8
+ - te
9
+ tags:
10
+ - text-classification
11
+ - onnx
12
+ - intent-classification
13
+ - customer-support
14
+ - multilingual
15
+ datasets:
16
+ - bitext-cs-train
17
+ ---
18
+
19
+ # Agent Intent Classification Model
20
+
21
+ This model is an ONNX-based classifier designed to route customer support queries to the appropriate agent. It was trained on the `bitext-cs-train.csv` dataset and supports both English and multiple Indian languages.
22
+
23
+ ## Preprocessing
24
+
25
+ To use this model, the input text must first be converted into dense vector embeddings using the `ibm-granite/granite-embedding-97m-multilingual-r2` model from Hugging Face.
26
+
27
+ ```python
28
+ from langchain_huggingface import HuggingFaceEmbeddings
29
+ import numpy as np
30
+
31
+ # 1. Load the embedding model
32
+ embeddings_model = HuggingFaceEmbeddings(model_name="ibm-granite/granite-embedding-97m-multilingual-r2")
33
+
34
+ # 2. Embed the text
35
+ text = "I need help with my order."
36
+ embedding = embeddings_model.embed_query(text)
37
+
38
+ # 3. Convert to float32 numpy array
39
+ X_dense = np.asarray([embedding], dtype=np.float32)
40
+ ```
41
+
42
+ ## Inference
43
+
44
+ Once the input is preprocessed into an embedding vector, it can be passed to the ONNX model for inference.
45
+
46
+ ```python
47
+ import onnxruntime as rt
48
+
49
+ # 1. Load the ONNX model
50
+ sess = rt.InferenceSession("model.onnx")
51
+ input_name = sess.get_inputs()[0].name
52
+
53
+ # 2. Run inference
54
+ preds, probs = sess.run(None, {input_name: X_dense})
55
+
56
+ # 3. Map prediction to string class
57
+ classes = ['billing', 'order', 'product', 'shipping', 'undefined']
58
+ predicted_class = classes[preds[0]]
59
+
60
+ print(f"Predicted Agent: {predicted_class}")
61
+ ```
62
+
63
+ ## Classes
64
+
65
+ The model predicts one of the following 5 classes:
66
+ - `billing`
67
+ - `order`
68
+ - `product`
69
+ - `shipping`
70
+ - `undefined`
71
+
72
+
73
+ # ONNX Model Multilingual Evaluation Report (Target: Agent)
74
+
75
+ This report evaluates the ONNX model's ability to predict the `agent` column across different Indian languages and English. Embeddings used: `ibm-granite/granite-embedding-97m-multilingual-r2`.
76
+
77
+ ## Overall Metrics
78
+
79
+ - **Accuracy**: 0.6914
80
+ - **Precision (Weighted)**: 0.7091
81
+ - **Recall (Weighted)**: 0.6914
82
+ - **F1 Score (Weighted)**: 0.6874
83
+
84
+ ## Metrics by Language
85
+
86
+ | Language | Samples | Accuracy | Precision | Recall | F1 Score |
87
+ |---|---|---|---|---|---|
88
+ | en | 54 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
89
+ | hi | 54 | 0.9074 | 0.9235 | 0.9074 | 0.9055 |
90
+ | bn | 54 | 0.7593 | 0.7661 | 0.7593 | 0.7497 |
91
+ | te | 54 | 0.5741 | 0.5756 | 0.5741 | 0.4982 |
92
+ | ta | 54 | 0.2407 | 0.2923 | 0.2407 | 0.2372 |
93
+ | mr | 54 | 0.6667 | 0.6665 | 0.6667 | 0.6623 |
94
+
95
+ ## Production Eligibility Analysis
96
+
97
+ For the model to be eligible for production in Indian languages, we expect the F1 score in the translated languages to be comparable to English. If scores are significantly lower, the ONNX model (which was likely trained on English embeddings) might not generalize well even with multilingual embeddings, or it needs fine-tuning on the multilingual embedding space.
98
+
conda.yaml ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ channels:
2
+ - conda-forge
3
+ dependencies:
4
+ - python=3.12.13
5
+ - pip<=24.1.2
6
+ - pip:
7
+ - mlflow==3.14.0
8
+ - bottleneck==1.4.2
9
+ - cupy-cuda12x==14.0.1
10
+ - defusedxml==0.7.1
11
+ - distributed-ucxx-cu12==0.48.0
12
+ - graphviz==0.21
13
+ - lightgbm==4.6.0
14
+ - matplotlib==3.10.0
15
+ - mmh3==5.2.1
16
+ - numpy==2.0.2
17
+ - optuna==4.9.0
18
+ - pandas==2.2.2
19
+ - pyarrow==18.1.0
20
+ - python-snappy==0.7.3
21
+ - scikit-learn==1.6.1
22
+ - scipy==1.16.3
23
+ - xarray==2025.12.0
24
+ - xxhash==3.7.0
25
+ - zstandard==0.25.0
26
+ name: mlflow-env
model.lgb ADDED
The diff for this file is too large to render. See raw diff
 
model.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1a722978f645d4c7bfb49f50c4d22bffda871bf58513f6b55484ea26b34fc57e
3
+ size 3060619
python_env.yaml ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ python: 3.12.13
2
+ build_dependencies:
3
+ - pip==24.1.2
4
+ - setuptools==75.2.0
5
+ - wheel==0.47.0
6
+ dependencies:
7
+ - -r requirements.txt
requirements.txt ADDED
@@ -0,0 +1,19 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ mlflow==3.14.0
2
+ bottleneck==1.4.2
3
+ cupy-cuda12x==14.0.1
4
+ defusedxml==0.7.1
5
+ distributed-ucxx-cu12==0.48.0
6
+ graphviz==0.21
7
+ lightgbm==4.6.0
8
+ matplotlib==3.10.0
9
+ mmh3==5.2.1
10
+ numpy==2.0.2
11
+ optuna==4.9.0
12
+ pandas==2.2.2
13
+ pyarrow==18.1.0
14
+ python-snappy==0.7.3
15
+ scikit-learn==1.6.1
16
+ scipy==1.16.3
17
+ xarray==2025.12.0
18
+ xxhash==3.7.0
19
+ zstandard==0.25.0