Upload 78 files

.gitattributes

@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+data/model/medical-image-classification.keras filter=lfs diff=lfs merge=lfs -text
+data/model/model.keras filter=lfs diff=lfs merge=lfs -text
+data/model/multi-classification.keras filter=lfs diff=lfs merge=lfs -text
+data/tokenizer/multi-classification-tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,13 +1,221 @@
----
-title: Ngt Ai Platfomr
-emoji: 🏃
-colorFrom: blue
-colorTo: yellow
-sdk: gradio
-sdk_version: 4.21.0
-app_file: app.py
-pinned: false
-license: apache-2.0
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+
+# NGT AI Platform
+
+La piattaforma si propone di esporre i seguenti moduli: 
+1. binary classification di un testo fornito in input
+2. image classification di una immagine fornita in input (Classi : Basket, Bowling, Calcio, Golf, Hockey, Rugby, Volley, Tennis)
+3. multilabel classification di un testo fornito in input (Classi: alt.atheism, comp.graphics, comp.os.ms-windows.misc, comp.sys.ibm.pc.hardware, comp.sys.mac.hardware, comp.windows.x, misc.forsale, rec.autos, rec.motorcycles, rec.sport.baseball, rec.sport.hockey, sci.crypt, sci.electronics, sci.med, sci.space, soc.religion.christian, talk.politics.guns, talk.politics.mideast, talk.politics.misc, talk.religion.misc)
+
+
+## Required
+
+Prima di procedere è necessario installare anaconda utilizzando la seguente [guida](https://docs.anaconda.com/free/anaconda/install/linux/)
+
+La lemmatizzazione del testo viene eseguita con la libreria [spacy](https://spacy.io/usage).  
+Procedere con i seguenti passaggi
+ 
+```bash
+  pip install -U pip setuptools wheel
+  pip install -U spacy
+  python -m spacy download it_core_news_lg
+```
+
+Fondamentale installare anche la libreria tensorflow
+
+```bash
+  pip install tensorflow
+```
+    
+## Run Locally
+
+Clona il progetto
+
+```bash
+  git clone git@github.com:gaeparente/ngt-ai-platform.git
+```
+
+Installa il micro-framework Flask
+
+```bash
+  python -m pip install flask
+```
+
+Installa libreria CORS di Flask
+
+```bash
+  pip install flask_cors
+```
+
+Posizionati nella directory del file app.py
+
+```bash
+  cd ngt-ai-platform/
+```
+
+Avvia il server
+
+```bash
+  flask run
+```
+
+I moduli saranno quindi raggiungibili: 
+1. binary classification all'indirizzo http://127.0.0.1:5000/binary-classification
+2. image classification all'indirizzo http://127.0.0.1:5000/image-classification 
+3. multilabel classification all'indirizzo http://127.0.0.1:5000/multi-classification 
+
+
+## Usage/Examples Binary classification
+
+Effettuare una chiamata POST all'indirizzo indicato in precedenza. Il body dovrà essere in formato form-data con le seguenti property:
+1. text (required) -> contenente la sentence per cui si richiede la classificazione
+2. model (optional) -> contenente il file del modello (.keras o .h5)
+3. token (optional) -> contenente il file del tokenizer (.json)
+
+La risposta sarà quindi
+
+```json
+{
+    "lemma": "che posto ragazzo ! uno cucina ricercare in piccolo cortile di altro tempo . bello , buone , bravissimo . prenotare con largo anticipo .",
+    "percent": "99.95895028114319",
+    "sentiment": "POSITIVE"
+}
+```
+
+## Usage/Examples Image Classification
+
+Effettuare una chiamata POST all'indirizzo indicato in precedenza. Il body dovrà essere in formato form-data con le seguenti property:
+1. image (required) -> contenente il file per cui si richiede la classificazione
+2. model (optional) -> contenente il file del modello (.keras o .h5)
+
+La risposta sarà quindi
+
+```json
+[
+    {
+        "classe": "Basket",
+        "percent": "0.02414761"
+    },
+    {
+        "classe": "Bowling",
+        "percent": "0.12304398"
+    },
+    {
+        "classe": "Calcio",
+        "percent": "0.00155318"
+    },
+    {
+        "classe": "Golf",
+        "percent": "0.00484183"
+    },
+    {
+        "classe": "Hockey",
+        "percent": "0.05853807"
+    },
+    {
+        "classe": "Rugby",
+        "percent": "0.26000361"
+    },
+    {
+        "classe": "Volley",
+        "percent": "99.52762127"
+    },
+    {
+        "classe": "Tennis",
+        "percent": "0.00024511"
+    }
+]
+```
+
+## Usage/Examples Multilabel classification
+
+Effettuare una chiamata POST all'indirizzo indicato in precedenza. Il body dovrà essere in formato form-data con le seguenti property:
+1. text (required) -> contenente la sentence per cui si richiede la classificazione
+2. model (optional) -> contenente il file del modello (.keras o .h5)
+3. token (optional) -> contenente il file del tokenizer (.json)
+
+La risposta sarà quindi
+
+```json
+[
+    {
+        "classe": "alt.atheism",
+        "percent": "20.58875114"
+    },
+    {
+        "classe": "comp.graphics",
+        "percent": "5.57006039"
+    },
+    {
+        "classe": "comp.os.ms-windows.misc",
+        "percent": "1.00294100"
+    },
+    {
+        "classe": "comp.sys.ibm.pc.hardware",
+        "percent": "0.17852880"
+    },
+    {
+        "classe": "comp.sys.mac.hardware",
+        "percent": "0.24781623"
+    },
+    {
+        "classe": "comp.windows.x",
+        "percent": "3.20503265"
+    },
+    {
+        "classe": "misc.forsale",
+        "percent": "0.16137564"
+    },
+    {
+        "classe": "rec.autos",
+        "percent": "0.23865439"
+    },
+    {
+        "classe": "rec.motorcycles",
+        "percent": "0.35177895"
+    },
+    {
+        "classe": "rec.sport.baseball",
+        "percent": "1.18482364"
+    },
+    {
+        "classe": "rec.sport.hockey",
+        "percent": "0.21046386"
+    },
+    {
+        "classe": "sci.crypt",
+        "percent": "4.29985709"
+    },
+    {
+        "classe": "sci.electronics",
+        "percent": "2.09880602"
+    },
+    {
+        "classe": "sci.med",
+        "percent": "19.70048994"
+    },
+    {
+        "classe": "sci.space",
+        "percent": "5.71478717"
+    },
+    {
+        "classe": "soc.religion.christian",
+        "percent": "11.07885465"
+    },
+    {
+        "classe": "talk.politics.guns",
+        "percent": "1.57866161"
+    },
+    {
+        "classe": "talk.politics.mideast",
+        "percent": "1.79922581"
+    },
+    {
+        "classe": "talk.politics.misc",
+        "percent": "3.07453331"
+    },
+    {
+        "classe": "talk.religion.misc",
+        "percent": "17.71455258"
+    }
+]
+```

app.py

@@ -0,0 +1,130 @@
+<<<<<<< HEAD
+from flask import Flask, request
+from flask_cors import CORS
+from modules.binary_classification import binary_classification as binary
+from modules.image_classification import image_classification as image
+from modules.multilabel_classification import multi_classification as multi
+
+app = Flask(__name__)
+CORS(app)
+
+@app.post("/binary-classification")
+def binary_classification():
+	return binary(request)
+
+@app.post("/image-classification")
+def image_classification():
+	return image(request)
+
+@app.post("/multi-classification")
+def multilabel_classification():
+	return multi(request)
+
+
+=======
+from flask import Flask, request, jsonify
+
+app = Flask(__name__)
+
+import keras.models as models
+import spacy
+import string
+import re
+import json
+from nltk.corpus import stopwords
+
+nlp = spacy.load("it_core_news_sm")
+BASE_PATH = 'data/'
+MODEL = BASE_PATH + 'model/'
+VOCAB = BASE_PATH + 'vocab.txt'
+
+def load_doc(filename):
+	# open the file as read only 
+	file = open(filename, 'r')
+	# read all text
+	text = file.read()
+	# close the file 
+	file.close()
+	return text
+
+def load_vocab():
+	vocab=load_doc(VOCAB)
+	vocab=vocab.split()
+	vocab = set(vocab)
+	return vocab
+
+from keras.preprocessing.text import Tokenizer
+from keras.preprocessing.text import tokenizer_from_json
+
+def load_tokenizer():
+	with open(MODEL + 'tokenizer.json') as f:
+		data = json.load(f)
+		tokenizer = Tokenizer()
+		tokenizer = tokenizer_from_json(data)
+	return tokenizer
+
+def lemma_text(text):
+	doc = nlp(text)
+	lemmatized_tokens = [token.lemma_ for token in doc]
+	lemmatized_text = ' '.join(lemmatized_tokens)
+	return lemmatized_text
+
+def clean_doc(text):
+	doc = lemma_text(text)
+	# split into tokens by white space
+	tokens = doc.split()
+	# prepare regex for char filtering
+	re_punc = re.compile('[%s]' % re.escape(string.punctuation)) # remove punctuation from each word
+	tokens = [re_punc.sub('', w) for w in tokens]
+	# remove remaining tokens that are not alphabetic
+	tokens = [word for word in tokens if word.isalpha()]
+	# filter out stop words
+	stop_words = set(stopwords.words('italian'))
+	tokens = [w for w in tokens if not w in stop_words]
+	# filter out short tokens
+	tokens = [word for word in tokens if len(word) > 1]
+	# rimuovo le parole show e less
+	tokens = [word for word in tokens if word not in ('show', 'less')]
+	return tokens
+
+def predict_sentiment(review, vocab, tokenizer, model):
+    # clean
+    tokens = clean_doc(review)
+    # filter by vocab
+    tokens = [w for w in tokens if w in vocab] 
+    # convert to line
+    line = ' '.join(tokens)
+    # encode
+    encoded = tokenizer.texts_to_matrix([line], mode='tfidf')
+    # predict sentiment
+    yhat = model.predict(encoded, verbose=0)
+    # retrieve predicted percentage and label
+    percent_pos = yhat[0,0]
+    if round(percent_pos) == 0:
+        return (1-percent_pos), 'NEGATIVE'
+    return percent_pos, 'POSITIVE'
+
+def predict(text) :
+    model = models.load_model(MODEL + 'model.keras', compile=False)
+    vocab = load_vocab()
+    tokenizer = load_tokenizer()
+    doc = lemma_text(text)
+    percent, sentiment = predict_sentiment(doc, vocab, tokenizer, model)
+    print('Review: [%s]\nSentiment: %s (%.3f%%)' % (doc, sentiment, percent*100))
+    return doc, sentiment, percent
+
+@app.post("/text-classification")
+def text_classification():
+    if request.is_json:
+        text = request.get_json()
+        sentence = text["text"]
+        doc, sentiment, percent = predict(sentence)
+        response = {
+              "lemma" : doc,
+              "sentiment" : sentiment,
+              "percent" : str(percent * 100)
+		}
+        return jsonify(response)
+    return {"error": "Request must be JSON"}, 415
+      
+>>>>>>> 255b26b (first commit)