license: apache-2.0
inference: false
Model Card for Model ID
slim-sentiment is part of the SLIM ("Structured Language Instruction Model") model series, providing a set of small, specialized decoder-based LLMs, fine-tuned for function-calling.
slim-sentiment has been fine-tuned for sentiment analysis function calls, generating output consisting of a python dictionary corresponding to specified keys.
Each slim model has a corresponding 'tool' in a separate repository, e.g.,
'slim-sentiment-tool', which a 4-bit quantized gguf version of the model that is intended to be used for inference.
Inference speed and loading time is much faster with the 'tool' versions of the model, and multiple tools can be deployed concurrently and run on a local CPU-based laptop or server.
Model Description
- Developed by: llmware
- Model type: SLIM - small, specialized LLM generating structured outputs
- Language(s) (NLP): English
- License: Apache 2.0
- Finetuned from model: Tiny Llama 1B
Uses
The intended use of SLIM models is to re-imagine traditional 'hard-coded' classifiers through the use of function calls, and to provide a natural language flexible tool that can be used as decision gates and processing steps in a complex LLM-based automation workflow.
Getting Started:
model = AutoModelForCausalLM.from_pretrained("llmware/slim-sentiment")
tokenizer = AutoTokenizer.from_pretrained("llmware/slim-sentiment")
function = "classify"
params = "sentiment"
text = "That was the worst earnings call of the year. The CEO should be fired."
text = "The stock market declined yesterday as investors worried increasingly about the slowing economy."
prompt = "<human>: " + text + "\n" + f"<{function}> {params} </{function}>\n<bot>:"
inputs = tokenizer(prompt, return_tensors="pt")
start_of_input = len(inputs.input_ids[0])
outputs = model.generate(
inputs.input_ids.to('cpu'),
eos_token_id=tokenizer.eos_token_id,
pad_token_id=tokenizer.eos_token_id,
do_sample=True,
temperature=0.3,
max_new_tokens=100
)
output_only = tokenizer.decode(outputs[0][start_of_input:], skip_special_tokens=True)
print("output only: ", output_only)
Sample output:
{"sentiment": ["negative"]}
All of the SLIM models use a novel prompt instruction structured as follows:
"<human> " + {text} + "\n" +
"<{function}> " + {keys} + "</{function}>" +
"/n<bot>:"
For example, in this case, the prompt would be as follows:
"<human>" + "The stock market declined yesterday ..." + "\n" + "<classify> sentiment </classify>" + "\n<bot>:"
The model generation output will be a string in the form of a python dictionary, which can be converted as follows:
try:
output_only = ast.literal_eval(llm_string_output)
print("success - converted to python dictionary automatically")
except:
print("fail - could not convert to python dictionary automatically - ", llm_string_output)
Using as Function Call in LLMWare
We envision the slim models deployed in a pipeline/workflow/templating framework that handles the prompt packaging more elegantly.
Check out llmware for one such implementation:
from llmware.models import ModelCatalog
slim_model = ModelCatalog().load_model("llmware/slim-sentiment")
response = slim_model.function_call(text,params=["sentiment"], function="classify")
print("llmware - llm_response: ", response)
Model Card Contact
Darren Oberst & llmware team