jackboyla/Phi-3-mini-4k-instruct-graph-GGUF

• Model creator: EmergentMethods
• Original model: EmergentMethods/Phi-3-mini-4k-instruct-graph

Description

jackboyla/Phi-3-mini-4k-instruct-graph-GGUF contains GGUF format model files for EmergentMethods/Phi-3-mini-4k-instruct-graph.

How to Get Started with the Model (Sample inference code)

This code snippets show how to get quickly started with running the model on a CPU with ollama:

# install ollama
ollama pull hf.co/jackboyla/Phi-3-mini-4k-instruct-graph-GGUF:Q8_0

import requests
import json

messages = [ 
    {"role": "system", "content": """
A chat between a curious user and an artificial intelligence Assistant. The Assistant is an expert at identifying entities and relationships in text. The Assistant responds in JSON output only.

The User provides text in the format:

-------Text begin-------
<User provided text>
-------Text end-------

The Assistant follows the following steps before replying to the User:

1. **identify the most important entities** The Assistant identifies the most important entities in the text. These entities are listed in the JSON output under the key "nodes", they follow the structure of a list of dictionaries where each dict is:

"nodes":[{"id": <entity N>, "type": <type>, "detailed_type": <detailed type>}, ...]

where "type": <type> is a broad categorization of the entity. "detailed type": <detailed_type>  is a very descriptive categorization of the entity.

2. **determine relationships** The Assistant uses the text between -------Text begin------- and -------Text end------- to determine the relationships between the entities identified in the "nodes" list defined above. These relationships are called "edges" and they follow the structure of:

"edges":[{"from": <entity 1>, "to": <entity 2>, "label": <relationship>}, ...]

The <entity N> must correspond to the "id" of an entity in the "nodes" list.

The Assistant never repeats the same node twice. The Assistant never repeats the same edge twice.
The Assistant responds to the User in JSON only, according to the following JSON schema:

{"type":"object","properties":{"nodes":{"type":"array","items":{"type":"object","properties":{"id":{"type":"string"},"type":{"type":"string"},"detailed_type":{"type":"string"}},"required":["id","type","detailed_type"],"additionalProperties":false}},"edges":{"type":"array","items":{"type":"object","properties":{"from":{"type":"string"},"to":{"type":"string"},"label":{"type":"string"}},"required":["from","to","label"],"additionalProperties":false}}},"required":["nodes","edges"],"additionalProperties":false}
     """}, 
    {"role": "user", "content": """
-------Text begin-------
OpenAI is an American artificial intelligence (AI) research organization founded in December 2015 and headquartered in San Francisco, California. Its mission is to develop "safe and beneficial" artificial general intelligence, which it defines as "highly autonomous systems that outperform humans at most economically valuable work".[4] As a leading organization in the ongoing AI boom,[5] OpenAI is known for the GPT family of large language models, the DALL-E series of text-to-image models, and a text-to-video model named Sora.[6][7] Its release of ChatGPT in November 2022 has been credited with catalyzing widespread interest in generative AI.
-------Text end-------
"""}
] 

# Define the API endpoint
url = "http://localhost:11434/api/chat"

# Prepare the request payload
payload = {
    "model": "hf.co/jackboyla/Phi-3-mini-4k-instruct-graph-GGUF:Q8_0",
    "messages": messages,
    "stream": False
}

# Send the POST request
response = requests.post(url, json=payload)

# Print the response
print(response.status_code)
out = json.loads(response.content.decode('utf-8'))['message']['content']
print(json.dumps(json.loads(out), indent=2))

Output:

{
    "nodes": [
        {
            "id": "OpenAI",
            "type": "organization",
            "detailed_type": "ai research organization"
        },
        {
            "id": "GPT family",
            "type": "technology",
            "detailed_type": "large language models"
        },
        {
            "id": "DALL-E series",
            "type": "technology",
            "detailed_type": "text-to-image models"
        },
        {
            "id": "Sora",
            "type": "technology",
            "detailed_type": "text-to-video model"
        },
        {
            "id": "ChatGPT",
            "type": "technology",
            "detailed_type": "generative ai"
        },
        {
            "id": "San Francisco",
            "type": "location",
            "detailed_type": "city"
        },
        {
            "id": "California",
            "type": "location",
            "detailed_type": "state"
        },
        {
            "id": "December 2015",
            "type": "date",
            "detailed_type": "foundation date"
        },
        {
            "id": "November 2022",
            "type": "date",
            "detailed_type": "release date"
        }
    ],
    "edges": [
        {
            "from": "OpenAI",
            "to": "San Francisco",
            "label": "headquartered in"
        },
        {
            "from": "San Francisco",
            "to": "California",
            "label": "located in"
        },
        {
            "from": "OpenAI",
            "to": "December 2015",
            "label": "founded in"
        },
        {
            "from": "OpenAI",
            "to": "GPT family",
            "label": "developed"
        },
        {
            "from": "OpenAI",
            "to": "DALL-E series",
            "label": "developed"
        },
        {
            "from": "OpenAI",
            "to": "Sora",
            "label": "developed"
        },
        {
            "from": "OpenAI",
            "to": "ChatGPT",
            "label": "released"
        },
        {
            "from": "ChatGPT",
            "to": "November 2022",
            "label": "released in"
        }
    ]
}

About GGUF

GGUF is a new format introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp.

Here is an incomplete list of clients and libraries that are known to support GGUF:

• llama.cpp. The source project for GGUF. Offers a CLI and a server option.
• llama-cpp-python, a Python library with GPU accel, LangChain support, and OpenAI-compatible API server.
• LM Studio, an easy-to-use and powerful local GUI for Windows and macOS (Silicon), with GPU acceleration. Linux available, in beta as of 27/11/2023.
• text-generation-webui, the most widely used web UI, with many features and powerful extensions. Supports GPU acceleration.
• KoboldCpp, a fully featured web UI, with GPU accel across all platforms and GPU architectures. Especially good for story telling.
• GPT4All, a free and open source local running GUI, supporting Windows, Linux and macOS with full GPU accel.
• LoLLMS Web UI, a great web UI with many interesting and unique features, including a full model library for easy model selection.
• Faraday.dev, an attractive and easy to use character-based chat GUI for Windows and macOS (both Silicon and Intel), with GPU acceleration.
• candle, a Rust ML framework with a focus on performance, including GPU support, and ease of use.
• ctransformers, a Python library with GPU accel, LangChain support, and OpenAI-compatible AI server. Note, as of time of writing (November 27th 2023), ctransformers has not been updated in a long time and does not support many recent models.

Special thanks

🙏 Special thanks to Georgi Gerganov and the whole team working on llama.cpp for making all of this possible. And thanks to MaziyarPanahi for the README template