Instructions to use girish00/ConicAI_LLM_model with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use girish00/ConicAI_LLM_model with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-0.5B-Instruct")
model = PeftModel.from_pretrained(base_model, "girish00/ConicAI_LLM_model")

Transformers

How to use girish00/ConicAI_LLM_model with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="girish00/ConicAI_LLM_model")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("girish00/ConicAI_LLM_model")
model = AutoModelForCausalLM.from_pretrained("girish00/ConicAI_LLM_model")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use girish00/ConicAI_LLM_model with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "girish00/ConicAI_LLM_model"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "girish00/ConicAI_LLM_model",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/girish00/ConicAI_LLM_model

SGLang

How to use girish00/ConicAI_LLM_model with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "girish00/ConicAI_LLM_model" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "girish00/ConicAI_LLM_model",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "girish00/ConicAI_LLM_model" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "girish00/ConicAI_LLM_model",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use girish00/ConicAI_LLM_model with Docker Model Runner:
```
docker model run hf.co/girish00/ConicAI_LLM_model
```

girish00 commited on Apr 22

Commit

4d657a7

verified ·

1 Parent(s): 99567d6

update model

Browse files

Files changed (1) hide show

README.md +130 -204

README.md CHANGED Viewed

@@ -1,281 +1,207 @@
 ---
-license: apache-2.0
-base_model: "Qwen/Qwen2.5-Coder-0.5B-Instruct"
 library_name: peft
 pipeline_tag: text-generation
 tags:
-  - code
-  - lora
-  - coding-assistant
-  - structured-output
 ---
-# ConicAI Coding LLM
 ## Model Details
-* **Model Name:** ConicAI LLM Model
-* **Developer:** GIRISH KUMAR DEWANGAN
-* **Base Model:** Qwen/Qwen2.5-Coder-0.5B-Instruct
-* **Architecture:** Transformer (Causal LM)
-* **Fine-tuning Method:** LoRA (PEFT)
-* **Task Domain:** Code Generation, Debugging, Explanation
-* **Primary Language:** Python
----
-## Model Description
-ConicAI Coding LLM is a parameter-efficient fine-tuned model optimized for structured coding tasks.
-It enhances the base model’s reasoning ability by introducing instruction-conditioned outputs and structured response generation.
-The model focuses on three key aspects:
-* **Accuracy** → Correct code generation
-* **Interpretability** → Explanation + confidence
-* **Efficiency** → Lightweight fine-tuning
----
-## Core Design Philosophy
-1. **Instruction Conditioning**
-   ```
-   Instruction → Input → Output
-   ```
-2. **Structured Output Learning**
-3. **Post-Generation Validation Awareness**
----
-## Capabilities
-* Code generation
-* Code debugging
-* Code explanation
-* Structured output generation
-* Confidence estimation
-* Hallucination detection
----
-## Output Schema
-```json
-{
-  "code": "string",
-  "explanation": "string",
-  "confidence": 0.0,
-  "important_tokens": [],
-  "relevancy_score": 0.0,
-  "hallucination": false,
-  "hallucination_check_reason": "",
-  "latency_ms": 0
-}
-```
----
-## How to Use This Model (Colab / Local)
-```python
-!pip -q install -U transformers peft accelerate huggingface_hub safetensors
-from google.colab import userdata
-HF_TOKEN = userdata.get('HF_TOKEN')
-model = "girish00/ConicAI_LLM_model"
-prompt = input("Please enter your prompt: ")
-from huggingface_hub import login, snapshot_download
-login(token=HF_TOKEN)
-repo = snapshot_download(model, token=HF_TOKEN)
-import sys
-sys.path.append(repo)
-from infer_local import build_instruction_prompt, build_structured_result
-from peft import PeftConfig, PeftModel
-from transformers import AutoTokenizer, AutoModelForCausalLM
-import torch, time, json
-cfg = PeftConfig.from_pretrained(repo)
-base = cfg.base_model_name_or_path
-tokenizer = AutoTokenizer.from_pretrained(base)
-base_model = AutoModelForCausalLM.from_pretrained(
-    base,
-    torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
-    device_map="auto"
-)
-llm = PeftModel.from_pretrained(base_model, repo)
-llm.eval()
-inputs = tokenizer(build_instruction_prompt(prompt), return_tensors="pt").to(llm.device)
-start = time.perf_counter()
-with torch.no_grad():
-    out = llm.generate(
-        **inputs,
-        max_new_tokens=320,
-        output_scores=True,
-        return_dict_in_generate=True,
-        do_sample=False,
-        pad_token_id=tokenizer.eos_token_id
-    )
-latency = int((time.perf_counter() - start) * 1000)
-gen_ids = out.sequences[0][inputs["input_ids"].shape[1]:].tolist()
-text = tokenizer.decode(gen_ids, skip_special_tokens=True)
-conf = []
-for tid, score in zip(gen_ids, out.scores):
-    probs = torch.softmax(score[0], dim=-1)
-    conf.append(float(probs[tid].item()))
-print(json.dumps(
-    build_structured_result(
-        prompt,
-        text,
-        latency,
-        tokenizer=tokenizer,
-        generated_ids=gen_ids,
-        token_confidences=conf
-    ),
-    indent=2
-))
-```
----
 ## Training Details
-### Dataset
-* Size: ~5K–10K samples
-* Instruction-based coding dataset
 ### Training Procedure
-* Method: LoRA fine-tuning
-* Framework: Transformers + PEFT
-* Precision: FP16 / Mixed
-### Training Hyperparameters
-| Parameter           | Value |
-| ------------------- | ----- |
-| Epochs              | 1–3   |
-| Batch Size          | 2     |
-| Learning Rate       | 2e-4  |
-| Max Sequence Length | 512   |
-| LoRA Rank (r)       | 8     |
-| LoRA Alpha          | 16    |
-| LoRA Dropout        | 0.05  |
----
-## Inference Configuration
-```text
-max_new_tokens = 200
-temperature = 0.2
-top_p = 0.9
-do_sample = True
-```
----
 ## Evaluation
-* Syntax validation
-* Prompt-based testing
-* Relevancy scoring
-* Hallucination detection
----
-## Strengths
-* Lightweight and efficient
-* Strong performance on structured prompts
-* Generates readable and correct Python code
-* Provides reasoning-aware outputs
----
-## Limitations
-* Sensitive to prompt structure
-* Confidence scores are heuristic
-* Limited generalization beyond training dataset
----
-## Risks & Considerations
-* Always validate generated code
-* Not suitable for critical production systems
-* May produce incorrect logic
----
-## Best Practices
-```
-Instruction:
-Input:
-Output:
-```
-* Use clear and specific prompts
-* Keep temperature low for reliability
-* Apply post-processing for cleaner output
----
-## Technical Specifications
-* Transformer-based causal language model
-* LoRA adaptation on attention layers
-* Hugging Face Transformers + PEFT
----
 ## Environmental Impact
-* Uses parameter-efficient fine-tuning
-* Lower compute compared to full fine-tuning
-* Suitable for local deployment
----
-## Intended Use
-### Direct Use
-* Coding assistant
-* Debugging support
-* Learning programming
-### Out-of-Scope Use
-* Security-critical systems
-* Autonomous production deployment
----
-## Author
-**GIRISH KUMAR DEWANGAN**
----
-## License
-Apache License 2.0
----

 ---
+base_model: Qwen/Qwen2.5-Coder-0.5B-Instruct
 library_name: peft
 pipeline_tag: text-generation
 tags:
+- base_model:adapter:Qwen/Qwen2.5-Coder-0.5B-Instruct
+- lora
+- transformers
 ---
+# Model Card for Model ID
+<!-- Provide a quick summary of what the model is/does. -->
 ## Model Details
+### Model Description
+<!-- Provide a longer summary of what this model is. -->
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+### Model Sources [optional]
+<!-- Provide the basic links for the model. -->
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+## Uses
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+### Direct Use
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+[More Information Needed]
+### Downstream Use [optional]
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+[More Information Needed]
+### Out-of-Scope Use
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+[More Information Needed]
+## Bias, Risks, and Limitations
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+[More Information Needed]
+### Recommendations
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+## How to Get Started with the Model
+Use the code below to get started with the model.
+[More Information Needed]
 ## Training Details
+### Training Data
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+[More Information Needed]
 ### Training Procedure
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+#### Preprocessing [optional]
+[More Information Needed]
+#### Training Hyperparameters
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+#### Speeds, Sizes, Times [optional]
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+[More Information Needed]
 ## Evaluation
+<!-- This section describes the evaluation protocols and provides the results. -->
+### Testing Data, Factors & Metrics
+#### Testing Data
+<!-- This should link to a Dataset Card if possible. -->
+[More Information Needed]
+#### Factors
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+[More Information Needed]
+#### Metrics
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+[More Information Needed]
+### Results
+[More Information Needed]
+#### Summary
+## Model Examination [optional]
+<!-- Relevant interpretability work for the model goes here -->
+[More Information Needed]
 ## Environmental Impact
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+## Technical Specifications [optional]
+### Model Architecture and Objective
+[More Information Needed]
+### Compute Infrastructure
+[More Information Needed]
+#### Hardware
+[More Information Needed]
+#### Software
+[More Information Needed]
+## Citation [optional]
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+**BibTeX:**
+[More Information Needed]
+**APA:**
+[More Information Needed]
+## Glossary [optional]
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+[More Information Needed]
+## More Information [optional]
+[More Information Needed]
+## Model Card Authors [optional]
+[More Information Needed]
+## Model Card Contact
+[More Information Needed]
+### Framework versions
+- PEFT 0.19.0