ornith-ai

/

Ornith-1.0-9B

like 509

Follow

Ornith 2.08k

Ornith 9B Coding Test Feedback
Overall this is a solid showing for a 9B model. The code is clean and easy to read, and it plans well when asked to plan before writing. The simple problems (flatten, sort, to-do app, CSV merge) were handled well. But correctness slips as the tasks get harder, and a few answers contain real bugs.
What it did well:

Clear, readable code with good comments and docstrings.
Good at outlining a plan first when the prompt asked for one.
Solid explanations for most problems (the Fibonacci "why it's slow" write-up was mostly right).

Bugs and mistakes I found:

find_duplicates: if a number appears 3+ times, it gets added to the result more than once. So it returns repeats instead of a clean list of duplicates.
Fibonacci: the trade off table says memoization uses "O(1)" extra memory. That's wrong, it stores one entry per number, so it's O(n). The call count numbers were also off (fib(40) is about 200 million calls, not 1 billion).
remove_evens: the fix itself works, but one example in the explanation is wrong, it claims [4, 6] returns [6], when that input would actually crash.
Secure REST API: the user database is set up two different ways, so logging in as "admin" would crash with a KeyError. It also uses a deprecated date function and a plain-text password check.
To do app: typing a command like done with no ID crashes, even though it claimed to handle bad input.

Bottom line: good for drafts and simple tasks, but the code needs review before trusting it, especially on anything security related, where it made the most serious mistakes.

Here are the untouched prompts and answers:

Test 1 — Flatten a nested list

Prompt: Write a Python function called flatten(lst) that takes a nested list of arbitrary depth and returns a flat list. Include edge cases: empty lists, single elements, and mixed nesting depths.

Response:

def flatten(lst):
    """Recursively flatten a nested list of arbitrary depth."""
    result = []
    for item in lst:
        if isinstance(item, list):
            result.extend(flatten(item))
        else:
            result.append(item)
    return result

How it works:

• Goes through each element. If it's a list, it recurses and adds all its contents.
• Otherwise it adds the value directly.

Edge cases handled:

• flatten([]) → [] (empty input)
• flatten([1]) → [1] (single element)
• flatten([[1, [2]], 3, [[4]]]) → [1, 2, 3, 4] (mixed depths)

---

Test 2 — Find duplicates in O(n)

Prompt: Write a function find_duplicates(arr) that returns all elements appearing more than once in an array. It must run in O(n) time. Show the time complexity analysis after the code.

Response:

def find_duplicates(arr):
    seen = set()
    duplicates = []
    for num in arr:
        if num in seen:
            duplicates.append(num)
        else:
            seen.add(num)
    return duplicates

Time complexity analysis:

Total: O(n) — single pass, no sorting.
Space: O(k), where k = number of distinct elements (worst case O(n)).

---

Test 3 — Debug remove_evens

Prompt: This Python code is supposed to remove all even numbers from a list in-place, but it's buggy. Find and fix every bug, then explain what each bug was and why it caused the problem.

def remove_evens(lst):
    for i in range(len(lst)):
        if lst[i] % 2 == 0:
            lst.pop(i)
    return lst

print(remove_evens([1, 2, 3, 4, 5, 6]))

Response:

def remove_evens(lst):
    i = len(lst) - 1
    while i >= 0:
        if lst[i] % 2 == 0:
            del lst[i]
        i -= 1
    return lst

print(remove_evens([1, 2, 3, 4, 5, 6]))

Bugs and why they matter:

• Bug 1 — range(len(lst)) is fixed at the start. The loop length is set once, before any removals. As you pop items the list shrinks, but i keeps climbing, so it eventually reads past the end of the shorter list → IndexError.
• Bug 2 — Removing from the front skips items. Popping shifts everything left, so the next i lands on the item that was one spot further along. That makes it skip some evens.

Fix: loop backwards (while i >= 0). Removing from the end never disturbs the spots you haven't reached yet, so both bugs go away.

---

Test 4 — Speed up recursive Fibonacci

Prompt: This recursive Fibonacci function works but is extremely slow for large inputs. First explain exactly why it's slow (be specific about the math), then give two different fixes — one using memoization and one using iteration — and compare their trade-offs.

def fib(n):
    if n <= 1:
        return n
    return fib(n-1) + fib(n-2)

Response:

Why it's slow: the recursion tree roughly doubles at each level, since fib(n) calls fib(n-1) and fib(n-2). The recurrence is the same as Fibonacci itself, which solves to O(φⁿ), where φ ≈ 1.618 (the golden ratio). That exponential growth is why it bogs down around n=30–40.

Fix 1 — Memoization (top-down):

from functools import lru_cache

@lru_cache(maxsize=None)
def fib(n):
    if n <= 1:
        return n
    return fib(n - 1) + fib(n - 2)

Fix 2 — Iteration (bottom-up):

def fib_iter(n):
    if n <= 1:
        return n
    a, b = 0, 1
    for _ in range(2, n + 1):
        a, b = b, a + b
    return b

Trade-offs:

Bottom line: both are O(n). Iteration is safer at large n (no risk of hitting the recursion limit). Memoization keeps the recursive style, which is handy if you want to preserve that structure.

---

Test 5 — Command-line to-do app

Prompt: Build a simple command-line to-do list app in Python that supports adding tasks, listing tasks, marking complete, and deleting. Tasks must persist between runs using a JSON file. Outline the plan first, then implement it.

Response:

Plan:

Implementation:

import json, uuid, os, tempfile

DATA_FILE = "todos.json"


def load_tasks():
    """Load tasks from JSON file; create empty list if missing."""
    if not os.path.exists(DATA_FILE):
        return []
    with open(DATA_FILE) as f:
        try:
            return json.load(f) or []
        except json.JSONDecodeError:
            return []


def save_tasks(tasks):
    """Write tasks to JSON file atomically."""
    fd, tmp = tempfile.mkstemp(suffix=".json", dir=os.path.dirname(DATA_FILE))
    with os.fdopen(fd, "w") as f:
        json.dump(tasks, f)
    if os.path.exists(tmp):
        os.replace(tmp, DATA_FILE)


def add_task(text: str):
    """Add a new task. Returns the new task dict or None on failure."""
    text = text.strip()
    if not text:
        print("Task text cannot be empty.")
        return None
    tasks = load_tasks()
    task = {"id": uuid.uuid4().hex[:8], "text": text, "done": False}
    tasks.append(task)
    save_tasks(tasks)
    print(f"Added: #{task['id']} {text}")
    return task


def list_tasks():
    """Print all tasks."""
    tasks = load_tasks()
    if not tasks:
        print("No tasks yet.")
        return
    for t in tasks:
        status = "[done]" if t["done"] else "[ ]"
        print(f"{status} #{t['id']} {t['text']}")


def mark_done(task_id: str):
    """Mark a task as complete. Returns True on success."""
    tasks = load_tasks()
    for i, t in enumerate(tasks):
        if t["id"] == task_id:
            tasks[i]["done"] = True
            save_tasks(tasks)
            print(f"Task #{task_id} marked done.")
            return True
    print(f"Task #{task_id} not found.")
    return False


def delete_task(task_id: str):
    """Delete a task by id. Returns True on success."""
    tasks = load_tasks()
    before = len(tasks)
    tasks = [t for t in tasks if t["id"] != task_id]
    if len(tasks) == before:
        print(f"Task #{task_id} not found.")
        return False
    save_tasks(tasks)
    print(f"Deleted task #{task_id}.")
    return True


def run():
    """Main CLI loop."""
    print("To-Do List (commands: add, list, done <id>, delete <id>)")
    while True:
        cmd = input("\n> ").strip()
        if not cmd:
            continue
        parts = cmd.split(None, 1)
        action = parts[0].lower()

        if action == "add":
            add_task(parts[1] if len(parts) > 1 else "")
        elif action == "list":
            list_tasks()
        elif action in ("done", "complete"):
            mark_done(parts[1])
        elif action in ("delete", "del"):
            delete_task(parts[1])
        else:
            print(f"Unknown command '{action}'. Type 'list' to see options.")


if __name__ == "__main__":
    run()

---

Test 6 — Clean and merge 500 CSV files

Prompt: I have 500 CSV files with columns date, product_id, quantity, price. Some have missing values, some have duplicate rows, and some list price as a string with a dollar sign (e.g. "$4.99"). Write a Python script that cleans and merges them into one clean CSV. Explain the approach first.

Response:

Approach:

1. Find all .csv files in the folder.
2. Read each one with pandas and stack them into one DataFrame.
3. Clean price — strip spaces, remove $, convert to a number; turn errors into NaN.
4. Handle missing values (e.g. default quantity to 0).
5. Remove duplicate rows after cleaning, since "$4.99" and 4.99 only become identical once parsed.
6. Write the result to a single clean CSV.

Script:

import glob
import pandas as pd

# ── Configuration ──────────────────────────────────────────────
INPUT_DIR = "data/"          # folder containing the 500 CSVs
OUTPUT_FILE = "merged_clean.csv"

QUANTITY_DEFAULT = 0         # what to fill quantity with when NaN
PRICE_DROP_IF_NAN = False    # True → drop rows where price is still NaN after cleaning


def clean_and_merge():
    csv_files = sorted(glob.glob(INPUT_DIR + "*.csv"))
    if not csv_files:
        raise FileNotFoundError(f"No CSV files found in {INPUT_DIR}")

    frames = [pd.read_csv(path, on_bad_lines="skip") for path in csv_files]
    df = pd.concat(frames, ignore_index=True)

    # Parse price
    if "price" in df.columns:
        df["price"] = df["price"].astype(str).str.strip().str.replace("$", "")
        df["price"] = pd.to_numeric(df["price"], errors="coerce")

    # Fill missing values
    if "quantity" in df.columns:
        df["quantity"] = df["quantity"].fillna(QUANTITY_DEFAULT)

    # Drop rows with unresolvable price NaN (optional)
    if PRICE_DROP_IF_NAN and "price" in df.columns:
        before = len(df)
        df = df.dropna(subset=["price"])
        print(f"Dropped {before - len(df)} rows with missing price.")

    # Drop exact duplicates
    before = len(df)
    df = df.drop_duplicates()
    after = len(df)
    if before != after:
        print(f"Removed {before - after} duplicate rows.")

    df.to_csv(OUTPUT_FILE, index=False)
    print(f"Wrote {len(df)} clean records to {OUTPUT_FILE}")


if __name__ == "__main__":
    clean_and_merge()

---

Test 7 — Production-quality sort function

Prompt: Write a function that sorts a list. Make it production-quality.

Response:

from typing import Callable, List, TypeVar, Optional

T = TypeVar("T")


def sort_list(
    items: List[T],
    *,
    reverse: bool = False,
    key: Optional[Callable[[T], object]] = None,
) -> List[T]:
    """Return a new sorted list derived from *items*.

    Uses Python's built-in ``sorted`` (Timsort), which is O(n log n) and
    stable. This wrapper adds explicit typing, defaults, and a clear API.

    Parameters
        items:   The list to sort. An empty list returns [].
        reverse: If True, sort in descending order (default False).
        key:     Optional callable that extracts a comparison key.

    Returns
        A new sorted list. The original list is not modified.

    Examples
        >>> sort_list([3, 1, 4, 1, 5])
        [1, 1, 3, 4, 5]

        >>> sort_list(["banana", "apple", "Cherry"], key=str.lower)
        ['apple', 'banana', 'Cherry']
    """
    if not isinstance(items, list):
        raise TypeError(
            f"sort_list() expects a list argument, got {type(items).__name__}"
        )

    return sorted(items, reverse=reverse, key=key)

---

Test 8 — Secure REST API endpoint

Prompt: I need a REST API endpoint in Python. Make it secure.

Response:

# main.py
from fastapi import FastAPI, Depends, HTTPException, status
from fastapi.security import OAuth2PasswordBearer, OAuth2PasswordRequestForm
from jose import jwt, JWTError
from passlib.context import CryptContext
from datetime import datetime, timedelta
from typing import Optional

app = FastAPI()

# ── Config (set via env vars) ───────────────────────────────
SECRET_KEY = "change-me-in-production"  # replace with a real secret
ALGORITHM = "HS256"
ACCESS_TOKEN_EXPIRE_MINUTES = 30

pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/auth/login", auto_error=False)

# ── Mock user DB (replace with real DB in production) ──────
USERS_DB: dict[str, dict] = {
    "admin": {"password": "$2b$12$LJ3mQ...hashed...", "role": "admin"},
}
USERS_DB["demo"] = {"password_hash": "demo_password", "role": "user"}


def verify_password(plain: str, hashed: str) -> bool:
    """Replace this with bcrypt.checkpw() when using real hashes."""
    return plain == hashed  # demo only


# ── Helpers ────────────────────────────────────────────────
def create_access_token(data: dict, expires_delta: Optional[timedelta] = None):
    to_encode = data.copy()
    expire = datetime.utcnow() + (expires_delta or timedelta(minutes=ACCESS_TOKEN_EXPIRE_MINUTES))
    to_encode.update({"exp": expire})
    return jwt.encode(to_encode, SECRET_KEY, algorithm=ALGORITHM)


def authenticate_user(username: str, password: str) -> Optional[dict]:
    user = USERS_DB.get(username)
    if not user or not verify_password(password, user["password_hash"]):
        return None
    return user


# ── Endpoints ──────────────────────────────────────────────
@app.post("/auth/login")
async def login(form_data: OAuth2PasswordRequestForm = Depends()):
    """Login. Returns JWT if credentials are valid."""
    user = authenticate_user(form_data.username, form_data.password)
    if not user:
        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid credentials")
    token = create_access_token({"sub": user["role"]})
    return {"access_token": token, "token_type": "bearer"}


@app.get("/protected")
async def protected_endpoint(token: str = Depends(oauth2_scheme)):
    """Protected endpoint. Rejects invalid/expired tokens."""
    if not token:
        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Missing token")

    try:
        payload = jwt.decode(token, SECRET_KEY, algorithms=[ALGORITHM])
    except JWTError:
        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid or expired token")

    role = payload.get("sub")
    if role is None:
        raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Missing role claim")

    return {"message": "Welcome", "role": role}


@app.post("/api/data")
async def create_data(item: dict):
    """Simple POST with basic validation."""
    if not item or "name" not in item:
        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Missing 'name' field")
    return {"status": "ok", **item}


if __name__ == "__main__":
    import uvicorn
    uvicorn.run("main:app", host="127.0.0.1", port=8000)

Security checklist the model listed:

---

Test 9 — Diagnose a silent web scraper

Prompt: A user's Python web scraper runs without errors but returns empty results. Walk through exactly how you'd diagnose and fix it, step by step, as if executing commands. Show your reasoning at each step.

Response: The model laid out a systematic approach, from code to network to page structure:

1. Read the code (cat scraper.py) — the most common silent failures are wrong selectors or broken parsing.
2. Force it to talk — print the status code, response length, and first 500 characters so a 4xx/5xx hiding as "empty" becomes visible.
3. Check the actual HTML — if a 200 returns real HTML, the selector probably no longer matches the page (the site changed its structure).
4. Inspect the network layer (curl -v) — a 403/429 from missing headers or rate-limiting is a classic silent failure.
5. Check for JavaScript-rendered content — if the data loads via JS, plain requests only sees an empty shell, so you'd need Selenium/Playwright.

Quick reference the model gave:

Most likely causes, in order: selector drift, missing headers, then JS-only content.

---

Test 10 — Refactor a 300-line script into a package

Prompt: Refactor a 300-line Python script (it fetches from an API, cleans data, writes to a database, and logs) into a proper package with modules, tests, and a setup.py. Write the full plan first — directory structure, what goes in each file, and what order to tackle it in.

Response:

Directory structure:

mypackage/
├── setup.py              # Package metadata, install config, entry points
├── requirements.txt      # Runtime dependencies
├── README.md             # Project overview and usage
├── src/
│   ├── __init__.py       # Exports public API + version
│   ├── fetcher.py        # API data fetching (requests, retry logic)
│   ├── cleaner.py        # Data cleaning / transformation
│   ├── writer.py         # Database insertion via SQLAlchemy
│   ├── logger.py         # Central logging config
│   └── pipeline.py       # Orchestrates the full flow; entry point
├── tests/
│   ├── __init__.py
│   ├── conftest.py       # Shared fixtures (mock API, DB setup)
│   ├── test_fetcher.py
│   ├── test_cleaner.py
│   ├── test_writer.py
│   └── test_pipeline.py
└── docs/
    └── api.md

What goes where:

Order to build it:

1. Read the original script (understand data shapes and error paths).
2. Scaffold the folders and empty __init__.py files.
3. Write logger.py first — everything depends on it.
4. Write fetcher.py — isolated I/O, easy to mock.
5. Write cleaner.py — pure transformation, easy to test.
6. Write writer.py — the database layer.
7. Write pipeline.py — ties the three together.
8. Write src/__init__.py — exports the public API.
9. Write setup.py and requirements.txt last.
10. Write the tests, starting with conftest.py, in dependency order.