unsloth

/

gpt-oss-120b

like 15

Follow

Unsloth AI 11.2k

Something is wrong with Q5_K_M. So many tokens are just "missing". I am running the llama.cpp webui but also see it from directly interacting with the model. Some tokens just appear to be dropped or something! It affects the output quality, since it produces Python programs which do not work. Spot the two missing tokens in this snippet:

    def delete_task(self, task_id: int) ->:
        """
 Delete a task by its primary key.

        Args:
            task_id: The integer identifier of the task.
        """
        sql = "DELETE FROM tasks WHERE id = ?"
        with self._lock, self._conn:  # type: ignore[misc]
            try:
                cur = self._conn.execute(sql, (task_id,))  # type: ignore[union-attr]
                if cur.rowcount == 0:
                    raise DatabaseError(f"No task with id={task_id} to delete.")
                logging.getLogger(__name__).info("Deleted task=%s", task_id)
            except sqlite3.Error as exc:
                raise DatabaseError(f"Failed to delete task: {exc}") from exc