bsamadi commited on
Commit
c032460
·
1 Parent(s): 2ce7bb4

Update to pixi env

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .dockerignore +60 -0
  2. .gemini/commands/speckit.analyze.toml +188 -0
  3. .gemini/commands/speckit.checklist.toml +298 -0
  4. .gemini/commands/speckit.clarify.toml +185 -0
  5. .gemini/commands/speckit.constitution.toml +86 -0
  6. .gemini/commands/speckit.implement.toml +139 -0
  7. .gemini/commands/speckit.plan.toml +93 -0
  8. .gemini/commands/speckit.specify.toml +262 -0
  9. .gemini/commands/speckit.tasks.toml +141 -0
  10. .gemini/commands/speckit.taskstoissues.toml +34 -0
  11. .gitignore +34 -2
  12. .specify/memory/constitution.md +50 -0
  13. .specify/scripts/bash/check-prerequisites.sh +166 -0
  14. .specify/scripts/bash/common.sh +156 -0
  15. .specify/scripts/bash/create-new-feature.sh +297 -0
  16. .specify/scripts/bash/setup-plan.sh +61 -0
  17. .specify/scripts/bash/update-agent-context.sh +799 -0
  18. .specify/templates/agent-file-template.md +28 -0
  19. .specify/templates/checklist-template.md +40 -0
  20. .specify/templates/plan-template.md +104 -0
  21. .specify/templates/spec-template.md +115 -0
  22. .specify/templates/tasks-template.md +251 -0
  23. Dockerfile +10 -7
  24. docs/content-plan.md +148 -0
  25. docs/docker-ai.md +487 -0
  26. docs/learning-path.md +1716 -0
  27. {src → docs}/notebooks/advanced_rag.qmd +0 -0
  28. {src → docs}/notebooks/automatic_embedding.ipynb +0 -0
  29. {src → docs}/notebooks/faiss.ipynb +0 -0
  30. {src → docs}/notebooks/rag_evaluation.qmd +0 -0
  31. {src → docs}/notebooks/rag_zephyr_langchain.qmd +0 -0
  32. {src → docs}/notebooks/single_gpu.ipynb +0 -0
  33. docs/projects/DataCrew.md +571 -0
  34. docs/projects/FileOrganizer.md +706 -0
  35. docs/projects/README.md +350 -0
  36. pixi.toml +27 -0
  37. requirements.txt +0 -3
  38. serve.py +20 -0
  39. src/.gitignore +2 -0
  40. src/_extensions/grantmcdermott/clean/_extension.yml +20 -0
  41. src/_extensions/grantmcdermott/clean/clean.scss +351 -0
  42. src/_extensions/grantmcdermott/clean/mathjax-config.js +18 -0
  43. src/_extensions/pandoc-ext/diagram/_extension.yaml +7 -0
  44. src/_extensions/pandoc-ext/diagram/diagram.lua +660 -0
  45. src/_quarto.yml +76 -27
  46. src/chapters/appendix-learning-resources.qmd +16 -0
  47. src/chapters/appendix-pixi-commands.qmd +16 -0
  48. src/chapters/ch01-development-environment.qmd +191 -0
  49. src/chapters/ch01-project-structure.qmd +328 -0
  50. src/chapters/ch02-building-with-typer.qmd +416 -0
.dockerignore ADDED
@@ -0,0 +1,60 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Git
2
+ .git/
3
+ .gitignore
4
+ .gitattributes
5
+
6
+ # Python
7
+ __pycache__/
8
+ *.pyc
9
+ *.pyo
10
+ *.pyd
11
+ .Python
12
+ *.py[cod]
13
+ *$py.class
14
+
15
+ # Pixi
16
+ .pixi/
17
+ pixi.lock
18
+
19
+ # Quarto build outputs (will be regenerated in container)
20
+ .quarto/
21
+ _site/
22
+ src/_site/
23
+ */_book/
24
+ */_site/
25
+
26
+ # Jupyter
27
+ .ipynb_checkpoints/
28
+
29
+ # Virtual environments
30
+ .venv/
31
+ venv/
32
+ ENV/
33
+ env/
34
+
35
+ # OS files
36
+ .DS_Store
37
+ Thumbs.db
38
+ *.swp
39
+ *.swo
40
+
41
+ # IDE
42
+ .vscode/
43
+ .idea/
44
+ *.sublime-*
45
+
46
+ # Documentation
47
+ README.md
48
+ LICENSE
49
+ *.md
50
+ !pixi.toml
51
+
52
+ # CI/CD
53
+ .github/
54
+ .gitlab-ci.yml
55
+ .travis.yml
56
+
57
+ # Docker
58
+ Dockerfile
59
+ .dockerignore
60
+ docker-compose.yml
.gemini/commands/speckit.analyze.toml ADDED
@@ -0,0 +1,188 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ description = "Perform a non-destructive cross-artifact consistency and quality analysis across spec.md, plan.md, and tasks.md after task generation."
2
+
3
+ prompt = """
4
+ ---
5
+ description: Perform a non-destructive cross-artifact consistency and quality analysis across spec.md, plan.md, and tasks.md after task generation.
6
+ ---
7
+
8
+ ## User Input
9
+
10
+ ```text
11
+ $ARGUMENTS
12
+ ```
13
+
14
+ You **MUST** consider the user input before proceeding (if not empty).
15
+
16
+ ## Goal
17
+
18
+ Identify inconsistencies, duplications, ambiguities, and underspecified items across the three core artifacts (`spec.md`, `plan.md`, `tasks.md`) before implementation. This command MUST run only after `/speckit.tasks` has successfully produced a complete `tasks.md`.
19
+
20
+ ## Operating Constraints
21
+
22
+ **STRICTLY READ-ONLY**: Do **not** modify any files. Output a structured analysis report. Offer an optional remediation plan (user must explicitly approve before any follow-up editing commands would be invoked manually).
23
+
24
+ **Constitution Authority**: The project constitution (`.specify/memory/constitution.md`) is **non-negotiable** within this analysis scope. Constitution conflicts are automatically CRITICAL and require adjustment of the spec, plan, or tasks—not dilution, reinterpretation, or silent ignoring of the principle. If a principle itself needs to change, that must occur in a separate, explicit constitution update outside `/speckit.analyze`.
25
+
26
+ ## Execution Steps
27
+
28
+ ### 1. Initialize Analysis Context
29
+
30
+ Run `.specify/scripts/bash/check-prerequisites.sh --json --require-tasks --include-tasks` once from repo root and parse JSON for FEATURE_DIR and AVAILABLE_DOCS. Derive absolute paths:
31
+
32
+ - SPEC = FEATURE_DIR/spec.md
33
+ - PLAN = FEATURE_DIR/plan.md
34
+ - TASKS = FEATURE_DIR/tasks.md
35
+
36
+ Abort with an error message if any required file is missing (instruct the user to run missing prerequisite command).
37
+ For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\\''m Groot' (or double-quote if possible: "I'm Groot").
38
+
39
+ ### 2. Load Artifacts (Progressive Disclosure)
40
+
41
+ Load only the minimal necessary context from each artifact:
42
+
43
+ **From spec.md:**
44
+
45
+ - Overview/Context
46
+ - Functional Requirements
47
+ - Non-Functional Requirements
48
+ - User Stories
49
+ - Edge Cases (if present)
50
+
51
+ **From plan.md:**
52
+
53
+ - Architecture/stack choices
54
+ - Data Model references
55
+ - Phases
56
+ - Technical constraints
57
+
58
+ **From tasks.md:**
59
+
60
+ - Task IDs
61
+ - Descriptions
62
+ - Phase grouping
63
+ - Parallel markers [P]
64
+ - Referenced file paths
65
+
66
+ **From constitution:**
67
+
68
+ - Load `.specify/memory/constitution.md` for principle validation
69
+
70
+ ### 3. Build Semantic Models
71
+
72
+ Create internal representations (do not include raw artifacts in output):
73
+
74
+ - **Requirements inventory**: Each functional + non-functional requirement with a stable key (derive slug based on imperative phrase; e.g., "User can upload file" → `user-can-upload-file`)
75
+ - **User story/action inventory**: Discrete user actions with acceptance criteria
76
+ - **Task coverage mapping**: Map each task to one or more requirements or stories (inference by keyword / explicit reference patterns like IDs or key phrases)
77
+ - **Constitution rule set**: Extract principle names and MUST/SHOULD normative statements
78
+
79
+ ### 4. Detection Passes (Token-Efficient Analysis)
80
+
81
+ Focus on high-signal findings. Limit to 50 findings total; aggregate remainder in overflow summary.
82
+
83
+ #### A. Duplication Detection
84
+
85
+ - Identify near-duplicate requirements
86
+ - Mark lower-quality phrasing for consolidation
87
+
88
+ #### B. Ambiguity Detection
89
+
90
+ - Flag vague adjectives (fast, scalable, secure, intuitive, robust) lacking measurable criteria
91
+ - Flag unresolved placeholders (TODO, TKTK, ???, `<placeholder>`, etc.)
92
+
93
+ #### C. Underspecification
94
+
95
+ - Requirements with verbs but missing object or measurable outcome
96
+ - User stories missing acceptance criteria alignment
97
+ - Tasks referencing files or components not defined in spec/plan
98
+
99
+ #### D. Constitution Alignment
100
+
101
+ - Any requirement or plan element conflicting with a MUST principle
102
+ - Missing mandated sections or quality gates from constitution
103
+
104
+ #### E. Coverage Gaps
105
+
106
+ - Requirements with zero associated tasks
107
+ - Tasks with no mapped requirement/story
108
+ - Non-functional requirements not reflected in tasks (e.g., performance, security)
109
+
110
+ #### F. Inconsistency
111
+
112
+ - Terminology drift (same concept named differently across files)
113
+ - Data entities referenced in plan but absent in spec (or vice versa)
114
+ - Task ordering contradictions (e.g., integration tasks before foundational setup tasks without dependency note)
115
+ - Conflicting requirements (e.g., one requires Next.js while other specifies Vue)
116
+
117
+ ### 5. Severity Assignment
118
+
119
+ Use this heuristic to prioritize findings:
120
+
121
+ - **CRITICAL**: Violates constitution MUST, missing core spec artifact, or requirement with zero coverage that blocks baseline functionality
122
+ - **HIGH**: Duplicate or conflicting requirement, ambiguous security/performance attribute, untestable acceptance criterion
123
+ - **MEDIUM**: Terminology drift, missing non-functional task coverage, underspecified edge case
124
+ - **LOW**: Style/wording improvements, minor redundancy not affecting execution order
125
+
126
+ ### 6. Produce Compact Analysis Report
127
+
128
+ Output a Markdown report (no file writes) with the following structure:
129
+
130
+ ## Specification Analysis Report
131
+
132
+ | ID | Category | Severity | Location(s) | Summary | Recommendation |
133
+ |----|----------|----------|-------------|---------|----------------|
134
+ | A1 | Duplication | HIGH | spec.md:L120-134 | Two similar requirements ... | Merge phrasing; keep clearer version |
135
+
136
+ (Add one row per finding; generate stable IDs prefixed by category initial.)
137
+
138
+ **Coverage Summary Table:**
139
+
140
+ | Requirement Key | Has Task? | Task IDs | Notes |
141
+ |-----------------|-----------|----------|-------|
142
+
143
+ **Constitution Alignment Issues:** (if any)
144
+
145
+ **Unmapped Tasks:** (if any)
146
+
147
+ **Metrics:**
148
+
149
+ - Total Requirements
150
+ - Total Tasks
151
+ - Coverage % (requirements with >=1 task)
152
+ - Ambiguity Count
153
+ - Duplication Count
154
+ - Critical Issues Count
155
+
156
+ ### 7. Provide Next Actions
157
+
158
+ At end of report, output a concise Next Actions block:
159
+
160
+ - If CRITICAL issues exist: Recommend resolving before `/speckit.implement`
161
+ - If only LOW/MEDIUM: User may proceed, but provide improvement suggestions
162
+ - Provide explicit command suggestions: e.g., "Run /speckit.specify with refinement", "Run /speckit.plan to adjust architecture", "Manually edit tasks.md to add coverage for 'performance-metrics'"
163
+
164
+ ### 8. Offer Remediation
165
+
166
+ Ask the user: "Would you like me to suggest concrete remediation edits for the top N issues?" (Do NOT apply them automatically.)
167
+
168
+ ## Operating Principles
169
+
170
+ ### Context Efficiency
171
+
172
+ - **Minimal high-signal tokens**: Focus on actionable findings, not exhaustive documentation
173
+ - **Progressive disclosure**: Load artifacts incrementally; don't dump all content into analysis
174
+ - **Token-efficient output**: Limit findings table to 50 rows; summarize overflow
175
+ - **Deterministic results**: Rerunning without changes should produce consistent IDs and counts
176
+
177
+ ### Analysis Guidelines
178
+
179
+ - **NEVER modify files** (this is read-only analysis)
180
+ - **NEVER hallucinate missing sections** (if absent, report them accurately)
181
+ - **Prioritize constitution violations** (these are always CRITICAL)
182
+ - **Use examples over exhaustive rules** (cite specific instances, not generic patterns)
183
+ - **Report zero issues gracefully** (emit success report with coverage statistics)
184
+
185
+ ## Context
186
+
187
+ {{args}}
188
+ """
.gemini/commands/speckit.checklist.toml ADDED
@@ -0,0 +1,298 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ description = "Generate a custom checklist for the current feature based on user requirements."
2
+
3
+ prompt = """
4
+ ---
5
+ description: Generate a custom checklist for the current feature based on user requirements.
6
+ ---
7
+
8
+ ## Checklist Purpose: "Unit Tests for English"
9
+
10
+ **CRITICAL CONCEPT**: Checklists are **UNIT TESTS FOR REQUIREMENTS WRITING** - they validate the quality, clarity, and completeness of requirements in a given domain.
11
+
12
+ **NOT for verification/testing**:
13
+
14
+ - ❌ NOT "Verify the button clicks correctly"
15
+ - ❌ NOT "Test error handling works"
16
+ - ❌ NOT "Confirm the API returns 200"
17
+ - ❌ NOT checking if code/implementation matches the spec
18
+
19
+ **FOR requirements quality validation**:
20
+
21
+ - ✅ "Are visual hierarchy requirements defined for all card types?" (completeness)
22
+ - ✅ "Is 'prominent display' quantified with specific sizing/positioning?" (clarity)
23
+ - ✅ "Are hover state requirements consistent across all interactive elements?" (consistency)
24
+ - ✅ "Are accessibility requirements defined for keyboard navigation?" (coverage)
25
+ - ✅ "Does the spec define what happens when logo image fails to load?" (edge cases)
26
+
27
+ **Metaphor**: If your spec is code written in English, the checklist is its unit test suite. You're testing whether the requirements are well-written, complete, unambiguous, and ready for implementation - NOT whether the implementation works.
28
+
29
+ ## User Input
30
+
31
+ ```text
32
+ $ARGUMENTS
33
+ ```
34
+
35
+ You **MUST** consider the user input before proceeding (if not empty).
36
+
37
+ ## Execution Steps
38
+
39
+ 1. **Setup**: Run `.specify/scripts/bash/check-prerequisites.sh --json` from repo root and parse JSON for FEATURE_DIR and AVAILABLE_DOCS list.
40
+ - All file paths must be absolute.
41
+ - For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\\''m Groot' (or double-quote if possible: "I'm Groot").
42
+
43
+ 2. **Clarify intent (dynamic)**: Derive up to THREE initial contextual clarifying questions (no pre-baked catalog). They MUST:
44
+ - Be generated from the user's phrasing + extracted signals from spec/plan/tasks
45
+ - Only ask about information that materially changes checklist content
46
+ - Be skipped individually if already unambiguous in `$ARGUMENTS`
47
+ - Prefer precision over breadth
48
+
49
+ Generation algorithm:
50
+ 1. Extract signals: feature domain keywords (e.g., auth, latency, UX, API), risk indicators ("critical", "must", "compliance"), stakeholder hints ("QA", "review", "security team"), and explicit deliverables ("a11y", "rollback", "contracts").
51
+ 2. Cluster signals into candidate focus areas (max 4) ranked by relevance.
52
+ 3. Identify probable audience & timing (author, reviewer, QA, release) if not explicit.
53
+ 4. Detect missing dimensions: scope breadth, depth/rigor, risk emphasis, exclusion boundaries, measurable acceptance criteria.
54
+ 5. Formulate questions chosen from these archetypes:
55
+ - Scope refinement (e.g., "Should this include integration touchpoints with X and Y or stay limited to local module correctness?")
56
+ - Risk prioritization (e.g., "Which of these potential risk areas should receive mandatory gating checks?")
57
+ - Depth calibration (e.g., "Is this a lightweight pre-commit sanity list or a formal release gate?")
58
+ - Audience framing (e.g., "Will this be used by the author only or peers during PR review?")
59
+ - Boundary exclusion (e.g., "Should we explicitly exclude performance tuning items this round?")
60
+ - Scenario class gap (e.g., "No recovery flows detected—are rollback / partial failure paths in scope?")
61
+
62
+ Question formatting rules:
63
+ - If presenting options, generate a compact table with columns: Option | Candidate | Why It Matters
64
+ - Limit to A–E options maximum; omit table if a free-form answer is clearer
65
+ - Never ask the user to restate what they already said
66
+ - Avoid speculative categories (no hallucination). If uncertain, ask explicitly: "Confirm whether X belongs in scope."
67
+
68
+ Defaults when interaction impossible:
69
+ - Depth: Standard
70
+ - Audience: Reviewer (PR) if code-related; Author otherwise
71
+ - Focus: Top 2 relevance clusters
72
+
73
+ Output the questions (label Q1/Q2/Q3). After answers: if ≥2 scenario classes (Alternate / Exception / Recovery / Non-Functional domain) remain unclear, you MAY ask up to TWO more targeted follow‑ups (Q4/Q5) with a one-line justification each (e.g., "Unresolved recovery path risk"). Do not exceed five total questions. Skip escalation if user explicitly declines more.
74
+
75
+ 3. **Understand user request**: Combine `$ARGUMENTS` + clarifying answers:
76
+ - Derive checklist theme (e.g., security, review, deploy, ux)
77
+ - Consolidate explicit must-have items mentioned by user
78
+ - Map focus selections to category scaffolding
79
+ - Infer any missing context from spec/plan/tasks (do NOT hallucinate)
80
+
81
+ 4. **Load feature context**: Read from FEATURE_DIR:
82
+ - spec.md: Feature requirements and scope
83
+ - plan.md (if exists): Technical details, dependencies
84
+ - tasks.md (if exists): Implementation tasks
85
+
86
+ **Context Loading Strategy**:
87
+ - Load only necessary portions relevant to active focus areas (avoid full-file dumping)
88
+ - Prefer summarizing long sections into concise scenario/requirement bullets
89
+ - Use progressive disclosure: add follow-on retrieval only if gaps detected
90
+ - If source docs are large, generate interim summary items instead of embedding raw text
91
+
92
+ 5. **Generate checklist** - Create "Unit Tests for Requirements":
93
+ - Create `FEATURE_DIR/checklists/` directory if it doesn't exist
94
+ - Generate unique checklist filename:
95
+ - Use short, descriptive name based on domain (e.g., `ux.md`, `api.md`, `security.md`)
96
+ - Format: `[domain].md`
97
+ - If file exists, append to existing file
98
+ - Number items sequentially starting from CHK001
99
+ - Each `/speckit.checklist` run creates a NEW file (never overwrites existing checklists)
100
+
101
+ **CORE PRINCIPLE - Test the Requirements, Not the Implementation**:
102
+ Every checklist item MUST evaluate the REQUIREMENTS THEMSELVES for:
103
+ - **Completeness**: Are all necessary requirements present?
104
+ - **Clarity**: Are requirements unambiguous and specific?
105
+ - **Consistency**: Do requirements align with each other?
106
+ - **Measurability**: Can requirements be objectively verified?
107
+ - **Coverage**: Are all scenarios/edge cases addressed?
108
+
109
+ **Category Structure** - Group items by requirement quality dimensions:
110
+ - **Requirement Completeness** (Are all necessary requirements documented?)
111
+ - **Requirement Clarity** (Are requirements specific and unambiguous?)
112
+ - **Requirement Consistency** (Do requirements align without conflicts?)
113
+ - **Acceptance Criteria Quality** (Are success criteria measurable?)
114
+ - **Scenario Coverage** (Are all flows/cases addressed?)
115
+ - **Edge Case Coverage** (Are boundary conditions defined?)
116
+ - **Non-Functional Requirements** (Performance, Security, Accessibility, etc. - are they specified?)
117
+ - **Dependencies & Assumptions** (Are they documented and validated?)
118
+ - **Ambiguities & Conflicts** (What needs clarification?)
119
+
120
+ **HOW TO WRITE CHECKLIST ITEMS - "Unit Tests for English"**:
121
+
122
+ ❌ **WRONG** (Testing implementation):
123
+ - "Verify landing page displays 3 episode cards"
124
+ - "Test hover states work on desktop"
125
+ - "Confirm logo click navigates home"
126
+
127
+ ✅ **CORRECT** (Testing requirements quality):
128
+ - "Are the exact number and layout of featured episodes specified?" [Completeness]
129
+ - "Is 'prominent display' quantified with specific sizing/positioning?" [Clarity]
130
+ - "Are hover state requirements consistent across all interactive elements?" [Consistency]
131
+ - "Are keyboard navigation requirements defined for all interactive UI?" [Coverage]
132
+ - "Is the fallback behavior specified when logo image fails to load?" [Edge Cases]
133
+ - "Are loading states defined for asynchronous episode data?" [Completeness]
134
+ - "Does the spec define visual hierarchy for competing UI elements?" [Clarity]
135
+
136
+ **ITEM STRUCTURE**:
137
+ Each item should follow this pattern:
138
+ - Question format asking about requirement quality
139
+ - Focus on what's WRITTEN (or not written) in the spec/plan
140
+ - Include quality dimension in brackets [Completeness/Clarity/Consistency/etc.]
141
+ - Reference spec section `[Spec §X.Y]` when checking existing requirements
142
+ - Use `[Gap]` marker when checking for missing requirements
143
+
144
+ **EXAMPLES BY QUALITY DIMENSION**:
145
+
146
+ Completeness:
147
+ - "Are error handling requirements defined for all API failure modes? [Gap]"
148
+ - "Are accessibility requirements specified for all interactive elements? [Completeness]"
149
+ - "Are mobile breakpoint requirements defined for responsive layouts? [Gap]"
150
+
151
+ Clarity:
152
+ - "Is 'fast loading' quantified with specific timing thresholds? [Clarity, Spec §NFR-2]"
153
+ - "Are 'related episodes' selection criteria explicitly defined? [Clarity, Spec §FR-5]"
154
+ - "Is 'prominent' defined with measurable visual properties? [Ambiguity, Spec §FR-4]"
155
+
156
+ Consistency:
157
+ - "Do navigation requirements align across all pages? [Consistency, Spec §FR-10]"
158
+ - "Are card component requirements consistent between landing and detail pages? [Consistency]"
159
+
160
+ Coverage:
161
+ - "Are requirements defined for zero-state scenarios (no episodes)? [Coverage, Edge Case]"
162
+ - "Are concurrent user interaction scenarios addressed? [Coverage, Gap]"
163
+ - "Are requirements specified for partial data loading failures? [Coverage, Exception Flow]"
164
+
165
+ Measurability:
166
+ - "Are visual hierarchy requirements measurable/testable? [Acceptance Criteria, Spec §FR-1]"
167
+ - "Can 'balanced visual weight' be objectively verified? [Measurability, Spec §FR-2]"
168
+
169
+ **Scenario Classification & Coverage** (Requirements Quality Focus):
170
+ - Check if requirements exist for: Primary, Alternate, Exception/Error, Recovery, Non-Functional scenarios
171
+ - For each scenario class, ask: "Are [scenario type] requirements complete, clear, and consistent?"
172
+ - If scenario class missing: "Are [scenario type] requirements intentionally excluded or missing? [Gap]"
173
+ - Include resilience/rollback when state mutation occurs: "Are rollback requirements defined for migration failures? [Gap]"
174
+
175
+ **Traceability Requirements**:
176
+ - MINIMUM: ≥80% of items MUST include at least one traceability reference
177
+ - Each item should reference: spec section `[Spec §X.Y]`, or use markers: `[Gap]`, `[Ambiguity]`, `[Conflict]`, `[Assumption]`
178
+ - If no ID system exists: "Is a requirement & acceptance criteria ID scheme established? [Traceability]"
179
+
180
+ **Surface & Resolve Issues** (Requirements Quality Problems):
181
+ Ask questions about the requirements themselves:
182
+ - Ambiguities: "Is the term 'fast' quantified with specific metrics? [Ambiguity, Spec §NFR-1]"
183
+ - Conflicts: "Do navigation requirements conflict between §FR-10 and §FR-10a? [Conflict]"
184
+ - Assumptions: "Is the assumption of 'always available podcast API' validated? [Assumption]"
185
+ - Dependencies: "Are external podcast API requirements documented? [Dependency, Gap]"
186
+ - Missing definitions: "Is 'visual hierarchy' defined with measurable criteria? [Gap]"
187
+
188
+ **Content Consolidation**:
189
+ - Soft cap: If raw candidate items > 40, prioritize by risk/impact
190
+ - Merge near-duplicates checking the same requirement aspect
191
+ - If >5 low-impact edge cases, create one item: "Are edge cases X, Y, Z addressed in requirements? [Coverage]"
192
+
193
+ **🚫 ABSOLUTELY PROHIBITED** - These make it an implementation test, not a requirements test:
194
+ - ❌ Any item starting with "Verify", "Test", "Confirm", "Check" + implementation behavior
195
+ - ❌ References to code execution, user actions, system behavior
196
+ - ❌ "Displays correctly", "works properly", "functions as expected"
197
+ - ❌ "Click", "navigate", "render", "load", "execute"
198
+ - ❌ Test cases, test plans, QA procedures
199
+ - ❌ Implementation details (frameworks, APIs, algorithms)
200
+
201
+ **✅ REQUIRED PATTERNS** - These test requirements quality:
202
+ - ✅ "Are [requirement type] defined/specified/documented for [scenario]?"
203
+ - ✅ "Is [vague term] quantified/clarified with specific criteria?"
204
+ - ✅ "Are requirements consistent between [section A] and [section B]?"
205
+ - ✅ "Can [requirement] be objectively measured/verified?"
206
+ - ✅ "Are [edge cases/scenarios] addressed in requirements?"
207
+ - ✅ "Does the spec define [missing aspect]?"
208
+
209
+ 6. **Structure Reference**: Generate the checklist following the canonical template in `.specify/templates/checklist-template.md` for title, meta section, category headings, and ID formatting. If template is unavailable, use: H1 title, purpose/created meta lines, `##` category sections containing `- [ ] CHK### <requirement item>` lines with globally incrementing IDs starting at CHK001.
210
+
211
+ 7. **Report**: Output full path to created checklist, item count, and remind user that each run creates a new file. Summarize:
212
+ - Focus areas selected
213
+ - Depth level
214
+ - Actor/timing
215
+ - Any explicit user-specified must-have items incorporated
216
+
217
+ **Important**: Each `/speckit.checklist` command invocation creates a checklist file using short, descriptive names unless file already exists. This allows:
218
+
219
+ - Multiple checklists of different types (e.g., `ux.md`, `test.md`, `security.md`)
220
+ - Simple, memorable filenames that indicate checklist purpose
221
+ - Easy identification and navigation in the `checklists/` folder
222
+
223
+ To avoid clutter, use descriptive types and clean up obsolete checklists when done.
224
+
225
+ ## Example Checklist Types & Sample Items
226
+
227
+ **UX Requirements Quality:** `ux.md`
228
+
229
+ Sample items (testing the requirements, NOT the implementation):
230
+
231
+ - "Are visual hierarchy requirements defined with measurable criteria? [Clarity, Spec §FR-1]"
232
+ - "Is the number and positioning of UI elements explicitly specified? [Completeness, Spec §FR-1]"
233
+ - "Are interaction state requirements (hover, focus, active) consistently defined? [Consistency]"
234
+ - "Are accessibility requirements specified for all interactive elements? [Coverage, Gap]"
235
+ - "Is fallback behavior defined when images fail to load? [Edge Case, Gap]"
236
+ - "Can 'prominent display' be objectively measured? [Measurability, Spec §FR-4]"
237
+
238
+ **API Requirements Quality:** `api.md`
239
+
240
+ Sample items:
241
+
242
+ - "Are error response formats specified for all failure scenarios? [Completeness]"
243
+ - "Are rate limiting requirements quantified with specific thresholds? [Clarity]"
244
+ - "Are authentication requirements consistent across all endpoints? [Consistency]"
245
+ - "Are retry/timeout requirements defined for external dependencies? [Coverage, Gap]"
246
+ - "Is versioning strategy documented in requirements? [Gap]"
247
+
248
+ **Performance Requirements Quality:** `performance.md`
249
+
250
+ Sample items:
251
+
252
+ - "Are performance requirements quantified with specific metrics? [Clarity]"
253
+ - "Are performance targets defined for all critical user journeys? [Coverage]"
254
+ - "Are performance requirements under different load conditions specified? [Completeness]"
255
+ - "Can performance requirements be objectively measured? [Measurability]"
256
+ - "Are degradation requirements defined for high-load scenarios? [Edge Case, Gap]"
257
+
258
+ **Security Requirements Quality:** `security.md`
259
+
260
+ Sample items:
261
+
262
+ - "Are authentication requirements specified for all protected resources? [Coverage]"
263
+ - "Are data protection requirements defined for sensitive information? [Completeness]"
264
+ - "Is the threat model documented and requirements aligned to it? [Traceability]"
265
+ - "Are security requirements consistent with compliance obligations? [Consistency]"
266
+ - "Are security failure/breach response requirements defined? [Gap, Exception Flow]"
267
+
268
+ ## Anti-Examples: What NOT To Do
269
+
270
+ **❌ WRONG - These test implementation, not requirements:**
271
+
272
+ ```markdown
273
+ - [ ] CHK001 - Verify landing page displays 3 episode cards [Spec §FR-001]
274
+ - [ ] CHK002 - Test hover states work correctly on desktop [Spec §FR-003]
275
+ - [ ] CHK003 - Confirm logo click navigates to home page [Spec §FR-010]
276
+ - [ ] CHK004 - Check that related episodes section shows 3-5 items [Spec §FR-005]
277
+ ```
278
+
279
+ **✅ CORRECT - These test requirements quality:**
280
+
281
+ ```markdown
282
+ - [ ] CHK001 - Are the number and layout of featured episodes explicitly specified? [Completeness, Spec §FR-001]
283
+ - [ ] CHK002 - Are hover state requirements consistently defined for all interactive elements? [Consistency, Spec §FR-003]
284
+ - [ ] CHK003 - Are navigation requirements clear for all clickable brand elements? [Clarity, Spec §FR-010]
285
+ - [ ] CHK004 - Is the selection criteria for related episodes documented? [Gap, Spec §FR-005]
286
+ - [ ] CHK005 - Are loading state requirements defined for asynchronous episode data? [Gap]
287
+ - [ ] CHK006 - Can "visual hierarchy" requirements be objectively measured? [Measurability, Spec §FR-001]
288
+ ```
289
+
290
+ **Key Differences:**
291
+
292
+ - Wrong: Tests if the system works correctly
293
+ - Correct: Tests if the requirements are written correctly
294
+ - Wrong: Verification of behavior
295
+ - Correct: Validation of requirement quality
296
+ - Wrong: "Does it do X?"
297
+ - Correct: "Is X clearly specified?"
298
+ """
.gemini/commands/speckit.clarify.toml ADDED
@@ -0,0 +1,185 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ description = "Identify underspecified areas in the current feature spec by asking up to 5 highly targeted clarification questions and encoding answers back into the spec."
2
+
3
+ prompt = """
4
+ ---
5
+ description: Identify underspecified areas in the current feature spec by asking up to 5 highly targeted clarification questions and encoding answers back into the spec.
6
+ handoffs:
7
+ - label: Build Technical Plan
8
+ agent: speckit.plan
9
+ prompt: Create a plan for the spec. I am building with...
10
+ ---
11
+
12
+ ## User Input
13
+
14
+ ```text
15
+ $ARGUMENTS
16
+ ```
17
+
18
+ You **MUST** consider the user input before proceeding (if not empty).
19
+
20
+ ## Outline
21
+
22
+ Goal: Detect and reduce ambiguity or missing decision points in the active feature specification and record the clarifications directly in the spec file.
23
+
24
+ Note: This clarification workflow is expected to run (and be completed) BEFORE invoking `/speckit.plan`. If the user explicitly states they are skipping clarification (e.g., exploratory spike), you may proceed, but must warn that downstream rework risk increases.
25
+
26
+ Execution steps:
27
+
28
+ 1. Run `.specify/scripts/bash/check-prerequisites.sh --json --paths-only` from repo root **once** (combined `--json --paths-only` mode / `-Json -PathsOnly`). Parse minimal JSON payload fields:
29
+ - `FEATURE_DIR`
30
+ - `FEATURE_SPEC`
31
+ - (Optionally capture `IMPL_PLAN`, `TASKS` for future chained flows.)
32
+ - If JSON parsing fails, abort and instruct user to re-run `/speckit.specify` or verify feature branch environment.
33
+ - For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\\''m Groot' (or double-quote if possible: "I'm Groot").
34
+
35
+ 2. Load the current spec file. Perform a structured ambiguity & coverage scan using this taxonomy. For each category, mark status: Clear / Partial / Missing. Produce an internal coverage map used for prioritization (do not output raw map unless no questions will be asked).
36
+
37
+ Functional Scope & Behavior:
38
+ - Core user goals & success criteria
39
+ - Explicit out-of-scope declarations
40
+ - User roles / personas differentiation
41
+
42
+ Domain & Data Model:
43
+ - Entities, attributes, relationships
44
+ - Identity & uniqueness rules
45
+ - Lifecycle/state transitions
46
+ - Data volume / scale assumptions
47
+
48
+ Interaction & UX Flow:
49
+ - Critical user journeys / sequences
50
+ - Error/empty/loading states
51
+ - Accessibility or localization notes
52
+
53
+ Non-Functional Quality Attributes:
54
+ - Performance (latency, throughput targets)
55
+ - Scalability (horizontal/vertical, limits)
56
+ - Reliability & availability (uptime, recovery expectations)
57
+ - Observability (logging, metrics, tracing signals)
58
+ - Security & privacy (authN/Z, data protection, threat assumptions)
59
+ - Compliance / regulatory constraints (if any)
60
+
61
+ Integration & External Dependencies:
62
+ - External services/APIs and failure modes
63
+ - Data import/export formats
64
+ - Protocol/versioning assumptions
65
+
66
+ Edge Cases & Failure Handling:
67
+ - Negative scenarios
68
+ - Rate limiting / throttling
69
+ - Conflict resolution (e.g., concurrent edits)
70
+
71
+ Constraints & Tradeoffs:
72
+ - Technical constraints (language, storage, hosting)
73
+ - Explicit tradeoffs or rejected alternatives
74
+
75
+ Terminology & Consistency:
76
+ - Canonical glossary terms
77
+ - Avoided synonyms / deprecated terms
78
+
79
+ Completion Signals:
80
+ - Acceptance criteria testability
81
+ - Measurable Definition of Done style indicators
82
+
83
+ Misc / Placeholders:
84
+ - TODO markers / unresolved decisions
85
+ - Ambiguous adjectives ("robust", "intuitive") lacking quantification
86
+
87
+ For each category with Partial or Missing status, add a candidate question opportunity unless:
88
+ - Clarification would not materially change implementation or validation strategy
89
+ - Information is better deferred to planning phase (note internally)
90
+
91
+ 3. Generate (internally) a prioritized queue of candidate clarification questions (maximum 5). Do NOT output them all at once. Apply these constraints:
92
+ - Maximum of 10 total questions across the whole session.
93
+ - Each question must be answerable with EITHER:
94
+ - A short multiple‑choice selection (2–5 distinct, mutually exclusive options), OR
95
+ - A one-word / short‑phrase answer (explicitly constrain: "Answer in <=5 words").
96
+ - Only include questions whose answers materially impact architecture, data modeling, task decomposition, test design, UX behavior, operational readiness, or compliance validation.
97
+ - Ensure category coverage balance: attempt to cover the highest impact unresolved categories first; avoid asking two low-impact questions when a single high-impact area (e.g., security posture) is unresolved.
98
+ - Exclude questions already answered, trivial stylistic preferences, or plan-level execution details (unless blocking correctness).
99
+ - Favor clarifications that reduce downstream rework risk or prevent misaligned acceptance tests.
100
+ - If more than 5 categories remain unresolved, select the top 5 by (Impact * Uncertainty) heuristic.
101
+
102
+ 4. Sequential questioning loop (interactive):
103
+ - Present EXACTLY ONE question at a time.
104
+ - For multiple‑choice questions:
105
+ - **Analyze all options** and determine the **most suitable option** based on:
106
+ - Best practices for the project type
107
+ - Common patterns in similar implementations
108
+ - Risk reduction (security, performance, maintainability)
109
+ - Alignment with any explicit project goals or constraints visible in the spec
110
+ - Present your **recommended option prominently** at the top with clear reasoning (1-2 sentences explaining why this is the best choice).
111
+ - Format as: `**Recommended:** Option [X] - <reasoning>`
112
+ - Then render all options as a Markdown table:
113
+
114
+ | Option | Description |
115
+ |--------|-------------|
116
+ | A | <Option A description> |
117
+ | B | <Option B description> |
118
+ | C | <Option C description> (add D/E as needed up to 5) |
119
+ | Short | Provide a different short answer (<=5 words) (Include only if free-form alternative is appropriate) |
120
+
121
+ - After the table, add: `You can reply with the option letter (e.g., "A"), accept the recommendation by saying "yes" or "recommended", or provide your own short answer.`
122
+ - For short‑answer style (no meaningful discrete options):
123
+ - Provide your **suggested answer** based on best practices and context.
124
+ - Format as: `**Suggested:** <your proposed answer> - <brief reasoning>`
125
+ - Then output: `Format: Short answer (<=5 words). You can accept the suggestion by saying "yes" or "suggested", or provide your own answer.`
126
+ - After the user answers:
127
+ - If the user replies with "yes", "recommended", or "suggested", use your previously stated recommendation/suggestion as the answer.
128
+ - Otherwise, validate the answer maps to one option or fits the <=5 word constraint.
129
+ - If ambiguous, ask for a quick disambiguation (count still belongs to same question; do not advance).
130
+ - Once satisfactory, record it in working memory (do not yet write to disk) and move to the next queued question.
131
+ - Stop asking further questions when:
132
+ - All critical ambiguities resolved early (remaining queued items become unnecessary), OR
133
+ - User signals completion ("done", "good", "no more"), OR
134
+ - You reach 5 asked questions.
135
+ - Never reveal future queued questions in advance.
136
+ - If no valid questions exist at start, immediately report no critical ambiguities.
137
+
138
+ 5. Integration after EACH accepted answer (incremental update approach):
139
+ - Maintain in-memory representation of the spec (loaded once at start) plus the raw file contents.
140
+ - For the first integrated answer in this session:
141
+ - Ensure a `## Clarifications` section exists (create it just after the highest-level contextual/overview section per the spec template if missing).
142
+ - Under it, create (if not present) a `### Session YYYY-MM-DD` subheading for today.
143
+ - Append a bullet line immediately after acceptance: `- Q: <question> → A: <final answer>`.
144
+ - Then immediately apply the clarification to the most appropriate section(s):
145
+ - Functional ambiguity → Update or add a bullet in Functional Requirements.
146
+ - User interaction / actor distinction → Update User Stories or Actors subsection (if present) with clarified role, constraint, or scenario.
147
+ - Data shape / entities → Update Data Model (add fields, types, relationships) preserving ordering; note added constraints succinctly.
148
+ - Non-functional constraint → Add/modify measurable criteria in Non-Functional / Quality Attributes section (convert vague adjective to metric or explicit target).
149
+ - Edge case / negative flow → Add a new bullet under Edge Cases / Error Handling (or create such subsection if template provides placeholder for it).
150
+ - Terminology conflict → Normalize term across spec; retain original only if necessary by adding `(formerly referred to as "X")` once.
151
+ - If the clarification invalidates an earlier ambiguous statement, replace that statement instead of duplicating; leave no obsolete contradictory text.
152
+ - Save the spec file AFTER each integration to minimize risk of context loss (atomic overwrite).
153
+ - Preserve formatting: do not reorder unrelated sections; keep heading hierarchy intact.
154
+ - Keep each inserted clarification minimal and testable (avoid narrative drift).
155
+
156
+ 6. Validation (performed after EACH write plus final pass):
157
+ - Clarifications session contains exactly one bullet per accepted answer (no duplicates).
158
+ - Total asked (accepted) questions ≤ 5.
159
+ - Updated sections contain no lingering vague placeholders the new answer was meant to resolve.
160
+ - No contradictory earlier statement remains (scan for now-invalid alternative choices removed).
161
+ - Markdown structure valid; only allowed new headings: `## Clarifications`, `### Session YYYY-MM-DD`.
162
+ - Terminology consistency: same canonical term used across all updated sections.
163
+
164
+ 7. Write the updated spec back to `FEATURE_SPEC`.
165
+
166
+ 8. Report completion (after questioning loop ends or early termination):
167
+ - Number of questions asked & answered.
168
+ - Path to updated spec.
169
+ - Sections touched (list names).
170
+ - Coverage summary table listing each taxonomy category with Status: Resolved (was Partial/Missing and addressed), Deferred (exceeds question quota or better suited for planning), Clear (already sufficient), Outstanding (still Partial/Missing but low impact).
171
+ - If any Outstanding or Deferred remain, recommend whether to proceed to `/speckit.plan` or run `/speckit.clarify` again later post-plan.
172
+ - Suggested next command.
173
+
174
+ Behavior rules:
175
+
176
+ - If no meaningful ambiguities found (or all potential questions would be low-impact), respond: "No critical ambiguities detected worth formal clarification." and suggest proceeding.
177
+ - If spec file missing, instruct user to run `/speckit.specify` first (do not create a new spec here).
178
+ - Never exceed 5 total asked questions (clarification retries for a single question do not count as new questions).
179
+ - Avoid speculative tech stack questions unless the absence blocks functional clarity.
180
+ - Respect user early termination signals ("stop", "done", "proceed").
181
+ - If no questions asked due to full coverage, output a compact coverage summary (all categories Clear) then suggest advancing.
182
+ - If quota reached with unresolved high-impact categories remaining, explicitly flag them under Deferred with rationale.
183
+
184
+ Context for prioritization: {{args}}
185
+ """
.gemini/commands/speckit.constitution.toml ADDED
@@ -0,0 +1,86 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ description = "Create or update the project constitution from interactive or provided principle inputs, ensuring all dependent templates stay in sync."
2
+
3
+ prompt = """
4
+ ---
5
+ description: Create or update the project constitution from interactive or provided principle inputs, ensuring all dependent templates stay in sync.
6
+ handoffs:
7
+ - label: Build Specification
8
+ agent: speckit.specify
9
+ prompt: Implement the feature specification based on the updated constitution. I want to build...
10
+ ---
11
+
12
+ ## User Input
13
+
14
+ ```text
15
+ $ARGUMENTS
16
+ ```
17
+
18
+ You **MUST** consider the user input before proceeding (if not empty).
19
+
20
+ ## Outline
21
+
22
+ You are updating the project constitution at `.specify/memory/constitution.md`. This file is a TEMPLATE containing placeholder tokens in square brackets (e.g. `[PROJECT_NAME]`, `[PRINCIPLE_1_NAME]`). Your job is to (a) collect/derive concrete values, (b) fill the template precisely, and (c) propagate any amendments across dependent artifacts.
23
+
24
+ Follow this execution flow:
25
+
26
+ 1. Load the existing constitution template at `.specify/memory/constitution.md`.
27
+ - Identify every placeholder token of the form `[ALL_CAPS_IDENTIFIER]`.
28
+ **IMPORTANT**: The user might require less or more principles than the ones used in the template. If a number is specified, respect that - follow the general template. You will update the doc accordingly.
29
+
30
+ 2. Collect/derive values for placeholders:
31
+ - If user input (conversation) supplies a value, use it.
32
+ - Otherwise infer from existing repo context (README, docs, prior constitution versions if embedded).
33
+ - For governance dates: `RATIFICATION_DATE` is the original adoption date (if unknown ask or mark TODO), `LAST_AMENDED_DATE` is today if changes are made, otherwise keep previous.
34
+ - `CONSTITUTION_VERSION` must increment according to semantic versioning rules:
35
+ - MAJOR: Backward incompatible governance/principle removals or redefinitions.
36
+ - MINOR: New principle/section added or materially expanded guidance.
37
+ - PATCH: Clarifications, wording, typo fixes, non-semantic refinements.
38
+ - If version bump type ambiguous, propose reasoning before finalizing.
39
+
40
+ 3. Draft the updated constitution content:
41
+ - Replace every placeholder with concrete text (no bracketed tokens left except intentionally retained template slots that the project has chosen not to define yet—explicitly justify any left).
42
+ - Preserve heading hierarchy and comments can be removed once replaced unless they still add clarifying guidance.
43
+ - Ensure each Principle section: succinct name line, paragraph (or bullet list) capturing non‑negotiable rules, explicit rationale if not obvious.
44
+ - Ensure Governance section lists amendment procedure, versioning policy, and compliance review expectations.
45
+
46
+ 4. Consistency propagation checklist (convert prior checklist into active validations):
47
+ - Read `.specify/templates/plan-template.md` and ensure any "Constitution Check" or rules align with updated principles.
48
+ - Read `.specify/templates/spec-template.md` for scope/requirements alignment—update if constitution adds/removes mandatory sections or constraints.
49
+ - Read `.specify/templates/tasks-template.md` and ensure task categorization reflects new or removed principle-driven task types (e.g., observability, versioning, testing discipline).
50
+ - Read each command file in `.specify/templates/commands/*.md` (including this one) to verify no outdated references (agent-specific names like CLAUDE only) remain when generic guidance is required.
51
+ - Read any runtime guidance docs (e.g., `README.md`, `docs/quickstart.md`, or agent-specific guidance files if present). Update references to principles changed.
52
+
53
+ 5. Produce a Sync Impact Report (prepend as an HTML comment at top of the constitution file after update):
54
+ - Version change: old → new
55
+ - List of modified principles (old title → new title if renamed)
56
+ - Added sections
57
+ - Removed sections
58
+ - Templates requiring updates (✅ updated / ⚠ pending) with file paths
59
+ - Follow-up TODOs if any placeholders intentionally deferred.
60
+
61
+ 6. Validation before final output:
62
+ - No remaining unexplained bracket tokens.
63
+ - Version line matches report.
64
+ - Dates ISO format YYYY-MM-DD.
65
+ - Principles are declarative, testable, and free of vague language ("should" → replace with MUST/SHOULD rationale where appropriate).
66
+
67
+ 7. Write the completed constitution back to `.specify/memory/constitution.md` (overwrite).
68
+
69
+ 8. Output a final summary to the user with:
70
+ - New version and bump rationale.
71
+ - Any files flagged for manual follow-up.
72
+ - Suggested commit message (e.g., `docs: amend constitution to vX.Y.Z (principle additions + governance update)`).
73
+
74
+ Formatting & Style Requirements:
75
+
76
+ - Use Markdown headings exactly as in the template (do not demote/promote levels).
77
+ - Wrap long rationale lines to keep readability (<100 chars ideally) but do not hard enforce with awkward breaks.
78
+ - Keep a single blank line between sections.
79
+ - Avoid trailing whitespace.
80
+
81
+ If the user supplies partial updates (e.g., only one principle revision), still perform validation and version decision steps.
82
+
83
+ If critical info missing (e.g., ratification date truly unknown), insert `TODO(<FIELD_NAME>): explanation` and include in the Sync Impact Report under deferred items.
84
+
85
+ Do not create a new template; always operate on the existing `.specify/memory/constitution.md` file.
86
+ """
.gemini/commands/speckit.implement.toml ADDED
@@ -0,0 +1,139 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ description = "Execute the implementation plan by processing and executing all tasks defined in tasks.md"
2
+
3
+ prompt = """
4
+ ---
5
+ description: Execute the implementation plan by processing and executing all tasks defined in tasks.md
6
+ ---
7
+
8
+ ## User Input
9
+
10
+ ```text
11
+ $ARGUMENTS
12
+ ```
13
+
14
+ You **MUST** consider the user input before proceeding (if not empty).
15
+
16
+ ## Outline
17
+
18
+ 1. Run `.specify/scripts/bash/check-prerequisites.sh --json --require-tasks --include-tasks` from repo root and parse FEATURE_DIR and AVAILABLE_DOCS list. All paths must be absolute. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\\''m Groot' (or double-quote if possible: "I'm Groot").
19
+
20
+ 2. **Check checklists status** (if FEATURE_DIR/checklists/ exists):
21
+ - Scan all checklist files in the checklists/ directory
22
+ - For each checklist, count:
23
+ - Total items: All lines matching `- [ ]` or `- [X]` or `- [x]`
24
+ - Completed items: Lines matching `- [X]` or `- [x]`
25
+ - Incomplete items: Lines matching `- [ ]`
26
+ - Create a status table:
27
+
28
+ ```text
29
+ | Checklist | Total | Completed | Incomplete | Status |
30
+ |-----------|-------|-----------|------------|--------|
31
+ | ux.md | 12 | 12 | 0 | ✓ PASS |
32
+ | test.md | 8 | 5 | 3 | ✗ FAIL |
33
+ | security.md | 6 | 6 | 0 | ✓ PASS |
34
+ ```
35
+
36
+ - Calculate overall status:
37
+ - **PASS**: All checklists have 0 incomplete items
38
+ - **FAIL**: One or more checklists have incomplete items
39
+
40
+ - **If any checklist is incomplete**:
41
+ - Display the table with incomplete item counts
42
+ - **STOP** and ask: "Some checklists are incomplete. Do you want to proceed with implementation anyway? (yes/no)"
43
+ - Wait for user response before continuing
44
+ - If user says "no" or "wait" or "stop", halt execution
45
+ - If user says "yes" or "proceed" or "continue", proceed to step 3
46
+
47
+ - **If all checklists are complete**:
48
+ - Display the table showing all checklists passed
49
+ - Automatically proceed to step 3
50
+
51
+ 3. Load and analyze the implementation context:
52
+ - **REQUIRED**: Read tasks.md for the complete task list and execution plan
53
+ - **REQUIRED**: Read plan.md for tech stack, architecture, and file structure
54
+ - **IF EXISTS**: Read data-model.md for entities and relationships
55
+ - **IF EXISTS**: Read contracts/ for API specifications and test requirements
56
+ - **IF EXISTS**: Read research.md for technical decisions and constraints
57
+ - **IF EXISTS**: Read quickstart.md for integration scenarios
58
+
59
+ 4. **Project Setup Verification**:
60
+ - **REQUIRED**: Create/verify ignore files based on actual project setup:
61
+
62
+ **Detection & Creation Logic**:
63
+ - Check if the following command succeeds to determine if the repository is a git repo (create/verify .gitignore if so):
64
+
65
+ ```sh
66
+ git rev-parse --git-dir 2>/dev/null
67
+ ```
68
+
69
+ - Check if Dockerfile* exists or Docker in plan.md → create/verify .dockerignore
70
+ - Check if .eslintrc* exists → create/verify .eslintignore
71
+ - Check if eslint.config.* exists → ensure the config's `ignores` entries cover required patterns
72
+ - Check if .prettierrc* exists → create/verify .prettierignore
73
+ - Check if .npmrc or package.json exists → create/verify .npmignore (if publishing)
74
+ - Check if terraform files (*.tf) exist → create/verify .terraformignore
75
+ - Check if .helmignore needed (helm charts present) → create/verify .helmignore
76
+
77
+ **If ignore file already exists**: Verify it contains essential patterns, append missing critical patterns only
78
+ **If ignore file missing**: Create with full pattern set for detected technology
79
+
80
+ **Common Patterns by Technology** (from plan.md tech stack):
81
+ - **Node.js/JavaScript/TypeScript**: `node_modules/`, `dist/`, `build/`, `*.log`, `.env*`
82
+ - **Python**: `__pycache__/`, `*.pyc`, `.venv/`, `venv/`, `dist/`, `*.egg-info/`
83
+ - **Java**: `target/`, `*.class`, `*.jar`, `.gradle/`, `build/`
84
+ - **C#/.NET**: `bin/`, `obj/`, `*.user`, `*.suo`, `packages/`
85
+ - **Go**: `*.exe`, `*.test`, `vendor/`, `*.out`
86
+ - **Ruby**: `.bundle/`, `log/`, `tmp/`, `*.gem`, `vendor/bundle/`
87
+ - **PHP**: `vendor/`, `*.log`, `*.cache`, `*.env`
88
+ - **Rust**: `target/`, `debug/`, `release/`, `*.rs.bk`, `*.rlib`, `*.prof*`, `.idea/`, `*.log`, `.env*`
89
+ - **Kotlin**: `build/`, `out/`, `.gradle/`, `.idea/`, `*.class`, `*.jar`, `*.iml`, `*.log`, `.env*`
90
+ - **C++**: `build/`, `bin/`, `obj/`, `out/`, `*.o`, `*.so`, `*.a`, `*.exe`, `*.dll`, `.idea/`, `*.log`, `.env*`
91
+ - **C**: `build/`, `bin/`, `obj/`, `out/`, `*.o`, `*.a`, `*.so`, `*.exe`, `Makefile`, `config.log`, `.idea/`, `*.log`, `.env*`
92
+ - **Swift**: `.build/`, `DerivedData/`, `*.swiftpm/`, `Packages/`
93
+ - **R**: `.Rproj.user/`, `.Rhistory`, `.RData`, `.Ruserdata`, `*.Rproj`, `packrat/`, `renv/`
94
+ - **Universal**: `.DS_Store`, `Thumbs.db`, `*.tmp`, `*.swp`, `.vscode/`, `.idea/`
95
+
96
+ **Tool-Specific Patterns**:
97
+ - **Docker**: `node_modules/`, `.git/`, `Dockerfile*`, `.dockerignore`, `*.log*`, `.env*`, `coverage/`
98
+ - **ESLint**: `node_modules/`, `dist/`, `build/`, `coverage/`, `*.min.js`
99
+ - **Prettier**: `node_modules/`, `dist/`, `build/`, `coverage/`, `package-lock.json`, `yarn.lock`, `pnpm-lock.yaml`
100
+ - **Terraform**: `.terraform/`, `*.tfstate*`, `*.tfvars`, `.terraform.lock.hcl`
101
+ - **Kubernetes/k8s**: `*.secret.yaml`, `secrets/`, `.kube/`, `kubeconfig*`, `*.key`, `*.crt`
102
+
103
+ 5. Parse tasks.md structure and extract:
104
+ - **Task phases**: Setup, Tests, Core, Integration, Polish
105
+ - **Task dependencies**: Sequential vs parallel execution rules
106
+ - **Task details**: ID, description, file paths, parallel markers [P]
107
+ - **Execution flow**: Order and dependency requirements
108
+
109
+ 6. Execute implementation following the task plan:
110
+ - **Phase-by-phase execution**: Complete each phase before moving to the next
111
+ - **Respect dependencies**: Run sequential tasks in order, parallel tasks [P] can run together
112
+ - **Follow TDD approach**: Execute test tasks before their corresponding implementation tasks
113
+ - **File-based coordination**: Tasks affecting the same files must run sequentially
114
+ - **Validation checkpoints**: Verify each phase completion before proceeding
115
+
116
+ 7. Implementation execution rules:
117
+ - **Setup first**: Initialize project structure, dependencies, configuration
118
+ - **Tests before code**: If you need to write tests for contracts, entities, and integration scenarios
119
+ - **Core development**: Implement models, services, CLI commands, endpoints
120
+ - **Integration work**: Database connections, middleware, logging, external services
121
+ - **Polish and validation**: Unit tests, performance optimization, documentation
122
+
123
+ 8. Progress tracking and error handling:
124
+ - Report progress after each completed task
125
+ - Halt execution if any non-parallel task fails
126
+ - For parallel tasks [P], continue with successful tasks, report failed ones
127
+ - Provide clear error messages with context for debugging
128
+ - Suggest next steps if implementation cannot proceed
129
+ - **IMPORTANT** For completed tasks, make sure to mark the task off as [X] in the tasks file.
130
+
131
+ 9. Completion validation:
132
+ - Verify all required tasks are completed
133
+ - Check that implemented features match the original specification
134
+ - Validate that tests pass and coverage meets requirements
135
+ - Confirm the implementation follows the technical plan
136
+ - Report final status with summary of completed work
137
+
138
+ Note: This command assumes a complete task breakdown exists in tasks.md. If tasks are incomplete or missing, suggest running `/speckit.tasks` first to regenerate the task list.
139
+ """
.gemini/commands/speckit.plan.toml ADDED
@@ -0,0 +1,93 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ description = "Execute the implementation planning workflow using the plan template to generate design artifacts."
2
+
3
+ prompt = """
4
+ ---
5
+ description: Execute the implementation planning workflow using the plan template to generate design artifacts.
6
+ handoffs:
7
+ - label: Create Tasks
8
+ agent: speckit.tasks
9
+ prompt: Break the plan into tasks
10
+ send: true
11
+ - label: Create Checklist
12
+ agent: speckit.checklist
13
+ prompt: Create a checklist for the following domain...
14
+ ---
15
+
16
+ ## User Input
17
+
18
+ ```text
19
+ $ARGUMENTS
20
+ ```
21
+
22
+ You **MUST** consider the user input before proceeding (if not empty).
23
+
24
+ ## Outline
25
+
26
+ 1. **Setup**: Run `.specify/scripts/bash/setup-plan.sh --json` from repo root and parse JSON for FEATURE_SPEC, IMPL_PLAN, SPECS_DIR, BRANCH. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\\''m Groot' (or double-quote if possible: "I'm Groot").
27
+
28
+ 2. **Load context**: Read FEATURE_SPEC and `.specify/memory/constitution.md`. Load IMPL_PLAN template (already copied).
29
+
30
+ 3. **Execute plan workflow**: Follow the structure in IMPL_PLAN template to:
31
+ - Fill Technical Context (mark unknowns as "NEEDS CLARIFICATION")
32
+ - Fill Constitution Check section from constitution
33
+ - Evaluate gates (ERROR if violations unjustified)
34
+ - Phase 0: Generate research.md (resolve all NEEDS CLARIFICATION)
35
+ - Phase 1: Generate data-model.md, contracts/, quickstart.md
36
+ - Phase 1: Update agent context by running the agent script
37
+ - Re-evaluate Constitution Check post-design
38
+
39
+ 4. **Stop and report**: Command ends after Phase 2 planning. Report branch, IMPL_PLAN path, and generated artifacts.
40
+
41
+ ## Phases
42
+
43
+ ### Phase 0: Outline & Research
44
+
45
+ 1. **Extract unknowns from Technical Context** above:
46
+ - For each NEEDS CLARIFICATION → research task
47
+ - For each dependency → best practices task
48
+ - For each integration → patterns task
49
+
50
+ 2. **Generate and dispatch research agents**:
51
+
52
+ ```text
53
+ For each unknown in Technical Context:
54
+ Task: "Research {unknown} for {feature context}"
55
+ For each technology choice:
56
+ Task: "Find best practices for {tech} in {domain}"
57
+ ```
58
+
59
+ 3. **Consolidate findings** in `research.md` using format:
60
+ - Decision: [what was chosen]
61
+ - Rationale: [why chosen]
62
+ - Alternatives considered: [what else evaluated]
63
+
64
+ **Output**: research.md with all NEEDS CLARIFICATION resolved
65
+
66
+ ### Phase 1: Design & Contracts
67
+
68
+ **Prerequisites:** `research.md` complete
69
+
70
+ 1. **Extract entities from feature spec** → `data-model.md`:
71
+ - Entity name, fields, relationships
72
+ - Validation rules from requirements
73
+ - State transitions if applicable
74
+
75
+ 2. **Generate API contracts** from functional requirements:
76
+ - For each user action → endpoint
77
+ - Use standard REST/GraphQL patterns
78
+ - Output OpenAPI/GraphQL schema to `/contracts/`
79
+
80
+ 3. **Agent context update**:
81
+ - Run `.specify/scripts/bash/update-agent-context.sh gemini`
82
+ - These scripts detect which AI agent is in use
83
+ - Update the appropriate agent-specific context file
84
+ - Add only new technology from current plan
85
+ - Preserve manual additions between markers
86
+
87
+ **Output**: data-model.md, /contracts/*, quickstart.md, agent-specific file
88
+
89
+ ## Key rules
90
+
91
+ - Use absolute paths
92
+ - ERROR on gate failures or unresolved clarifications
93
+ """
.gemini/commands/speckit.specify.toml ADDED
@@ -0,0 +1,262 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ description = "Create or update the feature specification from a natural language feature description."
2
+
3
+ prompt = """
4
+ ---
5
+ description: Create or update the feature specification from a natural language feature description.
6
+ handoffs:
7
+ - label: Build Technical Plan
8
+ agent: speckit.plan
9
+ prompt: Create a plan for the spec. I am building with...
10
+ - label: Clarify Spec Requirements
11
+ agent: speckit.clarify
12
+ prompt: Clarify specification requirements
13
+ send: true
14
+ ---
15
+
16
+ ## User Input
17
+
18
+ ```text
19
+ $ARGUMENTS
20
+ ```
21
+
22
+ You **MUST** consider the user input before proceeding (if not empty).
23
+
24
+ ## Outline
25
+
26
+ The text the user typed after `/speckit.specify` in the triggering message **is** the feature description. Assume you always have it available in this conversation even if `{{args}}` appears literally below. Do not ask the user to repeat it unless they provided an empty command.
27
+
28
+ Given that feature description, do this:
29
+
30
+ 1. **Generate a concise short name** (2-4 words) for the branch:
31
+ - Analyze the feature description and extract the most meaningful keywords
32
+ - Create a 2-4 word short name that captures the essence of the feature
33
+ - Use action-noun format when possible (e.g., "add-user-auth", "fix-payment-bug")
34
+ - Preserve technical terms and acronyms (OAuth2, API, JWT, etc.)
35
+ - Keep it concise but descriptive enough to understand the feature at a glance
36
+ - Examples:
37
+ - "I want to add user authentication" → "user-auth"
38
+ - "Implement OAuth2 integration for the API" → "oauth2-api-integration"
39
+ - "Create a dashboard for analytics" → "analytics-dashboard"
40
+ - "Fix payment processing timeout bug" → "fix-payment-timeout"
41
+
42
+ 2. **Check for existing branches before creating new one**:
43
+
44
+ a. First, fetch all remote branches to ensure we have the latest information:
45
+
46
+ ```bash
47
+ git fetch --all --prune
48
+ ```
49
+
50
+ b. Find the highest feature number across all sources for the short-name:
51
+ - Remote branches: `git ls-remote --heads origin | grep -E 'refs/heads/[0-9]+-<short-name>$'`
52
+ - Local branches: `git branch | grep -E '^[* ]*[0-9]+-<short-name>$'`
53
+ - Specs directories: Check for directories matching `specs/[0-9]+-<short-name>`
54
+
55
+ c. Determine the next available number:
56
+ - Extract all numbers from all three sources
57
+ - Find the highest number N
58
+ - Use N+1 for the new branch number
59
+
60
+ d. Run the script `.specify/scripts/bash/create-new-feature.sh --json "{{args}}"` with the calculated number and short-name:
61
+ - Pass `--number N+1` and `--short-name "your-short-name"` along with the feature description
62
+ - Bash example: `.specify/scripts/bash/create-new-feature.sh --json "{{args}}" --json --number 5 --short-name "user-auth" "Add user authentication"`
63
+ - PowerShell example: `.specify/scripts/bash/create-new-feature.sh --json "{{args}}" -Json -Number 5 -ShortName "user-auth" "Add user authentication"`
64
+
65
+ **IMPORTANT**:
66
+ - Check all three sources (remote branches, local branches, specs directories) to find the highest number
67
+ - Only match branches/directories with the exact short-name pattern
68
+ - If no existing branches/directories found with this short-name, start with number 1
69
+ - You must only ever run this script once per feature
70
+ - The JSON is provided in the terminal as output - always refer to it to get the actual content you're looking for
71
+ - The JSON output will contain BRANCH_NAME and SPEC_FILE paths
72
+ - For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\\''m Groot' (or double-quote if possible: "I'm Groot")
73
+
74
+ 3. Load `.specify/templates/spec-template.md` to understand required sections.
75
+
76
+ 4. Follow this execution flow:
77
+
78
+ 1. Parse user description from Input
79
+ If empty: ERROR "No feature description provided"
80
+ 2. Extract key concepts from description
81
+ Identify: actors, actions, data, constraints
82
+ 3. For unclear aspects:
83
+ - Make informed guesses based on context and industry standards
84
+ - Only mark with [NEEDS CLARIFICATION: specific question] if:
85
+ - The choice significantly impacts feature scope or user experience
86
+ - Multiple reasonable interpretations exist with different implications
87
+ - No reasonable default exists
88
+ - **LIMIT: Maximum 3 [NEEDS CLARIFICATION] markers total**
89
+ - Prioritize clarifications by impact: scope > security/privacy > user experience > technical details
90
+ 4. Fill User Scenarios & Testing section
91
+ If no clear user flow: ERROR "Cannot determine user scenarios"
92
+ 5. Generate Functional Requirements
93
+ Each requirement must be testable
94
+ Use reasonable defaults for unspecified details (document assumptions in Assumptions section)
95
+ 6. Define Success Criteria
96
+ Create measurable, technology-agnostic outcomes
97
+ Include both quantitative metrics (time, performance, volume) and qualitative measures (user satisfaction, task completion)
98
+ Each criterion must be verifiable without implementation details
99
+ 7. Identify Key Entities (if data involved)
100
+ 8. Return: SUCCESS (spec ready for planning)
101
+
102
+ 5. Write the specification to SPEC_FILE using the template structure, replacing placeholders with concrete details derived from the feature description (arguments) while preserving section order and headings.
103
+
104
+ 6. **Specification Quality Validation**: After writing the initial spec, validate it against quality criteria:
105
+
106
+ a. **Create Spec Quality Checklist**: Generate a checklist file at `FEATURE_DIR/checklists/requirements.md` using the checklist template structure with these validation items:
107
+
108
+ ```markdown
109
+ # Specification Quality Checklist: [FEATURE NAME]
110
+
111
+ **Purpose**: Validate specification completeness and quality before proceeding to planning
112
+ **Created**: [DATE]
113
+ **Feature**: [Link to spec.md]
114
+
115
+ ## Content Quality
116
+
117
+ - [ ] No implementation details (languages, frameworks, APIs)
118
+ - [ ] Focused on user value and business needs
119
+ - [ ] Written for non-technical stakeholders
120
+ - [ ] All mandatory sections completed
121
+
122
+ ## Requirement Completeness
123
+
124
+ - [ ] No [NEEDS CLARIFICATION] markers remain
125
+ - [ ] Requirements are testable and unambiguous
126
+ - [ ] Success criteria are measurable
127
+ - [ ] Success criteria are technology-agnostic (no implementation details)
128
+ - [ ] All acceptance scenarios are defined
129
+ - [ ] Edge cases are identified
130
+ - [ ] Scope is clearly bounded
131
+ - [ ] Dependencies and assumptions identified
132
+
133
+ ## Feature Readiness
134
+
135
+ - [ ] All functional requirements have clear acceptance criteria
136
+ - [ ] User scenarios cover primary flows
137
+ - [ ] Feature meets measurable outcomes defined in Success Criteria
138
+ - [ ] No implementation details leak into specification
139
+
140
+ ## Notes
141
+
142
+ - Items marked incomplete require spec updates before `/speckit.clarify` or `/speckit.plan`
143
+ ```
144
+
145
+ b. **Run Validation Check**: Review the spec against each checklist item:
146
+ - For each item, determine if it passes or fails
147
+ - Document specific issues found (quote relevant spec sections)
148
+
149
+ c. **Handle Validation Results**:
150
+
151
+ - **If all items pass**: Mark checklist complete and proceed to step 6
152
+
153
+ - **If items fail (excluding [NEEDS CLARIFICATION])**:
154
+ 1. List the failing items and specific issues
155
+ 2. Update the spec to address each issue
156
+ 3. Re-run validation until all items pass (max 3 iterations)
157
+ 4. If still failing after 3 iterations, document remaining issues in checklist notes and warn user
158
+
159
+ - **If [NEEDS CLARIFICATION] markers remain**:
160
+ 1. Extract all [NEEDS CLARIFICATION: ...] markers from the spec
161
+ 2. **LIMIT CHECK**: If more than 3 markers exist, keep only the 3 most critical (by scope/security/UX impact) and make informed guesses for the rest
162
+ 3. For each clarification needed (max 3), present options to user in this format:
163
+
164
+ ```markdown
165
+ ## Question [N]: [Topic]
166
+
167
+ **Context**: [Quote relevant spec section]
168
+
169
+ **What we need to know**: [Specific question from NEEDS CLARIFICATION marker]
170
+
171
+ **Suggested Answers**:
172
+
173
+ | Option | Answer | Implications |
174
+ |--------|--------|--------------|
175
+ | A | [First suggested answer] | [What this means for the feature] |
176
+ | B | [Second suggested answer] | [What this means for the feature] |
177
+ | C | [Third suggested answer] | [What this means for the feature] |
178
+ | Custom | Provide your own answer | [Explain how to provide custom input] |
179
+
180
+ **Your choice**: _[Wait for user response]_
181
+ ```
182
+
183
+ 4. **CRITICAL - Table Formatting**: Ensure markdown tables are properly formatted:
184
+ - Use consistent spacing with pipes aligned
185
+ - Each cell should have spaces around content: `| Content |` not `|Content|`
186
+ - Header separator must have at least 3 dashes: `|--------|`
187
+ - Test that the table renders correctly in markdown preview
188
+ 5. Number questions sequentially (Q1, Q2, Q3 - max 3 total)
189
+ 6. Present all questions together before waiting for responses
190
+ 7. Wait for user to respond with their choices for all questions (e.g., "Q1: A, Q2: Custom - [details], Q3: B")
191
+ 8. Update the spec by replacing each [NEEDS CLARIFICATION] marker with the user's selected or provided answer
192
+ 9. Re-run validation after all clarifications are resolved
193
+
194
+ d. **Update Checklist**: After each validation iteration, update the checklist file with current pass/fail status
195
+
196
+ 7. Report completion with branch name, spec file path, checklist results, and readiness for the next phase (`/speckit.clarify` or `/speckit.plan`).
197
+
198
+ **NOTE:** The script creates and checks out the new branch and initializes the spec file before writing.
199
+
200
+ ## General Guidelines
201
+
202
+ ## Quick Guidelines
203
+
204
+ - Focus on **WHAT** users need and **WHY**.
205
+ - Avoid HOW to implement (no tech stack, APIs, code structure).
206
+ - Written for business stakeholders, not developers.
207
+ - DO NOT create any checklists that are embedded in the spec. That will be a separate command.
208
+
209
+ ### Section Requirements
210
+
211
+ - **Mandatory sections**: Must be completed for every feature
212
+ - **Optional sections**: Include only when relevant to the feature
213
+ - When a section doesn't apply, remove it entirely (don't leave as "N/A")
214
+
215
+ ### For AI Generation
216
+
217
+ When creating this spec from a user prompt:
218
+
219
+ 1. **Make informed guesses**: Use context, industry standards, and common patterns to fill gaps
220
+ 2. **Document assumptions**: Record reasonable defaults in the Assumptions section
221
+ 3. **Limit clarifications**: Maximum 3 [NEEDS CLARIFICATION] markers - use only for critical decisions that:
222
+ - Significantly impact feature scope or user experience
223
+ - Have multiple reasonable interpretations with different implications
224
+ - Lack any reasonable default
225
+ 4. **Prioritize clarifications**: scope > security/privacy > user experience > technical details
226
+ 5. **Think like a tester**: Every vague requirement should fail the "testable and unambiguous" checklist item
227
+ 6. **Common areas needing clarification** (only if no reasonable default exists):
228
+ - Feature scope and boundaries (include/exclude specific use cases)
229
+ - User types and permissions (if multiple conflicting interpretations possible)
230
+ - Security/compliance requirements (when legally/financially significant)
231
+
232
+ **Examples of reasonable defaults** (don't ask about these):
233
+
234
+ - Data retention: Industry-standard practices for the domain
235
+ - Performance targets: Standard web/mobile app expectations unless specified
236
+ - Error handling: User-friendly messages with appropriate fallbacks
237
+ - Authentication method: Standard session-based or OAuth2 for web apps
238
+ - Integration patterns: RESTful APIs unless specified otherwise
239
+
240
+ ### Success Criteria Guidelines
241
+
242
+ Success criteria must be:
243
+
244
+ 1. **Measurable**: Include specific metrics (time, percentage, count, rate)
245
+ 2. **Technology-agnostic**: No mention of frameworks, languages, databases, or tools
246
+ 3. **User-focused**: Describe outcomes from user/business perspective, not system internals
247
+ 4. **Verifiable**: Can be tested/validated without knowing implementation details
248
+
249
+ **Good examples**:
250
+
251
+ - "Users can complete checkout in under 3 minutes"
252
+ - "System supports 10,000 concurrent users"
253
+ - "95% of searches return results in under 1 second"
254
+ - "Task completion rate improves by 40%"
255
+
256
+ **Bad examples** (implementation-focused):
257
+
258
+ - "API response time is under 200ms" (too technical, use "Users see results instantly")
259
+ - "Database can handle 1000 TPS" (implementation detail, use user-facing metric)
260
+ - "React components render efficiently" (framework-specific)
261
+ - "Redis cache hit rate above 80%" (technology-specific)
262
+ """
.gemini/commands/speckit.tasks.toml ADDED
@@ -0,0 +1,141 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ description = "Generate an actionable, dependency-ordered tasks.md for the feature based on available design artifacts."
2
+
3
+ prompt = """
4
+ ---
5
+ description: Generate an actionable, dependency-ordered tasks.md for the feature based on available design artifacts.
6
+ handoffs:
7
+ - label: Analyze For Consistency
8
+ agent: speckit.analyze
9
+ prompt: Run a project analysis for consistency
10
+ send: true
11
+ - label: Implement Project
12
+ agent: speckit.implement
13
+ prompt: Start the implementation in phases
14
+ send: true
15
+ ---
16
+
17
+ ## User Input
18
+
19
+ ```text
20
+ $ARGUMENTS
21
+ ```
22
+
23
+ You **MUST** consider the user input before proceeding (if not empty).
24
+
25
+ ## Outline
26
+
27
+ 1. **Setup**: Run `.specify/scripts/bash/check-prerequisites.sh --json` from repo root and parse FEATURE_DIR and AVAILABLE_DOCS list. All paths must be absolute. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\\''m Groot' (or double-quote if possible: "I'm Groot").
28
+
29
+ 2. **Load design documents**: Read from FEATURE_DIR:
30
+ - **Required**: plan.md (tech stack, libraries, structure), spec.md (user stories with priorities)
31
+ - **Optional**: data-model.md (entities), contracts/ (API endpoints), research.md (decisions), quickstart.md (test scenarios)
32
+ - Note: Not all projects have all documents. Generate tasks based on what's available.
33
+
34
+ 3. **Execute task generation workflow**:
35
+ - Load plan.md and extract tech stack, libraries, project structure
36
+ - Load spec.md and extract user stories with their priorities (P1, P2, P3, etc.)
37
+ - If data-model.md exists: Extract entities and map to user stories
38
+ - If contracts/ exists: Map endpoints to user stories
39
+ - If research.md exists: Extract decisions for setup tasks
40
+ - Generate tasks organized by user story (see Task Generation Rules below)
41
+ - Generate dependency graph showing user story completion order
42
+ - Create parallel execution examples per user story
43
+ - Validate task completeness (each user story has all needed tasks, independently testable)
44
+
45
+ 4. **Generate tasks.md**: Use `.specify/templates/tasks-template.md` as structure, fill with:
46
+ - Correct feature name from plan.md
47
+ - Phase 1: Setup tasks (project initialization)
48
+ - Phase 2: Foundational tasks (blocking prerequisites for all user stories)
49
+ - Phase 3+: One phase per user story (in priority order from spec.md)
50
+ - Each phase includes: story goal, independent test criteria, tests (if requested), implementation tasks
51
+ - Final Phase: Polish & cross-cutting concerns
52
+ - All tasks must follow the strict checklist format (see Task Generation Rules below)
53
+ - Clear file paths for each task
54
+ - Dependencies section showing story completion order
55
+ - Parallel execution examples per story
56
+ - Implementation strategy section (MVP first, incremental delivery)
57
+
58
+ 5. **Report**: Output path to generated tasks.md and summary:
59
+ - Total task count
60
+ - Task count per user story
61
+ - Parallel opportunities identified
62
+ - Independent test criteria for each story
63
+ - Suggested MVP scope (typically just User Story 1)
64
+ - Format validation: Confirm ALL tasks follow the checklist format (checkbox, ID, labels, file paths)
65
+
66
+ Context for task generation: {{args}}
67
+
68
+ The tasks.md should be immediately executable - each task must be specific enough that an LLM can complete it without additional context.
69
+
70
+ ## Task Generation Rules
71
+
72
+ **CRITICAL**: Tasks MUST be organized by user story to enable independent implementation and testing.
73
+
74
+ **Tests are OPTIONAL**: Only generate test tasks if explicitly requested in the feature specification or if user requests TDD approach.
75
+
76
+ ### Checklist Format (REQUIRED)
77
+
78
+ Every task MUST strictly follow this format:
79
+
80
+ ```text
81
+ - [ ] [TaskID] [P?] [Story?] Description with file path
82
+ ```
83
+
84
+ **Format Components**:
85
+
86
+ 1. **Checkbox**: ALWAYS start with `- [ ]` (markdown checkbox)
87
+ 2. **Task ID**: Sequential number (T001, T002, T003...) in execution order
88
+ 3. **[P] marker**: Include ONLY if task is parallelizable (different files, no dependencies on incomplete tasks)
89
+ 4. **[Story] label**: REQUIRED for user story phase tasks only
90
+ - Format: [US1], [US2], [US3], etc. (maps to user stories from spec.md)
91
+ - Setup phase: NO story label
92
+ - Foundational phase: NO story label
93
+ - User Story phases: MUST have story label
94
+ - Polish phase: NO story label
95
+ 5. **Description**: Clear action with exact file path
96
+
97
+ **Examples**:
98
+
99
+ - ✅ CORRECT: `- [ ] T001 Create project structure per implementation plan`
100
+ - ✅ CORRECT: `- [ ] T005 [P] Implement authentication middleware in src/middleware/auth.py`
101
+ - ✅ CORRECT: `- [ ] T012 [P] [US1] Create User model in src/models/user.py`
102
+ - ✅ CORRECT: `- [ ] T014 [US1] Implement UserService in src/services/user_service.py`
103
+ - ❌ WRONG: `- [ ] Create User model` (missing ID and Story label)
104
+ - ❌ WRONG: `T001 [US1] Create model` (missing checkbox)
105
+ - ❌ WRONG: `- [ ] [US1] Create User model` (missing Task ID)
106
+ - ❌ WRONG: `- [ ] T001 [US1] Create model` (missing file path)
107
+
108
+ ### Task Organization
109
+
110
+ 1. **From User Stories (spec.md)** - PRIMARY ORGANIZATION:
111
+ - Each user story (P1, P2, P3...) gets its own phase
112
+ - Map all related components to their story:
113
+ - Models needed for that story
114
+ - Services needed for that story
115
+ - Endpoints/UI needed for that story
116
+ - If tests requested: Tests specific to that story
117
+ - Mark story dependencies (most stories should be independent)
118
+
119
+ 2. **From Contracts**:
120
+ - Map each contract/endpoint → to the user story it serves
121
+ - If tests requested: Each contract → contract test task [P] before implementation in that story's phase
122
+
123
+ 3. **From Data Model**:
124
+ - Map each entity to the user story(ies) that need it
125
+ - If entity serves multiple stories: Put in earliest story or Setup phase
126
+ - Relationships → service layer tasks in appropriate story phase
127
+
128
+ 4. **From Setup/Infrastructure**:
129
+ - Shared infrastructure → Setup phase (Phase 1)
130
+ - Foundational/blocking tasks → Foundational phase (Phase 2)
131
+ - Story-specific setup → within that story's phase
132
+
133
+ ### Phase Structure
134
+
135
+ - **Phase 1**: Setup (project initialization)
136
+ - **Phase 2**: Foundational (blocking prerequisites - MUST complete before user stories)
137
+ - **Phase 3+**: User Stories in priority order (P1, P2, P3...)
138
+ - Within each story: Tests (if requested) → Models → Services → Endpoints → Integration
139
+ - Each phase should be a complete, independently testable increment
140
+ - **Final Phase**: Polish & Cross-Cutting Concerns
141
+ """
.gemini/commands/speckit.taskstoissues.toml ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ description = "Convert existing tasks into actionable, dependency-ordered GitHub issues for the feature based on available design artifacts."
2
+
3
+ prompt = """
4
+ ---
5
+ description: Convert existing tasks into actionable, dependency-ordered GitHub issues for the feature based on available design artifacts.
6
+ tools: ['github/github-mcp-server/issue_write']
7
+ ---
8
+
9
+ ## User Input
10
+
11
+ ```text
12
+ $ARGUMENTS
13
+ ```
14
+
15
+ You **MUST** consider the user input before proceeding (if not empty).
16
+
17
+ ## Outline
18
+
19
+ 1. Run `.specify/scripts/bash/check-prerequisites.sh --json --require-tasks --include-tasks` from repo root and parse FEATURE_DIR and AVAILABLE_DOCS list. All paths must be absolute. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\\''m Groot' (or double-quote if possible: "I'm Groot").
20
+ 1. From the executed script, extract the path to **tasks**.
21
+ 1. Get the Git remote by running:
22
+
23
+ ```bash
24
+ git config --get remote.origin.url
25
+ ```
26
+
27
+ > [!CAUTION]
28
+ > ONLY PROCEED TO NEXT STEPS IF THE REMOTE IS A GITHUB URL
29
+
30
+ 1. For each task in the list, use the GitHub MCP server to create a new issue in the repository that is representative of the Git remote.
31
+
32
+ > [!CAUTION]
33
+ > UNDER NO CIRCUMSTANCES EVER CREATE ISSUES IN REPOSITORIES THAT DO NOT MATCH THE REMOTE URL
34
+ """
.gitignore CHANGED
@@ -1,5 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
  /.quarto/
 
 
 
 
 
2
 
 
 
 
 
 
 
3
  .DS_Store
4
- .venv/**
5
- src/_site/
 
 
 
 
 
 
 
 
 
 
1
+ # Python
2
+ __pycache__/
3
+ *.pyc
4
+ *.pyo
5
+ *.pyd
6
+ .Python
7
+
8
+ # Pixi
9
+ .pixi/
10
+ pixi.lock
11
+
12
+ # Quarto
13
  /.quarto/
14
+ /_site/
15
+ src/_site/
16
+
17
+ # Jupyter
18
+ .ipynb_checkpoints/
19
 
20
+ # Virtual environments
21
+ .venv/
22
+ venv/
23
+ ENV/
24
+
25
+ # OS
26
  .DS_Store
27
+ Thumbs.db
28
+
29
+ # IDE
30
+ .vscode/
31
+ .idea/
32
+ *.swp
33
+ *.swo
34
+
35
+ # Quarto
36
+ */_book/
37
+ */_site/
.specify/memory/constitution.md ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # [PROJECT_NAME] Constitution
2
+ <!-- Example: Spec Constitution, TaskFlow Constitution, etc. -->
3
+
4
+ ## Core Principles
5
+
6
+ ### [PRINCIPLE_1_NAME]
7
+ <!-- Example: I. Library-First -->
8
+ [PRINCIPLE_1_DESCRIPTION]
9
+ <!-- Example: Every feature starts as a standalone library; Libraries must be self-contained, independently testable, documented; Clear purpose required - no organizational-only libraries -->
10
+
11
+ ### [PRINCIPLE_2_NAME]
12
+ <!-- Example: II. CLI Interface -->
13
+ [PRINCIPLE_2_DESCRIPTION]
14
+ <!-- Example: Every library exposes functionality via CLI; Text in/out protocol: stdin/args → stdout, errors → stderr; Support JSON + human-readable formats -->
15
+
16
+ ### [PRINCIPLE_3_NAME]
17
+ <!-- Example: III. Test-First (NON-NEGOTIABLE) -->
18
+ [PRINCIPLE_3_DESCRIPTION]
19
+ <!-- Example: TDD mandatory: Tests written → User approved → Tests fail → Then implement; Red-Green-Refactor cycle strictly enforced -->
20
+
21
+ ### [PRINCIPLE_4_NAME]
22
+ <!-- Example: IV. Integration Testing -->
23
+ [PRINCIPLE_4_DESCRIPTION]
24
+ <!-- Example: Focus areas requiring integration tests: New library contract tests, Contract changes, Inter-service communication, Shared schemas -->
25
+
26
+ ### [PRINCIPLE_5_NAME]
27
+ <!-- Example: V. Observability, VI. Versioning & Breaking Changes, VII. Simplicity -->
28
+ [PRINCIPLE_5_DESCRIPTION]
29
+ <!-- Example: Text I/O ensures debuggability; Structured logging required; Or: MAJOR.MINOR.BUILD format; Or: Start simple, YAGNI principles -->
30
+
31
+ ## [SECTION_2_NAME]
32
+ <!-- Example: Additional Constraints, Security Requirements, Performance Standards, etc. -->
33
+
34
+ [SECTION_2_CONTENT]
35
+ <!-- Example: Technology stack requirements, compliance standards, deployment policies, etc. -->
36
+
37
+ ## [SECTION_3_NAME]
38
+ <!-- Example: Development Workflow, Review Process, Quality Gates, etc. -->
39
+
40
+ [SECTION_3_CONTENT]
41
+ <!-- Example: Code review requirements, testing gates, deployment approval process, etc. -->
42
+
43
+ ## Governance
44
+ <!-- Example: Constitution supersedes all other practices; Amendments require documentation, approval, migration plan -->
45
+
46
+ [GOVERNANCE_RULES]
47
+ <!-- Example: All PRs/reviews must verify compliance; Complexity must be justified; Use [GUIDANCE_FILE] for runtime development guidance -->
48
+
49
+ **Version**: [CONSTITUTION_VERSION] | **Ratified**: [RATIFICATION_DATE] | **Last Amended**: [LAST_AMENDED_DATE]
50
+ <!-- Example: Version: 2.1.1 | Ratified: 2025-06-13 | Last Amended: 2025-07-16 -->
.specify/scripts/bash/check-prerequisites.sh ADDED
@@ -0,0 +1,166 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env bash
2
+
3
+ # Consolidated prerequisite checking script
4
+ #
5
+ # This script provides unified prerequisite checking for Spec-Driven Development workflow.
6
+ # It replaces the functionality previously spread across multiple scripts.
7
+ #
8
+ # Usage: ./check-prerequisites.sh [OPTIONS]
9
+ #
10
+ # OPTIONS:
11
+ # --json Output in JSON format
12
+ # --require-tasks Require tasks.md to exist (for implementation phase)
13
+ # --include-tasks Include tasks.md in AVAILABLE_DOCS list
14
+ # --paths-only Only output path variables (no validation)
15
+ # --help, -h Show help message
16
+ #
17
+ # OUTPUTS:
18
+ # JSON mode: {"FEATURE_DIR":"...", "AVAILABLE_DOCS":["..."]}
19
+ # Text mode: FEATURE_DIR:... \n AVAILABLE_DOCS: \n ✓/✗ file.md
20
+ # Paths only: REPO_ROOT: ... \n BRANCH: ... \n FEATURE_DIR: ... etc.
21
+
22
+ set -e
23
+
24
+ # Parse command line arguments
25
+ JSON_MODE=false
26
+ REQUIRE_TASKS=false
27
+ INCLUDE_TASKS=false
28
+ PATHS_ONLY=false
29
+
30
+ for arg in "$@"; do
31
+ case "$arg" in
32
+ --json)
33
+ JSON_MODE=true
34
+ ;;
35
+ --require-tasks)
36
+ REQUIRE_TASKS=true
37
+ ;;
38
+ --include-tasks)
39
+ INCLUDE_TASKS=true
40
+ ;;
41
+ --paths-only)
42
+ PATHS_ONLY=true
43
+ ;;
44
+ --help|-h)
45
+ cat << 'EOF'
46
+ Usage: check-prerequisites.sh [OPTIONS]
47
+
48
+ Consolidated prerequisite checking for Spec-Driven Development workflow.
49
+
50
+ OPTIONS:
51
+ --json Output in JSON format
52
+ --require-tasks Require tasks.md to exist (for implementation phase)
53
+ --include-tasks Include tasks.md in AVAILABLE_DOCS list
54
+ --paths-only Only output path variables (no prerequisite validation)
55
+ --help, -h Show this help message
56
+
57
+ EXAMPLES:
58
+ # Check task prerequisites (plan.md required)
59
+ ./check-prerequisites.sh --json
60
+
61
+ # Check implementation prerequisites (plan.md + tasks.md required)
62
+ ./check-prerequisites.sh --json --require-tasks --include-tasks
63
+
64
+ # Get feature paths only (no validation)
65
+ ./check-prerequisites.sh --paths-only
66
+
67
+ EOF
68
+ exit 0
69
+ ;;
70
+ *)
71
+ echo "ERROR: Unknown option '$arg'. Use --help for usage information." >&2
72
+ exit 1
73
+ ;;
74
+ esac
75
+ done
76
+
77
+ # Source common functions
78
+ SCRIPT_DIR="$(CDPATH="" cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
79
+ source "$SCRIPT_DIR/common.sh"
80
+
81
+ # Get feature paths and validate branch
82
+ eval $(get_feature_paths)
83
+ check_feature_branch "$CURRENT_BRANCH" "$HAS_GIT" || exit 1
84
+
85
+ # If paths-only mode, output paths and exit (support JSON + paths-only combined)
86
+ if $PATHS_ONLY; then
87
+ if $JSON_MODE; then
88
+ # Minimal JSON paths payload (no validation performed)
89
+ printf '{"REPO_ROOT":"%s","BRANCH":"%s","FEATURE_DIR":"%s","FEATURE_SPEC":"%s","IMPL_PLAN":"%s","TASKS":"%s"}\n' \
90
+ "$REPO_ROOT" "$CURRENT_BRANCH" "$FEATURE_DIR" "$FEATURE_SPEC" "$IMPL_PLAN" "$TASKS"
91
+ else
92
+ echo "REPO_ROOT: $REPO_ROOT"
93
+ echo "BRANCH: $CURRENT_BRANCH"
94
+ echo "FEATURE_DIR: $FEATURE_DIR"
95
+ echo "FEATURE_SPEC: $FEATURE_SPEC"
96
+ echo "IMPL_PLAN: $IMPL_PLAN"
97
+ echo "TASKS: $TASKS"
98
+ fi
99
+ exit 0
100
+ fi
101
+
102
+ # Validate required directories and files
103
+ if [[ ! -d "$FEATURE_DIR" ]]; then
104
+ echo "ERROR: Feature directory not found: $FEATURE_DIR" >&2
105
+ echo "Run /speckit.specify first to create the feature structure." >&2
106
+ exit 1
107
+ fi
108
+
109
+ if [[ ! -f "$IMPL_PLAN" ]]; then
110
+ echo "ERROR: plan.md not found in $FEATURE_DIR" >&2
111
+ echo "Run /speckit.plan first to create the implementation plan." >&2
112
+ exit 1
113
+ fi
114
+
115
+ # Check for tasks.md if required
116
+ if $REQUIRE_TASKS && [[ ! -f "$TASKS" ]]; then
117
+ echo "ERROR: tasks.md not found in $FEATURE_DIR" >&2
118
+ echo "Run /speckit.tasks first to create the task list." >&2
119
+ exit 1
120
+ fi
121
+
122
+ # Build list of available documents
123
+ docs=()
124
+
125
+ # Always check these optional docs
126
+ [[ -f "$RESEARCH" ]] && docs+=("research.md")
127
+ [[ -f "$DATA_MODEL" ]] && docs+=("data-model.md")
128
+
129
+ # Check contracts directory (only if it exists and has files)
130
+ if [[ -d "$CONTRACTS_DIR" ]] && [[ -n "$(ls -A "$CONTRACTS_DIR" 2>/dev/null)" ]]; then
131
+ docs+=("contracts/")
132
+ fi
133
+
134
+ [[ -f "$QUICKSTART" ]] && docs+=("quickstart.md")
135
+
136
+ # Include tasks.md if requested and it exists
137
+ if $INCLUDE_TASKS && [[ -f "$TASKS" ]]; then
138
+ docs+=("tasks.md")
139
+ fi
140
+
141
+ # Output results
142
+ if $JSON_MODE; then
143
+ # Build JSON array of documents
144
+ if [[ ${#docs[@]} -eq 0 ]]; then
145
+ json_docs="[]"
146
+ else
147
+ json_docs=$(printf '"%s",' "${docs[@]}")
148
+ json_docs="[${json_docs%,}]"
149
+ fi
150
+
151
+ printf '{"FEATURE_DIR":"%s","AVAILABLE_DOCS":%s}\n' "$FEATURE_DIR" "$json_docs"
152
+ else
153
+ # Text output
154
+ echo "FEATURE_DIR:$FEATURE_DIR"
155
+ echo "AVAILABLE_DOCS:"
156
+
157
+ # Show status of each potential document
158
+ check_file "$RESEARCH" "research.md"
159
+ check_file "$DATA_MODEL" "data-model.md"
160
+ check_dir "$CONTRACTS_DIR" "contracts/"
161
+ check_file "$QUICKSTART" "quickstart.md"
162
+
163
+ if $INCLUDE_TASKS; then
164
+ check_file "$TASKS" "tasks.md"
165
+ fi
166
+ fi
.specify/scripts/bash/common.sh ADDED
@@ -0,0 +1,156 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env bash
2
+ # Common functions and variables for all scripts
3
+
4
+ # Get repository root, with fallback for non-git repositories
5
+ get_repo_root() {
6
+ if git rev-parse --show-toplevel >/dev/null 2>&1; then
7
+ git rev-parse --show-toplevel
8
+ else
9
+ # Fall back to script location for non-git repos
10
+ local script_dir="$(CDPATH="" cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
11
+ (cd "$script_dir/../../.." && pwd)
12
+ fi
13
+ }
14
+
15
+ # Get current branch, with fallback for non-git repositories
16
+ get_current_branch() {
17
+ # First check if SPECIFY_FEATURE environment variable is set
18
+ if [[ -n "${SPECIFY_FEATURE:-}" ]]; then
19
+ echo "$SPECIFY_FEATURE"
20
+ return
21
+ fi
22
+
23
+ # Then check git if available
24
+ if git rev-parse --abbrev-ref HEAD >/dev/null 2>&1; then
25
+ git rev-parse --abbrev-ref HEAD
26
+ return
27
+ fi
28
+
29
+ # For non-git repos, try to find the latest feature directory
30
+ local repo_root=$(get_repo_root)
31
+ local specs_dir="$repo_root/specs"
32
+
33
+ if [[ -d "$specs_dir" ]]; then
34
+ local latest_feature=""
35
+ local highest=0
36
+
37
+ for dir in "$specs_dir"/*; do
38
+ if [[ -d "$dir" ]]; then
39
+ local dirname=$(basename "$dir")
40
+ if [[ "$dirname" =~ ^([0-9]{3})- ]]; then
41
+ local number=${BASH_REMATCH[1]}
42
+ number=$((10#$number))
43
+ if [[ "$number" -gt "$highest" ]]; then
44
+ highest=$number
45
+ latest_feature=$dirname
46
+ fi
47
+ fi
48
+ fi
49
+ done
50
+
51
+ if [[ -n "$latest_feature" ]]; then
52
+ echo "$latest_feature"
53
+ return
54
+ fi
55
+ fi
56
+
57
+ echo "main" # Final fallback
58
+ }
59
+
60
+ # Check if we have git available
61
+ has_git() {
62
+ git rev-parse --show-toplevel >/dev/null 2>&1
63
+ }
64
+
65
+ check_feature_branch() {
66
+ local branch="$1"
67
+ local has_git_repo="$2"
68
+
69
+ # For non-git repos, we can't enforce branch naming but still provide output
70
+ if [[ "$has_git_repo" != "true" ]]; then
71
+ echo "[specify] Warning: Git repository not detected; skipped branch validation" >&2
72
+ return 0
73
+ fi
74
+
75
+ if [[ ! "$branch" =~ ^[0-9]{3}- ]]; then
76
+ echo "ERROR: Not on a feature branch. Current branch: $branch" >&2
77
+ echo "Feature branches should be named like: 001-feature-name" >&2
78
+ return 1
79
+ fi
80
+
81
+ return 0
82
+ }
83
+
84
+ get_feature_dir() { echo "$1/specs/$2"; }
85
+
86
+ # Find feature directory by numeric prefix instead of exact branch match
87
+ # This allows multiple branches to work on the same spec (e.g., 004-fix-bug, 004-add-feature)
88
+ find_feature_dir_by_prefix() {
89
+ local repo_root="$1"
90
+ local branch_name="$2"
91
+ local specs_dir="$repo_root/specs"
92
+
93
+ # Extract numeric prefix from branch (e.g., "004" from "004-whatever")
94
+ if [[ ! "$branch_name" =~ ^([0-9]{3})- ]]; then
95
+ # If branch doesn't have numeric prefix, fall back to exact match
96
+ echo "$specs_dir/$branch_name"
97
+ return
98
+ fi
99
+
100
+ local prefix="${BASH_REMATCH[1]}"
101
+
102
+ # Search for directories in specs/ that start with this prefix
103
+ local matches=()
104
+ if [[ -d "$specs_dir" ]]; then
105
+ for dir in "$specs_dir"/"$prefix"-*; do
106
+ if [[ -d "$dir" ]]; then
107
+ matches+=("$(basename "$dir")")
108
+ fi
109
+ done
110
+ fi
111
+
112
+ # Handle results
113
+ if [[ ${#matches[@]} -eq 0 ]]; then
114
+ # No match found - return the branch name path (will fail later with clear error)
115
+ echo "$specs_dir/$branch_name"
116
+ elif [[ ${#matches[@]} -eq 1 ]]; then
117
+ # Exactly one match - perfect!
118
+ echo "$specs_dir/${matches[0]}"
119
+ else
120
+ # Multiple matches - this shouldn't happen with proper naming convention
121
+ echo "ERROR: Multiple spec directories found with prefix '$prefix': ${matches[*]}" >&2
122
+ echo "Please ensure only one spec directory exists per numeric prefix." >&2
123
+ echo "$specs_dir/$branch_name" # Return something to avoid breaking the script
124
+ fi
125
+ }
126
+
127
+ get_feature_paths() {
128
+ local repo_root=$(get_repo_root)
129
+ local current_branch=$(get_current_branch)
130
+ local has_git_repo="false"
131
+
132
+ if has_git; then
133
+ has_git_repo="true"
134
+ fi
135
+
136
+ # Use prefix-based lookup to support multiple branches per spec
137
+ local feature_dir=$(find_feature_dir_by_prefix "$repo_root" "$current_branch")
138
+
139
+ cat <<EOF
140
+ REPO_ROOT='$repo_root'
141
+ CURRENT_BRANCH='$current_branch'
142
+ HAS_GIT='$has_git_repo'
143
+ FEATURE_DIR='$feature_dir'
144
+ FEATURE_SPEC='$feature_dir/spec.md'
145
+ IMPL_PLAN='$feature_dir/plan.md'
146
+ TASKS='$feature_dir/tasks.md'
147
+ RESEARCH='$feature_dir/research.md'
148
+ DATA_MODEL='$feature_dir/data-model.md'
149
+ QUICKSTART='$feature_dir/quickstart.md'
150
+ CONTRACTS_DIR='$feature_dir/contracts'
151
+ EOF
152
+ }
153
+
154
+ check_file() { [[ -f "$1" ]] && echo " ✓ $2" || echo " ✗ $2"; }
155
+ check_dir() { [[ -d "$1" && -n $(ls -A "$1" 2>/dev/null) ]] && echo " ✓ $2" || echo " ✗ $2"; }
156
+
.specify/scripts/bash/create-new-feature.sh ADDED
@@ -0,0 +1,297 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env bash
2
+
3
+ set -e
4
+
5
+ JSON_MODE=false
6
+ SHORT_NAME=""
7
+ BRANCH_NUMBER=""
8
+ ARGS=()
9
+ i=1
10
+ while [ $i -le $# ]; do
11
+ arg="${!i}"
12
+ case "$arg" in
13
+ --json)
14
+ JSON_MODE=true
15
+ ;;
16
+ --short-name)
17
+ if [ $((i + 1)) -gt $# ]; then
18
+ echo 'Error: --short-name requires a value' >&2
19
+ exit 1
20
+ fi
21
+ i=$((i + 1))
22
+ next_arg="${!i}"
23
+ # Check if the next argument is another option (starts with --)
24
+ if [[ "$next_arg" == --* ]]; then
25
+ echo 'Error: --short-name requires a value' >&2
26
+ exit 1
27
+ fi
28
+ SHORT_NAME="$next_arg"
29
+ ;;
30
+ --number)
31
+ if [ $((i + 1)) -gt $# ]; then
32
+ echo 'Error: --number requires a value' >&2
33
+ exit 1
34
+ fi
35
+ i=$((i + 1))
36
+ next_arg="${!i}"
37
+ if [[ "$next_arg" == --* ]]; then
38
+ echo 'Error: --number requires a value' >&2
39
+ exit 1
40
+ fi
41
+ BRANCH_NUMBER="$next_arg"
42
+ ;;
43
+ --help|-h)
44
+ echo "Usage: $0 [--json] [--short-name <name>] [--number N] <feature_description>"
45
+ echo ""
46
+ echo "Options:"
47
+ echo " --json Output in JSON format"
48
+ echo " --short-name <name> Provide a custom short name (2-4 words) for the branch"
49
+ echo " --number N Specify branch number manually (overrides auto-detection)"
50
+ echo " --help, -h Show this help message"
51
+ echo ""
52
+ echo "Examples:"
53
+ echo " $0 'Add user authentication system' --short-name 'user-auth'"
54
+ echo " $0 'Implement OAuth2 integration for API' --number 5"
55
+ exit 0
56
+ ;;
57
+ *)
58
+ ARGS+=("$arg")
59
+ ;;
60
+ esac
61
+ i=$((i + 1))
62
+ done
63
+
64
+ FEATURE_DESCRIPTION="${ARGS[*]}"
65
+ if [ -z "$FEATURE_DESCRIPTION" ]; then
66
+ echo "Usage: $0 [--json] [--short-name <name>] [--number N] <feature_description>" >&2
67
+ exit 1
68
+ fi
69
+
70
+ # Function to find the repository root by searching for existing project markers
71
+ find_repo_root() {
72
+ local dir="$1"
73
+ while [ "$dir" != "/" ]; do
74
+ if [ -d "$dir/.git" ] || [ -d "$dir/.specify" ]; then
75
+ echo "$dir"
76
+ return 0
77
+ fi
78
+ dir="$(dirname "$dir")"
79
+ done
80
+ return 1
81
+ }
82
+
83
+ # Function to get highest number from specs directory
84
+ get_highest_from_specs() {
85
+ local specs_dir="$1"
86
+ local highest=0
87
+
88
+ if [ -d "$specs_dir" ]; then
89
+ for dir in "$specs_dir"/*; do
90
+ [ -d "$dir" ] || continue
91
+ dirname=$(basename "$dir")
92
+ number=$(echo "$dirname" | grep -o '^[0-9]\+' || echo "0")
93
+ number=$((10#$number))
94
+ if [ "$number" -gt "$highest" ]; then
95
+ highest=$number
96
+ fi
97
+ done
98
+ fi
99
+
100
+ echo "$highest"
101
+ }
102
+
103
+ # Function to get highest number from git branches
104
+ get_highest_from_branches() {
105
+ local highest=0
106
+
107
+ # Get all branches (local and remote)
108
+ branches=$(git branch -a 2>/dev/null || echo "")
109
+
110
+ if [ -n "$branches" ]; then
111
+ while IFS= read -r branch; do
112
+ # Clean branch name: remove leading markers and remote prefixes
113
+ clean_branch=$(echo "$branch" | sed 's/^[* ]*//; s|^remotes/[^/]*/||')
114
+
115
+ # Extract feature number if branch matches pattern ###-*
116
+ if echo "$clean_branch" | grep -q '^[0-9]\{3\}-'; then
117
+ number=$(echo "$clean_branch" | grep -o '^[0-9]\{3\}' || echo "0")
118
+ number=$((10#$number))
119
+ if [ "$number" -gt "$highest" ]; then
120
+ highest=$number
121
+ fi
122
+ fi
123
+ done <<< "$branches"
124
+ fi
125
+
126
+ echo "$highest"
127
+ }
128
+
129
+ # Function to check existing branches (local and remote) and return next available number
130
+ check_existing_branches() {
131
+ local specs_dir="$1"
132
+
133
+ # Fetch all remotes to get latest branch info (suppress errors if no remotes)
134
+ git fetch --all --prune 2>/dev/null || true
135
+
136
+ # Get highest number from ALL branches (not just matching short name)
137
+ local highest_branch=$(get_highest_from_branches)
138
+
139
+ # Get highest number from ALL specs (not just matching short name)
140
+ local highest_spec=$(get_highest_from_specs "$specs_dir")
141
+
142
+ # Take the maximum of both
143
+ local max_num=$highest_branch
144
+ if [ "$highest_spec" -gt "$max_num" ]; then
145
+ max_num=$highest_spec
146
+ fi
147
+
148
+ # Return next number
149
+ echo $((max_num + 1))
150
+ }
151
+
152
+ # Function to clean and format a branch name
153
+ clean_branch_name() {
154
+ local name="$1"
155
+ echo "$name" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9]/-/g' | sed 's/-\+/-/g' | sed 's/^-//' | sed 's/-$//'
156
+ }
157
+
158
+ # Resolve repository root. Prefer git information when available, but fall back
159
+ # to searching for repository markers so the workflow still functions in repositories that
160
+ # were initialised with --no-git.
161
+ SCRIPT_DIR="$(CDPATH="" cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
162
+
163
+ if git rev-parse --show-toplevel >/dev/null 2>&1; then
164
+ REPO_ROOT=$(git rev-parse --show-toplevel)
165
+ HAS_GIT=true
166
+ else
167
+ REPO_ROOT="$(find_repo_root "$SCRIPT_DIR")"
168
+ if [ -z "$REPO_ROOT" ]; then
169
+ echo "Error: Could not determine repository root. Please run this script from within the repository." >&2
170
+ exit 1
171
+ fi
172
+ HAS_GIT=false
173
+ fi
174
+
175
+ cd "$REPO_ROOT"
176
+
177
+ SPECS_DIR="$REPO_ROOT/specs"
178
+ mkdir -p "$SPECS_DIR"
179
+
180
+ # Function to generate branch name with stop word filtering and length filtering
181
+ generate_branch_name() {
182
+ local description="$1"
183
+
184
+ # Common stop words to filter out
185
+ local stop_words="^(i|a|an|the|to|for|of|in|on|at|by|with|from|is|are|was|were|be|been|being|have|has|had|do|does|did|will|would|should|could|can|may|might|must|shall|this|that|these|those|my|your|our|their|want|need|add|get|set)$"
186
+
187
+ # Convert to lowercase and split into words
188
+ local clean_name=$(echo "$description" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9]/ /g')
189
+
190
+ # Filter words: remove stop words and words shorter than 3 chars (unless they're uppercase acronyms in original)
191
+ local meaningful_words=()
192
+ for word in $clean_name; do
193
+ # Skip empty words
194
+ [ -z "$word" ] && continue
195
+
196
+ # Keep words that are NOT stop words AND (length >= 3 OR are potential acronyms)
197
+ if ! echo "$word" | grep -qiE "$stop_words"; then
198
+ if [ ${#word} -ge 3 ]; then
199
+ meaningful_words+=("$word")
200
+ elif echo "$description" | grep -q "\b${word^^}\b"; then
201
+ # Keep short words if they appear as uppercase in original (likely acronyms)
202
+ meaningful_words+=("$word")
203
+ fi
204
+ fi
205
+ done
206
+
207
+ # If we have meaningful words, use first 3-4 of them
208
+ if [ ${#meaningful_words[@]} -gt 0 ]; then
209
+ local max_words=3
210
+ if [ ${#meaningful_words[@]} -eq 4 ]; then max_words=4; fi
211
+
212
+ local result=""
213
+ local count=0
214
+ for word in "${meaningful_words[@]}"; do
215
+ if [ $count -ge $max_words ]; then break; fi
216
+ if [ -n "$result" ]; then result="$result-"; fi
217
+ result="$result$word"
218
+ count=$((count + 1))
219
+ done
220
+ echo "$result"
221
+ else
222
+ # Fallback to original logic if no meaningful words found
223
+ local cleaned=$(clean_branch_name "$description")
224
+ echo "$cleaned" | tr '-' '\n' | grep -v '^$' | head -3 | tr '\n' '-' | sed 's/-$//'
225
+ fi
226
+ }
227
+
228
+ # Generate branch name
229
+ if [ -n "$SHORT_NAME" ]; then
230
+ # Use provided short name, just clean it up
231
+ BRANCH_SUFFIX=$(clean_branch_name "$SHORT_NAME")
232
+ else
233
+ # Generate from description with smart filtering
234
+ BRANCH_SUFFIX=$(generate_branch_name "$FEATURE_DESCRIPTION")
235
+ fi
236
+
237
+ # Determine branch number
238
+ if [ -z "$BRANCH_NUMBER" ]; then
239
+ if [ "$HAS_GIT" = true ]; then
240
+ # Check existing branches on remotes
241
+ BRANCH_NUMBER=$(check_existing_branches "$SPECS_DIR")
242
+ else
243
+ # Fall back to local directory check
244
+ HIGHEST=$(get_highest_from_specs "$SPECS_DIR")
245
+ BRANCH_NUMBER=$((HIGHEST + 1))
246
+ fi
247
+ fi
248
+
249
+ # Force base-10 interpretation to prevent octal conversion (e.g., 010 → 8 in octal, but should be 10 in decimal)
250
+ FEATURE_NUM=$(printf "%03d" "$((10#$BRANCH_NUMBER))")
251
+ BRANCH_NAME="${FEATURE_NUM}-${BRANCH_SUFFIX}"
252
+
253
+ # GitHub enforces a 244-byte limit on branch names
254
+ # Validate and truncate if necessary
255
+ MAX_BRANCH_LENGTH=244
256
+ if [ ${#BRANCH_NAME} -gt $MAX_BRANCH_LENGTH ]; then
257
+ # Calculate how much we need to trim from suffix
258
+ # Account for: feature number (3) + hyphen (1) = 4 chars
259
+ MAX_SUFFIX_LENGTH=$((MAX_BRANCH_LENGTH - 4))
260
+
261
+ # Truncate suffix at word boundary if possible
262
+ TRUNCATED_SUFFIX=$(echo "$BRANCH_SUFFIX" | cut -c1-$MAX_SUFFIX_LENGTH)
263
+ # Remove trailing hyphen if truncation created one
264
+ TRUNCATED_SUFFIX=$(echo "$TRUNCATED_SUFFIX" | sed 's/-$//')
265
+
266
+ ORIGINAL_BRANCH_NAME="$BRANCH_NAME"
267
+ BRANCH_NAME="${FEATURE_NUM}-${TRUNCATED_SUFFIX}"
268
+
269
+ >&2 echo "[specify] Warning: Branch name exceeded GitHub's 244-byte limit"
270
+ >&2 echo "[specify] Original: $ORIGINAL_BRANCH_NAME (${#ORIGINAL_BRANCH_NAME} bytes)"
271
+ >&2 echo "[specify] Truncated to: $BRANCH_NAME (${#BRANCH_NAME} bytes)"
272
+ fi
273
+
274
+ if [ "$HAS_GIT" = true ]; then
275
+ git checkout -b "$BRANCH_NAME"
276
+ else
277
+ >&2 echo "[specify] Warning: Git repository not detected; skipped branch creation for $BRANCH_NAME"
278
+ fi
279
+
280
+ FEATURE_DIR="$SPECS_DIR/$BRANCH_NAME"
281
+ mkdir -p "$FEATURE_DIR"
282
+
283
+ TEMPLATE="$REPO_ROOT/.specify/templates/spec-template.md"
284
+ SPEC_FILE="$FEATURE_DIR/spec.md"
285
+ if [ -f "$TEMPLATE" ]; then cp "$TEMPLATE" "$SPEC_FILE"; else touch "$SPEC_FILE"; fi
286
+
287
+ # Set the SPECIFY_FEATURE environment variable for the current session
288
+ export SPECIFY_FEATURE="$BRANCH_NAME"
289
+
290
+ if $JSON_MODE; then
291
+ printf '{"BRANCH_NAME":"%s","SPEC_FILE":"%s","FEATURE_NUM":"%s"}\n' "$BRANCH_NAME" "$SPEC_FILE" "$FEATURE_NUM"
292
+ else
293
+ echo "BRANCH_NAME: $BRANCH_NAME"
294
+ echo "SPEC_FILE: $SPEC_FILE"
295
+ echo "FEATURE_NUM: $FEATURE_NUM"
296
+ echo "SPECIFY_FEATURE environment variable set to: $BRANCH_NAME"
297
+ fi
.specify/scripts/bash/setup-plan.sh ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env bash
2
+
3
+ set -e
4
+
5
+ # Parse command line arguments
6
+ JSON_MODE=false
7
+ ARGS=()
8
+
9
+ for arg in "$@"; do
10
+ case "$arg" in
11
+ --json)
12
+ JSON_MODE=true
13
+ ;;
14
+ --help|-h)
15
+ echo "Usage: $0 [--json]"
16
+ echo " --json Output results in JSON format"
17
+ echo " --help Show this help message"
18
+ exit 0
19
+ ;;
20
+ *)
21
+ ARGS+=("$arg")
22
+ ;;
23
+ esac
24
+ done
25
+
26
+ # Get script directory and load common functions
27
+ SCRIPT_DIR="$(CDPATH="" cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
28
+ source "$SCRIPT_DIR/common.sh"
29
+
30
+ # Get all paths and variables from common functions
31
+ eval $(get_feature_paths)
32
+
33
+ # Check if we're on a proper feature branch (only for git repos)
34
+ check_feature_branch "$CURRENT_BRANCH" "$HAS_GIT" || exit 1
35
+
36
+ # Ensure the feature directory exists
37
+ mkdir -p "$FEATURE_DIR"
38
+
39
+ # Copy plan template if it exists
40
+ TEMPLATE="$REPO_ROOT/.specify/templates/plan-template.md"
41
+ if [[ -f "$TEMPLATE" ]]; then
42
+ cp "$TEMPLATE" "$IMPL_PLAN"
43
+ echo "Copied plan template to $IMPL_PLAN"
44
+ else
45
+ echo "Warning: Plan template not found at $TEMPLATE"
46
+ # Create a basic plan file if template doesn't exist
47
+ touch "$IMPL_PLAN"
48
+ fi
49
+
50
+ # Output results
51
+ if $JSON_MODE; then
52
+ printf '{"FEATURE_SPEC":"%s","IMPL_PLAN":"%s","SPECS_DIR":"%s","BRANCH":"%s","HAS_GIT":"%s"}\n' \
53
+ "$FEATURE_SPEC" "$IMPL_PLAN" "$FEATURE_DIR" "$CURRENT_BRANCH" "$HAS_GIT"
54
+ else
55
+ echo "FEATURE_SPEC: $FEATURE_SPEC"
56
+ echo "IMPL_PLAN: $IMPL_PLAN"
57
+ echo "SPECS_DIR: $FEATURE_DIR"
58
+ echo "BRANCH: $CURRENT_BRANCH"
59
+ echo "HAS_GIT: $HAS_GIT"
60
+ fi
61
+
.specify/scripts/bash/update-agent-context.sh ADDED
@@ -0,0 +1,799 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env bash
2
+
3
+ # Update agent context files with information from plan.md
4
+ #
5
+ # This script maintains AI agent context files by parsing feature specifications
6
+ # and updating agent-specific configuration files with project information.
7
+ #
8
+ # MAIN FUNCTIONS:
9
+ # 1. Environment Validation
10
+ # - Verifies git repository structure and branch information
11
+ # - Checks for required plan.md files and templates
12
+ # - Validates file permissions and accessibility
13
+ #
14
+ # 2. Plan Data Extraction
15
+ # - Parses plan.md files to extract project metadata
16
+ # - Identifies language/version, frameworks, databases, and project types
17
+ # - Handles missing or incomplete specification data gracefully
18
+ #
19
+ # 3. Agent File Management
20
+ # - Creates new agent context files from templates when needed
21
+ # - Updates existing agent files with new project information
22
+ # - Preserves manual additions and custom configurations
23
+ # - Supports multiple AI agent formats and directory structures
24
+ #
25
+ # 4. Content Generation
26
+ # - Generates language-specific build/test commands
27
+ # - Creates appropriate project directory structures
28
+ # - Updates technology stacks and recent changes sections
29
+ # - Maintains consistent formatting and timestamps
30
+ #
31
+ # 5. Multi-Agent Support
32
+ # - Handles agent-specific file paths and naming conventions
33
+ # - Supports: Claude, Gemini, Copilot, Cursor, Qwen, opencode, Codex, Windsurf, Kilo Code, Auggie CLI, Roo Code, CodeBuddy CLI, Qoder CLI, Amp, SHAI, or Amazon Q Developer CLI
34
+ # - Can update single agents or all existing agent files
35
+ # - Creates default Claude file if no agent files exist
36
+ #
37
+ # Usage: ./update-agent-context.sh [agent_type]
38
+ # Agent types: claude|gemini|copilot|cursor-agent|qwen|opencode|codex|windsurf|kilocode|auggie|shai|q|bob|qoder
39
+ # Leave empty to update all existing agent files
40
+
41
+ set -e
42
+
43
+ # Enable strict error handling
44
+ set -u
45
+ set -o pipefail
46
+
47
+ #==============================================================================
48
+ # Configuration and Global Variables
49
+ #==============================================================================
50
+
51
+ # Get script directory and load common functions
52
+ SCRIPT_DIR="$(CDPATH="" cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
53
+ source "$SCRIPT_DIR/common.sh"
54
+
55
+ # Get all paths and variables from common functions
56
+ eval $(get_feature_paths)
57
+
58
+ NEW_PLAN="$IMPL_PLAN" # Alias for compatibility with existing code
59
+ AGENT_TYPE="${1:-}"
60
+
61
+ # Agent-specific file paths
62
+ CLAUDE_FILE="$REPO_ROOT/CLAUDE.md"
63
+ GEMINI_FILE="$REPO_ROOT/GEMINI.md"
64
+ COPILOT_FILE="$REPO_ROOT/.github/agents/copilot-instructions.md"
65
+ CURSOR_FILE="$REPO_ROOT/.cursor/rules/specify-rules.mdc"
66
+ QWEN_FILE="$REPO_ROOT/QWEN.md"
67
+ AGENTS_FILE="$REPO_ROOT/AGENTS.md"
68
+ WINDSURF_FILE="$REPO_ROOT/.windsurf/rules/specify-rules.md"
69
+ KILOCODE_FILE="$REPO_ROOT/.kilocode/rules/specify-rules.md"
70
+ AUGGIE_FILE="$REPO_ROOT/.augment/rules/specify-rules.md"
71
+ ROO_FILE="$REPO_ROOT/.roo/rules/specify-rules.md"
72
+ CODEBUDDY_FILE="$REPO_ROOT/CODEBUDDY.md"
73
+ QODER_FILE="$REPO_ROOT/QODER.md"
74
+ AMP_FILE="$REPO_ROOT/AGENTS.md"
75
+ SHAI_FILE="$REPO_ROOT/SHAI.md"
76
+ Q_FILE="$REPO_ROOT/AGENTS.md"
77
+ BOB_FILE="$REPO_ROOT/AGENTS.md"
78
+
79
+ # Template file
80
+ TEMPLATE_FILE="$REPO_ROOT/.specify/templates/agent-file-template.md"
81
+
82
+ # Global variables for parsed plan data
83
+ NEW_LANG=""
84
+ NEW_FRAMEWORK=""
85
+ NEW_DB=""
86
+ NEW_PROJECT_TYPE=""
87
+
88
+ #==============================================================================
89
+ # Utility Functions
90
+ #==============================================================================
91
+
92
+ log_info() {
93
+ echo "INFO: $1"
94
+ }
95
+
96
+ log_success() {
97
+ echo "✓ $1"
98
+ }
99
+
100
+ log_error() {
101
+ echo "ERROR: $1" >&2
102
+ }
103
+
104
+ log_warning() {
105
+ echo "WARNING: $1" >&2
106
+ }
107
+
108
+ # Cleanup function for temporary files
109
+ cleanup() {
110
+ local exit_code=$?
111
+ rm -f /tmp/agent_update_*_$$
112
+ rm -f /tmp/manual_additions_$$
113
+ exit $exit_code
114
+ }
115
+
116
+ # Set up cleanup trap
117
+ trap cleanup EXIT INT TERM
118
+
119
+ #==============================================================================
120
+ # Validation Functions
121
+ #==============================================================================
122
+
123
+ validate_environment() {
124
+ # Check if we have a current branch/feature (git or non-git)
125
+ if [[ -z "$CURRENT_BRANCH" ]]; then
126
+ log_error "Unable to determine current feature"
127
+ if [[ "$HAS_GIT" == "true" ]]; then
128
+ log_info "Make sure you're on a feature branch"
129
+ else
130
+ log_info "Set SPECIFY_FEATURE environment variable or create a feature first"
131
+ fi
132
+ exit 1
133
+ fi
134
+
135
+ # Check if plan.md exists
136
+ if [[ ! -f "$NEW_PLAN" ]]; then
137
+ log_error "No plan.md found at $NEW_PLAN"
138
+ log_info "Make sure you're working on a feature with a corresponding spec directory"
139
+ if [[ "$HAS_GIT" != "true" ]]; then
140
+ log_info "Use: export SPECIFY_FEATURE=your-feature-name or create a new feature first"
141
+ fi
142
+ exit 1
143
+ fi
144
+
145
+ # Check if template exists (needed for new files)
146
+ if [[ ! -f "$TEMPLATE_FILE" ]]; then
147
+ log_warning "Template file not found at $TEMPLATE_FILE"
148
+ log_warning "Creating new agent files will fail"
149
+ fi
150
+ }
151
+
152
+ #==============================================================================
153
+ # Plan Parsing Functions
154
+ #==============================================================================
155
+
156
+ extract_plan_field() {
157
+ local field_pattern="$1"
158
+ local plan_file="$2"
159
+
160
+ grep "^\*\*${field_pattern}\*\*: " "$plan_file" 2>/dev/null | \
161
+ head -1 | \
162
+ sed "s|^\*\*${field_pattern}\*\*: ||" | \
163
+ sed 's/^[ \t]*//;s/[ \t]*$//' | \
164
+ grep -v "NEEDS CLARIFICATION" | \
165
+ grep -v "^N/A$" || echo ""
166
+ }
167
+
168
+ parse_plan_data() {
169
+ local plan_file="$1"
170
+
171
+ if [[ ! -f "$plan_file" ]]; then
172
+ log_error "Plan file not found: $plan_file"
173
+ return 1
174
+ fi
175
+
176
+ if [[ ! -r "$plan_file" ]]; then
177
+ log_error "Plan file is not readable: $plan_file"
178
+ return 1
179
+ fi
180
+
181
+ log_info "Parsing plan data from $plan_file"
182
+
183
+ NEW_LANG=$(extract_plan_field "Language/Version" "$plan_file")
184
+ NEW_FRAMEWORK=$(extract_plan_field "Primary Dependencies" "$plan_file")
185
+ NEW_DB=$(extract_plan_field "Storage" "$plan_file")
186
+ NEW_PROJECT_TYPE=$(extract_plan_field "Project Type" "$plan_file")
187
+
188
+ # Log what we found
189
+ if [[ -n "$NEW_LANG" ]]; then
190
+ log_info "Found language: $NEW_LANG"
191
+ else
192
+ log_warning "No language information found in plan"
193
+ fi
194
+
195
+ if [[ -n "$NEW_FRAMEWORK" ]]; then
196
+ log_info "Found framework: $NEW_FRAMEWORK"
197
+ fi
198
+
199
+ if [[ -n "$NEW_DB" ]] && [[ "$NEW_DB" != "N/A" ]]; then
200
+ log_info "Found database: $NEW_DB"
201
+ fi
202
+
203
+ if [[ -n "$NEW_PROJECT_TYPE" ]]; then
204
+ log_info "Found project type: $NEW_PROJECT_TYPE"
205
+ fi
206
+ }
207
+
208
+ format_technology_stack() {
209
+ local lang="$1"
210
+ local framework="$2"
211
+ local parts=()
212
+
213
+ # Add non-empty parts
214
+ [[ -n "$lang" && "$lang" != "NEEDS CLARIFICATION" ]] && parts+=("$lang")
215
+ [[ -n "$framework" && "$framework" != "NEEDS CLARIFICATION" && "$framework" != "N/A" ]] && parts+=("$framework")
216
+
217
+ # Join with proper formatting
218
+ if [[ ${#parts[@]} -eq 0 ]]; then
219
+ echo ""
220
+ elif [[ ${#parts[@]} -eq 1 ]]; then
221
+ echo "${parts[0]}"
222
+ else
223
+ # Join multiple parts with " + "
224
+ local result="${parts[0]}"
225
+ for ((i=1; i<${#parts[@]}; i++)); do
226
+ result="$result + ${parts[i]}"
227
+ done
228
+ echo "$result"
229
+ fi
230
+ }
231
+
232
+ #==============================================================================
233
+ # Template and Content Generation Functions
234
+ #==============================================================================
235
+
236
+ get_project_structure() {
237
+ local project_type="$1"
238
+
239
+ if [[ "$project_type" == *"web"* ]]; then
240
+ echo "backend/\\nfrontend/\\ntests/"
241
+ else
242
+ echo "src/\\ntests/"
243
+ fi
244
+ }
245
+
246
+ get_commands_for_language() {
247
+ local lang="$1"
248
+
249
+ case "$lang" in
250
+ *"Python"*)
251
+ echo "cd src && pytest && ruff check ."
252
+ ;;
253
+ *"Rust"*)
254
+ echo "cargo test && cargo clippy"
255
+ ;;
256
+ *"JavaScript"*|*"TypeScript"*)
257
+ echo "npm test \\&\\& npm run lint"
258
+ ;;
259
+ *)
260
+ echo "# Add commands for $lang"
261
+ ;;
262
+ esac
263
+ }
264
+
265
+ get_language_conventions() {
266
+ local lang="$1"
267
+ echo "$lang: Follow standard conventions"
268
+ }
269
+
270
+ create_new_agent_file() {
271
+ local target_file="$1"
272
+ local temp_file="$2"
273
+ local project_name="$3"
274
+ local current_date="$4"
275
+
276
+ if [[ ! -f "$TEMPLATE_FILE" ]]; then
277
+ log_error "Template not found at $TEMPLATE_FILE"
278
+ return 1
279
+ fi
280
+
281
+ if [[ ! -r "$TEMPLATE_FILE" ]]; then
282
+ log_error "Template file is not readable: $TEMPLATE_FILE"
283
+ return 1
284
+ fi
285
+
286
+ log_info "Creating new agent context file from template..."
287
+
288
+ if ! cp "$TEMPLATE_FILE" "$temp_file"; then
289
+ log_error "Failed to copy template file"
290
+ return 1
291
+ fi
292
+
293
+ # Replace template placeholders
294
+ local project_structure
295
+ project_structure=$(get_project_structure "$NEW_PROJECT_TYPE")
296
+
297
+ local commands
298
+ commands=$(get_commands_for_language "$NEW_LANG")
299
+
300
+ local language_conventions
301
+ language_conventions=$(get_language_conventions "$NEW_LANG")
302
+
303
+ # Perform substitutions with error checking using safer approach
304
+ # Escape special characters for sed by using a different delimiter or escaping
305
+ local escaped_lang=$(printf '%s\n' "$NEW_LANG" | sed 's/[\[\.*^$()+{}|]/\\&/g')
306
+ local escaped_framework=$(printf '%s\n' "$NEW_FRAMEWORK" | sed 's/[\[\.*^$()+{}|]/\\&/g')
307
+ local escaped_branch=$(printf '%s\n' "$CURRENT_BRANCH" | sed 's/[\[\.*^$()+{}|]/\\&/g')
308
+
309
+ # Build technology stack and recent change strings conditionally
310
+ local tech_stack
311
+ if [[ -n "$escaped_lang" && -n "$escaped_framework" ]]; then
312
+ tech_stack="- $escaped_lang + $escaped_framework ($escaped_branch)"
313
+ elif [[ -n "$escaped_lang" ]]; then
314
+ tech_stack="- $escaped_lang ($escaped_branch)"
315
+ elif [[ -n "$escaped_framework" ]]; then
316
+ tech_stack="- $escaped_framework ($escaped_branch)"
317
+ else
318
+ tech_stack="- ($escaped_branch)"
319
+ fi
320
+
321
+ local recent_change
322
+ if [[ -n "$escaped_lang" && -n "$escaped_framework" ]]; then
323
+ recent_change="- $escaped_branch: Added $escaped_lang + $escaped_framework"
324
+ elif [[ -n "$escaped_lang" ]]; then
325
+ recent_change="- $escaped_branch: Added $escaped_lang"
326
+ elif [[ -n "$escaped_framework" ]]; then
327
+ recent_change="- $escaped_branch: Added $escaped_framework"
328
+ else
329
+ recent_change="- $escaped_branch: Added"
330
+ fi
331
+
332
+ local substitutions=(
333
+ "s|\[PROJECT NAME\]|$project_name|"
334
+ "s|\[DATE\]|$current_date|"
335
+ "s|\[EXTRACTED FROM ALL PLAN.MD FILES\]|$tech_stack|"
336
+ "s|\[ACTUAL STRUCTURE FROM PLANS\]|$project_structure|g"
337
+ "s|\[ONLY COMMANDS FOR ACTIVE TECHNOLOGIES\]|$commands|"
338
+ "s|\[LANGUAGE-SPECIFIC, ONLY FOR LANGUAGES IN USE\]|$language_conventions|"
339
+ "s|\[LAST 3 FEATURES AND WHAT THEY ADDED\]|$recent_change|"
340
+ )
341
+
342
+ for substitution in "${substitutions[@]}"; do
343
+ if ! sed -i.bak -e "$substitution" "$temp_file"; then
344
+ log_error "Failed to perform substitution: $substitution"
345
+ rm -f "$temp_file" "$temp_file.bak"
346
+ return 1
347
+ fi
348
+ done
349
+
350
+ # Convert \n sequences to actual newlines
351
+ newline=$(printf '\n')
352
+ sed -i.bak2 "s/\\\\n/${newline}/g" "$temp_file"
353
+
354
+ # Clean up backup files
355
+ rm -f "$temp_file.bak" "$temp_file.bak2"
356
+
357
+ return 0
358
+ }
359
+
360
+
361
+
362
+
363
+ update_existing_agent_file() {
364
+ local target_file="$1"
365
+ local current_date="$2"
366
+
367
+ log_info "Updating existing agent context file..."
368
+
369
+ # Use a single temporary file for atomic update
370
+ local temp_file
371
+ temp_file=$(mktemp) || {
372
+ log_error "Failed to create temporary file"
373
+ return 1
374
+ }
375
+
376
+ # Process the file in one pass
377
+ local tech_stack=$(format_technology_stack "$NEW_LANG" "$NEW_FRAMEWORK")
378
+ local new_tech_entries=()
379
+ local new_change_entry=""
380
+
381
+ # Prepare new technology entries
382
+ if [[ -n "$tech_stack" ]] && ! grep -q "$tech_stack" "$target_file"; then
383
+ new_tech_entries+=("- $tech_stack ($CURRENT_BRANCH)")
384
+ fi
385
+
386
+ if [[ -n "$NEW_DB" ]] && [[ "$NEW_DB" != "N/A" ]] && [[ "$NEW_DB" != "NEEDS CLARIFICATION" ]] && ! grep -q "$NEW_DB" "$target_file"; then
387
+ new_tech_entries+=("- $NEW_DB ($CURRENT_BRANCH)")
388
+ fi
389
+
390
+ # Prepare new change entry
391
+ if [[ -n "$tech_stack" ]]; then
392
+ new_change_entry="- $CURRENT_BRANCH: Added $tech_stack"
393
+ elif [[ -n "$NEW_DB" ]] && [[ "$NEW_DB" != "N/A" ]] && [[ "$NEW_DB" != "NEEDS CLARIFICATION" ]]; then
394
+ new_change_entry="- $CURRENT_BRANCH: Added $NEW_DB"
395
+ fi
396
+
397
+ # Check if sections exist in the file
398
+ local has_active_technologies=0
399
+ local has_recent_changes=0
400
+
401
+ if grep -q "^## Active Technologies" "$target_file" 2>/dev/null; then
402
+ has_active_technologies=1
403
+ fi
404
+
405
+ if grep -q "^## Recent Changes" "$target_file" 2>/dev/null; then
406
+ has_recent_changes=1
407
+ fi
408
+
409
+ # Process file line by line
410
+ local in_tech_section=false
411
+ local in_changes_section=false
412
+ local tech_entries_added=false
413
+ local changes_entries_added=false
414
+ local existing_changes_count=0
415
+ local file_ended=false
416
+
417
+ while IFS= read -r line || [[ -n "$line" ]]; do
418
+ # Handle Active Technologies section
419
+ if [[ "$line" == "## Active Technologies" ]]; then
420
+ echo "$line" >> "$temp_file"
421
+ in_tech_section=true
422
+ continue
423
+ elif [[ $in_tech_section == true ]] && [[ "$line" =~ ^##[[:space:]] ]]; then
424
+ # Add new tech entries before closing the section
425
+ if [[ $tech_entries_added == false ]] && [[ ${#new_tech_entries[@]} -gt 0 ]]; then
426
+ printf '%s\n' "${new_tech_entries[@]}" >> "$temp_file"
427
+ tech_entries_added=true
428
+ fi
429
+ echo "$line" >> "$temp_file"
430
+ in_tech_section=false
431
+ continue
432
+ elif [[ $in_tech_section == true ]] && [[ -z "$line" ]]; then
433
+ # Add new tech entries before empty line in tech section
434
+ if [[ $tech_entries_added == false ]] && [[ ${#new_tech_entries[@]} -gt 0 ]]; then
435
+ printf '%s\n' "${new_tech_entries[@]}" >> "$temp_file"
436
+ tech_entries_added=true
437
+ fi
438
+ echo "$line" >> "$temp_file"
439
+ continue
440
+ fi
441
+
442
+ # Handle Recent Changes section
443
+ if [[ "$line" == "## Recent Changes" ]]; then
444
+ echo "$line" >> "$temp_file"
445
+ # Add new change entry right after the heading
446
+ if [[ -n "$new_change_entry" ]]; then
447
+ echo "$new_change_entry" >> "$temp_file"
448
+ fi
449
+ in_changes_section=true
450
+ changes_entries_added=true
451
+ continue
452
+ elif [[ $in_changes_section == true ]] && [[ "$line" =~ ^##[[:space:]] ]]; then
453
+ echo "$line" >> "$temp_file"
454
+ in_changes_section=false
455
+ continue
456
+ elif [[ $in_changes_section == true ]] && [[ "$line" == "- "* ]]; then
457
+ # Keep only first 2 existing changes
458
+ if [[ $existing_changes_count -lt 2 ]]; then
459
+ echo "$line" >> "$temp_file"
460
+ ((existing_changes_count++))
461
+ fi
462
+ continue
463
+ fi
464
+
465
+ # Update timestamp
466
+ if [[ "$line" =~ \*\*Last\ updated\*\*:.*[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9] ]]; then
467
+ echo "$line" | sed "s/[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]/$current_date/" >> "$temp_file"
468
+ else
469
+ echo "$line" >> "$temp_file"
470
+ fi
471
+ done < "$target_file"
472
+
473
+ # Post-loop check: if we're still in the Active Technologies section and haven't added new entries
474
+ if [[ $in_tech_section == true ]] && [[ $tech_entries_added == false ]] && [[ ${#new_tech_entries[@]} -gt 0 ]]; then
475
+ printf '%s\n' "${new_tech_entries[@]}" >> "$temp_file"
476
+ tech_entries_added=true
477
+ fi
478
+
479
+ # If sections don't exist, add them at the end of the file
480
+ if [[ $has_active_technologies -eq 0 ]] && [[ ${#new_tech_entries[@]} -gt 0 ]]; then
481
+ echo "" >> "$temp_file"
482
+ echo "## Active Technologies" >> "$temp_file"
483
+ printf '%s\n' "${new_tech_entries[@]}" >> "$temp_file"
484
+ tech_entries_added=true
485
+ fi
486
+
487
+ if [[ $has_recent_changes -eq 0 ]] && [[ -n "$new_change_entry" ]]; then
488
+ echo "" >> "$temp_file"
489
+ echo "## Recent Changes" >> "$temp_file"
490
+ echo "$new_change_entry" >> "$temp_file"
491
+ changes_entries_added=true
492
+ fi
493
+
494
+ # Move temp file to target atomically
495
+ if ! mv "$temp_file" "$target_file"; then
496
+ log_error "Failed to update target file"
497
+ rm -f "$temp_file"
498
+ return 1
499
+ fi
500
+
501
+ return 0
502
+ }
503
+ #==============================================================================
504
+ # Main Agent File Update Function
505
+ #==============================================================================
506
+
507
+ update_agent_file() {
508
+ local target_file="$1"
509
+ local agent_name="$2"
510
+
511
+ if [[ -z "$target_file" ]] || [[ -z "$agent_name" ]]; then
512
+ log_error "update_agent_file requires target_file and agent_name parameters"
513
+ return 1
514
+ fi
515
+
516
+ log_info "Updating $agent_name context file: $target_file"
517
+
518
+ local project_name
519
+ project_name=$(basename "$REPO_ROOT")
520
+ local current_date
521
+ current_date=$(date +%Y-%m-%d)
522
+
523
+ # Create directory if it doesn't exist
524
+ local target_dir
525
+ target_dir=$(dirname "$target_file")
526
+ if [[ ! -d "$target_dir" ]]; then
527
+ if ! mkdir -p "$target_dir"; then
528
+ log_error "Failed to create directory: $target_dir"
529
+ return 1
530
+ fi
531
+ fi
532
+
533
+ if [[ ! -f "$target_file" ]]; then
534
+ # Create new file from template
535
+ local temp_file
536
+ temp_file=$(mktemp) || {
537
+ log_error "Failed to create temporary file"
538
+ return 1
539
+ }
540
+
541
+ if create_new_agent_file "$target_file" "$temp_file" "$project_name" "$current_date"; then
542
+ if mv "$temp_file" "$target_file"; then
543
+ log_success "Created new $agent_name context file"
544
+ else
545
+ log_error "Failed to move temporary file to $target_file"
546
+ rm -f "$temp_file"
547
+ return 1
548
+ fi
549
+ else
550
+ log_error "Failed to create new agent file"
551
+ rm -f "$temp_file"
552
+ return 1
553
+ fi
554
+ else
555
+ # Update existing file
556
+ if [[ ! -r "$target_file" ]]; then
557
+ log_error "Cannot read existing file: $target_file"
558
+ return 1
559
+ fi
560
+
561
+ if [[ ! -w "$target_file" ]]; then
562
+ log_error "Cannot write to existing file: $target_file"
563
+ return 1
564
+ fi
565
+
566
+ if update_existing_agent_file "$target_file" "$current_date"; then
567
+ log_success "Updated existing $agent_name context file"
568
+ else
569
+ log_error "Failed to update existing agent file"
570
+ return 1
571
+ fi
572
+ fi
573
+
574
+ return 0
575
+ }
576
+
577
+ #==============================================================================
578
+ # Agent Selection and Processing
579
+ #==============================================================================
580
+
581
+ update_specific_agent() {
582
+ local agent_type="$1"
583
+
584
+ case "$agent_type" in
585
+ claude)
586
+ update_agent_file "$CLAUDE_FILE" "Claude Code"
587
+ ;;
588
+ gemini)
589
+ update_agent_file "$GEMINI_FILE" "Gemini CLI"
590
+ ;;
591
+ copilot)
592
+ update_agent_file "$COPILOT_FILE" "GitHub Copilot"
593
+ ;;
594
+ cursor-agent)
595
+ update_agent_file "$CURSOR_FILE" "Cursor IDE"
596
+ ;;
597
+ qwen)
598
+ update_agent_file "$QWEN_FILE" "Qwen Code"
599
+ ;;
600
+ opencode)
601
+ update_agent_file "$AGENTS_FILE" "opencode"
602
+ ;;
603
+ codex)
604
+ update_agent_file "$AGENTS_FILE" "Codex CLI"
605
+ ;;
606
+ windsurf)
607
+ update_agent_file "$WINDSURF_FILE" "Windsurf"
608
+ ;;
609
+ kilocode)
610
+ update_agent_file "$KILOCODE_FILE" "Kilo Code"
611
+ ;;
612
+ auggie)
613
+ update_agent_file "$AUGGIE_FILE" "Auggie CLI"
614
+ ;;
615
+ roo)
616
+ update_agent_file "$ROO_FILE" "Roo Code"
617
+ ;;
618
+ codebuddy)
619
+ update_agent_file "$CODEBUDDY_FILE" "CodeBuddy CLI"
620
+ ;;
621
+ qoder)
622
+ update_agent_file "$QODER_FILE" "Qoder CLI"
623
+ ;;
624
+ amp)
625
+ update_agent_file "$AMP_FILE" "Amp"
626
+ ;;
627
+ shai)
628
+ update_agent_file "$SHAI_FILE" "SHAI"
629
+ ;;
630
+ q)
631
+ update_agent_file "$Q_FILE" "Amazon Q Developer CLI"
632
+ ;;
633
+ bob)
634
+ update_agent_file "$BOB_FILE" "IBM Bob"
635
+ ;;
636
+ *)
637
+ log_error "Unknown agent type '$agent_type'"
638
+ log_error "Expected: claude|gemini|copilot|cursor-agent|qwen|opencode|codex|windsurf|kilocode|auggie|roo|amp|shai|q|bob|qoder"
639
+ exit 1
640
+ ;;
641
+ esac
642
+ }
643
+
644
+ update_all_existing_agents() {
645
+ local found_agent=false
646
+
647
+ # Check each possible agent file and update if it exists
648
+ if [[ -f "$CLAUDE_FILE" ]]; then
649
+ update_agent_file "$CLAUDE_FILE" "Claude Code"
650
+ found_agent=true
651
+ fi
652
+
653
+ if [[ -f "$GEMINI_FILE" ]]; then
654
+ update_agent_file "$GEMINI_FILE" "Gemini CLI"
655
+ found_agent=true
656
+ fi
657
+
658
+ if [[ -f "$COPILOT_FILE" ]]; then
659
+ update_agent_file "$COPILOT_FILE" "GitHub Copilot"
660
+ found_agent=true
661
+ fi
662
+
663
+ if [[ -f "$CURSOR_FILE" ]]; then
664
+ update_agent_file "$CURSOR_FILE" "Cursor IDE"
665
+ found_agent=true
666
+ fi
667
+
668
+ if [[ -f "$QWEN_FILE" ]]; then
669
+ update_agent_file "$QWEN_FILE" "Qwen Code"
670
+ found_agent=true
671
+ fi
672
+
673
+ if [[ -f "$AGENTS_FILE" ]]; then
674
+ update_agent_file "$AGENTS_FILE" "Codex/opencode"
675
+ found_agent=true
676
+ fi
677
+
678
+ if [[ -f "$WINDSURF_FILE" ]]; then
679
+ update_agent_file "$WINDSURF_FILE" "Windsurf"
680
+ found_agent=true
681
+ fi
682
+
683
+ if [[ -f "$KILOCODE_FILE" ]]; then
684
+ update_agent_file "$KILOCODE_FILE" "Kilo Code"
685
+ found_agent=true
686
+ fi
687
+
688
+ if [[ -f "$AUGGIE_FILE" ]]; then
689
+ update_agent_file "$AUGGIE_FILE" "Auggie CLI"
690
+ found_agent=true
691
+ fi
692
+
693
+ if [[ -f "$ROO_FILE" ]]; then
694
+ update_agent_file "$ROO_FILE" "Roo Code"
695
+ found_agent=true
696
+ fi
697
+
698
+ if [[ -f "$CODEBUDDY_FILE" ]]; then
699
+ update_agent_file "$CODEBUDDY_FILE" "CodeBuddy CLI"
700
+ found_agent=true
701
+ fi
702
+
703
+ if [[ -f "$SHAI_FILE" ]]; then
704
+ update_agent_file "$SHAI_FILE" "SHAI"
705
+ found_agent=true
706
+ fi
707
+
708
+ if [[ -f "$QODER_FILE" ]]; then
709
+ update_agent_file "$QODER_FILE" "Qoder CLI"
710
+ found_agent=true
711
+ fi
712
+
713
+ if [[ -f "$Q_FILE" ]]; then
714
+ update_agent_file "$Q_FILE" "Amazon Q Developer CLI"
715
+ found_agent=true
716
+ fi
717
+
718
+ if [[ -f "$BOB_FILE" ]]; then
719
+ update_agent_file "$BOB_FILE" "IBM Bob"
720
+ found_agent=true
721
+ fi
722
+
723
+ # If no agent files exist, create a default Claude file
724
+ if [[ "$found_agent" == false ]]; then
725
+ log_info "No existing agent files found, creating default Claude file..."
726
+ update_agent_file "$CLAUDE_FILE" "Claude Code"
727
+ fi
728
+ }
729
+ print_summary() {
730
+ echo
731
+ log_info "Summary of changes:"
732
+
733
+ if [[ -n "$NEW_LANG" ]]; then
734
+ echo " - Added language: $NEW_LANG"
735
+ fi
736
+
737
+ if [[ -n "$NEW_FRAMEWORK" ]]; then
738
+ echo " - Added framework: $NEW_FRAMEWORK"
739
+ fi
740
+
741
+ if [[ -n "$NEW_DB" ]] && [[ "$NEW_DB" != "N/A" ]]; then
742
+ echo " - Added database: $NEW_DB"
743
+ fi
744
+
745
+ echo
746
+
747
+ log_info "Usage: $0 [claude|gemini|copilot|cursor-agent|qwen|opencode|codex|windsurf|kilocode|auggie|codebuddy|shai|q|bob|qoder]"
748
+ }
749
+
750
+ #==============================================================================
751
+ # Main Execution
752
+ #==============================================================================
753
+
754
+ main() {
755
+ # Validate environment before proceeding
756
+ validate_environment
757
+
758
+ log_info "=== Updating agent context files for feature $CURRENT_BRANCH ==="
759
+
760
+ # Parse the plan file to extract project information
761
+ if ! parse_plan_data "$NEW_PLAN"; then
762
+ log_error "Failed to parse plan data"
763
+ exit 1
764
+ fi
765
+
766
+ # Process based on agent type argument
767
+ local success=true
768
+
769
+ if [[ -z "$AGENT_TYPE" ]]; then
770
+ # No specific agent provided - update all existing agent files
771
+ log_info "No agent specified, updating all existing agent files..."
772
+ if ! update_all_existing_agents; then
773
+ success=false
774
+ fi
775
+ else
776
+ # Specific agent provided - update only that agent
777
+ log_info "Updating specific agent: $AGENT_TYPE"
778
+ if ! update_specific_agent "$AGENT_TYPE"; then
779
+ success=false
780
+ fi
781
+ fi
782
+
783
+ # Print summary
784
+ print_summary
785
+
786
+ if [[ "$success" == true ]]; then
787
+ log_success "Agent context update completed successfully"
788
+ exit 0
789
+ else
790
+ log_error "Agent context update completed with errors"
791
+ exit 1
792
+ fi
793
+ }
794
+
795
+ # Execute main function if script is run directly
796
+ if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
797
+ main "$@"
798
+ fi
799
+
.specify/templates/agent-file-template.md ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # [PROJECT NAME] Development Guidelines
2
+
3
+ Auto-generated from all feature plans. Last updated: [DATE]
4
+
5
+ ## Active Technologies
6
+
7
+ [EXTRACTED FROM ALL PLAN.MD FILES]
8
+
9
+ ## Project Structure
10
+
11
+ ```text
12
+ [ACTUAL STRUCTURE FROM PLANS]
13
+ ```
14
+
15
+ ## Commands
16
+
17
+ [ONLY COMMANDS FOR ACTIVE TECHNOLOGIES]
18
+
19
+ ## Code Style
20
+
21
+ [LANGUAGE-SPECIFIC, ONLY FOR LANGUAGES IN USE]
22
+
23
+ ## Recent Changes
24
+
25
+ [LAST 3 FEATURES AND WHAT THEY ADDED]
26
+
27
+ <!-- MANUAL ADDITIONS START -->
28
+ <!-- MANUAL ADDITIONS END -->
.specify/templates/checklist-template.md ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # [CHECKLIST TYPE] Checklist: [FEATURE NAME]
2
+
3
+ **Purpose**: [Brief description of what this checklist covers]
4
+ **Created**: [DATE]
5
+ **Feature**: [Link to spec.md or relevant documentation]
6
+
7
+ **Note**: This checklist is generated by the `/speckit.checklist` command based on feature context and requirements.
8
+
9
+ <!--
10
+ ============================================================================
11
+ IMPORTANT: The checklist items below are SAMPLE ITEMS for illustration only.
12
+
13
+ The /speckit.checklist command MUST replace these with actual items based on:
14
+ - User's specific checklist request
15
+ - Feature requirements from spec.md
16
+ - Technical context from plan.md
17
+ - Implementation details from tasks.md
18
+
19
+ DO NOT keep these sample items in the generated checklist file.
20
+ ============================================================================
21
+ -->
22
+
23
+ ## [Category 1]
24
+
25
+ - [ ] CHK001 First checklist item with clear action
26
+ - [ ] CHK002 Second checklist item
27
+ - [ ] CHK003 Third checklist item
28
+
29
+ ## [Category 2]
30
+
31
+ - [ ] CHK004 Another category item
32
+ - [ ] CHK005 Item with specific criteria
33
+ - [ ] CHK006 Final item in this category
34
+
35
+ ## Notes
36
+
37
+ - Check items off as completed: `[x]`
38
+ - Add comments or findings inline
39
+ - Link to relevant resources or documentation
40
+ - Items are numbered sequentially for easy reference
.specify/templates/plan-template.md ADDED
@@ -0,0 +1,104 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Implementation Plan: [FEATURE]
2
+
3
+ **Branch**: `[###-feature-name]` | **Date**: [DATE] | **Spec**: [link]
4
+ **Input**: Feature specification from `/specs/[###-feature-name]/spec.md`
5
+
6
+ **Note**: This template is filled in by the `/speckit.plan` command. See `.specify/templates/commands/plan.md` for the execution workflow.
7
+
8
+ ## Summary
9
+
10
+ [Extract from feature spec: primary requirement + technical approach from research]
11
+
12
+ ## Technical Context
13
+
14
+ <!--
15
+ ACTION REQUIRED: Replace the content in this section with the technical details
16
+ for the project. The structure here is presented in advisory capacity to guide
17
+ the iteration process.
18
+ -->
19
+
20
+ **Language/Version**: [e.g., Python 3.11, Swift 5.9, Rust 1.75 or NEEDS CLARIFICATION]
21
+ **Primary Dependencies**: [e.g., FastAPI, UIKit, LLVM or NEEDS CLARIFICATION]
22
+ **Storage**: [if applicable, e.g., PostgreSQL, CoreData, files or N/A]
23
+ **Testing**: [e.g., pytest, XCTest, cargo test or NEEDS CLARIFICATION]
24
+ **Target Platform**: [e.g., Linux server, iOS 15+, WASM or NEEDS CLARIFICATION]
25
+ **Project Type**: [single/web/mobile - determines source structure]
26
+ **Performance Goals**: [domain-specific, e.g., 1000 req/s, 10k lines/sec, 60 fps or NEEDS CLARIFICATION]
27
+ **Constraints**: [domain-specific, e.g., <200ms p95, <100MB memory, offline-capable or NEEDS CLARIFICATION]
28
+ **Scale/Scope**: [domain-specific, e.g., 10k users, 1M LOC, 50 screens or NEEDS CLARIFICATION]
29
+
30
+ ## Constitution Check
31
+
32
+ *GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
33
+
34
+ [Gates determined based on constitution file]
35
+
36
+ ## Project Structure
37
+
38
+ ### Documentation (this feature)
39
+
40
+ ```text
41
+ specs/[###-feature]/
42
+ ├── plan.md # This file (/speckit.plan command output)
43
+ ├── research.md # Phase 0 output (/speckit.plan command)
44
+ ├── data-model.md # Phase 1 output (/speckit.plan command)
45
+ ├── quickstart.md # Phase 1 output (/speckit.plan command)
46
+ ├── contracts/ # Phase 1 output (/speckit.plan command)
47
+ └── tasks.md # Phase 2 output (/speckit.tasks command - NOT created by /speckit.plan)
48
+ ```
49
+
50
+ ### Source Code (repository root)
51
+ <!--
52
+ ACTION REQUIRED: Replace the placeholder tree below with the concrete layout
53
+ for this feature. Delete unused options and expand the chosen structure with
54
+ real paths (e.g., apps/admin, packages/something). The delivered plan must
55
+ not include Option labels.
56
+ -->
57
+
58
+ ```text
59
+ # [REMOVE IF UNUSED] Option 1: Single project (DEFAULT)
60
+ src/
61
+ ├── models/
62
+ ├── services/
63
+ ├── cli/
64
+ └── lib/
65
+
66
+ tests/
67
+ ├── contract/
68
+ ├── integration/
69
+ └── unit/
70
+
71
+ # [REMOVE IF UNUSED] Option 2: Web application (when "frontend" + "backend" detected)
72
+ backend/
73
+ ├── src/
74
+ │ ├── models/
75
+ │ ├── services/
76
+ │ └── api/
77
+ └── tests/
78
+
79
+ frontend/
80
+ ├── src/
81
+ │ ├── components/
82
+ │ ├── pages/
83
+ │ └── services/
84
+ └── tests/
85
+
86
+ # [REMOVE IF UNUSED] Option 3: Mobile + API (when "iOS/Android" detected)
87
+ api/
88
+ └── [same as backend above]
89
+
90
+ ios/ or android/
91
+ └── [platform-specific structure: feature modules, UI flows, platform tests]
92
+ ```
93
+
94
+ **Structure Decision**: [Document the selected structure and reference the real
95
+ directories captured above]
96
+
97
+ ## Complexity Tracking
98
+
99
+ > **Fill ONLY if Constitution Check has violations that must be justified**
100
+
101
+ | Violation | Why Needed | Simpler Alternative Rejected Because |
102
+ |-----------|------------|-------------------------------------|
103
+ | [e.g., 4th project] | [current need] | [why 3 projects insufficient] |
104
+ | [e.g., Repository pattern] | [specific problem] | [why direct DB access insufficient] |
.specify/templates/spec-template.md ADDED
@@ -0,0 +1,115 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Feature Specification: [FEATURE NAME]
2
+
3
+ **Feature Branch**: `[###-feature-name]`
4
+ **Created**: [DATE]
5
+ **Status**: Draft
6
+ **Input**: User description: "$ARGUMENTS"
7
+
8
+ ## User Scenarios & Testing *(mandatory)*
9
+
10
+ <!--
11
+ IMPORTANT: User stories should be PRIORITIZED as user journeys ordered by importance.
12
+ Each user story/journey must be INDEPENDENTLY TESTABLE - meaning if you implement just ONE of them,
13
+ you should still have a viable MVP (Minimum Viable Product) that delivers value.
14
+
15
+ Assign priorities (P1, P2, P3, etc.) to each story, where P1 is the most critical.
16
+ Think of each story as a standalone slice of functionality that can be:
17
+ - Developed independently
18
+ - Tested independently
19
+ - Deployed independently
20
+ - Demonstrated to users independently
21
+ -->
22
+
23
+ ### User Story 1 - [Brief Title] (Priority: P1)
24
+
25
+ [Describe this user journey in plain language]
26
+
27
+ **Why this priority**: [Explain the value and why it has this priority level]
28
+
29
+ **Independent Test**: [Describe how this can be tested independently - e.g., "Can be fully tested by [specific action] and delivers [specific value]"]
30
+
31
+ **Acceptance Scenarios**:
32
+
33
+ 1. **Given** [initial state], **When** [action], **Then** [expected outcome]
34
+ 2. **Given** [initial state], **When** [action], **Then** [expected outcome]
35
+
36
+ ---
37
+
38
+ ### User Story 2 - [Brief Title] (Priority: P2)
39
+
40
+ [Describe this user journey in plain language]
41
+
42
+ **Why this priority**: [Explain the value and why it has this priority level]
43
+
44
+ **Independent Test**: [Describe how this can be tested independently]
45
+
46
+ **Acceptance Scenarios**:
47
+
48
+ 1. **Given** [initial state], **When** [action], **Then** [expected outcome]
49
+
50
+ ---
51
+
52
+ ### User Story 3 - [Brief Title] (Priority: P3)
53
+
54
+ [Describe this user journey in plain language]
55
+
56
+ **Why this priority**: [Explain the value and why it has this priority level]
57
+
58
+ **Independent Test**: [Describe how this can be tested independently]
59
+
60
+ **Acceptance Scenarios**:
61
+
62
+ 1. **Given** [initial state], **When** [action], **Then** [expected outcome]
63
+
64
+ ---
65
+
66
+ [Add more user stories as needed, each with an assigned priority]
67
+
68
+ ### Edge Cases
69
+
70
+ <!--
71
+ ACTION REQUIRED: The content in this section represents placeholders.
72
+ Fill them out with the right edge cases.
73
+ -->
74
+
75
+ - What happens when [boundary condition]?
76
+ - How does system handle [error scenario]?
77
+
78
+ ## Requirements *(mandatory)*
79
+
80
+ <!--
81
+ ACTION REQUIRED: The content in this section represents placeholders.
82
+ Fill them out with the right functional requirements.
83
+ -->
84
+
85
+ ### Functional Requirements
86
+
87
+ - **FR-001**: System MUST [specific capability, e.g., "allow users to create accounts"]
88
+ - **FR-002**: System MUST [specific capability, e.g., "validate email addresses"]
89
+ - **FR-003**: Users MUST be able to [key interaction, e.g., "reset their password"]
90
+ - **FR-004**: System MUST [data requirement, e.g., "persist user preferences"]
91
+ - **FR-005**: System MUST [behavior, e.g., "log all security events"]
92
+
93
+ *Example of marking unclear requirements:*
94
+
95
+ - **FR-006**: System MUST authenticate users via [NEEDS CLARIFICATION: auth method not specified - email/password, SSO, OAuth?]
96
+ - **FR-007**: System MUST retain user data for [NEEDS CLARIFICATION: retention period not specified]
97
+
98
+ ### Key Entities *(include if feature involves data)*
99
+
100
+ - **[Entity 1]**: [What it represents, key attributes without implementation]
101
+ - **[Entity 2]**: [What it represents, relationships to other entities]
102
+
103
+ ## Success Criteria *(mandatory)*
104
+
105
+ <!--
106
+ ACTION REQUIRED: Define measurable success criteria.
107
+ These must be technology-agnostic and measurable.
108
+ -->
109
+
110
+ ### Measurable Outcomes
111
+
112
+ - **SC-001**: [Measurable metric, e.g., "Users can complete account creation in under 2 minutes"]
113
+ - **SC-002**: [Measurable metric, e.g., "System handles 1000 concurrent users without degradation"]
114
+ - **SC-003**: [User satisfaction metric, e.g., "90% of users successfully complete primary task on first attempt"]
115
+ - **SC-004**: [Business metric, e.g., "Reduce support tickets related to [X] by 50%"]
.specify/templates/tasks-template.md ADDED
@@ -0,0 +1,251 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+
3
+ description: "Task list template for feature implementation"
4
+ ---
5
+
6
+ # Tasks: [FEATURE NAME]
7
+
8
+ **Input**: Design documents from `/specs/[###-feature-name]/`
9
+ **Prerequisites**: plan.md (required), spec.md (required for user stories), research.md, data-model.md, contracts/
10
+
11
+ **Tests**: The examples below include test tasks. Tests are OPTIONAL - only include them if explicitly requested in the feature specification.
12
+
13
+ **Organization**: Tasks are grouped by user story to enable independent implementation and testing of each story.
14
+
15
+ ## Format: `[ID] [P?] [Story] Description`
16
+
17
+ - **[P]**: Can run in parallel (different files, no dependencies)
18
+ - **[Story]**: Which user story this task belongs to (e.g., US1, US2, US3)
19
+ - Include exact file paths in descriptions
20
+
21
+ ## Path Conventions
22
+
23
+ - **Single project**: `src/`, `tests/` at repository root
24
+ - **Web app**: `backend/src/`, `frontend/src/`
25
+ - **Mobile**: `api/src/`, `ios/src/` or `android/src/`
26
+ - Paths shown below assume single project - adjust based on plan.md structure
27
+
28
+ <!--
29
+ ============================================================================
30
+ IMPORTANT: The tasks below are SAMPLE TASKS for illustration purposes only.
31
+
32
+ The /speckit.tasks command MUST replace these with actual tasks based on:
33
+ - User stories from spec.md (with their priorities P1, P2, P3...)
34
+ - Feature requirements from plan.md
35
+ - Entities from data-model.md
36
+ - Endpoints from contracts/
37
+
38
+ Tasks MUST be organized by user story so each story can be:
39
+ - Implemented independently
40
+ - Tested independently
41
+ - Delivered as an MVP increment
42
+
43
+ DO NOT keep these sample tasks in the generated tasks.md file.
44
+ ============================================================================
45
+ -->
46
+
47
+ ## Phase 1: Setup (Shared Infrastructure)
48
+
49
+ **Purpose**: Project initialization and basic structure
50
+
51
+ - [ ] T001 Create project structure per implementation plan
52
+ - [ ] T002 Initialize [language] project with [framework] dependencies
53
+ - [ ] T003 [P] Configure linting and formatting tools
54
+
55
+ ---
56
+
57
+ ## Phase 2: Foundational (Blocking Prerequisites)
58
+
59
+ **Purpose**: Core infrastructure that MUST be complete before ANY user story can be implemented
60
+
61
+ **⚠️ CRITICAL**: No user story work can begin until this phase is complete
62
+
63
+ Examples of foundational tasks (adjust based on your project):
64
+
65
+ - [ ] T004 Setup database schema and migrations framework
66
+ - [ ] T005 [P] Implement authentication/authorization framework
67
+ - [ ] T006 [P] Setup API routing and middleware structure
68
+ - [ ] T007 Create base models/entities that all stories depend on
69
+ - [ ] T008 Configure error handling and logging infrastructure
70
+ - [ ] T009 Setup environment configuration management
71
+
72
+ **Checkpoint**: Foundation ready - user story implementation can now begin in parallel
73
+
74
+ ---
75
+
76
+ ## Phase 3: User Story 1 - [Title] (Priority: P1) 🎯 MVP
77
+
78
+ **Goal**: [Brief description of what this story delivers]
79
+
80
+ **Independent Test**: [How to verify this story works on its own]
81
+
82
+ ### Tests for User Story 1 (OPTIONAL - only if tests requested) ⚠️
83
+
84
+ > **NOTE: Write these tests FIRST, ensure they FAIL before implementation**
85
+
86
+ - [ ] T010 [P] [US1] Contract test for [endpoint] in tests/contract/test_[name].py
87
+ - [ ] T011 [P] [US1] Integration test for [user journey] in tests/integration/test_[name].py
88
+
89
+ ### Implementation for User Story 1
90
+
91
+ - [ ] T012 [P] [US1] Create [Entity1] model in src/models/[entity1].py
92
+ - [ ] T013 [P] [US1] Create [Entity2] model in src/models/[entity2].py
93
+ - [ ] T014 [US1] Implement [Service] in src/services/[service].py (depends on T012, T013)
94
+ - [ ] T015 [US1] Implement [endpoint/feature] in src/[location]/[file].py
95
+ - [ ] T016 [US1] Add validation and error handling
96
+ - [ ] T017 [US1] Add logging for user story 1 operations
97
+
98
+ **Checkpoint**: At this point, User Story 1 should be fully functional and testable independently
99
+
100
+ ---
101
+
102
+ ## Phase 4: User Story 2 - [Title] (Priority: P2)
103
+
104
+ **Goal**: [Brief description of what this story delivers]
105
+
106
+ **Independent Test**: [How to verify this story works on its own]
107
+
108
+ ### Tests for User Story 2 (OPTIONAL - only if tests requested) ⚠️
109
+
110
+ - [ ] T018 [P] [US2] Contract test for [endpoint] in tests/contract/test_[name].py
111
+ - [ ] T019 [P] [US2] Integration test for [user journey] in tests/integration/test_[name].py
112
+
113
+ ### Implementation for User Story 2
114
+
115
+ - [ ] T020 [P] [US2] Create [Entity] model in src/models/[entity].py
116
+ - [ ] T021 [US2] Implement [Service] in src/services/[service].py
117
+ - [ ] T022 [US2] Implement [endpoint/feature] in src/[location]/[file].py
118
+ - [ ] T023 [US2] Integrate with User Story 1 components (if needed)
119
+
120
+ **Checkpoint**: At this point, User Stories 1 AND 2 should both work independently
121
+
122
+ ---
123
+
124
+ ## Phase 5: User Story 3 - [Title] (Priority: P3)
125
+
126
+ **Goal**: [Brief description of what this story delivers]
127
+
128
+ **Independent Test**: [How to verify this story works on its own]
129
+
130
+ ### Tests for User Story 3 (OPTIONAL - only if tests requested) ⚠️
131
+
132
+ - [ ] T024 [P] [US3] Contract test for [endpoint] in tests/contract/test_[name].py
133
+ - [ ] T025 [P] [US3] Integration test for [user journey] in tests/integration/test_[name].py
134
+
135
+ ### Implementation for User Story 3
136
+
137
+ - [ ] T026 [P] [US3] Create [Entity] model in src/models/[entity].py
138
+ - [ ] T027 [US3] Implement [Service] in src/services/[service].py
139
+ - [ ] T028 [US3] Implement [endpoint/feature] in src/[location]/[file].py
140
+
141
+ **Checkpoint**: All user stories should now be independently functional
142
+
143
+ ---
144
+
145
+ [Add more user story phases as needed, following the same pattern]
146
+
147
+ ---
148
+
149
+ ## Phase N: Polish & Cross-Cutting Concerns
150
+
151
+ **Purpose**: Improvements that affect multiple user stories
152
+
153
+ - [ ] TXXX [P] Documentation updates in docs/
154
+ - [ ] TXXX Code cleanup and refactoring
155
+ - [ ] TXXX Performance optimization across all stories
156
+ - [ ] TXXX [P] Additional unit tests (if requested) in tests/unit/
157
+ - [ ] TXXX Security hardening
158
+ - [ ] TXXX Run quickstart.md validation
159
+
160
+ ---
161
+
162
+ ## Dependencies & Execution Order
163
+
164
+ ### Phase Dependencies
165
+
166
+ - **Setup (Phase 1)**: No dependencies - can start immediately
167
+ - **Foundational (Phase 2)**: Depends on Setup completion - BLOCKS all user stories
168
+ - **User Stories (Phase 3+)**: All depend on Foundational phase completion
169
+ - User stories can then proceed in parallel (if staffed)
170
+ - Or sequentially in priority order (P1 → P2 → P3)
171
+ - **Polish (Final Phase)**: Depends on all desired user stories being complete
172
+
173
+ ### User Story Dependencies
174
+
175
+ - **User Story 1 (P1)**: Can start after Foundational (Phase 2) - No dependencies on other stories
176
+ - **User Story 2 (P2)**: Can start after Foundational (Phase 2) - May integrate with US1 but should be independently testable
177
+ - **User Story 3 (P3)**: Can start after Foundational (Phase 2) - May integrate with US1/US2 but should be independently testable
178
+
179
+ ### Within Each User Story
180
+
181
+ - Tests (if included) MUST be written and FAIL before implementation
182
+ - Models before services
183
+ - Services before endpoints
184
+ - Core implementation before integration
185
+ - Story complete before moving to next priority
186
+
187
+ ### Parallel Opportunities
188
+
189
+ - All Setup tasks marked [P] can run in parallel
190
+ - All Foundational tasks marked [P] can run in parallel (within Phase 2)
191
+ - Once Foundational phase completes, all user stories can start in parallel (if team capacity allows)
192
+ - All tests for a user story marked [P] can run in parallel
193
+ - Models within a story marked [P] can run in parallel
194
+ - Different user stories can be worked on in parallel by different team members
195
+
196
+ ---
197
+
198
+ ## Parallel Example: User Story 1
199
+
200
+ ```bash
201
+ # Launch all tests for User Story 1 together (if tests requested):
202
+ Task: "Contract test for [endpoint] in tests/contract/test_[name].py"
203
+ Task: "Integration test for [user journey] in tests/integration/test_[name].py"
204
+
205
+ # Launch all models for User Story 1 together:
206
+ Task: "Create [Entity1] model in src/models/[entity1].py"
207
+ Task: "Create [Entity2] model in src/models/[entity2].py"
208
+ ```
209
+
210
+ ---
211
+
212
+ ## Implementation Strategy
213
+
214
+ ### MVP First (User Story 1 Only)
215
+
216
+ 1. Complete Phase 1: Setup
217
+ 2. Complete Phase 2: Foundational (CRITICAL - blocks all stories)
218
+ 3. Complete Phase 3: User Story 1
219
+ 4. **STOP and VALIDATE**: Test User Story 1 independently
220
+ 5. Deploy/demo if ready
221
+
222
+ ### Incremental Delivery
223
+
224
+ 1. Complete Setup + Foundational → Foundation ready
225
+ 2. Add User Story 1 → Test independently → Deploy/Demo (MVP!)
226
+ 3. Add User Story 2 → Test independently → Deploy/Demo
227
+ 4. Add User Story 3 → Test independently → Deploy/Demo
228
+ 5. Each story adds value without breaking previous stories
229
+
230
+ ### Parallel Team Strategy
231
+
232
+ With multiple developers:
233
+
234
+ 1. Team completes Setup + Foundational together
235
+ 2. Once Foundational is done:
236
+ - Developer A: User Story 1
237
+ - Developer B: User Story 2
238
+ - Developer C: User Story 3
239
+ 3. Stories complete and integrate independently
240
+
241
+ ---
242
+
243
+ ## Notes
244
+
245
+ - [P] tasks = different files, no dependencies
246
+ - [Story] label maps task to specific user story for traceability
247
+ - Each user story should be independently completable and testable
248
+ - Verify tests fail before implementing
249
+ - Commit after each task or logical group
250
+ - Stop at any checkpoint to validate story independently
251
+ - Avoid: vague tasks, same file conflicts, cross-story dependencies that break independence
Dockerfile CHANGED
@@ -3,16 +3,19 @@ ARG QUARTO_VERSION="1.4.550"
3
  # Use the Quarto base image
4
  FROM ghcr.io/quarto-dev/quarto:${QUARTO_VERSION} AS builder
5
 
6
- COPY src /app
 
7
  WORKDIR /app
8
 
9
- # Install Python requirements
10
  USER root
11
- RUN apt-get update && apt-get install -y python3 python3-pip
12
- COPY requirements.txt /app/
13
- RUN pip3 install -r requirements.txt
14
 
15
- RUN quarto render .
 
 
16
 
 
17
  EXPOSE 7860
18
- CMD ["python3", "-m", "http.server", "7860", "--directory", "_site"]
 
3
  # Use the Quarto base image
4
  FROM ghcr.io/quarto-dev/quarto:${QUARTO_VERSION} AS builder
5
 
6
+ # Copy project files
7
+ COPY . /app
8
  WORKDIR /app
9
 
10
+ # Install pixi
11
  USER root
12
+ RUN curl -fsSL https://pixi.sh/install.sh | bash
13
+ ENV PATH="/root/.pixi/bin:$PATH"
 
14
 
15
+ # Install dependencies and render the site
16
+ RUN pixi install
17
+ RUN pixi run render
18
 
19
+ # Expose port and serve the site using pixi
20
  EXPOSE 7860
21
+ CMD ["pixi", "run", "serve"]
docs/content-plan.md ADDED
@@ -0,0 +1,148 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 12-Month Content Plan for Kashi Coding Handbook
2
+
3
+ ## "Modern Python: From Packages to Production AI"
4
+
5
+ ---
6
+
7
+ ## Q1 2026: Modern Python Foundation (Jan-Mar)
8
+
9
+ **Theme:** Setting up the modern Python development stack
10
+
11
+ ### January: Package Management Evolution
12
+
13
+ **Post 1: "Beyond pip: Package Management with pixi.sh"**
14
+
15
+ - Why conda-forge matters for data science/AI
16
+ - Installing pixi and basic commands
17
+ - Creating reproducible environments
18
+ - Comparison with Poetry, PDM, uv
19
+ - When to use each tool
20
+ - GitHub repo: `pixi-starter-templates`
21
+
22
+ **Post 2: "Multi-Project Workspace with pixi"**
23
+
24
+ - Managing multiple related projects
25
+ - Shared dependencies across projects
26
+ - Lock files and reproducibility
27
+ - Integration with Docker
28
+ - GitHub repo: `pixi-monorepo-example`
29
+
30
+ ### February: Professional CLI Development
31
+
32
+ **Post 3: "Building CLI Tools with Typer"**
33
+
34
+ - Type hints-driven interfaces
35
+ - Commands, options, and arguments
36
+ - Auto-generated documentation
37
+ - Testing CLI applications with pytest
38
+ - GitHub repo: `typer-starter-kit`
39
+
40
+ **Post 4: "Advanced CLI Patterns: Progress Bars, Config Files, and Plugins"**
41
+
42
+ - Rich integration for beautiful output
43
+ - Configuration management (TOML, YAML)
44
+ - Plugin architecture
45
+ - Distributing CLI tools
46
+ - GitHub repo: `advanced-cli-patterns`
47
+
48
+ ### March: Docker & Containerization
49
+
50
+ **Post 5: "Docker for Python Development"**
51
+
52
+ - Multi-stage builds for Python apps
53
+ - Managing dependencies in containers
54
+ - Docker Compose for development stacks
55
+ - Volume mounting and hot reload
56
+ - GitHub repo: `python-docker-templates`
57
+
58
+ **Post 6: "Containerizing AI Applications"**
59
+
60
+ - GPU support in Docker
61
+ - Model serving containers
62
+ - Environment variables and secrets
63
+ - Your TrueNAS/Coolify setup as case study
64
+ - GitHub repo: `ai-docker-stack`
65
+
66
+ ---
67
+
68
+ ## Q2 2026: LLM Applications & Foundations (Apr-Jun)
69
+
70
+ **Theme:** Building blocks for AI applications
71
+
72
+ ### April: LangChain Fundamentals
73
+
74
+ **Post 7: "Building Your First RAG Application with LangChain"**
75
+
76
+ - Document loaders and text splitters
77
+ - Vector stores (using Supabase pgvector!)
78
+ - Retrieval chains
79
+ - Basic prompt engineering
80
+ - GitHub repo: `langchain-rag-starter`
81
+
82
+ **Post 8: "LangChain Memory Systems"**
83
+
84
+ - Conversation memory patterns
85
+ - Message history with Supabase
86
+ - Context window management
87
+ - When to use which memory type
88
+ - GitHub repo: `langchain-memory-examples`
89
+
90
+ ### May: MCP (Model Context Protocol)
91
+
92
+ **Post 9: "Building Your First MCP Server"**
93
+
94
+ - MCP architecture overview
95
+ - Database connector server (your actual work!)
96
+ - Tool creation and registration
97
+ - Testing MCP servers
98
+ - GitHub repo: `mcp-starter-pack`
99
+
100
+ **Post 10: "Advanced MCP: Custom Tools and Integrations"**
101
+
102
+ - File system access tools
103
+ - API integration tools
104
+ - Docker container management tools
105
+ - Connecting MCP to LangChain agents
106
+ - GitHub repo: `mcp-advanced-tools`
107
+
108
+ ### June: Agent Observability
109
+
110
+ **Post 11: "Observability for LLM Applications"**
111
+
112
+ - Logging strategies for LLM calls
113
+ - Token counting and cost tracking
114
+ - LangSmith integration
115
+ - Debugging agent decisions
116
+ - GitHub repo: `llm-observability-toolkit`
117
+
118
+ **Post 12: "Testing Non-Deterministic Systems"**
119
+
120
+ - Testing strategies for LLM apps
121
+ - Assertion patterns for AI outputs
122
+ - Evaluation frameworks
123
+ - CI/CD for AI applications
124
+ - GitHub repo: `ai-testing-patterns`
125
+
126
+ ---
127
+
128
+ ## Q3 2026: Multi-Agent Systems & Robotics (Jul-Sep)
129
+
130
+ **Theme:** Orchestrating multiple agents and physical systems
131
+
132
+ ### July: CrewAI Deep Dive
133
+
134
+ **Post 13: "Building Multi-Agent Systems with CrewAI"**
135
+
136
+ - Agent roles and goals
137
+ - Task delegation patterns
138
+ - Sequential vs hierarchical crews
139
+ - Real-world example: research automation
140
+ - GitHub repo: `crewai-examples`
141
+
142
+ **Post 14: "CrewAI Advanced Patterns"**
143
+
144
+ - Custom tools for agents
145
+ - Inter-agent communication
146
+ - Error handling and recovery
147
+ - Observability for CrewAI (connects to Post 11)
148
+ - GitHub repo: `crewai-advanced`
docs/docker-ai.md ADDED
@@ -0,0 +1,487 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Docker AI: Deploying Local LLMs and MCP Servers
2
+
3
+ This guide covers the latest ways to use Docker for deploying local Large Language Model (LLM) models and Model Context Protocol (MCP) servers.
4
+
5
+ ## Table of Contents
6
+
7
+ - [Docker Compose Models](#docker-compose-models)
8
+ - [Docker Model Runner](#docker-model-runner)
9
+ - [Docker MCP Toolkit](#docker-mcp-toolkit)
10
+ - [Common Use Cases](#common-use-cases)
11
+
12
+ ---
13
+
14
+ ## Docker Compose Models
15
+
16
+ Docker Compose v2.38+ introduces a standardized way to define AI model dependencies using the `models` top-level element in your Compose files.
17
+
18
+ ### Prerequisites
19
+
20
+ - Docker Compose v2.38 or later
21
+ - Docker Model Runner (DMR) or compatible cloud providers
22
+ - For DMR: See [requirements](https://docs.docker.com/ai/model-runner/#requirements)
23
+
24
+ ### Basic Model Definition
25
+
26
+ Define models in your `docker-compose.yml`:
27
+
28
+ ```yaml
29
+ services:
30
+ chat-app:
31
+ image: my-chat-app
32
+ models:
33
+ - llm
34
+
35
+ models:
36
+ llm:
37
+ model: ai/smollm2
38
+ ```
39
+
40
+ This configuration:
41
+ - Defines a service `chat-app` that uses a model named `llm`
42
+ - References the `ai/smollm2` model image from Docker Hub
43
+
44
+ ### Model Configuration Options
45
+
46
+ Configure models with various runtime parameters:
47
+
48
+ ```yaml
49
+ models:
50
+ llm:
51
+ model: ai/smollm2
52
+ context_size: 1024
53
+ runtime_flags:
54
+ - "--a-flag"
55
+ - "--another-flag=42"
56
+ ```
57
+
58
+ **Key configuration options:**
59
+
60
+ - **`model`** (required): OCI artifact identifier for the model
61
+ - **`context_size`**: Maximum token context size (keep as small as feasible for your needs)
62
+ - **`runtime_flags`**: Command-line flags passed to the inference engine (e.g., [llama.cpp parameters](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md))
63
+ - **`x-*`**: Platform-specific extension attributes
64
+
65
+ ### Service Model Binding
66
+
67
+ #### Short Syntax
68
+
69
+ The simplest way to bind models to services:
70
+
71
+ ```yaml
72
+ services:
73
+ app:
74
+ image: my-app
75
+ models:
76
+ - llm
77
+ - embedding-model
78
+
79
+ models:
80
+ llm:
81
+ model: ai/smollm2
82
+ embedding-model:
83
+ model: ai/all-minilm
84
+ ```
85
+
86
+ Auto-generated environment variables:
87
+ - `LLM_URL` - URL to access the LLM model
88
+ - `LLM_MODEL` - Model identifier
89
+ - `EMBEDDING_MODEL_URL` - URL to access the embedding model
90
+ - `EMBEDDING_MODEL_MODEL` - Model identifier
91
+
92
+ #### Long Syntax
93
+
94
+ Customize environment variable names:
95
+
96
+ ```yaml
97
+ services:
98
+ app:
99
+ image: my-app
100
+ models:
101
+ llm:
102
+ endpoint_var: AI_MODEL_URL
103
+ model_var: AI_MODEL_NAME
104
+ embedding-model:
105
+ endpoint_var: EMBEDDING_URL
106
+ model_var: EMBEDDING_NAME
107
+
108
+ models:
109
+ llm:
110
+ model: ai/smollm2
111
+ embedding-model:
112
+ model: ai/all-minilm
113
+ ```
114
+
115
+ ### Platform Portability
116
+
117
+ #### Docker Model Runner
118
+
119
+ When Docker Model Runner is enabled locally:
120
+
121
+ ```yaml
122
+ services:
123
+ chat-app:
124
+ image: my-chat-app
125
+ models:
126
+ llm:
127
+ endpoint_var: AI_MODEL_URL
128
+ model_var: AI_MODEL_NAME
129
+
130
+ models:
131
+ llm:
132
+ model: ai/smollm2
133
+ context_size: 4096
134
+ runtime_flags:
135
+ - "--no-prefill-assistant"
136
+ ```
137
+
138
+ Docker Model Runner will:
139
+ - Pull and run the model locally
140
+ - Provide endpoint URLs
141
+ - Inject environment variables into the service
142
+
143
+ #### Cloud Providers
144
+
145
+ The same Compose file works on cloud providers:
146
+
147
+ ```yaml
148
+ services:
149
+ chat-app:
150
+ image: my-chat-app
151
+ models:
152
+ - llm
153
+
154
+ models:
155
+ llm:
156
+ model: ai/smollm2
157
+ # Cloud-specific configurations
158
+ x-cloud-options:
159
+ - "cloud.instance-type=gpu-small"
160
+ - "cloud.region=us-west-2"
161
+ ```
162
+
163
+ Cloud providers may:
164
+ - Use managed AI services
165
+ - Apply cloud-specific optimizations
166
+ - Provide monitoring and logging
167
+ - Handle model versioning automatically
168
+
169
+ ### Common Runtime Configurations
170
+
171
+ #### Development Mode
172
+
173
+ ```yaml
174
+ models:
175
+ dev_model:
176
+ model: ai/model
177
+ context_size: 4096
178
+ runtime_flags:
179
+ - "--verbose"
180
+ - "--verbose-prompt"
181
+ - "--log-prefix"
182
+ - "--log-timestamps"
183
+ - "--log-colors"
184
+ ```
185
+
186
+ #### Conservative (Disabled Reasoning)
187
+
188
+ ```yaml
189
+ models:
190
+ conservative_model:
191
+ model: ai/model
192
+ context_size: 4096
193
+ runtime_flags:
194
+ - "--temp"
195
+ - "0.1"
196
+ - "--top-k"
197
+ - "1"
198
+ - "--reasoning-budget"
199
+ - "0"
200
+ ```
201
+
202
+ #### Creative (High Randomness)
203
+
204
+ ```yaml
205
+ models:
206
+ creative_model:
207
+ model: ai/model
208
+ context_size: 4096
209
+ runtime_flags:
210
+ - "--temp"
211
+ - "1"
212
+ - "--top-p"
213
+ - "0.9"
214
+ ```
215
+
216
+ #### Highly Deterministic
217
+
218
+ ```yaml
219
+ models:
220
+ deterministic_model:
221
+ model: ai/model
222
+ context_size: 4096
223
+ runtime_flags:
224
+ - "--temp"
225
+ - "0"
226
+ - "--top-k"
227
+ - "1"
228
+ ```
229
+
230
+ #### Concurrent Processing
231
+
232
+ ```yaml
233
+ models:
234
+ concurrent_model:
235
+ model: ai/model
236
+ context_size: 2048
237
+ runtime_flags:
238
+ - "--threads"
239
+ - "8"
240
+ - "--mlock" # Lock memory to prevent swapping
241
+ ```
242
+
243
+ ---
244
+
245
+ ## Docker Model Runner
246
+
247
+ Docker Model Runner (DMR) enables running AI models locally with minimal setup. It integrates seamlessly with Docker Compose models.
248
+
249
+ ### Key Features
250
+
251
+ - **Local model execution**: Run models on your local machine
252
+ - **GPU support**: Leverage local GPU resources
253
+ - **Automatic model pulling**: Models are pulled from Docker Hub as needed
254
+ - **OpenAI-compatible API**: Expose models via standard API endpoints
255
+
256
+ ### Use Cases
257
+
258
+ - **Development and testing**: Test AI applications locally before cloud deployment
259
+ - **Privacy-sensitive workloads**: Keep data on-premises
260
+ - **Offline development**: Work without internet connectivity
261
+ - **Cost optimization**: Avoid cloud inference costs
262
+
263
+ ---
264
+
265
+ ## Docker MCP Toolkit
266
+
267
+ The Docker MCP (Model Context Protocol) Toolkit provides a secure, standardized way to connect AI agents to external tools and data sources.
268
+
269
+ ### What is MCP?
270
+
271
+ Model Context Protocol (MCP) is an open, client-server protocol that standardizes how applications provide context and functionality to Large Language Models. It allows AI agents to:
272
+
273
+ - Interact with external tools and APIs
274
+ - Access databases and services
275
+ - Execute code in isolated environments
276
+ - Retrieve real-world data
277
+
278
+ ### Docker MCP Components
279
+
280
+ #### 1. MCP Gateway
281
+
282
+ The [Docker MCP Gateway](https://github.com/docker/mcp-gateway/) is the core open-source component that:
283
+
284
+ - Manages MCP containers
285
+ - Provides a unified endpoint for AI applications
286
+ - Mediates between agents and MCP servers
287
+ - Enables dynamic MCP discovery and configuration
288
+
289
+ #### 2. MCP Catalog
290
+
291
+ The [Docker MCP Catalog](https://hub.docker.com/mcp) is a curated collection of trusted, containerized MCP servers including:
292
+
293
+ - **270+ curated servers** from publishers like Stripe, Elastic, Grafana
294
+ - **Publisher trust levels**: Distinguish between official, verified, and community servers
295
+ - **Commit pinning**: Each server tied to specific Git commits for verifiability
296
+ - **AI-audited updates**: Automated reviews of code changes
297
+
298
+ #### 3. MCP Toolkit (Docker Desktop)
299
+
300
+ A management interface in Docker Desktop for:
301
+
302
+ - Discovering MCP servers
303
+ - Configuring and managing servers
304
+ - One-click deployment
305
+ - Centralized authentication
306
+
307
+ ### Dynamic MCPs: Smart Search and Tool Composition
308
+
309
+ Recent enhancements enable agents to dynamically discover and configure MCP servers:
310
+
311
+ #### Smart Search Features
312
+
313
+ **`mcp-find`**: Find MCP servers by name or description
314
+ ```
315
+ Agent: "Find MCP servers for web searching"
316
+ → Returns: DuckDuckGo MCP, Brave Search MCP, etc.
317
+ ```
318
+
319
+ **`mcp-add`**: Add MCP servers to the current session
320
+ ```
321
+ Agent: "Add the DuckDuckGo MCP server"
322
+ → Server is pulled, configured, and made available
323
+ ```
324
+
325
+ #### Benefits of Dynamic MCPs
326
+
327
+ 1. **No manual configuration**: Agents discover and add tools as needed
328
+ 2. **Reduced token usage**: Only load tools when required
329
+ 3. **Autonomous operation**: Agents manage their own tool ecosystem
330
+ 4. **Secure sandbox**: Tool composition happens in isolated environments
331
+
332
+ ### Security Features
333
+
334
+ Docker MCP Toolkit implements multiple security layers:
335
+
336
+ 1. **Containerization**: Strong isolation limits blast radius
337
+ 2. **Commit pinning**: Precise attribution and verification
338
+ 3. **Automated auditing**: AI-powered code reviews
339
+ 4. **Publisher trust levels**: Clear indicators of server origin
340
+ 5. **Isolated execution**: MCP servers run in separate containers
341
+
342
+ ### Using MCP with Docker Compose
343
+
344
+ Example `docker-compose.yml` for MCP servers:
345
+
346
+ ```yaml
347
+ services:
348
+ mcp-gateway:
349
+ image: docker/mcp-gateway:latest
350
+ ports:
351
+ - "3000:3000"
352
+ volumes:
353
+ - mcp-data:/data
354
+ environment:
355
+ - MCP_CATALOG_URL=https://hub.docker.com/mcp
356
+
357
+ my-app:
358
+ image: my-ai-app
359
+ depends_on:
360
+ - mcp-gateway
361
+ environment:
362
+ - MCP_GATEWAY_URL=http://mcp-gateway:3000
363
+
364
+ volumes:
365
+ mcp-data:
366
+ ```
367
+
368
+ ### Submitting MCP Servers
369
+
370
+ To contribute MCP servers to the Docker MCP Catalog:
371
+
372
+ 1. Follow the [submission guidance](https://github.com/docker/mcp-registry/blob/main/CONTRIBUTING.md)
373
+ 2. Submit to the [MCP Registry](https://github.com/docker/mcp-registry)
374
+ 3. Servers undergo automated and manual review
375
+ 4. Approved servers appear in the catalog with appropriate trust levels
376
+
377
+ ---
378
+
379
+ ## Common Use Cases
380
+
381
+ ### 1. Local AI Development Environment
382
+
383
+ ```yaml
384
+ services:
385
+ dev-app:
386
+ build: .
387
+ models:
388
+ - llm
389
+ - embeddings
390
+ depends_on:
391
+ - mcp-gateway
392
+
393
+ mcp-gateway:
394
+ image: docker/mcp-gateway:latest
395
+ ports:
396
+ - "3000:3000"
397
+
398
+ models:
399
+ llm:
400
+ model: ai/smollm2
401
+ context_size: 4096
402
+ runtime_flags:
403
+ - "--verbose"
404
+ - "--log-colors"
405
+
406
+ embeddings:
407
+ model: ai/all-minilm
408
+ ```
409
+
410
+ ### 2. Multi-Model AI Application
411
+
412
+ ```yaml
413
+ services:
414
+ chat-service:
415
+ image: my-chat-service
416
+ models:
417
+ chat-model:
418
+ endpoint_var: CHAT_MODEL_URL
419
+ model_var: CHAT_MODEL_NAME
420
+
421
+ code-service:
422
+ image: my-code-service
423
+ models:
424
+ code-model:
425
+ endpoint_var: CODE_MODEL_URL
426
+ model_var: CODE_MODEL_NAME
427
+
428
+ models:
429
+ chat-model:
430
+ model: ai/smollm2
431
+ context_size: 2048
432
+ runtime_flags:
433
+ - "--temp"
434
+ - "0.7"
435
+
436
+ code-model:
437
+ model: ai/codellama
438
+ context_size: 4096
439
+ runtime_flags:
440
+ - "--temp"
441
+ - "0.2"
442
+ ```
443
+
444
+ ### 3. AI Agent with Dynamic MCP Tools
445
+
446
+ ```yaml
447
+ services:
448
+ ai-agent:
449
+ image: my-ai-agent
450
+ environment:
451
+ - MCP_GATEWAY_URL=http://mcp-gateway:3000
452
+ - ENABLE_DYNAMIC_MCPS=true
453
+ depends_on:
454
+ - mcp-gateway
455
+ - llm-service
456
+ models:
457
+ - agent-llm
458
+
459
+ mcp-gateway:
460
+ image: docker/mcp-gateway:latest
461
+ ports:
462
+ - "3000:3000"
463
+ volumes:
464
+ - ./mcp-catalog.yml:/config/catalog.yml
465
+
466
+ models:
467
+ agent-llm:
468
+ model: ai/smollm2
469
+ context_size: 4096
470
+ ```
471
+
472
+ ---
473
+
474
+ ## Additional Resources
475
+
476
+ - [Docker AI Documentation](https://docs.docker.com/ai/)
477
+ - [Docker Compose Models Reference](https://docs.docker.com/ai/compose/models-and-compose/)
478
+ - [Docker Model Runner](https://docs.docker.com/ai/model-runner/)
479
+ - [Docker MCP Gateway (GitHub)](https://github.com/docker/mcp-gateway/)
480
+ - [Docker MCP Catalog](https://hub.docker.com/mcp)
481
+ - [MCP Registry](https://github.com/docker/mcp-registry)
482
+ - [Dynamic MCPs Blog Post](https://www.docker.com/blog/dynamic-mcps-stop-hardcoding-your-agents-world/)
483
+ - [MCP Security Blog Post](https://www.docker.com/blog/enhancing-mcp-trust-with-the-docker-mcp-catalog/)
484
+
485
+ ---
486
+
487
+ **Last Updated**: December 2024
docs/learning-path.md ADDED
@@ -0,0 +1,1716 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Learning Path: Building AI-Powered CLI Tools with Python
2
+
3
+ A structured learning path for developers with basic Python knowledge who want to build AI-powered CLI tools using modern development practices.
4
+
5
+ ## 🎯 Prerequisites
6
+
7
+ **What You Should Know:**
8
+ - Basic Python syntax (variables, functions, loops, conditionals)
9
+ - How to run Python scripts from the command line
10
+ - Basic understanding of files and directories
11
+ - Familiarity with text editors or IDEs
12
+
13
+ **What You'll Learn:**
14
+ - Building professional CLI applications
15
+ - Integrating AI/LLM capabilities
16
+ - Modern Python package management with pixi
17
+ - AI-assisted development with GitHub Copilot
18
+ - Package publishing and distribution
19
+
20
+ ---
21
+
22
+ ## 📚 Learning Phases
23
+
24
+ ### Phase 1: Foundation Setup (Week 1)
25
+
26
+ #### 1.1 Development Environment Setup
27
+
28
+ **Install Required Tools:**
29
+
30
+ ```bash
31
+ # Install pixi (cross-platform package manager)
32
+ curl -fsSL https://pixi.sh/install.sh | bash
33
+
34
+ # Verify installation
35
+ pixi --version
36
+
37
+ # Install Git (if not already installed)
38
+ # Linux: sudo apt install git
39
+ # macOS: brew install git
40
+ # Windows: Download from git-scm.com
41
+ ```
42
+
43
+ **Set Up GitHub Copilot:**
44
+ 1. Install VS Code or your preferred IDE
45
+ 2. Install GitHub Copilot extension
46
+ 3. Sign in with your GitHub account (requires Copilot subscription)
47
+ 4. Complete the Copilot quickstart tutorial
48
+
49
+ **Resources:**
50
+ - [Pixi Documentation](https://pixi.sh/latest/)
51
+ - [GitHub Copilot Getting Started](https://docs.github.com/en/copilot/getting-started-with-github-copilot)
52
+ - [VS Code Python Setup](https://code.visualstudio.com/docs/python/python-tutorial)
53
+
54
+ #### 1.2 Understanding Modern Python Project Structure
55
+
56
+ **Learn About:**
57
+ - Project organization (src layout vs flat layout)
58
+ - Virtual environments and dependency isolation
59
+ - Configuration files (`pyproject.toml`, `pixi.toml`)
60
+ - Version control with Git
61
+
62
+ **Hands-On Exercise:**
63
+ Create a simple "Hello World" project with pixi:
64
+
65
+ ```bash
66
+ # Create new project
67
+ pixi init my-first-cli
68
+ cd my-first-cli
69
+
70
+ # Add Python dependency
71
+ pixi add python
72
+
73
+ # Create a simple script
74
+ mkdir src
75
+ echo 'print("Hello from pixi!")' > src/hello.py
76
+
77
+ # Run it
78
+ pixi run python src/hello.py
79
+ ```
80
+
81
+ **Use Copilot to:**
82
+ - Generate a `.gitignore` file for Python projects
83
+ - Create a basic `README.md` template
84
+ - Write docstrings for your functions
85
+
86
+ ---
87
+
88
+ ### Phase 2: CLI Development Fundamentals (Week 2-3)
89
+
90
+ #### 2.1 Building Your First CLI with Typer
91
+
92
+ **Learning Objectives:**
93
+ - Understand command-line argument parsing with type hints
94
+ - Create commands, options, and flags using Python types
95
+ - Handle user input and validation
96
+ - Display formatted output with Rich integration
97
+
98
+ **Project: Simple File Organizer CLI**
99
+
100
+ > **Note**: This is a simplified version for learning CLI basics. For a comprehensive, production-ready example that integrates Docker AI, MCP servers, and multi-agent systems, see the [FileOrganizer project](projects/FileOrganizer.md) in Phase 7.
101
+
102
+ ```bash
103
+ # Initialize project with pixi
104
+ pixi init file-organizer-cli
105
+ cd file-organizer-cli
106
+
107
+ # Add dependencies
108
+ pixi add python typer rich
109
+
110
+ # Create project structure
111
+ mkdir -p src/file_organizer
112
+ touch src/file_organizer/__init__.py
113
+ touch src/file_organizer/cli.py
114
+ ```
115
+
116
+ **Example CLI Structure (use Copilot to help generate):**
117
+
118
+ ```python
119
+ # src/file_organizer/cli.py
120
+ import typer
121
+ from pathlib import Path
122
+ from rich.console import Console
123
+ from typing import Optional
124
+
125
+ app = typer.Typer(help="File organizer CLI tool")
126
+ console = Console()
127
+
128
+ @app.command()
129
+ def organize(
130
+ directory: Path = typer.Argument(..., help="Directory to organize", exists=True),
131
+ dry_run: bool = typer.Option(False, "--dry-run", help="Preview changes without executing"),
132
+ verbose: bool = typer.Option(False, "--verbose", "-v", help="Show detailed output")
133
+ ):
134
+ """Organize files in DIRECTORY by extension."""
135
+ if verbose:
136
+ console.print(f"[blue]Organizing files in: {directory}[/blue]")
137
+
138
+ # Use Copilot to generate the organization logic
139
+ if dry_run:
140
+ console.print("[yellow]DRY RUN - No changes will be made[/yellow]")
141
+
142
+ pass
143
+
144
+ @app.command()
145
+ def stats(directory: Path = typer.Argument(..., exists=True)):
146
+ """Show statistics about files in DIRECTORY."""
147
+ # Use Copilot to generate statistics logic
148
+ pass
149
+
150
+ if __name__ == '__main__':
151
+ app()
152
+ ```
153
+
154
+ **Copilot Prompts to Try:**
155
+ - "Create a function to organize files by extension using pathlib"
156
+ - "Add error handling for file operations with try-except"
157
+ - "Generate help text and docstrings for CLI commands"
158
+ - "Add progress bar using rich library for file processing"
159
+
160
+ **Resources:**
161
+ - [Typer Documentation](https://typer.tiangolo.com/)
162
+ - [Typer Tutorial](https://typer.tiangolo.com/tutorial/)
163
+ - [Rich Documentation](https://rich.readthedocs.io/)
164
+
165
+ #### 2.2 Configuration and Settings Management
166
+
167
+ **Learn About:**
168
+ - Reading configuration files (YAML, TOML, JSON)
169
+ - Environment variables
170
+ - User preferences and defaults
171
+ - Configuration validation with Pydantic
172
+
173
+ **Add to Your Project:**
174
+
175
+ ```bash
176
+ # Add configuration dependencies
177
+ pixi add pydantic pyyaml python-dotenv
178
+ ```
179
+
180
+ **Use Copilot to Generate:**
181
+ - Configuration schema with Pydantic
182
+ - Config file loader functions
183
+ - Environment variable handling
184
+
185
+ ---
186
+
187
+ ### Phase 3: AI Integration Basics (Week 4-5)
188
+
189
+ #### 3.1 Understanding HuggingFace and LLM APIs
190
+
191
+ **Learning Objectives:**
192
+ - API authentication and token management
193
+ - Using HuggingFace Inference API and local models
194
+ - Making API requests with transformers and huggingface_hub
195
+ - Handling streaming responses
196
+ - Error handling and rate limiting
197
+
198
+ **Project: Add AI Capabilities to Your CLI**
199
+
200
+ ```bash
201
+ # Add AI dependencies
202
+ pixi add transformers huggingface-hub python-dotenv
203
+
204
+ # For local inference (optional)
205
+ pixi add torch
206
+
207
+ # Create .env file for API keys
208
+ echo "HUGGINGFACE_TOKEN=your-token-here" > .env
209
+ echo ".env" >> .gitignore
210
+ ```
211
+
212
+ **Simple AI Integration Example:**
213
+
214
+ ```python
215
+ # src/file_organizer/ai_helper.py
216
+ from huggingface_hub import InferenceClient
217
+ import os
218
+ from dotenv import load_dotenv
219
+
220
+ load_dotenv()
221
+
222
+ def suggest_organization_strategy(file_list: list[str]) -> str:
223
+ """Use AI to suggest file organization strategy."""
224
+ client = InferenceClient(token=os.getenv("HUGGINGFACE_TOKEN"))
225
+
226
+ prompt = f"""Given these files: {', '.join(file_list)}
227
+
228
+ Suggest an intelligent organization strategy. Group related files and explain your reasoning.
229
+ Respond in JSON format."""
230
+
231
+ # Use a free model like Mistral or Llama
232
+ response = client.text_generation(
233
+ prompt,
234
+ model="mistralai/Mistral-7B-Instruct-v0.2",
235
+ max_new_tokens=500,
236
+ temperature=0.7
237
+ )
238
+
239
+ return response
240
+
241
+ # Alternative: Using local models with transformers
242
+ from transformers import pipeline
243
+
244
+ def analyze_file_content_local(content: str) -> str:
245
+ """Analyze file content using a local model."""
246
+ # Use Copilot to complete this function
247
+ # Prompt: "Create a function that uses a local HuggingFace model
248
+ # to analyze and categorize file content"
249
+
250
+ classifier = pipeline(
251
+ "text-classification",
252
+ model="distilbert-base-uncased-finetuned-sst-2-english"
253
+ )
254
+
255
+ result = classifier(content[:512]) # Truncate for model limits
256
+ return result
257
+ ```
258
+
259
+ **Copilot Exercises:**
260
+ - "Create a function to summarize file contents using HuggingFace models"
261
+ - "Add retry logic for API failures with exponential backoff"
262
+ - "Implement streaming response handler for long-form generation"
263
+ - "Create a model selector that chooses between local and API inference"
264
+
265
+ **Resources:**
266
+ - [HuggingFace Hub Documentation](https://huggingface.co/docs/huggingface_hub/)
267
+ - [Transformers Documentation](https://huggingface.co/docs/transformers/)
268
+ - [HuggingFace Inference API](https://huggingface.co/docs/api-inference/)
269
+ - [Free Models on HuggingFace](https://huggingface.co/models)
270
+
271
+ **Popular Models to Try:**
272
+ - **Text Generation**: `mistralai/Mistral-7B-Instruct-v0.2`, `meta-llama/Llama-2-7b-chat-hf`
273
+ - **Summarization**: `facebook/bart-large-cnn`, `google/pegasus-xsum`
274
+ - **Classification**: `distilbert-base-uncased`, `roberta-base`
275
+ - **Embeddings**: `sentence-transformers/all-MiniLM-L6-v2`
276
+
277
+ **Local vs API Inference:**
278
+
279
+ ```python
280
+ # src/file_organizer/inference.py
281
+ from typing import Literal
282
+ import os
283
+
284
+ class AIHelper:
285
+ """Flexible AI helper supporting both local and API inference."""
286
+
287
+ def __init__(self, mode: Literal["local", "api"] = "api"):
288
+ self.mode = mode
289
+
290
+ if mode == "api":
291
+ from huggingface_hub import InferenceClient
292
+ self.client = InferenceClient(token=os.getenv("HUGGINGFACE_TOKEN"))
293
+ else:
294
+ from transformers import pipeline
295
+ # Load model once at initialization
296
+ self.pipeline = pipeline(
297
+ "text-generation",
298
+ model="distilgpt2", # Smaller model for local use
299
+ device=-1 # CPU, use 0 for GPU
300
+ )
301
+
302
+ def generate(self, prompt: str) -> str:
303
+ """Generate text using configured mode."""
304
+ if self.mode == "api":
305
+ return self.client.text_generation(
306
+ prompt,
307
+ model="mistralai/Mistral-7B-Instruct-v0.2",
308
+ max_new_tokens=500
309
+ )
310
+ else:
311
+ result = self.pipeline(prompt, max_new_tokens=100)
312
+ return result[0]['generated_text']
313
+
314
+ # Usage in CLI
315
+ # Use Copilot: "Add a --local flag to switch between API and local inference"
316
+ ```
317
+
318
+ **When to Use Each:**
319
+ - **API Inference**: Better quality, larger models, no local resources needed, requires internet
320
+ - **Local Inference**: Privacy, offline use, no API costs, but requires more RAM/GPU
321
+ - **vLLM Server**: Best of both worlds - local privacy with high performance and OpenAI-compatible API
322
+
323
+ **Advanced: Serving Local Models with vLLM**
324
+
325
+ vLLM is a high-performance inference engine that can serve local models with significantly better throughput and lower latency than standard transformers.
326
+
327
+ ```bash
328
+ # Install vLLM (requires GPU for best performance)
329
+ pixi add vllm
330
+
331
+ # Or install with specific CUDA version
332
+ pixi add "vllm[cuda12]"
333
+ ```
334
+
335
+ **Starting a vLLM Server:**
336
+
337
+ ```bash
338
+ # Start vLLM server with a model
339
+ # This creates an OpenAI-compatible API endpoint
340
+ vllm serve mistralai/Mistral-7B-Instruct-v0.2 \
341
+ --host 0.0.0.0 \
342
+ --port 8000 \
343
+ --max-model-len 4096
344
+
345
+ # For smaller GPUs, use quantized models
346
+ vllm serve TheBloke/Mistral-7B-Instruct-v0.2-GPTQ \
347
+ --quantization gptq \
348
+ --dtype half
349
+ ```
350
+
351
+ **Using vLLM Server in Your CLI:**
352
+
353
+ ```python
354
+ # src/file_organizer/vllm_client.py
355
+ from openai import OpenAI
356
+ from typing import Optional
357
+
358
+ class vLLMClient:
359
+ """Client for vLLM server with OpenAI-compatible API."""
360
+
361
+ def __init__(self, base_url: str = "http://localhost:8000/v1"):
362
+ # vLLM provides OpenAI-compatible endpoints
363
+ self.client = OpenAI(
364
+ base_url=base_url,
365
+ api_key="not-needed" # vLLM doesn't require API key
366
+ )
367
+
368
+ def generate(
369
+ self,
370
+ prompt: str,
371
+ model: str = "mistralai/Mistral-7B-Instruct-v0.2",
372
+ max_tokens: int = 500,
373
+ temperature: float = 0.7
374
+ ) -> str:
375
+ """Generate text using vLLM server."""
376
+ response = self.client.completions.create(
377
+ model=model,
378
+ prompt=prompt,
379
+ max_tokens=max_tokens,
380
+ temperature=temperature
381
+ )
382
+ return response.choices[0].text
383
+
384
+ def chat_generate(
385
+ self,
386
+ messages: list[dict],
387
+ model: str = "mistralai/Mistral-7B-Instruct-v0.2",
388
+ max_tokens: int = 500
389
+ ) -> str:
390
+ """Generate using chat completion format."""
391
+ response = self.client.chat.completions.create(
392
+ model=model,
393
+ messages=messages,
394
+ max_tokens=max_tokens
395
+ )
396
+ return response.choices[0].message.content
397
+
398
+ # Usage in your CLI
399
+ def suggest_organization_with_vllm(file_list: list[str]) -> str:
400
+ """Use local vLLM server for suggestions."""
401
+ client = vLLMClient()
402
+
403
+ messages = [
404
+ {"role": "system", "content": "You are a file organization assistant."},
405
+ {"role": "user", "content": f"Organize these files: {', '.join(file_list)}"}
406
+ ]
407
+
408
+ return client.chat_generate(messages)
409
+ ```
410
+
411
+ **Complete Inference Strategy:**
412
+
413
+ ```python
414
+ # src/file_organizer/ai_strategy.py
415
+ from typing import Literal
416
+ import os
417
+ from enum import Enum
418
+
419
+ class InferenceMode(str, Enum):
420
+ """Available inference modes."""
421
+ API = "api" # HuggingFace Inference API
422
+ LOCAL = "local" # Direct transformers
423
+ VLLM = "vllm" # vLLM server
424
+ AUTO = "auto" # Auto-detect best option
425
+
426
+ class UnifiedAIClient:
427
+ """Unified client supporting multiple inference backends."""
428
+
429
+ def __init__(self, mode: InferenceMode = InferenceMode.AUTO):
430
+ self.mode = self._resolve_mode(mode)
431
+ self._setup_client()
432
+
433
+ def _resolve_mode(self, mode: InferenceMode) -> InferenceMode:
434
+ """Auto-detect best available mode."""
435
+ if mode != InferenceMode.AUTO:
436
+ return mode
437
+
438
+ # Check if vLLM server is running
439
+ try:
440
+ import requests
441
+ requests.get("http://localhost:8000/health", timeout=1)
442
+ return InferenceMode.VLLM
443
+ except:
444
+ pass
445
+
446
+ # Check if HuggingFace token is available
447
+ if os.getenv("HUGGINGFACE_TOKEN"):
448
+ return InferenceMode.API
449
+
450
+ # Fall back to local
451
+ return InferenceMode.LOCAL
452
+
453
+ def _setup_client(self):
454
+ """Initialize the appropriate client."""
455
+ if self.mode == InferenceMode.VLLM:
456
+ from openai import OpenAI
457
+ self.client = OpenAI(
458
+ base_url="http://localhost:8000/v1",
459
+ api_key="not-needed"
460
+ )
461
+ elif self.mode == InferenceMode.API:
462
+ from huggingface_hub import InferenceClient
463
+ self.client = InferenceClient(token=os.getenv("HUGGINGFACE_TOKEN"))
464
+ else: # LOCAL
465
+ from transformers import pipeline
466
+ self.client = pipeline("text-generation", model="distilgpt2")
467
+
468
+ def generate(self, prompt: str, **kwargs) -> str:
469
+ """Generate text using configured backend."""
470
+ if self.mode == InferenceMode.VLLM:
471
+ response = self.client.completions.create(
472
+ model="mistralai/Mistral-7B-Instruct-v0.2",
473
+ prompt=prompt,
474
+ max_tokens=kwargs.get("max_tokens", 500)
475
+ )
476
+ return response.choices[0].text
477
+
478
+ elif self.mode == InferenceMode.API:
479
+ return self.client.text_generation(
480
+ prompt,
481
+ model="mistralai/Mistral-7B-Instruct-v0.2",
482
+ max_new_tokens=kwargs.get("max_tokens", 500)
483
+ )
484
+
485
+ else: # LOCAL
486
+ result = self.client(prompt, max_new_tokens=kwargs.get("max_tokens", 100))
487
+ return result[0]['generated_text']
488
+
489
+ # Use in CLI with Typer
490
+ import typer
491
+
492
+ @app.command()
493
+ def organize(
494
+ directory: Path,
495
+ inference_mode: InferenceMode = typer.Option(
496
+ InferenceMode.AUTO,
497
+ "--mode",
498
+ help="Inference mode: api, local, vllm, or auto"
499
+ )
500
+ ):
501
+ """Organize files using AI."""
502
+ ai_client = UnifiedAIClient(mode=inference_mode)
503
+ # Use ai_client.generate() for suggestions
504
+ ```
505
+
506
+ **vLLM Performance Tips:**
507
+
508
+ 1. **GPU Memory**: Use `--gpu-memory-utilization 0.9` to maximize GPU usage
509
+ 2. **Batch Size**: vLLM automatically batches requests for better throughput
510
+ 3. **Quantization**: Use GPTQ or AWQ quantized models for lower memory usage
511
+ 4. **Tensor Parallelism**: For multi-GPU: `--tensor-parallel-size 2`
512
+
513
+ **Docker Compose for vLLM (Optional):**
514
+
515
+ ```yaml
516
+ # docker-compose.vllm.yml
517
+ version: '3.8'
518
+
519
+ services:
520
+ vllm:
521
+ image: vllm/vllm-openai:latest
522
+ ports:
523
+ - "8000:8000"
524
+ environment:
525
+ - MODEL=mistralai/Mistral-7B-Instruct-v0.2
526
+ - MAX_MODEL_LEN=4096
527
+ deploy:
528
+ resources:
529
+ reservations:
530
+ devices:
531
+ - driver: nvidia
532
+ count: 1
533
+ capabilities: [gpu]
534
+ command: >
535
+ --host 0.0.0.0
536
+ --port 8000
537
+ --model ${MODEL}
538
+ --max-model-len ${MAX_MODEL_LEN}
539
+ ```
540
+
541
+ **Comparison:**
542
+
543
+ | Feature | HF API | Transformers | vLLM |
544
+ |---------|--------|--------------|------|
545
+ | Setup | Easy | Easy | Medium |
546
+ | Speed | Fast | Slow | Very Fast |
547
+ | Cost | Pay per use | Free | Free (local) |
548
+ | GPU Required | No | Optional | Recommended |
549
+ | Offline | No | Yes | Yes |
550
+ | Batch Processing | Limited | Poor | Excellent |
551
+ | Memory Efficient | N/A | No | Yes |
552
+ | OpenAI Compatible | No | No | Yes |
553
+
554
+ **Recommended Workflow:**
555
+ 1. **Development**: Use HuggingFace API for quick prototyping
556
+ 2. **Testing**: Use vLLM locally for faster iteration
557
+ 3. **Production**: Deploy vLLM server for best performance and privacy
558
+
559
+ #### 3.2 Docker-Based Model Deployment
560
+
561
+ Docker provides a modern, standardized way to deploy local LLM models with minimal configuration using Docker Compose v2.38+.
562
+
563
+ **Why Use Docker for AI Models?**
564
+ - **Consistent environments**: Same setup across development, testing, and production
565
+ - **Easy deployment**: One command to start models and services
566
+ - **Resource isolation**: Models run in containers with defined resource limits
567
+ - **Portability**: Works locally with Docker Model Runner or on cloud providers
568
+ - **Version control**: Pin specific model versions with OCI artifacts
569
+
570
+ **Prerequisites:**
571
+
572
+ ```bash
573
+ # Ensure Docker Compose v2.38 or later
574
+ docker compose version
575
+
576
+ # Enable Docker Model Runner in Docker Desktop settings
577
+ # Or install separately: https://docs.docker.com/ai/model-runner/
578
+ ```
579
+
580
+ **Basic Model Deployment with Docker Compose:**
581
+
582
+ Create a `docker-compose.yml` for your CLI project:
583
+
584
+ ```yaml
585
+ # docker-compose.yml
586
+ services:
587
+ # Your CLI application
588
+ file-organizer:
589
+ build: .
590
+ models:
591
+ - llm # Reference to the model defined below
592
+ environment:
593
+ # Auto-injected by Docker:
594
+ # LLM_URL - endpoint to access the model
595
+ # LLM_MODEL - model identifier
596
+ volumes:
597
+ - ./data:/app/data
598
+
599
+ models:
600
+ llm:
601
+ model: ai/smollm2 # Model from Docker Hub
602
+ context_size: 4096
603
+ runtime_flags:
604
+ - "--verbose"
605
+ - "--log-colors"
606
+ ```
607
+
608
+ **Using Models in Your Python CLI:**
609
+
610
+ ```python
611
+ # src/file_organizer/docker_ai.py
612
+ import os
613
+ from openai import OpenAI
614
+
615
+ class DockerModelClient:
616
+ """Client for Docker-deployed models with OpenAI-compatible API."""
617
+
618
+ def __init__(self):
619
+ # Docker automatically injects these environment variables
620
+ model_url = os.getenv("LLM_URL")
621
+ model_name = os.getenv("LLM_MODEL")
622
+
623
+ if not model_url:
624
+ raise ValueError("LLM_URL not set. Are you running with Docker Compose?")
625
+
626
+ # Docker models provide OpenAI-compatible endpoints
627
+ self.client = OpenAI(
628
+ base_url=model_url,
629
+ api_key="not-needed" # Docker models don't require API keys
630
+ )
631
+ self.model_name = model_name
632
+
633
+ def generate(self, prompt: str, max_tokens: int = 500) -> str:
634
+ """Generate text using Docker-deployed model."""
635
+ response = self.client.completions.create(
636
+ model=self.model_name,
637
+ prompt=prompt,
638
+ max_tokens=max_tokens,
639
+ temperature=0.7
640
+ )
641
+ return response.choices[0].text
642
+
643
+ def chat_generate(self, messages: list[dict], max_tokens: int = 500) -> str:
644
+ """Generate using chat completion format."""
645
+ response = self.client.chat.completions.create(
646
+ model=self.model_name,
647
+ messages=messages,
648
+ max_tokens=max_tokens
649
+ )
650
+ return response.choices[0].message.content
651
+
652
+ # Usage in your CLI
653
+ import typer
654
+
655
+ @app.command()
656
+ def organize(directory: Path):
657
+ """Organize files using Docker-deployed AI model."""
658
+ try:
659
+ ai_client = DockerModelClient()
660
+ # Use the model for suggestions
661
+ suggestion = ai_client.generate(f"Organize these files: {list(directory.iterdir())}")
662
+ console.print(suggestion)
663
+ except ValueError as e:
664
+ console.print(f"[red]Error: {e}[/red]")
665
+ console.print("[yellow]Run with: docker compose up[/yellow]")
666
+ ```
667
+
668
+ **Multi-Model Setup:**
669
+
670
+ Deploy multiple models for different tasks:
671
+
672
+ ```yaml
673
+ services:
674
+ file-organizer:
675
+ build: .
676
+ models:
677
+ chat-model:
678
+ endpoint_var: CHAT_MODEL_URL
679
+ model_var: CHAT_MODEL_NAME
680
+ embeddings:
681
+ endpoint_var: EMBEDDING_URL
682
+ model_var: EMBEDDING_NAME
683
+
684
+ models:
685
+ chat-model:
686
+ model: ai/smollm2
687
+ context_size: 4096
688
+ runtime_flags:
689
+ - "--temp"
690
+ - "0.7"
691
+
692
+ embeddings:
693
+ model: ai/all-minilm
694
+ context_size: 512
695
+ ```
696
+
697
+ **Model Configuration Presets:**
698
+
699
+ ```yaml
700
+ # Development mode - verbose logging
701
+ models:
702
+ dev_model:
703
+ model: ai/smollm2
704
+ context_size: 4096
705
+ runtime_flags:
706
+ - "--verbose"
707
+ - "--verbose-prompt"
708
+ - "--log-timestamps"
709
+ - "--log-colors"
710
+
711
+ # Production mode - deterministic output
712
+ models:
713
+ prod_model:
714
+ model: ai/smollm2
715
+ context_size: 4096
716
+ runtime_flags:
717
+ - "--temp"
718
+ - "0.1" # Low temperature for consistency
719
+ - "--top-k"
720
+ - "1"
721
+
722
+ # Creative mode - high randomness
723
+ models:
724
+ creative_model:
725
+ model: ai/smollm2
726
+ context_size: 4096
727
+ runtime_flags:
728
+ - "--temp"
729
+ - "1.0"
730
+ - "--top-p"
731
+ - "0.9"
732
+ ```
733
+
734
+ **Running Your Dockerized CLI:**
735
+
736
+ ```bash
737
+ # Start models and services
738
+ docker compose up -d
739
+
740
+ # Check model status
741
+ docker compose ps
742
+
743
+ # View model logs
744
+ docker compose logs llm
745
+
746
+ # Run your CLI (models are available via environment variables)
747
+ docker compose exec file-organizer python -m file_organizer organize ./data
748
+
749
+ # Stop everything
750
+ docker compose down
751
+ ```
752
+
753
+ **Complete Example Dockerfile:**
754
+
755
+ ```dockerfile
756
+ # Dockerfile
757
+ FROM python:3.11-slim
758
+
759
+ WORKDIR /app
760
+
761
+ # Install dependencies
762
+ COPY pyproject.toml .
763
+ RUN pip install -e .
764
+
765
+ # Copy application code
766
+ COPY src/ ./src/
767
+
768
+ # The CLI will use environment variables injected by Docker Compose
769
+ CMD ["python", "-m", "file_organizer.cli"]
770
+ ```
771
+
772
+ **Benefits of Docker Deployment:**
773
+
774
+ | Feature | Docker Compose | Manual Setup |
775
+ |---------|----------------|-------------|
776
+ | Setup Time | Minutes | Hours |
777
+ | Consistency | ✅ Same everywhere | ❌ Varies by system |
778
+ | Resource Control | ✅ Built-in limits | ⚠️ Manual config |
779
+ | Multi-model | ✅ Easy | ❌ Complex |
780
+ | Cloud Portability | ✅ Same config | ❌ Rewrite needed |
781
+ | Version Control | ✅ Git-friendly | ⚠️ Documentation |
782
+
783
+ **Cloud Deployment:**
784
+
785
+ The same `docker-compose.yml` works on cloud providers with extensions:
786
+
787
+ ```yaml
788
+ models:
789
+ llm:
790
+ model: ai/smollm2
791
+ context_size: 4096
792
+ # Cloud-specific options (provider-dependent)
793
+ x-cloud-options:
794
+ - "cloud.instance-type=gpu-small"
795
+ - "cloud.region=us-west-2"
796
+ - "cloud.auto-scaling=true"
797
+ ```
798
+
799
+ **Resources:**
800
+ - [Docker AI Documentation](https://docs.docker.com/ai/)
801
+ - [Docker Compose Models Reference](https://docs.docker.com/ai/compose/models-and-compose/)
802
+ - [Docker Model Runner](https://docs.docker.com/ai/model-runner/)
803
+ - [Available Models on Docker Hub](https://hub.docker.com/search?q=ai%2F)
804
+
805
+ #### 3.3 Docker MCP Toolkit: Secure Tool Integration
806
+
807
+ The Model Context Protocol (MCP) provides a standardized way for AI agents to interact with external tools and data sources. Docker's MCP Toolkit makes this secure and easy.
808
+
809
+ **What is MCP?**
810
+
811
+ MCP is an open protocol that allows AI models to:
812
+ - Execute code in isolated environments
813
+ - Access databases and APIs securely
814
+ - Use external tools (web search, calculators, etc.)
815
+ - Retrieve real-world data
816
+
817
+ **Why Docker MCP?**
818
+
819
+ 1. **Security**: Tools run in isolated containers
820
+ 2. **Trust**: Curated catalog with publisher verification
821
+ 3. **Simplicity**: One-click deployment from Docker Desktop
822
+ 4. **Dynamic Discovery**: Agents find and add tools as needed
823
+
824
+ **Docker MCP Components:**
825
+
826
+ ```yaml
827
+ # docker-compose.yml with MCP Gateway
828
+ services:
829
+ # Your AI-powered CLI
830
+ file-organizer:
831
+ build: .
832
+ models:
833
+ - llm
834
+ environment:
835
+ - MCP_GATEWAY_URL=http://mcp-gateway:3000
836
+ depends_on:
837
+ - mcp-gateway
838
+
839
+ # MCP Gateway - manages MCP servers
840
+ mcp-gateway:
841
+ image: docker/mcp-gateway:latest
842
+ ports:
843
+ - "3000:3000"
844
+ volumes:
845
+ - mcp-data:/data
846
+ environment:
847
+ - MCP_CATALOG_URL=https://hub.docker.com/mcp
848
+
849
+ models:
850
+ llm:
851
+ model: ai/smollm2
852
+ context_size: 4096
853
+
854
+ volumes:
855
+ mcp-data:
856
+ ```
857
+
858
+ **Using MCP in Your CLI:**
859
+
860
+ ```python
861
+ # src/file_organizer/mcp_client.py
862
+ import os
863
+ import requests
864
+ from typing import Any
865
+
866
+ class MCPClient:
867
+ """Client for Docker MCP Gateway."""
868
+
869
+ def __init__(self):
870
+ self.gateway_url = os.getenv("MCP_GATEWAY_URL", "http://localhost:3000")
871
+
872
+ def find_servers(self, query: str) -> list[dict]:
873
+ """Find MCP servers by name or description."""
874
+ response = requests.post(
875
+ f"{self.gateway_url}/mcp-find",
876
+ json={"query": query}
877
+ )
878
+ return response.json()["servers"]
879
+
880
+ def add_server(self, server_name: str) -> dict:
881
+ """Add an MCP server to the current session."""
882
+ response = requests.post(
883
+ f"{self.gateway_url}/mcp-add",
884
+ json={"server": server_name}
885
+ )
886
+ return response.json()
887
+
888
+ def call_tool(self, server: str, tool: str, params: dict) -> Any:
889
+ """Call a tool from an MCP server."""
890
+ response = requests.post(
891
+ f"{self.gateway_url}/mcp-call",
892
+ json={
893
+ "server": server,
894
+ "tool": tool,
895
+ "parameters": params
896
+ }
897
+ )
898
+ return response.json()["result"]
899
+
900
+ # Example: Web search integration
901
+ @app.command()
902
+ def research(topic: str):
903
+ """Research a topic using web search MCP."""
904
+ mcp = MCPClient()
905
+
906
+ # Find web search servers
907
+ servers = mcp.find_servers("web search")
908
+ console.print(f"Found {len(servers)} search servers")
909
+
910
+ # Add DuckDuckGo MCP
911
+ mcp.add_server("duckduckgo-mcp")
912
+
913
+ # Use the search tool
914
+ results = mcp.call_tool(
915
+ server="duckduckgo-mcp",
916
+ tool="search",
917
+ params={"query": topic, "max_results": 5}
918
+ )
919
+
920
+ # Display results
921
+ for result in results:
922
+ console.print(f"[bold]{result['title']}[/bold]")
923
+ console.print(f" {result['url']}")
924
+ console.print(f" {result['snippet']}\n")
925
+ ```
926
+
927
+ **Dynamic MCP Discovery:**
928
+
929
+ Let AI agents discover and use tools automatically:
930
+
931
+ ```python
932
+ # src/file_organizer/ai_agent.py
933
+ from openai import OpenAI
934
+ import json
935
+
936
+ class AIAgentWithMCP:
937
+ """AI agent that can discover and use MCP tools."""
938
+
939
+ def __init__(self):
940
+ self.llm = OpenAI(base_url=os.getenv("LLM_URL"), api_key="not-needed")
941
+ self.mcp = MCPClient()
942
+ self.available_tools = []
943
+
944
+ def discover_tools(self, task_description: str):
945
+ """Ask LLM what tools are needed for a task."""
946
+ prompt = f"""Task: {task_description}
947
+
948
+ What MCP tools would be helpful? Respond with JSON:
949
+ {{"tools": ["tool-name-1", "tool-name-2"]}}
950
+ """
951
+
952
+ response = self.llm.completions.create(
953
+ model=os.getenv("LLM_MODEL"),
954
+ prompt=prompt,
955
+ max_tokens=200
956
+ )
957
+
958
+ tools_needed = json.loads(response.choices[0].text)
959
+
960
+ # Add each tool
961
+ for tool in tools_needed["tools"]:
962
+ servers = self.mcp.find_servers(tool)
963
+ if servers:
964
+ self.mcp.add_server(servers[0]["name"])
965
+ self.available_tools.append(servers[0])
966
+
967
+ def execute_task(self, task: str):
968
+ """Execute a task using available tools."""
969
+ # First, discover what tools we need
970
+ self.discover_tools(task)
971
+
972
+ # Then execute with those tools
973
+ # (Implementation depends on your specific use case)
974
+ pass
975
+
976
+ # Usage
977
+ @app.command()
978
+ def smart_organize(directory: Path, strategy: str):
979
+ """Organize files using AI with dynamic tool discovery."""
980
+ agent = AIAgentWithMCP()
981
+
982
+ task = f"Organize files in {directory} using strategy: {strategy}"
983
+ agent.execute_task(task)
984
+ ```
985
+
986
+ **Available MCP Servers:**
987
+
988
+ The [Docker MCP Catalog](https://hub.docker.com/mcp) includes 270+ servers:
989
+
990
+ - **Web Search**: DuckDuckGo, Brave Search
991
+ - **Databases**: PostgreSQL, MongoDB, Elasticsearch
992
+ - **APIs**: Stripe, GitHub, Slack
993
+ - **Monitoring**: Grafana, Prometheus
994
+ - **File Systems**: Local files, S3, Google Drive
995
+ - **Development**: Git, Docker, Kubernetes
996
+
997
+ **Security Features:**
998
+
999
+ 1. **Container Isolation**: Each MCP server runs in its own container
1000
+ 2. **Commit Pinning**: Servers tied to specific Git commits
1001
+ 3. **Publisher Trust Levels**: Official, verified, and community servers
1002
+ 4. **AI-Audited Updates**: Automated code review for changes
1003
+ 5. **Resource Limits**: CPU and memory constraints per server
1004
+
1005
+ **Complete Example with MCP:**
1006
+
1007
+ ```yaml
1008
+ # docker-compose.yml - Full AI CLI with MCP
1009
+ services:
1010
+ file-organizer:
1011
+ build: .
1012
+ models:
1013
+ - llm
1014
+ environment:
1015
+ - MCP_GATEWAY_URL=http://mcp-gateway:3000
1016
+ - ENABLE_DYNAMIC_MCPS=true
1017
+ depends_on:
1018
+ - mcp-gateway
1019
+ volumes:
1020
+ - ./data:/app/data
1021
+
1022
+ mcp-gateway:
1023
+ image: docker/mcp-gateway:latest
1024
+ ports:
1025
+ - "3000:3000"
1026
+ volumes:
1027
+ - mcp-data:/data
1028
+ - ./mcp-config.yml:/config/catalog.yml
1029
+
1030
+ models:
1031
+ llm:
1032
+ model: ai/smollm2
1033
+ context_size: 4096
1034
+ runtime_flags:
1035
+ - "--temp"
1036
+ - "0.7"
1037
+
1038
+ volumes:
1039
+ mcp-data:
1040
+ ```
1041
+
1042
+ **MCP Best Practices:**
1043
+
1044
+ 1. **Start with trusted servers**: Use official and verified publishers
1045
+ 2. **Enable only needed tools**: Reduce attack surface
1046
+ 3. **Monitor MCP usage**: Track which tools are called
1047
+ 4. **Set resource limits**: Prevent runaway processes
1048
+ 5. **Review permissions**: Understand what each MCP can access
1049
+
1050
+ **Resources:**
1051
+ - [Docker MCP Gateway (GitHub)](https://github.com/docker/mcp-gateway/)
1052
+ - [Docker MCP Catalog](https://hub.docker.com/mcp)
1053
+ - [MCP Registry](https://github.com/docker/mcp-registry)
1054
+ - [Dynamic MCPs Blog](https://www.docker.com/blog/dynamic-mcps-stop-hardcoding-your-agents-world/)
1055
+ - [MCP Security Blog](https://www.docker.com/blog/enhancing-mcp-trust-with-the-docker-mcp-catalog/)
1056
+
1057
+ #### 3.4 Prompt Engineering for CLI Tools
1058
+
1059
+ **Learn About:**
1060
+ - Crafting effective prompts for different model types
1061
+ - Understanding model-specific prompt formats (Mistral, Llama, etc.)
1062
+ - System vs user messages (for chat models)
1063
+ - Few-shot learning examples
1064
+ - Prompt templates and variables
1065
+
1066
+ **Hands-On:**
1067
+ Create a prompt template system:
1068
+
1069
+ ```python
1070
+ # src/file_organizer/prompts.py
1071
+
1072
+ # For instruction-tuned models like Mistral
1073
+ MISTRAL_ORGANIZATION_PROMPT = """[INST] You are a helpful file organization assistant.
1074
+
1075
+ Given the following list of files:
1076
+ {file_list}
1077
+
1078
+ Suggest an intelligent organization strategy that:
1079
+ 1. Groups related files together
1080
+ 2. Creates meaningful folder names
1081
+ 3. Explains the reasoning
1082
+
1083
+ Respond in JSON format with this structure:
1084
+ {{
1085
+ "strategy": "description",
1086
+ "folders": [
1087
+ {{"name": "folder_name", "files": ["file1", "file2"], "reason": "why"}}
1088
+ ]
1089
+ }} [/INST]"""
1090
+
1091
+ # For Llama-2 chat models
1092
+ LLAMA_SYSTEM_PROMPT = """You are a helpful file organization assistant.
1093
+ Always respond in valid JSON format."""
1094
+
1095
+ def format_llama_prompt(user_message: str) -> str:
1096
+ """Format prompt for Llama-2 chat models."""
1097
+ return f"""<s>[INST] <<SYS>>
1098
+ {LLAMA_SYSTEM_PROMPT}
1099
+ <</SYS>>
1100
+
1101
+ {user_message} [/INST]"""
1102
+
1103
+ # For general models without special formatting
1104
+ GENERIC_PROMPT_TEMPLATE = """Task: Organize the following files intelligently.
1105
+
1106
+ Files: {file_list}
1107
+
1108
+ Instructions:
1109
+ - Group related files together
1110
+ - Suggest meaningful folder names
1111
+ - Explain your reasoning
1112
+ - Output as JSON
1113
+
1114
+ Response:"""
1115
+
1116
+ # Use Copilot to generate more prompt templates for different tasks
1117
+ ```
1118
+
1119
+ **Model-Specific Considerations:**
1120
+
1121
+ ```python
1122
+ # src/file_organizer/model_config.py
1123
+
1124
+ MODEL_CONFIGS = {
1125
+ "mistralai/Mistral-7B-Instruct-v0.2": {
1126
+ "max_tokens": 8192,
1127
+ "prompt_format": "mistral",
1128
+ "temperature": 0.7,
1129
+ "use_case": "general instruction following"
1130
+ },
1131
+ "meta-llama/Llama-2-7b-chat-hf": {
1132
+ "max_tokens": 4096,
1133
+ "prompt_format": "llama2",
1134
+ "temperature": 0.7,
1135
+ "use_case": "conversational tasks"
1136
+ },
1137
+ "facebook/bart-large-cnn": {
1138
+ "max_tokens": 1024,
1139
+ "prompt_format": "none",
1140
+ "use_case": "summarization only"
1141
+ }
1142
+ }
1143
+
1144
+ def get_model_config(model_name: str) -> dict:
1145
+ """Get configuration for a specific model."""
1146
+ return MODEL_CONFIGS.get(model_name, {})
1147
+ ```
1148
+
1149
+ **Copilot Prompts:**
1150
+ - "Create a function to format prompts based on model type"
1151
+ - "Generate few-shot examples for file categorization"
1152
+ - "Build a prompt validator that checks token limits"
1153
+ - "Create a prompt optimization function that reduces token usage"
1154
+
1155
+ ---
1156
+
1157
+ ### Phase 4: Advanced CLI Features (Week 6-7)
1158
+
1159
+ #### 4.1 Interactive CLI Elements
1160
+
1161
+ **Add Dependencies:**
1162
+
1163
+ ```bash
1164
+ pixi add questionary rich typer
1165
+ ```
1166
+
1167
+ **Learn to Build:**
1168
+ - Interactive prompts and menus
1169
+ - Progress bars and spinners
1170
+ - Tables and formatted output
1171
+ - Color-coded messages
1172
+
1173
+ **Example with Copilot:**
1174
+
1175
+ ```python
1176
+ # Ask Copilot: "Create an interactive menu using questionary
1177
+ # to select file organization options"
1178
+
1179
+ import questionary
1180
+ from rich.progress import track
1181
+
1182
+ def interactive_organize():
1183
+ # Copilot will help generate this
1184
+ pass
1185
+ ```
1186
+
1187
+ #### 4.2 Batch Processing and Async Operations
1188
+
1189
+ **Learn About:**
1190
+ - Processing multiple files efficiently
1191
+ - Async/await for concurrent API calls
1192
+ - Rate limiting and throttling
1193
+ - Progress tracking for long operations
1194
+
1195
+ ```bash
1196
+ # Add async dependencies
1197
+ pixi add aiohttp asyncio
1198
+ ```
1199
+
1200
+ **Copilot Exercise:**
1201
+ - "Create an async function to process multiple files with OpenAI API"
1202
+ - "Add rate limiting to prevent API quota exhaustion"
1203
+ - "Implement a queue system for batch processing"
1204
+
1205
+ ---
1206
+
1207
+ ### Phase 5: Testing and Quality (Week 8)
1208
+
1209
+ #### 5.1 Writing Tests
1210
+
1211
+ **Add Testing Dependencies:**
1212
+
1213
+ ```bash
1214
+ pixi add pytest pytest-cov pytest-asyncio pytest-mock
1215
+ ```
1216
+
1217
+ **Learn to Test:**
1218
+ - Unit tests for individual functions
1219
+ - Integration tests for CLI commands
1220
+ - Mocking API calls
1221
+ - Test coverage reporting
1222
+
1223
+ **Example Test Structure:**
1224
+
1225
+ ```python
1226
+ # tests/test_cli.py
1227
+ import pytest
1228
+ from typer.testing import CliRunner
1229
+ from file_organizer.cli import app
1230
+
1231
+ runner = CliRunner()
1232
+
1233
+ def test_organize_command():
1234
+ # Use Copilot to generate test cases
1235
+ result = runner.invoke(app, ['organize', 'test_dir', '--dry-run'])
1236
+ assert result.exit_code == 0
1237
+ assert "DRY RUN" in result.stdout
1238
+
1239
+ def test_organize_with_verbose():
1240
+ result = runner.invoke(app, ['organize', 'test_dir', '--verbose'])
1241
+ assert result.exit_code == 0
1242
+
1243
+ def test_stats_command():
1244
+ result = runner.invoke(app, ['stats', 'test_dir'])
1245
+ assert result.exit_code == 0
1246
+ ```
1247
+
1248
+ **Copilot Prompts:**
1249
+ - "Generate pytest fixtures for mocking HuggingFace Inference API"
1250
+ - "Create test cases for error handling with API timeouts"
1251
+ - "Write integration tests for the organize command"
1252
+ - "Mock transformers pipeline for local model testing"
1253
+
1254
+ **Example Mocking HuggingFace:**
1255
+
1256
+ ```python
1257
+ # tests/conftest.py
1258
+ import pytest
1259
+ from unittest.mock import Mock, patch
1260
+
1261
+ @pytest.fixture
1262
+ def mock_hf_client():
1263
+ """Mock HuggingFace InferenceClient."""
1264
+ with patch('huggingface_hub.InferenceClient') as mock:
1265
+ mock_instance = Mock()
1266
+ mock_instance.text_generation.return_value = '{"strategy": "test"}'
1267
+ mock.return_value = mock_instance
1268
+ yield mock_instance
1269
+
1270
+ @pytest.fixture
1271
+ def mock_transformers_pipeline():
1272
+ """Mock transformers pipeline for local models."""
1273
+ with patch('transformers.pipeline') as mock:
1274
+ mock_pipeline = Mock()
1275
+ mock_pipeline.return_value = [{"label": "POSITIVE", "score": 0.99}]
1276
+ mock.return_value = mock_pipeline
1277
+ yield mock_pipeline
1278
+ ```
1279
+
1280
+ #### 5.2 Code Quality Tools
1281
+
1282
+ ```bash
1283
+ # Add quality tools
1284
+ pixi add ruff mypy black isort
1285
+ ```
1286
+
1287
+ **Set Up:**
1288
+ - Linting with ruff
1289
+ - Type checking with mypy
1290
+ - Code formatting with black
1291
+ - Import sorting with isort
1292
+
1293
+ **Create `pyproject.toml` configuration (use Copilot):**
1294
+
1295
+ ```toml
1296
+ [tool.ruff]
1297
+ line-length = 100
1298
+ target-version = "py311"
1299
+
1300
+ [tool.mypy]
1301
+ python_version = "3.11"
1302
+ strict = true
1303
+
1304
+ [tool.black]
1305
+ line-length = 100
1306
+ ```
1307
+
1308
+ ---
1309
+
1310
+ ### Phase 6: Package Publishing with Pixi (Week 9)
1311
+
1312
+ #### 6.1 Preparing for Publication
1313
+
1314
+ **Project Structure:**
1315
+
1316
+ ```
1317
+ my-cli-tool/
1318
+ ├── pixi.toml # Pixi configuration
1319
+ ├── pyproject.toml # Python package metadata
1320
+ ├── README.md # Documentation
1321
+ ├── LICENSE # License file
1322
+ ├── src/
1323
+ │ └── my_cli_tool/
1324
+ │ ├── __init__.py
1325
+ │ ├── cli.py
1326
+ │ └── ...
1327
+ ├── tests/
1328
+ │ └── test_*.py
1329
+ └── docs/
1330
+ └── ...
1331
+ ```
1332
+
1333
+ **Configure `pyproject.toml` for Publishing:**
1334
+
1335
+ ```toml
1336
+ [build-system]
1337
+ requires = ["hatchling"]
1338
+ build-backend = "hatchling.build"
1339
+
1340
+ [project]
1341
+ name = "my-cli-tool"
1342
+ version = "0.1.0"
1343
+ description = "AI-powered file organization CLI"
1344
+ authors = [{name = "Your Name", email = "you@example.com"}]
1345
+ readme = "README.md"
1346
+ requires-python = ">=3.11"
1347
+ dependencies = [
1348
+ "typer>=0.9",
1349
+ "rich>=13.0",
1350
+ "transformers>=4.30",
1351
+ "huggingface-hub>=0.16",
1352
+ ]
1353
+
1354
+ [project.scripts]
1355
+ my-cli = "my_cli_tool.cli:cli"
1356
+
1357
+ [project.urls]
1358
+ Homepage = "https://github.com/yourusername/my-cli-tool"
1359
+ Documentation = "https://my-cli-tool.readthedocs.io"
1360
+ ```
1361
+
1362
+ **Use Copilot to:**
1363
+ - Generate comprehensive README with usage examples
1364
+ - Create CHANGELOG.md
1365
+ - Write contributing guidelines
1366
+ - Generate documentation
1367
+
1368
+ #### 6.2 Building and Publishing
1369
+
1370
+ **Build Package:**
1371
+
1372
+ ```bash
1373
+ # Add build tools
1374
+ pixi add hatchling build twine
1375
+
1376
+ # Build the package
1377
+ pixi run python -m build
1378
+
1379
+ # This creates:
1380
+ # - dist/my_cli_tool-0.1.0.tar.gz
1381
+ # - dist/my_cli_tool-0.1.0-py3-none-any.whl
1382
+ ```
1383
+
1384
+ **Publish to PyPI:**
1385
+
1386
+ ```bash
1387
+ # Test on TestPyPI first
1388
+ pixi run twine upload --repository testpypi dist/*
1389
+
1390
+ # Then publish to PyPI
1391
+ pixi run twine upload dist/*
1392
+ ```
1393
+
1394
+ **Publish as Pixi Package:**
1395
+
1396
+ ```bash
1397
+ # Create pixi.toml with package metadata
1398
+ pixi project init --name my-cli-tool
1399
+
1400
+ # Add to pixi.toml:
1401
+ [project]
1402
+ name = "my-cli-tool"
1403
+ version = "0.1.0"
1404
+ description = "AI-powered file organization CLI"
1405
+ channels = ["conda-forge"]
1406
+ platforms = ["linux-64", "osx-64", "win-64"]
1407
+
1408
+ [dependencies]
1409
+ python = ">=3.11"
1410
+ typer = ">=0.9"
1411
+ rich = ">=13.0"
1412
+
1413
+ [tasks]
1414
+ start = "my-cli"
1415
+ ```
1416
+
1417
+ **Resources:**
1418
+ - [Python Packaging Guide](https://packaging.python.org/)
1419
+ - [Pixi Publishing Guide](https://pixi.sh/latest/advanced/publishing/)
1420
+ - [Semantic Versioning](https://semver.org/)
1421
+
1422
+ ---
1423
+
1424
+ ### Phase 7: Real-World Project (Week 10-12)
1425
+
1426
+ #### 7.1 Choose a Project from the Ideas List
1427
+
1428
+ **Comprehensive Example Project:**
1429
+
1430
+ **[FileOrganizer](projects/FileOrganizer.md)** - AI-Powered File Organization CLI
1431
+ - **What it demonstrates**: Complete integration of all concepts from this learning path
1432
+ - **Key technologies**: Docker Model Runner, MCP servers, CrewAI multi-agent system, Typer CLI
1433
+ - **Complexity**: Advanced
1434
+ - **Best for**: Learners who have completed Phases 1-6 and want to see a production-ready example
1435
+ - **Features**:
1436
+ - Multi-agent system (Scanner, Classifier, Organizer, Deduplicator)
1437
+ - Docker-based LLM deployment
1438
+ - MCP server for file operations
1439
+ - Research paper management with metadata extraction
1440
+ - Comprehensive CLI with multiple commands
1441
+ - **Learning outcomes**: See how Docker AI, MCP, multi-agent systems, and CLI development work together in a real project
1442
+
1443
+ **Recommended Starter Projects:**
1444
+
1445
+ 1. **smart-csv** (Data & Analytics)
1446
+ - Good for: Learning data manipulation
1447
+ - Key skills: Pandas, CSV processing, LLM integration
1448
+ - Complexity: Medium
1449
+
1450
+ 2. **smart-summarize** (Document Processing)
1451
+ - Good for: Text processing and AI integration
1452
+ - Key skills: File I/O, API integration, prompt engineering
1453
+ - Complexity: Low-Medium
1454
+
1455
+ 3. **error-translator** (DevOps)
1456
+ - Good for: String processing and knowledge retrieval
1457
+ - Key skills: Pattern matching, API usage, caching
1458
+ - Complexity: Medium
1459
+
1460
+ 4. **task-prioritizer** (Productivity)
1461
+ - Good for: Building practical tools
1462
+ - Key skills: Data structures, AI reasoning, persistence
1463
+ - Complexity: Medium
1464
+
1465
+ > **💡 Tip**: Start with one of the simpler projects (2-4) to build confidence, then tackle FileOrganizer to see how all the concepts integrate in a production-ready application.
1466
+
1467
+ #### 7.2 Development Workflow with GitHub Copilot
1468
+
1469
+ **Step-by-Step Process:**
1470
+
1471
+ 1. **Planning Phase:**
1472
+ - Use Copilot Chat to brainstorm features
1473
+ - Generate project structure
1474
+ - Create initial documentation
1475
+
1476
+ 2. **Implementation Phase:**
1477
+ - Use Copilot for boilerplate code
1478
+ - Ask Copilot to explain unfamiliar concepts
1479
+ - Generate test cases alongside code
1480
+
1481
+ 3. **Refinement Phase:**
1482
+ - Use Copilot to suggest optimizations
1483
+ - Generate documentation and examples
1484
+ - Create user guides
1485
+
1486
+ **Effective Copilot Prompts:**
1487
+
1488
+ ```python
1489
+ # In comments, be specific:
1490
+ # "Create a function that reads a CSV file, analyzes column types,
1491
+ # and returns a dictionary with column names as keys and suggested
1492
+ # data types as values. Handle errors gracefully."
1493
+
1494
+ # Use descriptive function names:
1495
+ def analyze_csv_column_types(filepath: str) -> dict[str, str]:
1496
+ # Copilot will suggest implementation
1497
+ pass
1498
+
1499
+ # Ask for explanations:
1500
+ # "Explain how to use asyncio to make concurrent API calls with rate limiting"
1501
+ ```
1502
+
1503
+ #### 7.3 Project Milestones
1504
+
1505
+ **Week 10: MVP (Minimum Viable Product)**
1506
+ - [ ] Core functionality working
1507
+ - [ ] Basic CLI interface
1508
+ - [ ] Simple AI integration
1509
+ - [ ] README with usage examples
1510
+
1511
+ **Week 11: Enhancement**
1512
+ - [ ] Add configuration system
1513
+ - [ ] Implement error handling
1514
+ - [ ] Add progress indicators
1515
+ - [ ] Write tests (>70% coverage)
1516
+
1517
+ **Week 12: Polish & Publish**
1518
+ - [ ] Complete documentation
1519
+ - [ ] Add examples and tutorials
1520
+ - [ ] Set up CI/CD (GitHub Actions)
1521
+ - [ ] Publish to PyPI
1522
+ - [ ] Share on GitHub/social media
1523
+
1524
+ ---
1525
+
1526
+ ## 🛠️ Essential Pixi Commands Reference
1527
+
1528
+ ```bash
1529
+ # Project initialization
1530
+ pixi init my-project
1531
+ pixi init --channel conda-forge --channel bioconda
1532
+
1533
+ # Dependency management
1534
+ pixi add package-name # Add runtime dependency
1535
+ pixi add --dev pytest # Add dev dependency
1536
+ pixi add "package>=1.0,<2.0" # Version constraints
1537
+ pixi remove package-name # Remove dependency
1538
+ pixi update # Update all dependencies
1539
+
1540
+ # Environment management
1541
+ pixi shell # Activate environment
1542
+ pixi run python script.py # Run command in environment
1543
+ pixi run --environment prod start # Run in specific environment
1544
+
1545
+ # Task management
1546
+ pixi task add start "python -m my_cli"
1547
+ pixi task add test "pytest tests/"
1548
+ pixi task add lint "ruff check src/"
1549
+ pixi run start # Run defined task
1550
+
1551
+ # Multi-environment setup
1552
+ [feature.dev.dependencies]
1553
+ pytest = "*"
1554
+ ruff = "*"
1555
+
1556
+ [environments]
1557
+ default = ["dev"]
1558
+ prod = []
1559
+ ```
1560
+
1561
+ ---
1562
+
1563
+ ## 🎓 Learning Resources
1564
+
1565
+ ### Documentation
1566
+ - [Pixi Official Docs](https://pixi.sh/latest/)
1567
+ - [Python Packaging Guide](https://packaging.python.org/)
1568
+ - [Click Documentation](https://click.palletsprojects.com/)
1569
+ - [OpenAI API Reference](https://platform.openai.com/docs/)
1570
+ - [Docker AI Documentation](https://docs.docker.com/ai/)
1571
+ - [Docker Compose Models Reference](https://docs.docker.com/ai/compose/models-and-compose/)
1572
+ - [Docker MCP Gateway](https://github.com/docker/mcp-gateway/)
1573
+ - [Docker MCP Catalog](https://hub.docker.com/mcp)
1574
+
1575
+ ### Tutorials & Courses
1576
+ - [Real Python: Building CLI Applications](https://realpython.com/command-line-interfaces-python-argparse/)
1577
+ - [GitHub Copilot Learning Path](https://github.com/skills/copilot)
1578
+ - [LangChain Tutorials](https://python.langchain.com/docs/tutorials/)
1579
+
1580
+ ### Example Projects
1581
+ - [Typer Examples](https://github.com/tiangolo/typer/tree/master/docs_src)
1582
+ - [Rich Examples](https://github.com/Textualize/rich/tree/master/examples)
1583
+ - [AI CLI Tools on GitHub](https://github.com/topics/ai-cli)
1584
+
1585
+ ### Community
1586
+ - [Python Discord](https://discord.gg/python)
1587
+ - [r/Python](https://reddit.com/r/Python)
1588
+ - [Pixi GitHub Discussions](https://github.com/prefix-dev/pixi/discussions)
1589
+
1590
+ ---
1591
+
1592
+ ## 💡 Tips for Success
1593
+
1594
+ ### Using GitHub Copilot Effectively
1595
+
1596
+ 1. **Write Clear Comments:**
1597
+ ```python
1598
+ # Create a function that takes a list of file paths,
1599
+ # sends them to GPT-4 for analysis, and returns
1600
+ # a structured JSON response with organization suggestions
1601
+ ```
1602
+
1603
+ 2. **Use Descriptive Names:**
1604
+ - Good: `analyze_and_categorize_files()`
1605
+ - Bad: `process()`
1606
+
1607
+ 3. **Break Down Complex Tasks:**
1608
+ - Don't ask Copilot to generate entire applications
1609
+ - Build incrementally, function by function
1610
+
1611
+ 4. **Review and Understand:**
1612
+ - Always review Copilot's suggestions
1613
+ - Understand the code before accepting it
1614
+ - Test thoroughly
1615
+
1616
+ 5. **Use Copilot Chat for:**
1617
+ - Explaining error messages
1618
+ - Suggesting alternative approaches
1619
+ - Generating test cases
1620
+ - Writing documentation
1621
+
1622
+ ### Pixi Best Practices
1623
+
1624
+ 1. **Use Feature Flags:**
1625
+ ```toml
1626
+ [feature.ai]
1627
+ dependencies = {openai = "*", anthropic = "*"}
1628
+
1629
+ [feature.dev]
1630
+ dependencies = {pytest = "*", ruff = "*"}
1631
+
1632
+ [environments]
1633
+ default = ["ai"]
1634
+ dev = ["ai", "dev"]
1635
+ ```
1636
+
1637
+ 2. **Define Tasks:**
1638
+ ```toml
1639
+ [tasks]
1640
+ dev = "python -m my_cli --debug"
1641
+ test = "pytest tests/ -v"
1642
+ lint = "ruff check src/"
1643
+ format = "black src/ tests/"
1644
+ ```
1645
+
1646
+ 3. **Lock Dependencies:**
1647
+ - Commit `pixi.lock` to version control
1648
+ - Ensures reproducible builds
1649
+
1650
+ 4. **Use Channels Wisely:**
1651
+ - Start with `conda-forge`
1652
+ - Add specialized channels as needed
1653
+
1654
+ ### Development Workflow
1655
+
1656
+ 1. **Start Small:**
1657
+ - Build the simplest version first
1658
+ - Add features incrementally
1659
+ - Test each addition
1660
+
1661
+ 2. **Iterate Based on Feedback:**
1662
+ - Share early with friends/colleagues
1663
+ - Gather feedback
1664
+ - Improve based on real usage
1665
+
1666
+ 3. **Document as You Go:**
1667
+ - Write docstrings immediately
1668
+ - Update README with new features
1669
+ - Keep CHANGELOG current
1670
+
1671
+ 4. **Test Continuously:**
1672
+ - Write tests alongside code
1673
+ - Run tests before committing
1674
+ - Aim for >80% coverage
1675
+
1676
+ ---
1677
+
1678
+ ## 🎯 Success Metrics
1679
+
1680
+ By the end of this learning path, you should be able to:
1681
+
1682
+ - ✅ Set up a Python project with pixi
1683
+ - ✅ Build a CLI application with commands and options
1684
+ - ✅ Integrate AI/LLM capabilities effectively
1685
+ - ✅ Write tests and maintain code quality
1686
+ - ✅ Publish a package to PyPI
1687
+ - ✅ Use GitHub Copilot to accelerate development
1688
+ - ✅ Build one complete AI-powered CLI tool
1689
+
1690
+ ---
1691
+
1692
+ ## 📅 Next Steps
1693
+
1694
+ After completing this learning path:
1695
+
1696
+ 1. **Build More Projects:**
1697
+ - Try different project ideas from the list
1698
+ - Experiment with different AI models
1699
+ - Contribute to open-source CLI tools
1700
+
1701
+ 2. **Advanced Topics:**
1702
+ - Plugin architectures
1703
+ - Multi-command CLIs
1704
+ - Database integration
1705
+ - Web dashboards for CLI tools
1706
+ - CI/CD automation
1707
+
1708
+ 3. **Share Your Work:**
1709
+ - Write blog posts about your projects
1710
+ - Create video tutorials
1711
+ - Contribute to the community
1712
+ - Help others learn
1713
+
1714
+ ---
1715
+
1716
+ *Last Updated: 2024-12-04*
{src → docs}/notebooks/advanced_rag.qmd RENAMED
File without changes
{src → docs}/notebooks/automatic_embedding.ipynb RENAMED
File without changes
{src → docs}/notebooks/faiss.ipynb RENAMED
File without changes
{src → docs}/notebooks/rag_evaluation.qmd RENAMED
File without changes
{src → docs}/notebooks/rag_zephyr_langchain.qmd RENAMED
File without changes
{src → docs}/notebooks/single_gpu.ipynb RENAMED
File without changes
docs/projects/DataCrew.md ADDED
@@ -0,0 +1,571 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Course Project: DataCrew
2
+
3
+ **A CLI tool that uses local LLMs and multi-agent systems to transform spreadsheets into intelligent PDF reports.**
4
+
5
+ ---
6
+
7
+ ## Overview
8
+
9
+ ```
10
+ ┌─────────────────────────────────────────────────────────────────┐
11
+ │ DataCrew CLI │
12
+ ├─────────────────────────────────────────────────────────────────┤
13
+ │ $ datacrew ingest sales_2024.xlsx │
14
+ │ $ datacrew ask "What were the top 5 products by revenue?" │
15
+ │ $ datacrew report "Q4 Executive Summary" --output report.pdf │
16
+ └─────────────────────────────────────────────────────────────────┘
17
+ ```
18
+
19
+ ---
20
+
21
+ ## Architecture
22
+
23
+ ```
24
+ CSV/XLSX ──► SQLite ──► MCP Server ──► Multi-Agent Crew ──► PDF Report
25
+
26
+
27
+ Docker Model Runner
28
+ (Local LLM)
29
+ ```
30
+
31
+ ### Data Flow
32
+
33
+ ```
34
+ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
35
+ │ CSV/XLSX │────►│ SQLite │────►│ MCP Server │
36
+ │ Files │ │ Database │ │ (Tools) │
37
+ └──────────────┘ └──────────────┘ └──────┬───────┘
38
+
39
+
40
+ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
41
+ │ PDF Report │◄────│ Agent Crew │◄────│ Local LLM │
42
+ │ Output │ │ (CrewAI) │ │ (Docker) │
43
+ └──────────────┘ └──────────────┘ └──────────────┘
44
+ ```
45
+
46
+ ---
47
+
48
+ ## Agent System
49
+
50
+ | Agent | Role | Tools | Output |
51
+ |-------|------|-------|--------|
52
+ | **Data Analyst** | Understands schema, writes SQL queries | MCP Database Tools | Query results, data summaries |
53
+ | **Insights Agent** | Interprets results, finds patterns | Python Analysis, Statistics | Key findings, trends, anomalies |
54
+ | **Report Writer** | Creates narrative sections | LLM Generation | Executive summary, section text |
55
+ | **PDF Composer** | Formats and assembles final report | ReportLab/WeasyPrint | Formatted PDF document |
56
+
57
+ ### Agent Workflow
58
+
59
+ ```
60
+ User Request: "Generate Q4 Executive Summary"
61
+
62
+
63
+ ┌─────────────────────┐
64
+ │ Data Analyst │
65
+ │ "What data do we │
66
+ │ need for Q4?" │
67
+ └──────────┬──────────┘
68
+ │ SQL Queries
69
+
70
+ ┌─────────────────────┐
71
+ │ Insights Agent │
72
+ │ "What patterns │
73
+ │ emerge from │
74
+ │ this data?" │
75
+ └──────────┬──────────┘
76
+ │ Key Findings
77
+
78
+ ┌─────────────────────┐
79
+ │ Report Writer │
80
+ │ "Write narrative │
81
+ │ sections for │
82
+ │ each finding" │
83
+ └──────────┬──────────┘
84
+ │ Text Sections
85
+
86
+ ┌─────────────────────┐
87
+ │ PDF Composer │
88
+ │ "Assemble into │
89
+ │ formatted PDF" │
90
+ └──────────┬──────────┘
91
+
92
+
93
+ report.pdf
94
+ ```
95
+
96
+ ---
97
+
98
+ ## CLI Commands
99
+
100
+ ### `datacrew ingest`
101
+
102
+ Ingest CSV or XLSX files into the local SQLite database.
103
+
104
+ ```bash
105
+ # Ingest a single file
106
+ datacrew ingest sales_2024.xlsx
107
+
108
+ # Ingest with custom table name
109
+ datacrew ingest sales_2024.xlsx --table quarterly_sales
110
+
111
+ # Ingest multiple files
112
+ datacrew ingest data/*.csv
113
+
114
+ # Ingest with schema inference options
115
+ datacrew ingest sales.csv --infer-types --date-columns "order_date,ship_date"
116
+ ```
117
+
118
+ **Options:**
119
+
120
+ | Flag | Description | Default |
121
+ |------|-------------|---------|
122
+ | `--table` | Custom table name | Filename (sanitized) |
123
+ | `--if-exists` | Behavior if table exists: `fail`, `replace`, `append` | `fail` |
124
+ | `--infer-types` | Automatically infer column types | `true` |
125
+ | `--date-columns` | Comma-separated list of date columns | Auto-detect |
126
+ | `--db` | Database file path | `./data/datacrew.db` |
127
+
128
+ ### `datacrew ask`
129
+
130
+ Query the database using natural language.
131
+
132
+ ```bash
133
+ # Simple query
134
+ datacrew ask "What were the top 5 products by revenue?"
135
+
136
+ # Query with output format
137
+ datacrew ask "Show monthly sales trends" --format table
138
+
139
+ # Query with export
140
+ datacrew ask "List all customers from California" --export customers_ca.csv
141
+
142
+ # Interactive mode
143
+ datacrew ask --interactive
144
+ ```
145
+
146
+ **Options:**
147
+
148
+ | Flag | Description | Default |
149
+ |------|-------------|---------|
150
+ | `--format` | Output format: `table`, `json`, `csv`, `markdown` | `table` |
151
+ | `--export` | Export results to file | None |
152
+ | `--explain` | Show generated SQL query | `false` |
153
+ | `--interactive` | Enter interactive query mode | `false` |
154
+ | `--limit` | Maximum rows to return | `100` |
155
+
156
+ ### `datacrew report`
157
+
158
+ Generate PDF reports using the multi-agent system.
159
+
160
+ ```bash
161
+ # Generate a report
162
+ datacrew report "Q4 Executive Summary"
163
+
164
+ # Specify output file
165
+ datacrew report "Q4 Executive Summary" --output reports/q4_summary.pdf
166
+
167
+ # Use a template
168
+ datacrew report "Monthly Sales" --template executive
169
+
170
+ # Include specific analyses
171
+ datacrew report "Product Analysis" --include trends,comparisons,recommendations
172
+ ```
173
+
174
+ **Options:**
175
+
176
+ | Flag | Description | Default |
177
+ |------|-------------|---------|
178
+ | `--output`, `-o` | Output PDF file path | `./report.pdf` |
179
+ | `--template` | Report template: `executive`, `detailed`, `minimal` | `executive` |
180
+ | `--include` | Analyses to include | All |
181
+ | `--date-range` | Date range for analysis | All data |
182
+ | `--verbose`, `-v` | Show agent reasoning | `false` |
183
+
184
+ ### `datacrew config`
185
+
186
+ Manage configuration settings.
187
+
188
+ ```bash
189
+ # Show current config
190
+ datacrew config show
191
+
192
+ # Set LLM model
193
+ datacrew config set llm.model "llama3.2:3b"
194
+
195
+ # Set database path
196
+ datacrew config set database.path "./data/mydata.db"
197
+
198
+ # Reset to defaults
199
+ datacrew config reset
200
+ ```
201
+
202
+ ### `datacrew schema`
203
+
204
+ Inspect database schema.
205
+
206
+ ```bash
207
+ # List all tables
208
+ datacrew schema list
209
+
210
+ # Show table details
211
+ datacrew schema describe sales
212
+
213
+ # Show sample data
214
+ datacrew schema sample sales --rows 5
215
+ ```
216
+
217
+ ---
218
+
219
+ ## Configuration
220
+
221
+ Configuration is stored in `~/.config/datacrew/config.toml` or `./datacrew.toml` in the project directory.
222
+
223
+ ```toml
224
+ [datacrew]
225
+ version = "1.0.0"
226
+
227
+ [database]
228
+ path = "./data/datacrew.db"
229
+ echo = false
230
+
231
+ [llm]
232
+ provider = "docker" # docker, ollama, openai
233
+ model = "llama3.2:3b"
234
+ temperature = 0.7
235
+ max_tokens = 4096
236
+ base_url = "http://localhost:11434"
237
+
238
+ [llm.docker]
239
+ runtime = "nvidia" # nvidia, cpu
240
+ memory_limit = "8g"
241
+
242
+ [agents]
243
+ verbose = false
244
+ max_iterations = 10
245
+
246
+ [agents.analyst]
247
+ role = "Data Analyst"
248
+ goal = "Analyze data and write accurate SQL queries"
249
+
250
+ [agents.insights]
251
+ role = "Insights Specialist"
252
+ goal = "Find meaningful patterns and trends in data"
253
+
254
+ [agents.writer]
255
+ role = "Report Writer"
256
+ goal = "Create clear, compelling narrative content"
257
+
258
+ [agents.composer]
259
+ role = "PDF Composer"
260
+ goal = "Assemble professional PDF reports"
261
+
262
+ [reports]
263
+ output_dir = "./reports"
264
+ default_template = "executive"
265
+
266
+ [reports.templates.executive]
267
+ include_charts = true
268
+ include_recommendations = true
269
+ max_pages = 10
270
+
271
+ [reports.templates.detailed]
272
+ include_charts = true
273
+ include_recommendations = true
274
+ include_raw_data = true
275
+ max_pages = 50
276
+
277
+ [observability]
278
+ enabled = true
279
+ provider = "langfuse" # langfuse, langsmith, console
280
+ trace_agents = true
281
+ log_tokens = true
282
+ ```
283
+
284
+ ---
285
+
286
+ ## Docker Stack
287
+
288
+ ### docker-compose.yml
289
+
290
+ ```yaml
291
+ version: "3.9"
292
+
293
+ services:
294
+ # Local LLM via Docker Model Runner
295
+ llm:
296
+ image: ollama/ollama:latest
297
+ runtime: nvidia
298
+ environment:
299
+ - OLLAMA_HOST=0.0.0.0
300
+ volumes:
301
+ - ollama_data:/root/.ollama
302
+ ports:
303
+ - "11434:11434"
304
+ deploy:
305
+ resources:
306
+ reservations:
307
+ devices:
308
+ - driver: nvidia
309
+ count: 1
310
+ capabilities: [gpu]
311
+ healthcheck:
312
+ test: ["CMD", "curl", "-f", "http://localhost:11434/api/tags"]
313
+ interval: 30s
314
+ timeout: 10s
315
+ retries: 3
316
+
317
+ # MCP Server for database access
318
+ mcp-server:
319
+ build:
320
+ context: ./src/datacrew/mcp
321
+ dockerfile: Dockerfile
322
+ environment:
323
+ - DATABASE_PATH=/data/datacrew.db
324
+ - MCP_PORT=3000
325
+ volumes:
326
+ - ./data:/data
327
+ ports:
328
+ - "3000:3000"
329
+ depends_on:
330
+ - llm
331
+
332
+ # Main application (for containerized usage)
333
+ datacrew:
334
+ build:
335
+ context: .
336
+ dockerfile: Dockerfile
337
+ environment:
338
+ - LLM_BASE_URL=http://llm:11434
339
+ - MCP_SERVER_URL=http://mcp-server:3000
340
+ - DATABASE_PATH=/data/datacrew.db
341
+ volumes:
342
+ - ./data:/data
343
+ - ./reports:/reports
344
+ - ./input:/input:ro
345
+ depends_on:
346
+ llm:
347
+ condition: service_healthy
348
+ mcp-server:
349
+ condition: service_started
350
+ profiles:
351
+ - cli
352
+
353
+ volumes:
354
+ ollama_data:
355
+ ```
356
+
357
+ ### Running the Stack
358
+
359
+ ```bash
360
+ # Start LLM and MCP server
361
+ docker compose up -d llm mcp-server
362
+
363
+ # Pull the model (first time only)
364
+ docker compose exec llm ollama pull llama3.2:3b
365
+
366
+ # Run DataCrew commands
367
+ docker compose run --rm datacrew ingest /input/sales.xlsx
368
+ docker compose run --rm datacrew ask "What is total revenue?"
369
+ docker compose run --rm datacrew report "Sales Summary" -o /reports/summary.pdf
370
+
371
+ # Or run locally with Docker backend
372
+ datacrew ingest sales.xlsx
373
+ datacrew ask "What is total revenue?"
374
+ datacrew report "Sales Summary"
375
+ ```
376
+
377
+ ---
378
+
379
+ ## Project Structure
380
+
381
+ ```
382
+ datacrew/
383
+ ├── pyproject.toml # pixi/uv project config
384
+ ├── pixi.lock
385
+ ├── docker-compose.yml # Full stack orchestration
386
+ ├── Dockerfile
387
+ ├── datacrew.toml # Default configuration
388
+ ├── README.md
389
+
390
+ ├── src/
391
+ │ └── datacrew/
392
+ │ ├── __init__.py
393
+ │ ├── __main__.py # Entry point
394
+ │ ├── cli.py # Typer CLI commands
395
+ │ ├── config.py # TOML configuration loader
396
+ │ │
397
+ │ ├── ingestion/ # CSV/XLSX → SQLite
398
+ │ │ ├── __init__.py
399
+ │ │ ├── readers.py # File readers (pandas, openpyxl)
400
+ │ │ ├── schema.py # Schema inference
401
+ │ │ └── database.py # SQLite operations
402
+ │ │
403
+ │ ├── query/ # Natural language queries
404
+ │ │ ├── __init__.py
405
+ │ │ ├── nl2sql.py # NL to SQL conversion
406
+ │ │ ├── executor.py # Query execution
407
+ │ │ └── formatter.py # Result formatting
408
+ │ │
409
+ │ ├── agents/ # CrewAI agents
410
+ │ │ ├── __init__.py
411
+ │ │ ├── crew.py # Crew orchestration
412
+ │ │ ├── analyst.py # Data Analyst agent
413
+ │ │ ├── insights.py # Insights Specialist agent
414
+ │ │ ├── writer.py # Report Writer agent
415
+ │ │ └── composer.py # PDF Composer agent
416
+ │ │
417
+ │ ├── tools/ # Agent tools
418
+ │ │ ├── __init__.py
419
+ │ │ ├── sql_tools.py # SQL execution tools
420
+ │ │ ├── analysis.py # Statistical analysis tools
421
+ │ │ └── charts.py # Chart generation tools
422
+ │ │
423
+ │ ├── mcp/ # MCP server
424
+ │ │ ├── __init__.py
425
+ │ │ ├── server.py # MCP server implementation
426
+ │ │ ├── tools.py # MCP tool definitions
427
+ │ │ └── Dockerfile # MCP server container
428
+ │ │
429
+ │ ├── reports/ # PDF generation
430
+ │ │ ├── __init__.py
431
+ │ │ ├── generator.py # Report generation orchestrator
432
+ │ │ ├── pdf.py # PDF creation (WeasyPrint)
433
+ │ │ ├── charts.py # Chart rendering
434
+ │ │ └── templates/ # HTML/CSS templates
435
+ │ │ ├── executive.html
436
+ │ │ ├── detailed.html
437
+ │ │ ├── minimal.html
438
+ │ │ └── styles.css
439
+ │ │
440
+ │ ├── llm/ # LLM integration
441
+ │ │ ├── __init__.py
442
+ │ │ ├── client.py # LLM client (Docker/Ollama/OpenAI)
443
+ │ │ └── prompts.py # Prompt templates
444
+ │ │
445
+ │ └── observability/ # Logging & tracing
446
+ │ ├── __init__.py
447
+ │ ├── tracing.py # Distributed tracing
448
+ │ └── metrics.py # Token/cost tracking
449
+
450
+ ├── tests/
451
+ │ ├── __init__.py
452
+ │ ├── conftest.py # Pytest fixtures
453
+ │ ├── test_cli.py
454
+ │ ├── test_ingestion.py
455
+ │ ├── test_query.py
456
+ │ ├── test_agents.py
457
+ │ ├── test_reports.py
458
+ │ └── fixtures/
459
+ │ ├── sample_sales.csv
460
+ │ ├── sample_products.xlsx
461
+ │ └── expected_outputs/
462
+
463
+ ├── data/ # Local data directory
464
+ │ └── .gitkeep
465
+
466
+ ├── reports/ # Generated reports
467
+ │ └── .gitkeep
468
+
469
+ └── docs/ # Documentation (Quarto)
470
+ ├── _quarto.yml
471
+ ├── index.qmd
472
+ └── chapters/
473
+ ```
474
+
475
+ ---
476
+
477
+ ## Technology Stack
478
+
479
+ | Category | Tools |
480
+ |----------|-------|
481
+ | **Package Management** | pixi, uv |
482
+ | **CLI Framework** | Typer, Rich |
483
+ | **Local LLM** | Docker Model Runner, Ollama |
484
+ | **LLM Framework** | LangChain |
485
+ | **Multi-Agent** | CrewAI |
486
+ | **MCP** | Docker MCP Toolkit |
487
+ | **Database** | SQLite |
488
+ | **Data Processing** | pandas, openpyxl |
489
+ | **PDF Generation** | WeasyPrint |
490
+ | **Charts** | matplotlib, plotly |
491
+ | **Observability** | Langfuse, OpenTelemetry |
492
+ | **Testing** | pytest, DeepEval |
493
+ | **Containerization** | Docker, Docker Compose |
494
+
495
+ ---
496
+
497
+ ## Example Usage
498
+
499
+ ### End-to-End Workflow
500
+
501
+ ```bash
502
+ # 1. Start the Docker stack
503
+ docker compose up -d
504
+
505
+ # 2. Ingest your data
506
+ datacrew ingest quarterly_sales_2024.xlsx
507
+ datacrew ingest product_catalog.csv
508
+ datacrew ingest customer_data.csv
509
+
510
+ # 3. Explore with natural language queries
511
+ datacrew ask "How many records are in each table?"
512
+ datacrew ask "What are the top 10 products by revenue in Q4?"
513
+ datacrew ask "Show me the monthly sales trend for 2024"
514
+
515
+ # 4. Generate a comprehensive report
516
+ datacrew report "2024 Annual Sales Analysis" \
517
+ --template detailed \
518
+ --output reports/annual_2024.pdf \
519
+ --include trends,top_products,regional_breakdown,recommendations \
520
+ --verbose
521
+
522
+ # 5. View agent reasoning (verbose mode)
523
+ # [Data Analyst] Analyzing schema... found 3 tables
524
+ # [Data Analyst] Executing: SELECT strftime('%Y-%m', order_date) as month, SUM(revenue) ...
525
+ # [Insights Agent] Identified trend: 23% YoY growth in Q4
526
+ # [Insights Agent] Anomaly detected: December spike in electronics category
527
+ # [Report Writer] Generating executive summary...
528
+ # [PDF Composer] Assembling 12-page report...
529
+ # ✓ Report saved to reports/annual_2024.pdf
530
+ ```
531
+
532
+ ### Sample Report Output
533
+
534
+ ```
535
+ ┌────────────────────────────────────────────────────────────────┐
536
+ │ 2024 Annual Sales Analysis │
537
+ │ Executive Summary │
538
+ ├────────────────────────────────────────────────────────────────┤
539
+ │ │
540
+ │ Key Findings: │
541
+ │ • Total revenue: $4.2M (+23% YoY) │
542
+ │ • Top product category: Electronics (38% of revenue) │
543
+ │ • Strongest region: West Coast (42% of sales) │
544
+ │ • Customer retention rate: 78% │
545
+ │ │
546
+ │ [Monthly Revenue Trend Chart] │
547
+ │ │
548
+ │ Recommendations: │
549
+ │ 1. Expand electronics inventory for Q1 2025 │
550
+ │ 2. Increase marketing spend in Midwest region │
551
+ │ 3. Launch loyalty program to improve retention │
552
+ │ │
553
+ └────────────────────────────────────────────────────────────────┘
554
+ ```
555
+
556
+ ---
557
+
558
+ ## Learning Outcomes
559
+
560
+ By building DataCrew, learners will be able to:
561
+
562
+ 1. ✅ Set up modern Python projects with pixi and reproducible environments
563
+ 2. ✅ Build professional CLI tools with Typer and Rich
564
+ 3. ✅ Run local LLMs using Docker Model Runner
565
+ 4. ✅ Ingest and query data from spreadsheets using natural language
566
+ 5. ✅ Build MCP servers to connect AI agents to data sources
567
+ 6. ✅ Design multi-agent systems with CrewAI
568
+ 7. ✅ Generate PDF reports programmatically
569
+ 8. ✅ Implement observability for AI applications
570
+ 9. ✅ Test non-deterministic systems effectively
571
+ 10. ✅ Deploy self-hosted AI applications with Docker Compose
docs/projects/FileOrganizer.md ADDED
@@ -0,0 +1,706 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Course Project: FileOrganizer
2
+
3
+ **A CLI tool that uses local LLMs and AI agents to intelligently organize files, with special focus on research paper management.**
4
+
5
+ ---
6
+
7
+ ## Overview
8
+
9
+ ```
10
+ ┌─────────────────────────────────────────────────────────────────┐
11
+ │ FileOrganizer CLI │
12
+ ├─────────────────────────────────────────────────────────────────┤
13
+ │ $ fileorg scan ~/Downloads │
14
+ │ $ fileorg organize ~/Papers --strategy=by-topic │
15
+ │ $ fileorg deduplicate ~/Research --similarity=0.9 │
16
+ └─────────────────────────────────────────────────────────────────┘
17
+ ```
18
+
19
+ ---
20
+
21
+ ## Architecture
22
+
23
+ ```
24
+ Files ──► Content Analysis ──► AI Classification ──► Organized Structure
25
+ │ │
26
+ ▼ ▼
27
+ PDF Extraction Docker Model Runner
28
+ Metadata Tools (Local LLM)
29
+ │ │
30
+ └────────►MCP◄───────────┘
31
+ ```
32
+
33
+ ### Data Flow
34
+
35
+ ```
36
+ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
37
+ │ Files/PDFs │────►│ Content │────►│ MCP Server │
38
+ │ (Input) │ │ Extraction │ │ (Tools) │
39
+ └──────────────┘ └──────────────┘ └──────┬───────┘
40
+
41
+
42
+ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
43
+ │ Organized │◄────│ Agent Crew │◄────│ Local LLM │
44
+ │ Structure │ │ (CrewAI) │ │ (Docker) │
45
+ └──────────────┘ └──────────────┘ └──────────────┘
46
+ ```
47
+
48
+ ---
49
+
50
+ ## Agent System
51
+
52
+ | Agent | Role | Tools | Output |
53
+ |-------|------|-------|--------|
54
+ | **Scanner Agent** | Discovers files, extracts metadata | File I/O, PDF extraction, hash generation | File inventory, metadata catalog |
55
+ | **Classifier Agent** | Categorizes files by content and context | LLM analysis, embeddings, similarity | Category assignments, topic tags |
56
+ | **Organizer Agent** | Creates folder structure and moves files | File operations, naming strategies | Organized directory tree |
57
+ | **Deduplicator Agent** | Finds and handles duplicate files | Hash comparison, content similarity | Duplicate reports, cleanup actions |
58
+
59
+ ### Agent Workflow
60
+
61
+ ```
62
+ User Request: "Organize research papers by topic"
63
+
64
+
65
+ ┌─────────────────────┐
66
+ │ Scanner Agent │
67
+ │ "What files do we │
68
+ │ have and what │
69
+ │ are they about?" │
70
+ └──────────┬──────────┘
71
+ │ File Inventory
72
+
73
+ ┌─────────────────────┐
74
+ │ Classifier Agent │
75
+ │ "What topics and │
76
+ │ categories emerge │
77
+ │ from the content?"│
78
+ └──────────┬──────────┘
79
+ │ Categories
80
+
81
+ ┌─────────────────────┐
82
+ │ Organizer Agent │
83
+ │ "Create folder │
84
+ │ structure and │
85
+ │ move files" │
86
+ └──────────┬──────────┘
87
+ │ Organization Plan
88
+
89
+ ┌─────────────────────┐
90
+ │ Deduplicator Agent │
91
+ │ "Find and handle │
92
+ │ duplicate files" │
93
+ └──────────┬──────────┘
94
+
95
+
96
+ Organized Directory
97
+ ```
98
+
99
+ ---
100
+
101
+ ## CLI Commands
102
+
103
+ ### `fileorg scan`
104
+
105
+ Scan a directory and analyze its contents.
106
+
107
+ ```bash
108
+ # Scan a directory
109
+ fileorg scan ~/Downloads
110
+
111
+ # Scan with detailed analysis
112
+ fileorg scan ~/Papers --analyze-content
113
+
114
+ # Scan and export inventory
115
+ fileorg scan ~/Research --export inventory.json
116
+
117
+ # Scan specific file types
118
+ fileorg scan ~/Documents --types pdf,docx,txt
119
+ ```
120
+
121
+ **Options:**
122
+
123
+ | Flag | Description | Default |
124
+ |------|-------------|---------|
125
+ | `--analyze-content` | Extract and analyze file contents | `false` |
126
+ | `--export` | Export inventory to JSON/CSV | None |
127
+ | `--types` | Comma-separated file extensions to scan | All |
128
+ | `--recursive` | Scan subdirectories | `true` |
129
+ | `--max-depth` | Maximum directory depth | `10` |
130
+
131
+ ### `fileorg organize`
132
+
133
+ Organize files using AI-powered strategies.
134
+
135
+ ```bash
136
+ # Organize by topic (AI-powered)
137
+ fileorg organize ~/Papers --strategy=by-topic
138
+
139
+ # Organize by date
140
+ fileorg organize ~/Photos --strategy=by-date --format="%Y/%m"
141
+
142
+ # Organize with custom naming
143
+ fileorg organize ~/Papers --rename --pattern="{year}_{author}_{title}"
144
+
145
+ # Dry run to preview changes
146
+ fileorg organize ~/Downloads --dry-run
147
+
148
+ # Interactive mode
149
+ fileorg organize ~/Research --interactive
150
+ ```
151
+
152
+ **Options:**
153
+
154
+ | Flag | Description | Default |
155
+ |------|-------------|---------|
156
+ | `--strategy` | Organization strategy: `by-topic`, `by-date`, `by-type`, `by-author`, `smart` | `smart` |
157
+ | `--rename` | Rename files intelligently | `false` |
158
+ | `--pattern` | Naming pattern for renamed files | `{original}` |
159
+ | `--dry-run` | Preview changes without executing | `false` |
160
+ | `--interactive` | Confirm each action | `false` |
161
+ | `--output` | Output directory | Same as input |
162
+
163
+ ### `fileorg deduplicate`
164
+
165
+ Find and handle duplicate files.
166
+
167
+ ```bash
168
+ # Find duplicates by hash
169
+ fileorg deduplicate ~/Downloads
170
+
171
+ # Find similar files (content-based)
172
+ fileorg deduplicate ~/Papers --similarity=0.9
173
+
174
+ # Auto-delete duplicates (keep newest)
175
+ fileorg deduplicate ~/Photos --auto-delete --keep=newest
176
+
177
+ # Move duplicates to folder
178
+ fileorg deduplicate ~/Documents --move-to=./duplicates
179
+ ```
180
+
181
+ **Options:**
182
+
183
+ | Flag | Description | Default |
184
+ |------|-------------|---------|
185
+ | `--similarity` | Similarity threshold (0.0-1.0) for content matching | `1.0` (exact) |
186
+ | `--method` | Detection method: `hash`, `content`, `metadata` | `hash` |
187
+ | `--auto-delete` | Automatically delete duplicates | `false` |
188
+ | `--keep` | Which to keep: `newest`, `oldest`, `largest`, `smallest` | `newest` |
189
+ | `--move-to` | Move duplicates to directory instead of deleting | None |
190
+
191
+ ### `fileorg research`
192
+
193
+ Special commands for research paper management.
194
+
195
+ ```bash
196
+ # Extract metadata from PDFs
197
+ fileorg research extract ~/Papers
198
+
199
+ # Generate bibliography
200
+ fileorg research bibliography ~/Papers --format=bibtex --output=refs.bib
201
+
202
+ # Find related papers
203
+ fileorg research related "attention mechanisms" --in ~/Papers
204
+
205
+ # Create reading list
206
+ fileorg research reading-list ~/Papers --topic "transformers" --order=citations
207
+ ```
208
+
209
+ **Options:**
210
+
211
+ | Flag | Description | Default |
212
+ |------|-------------|---------|
213
+ | `--format` | Bibliography format: `bibtex`, `apa`, `mla` | `bibtex` |
214
+ | `--output` | Output file path | `stdout` |
215
+ | `--order` | Sort order: `date`, `citations`, `relevance` | `relevance` |
216
+
217
+ ### `fileorg config`
218
+
219
+ Manage configuration settings.
220
+
221
+ ```bash
222
+ # Show current config
223
+ fileorg config show
224
+
225
+ # Set LLM model
226
+ fileorg config set llm.model "llama3.2:3b"
227
+
228
+ # Set default strategy
229
+ fileorg config set organize.default_strategy "by-topic"
230
+
231
+ # Reset to defaults
232
+ fileorg config reset
233
+ ```
234
+
235
+ ### `fileorg stats`
236
+
237
+ Show statistics about files and organization.
238
+
239
+ ```bash
240
+ # Show directory statistics
241
+ fileorg stats ~/Papers
242
+
243
+ # Show organization suggestions
244
+ fileorg stats ~/Downloads --suggest
245
+
246
+ # Export statistics
247
+ fileorg stats ~/Research --export stats.json
248
+ ```
249
+
250
+ ---
251
+
252
+ ## Configuration
253
+
254
+ Configuration is stored in `~/.config/fileorg/config.toml` or `./fileorg.toml` in the project directory.
255
+
256
+ ```toml
257
+ [fileorg]
258
+ version = "1.0.0"
259
+
260
+ [llm]
261
+ provider = "docker" # docker, ollama, openai
262
+ model = "llama3.2:3b"
263
+ temperature = 0.7
264
+ max_tokens = 4096
265
+ base_url = "http://localhost:11434"
266
+
267
+ [llm.docker]
268
+ runtime = "nvidia" # nvidia, cpu
269
+ memory_limit = "8g"
270
+
271
+ [agents]
272
+ verbose = false
273
+ max_iterations = 10
274
+
275
+ [agents.scanner]
276
+ role = "File Scanner"
277
+ goal = "Discover and catalog all files with metadata"
278
+
279
+ [agents.classifier]
280
+ role = "Content Classifier"
281
+ goal = "Categorize files by content and context"
282
+
283
+ [agents.organizer]
284
+ role = "File Organizer"
285
+ goal = "Create optimal folder structure and organize files"
286
+
287
+ [agents.deduplicator]
288
+ role = "Duplicate Detector"
289
+ goal = "Find and handle duplicate files efficiently"
290
+
291
+ [organize]
292
+ default_strategy = "smart"
293
+ create_backups = true
294
+ backup_dir = "./.fileorg_backup"
295
+
296
+ [organize.naming]
297
+ sanitize = true
298
+ max_length = 255
299
+ replace_spaces = "_"
300
+
301
+ [research]
302
+ extract_metadata = true
303
+ auto_rename = true
304
+ naming_pattern = "{year}_{author}_{title}"
305
+ generate_bibliography = true
306
+
307
+ [deduplication]
308
+ default_method = "hash"
309
+ similarity_threshold = 0.95
310
+ auto_delete = false
311
+ keep_strategy = "newest"
312
+
313
+ [pdf]
314
+ extract_text = true
315
+ extract_metadata = true
316
+ ocr_enabled = false # Enable OCR for scanned PDFs
317
+
318
+ [observability]
319
+ enabled = true
320
+ provider = "langfuse" # langfuse, langsmith, console
321
+ trace_agents = true
322
+ log_tokens = true
323
+ ```
324
+
325
+ ---
326
+
327
+ ## Docker Stack
328
+
329
+ ### docker-compose.yml
330
+
331
+ ```yaml
332
+ version: "3.9"
333
+
334
+ services:
335
+ # Local LLM via Docker Model Runner
336
+ llm:
337
+ image: ollama/ollama:latest
338
+ runtime: nvidia
339
+ environment:
340
+ - OLLAMA_HOST=0.0.0.0
341
+ volumes:
342
+ - ollama_data:/root/.ollama
343
+ ports:
344
+ - "11434:11434"
345
+ deploy:
346
+ resources:
347
+ reservations:
348
+ devices:
349
+ - driver: nvidia
350
+ count: 1
351
+ capabilities: [gpu]
352
+ healthcheck:
353
+ test: ["CMD", "curl", "-f", "http://localhost:11434/api/tags"]
354
+ interval: 30s
355
+ timeout: 10s
356
+ retries: 3
357
+
358
+ # MCP Server for file operations and PDF tools
359
+ mcp-server:
360
+ build:
361
+ context: ./src/fileorg/mcp
362
+ dockerfile: Dockerfile
363
+ environment:
364
+ - MCP_PORT=3000
365
+ volumes:
366
+ - ./workspace:/workspace
367
+ ports:
368
+ - "3000:3000"
369
+ depends_on:
370
+ - llm
371
+
372
+ # Main application (for containerized usage)
373
+ fileorg:
374
+ build:
375
+ context: .
376
+ dockerfile: Dockerfile
377
+ environment:
378
+ - LLM_BASE_URL=http://llm:11434
379
+ - MCP_SERVER_URL=http://mcp-server:3000
380
+ volumes:
381
+ - ./workspace:/workspace
382
+ - ./config:/config:ro
383
+ depends_on:
384
+ llm:
385
+ condition: service_healthy
386
+ mcp-server:
387
+ condition: service_started
388
+ profiles:
389
+ - cli
390
+
391
+ volumes:
392
+ ollama_data:
393
+ ```
394
+
395
+ ### Running the Stack
396
+
397
+ ```bash
398
+ # Start LLM and MCP server
399
+ docker compose up -d llm mcp-server
400
+
401
+ # Pull the model (first time only)
402
+ docker compose exec llm ollama pull llama3.2:3b
403
+
404
+ # Run FileOrganizer commands
405
+ docker compose run --rm fileorg scan /workspace/papers
406
+ docker compose run --rm fileorg organize /workspace/papers --strategy=by-topic
407
+ docker compose run --rm fileorg deduplicate /workspace/downloads
408
+
409
+ # Or run locally with Docker backend
410
+ fileorg scan ~/Papers
411
+ fileorg organize ~/Papers --strategy=by-topic
412
+ fileorg deduplicate ~/Downloads
413
+ ```
414
+
415
+ ---
416
+
417
+ ## Project Structure
418
+
419
+ ```
420
+ fileorg/
421
+ ├── pyproject.toml # pixi/uv project config
422
+ ├── pixi.lock
423
+ ├── docker-compose.yml # Full stack orchestration
424
+ ├── Dockerfile
425
+ ├── fileorg.toml # Default configuration
426
+ ├── README.md
427
+
428
+ ├── src/
429
+ │ └── fileorg/
430
+ │ ├── __init__.py
431
+ │ ├── __main__.py # Entry point
432
+ │ ├── cli.py # Typer CLI commands
433
+ │ ├── config.py # TOML configuration loader
434
+ │ │
435
+ │ ├── scanner/ # File discovery and analysis
436
+ │ │ ├── __init__.py
437
+ │ │ ├── discovery.py # File system traversal
438
+ │ │ ├── metadata.py # Metadata extraction
439
+ │ │ ├── pdf_reader.py # PDF text/metadata extraction
440
+ │ │ └── hashing.py # File hashing utilities
441
+ │ │
442
+ │ ├── classifier/ # Content classification
443
+ │ │ ├── __init__.py
444
+ │ │ ├── embeddings.py # Generate embeddings
445
+ │ │ ├── clustering.py # Topic clustering
446
+ │ │ ├── categorizer.py # AI-powered categorization
447
+ │ │ └── similarity.py # Content similarity
448
+ │ │
449
+ │ ├── organizer/ # File organization
450
+ │ │ ├── __init__.py
451
+ │ │ ├── strategies.py # Organization strategies
452
+ │ │ ├── naming.py # File naming logic
453
+ │ │ ├── structure.py # Directory structure creation
454
+ │ │ └── mover.py # Safe file operations
455
+ │ │
456
+ │ ├── deduplicator/ # Duplicate detection
457
+ │ │ ├── __init__.py
458
+ │ │ ├── hash_based.py # Hash-based detection
459
+ │ │ ├── content_based.py # Content similarity detection
460
+ │ │ └── handler.py # Duplicate handling
461
+ │ │
462
+ │ ├── research/ # Research paper tools
463
+ │ │ ├── __init__.py
464
+ │ │ ├── extractor.py # PDF metadata extraction
465
+ │ │ ├── bibliography.py # Bibliography generation
466
+ │ │ ├── citation.py # Citation parsing
467
+ │ │ └── scholar.py # Academic search integration
468
+ │ │
469
+ │ ├── agents/ # CrewAI agents
470
+ │ │ ├── __init__.py
471
+ │ │ ├── crew.py # Crew orchestration
472
+ │ │ ├── scanner.py # Scanner agent
473
+ │ │ ├── classifier.py # Classifier agent
474
+ │ │ ├── organizer.py # Organizer agent
475
+ │ │ └── deduplicator.py # Deduplicator agent
476
+ │ │
477
+ │ ├── tools/ # Agent tools
478
+ │ │ ├── __init__.py
479
+ │ │ ├── file_tools.py # File operation tools
480
+ │ │ ├── pdf_tools.py # PDF processing tools
481
+ │ │ ├── search_tools.py # Search and query tools
482
+ │ │ └── analysis.py # Content analysis tools
483
+ │ │
484
+ │ ├── mcp/ # MCP server
485
+ │ │ ├── __init__.py
486
+ │ │ ├── server.py # MCP server implementation
487
+ │ │ ├── tools.py # MCP tool definitions
488
+ │ │ └── Dockerfile # MCP server container
489
+ │ │
490
+ │ ├── llm/ # LLM integration
491
+ │ │ ├── __init__.py
492
+ │ │ ├── client.py # LLM client (Docker/Ollama/OpenAI)
493
+ │ │ └── prompts.py # Prompt templates
494
+ │ │
495
+ │ └── observability/ # Logging & tracing
496
+ │ ├── __init__.py
497
+ │ ├── tracing.py # Distributed tracing
498
+ │ └── metrics.py # Token/cost tracking
499
+
500
+ ├── tests/
501
+ │ ├── __init__.py
502
+ │ ├── conftest.py # Pytest fixtures
503
+ │ ├── test_cli.py
504
+ │ ├── test_scanner.py
505
+ │ ├── test_classifier.py
506
+ │ ├── test_organizer.py
507
+ │ ├── test_deduplicator.py
508
+ │ ├── test_research.py
509
+ │ └── fixtures/
510
+ │ ├── sample_papers/
511
+ │ │ ├── paper1.pdf
512
+ │ │ ├── paper2.pdf
513
+ │ │ └── paper3.pdf
514
+ │ ├── sample_files/
515
+ │ └── expected_outputs/
516
+
517
+ ├── workspace/ # Working directory
518
+ │ └── .gitkeep
519
+
520
+ └── docs/ # Documentation (Quarto)
521
+ ├── _quarto.yml
522
+ ├── index.qmd
523
+ └── chapters/
524
+ ```
525
+
526
+ ---
527
+
528
+ ## Technology Stack
529
+
530
+ | Category | Tools |
531
+ |----------|-------|
532
+ | **Package Management** | pixi, uv |
533
+ | **CLI Framework** | Typer, Rich |
534
+ | **Local LLM** | Docker Model Runner, Ollama |
535
+ | **LLM Framework** | LangChain |
536
+ | **Multi-Agent** | CrewAI |
537
+ | **MCP** | Docker MCP Toolkit |
538
+ | **PDF Processing** | PyPDF2, pdfplumber, pypdf |
539
+ | **Embeddings** | sentence-transformers |
540
+ | **File Operations** | pathlib, shutil |
541
+ | **Hashing** | hashlib, xxhash |
542
+ | **Metadata** | exifread, mutagen |
543
+ | **Similarity** | scikit-learn, faiss |
544
+ | **Observability** | Langfuse, OpenTelemetry |
545
+ | **Testing** | pytest, DeepEval |
546
+ | **Containerization** | Docker, Docker Compose |
547
+
548
+ ---
549
+
550
+ ## Example Usage
551
+
552
+ ### End-to-End Workflow
553
+
554
+ ```bash
555
+ # 1. Start the Docker stack
556
+ docker compose up -d
557
+
558
+ # 2. Scan your messy Downloads folder
559
+ fileorg scan ~/Downloads --analyze-content --export downloads_inventory.json
560
+
561
+ # 3. Organize files by type and date
562
+ fileorg organize ~/Downloads --strategy=smart --dry-run
563
+ # Review the plan, then execute
564
+ fileorg organize ~/Downloads --strategy=smart
565
+
566
+ # 4. Organize research papers by topic
567
+ fileorg scan ~/Papers --types=pdf --analyze-content
568
+ fileorg organize ~/Papers --strategy=by-topic --rename --pattern="{year}_{author}_{title}"
569
+
570
+ # 5. Find and handle duplicates
571
+ fileorg deduplicate ~/Papers --similarity=0.95 --move-to=./duplicates
572
+
573
+ # 6. Extract metadata and generate bibliography
574
+ fileorg research extract ~/Papers
575
+ fileorg research bibliography ~/Papers --format=bibtex --output=references.bib
576
+
577
+ # 7. Create a reading list on a specific topic
578
+ fileorg research reading-list ~/Papers --topic "transformers" --order=citations
579
+
580
+ # 8. View statistics
581
+ fileorg stats ~/Papers
582
+ ```
583
+
584
+ ### Research Paper Organization Example
585
+
586
+ ```bash
587
+ # Before:
588
+ ~/Papers/
589
+ ├── paper_final.pdf
590
+ ├── attention_is_all_you_need.pdf
591
+ ├── bert_paper.pdf
592
+ ├── gpt3.pdf
593
+ ├── vision_transformer.pdf
594
+ ├── download (1).pdf
595
+ ├── download (2).pdf
596
+ └── thesis_draft_v5.pdf
597
+
598
+ # Run organization
599
+ fileorg organize ~/Papers --strategy=by-topic --rename
600
+
601
+ # After:
602
+ ~/Papers/
603
+ ├── Natural_Language_Processing/
604
+ │ ├── Transformers/
605
+ │ │ ├── 2017_Vaswani_Attention_Is_All_You_Need.pdf
606
+ │ │ ├── 2018_Devlin_BERT_Pretraining.pdf
607
+ │ │ └── 2020_Brown_GPT3_Language_Models.pdf
608
+ │ └── Other/
609
+ │ └── 2023_Smith_Thesis_Draft.pdf
610
+ ├── Computer_Vision/
611
+ │ └── Transformers/
612
+ │ └── 2020_Dosovitskiy_Vision_Transformer.pdf
613
+ └── Uncategorized/
614
+ └── 2024_Unknown_Document.pdf
615
+ ```
616
+
617
+ ### Duplicate Detection Example
618
+
619
+ ```bash
620
+ # Find exact duplicates
621
+ fileorg deduplicate ~/Downloads
622
+ # Found 15 duplicate files (45 MB)
623
+ # • download.pdf (3 copies)
624
+ # • image.jpg (2 copies)
625
+ # • report.docx (2 copies)
626
+
627
+ # Find similar papers (different versions)
628
+ fileorg deduplicate ~/Papers --similarity=0.9 --method=content
629
+ # Found 3 similar file groups:
630
+ # • attention_paper.pdf, attention_is_all_you_need.pdf (95% similar)
631
+ # • bert_preprint.pdf, bert_final.pdf (98% similar)
632
+
633
+ # Auto-cleanup (keep newest)
634
+ fileorg deduplicate ~/Downloads --auto-delete --keep=newest
635
+ # ✓ Deleted 15 duplicate files, freed 45 MB
636
+ ```
637
+
638
+ ---
639
+
640
+ ## Learning Outcomes
641
+
642
+ By building FileOrganizer, learners will be able to:
643
+
644
+ 1. ✅ Set up modern Python projects with pixi and reproducible environments
645
+ 2. ✅ Build professional CLI tools with Typer and Rich
646
+ 3. ✅ Run local LLMs using Docker Model Runner
647
+ 4. ✅ Process and extract content from PDF files
648
+ 5. ✅ Build MCP servers to connect AI agents to file systems
649
+ 6. ✅ Design multi-agent systems with CrewAI
650
+ 7. ✅ Implement content-based similarity and clustering
651
+ 8. ✅ Generate embeddings for semantic search
652
+ 9. ✅ Handle file operations safely with backups and dry-run modes
653
+ 10. ✅ Implement observability for AI applications
654
+ 11. ✅ Test non-deterministic systems effectively
655
+ 12. ✅ Deploy self-hosted AI applications with Docker Compose
656
+
657
+ ---
658
+
659
+ ## Advanced Features
660
+
661
+ ### Smart Organization Strategy
662
+
663
+ The `smart` strategy uses AI to analyze file content and context to determine the best organization approach:
664
+
665
+ ```python
666
+ # Pseudocode for smart strategy
667
+ def smart_organize(files):
668
+ # 1. Analyze file types and content
669
+ file_analysis = scanner_agent.analyze(files)
670
+
671
+ # 2. Determine optimal strategy
672
+ if mostly_pdfs_with_academic_content:
673
+ strategy = "by-topic-hierarchical"
674
+ elif mostly_media_files:
675
+ strategy = "by-date-and-type"
676
+ elif mixed_work_documents:
677
+ strategy = "by-project-and-date"
678
+
679
+ # 3. Execute with AI-powered categorization
680
+ classifier_agent.categorize(files, strategy)
681
+ organizer_agent.execute(strategy)
682
+ ```
683
+
684
+ ### Research Paper Features
685
+
686
+ Special handling for academic PDFs:
687
+
688
+ - **Metadata Extraction**: Title, authors, year, abstract, keywords
689
+ - **Citation Parsing**: Extract and parse references
690
+ - **Smart Naming**: `{year}_{first_author}_{short_title}.pdf`
691
+ - **Topic Clustering**: Group papers by research area
692
+ - **Citation Network**: Identify related papers
693
+ - **Bibliography Generation**: BibTeX, APA, MLA formats
694
+
695
+ ### Deduplication Strategies
696
+
697
+ Multiple methods for finding duplicates:
698
+
699
+ 1. **Hash-based**: Exact file matches (fastest)
700
+ 2. **Content-based**: Similar content using embeddings
701
+ 3. **Metadata-based**: Same title/author but different files
702
+ 4. **Fuzzy matching**: Handle renamed or modified files
703
+
704
+ ---
705
+
706
+ *This project serves as the main example in the [Learning Path](../learning-path.md) for building AI-powered CLI tools.*
docs/projects/README.md ADDED
@@ -0,0 +1,350 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Python Package Ideas: CLI Tools for AI & Daily Operations
2
+
3
+ A curated list of Python package ideas that combine CLI interfaces with AI capabilities for day-to-day productivity and agent automation.
4
+
5
+ ## 📊 Data & Analytics Tools
6
+
7
+ ### 1. **smart-csv**
8
+ - **Purpose**: Intelligent CSV manipulation with natural language queries
9
+ - **CLI Features**:
10
+ - `smart-csv query data.csv "show me top 10 customers by revenue"`
11
+ - `smart-csv transform data.csv --operation "normalize dates"`
12
+ - `smart-csv merge file1.csv file2.csv --ai-match-columns`
13
+ - **AI Integration**: LLM-powered column matching, data cleaning suggestions, anomaly detection
14
+ - **Agent Use**: Data preprocessing, ETL operations, report generation
15
+
16
+ ### 2. **log-insight**
17
+ - **Purpose**: AI-powered log analysis and troubleshooting
18
+ - **CLI Features**:
19
+ - `log-insight analyze app.log --find-errors`
20
+ - `log-insight pattern-detect --timerange "last 24h"`
21
+ - `log-insight explain-error "stack trace here"`
22
+ - **AI Integration**: Pattern recognition, root cause analysis, predictive alerts
23
+ - **Agent Use**: Automated debugging, system monitoring, incident response
24
+
25
+ ### 3. **data-profiler-ai**
26
+ - **Purpose**: Automated data quality assessment and profiling
27
+ - **CLI Features**:
28
+ - `data-profiler scan dataset.parquet --generate-report`
29
+ - `data-profiler validate schema.json data.csv`
30
+ - `data-profiler suggest-types unknown-data.csv`
31
+ - **AI Integration**: Schema inference, data type detection, quality scoring
32
+ - **Agent Use**: Data validation pipelines, schema evolution tracking
33
+
34
+ ## 🤖 AI Agent Utilities
35
+
36
+ ### 4. **prompt-forge**
37
+ - **Purpose**: Prompt template management and optimization
38
+ - **CLI Features**:
39
+ - `prompt-forge create --template customer-support`
40
+ - `prompt-forge test prompt.txt --model gpt-4 --iterations 10`
41
+ - `prompt-forge optimize --goal "reduce tokens by 20%"`
42
+ - **AI Integration**: Prompt optimization, A/B testing, version control
43
+ - **Agent Use**: Dynamic prompt generation, context management
44
+
45
+ ### 5. **agent-memory**
46
+ - **Purpose**: Persistent memory and context management for AI agents
47
+ - **CLI Features**:
48
+ - `agent-memory store --key "user_preferences" --value "..."`
49
+ - `agent-memory recall --query "what did user say about deadlines?"`
50
+ - `agent-memory summarize --session-id abc123`
51
+ - **AI Integration**: Semantic search, context compression, memory consolidation
52
+ - **Agent Use**: Long-term memory, conversation history, knowledge graphs
53
+
54
+ ### 6. **tool-registry**
55
+ - **Purpose**: Discover, validate, and manage tools for AI agents
56
+ - **CLI Features**:
57
+ - `tool-registry search "weather API"`
58
+ - `tool-registry validate my-tool.json`
59
+ - `tool-registry generate-schema --from-function get_weather`
60
+ - **AI Integration**: Tool recommendation, capability matching, auto-documentation
61
+ - **Agent Use**: Dynamic tool discovery, function calling, API integration
62
+
63
+ ## 📝 Document & Content Processing
64
+
65
+ ### 7. **doc-extract-ai**
66
+ - **Purpose**: Intelligent document parsing and information extraction
67
+ - **CLI Features**:
68
+ - `doc-extract parse invoice.pdf --extract "vendor, amount, date"`
69
+ - `doc-extract batch-process ./documents --type receipts`
70
+ - `doc-extract to-markdown presentation.pptx`
71
+ - **AI Integration**: Layout understanding, entity extraction, format conversion
72
+ - **Agent Use**: Document automation, data entry, content migration
73
+
74
+ ### 8. **smart-summarize**
75
+ - **Purpose**: Multi-format content summarization
76
+ - **CLI Features**:
77
+ - `smart-summarize article.txt --length 100`
78
+ - `smart-summarize meeting-notes.md --extract-action-items`
79
+ - `smart-summarize youtube-url --transcript-summary`
80
+ - **AI Integration**: Abstractive summarization, key point extraction, multi-document synthesis
81
+ - **Agent Use**: Report generation, meeting notes, research assistance
82
+
83
+ ### 9. **content-repurpose**
84
+ - **Purpose**: Transform content across formats and styles
85
+ - **CLI Features**:
86
+ - `content-repurpose blog-post.md --to twitter-thread`
87
+ - `content-repurpose presentation.pptx --to blog-post`
88
+ - `content-repurpose --style "technical to beginner-friendly"`
89
+ - **AI Integration**: Style transfer, format adaptation, audience targeting
90
+ - **Agent Use**: Content marketing, documentation, social media automation
91
+
92
+ ## 🔍 Code & Development Tools
93
+
94
+ ### 10. **code-review-ai**
95
+ - **Purpose**: Automated code review and improvement suggestions
96
+ - **CLI Features**:
97
+ - `code-review analyze src/ --focus security`
98
+ - `code-review suggest-refactor messy-function.py`
99
+ - `code-review explain --file complex.py --line 42`
100
+ - **AI Integration**: Code understanding, best practice detection, vulnerability scanning
101
+ - **Agent Use**: CI/CD integration, code quality gates, learning assistant
102
+
103
+ ### 11. **test-gen-ai**
104
+ - **Purpose**: Intelligent test case generation
105
+ - **CLI Features**:
106
+ - `test-gen create --file calculator.py --coverage 90`
107
+ - `test-gen edge-cases --function parse_date`
108
+ - `test-gen from-spec requirements.md`
109
+ - **AI Integration**: Code analysis, edge case discovery, test oracle generation
110
+ - **Agent Use**: Test automation, TDD assistance, regression testing
111
+
112
+ ### 12. **doc-string-ai**
113
+ - **Purpose**: Automated documentation generation
114
+ - **CLI Features**:
115
+ - `doc-string generate src/ --style google`
116
+ - `doc-string update --file api.py --sync-with-code`
117
+ - `doc-string readme --project-root .`
118
+ - **AI Integration**: Code comprehension, example generation, API documentation
119
+ - **Agent Use**: Documentation maintenance, onboarding, API reference
120
+
121
+ ## 🌐 Web & API Tools
122
+
123
+ ### 13. **api-explorer-ai**
124
+ - **Purpose**: Intelligent API testing and exploration
125
+ - **CLI Features**:
126
+ - `api-explorer discover https://api.example.com`
127
+ - `api-explorer test --endpoint /users --generate-scenarios`
128
+ - `api-explorer mock --from-openapi spec.yaml`
129
+ - **AI Integration**: Endpoint discovery, test case generation, response validation
130
+ - **Agent Use**: API integration, testing automation, mock server generation
131
+
132
+ ### 14. **web-scrape-smart**
133
+ - **Purpose**: AI-powered web scraping with anti-detection
134
+ - **CLI Features**:
135
+ - `web-scrape extract https://example.com --schema product-listing`
136
+ - `web-scrape monitor --url news-site.com --alert-on-change`
137
+ - `web-scrape batch urls.txt --parallel 5`
138
+ - **AI Integration**: Layout understanding, content extraction, CAPTCHA solving
139
+ - **Agent Use**: Data collection, price monitoring, content aggregation
140
+
141
+ ## 📧 Communication & Workflow
142
+
143
+ ### 15. **email-assistant-cli**
144
+ - **Purpose**: Email management and automation
145
+ - **CLI Features**:
146
+ - `email-assistant draft --to client@example.com --context "project update"`
147
+ - `email-assistant triage inbox --auto-label --priority-score`
148
+ - `email-assistant respond --template "meeting-request"`
149
+ - **AI Integration**: Email classification, response generation, sentiment analysis
150
+ - **Agent Use**: Email automation, customer support, scheduling
151
+
152
+ ### 16. **meeting-prep-ai**
153
+ - **Purpose**: Automated meeting preparation and follow-up
154
+ - **CLI Features**:
155
+ - `meeting-prep agenda --topic "Q4 planning" --attendees team.json`
156
+ - `meeting-prep transcribe recording.mp3 --extract-decisions`
157
+ - `meeting-prep follow-up --assign-tasks`
158
+ - **AI Integration**: Agenda generation, transcription, action item extraction
159
+ - **Agent Use**: Meeting automation, task management, documentation
160
+
161
+ ### 17. **slack-digest**
162
+ - **Purpose**: Intelligent Slack/Teams message summarization
163
+ - **CLI Features**:
164
+ - `slack-digest summarize --channel engineering --since yesterday`
165
+ - `slack-digest extract-decisions --thread-url "..."`
166
+ - `slack-digest notify --important-only`
167
+ - **AI Integration**: Message clustering, importance scoring, thread summarization
168
+ - **Agent Use**: Team communication, information retrieval, notification management
169
+
170
+ ## 🔧 System & DevOps Tools
171
+
172
+ ### 18. **config-validator-ai**
173
+ - **Purpose**: Intelligent configuration validation and optimization
174
+ - **CLI Features**:
175
+ - `config-validator check docker-compose.yml --suggest-improvements`
176
+ - `config-validator migrate --from v1 --to v2`
177
+ - `config-validator security-scan kubernetes/`
178
+ - **AI Integration**: Best practice detection, security analysis, migration assistance
179
+ - **Agent Use**: Infrastructure as code, deployment automation, security compliance
180
+
181
+ ### 19. **error-translator**
182
+ - **Purpose**: Translate technical errors to actionable solutions
183
+ - **CLI Features**:
184
+ - `error-translator explain "segmentation fault core dumped"`
185
+ - `error-translator fix --error-log build.log --suggest-commands`
186
+ - `error-translator learn-from-fix --before error.txt --after solution.txt`
187
+ - **AI Integration**: Error pattern matching, solution database, context-aware suggestions
188
+ - **Agent Use**: Debugging assistance, self-healing systems, knowledge base
189
+
190
+ ### 20. **dependency-doctor**
191
+ - **Purpose**: Smart dependency management and conflict resolution
192
+ - **CLI Features**:
193
+ - `dependency-doctor audit --fix-vulnerabilities`
194
+ - `dependency-doctor optimize --reduce-size`
195
+ - `dependency-doctor explain --package requests --why-needed`
196
+ - **AI Integration**: Dependency graph analysis, alternative suggestions, impact prediction
197
+ - **Agent Use**: Security patching, build optimization, dependency updates
198
+
199
+ ## 🎨 Creative & Media Tools
200
+
201
+ ### 21. **image-caption-cli**
202
+ - **Purpose**: Automated image analysis and captioning
203
+ - **CLI Features**:
204
+ - `image-caption generate photo.jpg --style descriptive`
205
+ - `image-caption batch-process ./photos --alt-text`
206
+ - `image-caption search ./images --query "sunset beach"`
207
+ - **AI Integration**: Vision models, semantic search, accessibility text generation
208
+ - **Agent Use**: Content management, SEO optimization, accessibility compliance
209
+
210
+ ### 22. **video-chapter-ai**
211
+ - **Purpose**: Automatic video chapter generation and editing
212
+ - **CLI Features**:
213
+ - `video-chapter detect video.mp4 --create-chapters`
214
+ - `video-chapter highlight --topic "product demo"`
215
+ - `video-chapter transcribe --with-timestamps`
216
+ - **AI Integration**: Scene detection, topic segmentation, speech recognition
217
+ - **Agent Use**: Video processing, content creation, accessibility
218
+
219
+ ## 🔐 Security & Privacy Tools
220
+
221
+ ### 23. **secret-scanner-ai**
222
+ - **Purpose**: Intelligent secret detection and rotation
223
+ - **CLI Features**:
224
+ - `secret-scanner scan . --deep`
225
+ - `secret-scanner rotate --service aws --auto-update-configs`
226
+ - `secret-scanner audit-history --find-leaks`
227
+ - **AI Integration**: Pattern learning, false positive reduction, context-aware detection
228
+ - **Agent Use**: Security automation, compliance, incident response
229
+
230
+ ### 24. **privacy-guard-cli**
231
+ - **Purpose**: PII detection and data anonymization
232
+ - **CLI Features**:
233
+ - `privacy-guard scan dataset.csv --detect-pii`
234
+ - `privacy-guard anonymize --method k-anonymity`
235
+ - `privacy-guard compliance-check --regulation GDPR`
236
+ - **AI Integration**: Entity recognition, anonymization strategies, compliance validation
237
+ - **Agent Use**: Data processing, compliance automation, privacy engineering
238
+
239
+ ## 📚 Knowledge & Research Tools
240
+
241
+ ### 25. **paper-digest**
242
+ - **Purpose**: Academic paper summarization and analysis
243
+ - **CLI Features**:
244
+ - `paper-digest summarize paper.pdf --technical-level intermediate`
245
+ - `paper-digest compare paper1.pdf paper2.pdf`
246
+ - `paper-digest extract-methods --output markdown`
247
+ - **AI Integration**: Scientific text understanding, citation analysis, methodology extraction
248
+ - **Agent Use**: Research assistance, literature review, knowledge synthesis
249
+
250
+ ### 26. **kb-builder**
251
+ - **Purpose**: Automated knowledge base construction
252
+ - **CLI Features**:
253
+ - `kb-builder index ./docs --create-embeddings`
254
+ - `kb-builder query "how to deploy?" --with-sources`
255
+ - `kb-builder update --incremental --watch`
256
+ - **AI Integration**: Semantic search, document chunking, question answering
257
+ - **Agent Use**: Documentation search, customer support, internal wiki
258
+
259
+ ### 27. **citation-manager-ai**
260
+ - **Purpose**: Smart citation management and bibliography generation
261
+ - **CLI Features**:
262
+ - `citation-manager extract paper.pdf --format bibtex`
263
+ - `citation-manager validate references.bib`
264
+ - `citation-manager suggest-related --topic "machine learning"`
265
+ - **AI Integration**: Citation extraction, duplicate detection, related work discovery
266
+ - **Agent Use**: Academic writing, research management, bibliography maintenance
267
+
268
+ ## 🎯 Productivity & Automation
269
+
270
+ ### 28. **task-prioritizer**
271
+ - **Purpose**: AI-powered task management and prioritization
272
+ - **CLI Features**:
273
+ - `task-prioritizer add "implement feature X" --context project.md`
274
+ - `task-prioritizer suggest-next --based-on energy-level:low`
275
+ - `task-prioritizer estimate --task "write tests"`
276
+ - **AI Integration**: Priority scoring, time estimation, dependency detection
277
+ - **Agent Use**: Project management, personal productivity, resource allocation
278
+
279
+ ### 29. **workflow-optimizer**
280
+ - **Purpose**: Analyze and optimize repetitive workflows
281
+ - **CLI Features**:
282
+ - `workflow-optimizer record --name "daily-standup-prep"`
283
+ - `workflow-optimizer analyze --suggest-automation`
284
+ - `workflow-optimizer generate-script --workflow deployment`
285
+ - **AI Integration**: Pattern detection, automation suggestions, efficiency analysis
286
+ - **Agent Use**: Process automation, productivity improvement, workflow design
287
+
288
+ ### 30. **context-switch-helper**
289
+ - **Purpose**: Manage context switching with AI assistance
290
+ - **CLI Features**:
291
+ - `context-switch save --project backend-api`
292
+ - `context-switch restore --project frontend`
293
+ - `context-switch summarize --what-changed-since-last`
294
+ - **AI Integration**: State capture, change summarization, context reconstruction
295
+ - **Agent Use**: Developer productivity, project management, focus optimization
296
+
297
+ ---
298
+
299
+ ## 🏗️ Implementation Considerations
300
+
301
+ ### Common Features Across Tools:
302
+ - **Streaming Output**: Real-time progress for long-running operations
303
+ - **Configuration Files**: YAML/TOML config for defaults and presets
304
+ - **Plugin Architecture**: Extensible with custom processors/models
305
+ - **Multi-Model Support**: OpenAI, Anthropic, local models (Ollama)
306
+ - **Caching**: Intelligent caching to reduce API costs
307
+ - **Batch Processing**: Handle multiple files/inputs efficiently
308
+ - **Output Formats**: JSON, Markdown, CSV, HTML for different use cases
309
+ - **Logging & Telemetry**: Track usage, errors, and performance
310
+ - **Offline Mode**: Local model support for privacy/offline use
311
+
312
+ ### Agent-Friendly Design Patterns:
313
+ 1. **Structured Output**: JSON schemas for reliable parsing
314
+ 2. **Idempotency**: Safe to retry operations
315
+ 3. **Progress Tracking**: Machine-readable status updates
316
+ 4. **Error Codes**: Standardized exit codes and error messages
317
+ 5. **Composability**: Unix philosophy - do one thing well, pipe-friendly
318
+ 6. **API Mode**: HTTP server mode for agent integration
319
+ 7. **Webhooks**: Event-driven notifications for async operations
320
+
321
+ ### Technology Stack Suggestions:
322
+ - **CLI Framework**: Click, Typer, or argparse
323
+ - **AI Integration**: LangChain, LlamaIndex, or direct API calls
324
+ - **Async Operations**: asyncio, aiohttp for concurrent processing
325
+ - **Configuration**: Pydantic for validation, dynaconf for management
326
+ - **Testing**: pytest with fixtures for AI mocking
327
+ - **Packaging**: Poetry or uv for dependency management
328
+ - **Distribution**: PyPI, with optional Docker images
329
+
330
+ ---
331
+
332
+ ## 📈 Market Opportunities
333
+
334
+ ### High-Impact Categories:
335
+ 1. **Developer Tools** (code-review-ai, test-gen-ai, doc-string-ai)
336
+ 2. **Data Processing** (smart-csv, data-profiler-ai, doc-extract-ai)
337
+ 3. **Content Creation** (smart-summarize, content-repurpose)
338
+ 4. **Security** (secret-scanner-ai, privacy-guard-cli)
339
+ 5. **Productivity** (task-prioritizer, email-assistant-cli)
340
+
341
+ ### Differentiation Strategies:
342
+ - **Local-First**: Privacy-focused with local model support
343
+ - **Cost-Optimized**: Intelligent caching and batching to reduce API costs
344
+ - **Domain-Specific**: Deep expertise in particular verticals
345
+ - **Integration-Rich**: Works with existing tools (Git, Jira, Slack)
346
+ - **Open Source**: Community-driven with premium features
347
+
348
+ ---
349
+
350
+ *Generated: 2025-12-04*
pixi.toml ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [project]
2
+ name = "kashi-coding-handbook"
3
+ version = "0.1.0"
4
+ description = "Kashi Coding Handbook - Building AI-Powered CLI Tools with Python"
5
+ authors = ["Kashi School of Computing"]
6
+ channels = ["conda-forge"]
7
+ platforms = ["linux-64", "osx-64", "osx-arm64", "win-64"]
8
+
9
+ [dependencies]
10
+ python = ">=3.11"
11
+ quarto = ">=1.8.26,<2"
12
+
13
+ [tasks]
14
+ install-quarto-extensions = "cd src; quarto install extension grantmcdermott/quarto-revealjs-clean; quarto install extension pandoc-ext/diagram"
15
+ install-tinytex = "quarto install tinytex"
16
+ preview = "quarto preview src"
17
+ render = "quarto render src"
18
+ serve = "python serve.py"
19
+
20
+ [feature.dev.dependencies]
21
+ black = ">=25.1.0,<26"
22
+ ruff = ">=0.14.8,<0.15"
23
+ mypy = ">=1.19.0,<2"
24
+
25
+ [environments]
26
+ default = []
27
+ dev = ["dev"]
requirements.txt DELETED
@@ -1,3 +0,0 @@
1
- pandas
2
- seaborn
3
- jupyter
 
 
 
 
serve.py CHANGED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """Simple HTTP server for serving the rendered Quarto site."""
3
+
4
+ import http.server
5
+ import socketserver
6
+ import os
7
+
8
+ PORT = 7860
9
+ DIRECTORY = "src/_site"
10
+
11
+ class Handler(http.server.SimpleHTTPRequestHandler):
12
+ def __init__(self, *args, **kwargs):
13
+ super().__init__(*args, directory=DIRECTORY, **kwargs)
14
+
15
+ if __name__ == "__main__":
16
+ os.chdir(os.path.dirname(os.path.abspath(__file__)))
17
+
18
+ with socketserver.TCPServer(("", PORT), Handler) as httpd:
19
+ print(f"Serving {DIRECTORY} at http://0.0.0.0:{PORT}")
20
+ httpd.serve_forever()
src/.gitignore CHANGED
@@ -1 +1,3 @@
1
  /.quarto/
 
 
 
1
  /.quarto/
2
+
3
+ **/*.quarto_ipynb
src/_extensions/grantmcdermott/clean/_extension.yml ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ title: clean
2
+ author: Grant McDermott
3
+ version: 1.4.1
4
+ quarto-required: ">=1.3.0"
5
+ contributes:
6
+ formats:
7
+ revealjs:
8
+ theme: [default, clean.scss]
9
+ menu:
10
+ side: left
11
+ slide-number: true
12
+ date-format: long
13
+ html-math-method:
14
+ method: mathjax
15
+ url: "https://cdn.jsdelivr.net/npm/mathjax@4/tex-mml-chtml.js"
16
+ include-before-body:
17
+ - text: |
18
+ <script src="mathjax-config.js"></script>
19
+ format-resources:
20
+ - mathjax-config.js
src/_extensions/grantmcdermott/clean/clean.scss ADDED
@@ -0,0 +1,351 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /*-- scss:defaults --*/
2
+
3
+ // Custom colours and variables
4
+
5
+ $jet: #131516;
6
+ $accent: #107895;
7
+ $accent2: #9a2515;
8
+ // $accent2: #e64173;
9
+ $right-arrow: "\2192"; // Unicode character for right arrow
10
+
11
+ // fonts
12
+
13
+ /*
14
+ Note: This theme uses the Roboto font family, which it imports from Google
15
+ Fonts to ensure consistent weighting in addition to availability. While
16
+ you can use a local installation of Roboto, this is generally not
17
+ recommended since the weighting will likely be wrong (probably too
18
+ light). OTOH, importing from Google Fonts can cause some issues in
19
+ certain secure environments due the external CDN (see:
20
+ https://github.com/grantmcdermott/quarto-revealjs-clean/issues/7). If
21
+ that's the case for you, simply comment out the `@import url(...)` line
22
+ below and it will default for the default Sans Serif font on your system
23
+ (e.g., Helvetica on a Mac). Circling back to the earlier point about
24
+ preserving consistent font weights, you may also wish to remove "Roboto"
25
+ from the choice set if the family is installed locally.
26
+ */
27
+ @import url('https://fonts.googleapis.com/css?family=Roboto:200,200i,300,300i,350,350i,400,400i&display=swap');
28
+
29
+ $font-family-sans-serif: "Roboto", sans-serif !default;
30
+ $presentation-heading-font: "Roboto", sans-serif !default;
31
+
32
+ $presentation-heading-color: $jet !default;
33
+ $presentation-heading-font-weight: lighter;
34
+ //$presentation-heading-line-height: 2;
35
+ //$presentation-block-margin: 28px;
36
+ $presentation-font-size-root: 32px;
37
+
38
+ // colors
39
+ //$body-bg: #f0f1eb !default;
40
+ $body-color: $jet !default;
41
+ $link-color: $accent !default;
42
+ $selection-bg: #26351c !default;
43
+
44
+
45
+ /*-- scss:rules --*/
46
+
47
+ .reveal a {
48
+ line-height: 1.5em;
49
+ }
50
+
51
+ .reveal p {
52
+ // font-weight: 300;
53
+ font-weight: lighter;
54
+ margin-top: 1.25em;
55
+ }
56
+
57
+ // title and headings
58
+
59
+ #title-slide {
60
+ text-align: left;
61
+
62
+ .title {
63
+ color: $body-color;
64
+ font-size: 1.4em;
65
+ // font-weight: 350;
66
+ font-weight: lighter;
67
+ }
68
+
69
+ .subtitle {
70
+ color: $accent;
71
+ font-style: italic;
72
+ margin-top: 0em;
73
+ font-weight: lighter;
74
+ }
75
+
76
+ .institute,
77
+ .quarto-title-affiliation,
78
+ .quarto-title-author-email {
79
+ font-style: italic;
80
+ // font-size: 80%;
81
+ // color: #7F7F7F;
82
+ }
83
+
84
+ .author,
85
+ .quarto-title-author-name {
86
+ color: $body-color;
87
+ }
88
+
89
+ .quarto-title-authors {
90
+ display: flex;
91
+ justify-content: left;
92
+
93
+ .quarto-title-author {
94
+ padding-left: 0em;
95
+ padding-right: 0em;
96
+ width: 100%;
97
+ }
98
+ }
99
+
100
+ }
101
+
102
+
103
+ .reveal h2 {
104
+ // font-weight: 350;
105
+ font-weight: lighter;
106
+ font-size: 1.4em;
107
+ }
108
+
109
+ .reveal h3 {
110
+ color: $accent;
111
+ font-style: italic;
112
+ // font-weight: 350;
113
+ font-weight: lighter;
114
+ font-size: 0.95em;
115
+ }
116
+
117
+ .reveal h4 {
118
+ color: $accent2;
119
+ // font-weight: 350;
120
+ font-weight: normal;
121
+ margin-top: 1.25em;
122
+ }
123
+
124
+ // alerts etc.
125
+
126
+ .alert {
127
+ color: $accent2;
128
+ }
129
+
130
+ .fg {
131
+ color: var(--col, $jet);
132
+ }
133
+
134
+ .bg {
135
+ background-color: var(--col, #fff);
136
+ padding: 0.1em;
137
+ border-radius: 5px;
138
+ display: inline-block;
139
+ }
140
+
141
+ // lists
142
+
143
+ // Unordered lists
144
+
145
+ .reveal ul {
146
+ // font-weight: 300;
147
+ font-weight: lighter;
148
+ padding-left: 16px;
149
+
150
+ li::marker {
151
+ color: mix($accent, white, 70%);
152
+ }
153
+ }
154
+
155
+ .reveal ul ul {
156
+ list-style: none;
157
+
158
+ li:before {
159
+ content: $right-arrow;
160
+ color: mix($accent, white, 60%);
161
+ display: inline-block;
162
+ width: 1em;
163
+ margin-left: -1em;
164
+ margin-right: 0.5em;
165
+ }
166
+ }
167
+
168
+ // Ordered lists
169
+
170
+ .reveal ol {
171
+ // font-weight: 300;
172
+ font-weight: lighter;
173
+ padding-left: 16px;
174
+
175
+ li::marker {
176
+ color: $accent;
177
+ }
178
+ }
179
+
180
+ // Move "hamburger" menu button to top right
181
+
182
+ .reveal .slide-menu-button {
183
+ position: fixed;
184
+ top: 6px;
185
+ right: 0;
186
+ display: flex;
187
+ justify-content: flex-end;
188
+ align-items: flex-start;
189
+ pointer-events: none;
190
+ }
191
+
192
+ .reveal .slide-menu-button > * {
193
+ pointer-events: auto;
194
+ }
195
+
196
+ // Same for chalkboard buttons (with an offset)
197
+
198
+ .reveal .slide-chalkboard-buttons {
199
+ position: fixed;
200
+ top: 12px;
201
+ right: 24px;
202
+ display: flex;
203
+ justify-content: flex-end;
204
+ align-items: flex-start;
205
+ pointer-events: none;
206
+ }
207
+
208
+ .reveal .slide-chalkboard-buttons > * {
209
+ pointer-events: auto;
210
+ }
211
+
212
+ // Logo to the bottom-left
213
+ .slide-logo {
214
+ display: block !important;
215
+ position: fixed !important;
216
+ bottom: 0 !important;
217
+ left: 10px !important;
218
+ max-width: 150px; // Adjust if necessary
219
+ max-height: 50px;
220
+ width: auto !important;
221
+ color: $body-color !important;
222
+ }
223
+
224
+ // Also need to enforce slide numbers at bottom-right (if logo is present)
225
+ .slide-number, .reveal.has-logo .slide-number {
226
+ bottom: 6px !important;
227
+ right: 10px !important;
228
+ top: unset !important;
229
+ color: #777777 !important;
230
+ }
231
+
232
+ // Beamer-style button link environment
233
+
234
+ .button {
235
+ display: inline-block;
236
+ padding: 6px 12px;
237
+ margin-bottom: 0;
238
+ font-size: 14px;
239
+ font-weight: 400;
240
+ line-height: 1.42857143;
241
+ text-align: center;
242
+ white-space: nowrap;
243
+ vertical-align: middle;
244
+ cursor: pointer;
245
+ background-color: $accent;
246
+ border: 1px solid $accent;
247
+ color: #fff !important;
248
+ text-decoration: none;
249
+ border-radius: 4px;
250
+ transition: all 0.2s ease-in-out;
251
+ }
252
+
253
+ .button:hover {
254
+ background-color: #0056b3;
255
+ border-color: #0056b3;
256
+ }
257
+
258
+ .button::before {
259
+ content: "▶";
260
+ margin-right: 5px;
261
+ }
262
+
263
+ // tables
264
+
265
+ .reveal table {
266
+ // height: auto; /* Adjust table width to fit content up to the available slide space */
267
+ margin: auto;
268
+ border-collapse: collapse;
269
+ border-spacing: 0;
270
+ font-size: 0.8em;
271
+ }
272
+
273
+ .reveal table th,
274
+ .reveal table td {
275
+ border: none; /* Remove internal row lines */
276
+ padding: .23em; /* Adjust padding as needed */
277
+ text-align: left; /* Adjust text alignment as needed */
278
+ font-weight: lighter; /* Lighter font weight for main table text */
279
+ }
280
+
281
+ /* Adds a bottom border to the table header row for distinction */
282
+ .reveal table thead th,
283
+ .reveal .slides table tr:last-child td,
284
+ .reveal .slides table {
285
+ border-bottom: 2px solid #D3D3D3; /* Dark grey color for the bottom border */
286
+ }
287
+
288
+ /* Make column headers bold */
289
+ .reveal table thead th {
290
+ font-weight: bold;
291
+ }
292
+
293
+ /* Styling table captions */
294
+ .reveal table caption {
295
+ color: #666666; /* Dark grey color for the caption */
296
+ font-variant: small-caps; /* Use small caps for the caption text */
297
+ }
298
+
299
+ // Special catch for etable environment to ensure these table images
300
+ // don't overflow the slide.
301
+ // See: https://lrberge.github.io/fixest/articles/etable_new_features.html
302
+
303
+ .etable {
304
+ width: 100%;
305
+ height: calc(100% - 3em); /* Adjust 3em based on the height of your header, if necessary */
306
+ display: flex;
307
+ align-items: center;
308
+ justify-content: center;
309
+ }
310
+
311
+ .etable img {
312
+ max-width: 100%;
313
+ max-height: 100%;
314
+ width: auto;
315
+ height: auto;
316
+ object-fit: contain;
317
+ }
318
+
319
+ // Change the relative widths of `output-location: column`.
320
+ // See: https://github.com/grantmcdermott/quarto-revealjs-clean/pull/16
321
+ // Example usage:
322
+ // ```{python}
323
+ // #| echo: true
324
+ // #| output-location: column
325
+ // #| classes: columns3070
326
+ // <code>
327
+ // ```
328
+ .reveal .columns3070 > div.column:first-child {
329
+ width: 30%;
330
+ }
331
+ .reveal .columns3070 div.column:not(:first-child) {
332
+ width: 70%;
333
+ }
334
+ .reveal .columns7030 > div.column:first-child {
335
+ width: 70%;
336
+ }
337
+ .reveal .columns7030 div.column:not(:first-child) {
338
+ width: 30%;
339
+ }
340
+ .reveal .columns4060 > div.column:first-child {
341
+ width: 40%;
342
+ }
343
+ .reveal .columns4060 div.column:not(:first-child) {
344
+ width: 60%;
345
+ }
346
+ .reveal .columns6040 > div.column:first-child {
347
+ width: 60%;
348
+ }
349
+ .reveal .columns6040 div.column:not(:first-child) {
350
+ width: 40%;
351
+ }
src/_extensions/grantmcdermott/clean/mathjax-config.js ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ window.MathJax = {
2
+ tex: {
3
+ inlineMath: [['\\(', '\\)']],
4
+ displayMath: [['\\[', '\\]']],
5
+ processEscapes: true,
6
+ processRefs: true,
7
+ processEnvironments: true
8
+ },
9
+ chtml: {
10
+ font: 'mathjax-asana'
11
+ },
12
+ startup: {
13
+ ready: () => {
14
+ console.log('MathJax is loaded and ready with font: mathjax-asana (Asana Math)');
15
+ MathJax.startup.defaultReady();
16
+ }
17
+ }
18
+ };
src/_extensions/pandoc-ext/diagram/_extension.yaml ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ title: diagram
2
+ author: Albert Krewinkel
3
+ version: 1.2.0
4
+ quarto-required: ">=1.3"
5
+ contributes:
6
+ filters:
7
+ - diagram.lua
src/_extensions/pandoc-ext/diagram/diagram.lua ADDED
@@ -0,0 +1,660 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ --[[
2
+ diagram – create images and figures from code blocks.
3
+
4
+ See copyright notice in file LICENSE.
5
+ ]]
6
+ -- The filter uses the Figure AST element, which was added in pandoc 3.
7
+ PANDOC_VERSION:must_be_at_least '3.0'
8
+
9
+ local version = pandoc.types.Version '1.2.0'
10
+
11
+ -- Report Lua warnings to stderr if the `warn` function is not plugged into
12
+ -- pandoc's logging system.
13
+ if not warn then
14
+ -- fallback
15
+ warn = function(...) io.stderr:write(table.concat({ ... })) end
16
+ elseif PANDOC_VERSION < '3.1.4' then
17
+ -- starting with pandoc 3.1.4, warnings are reported to pandoc's logging
18
+ -- system, so no need to print warnings to stderr.
19
+ warn '@on'
20
+ end
21
+
22
+ local io = require 'io'
23
+ local pandoc = require 'pandoc'
24
+ local system = require 'pandoc.system'
25
+ local utils = require 'pandoc.utils'
26
+ local List = require 'pandoc.List'
27
+ local stringify = utils.stringify
28
+ local with_temporary_directory = system.with_temporary_directory
29
+ local with_working_directory = system.with_working_directory
30
+
31
+ --- Returns a filter-specific directory in which cache files can be
32
+ --- stored, or nil if no such directory is available.
33
+ local function cachedir ()
34
+ local cache_home = os.getenv 'XDG_CACHE_HOME'
35
+ if not cache_home or cache_home == '' then
36
+ local user_home = system.os == 'windows'
37
+ and os.getenv 'USERPROFILE'
38
+ or os.getenv 'HOME'
39
+
40
+ if not user_home or user_home == '' then
41
+ return nil
42
+ end
43
+ cache_home = pandoc.path.join{user_home, '.cache'} or nil
44
+ end
45
+
46
+ -- Create filter cache directory
47
+ return pandoc.path.join{cache_home, 'pandoc-diagram-filter'}
48
+ end
49
+
50
+ --- Path holding the image cache, or `nil` if the cache is not used.
51
+ local image_cache = nil
52
+
53
+ local mimetype_for_extension = {
54
+ jpeg = 'image/jpeg',
55
+ jpg = 'image/jpeg',
56
+ pdf = 'application/pdf',
57
+ png = 'image/png',
58
+ svg = 'image/svg+xml',
59
+ }
60
+
61
+ local extension_for_mimetype = {
62
+ ['application/pdf'] = 'pdf',
63
+ ['image/jpeg'] = 'jpg',
64
+ ['image/png'] = 'png',
65
+ ['image/svg+xml'] = 'svg',
66
+ }
67
+
68
+ --- Converts a list of format specifiers to a set of MIME types.
69
+ local function mime_types_set (tbl)
70
+ local set = {}
71
+ local mime_type
72
+ for _, image_format_spec in ipairs(tbl) do
73
+ mime_type = mimetype_for_extension[image_format_spec] or image_format_spec
74
+ set[mime_type] = true
75
+ end
76
+ return set
77
+ end
78
+
79
+ --- Reads the contents of a file.
80
+ local function read_file (filepath)
81
+ local fh = io.open(filepath, 'rb')
82
+ local contents = fh:read('a')
83
+ fh:close()
84
+ return contents
85
+ end
86
+
87
+ --- Writes the contents into a file at the given path.
88
+ local function write_file (filepath, content)
89
+ local fh = io.open(filepath, 'wb')
90
+ fh:write(content)
91
+ fh:close()
92
+ end
93
+
94
+ --- Like `pandoc.pipe`, but allows "multi word" paths:
95
+ -- Supplying a list as the first argument will use the first element as
96
+ -- the executable path and prepend the remaining elements to the list of
97
+ -- arguments.
98
+ local function pipe (command, args, input)
99
+ local cmd
100
+ if pandoc.utils.type(command) == 'List' then
101
+ command = command:map(stringify)
102
+ cmd = command:remove(1)
103
+ args = command .. args
104
+ else
105
+ cmd = stringify(command)
106
+ end
107
+ return pandoc.pipe(cmd, args, input)
108
+ end
109
+
110
+
111
+ --
112
+ -- Diagram Engines
113
+ --
114
+
115
+ -- PlantUML engine; assumes that there's a `plantuml` binary.
116
+ local plantuml = {
117
+ line_comment_start = [[']],
118
+ mime_types = mime_types_set{'pdf', 'png', 'svg'},
119
+ compile = function (self, puml)
120
+ local mime_type = self.mime_type or 'image/svg+xml'
121
+ -- PlantUML format identifiers correspond to common file extensions.
122
+ local format = extension_for_mimetype[mime_type]
123
+ if not format then
124
+ format, mime_type = 'svg', 'image/svg+xml'
125
+ end
126
+ local args = {'-t' .. format, "-pipe", "-charset", "UTF8"}
127
+ return pipe(self.execpath or 'plantuml', args, puml), mime_type
128
+ end,
129
+ }
130
+
131
+ --- GraphViz engine for the dot language
132
+ local graphviz = {
133
+ line_comment_start = '//',
134
+ mime_types = mime_types_set{'jpg', 'pdf', 'png', 'svg'},
135
+ mime_type = 'image/svg+xml',
136
+ compile = function (self, code)
137
+ local mime_type = self.mime_type
138
+ -- GraphViz format identifiers correspond to common file extensions.
139
+ local format = extension_for_mimetype[mime_type]
140
+ if not format then
141
+ format, mime_type = 'svg', 'image/svg+xml'
142
+ end
143
+ return pipe(self.execpath or 'dot', {"-T"..format}, code), mime_type
144
+ end,
145
+ }
146
+
147
+ --- Mermaid engine
148
+ local mermaid = {
149
+ line_comment_start = '%%',
150
+ mime_types = mime_types_set{'pdf', 'png', 'svg'},
151
+ compile = function (self, code)
152
+ local mime_type = self.mime_type or 'image/svg+xml'
153
+ local file_extension = extension_for_mimetype[mime_type]
154
+ return with_temporary_directory("diagram", function (tmpdir)
155
+ return with_working_directory(tmpdir, function ()
156
+ local infile = 'diagram.mmd'
157
+ local outfile = 'diagram.' .. file_extension
158
+ write_file(infile, code)
159
+ pipe(
160
+ self.execpath or 'mmdc',
161
+ {"--pdfFit", "--input", infile, "--output", outfile},
162
+ ''
163
+ )
164
+ return read_file(outfile), mime_type
165
+ end)
166
+ end)
167
+ end,
168
+ }
169
+
170
+ --- TikZ
171
+ --
172
+
173
+ --- LaTeX template used to compile TikZ images.
174
+ local tikz_template = pandoc.template.compile [[
175
+ \documentclass{standalone}
176
+ \usepackage{tikz}
177
+ $for(header-includes)$
178
+ $it$
179
+ $endfor$
180
+ $additional-packages$
181
+ \begin{document}
182
+ $body$
183
+ \end{document}
184
+ ]]
185
+
186
+ --- The TikZ engine uses pdflatex to compile TikZ code to an image
187
+ local tikz = {
188
+ line_comment_start = '%%',
189
+
190
+ mime_types = {
191
+ ['application/pdf'] = true,
192
+ },
193
+
194
+ --- Compile LaTeX with TikZ code to an image
195
+ compile = function (self, src, user_opts)
196
+ return with_temporary_directory("tikz", function (tmpdir)
197
+ return with_working_directory(tmpdir, function ()
198
+ -- Define file names:
199
+ local file_template = "%s/tikz-image.%s"
200
+ local tikz_file = file_template:format(tmpdir, "tex")
201
+ local pdf_file = file_template:format(tmpdir, "pdf")
202
+
203
+ -- Treat string values as raw LaTeX
204
+ local meta = {
205
+ ['header-includes'] = user_opts['header-includes'],
206
+ ['additional-packages'] = {pandoc.RawInline(
207
+ 'latex',
208
+ stringify(user_opts['additional-packages'] or '')
209
+ )},
210
+ }
211
+ local tex_code = pandoc.write(
212
+ pandoc.Pandoc({pandoc.RawBlock('latex', src)}, meta),
213
+ 'latex',
214
+ {template = tikz_template}
215
+ )
216
+ write_file(tikz_file, tex_code)
217
+
218
+ -- Execute the LaTeX compiler:
219
+ local success, result = pcall(
220
+ pipe,
221
+ self.execpath or 'pdflatex',
222
+ { '-interaction=nonstopmode', '-output-directory', tmpdir, tikz_file },
223
+ ''
224
+ )
225
+ if not success then
226
+ warn(string.format(
227
+ "The call\n%s\nfailed with error code %s. Output:\n%s",
228
+ result.command,
229
+ result.error_code,
230
+ result.output
231
+ ))
232
+ end
233
+ return read_file(pdf_file), 'application/pdf'
234
+ end)
235
+ end)
236
+ end
237
+ }
238
+
239
+ --- Asymptote diagram engine
240
+ local asymptote = {
241
+ line_comment_start = '%%',
242
+ mime_types = {
243
+ ['application/pdf'] = true,
244
+ },
245
+ compile = function (self, code)
246
+ return with_temporary_directory("asymptote", function(tmpdir)
247
+ return with_working_directory(tmpdir, function ()
248
+ local pdf_file = "pandoc_diagram.pdf"
249
+ local args = {'-tex', 'pdflatex', "-o", "pandoc_diagram", '-'}
250
+ pipe(self.execpath or 'asy', args, code)
251
+ return read_file(pdf_file), 'application/pdf'
252
+ end)
253
+ end)
254
+ end,
255
+ }
256
+
257
+ --- Cetz diagram engine
258
+ local cetz = {
259
+ line_comment_start = '%%',
260
+ mime_types = mime_types_set{'jpg', 'pdf', 'png', 'svg'},
261
+ mime_type = 'image/svg+xml',
262
+ compile = function (self, code)
263
+ local mime_type = self.mime_type
264
+ local format = extension_for_mimetype[mime_type]
265
+ if not format then
266
+ format, mime_type = 'svg', 'image/svg+xml'
267
+ end
268
+ local preamble = [[
269
+ #import "@preview/cetz:0.3.4"
270
+ #set page(width: auto, height: auto, margin: .5cm)
271
+ ]]
272
+
273
+ local typst_code = preamble .. code
274
+
275
+ return with_temporary_directory("diagram", function (tmpdir)
276
+ return with_working_directory(tmpdir, function ()
277
+ local outfile = 'diagram.' .. format
278
+ local execpath = self.execpath
279
+ if not execpath and quarto and quarto.version >= '1.4' then
280
+ -- fall back to the Typst exec shipped with Quarto.
281
+ execpath = List{'quarto', 'typst'}
282
+ end
283
+ pipe(
284
+ execpath or 'typst',
285
+ {"compile", "-f", format, "-", outfile},
286
+ typst_code
287
+ )
288
+ return read_file(outfile), mime_type
289
+ end)
290
+ end)
291
+ end,
292
+ }
293
+
294
+ --- D2 engine for the D2 language
295
+ local d2 = {
296
+ line_comment_start = '#',
297
+ mime_types = mime_types_set{'png', 'svg'},
298
+
299
+ compile = function (self, code, user_opts)
300
+ return with_temporary_directory('diagram', function (tmpdir)
301
+ return with_working_directory(tmpdir, function ()
302
+ -- D2 format identifiers correspond to common file extensions.
303
+ local mime_type = self.mime_type or 'image/svg+xml'
304
+ local file_extension = extension_for_mimetype[mime_type]
305
+ local infile = 'diagram.d2'
306
+ local outfile = 'diagram.' .. file_extension
307
+
308
+ args = {'--bundle', '--pad=0', '--scale=1'}
309
+
310
+ d2_user_opts = {
311
+ 'layout',
312
+ }
313
+ for _, d2_user_opt in pairs(d2_user_opts) do
314
+ if user_opts[d2_user_opt] then
315
+ table.insert(args, '--' .. d2_user_opt .. '=' .. user_opts[d2_user_opt])
316
+ end
317
+ end
318
+
319
+ table.insert(args, infile)
320
+ table.insert(args, outfile)
321
+
322
+ write_file(infile, code)
323
+
324
+ pipe(self.execpath or 'd2', args, '')
325
+
326
+ return read_file(outfile), mime_type
327
+ end)
328
+ end)
329
+ end,
330
+ }
331
+
332
+ local default_engines = {
333
+ asymptote = asymptote,
334
+ dot = graphviz,
335
+ mermaid = mermaid,
336
+ plantuml = plantuml,
337
+ tikz = tikz,
338
+ cetz = cetz,
339
+ d2 = d2,
340
+ }
341
+
342
+ --
343
+ -- Configuration
344
+ --
345
+
346
+ --- Options for the output format of the given name.
347
+ local function format_options (name)
348
+ local pdf2svg = name ~= 'latex' and name ~= 'context'
349
+ local is_office_format = name == 'docx' or name == 'odt'
350
+ -- Office formats seem to work better with PNG than with SVG.
351
+ local preferred_mime_types = is_office_format
352
+ and pandoc.List{'image/png', 'application/pdf'}
353
+ or pandoc.List{'application/pdf', 'image/png'}
354
+ -- Prefer SVG for non-PDF output formats, except for Office formats
355
+ if is_office_format then
356
+ preferred_mime_types:insert('image/svg+xml')
357
+ elseif pdf2svg then
358
+ preferred_mime_types:insert(1, 'image/svg+xml')
359
+ end
360
+ return {
361
+ name = name,
362
+ pdf2svg = pdf2svg,
363
+ preferred_mime_types = preferred_mime_types,
364
+ best_mime_type = function (self, supported_mime_types, requested)
365
+ return self.preferred_mime_types:find_if(function (preferred)
366
+ return supported_mime_types[preferred] and
367
+ (not requested or
368
+ (pandoc.utils.type(requested) == 'List' and
369
+ requested:includes(preferred)) or
370
+ (pandoc.utils.type(requested) == 'table' and
371
+ requested[preferred]) or
372
+
373
+ -- Assume string, Inlines, and Blocks values specify the only
374
+ -- acceptable MIME type.
375
+ stringify(requested) == preferred)
376
+ end)
377
+ end
378
+ }
379
+ end
380
+
381
+ --- Returns a configured diagram engine.
382
+ local function get_engine (name, engopts, format)
383
+ local engine = default_engines[name] or
384
+ select(2, pcall(require, stringify(engopts.package)))
385
+
386
+ -- Sanity check
387
+ if not engine then
388
+ warn(PANDOC_SCRIPT_FILE, ": No such engine '", name, "'.")
389
+ return nil
390
+ elseif engopts == false then
391
+ -- engine is disabled
392
+ return nil
393
+ elseif engopts == true then
394
+ -- use default options
395
+ return engine
396
+ end
397
+
398
+ local execpath = engopts.execpath or os.getenv(name:upper() .. '_BIN')
399
+
400
+ local mime_type = format:best_mime_type(
401
+ engine.mime_types,
402
+ engopts['mime-type'] or engopts['mime-types']
403
+ )
404
+ if not mime_type then
405
+ warn(PANDOC_SCRIPT_FILE, ": Cannot use ", name, " with ", format.name)
406
+ return nil
407
+ end
408
+
409
+ return {
410
+ execpath = execpath,
411
+ compile = engine.compile,
412
+ line_comment_start = engine.line_comment_start,
413
+ mime_type = mime_type,
414
+ opt = engopts or {},
415
+ }
416
+ end
417
+
418
+ --- Returns the diagram engine configs.
419
+ local function configure (meta, format_name)
420
+ local conf = meta.diagram or {}
421
+ local format = format_options(format_name)
422
+ meta.diagram = nil
423
+
424
+ -- cache for image files
425
+ if conf.cache then
426
+ image_cache = conf['cache-dir']
427
+ and stringify(conf['cache-dir'])
428
+ or cachedir()
429
+ pandoc.system.make_directory(image_cache, true)
430
+ end
431
+
432
+ -- engine configs
433
+ local engine = {}
434
+ for name, engopts in pairs(conf.engine or default_engines) do
435
+ engine[name] = get_engine(name, engopts, format)
436
+ end
437
+
438
+ return {
439
+ engine = engine,
440
+ format = format,
441
+ cache = image_cache and true,
442
+ image_cache = image_cache,
443
+ }
444
+ end
445
+
446
+ --
447
+ -- Format conversion
448
+ --
449
+
450
+ --- Converts a PDF to SVG.
451
+ local pdf2svg = function (imgdata)
452
+ -- Using `os.tmpname()` instead of a hash would be slightly cleaner, but the
453
+ -- function causes problems on Windows (and wasm). See, e.g.,
454
+ -- https://github.com/pandoc-ext/diagram/issues/49
455
+ local pdf_file = 'diagram-' .. pandoc.utils.sha1(imgdata) .. '.pdf'
456
+ write_file(pdf_file, imgdata)
457
+ local args = {
458
+ '--export-type=svg',
459
+ '--export-plain-svg',
460
+ '--export-filename=-',
461
+ pdf_file
462
+ }
463
+ return pandoc.pipe('inkscape', args, ''), os.remove(pdf_file)
464
+ end
465
+
466
+ local function properties_from_code (code, comment_start)
467
+ local props = {}
468
+ local pattern = comment_start:gsub('%p', '%%%1') .. '| ' ..
469
+ '([-_%w]+): ([^\n]*)\n'
470
+ for key, value in code:gmatch(pattern) do
471
+ if key == 'fig-cap' then
472
+ props['caption'] = value
473
+ else
474
+ props[key] = value
475
+ end
476
+ end
477
+ return props
478
+ end
479
+
480
+ local function diagram_options (cb, comment_start)
481
+ local attribs = comment_start
482
+ and properties_from_code(cb.text, comment_start)
483
+ or {}
484
+ for key, value in pairs(cb.attributes) do
485
+ attribs[key] = value
486
+ end
487
+
488
+ local alt
489
+ local caption
490
+ local fig_attr = {id = cb.identifier}
491
+ local filename
492
+ local image_attr = {}
493
+ local user_opt = {}
494
+
495
+ for attr_name, value in pairs(attribs) do
496
+ if attr_name == 'alt' then
497
+ alt = value
498
+ elseif attr_name == 'caption' then
499
+ -- Read caption attribute as Markdown
500
+ caption = attribs.caption
501
+ and pandoc.read(attribs.caption).blocks
502
+ or nil
503
+ elseif attr_name == 'filename' then
504
+ filename = value
505
+ elseif attr_name == 'label' then
506
+ fig_attr.id = value
507
+ elseif attr_name == 'name' then
508
+ fig_attr.name = value
509
+ else
510
+ -- Check for prefixed attributes
511
+ local prefix, key = attr_name:match '^(%a+)%-(%a[-%w]*)$'
512
+ if prefix == 'fig' then
513
+ fig_attr[key] = value
514
+ elseif prefix == 'image' or prefix == 'img' then
515
+ image_attr[key] = value
516
+ elseif prefix == 'opt' then
517
+ user_opt[key] = value
518
+ else
519
+ -- Use as image attribute
520
+ image_attr[attr_name] = value
521
+ end
522
+ end
523
+ end
524
+
525
+ return {
526
+ ['alt'] = alt or
527
+ (caption and pandoc.utils.blocks_to_inlines(caption)) or
528
+ {},
529
+ ['caption'] = caption,
530
+ ['fig-attr'] = fig_attr,
531
+ ['filename'] = filename,
532
+ ['image-attr'] = image_attr,
533
+ ['opt'] = user_opt,
534
+ }
535
+ end
536
+
537
+ local function get_cached_image (hash, mime_type)
538
+ if not image_cache then
539
+ return nil
540
+ end
541
+ local filename = hash .. '.' .. extension_for_mimetype[mime_type]
542
+ local imgpath = pandoc.path.join{image_cache, filename}
543
+ local success, imgdata = pcall(read_file, imgpath)
544
+ if success then
545
+ return imgdata, mime_type
546
+ end
547
+ return nil
548
+ end
549
+
550
+ local function cache_image (codeblock, imgdata, mimetype)
551
+ -- do nothing if caching is disabled or not possible.
552
+ if not image_cache then
553
+ return
554
+ end
555
+ local ext = extension_for_mimetype[mimetype]
556
+ local filename = pandoc.sha1(codeblock.text) .. '.' .. ext
557
+ local imgpath = pandoc.path.join{image_cache, filename}
558
+ write_file(imgpath, imgdata)
559
+ end
560
+
561
+ -- Executes each document's code block to find matching code blocks:
562
+ local function code_to_figure (conf)
563
+ return function (block)
564
+ -- Check if a converter exists for this block. If not, return the block
565
+ -- unchanged.
566
+ local diagram_type = block.classes[1]
567
+ if not diagram_type then
568
+ return nil
569
+ end
570
+
571
+ local engine = conf.engine[diagram_type]
572
+ if not engine then
573
+ return nil
574
+ end
575
+
576
+ -- Unified properties.
577
+ local dgr_opt = diagram_options(block, engine.line_comment_start)
578
+ for optname, value in pairs(engine.opt or {}) do
579
+ dgr_opt.opt[optname] = dgr_opt.opt[optname] or value
580
+ end
581
+
582
+ local run_pdf2svg = engine.mime_type == 'application/pdf'
583
+ and conf.format.pdf2svg
584
+
585
+ -- Try to retrieve the image data from the cache.
586
+ local imgdata, imgtype
587
+ if conf.cache then
588
+ imgdata, imgtype = get_cached_image(
589
+ pandoc.sha1(block.text),
590
+ run_pdf2svg and 'image/svg+xml' or engine.mime_type
591
+ )
592
+ end
593
+
594
+ if not imgdata or not imgtype then
595
+ -- No cached image; call the converter
596
+ local success
597
+ success, imgdata, imgtype =
598
+ pcall(engine.compile, engine, block.text, dgr_opt.opt)
599
+
600
+ -- Bail if an error occurred; imgdata contains the error message
601
+ -- when that happens.
602
+ if not success then
603
+ warn(PANDOC_SCRIPT_FILE, ': ', tostring(imgdata))
604
+ return nil
605
+ elseif not imgdata then
606
+ warn(PANDOC_SCRIPT_FILE, ': Diagram engine returned no image data.')
607
+ return nil
608
+ elseif not imgtype then
609
+ warn(PANDOC_SCRIPT_FILE, ': Diagram engine did not return a MIME type.')
610
+ return nil
611
+ end
612
+
613
+ -- Convert SVG if necessary.
614
+ if imgtype == 'application/pdf' and conf.format.pdf2svg then
615
+ imgdata, imgtype = pdf2svg(imgdata), 'image/svg+xml'
616
+ end
617
+
618
+ -- If we got here, then the transformation went ok and `img` contains
619
+ -- the image data.
620
+ cache_image(block, imgdata, imgtype)
621
+ end
622
+
623
+ -- Use the block's filename attribute or create a new name by hashing the
624
+ -- image content.
625
+ local basename, _extension = pandoc.path.split_extension(
626
+ dgr_opt.filename or pandoc.sha1(imgdata)
627
+ )
628
+ local fname = basename .. '.' .. extension_for_mimetype[imgtype]
629
+
630
+ -- Store the data in the media bag:
631
+ pandoc.mediabag.insert(fname, imgtype, imgdata)
632
+
633
+ -- Create the image object.
634
+ local image = pandoc.Image(dgr_opt.alt, fname, "", dgr_opt['image-attr'])
635
+
636
+ -- Create a figure if the diagram has a caption; otherwise return
637
+ -- just the image.
638
+ return dgr_opt.caption and
639
+ pandoc.Figure(
640
+ pandoc.Plain{image},
641
+ dgr_opt.caption,
642
+ dgr_opt['fig-attr']
643
+ ) or
644
+ pandoc.Plain{image}
645
+ end
646
+ end
647
+
648
+ return setmetatable(
649
+ {{
650
+ Pandoc = function (doc)
651
+ local conf = configure(doc.meta, FORMAT)
652
+ return doc:walk {
653
+ CodeBlock = code_to_figure(conf),
654
+ }
655
+ end
656
+ }},
657
+ {
658
+ version = version,
659
+ }
660
+ )
src/_quarto.yml CHANGED
@@ -1,34 +1,83 @@
1
  project:
2
  type: website
 
3
  website:
4
- title: "Open-Source AI Cookbook"
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5
  sidebar:
6
- style: "docked"
7
- search: true
8
- collapse-level: 3
9
- contents:
10
- - section: "About"
11
- contents:
12
- - href: index.qmd
13
- text: About Quarto
14
- - section: "Open-Source AI Cookbook"
15
- contents:
16
- - section: "RAG Techniques"
17
- contents:
18
- - href: notebooks/rag_zephyr_langchain.qmd
19
- text: "RAG Zephyr & LangChain"
20
- - href: notebooks/advanced_rag.qmd
21
- text: "Advanced RAG"
22
- - href: notebooks/rag_evaluation.qmd
23
- text: "RAG Evaluation"
24
- - section: "Additional Techniques"
25
- contents:
26
- - href: notebooks/automatic_embedding.ipynb
27
- text: "Automatic Embedding"
28
- - href: notebooks/faiss.ipynb
29
- text: "FAISS for Efficient Search"
30
- - href: notebooks/single_gpu.ipynb
31
- text: "Single GPU Optimization"
 
 
 
 
 
 
 
 
 
 
 
32
 
33
  format:
34
  html:
 
1
  project:
2
  type: website
3
+
4
  website:
5
+ title: "Kashi Coding Handbook"
6
+ navbar:
7
+ left:
8
+ - text: "Home"
9
+ href: index.qmd
10
+ - text: "Foundation Setup"
11
+ menu:
12
+ - chapters/ch01-development-environment.qmd
13
+ - chapters/ch01-project-structure.qmd
14
+ - text: "CLI Development"
15
+ menu:
16
+ - chapters/ch02-building-with-typer.qmd
17
+ - chapters/ch02-configuration-management.qmd
18
+ - text: "AI Integration"
19
+ menu:
20
+ - chapters/ch03-huggingface-llm-apis.qmd
21
+ - chapters/ch03-docker-model-deployment.qmd
22
+ - chapters/ch03-mcp-toolkit.qmd
23
+ - chapters/ch03-prompt-engineering.qmd
24
+ - text: "Advanced Features"
25
+ menu:
26
+ - chapters/ch04-interactive-elements.qmd
27
+ - chapters/ch04-batch-processing.qmd
28
+ - text: "Testing & Quality"
29
+ menu:
30
+ - chapters/ch05-writing-tests.qmd
31
+ - chapters/ch05-code-quality.qmd
32
+ - text: "Publishing"
33
+ menu:
34
+ - chapters/ch06-package-preparation.qmd
35
+ - chapters/ch06-building-publishing.qmd
36
+ - text: "Projects"
37
+ menu:
38
+ - chapters/ch07-fileorganizer-project.qmd
39
+ - text: "Appendices"
40
+ menu:
41
+ - chapters/appendix-pixi-commands.qmd
42
+ - chapters/appendix-learning-resources.qmd
43
  sidebar:
44
+ - title: "Kashi Coding Handbook"
45
+ subtitle: "Building AI-Powered CLI Tools with Python"
46
+ contents:
47
+ - index.qmd
48
+ - section: "Chapter 1: Foundation Setup"
49
+ contents:
50
+ - chapters/ch01-development-environment.qmd
51
+ - chapters/ch01-project-structure.qmd
52
+ - section: "Chapter 2: CLI Development"
53
+ contents:
54
+ - chapters/ch02-building-with-typer.qmd
55
+ - chapters/ch02-configuration-management.qmd
56
+ - section: "Chapter 3: AI Integration"
57
+ contents:
58
+ - chapters/ch03-huggingface-llm-apis.qmd
59
+ - chapters/ch03-docker-model-deployment.qmd
60
+ - chapters/ch03-mcp-toolkit.qmd
61
+ - chapters/ch03-prompt-engineering.qmd
62
+ - section: "Chapter 4: Advanced Features"
63
+ contents:
64
+ - chapters/ch04-interactive-elements.qmd
65
+ - chapters/ch04-batch-processing.qmd
66
+ - section: "Chapter 5: Testing & Quality"
67
+ contents:
68
+ - chapters/ch05-writing-tests.qmd
69
+ - chapters/ch05-code-quality.qmd
70
+ - section: "Chapter 6: Publishing"
71
+ contents:
72
+ - chapters/ch06-package-preparation.qmd
73
+ - chapters/ch06-building-publishing.qmd
74
+ - section: "Chapter 7: Real-World Projects"
75
+ contents:
76
+ - chapters/ch07-fileorganizer-project.qmd
77
+ - section: "Appendices"
78
+ contents:
79
+ - chapters/appendix-pixi-commands.qmd
80
+ - chapters/appendix-learning-resources.qmd
81
 
82
  format:
83
  html:
src/chapters/appendix-learning-resources.qmd ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Learning Resources
2
+
3
+ ## Coming Soon
4
+
5
+ This chapter is currently being developed. Content will be extracted from the learning path document.
6
+
7
+ ## Topics Covered
8
+
9
+ - Topic 1
10
+ - Topic 2
11
+ - Topic 3
12
+
13
+ ## Resources
14
+
15
+ - [Resource 1](#)
16
+ - [Resource 2](#)
src/chapters/appendix-pixi-commands.qmd ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Pixi Commands
2
+
3
+ ## Coming Soon
4
+
5
+ This chapter is currently being developed. Content will be extracted from the learning path document.
6
+
7
+ ## Topics Covered
8
+
9
+ - Topic 1
10
+ - Topic 2
11
+ - Topic 3
12
+
13
+ ## Resources
14
+
15
+ - [Resource 1](#)
16
+ - [Resource 2](#)
src/chapters/ch01-development-environment.qmd ADDED
@@ -0,0 +1,191 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Development Environment Setup
2
+
3
+ ## Learning Objectives
4
+
5
+ By the end of this section, you will be able to:
6
+
7
+ - Install and configure pixi for Python package management
8
+ - Set up GitHub Copilot for AI-assisted development
9
+ - Configure VS Code or your preferred IDE for Python development
10
+ - Verify your development environment is working correctly
11
+
12
+ ## Installing Pixi
13
+
14
+ Pixi is a modern, cross-platform package manager that simplifies Python project management and dependency handling.
15
+
16
+ ### Installation
17
+
18
+ ```bash
19
+ # Install pixi (works on Linux, macOS, and Windows)
20
+ curl -fsSL https://pixi.sh/install.sh | bash
21
+
22
+ # Verify installation
23
+ pixi --version
24
+ ```
25
+
26
+ ### Why Pixi?
27
+
28
+ - **Fast**: Uses conda packages with parallel downloads
29
+ - **Reproducible**: Lock files ensure consistent environments
30
+ - **Cross-platform**: Works identically on all operating systems
31
+ - **Modern**: Built with Rust for performance and reliability
32
+
33
+ ## Setting Up GitHub Copilot
34
+
35
+ GitHub Copilot is an AI pair programmer that helps you write code faster and with fewer errors.
36
+
37
+ ### Prerequisites
38
+
39
+ - GitHub account with Copilot subscription (free for students and open-source maintainers)
40
+ - VS Code or compatible IDE
41
+
42
+ ### Installation Steps
43
+
44
+ 1. **Install VS Code** (if not already installed)
45
+ - Download from [code.visualstudio.com](https://code.visualstudio.com/)
46
+
47
+ 2. **Install GitHub Copilot Extension**
48
+ - Open VS Code
49
+ - Go to Extensions (Ctrl+Shift+X / Cmd+Shift+X)
50
+ - Search for "GitHub Copilot"
51
+ - Click Install
52
+
53
+ 3. **Sign In**
54
+ - Click "Sign in to GitHub" when prompted
55
+ - Authorize VS Code to access your GitHub account
56
+ - Complete the authentication flow
57
+
58
+ 4. **Verify Setup**
59
+ - Create a new Python file
60
+ - Start typing a function definition
61
+ - You should see gray suggestions from Copilot
62
+
63
+ ### Copilot Best Practices
64
+
65
+ - **Write clear comments**: Copilot uses comments to understand your intent
66
+ - **Use descriptive names**: Function and variable names guide suggestions
67
+ - **Review suggestions**: Always review and understand generated code
68
+ - **Iterate**: If the first suggestion isn't right, try rephrasing your comment
69
+
70
+ ## Installing Git
71
+
72
+ Git is essential for version control and collaboration.
73
+
74
+ ```bash
75
+ # Linux (Debian/Ubuntu)
76
+ sudo apt install git
77
+
78
+ # macOS (with Homebrew)
79
+ brew install git
80
+
81
+ # Windows
82
+ # Download from git-scm.com
83
+ ```
84
+
85
+ Verify installation:
86
+
87
+ ```bash
88
+ git --version
89
+ ```
90
+
91
+ ## IDE Configuration
92
+
93
+ ### VS Code Extensions
94
+
95
+ Install these recommended extensions:
96
+
97
+ - **Python** (Microsoft) - Python language support
98
+ - **GitHub Copilot** - AI pair programmer
99
+ - **Pylance** - Fast Python language server
100
+ - **Ruff** - Fast Python linter
101
+ - **GitLens** - Enhanced Git capabilities
102
+
103
+ ### VS Code Settings
104
+
105
+ Create or update `.vscode/settings.json` in your project:
106
+
107
+ ```json
108
+ {
109
+ "python.linting.enabled": true,
110
+ "python.linting.ruffEnabled": true,
111
+ "python.formatting.provider": "black",
112
+ "editor.formatOnSave": true,
113
+ "editor.codeActionsOnSave": {
114
+ "source.organizeImports": true
115
+ },
116
+ "github.copilot.enable": {
117
+ "*": true,
118
+ "python": true
119
+ }
120
+ }
121
+ ```
122
+
123
+ ## Verification
124
+
125
+ Let's verify everything is working:
126
+
127
+ ```bash
128
+ # Check pixi
129
+ pixi --version
130
+
131
+ # Check Git
132
+ git --version
133
+
134
+ # Check Python (via pixi)
135
+ pixi run python --version
136
+ ```
137
+
138
+ ## Creating Your First Pixi Project
139
+
140
+ ```bash
141
+ # Create a new directory
142
+ mkdir my-first-cli
143
+ cd my-first-cli
144
+
145
+ # Initialize pixi project
146
+ pixi init
147
+
148
+ # Add Python
149
+ pixi add python
150
+
151
+ # Create a simple script
152
+ mkdir src
153
+ echo 'print("Hello from Kashi Coding Handbook!")' > src/hello.py
154
+
155
+ # Run it
156
+ pixi run python src/hello.py
157
+ ```
158
+
159
+ You should see: `Hello from Kashi Coding Handbook!`
160
+
161
+ ## Troubleshooting
162
+
163
+ ### Pixi not found after installation
164
+
165
+ - **Linux/macOS**: Add pixi to your PATH by restarting your terminal or running:
166
+ ```bash
167
+ source ~/.bashrc # or ~/.zshrc
168
+ ```
169
+
170
+ - **Windows**: Restart your terminal or add pixi to your system PATH
171
+
172
+ ### Copilot not showing suggestions
173
+
174
+ - Verify you're signed in to GitHub in VS Code
175
+ - Check that Copilot is enabled in VS Code settings
176
+ - Try reloading VS Code (Ctrl+Shift+P → "Reload Window")
177
+
178
+ ### Permission errors with pixi
179
+
180
+ - Don't use `sudo` with pixi
181
+ - Ensure you have write permissions in your home directory
182
+
183
+ ## Next Steps
184
+
185
+ Now that your development environment is set up, proceed to the next section to learn about modern Python project structure.
186
+
187
+ ## Resources
188
+
189
+ - [Pixi Documentation](https://pixi.sh/latest/)
190
+ - [GitHub Copilot Docs](https://docs.github.com/en/copilot)
191
+ - [VS Code Python Tutorial](https://code.visualstudio.com/docs/python/python-tutorial)
src/chapters/ch01-project-structure.qmd ADDED
@@ -0,0 +1,328 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Modern Python Project Structure
2
+
3
+ ## Learning Objectives
4
+
5
+ - Understand modern Python project layouts
6
+ - Learn about `pyproject.toml` and `pixi.toml` configuration files
7
+ - Set up proper directory structure for CLI applications
8
+ - Implement version control best practices
9
+
10
+ ## Project Layout Options
11
+
12
+ There are two main approaches to organizing Python projects:
13
+
14
+ ### Flat Layout
15
+
16
+ ```
17
+ my-cli/
18
+ ├── my_cli/
19
+ │ ├── __init__.py
20
+ │ ├── cli.py
21
+ │ └── utils.py
22
+ ├── tests/
23
+ ├── pyproject.toml
24
+ └── README.md
25
+ ```
26
+
27
+ ### Src Layout (Recommended)
28
+
29
+ ```
30
+ my-cli/
31
+ ├── src/
32
+ │ └── my_cli/
33
+ │ ├── __init__.py
34
+ │ ├── cli.py
35
+ │ └── utils.py
36
+ ├── tests/
37
+ ├── pyproject.toml
38
+ ├── pixi.toml
39
+ └── README.md
40
+ ```
41
+
42
+ **Why src layout?**
43
+ - Prevents accidental imports from development directory
44
+ - Clearer separation between source and other files
45
+ - Better for testing and packaging
46
+
47
+ ## Essential Configuration Files
48
+
49
+ ### pyproject.toml
50
+
51
+ The modern standard for Python project metadata:
52
+
53
+ ```toml
54
+ [build-system]
55
+ requires = ["hatchling"]
56
+ build-backend = "hatchling.build"
57
+
58
+ [project]
59
+ name = "my-cli-tool"
60
+ version = "0.1.0"
61
+ description = "An AI-powered CLI tool"
62
+ authors = [{name = "Your Name", email = "you@example.com"}]
63
+ readme = "README.md"
64
+ requires-python = ">=3.11"
65
+ dependencies = [
66
+ "typer>=0.9",
67
+ "rich>=13.0",
68
+ ]
69
+
70
+ [project.scripts]
71
+ my-cli = "my_cli.cli:app"
72
+
73
+ [tool.ruff]
74
+ line-length = 100
75
+ target-version = "py311"
76
+
77
+ [tool.mypy]
78
+ python_version = "3.11"
79
+ strict = true
80
+ ```
81
+
82
+ ### pixi.toml
83
+
84
+ Pixi-specific configuration for environment management:
85
+
86
+ ```toml
87
+ [project]
88
+ name = "my-cli-tool"
89
+ version = "0.1.0"
90
+ description = "An AI-powered CLI tool"
91
+ channels = ["conda-forge"]
92
+ platforms = ["linux-64", "osx-64", "win-64"]
93
+
94
+ [dependencies]
95
+ python = ">=3.11"
96
+ typer = ">=0.9"
97
+ rich = ">=13.0"
98
+
99
+ [feature.dev.dependencies]
100
+ pytest = "*"
101
+ ruff = "*"
102
+ mypy = "*"
103
+ black = "*"
104
+
105
+ [environments]
106
+ default = []
107
+ dev = ["dev"]
108
+
109
+ [tasks]
110
+ start = "python -m my_cli.cli"
111
+ test = "pytest tests/"
112
+ lint = "ruff check src/"
113
+ format = "black src/ tests/"
114
+ ```
115
+
116
+ ## Complete Project Structure
117
+
118
+ Here's a complete, production-ready project structure:
119
+
120
+ ```
121
+ my-cli-tool/
122
+ ├── .git/ # Git repository
123
+ ├── .gitignore # Git ignore rules
124
+ ├── .vscode/ # VS Code settings
125
+ │ └── settings.json
126
+ ├── src/
127
+ │ └── my_cli/
128
+ │ ├── __init__.py # Package initialization
129
+ │ ├── cli.py # Main CLI entry point
130
+ │ ├── commands/ # Command modules
131
+ │ │ ├── __init__.py
132
+ │ │ ├── organize.py
133
+ │ │ └── stats.py
134
+ │ ├── core/ # Core functionality
135
+ │ │ ├── __init__.py
136
+ │ │ └── processor.py
137
+ │ └── utils/ # Utility functions
138
+ │ ├── __init__.py
139
+ │ └── helpers.py
140
+ ├── tests/
141
+ │ ├── __init__.py
142
+ │ ├── conftest.py # Pytest configuration
143
+ │ ├── test_cli.py
144
+ │ └── test_core.py
145
+ ├── docs/ # Documentation
146
+ │ └── README.md
147
+ ├── .gitignore
148
+ ├── pyproject.toml # Python project config
149
+ ├── pixi.toml # Pixi environment config
150
+ ├── pixi.lock # Locked dependencies
151
+ ├── LICENSE # License file
152
+ └── README.md # Project documentation
153
+ ```
154
+
155
+ ## Creating the Structure
156
+
157
+ Use this script to create the structure:
158
+
159
+ ```bash
160
+ #!/bin/bash
161
+ # create-project.sh
162
+
163
+ PROJECT_NAME="my-cli-tool"
164
+ PACKAGE_NAME="my_cli"
165
+
166
+ # Create directories
167
+ mkdir -p $PROJECT_NAME/{src/$PACKAGE_NAME/{commands,core,utils},tests,docs,.vscode}
168
+
169
+ # Create __init__.py files
170
+ touch $PROJECT_NAME/src/$PACKAGE_NAME/__init__.py
171
+ touch $PROJECT_NAME/src/$PACKAGE_NAME/commands/__init__.py
172
+ touch $PROJECT_NAME/src/$PACKAGE_NAME/core/__init__.py
173
+ touch $PROJECT_NAME/src/$PACKAGE_NAME/utils/__init__.py
174
+ touch $PROJECT_NAME/tests/__init__.py
175
+
176
+ # Create main files
177
+ touch $PROJECT_NAME/src/$PACKAGE_NAME/cli.py
178
+ touch $PROJECT_NAME/README.md
179
+ touch $PROJECT_NAME/LICENSE
180
+
181
+ echo "Project structure created!"
182
+ ```
183
+
184
+ Or use pixi to create it:
185
+
186
+ ```bash
187
+ # Initialize with pixi
188
+ pixi init my-cli-tool
189
+ cd my-cli-tool
190
+
191
+ # Add Python and dependencies
192
+ pixi add python typer rich
193
+
194
+ # Create src layout
195
+ mkdir -p src/my_cli/{commands,core,utils}
196
+ touch src/my_cli/__init__.py
197
+ touch src/my_cli/cli.py
198
+
199
+ # Create tests
200
+ mkdir tests
201
+ touch tests/__init__.py
202
+ ```
203
+
204
+ ## .gitignore
205
+
206
+ Essential patterns for Python projects:
207
+
208
+ ```gitignore
209
+ # Python
210
+ __pycache__/
211
+ *.py[cod]
212
+ *$py.class
213
+ *.so
214
+ .Python
215
+ build/
216
+ develop-eggs/
217
+ dist/
218
+ downloads/
219
+ eggs/
220
+ .eggs/
221
+ lib/
222
+ lib64/
223
+ parts/
224
+ sdist/
225
+ var/
226
+ wheels/
227
+ *.egg-info/
228
+ .installed.cfg
229
+ *.egg
230
+
231
+ # Virtual environments
232
+ .env
233
+ .venv
234
+ env/
235
+ venv/
236
+ ENV/
237
+ env.bak/
238
+ venv.bak/
239
+
240
+ # Pixi
241
+ .pixi/
242
+ pixi.lock
243
+
244
+ # IDEs
245
+ .vscode/
246
+ .idea/
247
+ *.swp
248
+ *.swo
249
+ *~
250
+
251
+ # Testing
252
+ .pytest_cache/
253
+ .coverage
254
+ htmlcov/
255
+
256
+ # OS
257
+ .DS_Store
258
+ Thumbs.db
259
+ ```
260
+
261
+ ## Version Control Setup
262
+
263
+ Initialize Git and make your first commit:
264
+
265
+ ```bash
266
+ # Initialize repository
267
+ git init
268
+
269
+ # Create .gitignore (use content above)
270
+ # Use Copilot: "Generate a comprehensive .gitignore for Python projects"
271
+
272
+ # Add files
273
+ git add .
274
+
275
+ # First commit
276
+ git commit -m "Initial project structure"
277
+
278
+ # Create GitHub repository (optional)
279
+ gh repo create my-cli-tool --public --source=. --remote=origin
280
+ git push -u origin main
281
+ ```
282
+
283
+ ## Package Initialization
284
+
285
+ ### src/my_cli/__init__.py
286
+
287
+ ```python
288
+ """My CLI Tool - An AI-powered command-line application."""
289
+
290
+ __version__ = "0.1.0"
291
+ __author__ = "Your Name"
292
+ __email__ = "you@example.com"
293
+
294
+ # Export main components
295
+ from .cli import app
296
+
297
+ __all__ = ["app"]
298
+ ```
299
+
300
+ ## Best Practices
301
+
302
+ 1. **Use src layout**: Prevents import issues and improves testing
303
+ 2. **Lock dependencies**: Commit `pixi.lock` for reproducibility
304
+ 3. **Separate concerns**: Use subdirectories for commands, core logic, and utilities
305
+ 4. **Write tests early**: Create test files alongside implementation
306
+ 5. **Document as you go**: Update README with each feature
307
+ 6. **Use type hints**: Enable better IDE support and catch errors early
308
+
309
+ ## Using Copilot for Project Setup
310
+
311
+ Ask Copilot to help with:
312
+
313
+ ```python
314
+ # In your IDE, write comments like:
315
+ # "Create a pyproject.toml for a CLI tool with typer and rich dependencies"
316
+ # "Generate a comprehensive .gitignore for a Python project"
317
+ # "Create a README template for a CLI application"
318
+ ```
319
+
320
+ ## Next Steps
321
+
322
+ With your project structure in place, you're ready to start building your CLI application. In the next chapter, we'll use Typer to create powerful command-line interfaces.
323
+
324
+ ## Resources
325
+
326
+ - [Python Packaging Guide](https://packaging.python.org/)
327
+ - [Pixi Project Configuration](https://pixi.sh/latest/reference/project_configuration/)
328
+ - [PEP 518 - pyproject.toml](https://peps.python.org/pep-0518/)
src/chapters/ch02-building-with-typer.qmd ADDED
@@ -0,0 +1,416 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Building CLI Applications with Typer
2
+
3
+ ## Learning Objectives
4
+
5
+ - Understand Typer's type-hint based approach to CLI development
6
+ - Create commands with arguments and options
7
+ - Implement subcommands for complex CLIs
8
+ - Add rich formatting and user-friendly output
9
+ - Handle errors and validation gracefully
10
+
11
+ ## Introduction to Typer
12
+
13
+ Typer is a modern CLI framework that uses Python type hints to automatically generate command-line interfaces. It's built on top of Click but provides a more intuitive, type-safe API.
14
+
15
+ ### Why Typer?
16
+
17
+ - **Type hints**: Uses Python's type system for automatic validation
18
+ - **Auto-completion**: Built-in shell completion support
19
+ - **Rich integration**: Beautiful terminal output out of the box
20
+ - **Less boilerplate**: Minimal code for maximum functionality
21
+ - **Great docs**: Excellent documentation and examples
22
+
23
+ ## Installation
24
+
25
+ ```bash
26
+ pixi add typer rich
27
+ ```
28
+
29
+ ## Your First Typer CLI
30
+
31
+ Create `src/my_cli/cli.py`:
32
+
33
+ ```python
34
+ import typer
35
+ from rich.console import Console
36
+
37
+ app = typer.Typer(help="My awesome CLI tool")
38
+ console = Console()
39
+
40
+ @app.command()
41
+ def hello(name: str = typer.Argument(..., help="Your name")):
42
+ """Say hello to NAME."""
43
+ console.print(f"[bold green]Hello {name}![/bold green]")
44
+
45
+ if __name__ == "__main__":
46
+ app()
47
+ ```
48
+
49
+ Run it:
50
+
51
+ ```bash
52
+ pixi run python -m my_cli.cli hello World
53
+ # Output: Hello World!
54
+ ```
55
+
56
+ ## Commands, Arguments, and Options
57
+
58
+ ### Arguments
59
+
60
+ Required positional parameters:
61
+
62
+ ```python
63
+ @app.command()
64
+ def process(
65
+ filename: str = typer.Argument(..., help="File to process"),
66
+ output: str = typer.Argument("output.txt", help="Output file")
67
+ ):
68
+ """Process FILENAME and save to OUTPUT."""
69
+ console.print(f"Processing {filename} → {output}")
70
+ ```
71
+
72
+ ### Options
73
+
74
+ Optional named parameters:
75
+
76
+ ```python
77
+ from pathlib import Path
78
+
79
+ @app.command()
80
+ def organize(
81
+ directory: Path = typer.Option(
82
+ ".",
83
+ "--dir",
84
+ "-d",
85
+ help="Directory to organize",
86
+ exists=True,
87
+ file_okay=False,
88
+ dir_okay=True
89
+ ),
90
+ dry_run: bool = typer.Option(
91
+ False,
92
+ "--dry-run",
93
+ help="Preview changes without executing"
94
+ ),
95
+ verbose: bool = typer.Option(
96
+ False,
97
+ "--verbose",
98
+ "-v",
99
+ help="Show detailed output"
100
+ )
101
+ ):
102
+ """Organize files in DIRECTORY."""
103
+ if dry_run:
104
+ console.print("[yellow]DRY RUN MODE[/yellow]")
105
+
106
+ if verbose:
107
+ console.print(f"[blue]Processing: {directory}[/blue]")
108
+ ```
109
+
110
+ ## Rich Output Formatting
111
+
112
+ ### Console Printing
113
+
114
+ ```python
115
+ from rich.console import Console
116
+ from rich.table import Table
117
+ from rich.progress import track
118
+ import time
119
+
120
+ console = Console()
121
+
122
+ @app.command()
123
+ def stats(directory: Path):
124
+ """Show statistics about files."""
125
+
126
+ # Colored output
127
+ console.print("[bold cyan]File Statistics[/bold cyan]")
128
+
129
+ # Tables
130
+ table = Table(title="File Types")
131
+ table.add_column("Extension", style="cyan")
132
+ table.add_column("Count", justify="right", style="green")
133
+
134
+ table.add_row(".py", "42")
135
+ table.add_row(".txt", "15")
136
+ table.add_row(".md", "8")
137
+
138
+ console.print(table)
139
+
140
+ # Progress bars
141
+ console.print("\n[bold]Processing files...[/bold]")
142
+ for _ in track(range(100), description="Scanning..."):
143
+ time.sleep(0.01)
144
+ ```
145
+
146
+ ## Subcommands
147
+
148
+ For complex CLIs, organize commands into groups:
149
+
150
+ ```python
151
+ import typer
152
+
153
+ app = typer.Typer()
154
+
155
+ # Create subcommand groups
156
+ files_app = typer.Typer(help="File operations")
157
+ config_app = typer.Typer(help="Configuration management")
158
+
159
+ app.add_typer(files_app, name="files")
160
+ app.add_typer(config_app, name="config")
161
+
162
+ # File commands
163
+ @files_app.command("organize")
164
+ def files_organize(directory: Path):
165
+ """Organize files in directory."""
166
+ console.print(f"Organizing {directory}")
167
+
168
+ @files_app.command("stats")
169
+ def files_stats(directory: Path):
170
+ """Show file statistics."""
171
+ console.print(f"Stats for {directory}")
172
+
173
+ # Config commands
174
+ @config_app.command("show")
175
+ def config_show():
176
+ """Show current configuration."""
177
+ console.print("Current config...")
178
+
179
+ @config_app.command("set")
180
+ def config_set(key: str, value: str):
181
+ """Set configuration value."""
182
+ console.print(f"Set {key} = {value}")
183
+ ```
184
+
185
+ Usage:
186
+
187
+ ```bash
188
+ my-cli files organize ./data
189
+ my-cli files stats ./data
190
+ my-cli config show
191
+ my-cli config set theme dark
192
+ ```
193
+
194
+ ## Input Validation
195
+
196
+ ### Type Validation
197
+
198
+ ```python
199
+ from typing import Optional
200
+ from enum import Enum
201
+
202
+ class OutputFormat(str, Enum):
203
+ JSON = "json"
204
+ YAML = "yaml"
205
+ TEXT = "text"
206
+
207
+ @app.command()
208
+ def export(
209
+ count: int = typer.Option(10, min=1, max=100),
210
+ format: OutputFormat = typer.Option(OutputFormat.JSON),
211
+ email: Optional[str] = typer.Option(None)
212
+ ):
213
+ """Export data in specified format."""
214
+ if email and "@" not in email:
215
+ raise typer.BadParameter("Invalid email address")
216
+
217
+ console.print(f"Exporting {count} items as {format.value}")
218
+ ```
219
+
220
+ ### Custom Validation
221
+
222
+ ```python
223
+ def validate_path(path: Path) -> Path:
224
+ """Validate that path exists and is readable."""
225
+ if not path.exists():
226
+ raise typer.BadParameter(f"Path does not exist: {path}")
227
+ if not path.is_dir():
228
+ raise typer.BadParameter(f"Path is not a directory: {path}")
229
+ return path
230
+
231
+ @app.command()
232
+ def process(
233
+ directory: Path = typer.Argument(..., callback=validate_path)
234
+ ):
235
+ """Process files in directory."""
236
+ console.print(f"Processing {directory}")
237
+ ```
238
+
239
+ ## Error Handling
240
+
241
+ ```python
242
+ from rich.console import Console
243
+
244
+ console = Console()
245
+
246
+ @app.command()
247
+ def risky_operation(filename: str):
248
+ """Perform a risky operation."""
249
+ try:
250
+ with open(filename) as f:
251
+ data = f.read()
252
+ console.print("[green]Success![/green]")
253
+ except FileNotFoundError:
254
+ console.print(f"[red]Error: File not found: {filename}[/red]")
255
+ raise typer.Exit(code=1)
256
+ except PermissionError:
257
+ console.print(f"[red]Error: Permission denied: {filename}[/red]")
258
+ raise typer.Exit(code=1)
259
+ except Exception as e:
260
+ console.print(f"[red]Unexpected error: {e}[/red]")
261
+ raise typer.Exit(code=1)
262
+ ```
263
+
264
+ ## Interactive Prompts
265
+
266
+ ```python
267
+ @app.command()
268
+ def delete(filename: str):
269
+ """Delete a file with confirmation."""
270
+ if not typer.confirm(f"Are you sure you want to delete {filename}?"):
271
+ console.print("[yellow]Cancelled[/yellow]")
272
+ raise typer.Abort()
273
+
274
+ # Proceed with deletion
275
+ console.print(f"[red]Deleted {filename}[/red]")
276
+ ```
277
+
278
+ ## Complete Example: File Organizer CLI
279
+
280
+ ```python
281
+ import typer
282
+ from pathlib import Path
283
+ from rich.console import Console
284
+ from rich.table import Table
285
+ from rich.progress import track
286
+ from typing import Optional
287
+ from enum import Enum
288
+
289
+ app = typer.Typer(
290
+ name="file-organizer",
291
+ help="Organize files intelligently",
292
+ add_completion=True
293
+ )
294
+ console = Console()
295
+
296
+ class Strategy(str, Enum):
297
+ EXTENSION = "extension"
298
+ DATE = "date"
299
+ SIZE = "size"
300
+
301
+ @app.command()
302
+ def organize(
303
+ directory: Path = typer.Argument(
304
+ ...,
305
+ help="Directory to organize",
306
+ exists=True,
307
+ file_okay=False,
308
+ dir_okay=True
309
+ ),
310
+ strategy: Strategy = typer.Option(
311
+ Strategy.EXTENSION,
312
+ "--strategy",
313
+ "-s",
314
+ help="Organization strategy"
315
+ ),
316
+ dry_run: bool = typer.Option(
317
+ False,
318
+ "--dry-run",
319
+ help="Preview without making changes"
320
+ ),
321
+ verbose: bool = typer.Option(
322
+ False,
323
+ "--verbose",
324
+ "-v",
325
+ help="Show detailed output"
326
+ )
327
+ ):
328
+ """Organize files in DIRECTORY using STRATEGY."""
329
+
330
+ if dry_run:
331
+ console.print("[yellow]DRY RUN - No changes will be made[/yellow]\n")
332
+
333
+ console.print(f"[bold cyan]Organizing {directory}[/bold cyan]")
334
+ console.print(f"Strategy: [green]{strategy.value}[/green]\n")
335
+
336
+ # Scan files
337
+ files = list(directory.glob("*"))
338
+ files = [f for f in files if f.is_file()]
339
+
340
+ if verbose:
341
+ console.print(f"Found {len(files)} files\n")
342
+
343
+ # Process files
344
+ for file in track(files, description="Processing..."):
345
+ if verbose:
346
+ console.print(f" Processing: {file.name}")
347
+
348
+ console.print("\n[bold green]✓ Complete![/bold green]")
349
+
350
+ @app.command()
351
+ def stats(
352
+ directory: Path = typer.Argument(..., exists=True, file_okay=False)
353
+ ):
354
+ """Show statistics about files in DIRECTORY."""
355
+
356
+ files = list(directory.glob("*"))
357
+ files = [f for f in files if f.is_file()]
358
+
359
+ # Count by extension
360
+ extensions = {}
361
+ for file in files:
362
+ ext = file.suffix or "no extension"
363
+ extensions[ext] = extensions.get(ext, 0) + 1
364
+
365
+ # Display table
366
+ table = Table(title=f"File Statistics: {directory}")
367
+ table.add_column("Extension", style="cyan")
368
+ table.add_column("Count", justify="right", style="green")
369
+
370
+ for ext, count in sorted(extensions.items(), key=lambda x: x[1], reverse=True):
371
+ table.add_row(ext, str(count))
372
+
373
+ console.print(table)
374
+ console.print(f"\n[bold]Total files: {len(files)}[/bold]")
375
+
376
+ if __name__ == "__main__":
377
+ app()
378
+ ```
379
+
380
+ ## Testing Your CLI
381
+
382
+ ```python
383
+ # tests/test_cli.py
384
+ from typer.testing import CliRunner
385
+ from my_cli.cli import app
386
+
387
+ runner = CliRunner()
388
+
389
+ def test_organize_dry_run():
390
+ result = runner.invoke(app, ["organize", ".", "--dry-run"])
391
+ assert result.exit_code == 0
392
+ assert "DRY RUN" in result.stdout
393
+
394
+ def test_stats():
395
+ result = runner.invoke(app, ["stats", "."])
396
+ assert result.exit_code == 0
397
+ ```
398
+
399
+ ## Using Copilot
400
+
401
+ Ask Copilot to help with:
402
+
403
+ - "Create a Typer command that accepts a file path and validates it exists"
404
+ - "Add a progress bar for processing multiple files"
405
+ - "Generate help text and docstrings for CLI commands"
406
+ - "Create an enum for output format options"
407
+
408
+ ## Next Steps
409
+
410
+ Now that you can build CLIs with Typer, the next section covers configuration management for your applications.
411
+
412
+ ## Resources
413
+
414
+ - [Typer Documentation](https://typer.tiangolo.com/)
415
+ - [Rich Documentation](https://rich.readthedocs.io/)
416
+ - [Typer Tutorial](https://typer.tiangolo.com/tutorial/)