File size: 34,712 Bytes
4eecd03
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
#!/usr/bin/env python3
"""The missing link: a supervised hardware.py job -> G-code a real motion controller runs, streamed under the supervisor.

``hardware.py`` stops at a command list in wall-plane millimetres. This module is the adapter docs/HARDWARE.md said was
"to build", for the cheapest machine that can actually put paint on a wall: a gantry or mast carriage on a GRBL /
FluidNC / grblHAL board (three axes: X along the wall, Z up the mast, Y towards the wall; tool outputs on M-codes).
ROS 2 or an industrial controller would want a different emitter; the program model here (one line, the commands it
carries out) is what they would share.

What it adds, beyond translating coordinates:

* **stations as stops.** Each base re-park is an ``M0`` pause with the station's wall range in the comment; streamed,
  it is an operator callback (park, register, confirm), and the base move is never a G-code move.
* **tool engagement.** Contact tools plunge on Y to a press depth computed from the force target and the *measured*
  nap stiffness (``measurements.py``), and retract between strokes; the spray opens ``trigger_lead_mm`` before the
  stroke and closes ``release_lead_mm`` before its end, so the valve latency lands the fan on the line.
* **refuse, don't clip.** A move outside the machine's travel is a ``ProgramError``, never a clamped coordinate.
* **air-cut first.** ``dry_run=True`` emits the same motion with every output off and contact depth held
  ``aircut_clearance_mm`` short of the wall. A wet run of a program is refused until an air-cut of the *same program*
  (by hash) has finished cleanly, and until commissioning (``commissioning.py``) says the machine is ready.
* **the supervisor stays live.** ``Streamer`` checks every wall-plane command a line carries through
  ``SafetySupervisor.check`` before the line is sent; status polls (``?``, answered even with the planner full) are the
  heartbeat; an ``ALARM``, an ``error``, a silent controller, or a reported position the program never visits trips the
  supervisor and sends feed-hold (``!``) then soft-reset (Ctrl-X, which drops spindle and coolant outputs on GRBL).
* **as-run, not as-planned.** Every status sample is logged; ``as_run_job`` turns the log back into a wall-plane job
  that ``hardware.rasterize`` / ``coverage_loop`` can score, so coverage comes from where the machine went.

Not in this module, and not optional on a real machine: a hard-wired e-stop that cuts motor and pump power without
software, limit switches, and drive-level current limits. The supervisor is the software half (safety_supervisor.py).

Fixed-depth warning: GRBL holds position, not force. ``roller_physics`` shows a fixed-depth roller inside its force
window ~37 % of the time on a +-5 mm wall; mount contact tools on a compliant (spring or pneumatic, constant-force)
slide and set ``force_control: "compliant"`` in the machine file, or the plan's film numbers do not apply.

    python3 controller_adapter.py gcode --job job.json --machine machine.json --out wall.nc [--dry-run]
    python3 controller_adapter.py stream --job job.json --machine machine.json --port /dev/ttyUSB0 --gate gate.json --dry-run
    python3 controller_adapter.py stream ... --sim                       # against the built-in simulated GRBL
"""
from __future__ import annotations

import argparse
import hashlib
import json
import math
import re
import time
from dataclasses import dataclass, field, asdict
from pathlib import Path

import hardware
import measurements
from safety_supervisor import SafeStop

HERE = Path(__file__).resolve().parent
RUNS_DIR = HERE / "artifacts" / "machine_runs"


class ProgramError(ValueError):
    """The job cannot be turned into a program this machine can run safely."""


@dataclass
class Machine:
    name: str = "prototype"
    platform: str = "mast_arm"
    x_travel_mm: tuple = (0.0, 1200.0)          # carriage travel along the wall at one station
    x_origin_mm: float = 0.0                    # machine X of the station's left edge: travel beyond the station on
                                                # both sides is runway for the spray to open before a stroke
    z_travel_mm: tuple = (150.0, 3200.0)        # mast travel, machine Z = wall z - z_offset_mm
    y_travel_mm: tuple = (0.0, 400.0)           # towards the wall; 0 = fully retracted
    z_offset_mm: float = 0.0                    # wall z at machine Z = 0 (0: machine Z is height above the floor)
    y_wall_mm: float | None = None              # Y at which the tool face touches the wall (commissioning touch-off)
    y_retract_mm: float = 0.0
    spray_y_mm: float | None = None             # Y for the spray standoff; default y_wall - standoff
    press_mm: dict = field(default_factory=lambda: {"brush": 3.0, "sander": 2.0, "scorer": 2.0, "steamer": 2.0,
                                                    "scraper": 2.0, "smoother": 3.0, "taper": 2.0})
    outputs: dict = field(default_factory=lambda: {"spray": ["M8", "M9"], "sander": ["M3 S1000", "M5"],
                                                   "steamer": ["M3 S1000", "M5"]})
    plunge_feed_mm_min: float = 600.0
    max_feed_mm_min: float = 90000.0
    aircut_clearance_mm: float = 20.0
    force_control: str = "fixed"                # fixed | compliant (spring / pneumatic constant-force slide)
    status_period_s: float = 0.1                # 10 Hz: the as-run path is only as fine as this (25 mm at 250 mm/s)
    heartbeat_timeout_s: float = 1.0
    position_tolerance_mm: float = 5.0

    @classmethod
    def load(cls, path=None, **over):
        data = json.loads(Path(path).read_text(encoding="utf-8")) if path else {}
        data.update(over)
        known = {k: v for k, v in data.items() if k in cls.__dataclass_fields__}
        for k in ("x_travel_mm", "z_travel_mm", "y_travel_mm"):
            if k in known:
                known[k] = tuple(known[k])
        return cls(**known)


def roller_press_mm(tool):
    """Press depth that gives the middle of the force window at the (measured, else assumed) nap stiffness."""
    import roller_physics
    p = roller_physics.params()
    return 0.5 * sum(tool["force_n"]) / p["nap_stiffness_n_mm"]


@dataclass
class Line:
    text: str
    commands: tuple = ()          # indices into job["commands"] this line carries out
    kind: str = "motion"          # motion | output | station | comment
    station: int | None = None
    target: tuple | None = None   # machine (x, y, z) after this line


def _f(v):
    return f"{v:.3f}".rstrip("0").rstrip(".")


def build_program(job, machine, dry_run=False):
    """Lines for one job. Raises ProgramError for anything the machine cannot do as planned."""
    tool = job["tool"]
    name = tool["name"]
    platform = hardware.PLATFORMS[job["platform"]]
    if machine.platform != job["platform"]:
        raise ProgramError(f"job is for {job['platform']}, machine is a {machine.platform}")
    if machine.y_wall_mm is None:
        raise ProgramError("machine has no y_wall_mm: run the commissioning touch-off first")
    reach = platform.reach_x_mm
    if machine.x_origin_mm < machine.x_travel_mm[0] - 1e-6 or machine.x_origin_mm + reach > machine.x_travel_mm[1] + 1e-6:
        raise ProgramError(f"carriage travel {machine.x_travel_mm} does not hold the {reach:.0f} mm station the job was "
                           f"compiled for (from X {machine.x_origin_mm:g}); recompile with the machine's reach")
    contact = bool(tool["contact"])
    if contact:
        press = roller_press_mm(tool) if name == "roller" else machine.press_mm.get(name, 2.0)
        y_work = machine.y_wall_mm + press
    else:
        y_work = machine.spray_y_mm if machine.spray_y_mm is not None else machine.y_wall_mm - tool["standoff_mm"]
    if dry_run and contact:
        y_work = machine.y_wall_mm - machine.aircut_clearance_mm
    on, off = (machine.outputs.get(name) or [None, None])      # in an air-cut these become comments, motion unchanged
    lines, warnings = [], []
    if contact and machine.force_control != "compliant":
        warnings.append(f"{name}: fixed-depth contact (no compliant slide); film and force numbers from the plan "
                        "assume a force loop (roller_physics: ~37 % in window at fixed depth on a +-5 mm wall)")

    def check(x, y, z):
        for axis, v, (lo, hi) in (("X", x, machine.x_travel_mm), ("Y", y, machine.y_travel_mm), ("Z", z, machine.z_travel_mm)):
            if not lo - 1e-6 <= v <= hi + 1e-6:
                raise ProgramError(f"{axis} {v:.1f} mm is outside the machine's travel [{lo:g}, {hi:g}]")

    state = {"x": None, "z": None, "y": machine.y_retract_mm, "engaged": False, "station": None}
    late = {"n": 0, "worst": 0.0}

    def mach(c, station):
        return c["x"] - station * reach + machine.x_origin_mm, c["y"] - machine.z_offset_mm

    def move(x, z, feed, idx, kind="motion"):
        check(x, state["y"], z)
        state["x"], state["z"] = x, z
        lines.append(Line(f"G1 X{_f(x)} Z{_f(z)} F{_f(min(feed, machine.max_feed_mm_min))}", tuple(idx), kind,
                          state["station"], (x, state["y"], z)))

    def set_y(y, idx=()):
        check(state["x"] if state["x"] is not None else machine.x_travel_mm[0], y,
              state["z"] if state["z"] is not None else machine.z_travel_mm[0])
        state["y"] = y
        lines.append(Line(f"G1 Y{_f(y)} F{_f(machine.plunge_feed_mm_min)}", tuple(idx), "motion", state["station"],
                          (state["x"], y, state["z"])))

    def output(text, comment):
        if dry_run:
            lines.append(Line(f"({comment}: {text} withheld, air-cut)", (), "comment", state["station"]))
        else:
            lines.append(Line(text, (), "output", state["station"]))

    def engage(idx=()):
        if not state["engaged"]:
            set_y(y_work, idx)
            if on:
                output(on, f"{name} on")
            state["engaged"] = True

    def disengage():
        if state["engaged"]:
            if off:
                output(off, f"{name} off")
            set_y(machine.y_retract_mm)
            state["engaged"] = False

    lines.append(Line(f"({job['operation']} on {job['platform']}, wall {job['commands'][0].get('wall_id') if job['commands'] else None}"
                      f"{', AIR-CUT' if dry_run else ''})", (), "comment"))
    lines.append(Line("G21 G90 G94", (), "comment"))
    lines.append(Line(f"G1 Y{_f(machine.y_retract_mm)} F{_f(machine.plunge_feed_mm_min)}", (), "motion", None,
                      None))
    cmds = job["commands"]
    travel_idx, travel_to = [], None
    feed_travel = 0.8 * platform.max_speed_mm_s * 60

    def flush_travel():
        nonlocal travel_idx, travel_to
        if travel_to is not None:
            disengage()
            move(*travel_to, feed_travel, travel_idx)
        travel_idx, travel_to = [], None

    i = 0
    while i < len(cmds):
        c = cmds[i]
        station = c.get("station", 0) or 0
        if station != state["station"]:
            flush_travel()
            disengage()
            lo = station * reach
            state["station"] = station
            state["x"] = state["z"] = None
            lines.append(Line(f"M0 (park base at station {station}: wall x {lo:.0f}-{lo + reach:.0f} mm, register, resume)",
                              (), "station", station))
        if not c["paint"]:
            travel_idx.append(i)
            travel_to = mach(c, station)
            i += 1
            continue
        if c.get("dwell_s"):
            flush_travel()
            engage((i,))
            lines.append(Line(f"G4 P{_f(c['dwell_s'])}", (i,), "motion", station, (state["x"], state["y"], state["z"])))
            # the next command (paint off at the same spot) disengages through the travel path
            i += 1
            continue
        # a stroke: consecutive paint commands
        flush_travel()
        start = (state["x"], state["z"])
        if start[0] is None:
            raise ProgramError(f"command {i} paints without a preceding position")
        feed = (c.get("speed_mm_s") or job["speed_mm_s"]) * 60
        lead = c.get("trigger_lead_mm") or 0.0
        if lead and on and not contact:
            x1, z1 = mach(c, station)
            d = math.dist(start, (x1, z1)) or 1.0
            ux, uz = (x1 - start[0]) / d, (z1 - start[1]) / d
            room = lead
            for v, u, (lo, hi) in ((start[0], ux, machine.x_travel_mm), (start[1], uz, machine.z_travel_mm)):
                if u > 1e-9:
                    room = min(room, (v - lo) / u)
                elif u < -1e-9:
                    room = min(room, (hi - v) / -u)
            if room < lead - 0.5:
                late["n"] += 1
                late["worst"] = max(late["worst"], lead - room)
            move(start[0] - ux * room, start[1] - uz * room, feed_travel, ())       # back off so the gun opens early
            set_y(y_work)
            output(on, f"{name} on, {lead:.0f} mm early for valve latency")
            state["engaged"] = True
            move(*start, feed, ())
        else:
            engage()
        while i < len(cmds) and cmds[i]["paint"] and not cmds[i].get("dwell_s") and (cmds[i].get("station", 0) or 0) == station:
            c = cmds[i]
            x, z = mach(c, station)
            release = c.get("release_lead_mm") or 0.0
            if release and off and not contact:
                d = math.dist((state["x"], state["z"]), (x, z)) or 1.0
                f = max(0.0, 1 - release / d)
                px, pz = state["x"] + (x - state["x"]) * f, state["z"] + (z - state["z"]) * f
                move(px, pz, feed, (i,))
                output(off, f"{name} off, {release:.0f} mm early")
                state["engaged"] = False
                move(x, z, feed, ())
                set_y(machine.y_retract_mm)
            else:
                move(x, z, feed, (i,))
            i += 1
    flush_travel()
    disengage()
    lines.append(Line("M2", (), "comment"))
    if late["n"]:
        warnings.append(f"{name}: {late['n']} stroke(s) start at the end of carriage travel, so the gun opens up to "
                        f"{late['worst']:.0f} mm late (a thin start); give the carriage that much runway beyond the "
                        "station (x_travel_mm wider than the reach, x_origin_mm inside it)")
    covered = sorted({j for ln in lines for j in ln.commands})
    if covered != list(range(len(cmds))):
        missing = sorted(set(range(len(cmds))) - set(covered))
        raise ProgramError(f"{len(missing)} command(s) did not make it into the program (first: {missing[:5]})")
    return {"lines": lines, "warnings": warnings, "dry_run": dry_run, "hash": program_hash(lines, ignore_outputs=True),
            "tool": name}


def program_hash(lines, ignore_outputs=True):
    """Hash of the motion a program makes. Outputs and the air-cut depth are left out (``ignore_outputs``) so the
    air-cut and the wet run of the same job share a hash; everything that moves the carriage in X and Z is in it."""
    h = hashlib.sha256()
    for ln in lines:
        if ln.kind == "motion" and not re.match(r"G1 Y", ln.text):
            h.update(ln.text.encode())
        elif ln.kind == "station":
            h.update(ln.text.encode())
        elif not ignore_outputs:
            h.update(ln.text.encode())
    return h.hexdigest()[:16]


def program_envelope(program):
    """Per station, the X/Z box the program's motion stays inside: ``{station: [x_lo, x_hi, z_lo, z_hi]}``."""
    env = {}
    for ln in program["lines"]:
        if ln.kind == "motion" and ln.target and ln.station is not None and None not in (ln.target[0], ln.target[2]):
            x, _, z = ln.target
            b = env.setdefault(str(ln.station), [x, x, z, z])
            b[0], b[1], b[2], b[3] = min(b[0], x), max(b[1], x), min(b[2], z), max(b[3], z)
    return env


def _inside(inner, outer, tol=1.0):
    return all(k in outer and outer[k][0] - tol <= b[0] and b[1] <= outer[k][1] + tol and outer[k][2] - tol <= b[2]
               and b[3] <= outer[k][3] + tol for k, b in inner.items())


def to_gcode(program):
    return "\n".join(ln.text for ln in program["lines"]) + "\n"


# ------------------------------------------------------------------ transports
class SerialTransport:
    """A GRBL board on a serial port (needs pyserial)."""

    def __init__(self, port, baud=115200):
        import serial                                         # imported here: only a real machine needs it
        self.s = serial.Serial(port, baud, timeout=0.05)
        time.sleep(2.0)                                       # GRBL resets on open
        self.s.reset_input_buffer()

    def write(self, data: bytes):
        self.s.write(data)

    def readline(self, timeout=0.05):
        self.s.timeout = timeout
        raw = self.s.readline()
        return raw.decode(errors="replace").strip() if raw else None


class SimGrbl:
    """A simulated GRBL for tests and rehearsal: executes moves instantly, answers ``ok`` and ``?``.

    Faults to inject: ``alarm_after`` (ALARM:1 on that line, e.g. a hard limit), ``hang_after`` (stops answering
    anything), ``drift_after`` (reported position jumps by ``drift_mm``, as after lost steps).
    """

    def __init__(self, alarm_after=None, hang_after=None, drift_after=None, drift_mm=50.0, home=(0.0, 0.0, 0.0)):
        self.pos = list(home)                              # where homing left the machine
        self.flood = self.spindle = False
        self.n = 0
        self.alarm_after, self.hang_after, self.drift_after, self.drift_mm = alarm_after, hang_after, drift_after, drift_mm
        self.out, self.received, self.state = [], [], "Idle"
        self.feed_hold = self.reset = False

    def write(self, data: bytes):
        self.received.append(data)
        if data == b"!":
            self.feed_hold, self.state = True, "Hold"
            return
        if data == b"\x18":
            self.reset, self.flood, self.spindle, self.state = True, False, False, "Alarm"
            return
        if data == b"~":
            self.state = "Idle"
            return
        if self.hang_after is not None and self.n >= self.hang_after:
            return
        if data == b"?":
            drift = self.drift_mm if self.drift_after is not None and self.n >= self.drift_after else 0.0
            acc = ("S" if self.spindle else "") + ("F" if self.flood else "")
            self.out.append(f"<{self.state}|MPos:{_f(self.pos[0] + drift)},{_f(self.pos[1])},{_f(self.pos[2])}|FS:0,0"
                            + (f"|A:{acc}" if acc else "") + ">")
            return
        line = data.decode().strip()
        self.n += 1
        if self.alarm_after is not None and self.n >= self.alarm_after:
            self.state = "Alarm"
            self.out.append("ALARM:1")
            return
        code = line.split("(")[0].strip()
        for axis, k in (("X", 0), ("Y", 1), ("Z", 2)):
            m = re.search(axis + r"(-?[\d.]+)", code)
            if m and code.startswith("G1"):
                self.pos[k] = float(m.group(1))
        if code.startswith("M8"):
            self.flood = True
        elif code.startswith("M9"):
            self.flood = False
        elif code.startswith("M3"):
            self.spindle = True
        elif code.startswith("M5"):
            self.spindle = False
        self.out.append("ok")

    def readline(self, timeout=0.05):
        if self.out:
            return self.out.pop(0)
        time.sleep(min(timeout, 0.01))
        return None


# ------------------------------------------------------------------ the streamer
_STATUS = re.compile(r"<(\w+)[^|]*\|MPos:(-?[\d.]+),(-?[\d.]+),(-?[\d.]+)(.*)>")


class Streamer:
    """Send a program line by line, the supervisor checking each command first and the controller's status as heartbeat.

    ``on_station(k, line)`` is called at each ``M0`` (operator re-parks and registers the base); return False to stop.
    Returns the run record: completed or stopped, why, the lines acknowledged, and the as-run status samples.
    """

    def __init__(self, transport, supervisor, machine, job, clock=time.monotonic, on_station=None, watchdog=None,
                 operator=None):
        self.t, self.sup, self.m, self.job, self.clock = transport, supervisor, machine, job, clock
        self.watchdog, self.operator = watchdog, operator           # pump_watchdog.WatchdogLink, already armed
        self.on_station = on_station or (lambda k, line: True)
        self.samples, self.acked, self.log = [], 0, []
        self.last_seen = None
        self._next_poll = 0.0

    def _stop(self, reason):
        if self.watchdog is not None:
            self.watchdog.stop()                           # relay off and GRBL abort, independent of this port
        for data in (b"!", b"\x18"):                       # feed hold, then soft reset (outputs off on GRBL)
            try:
                self.t.write(data)
            except Exception:                              # a dead port must not stop us latching
                pass
        if not self.sup.latched:
            self.sup.trip(reason, now=self.clock())
        self.log.append({"t": self.clock(), "event": "safe_stop", "reason": reason})
        raise SafeStop(reason)

    def _bounds_ok(self, x, y, z):
        tol = self.m.position_tolerance_mm
        return all(lo - tol <= v <= hi + tol for v, (lo, hi) in
                   ((x, self.m.x_travel_mm), (y, self.m.y_travel_mm), (z, self.m.z_travel_mm)))

    def _handle(self, reply, waiting_ok, expected=None):
        now = self.clock()
        if reply.startswith("ALARM") or reply.startswith("error"):
            self._stop(f"controller reported {reply}")
        m = _STATUS.match(reply)
        if m:
            state, x, y, z, rest = m.group(1), float(m.group(2)), float(m.group(3)), float(m.group(4)), m.group(5)
            self.last_seen = now
            self.sup.heartbeat(now=now)
            acc = re.search(r"\|A:([A-Z]*)", rest)
            self.samples.append({"t": round(now, 3), "state": state, "x": x, "y": y, "z": z,
                                 "outputs": acc.group(1) if acc else "", "station": self.station})
            if state == "Alarm":
                self._stop("controller is in Alarm")
            if not self._bounds_ok(x, y, z):
                self._stop(f"controller reports ({x:.1f}, {y:.1f}, {z:.1f}) mm, outside the travel the program uses: "
                           "lost steps, a crash, or the wrong machine file")
            if expected is not None and state == "Idle" and math.dist((x, y, z), expected) > self.m.position_tolerance_mm:
                self._stop(f"idle at ({x:.1f}, {y:.1f}, {z:.1f}) mm but the last line ended at "
                           f"({expected[0]:.1f}, {expected[1]:.1f}, {expected[2]:.1f}): position lost")
            return False
        if reply == "ok":
            self.last_seen = now
            self.sup.heartbeat(now=now)
            return waiting_ok
        return False

    def _beat_watchdog(self):
        if self.watchdog is not None:
            try:
                self.watchdog.beat()
            except SafeStop as stop:
                self._stop(str(stop))

    def _wait_ok(self, expected=None):
        while True:
            self._beat_watchdog()
            now = self.clock()
            if now >= self._next_poll:
                self.t.write(b"?")
                self._next_poll = now + self.m.status_period_s
            if self.last_seen is not None and now - self.last_seen > self.m.heartbeat_timeout_s:
                self._stop(f"controller silent for {now - self.last_seen:.2f}s (limit {self.m.heartbeat_timeout_s:.2f}s)")
            reply = self.t.readline(timeout=0.02)
            if reply and self._handle(reply, True, expected):
                return

    def run(self, program):
        self.station = None
        self.dry_run = program["dry_run"]
        self.last_seen = self.clock()
        cmds = self.job["commands"]
        last_target = None
        try:
            for n, ln in enumerate(program["lines"]):
                for j in ln.commands:                  # the supervisor sees the wall-plane commands before the machine does
                    try:
                        self.sup.check(cmds[j], now=self.clock())
                    except SafeStop as stop:
                        self._stop(f"supervisor refused command {j}: {stop}")
                if ln.kind == "station":
                    self.station = ln.station
                    self._wait_idle(last_target)
                    self.log.append({"t": self.clock(), "event": "station", "station": ln.station})
                    if self.watchdog is not None:
                        self.watchdog.pause()              # pump power off while a person moves the base: not a fault
                    if not self.on_station(ln.station, ln.text):
                        self._stop(f"operator did not confirm station {ln.station}")
                    if self.watchdog is not None:
                        try:
                            self.watchdog.arm(self.operator)
                        except SafeStop as stop:
                            self._stop(str(stop))
                    self.last_seen = self.clock()
                    continue
                if ln.kind == "comment" and ln.text.startswith("("):
                    continue
                self.t.write((ln.text + "\n").encode())
                self._wait_ok()
                self.acked = n + 1
                if ln.target and None not in ln.target:
                    last_target = ln.target
            self._wait_idle(last_target)
            return self.record(True, None)
        except SafeStop as stop:
            return self.record(False, str(stop))

    def _wait_idle(self, expected, limit_s=600.0):
        """Block until the controller reports Idle (the planner has drained), checking the position it stops at."""
        t0 = self.clock()
        while self.clock() - t0 < limit_s:
            self._beat_watchdog()
            now = self.clock()
            if now >= self._next_poll:
                self.t.write(b"?")
                self._next_poll = now + self.m.status_period_s
            if self.last_seen is not None and now - self.last_seen > self.m.heartbeat_timeout_s:
                self._stop(f"controller silent for {now - self.last_seen:.2f}s (limit {self.m.heartbeat_timeout_s:.2f}s)")
            reply = self.t.readline(timeout=0.02)
            if reply:
                self._handle(reply, False, expected)
                if reply.startswith("<Idle"):
                    return
        self._stop(f"controller never went idle in {limit_s:.0f}s")

    def record(self, completed, reason):
        return {"completed": completed, "reason": reason, "dry_run": getattr(self, "dry_run", None),
                "lines_acked": self.acked, "samples": self.samples,
                "log": self.log, "supervisor": self.sup.status()}


# ------------------------------------------------------------------ run records and the wet-run interlock
def _runs_file(runs_dir=None):
    d = Path(runs_dir) if runs_dir else RUNS_DIR
    d.mkdir(parents=True, exist_ok=True)
    return d / "runs.json"


def load_runs(runs_dir=None):
    f = _runs_file(runs_dir)
    return json.loads(f.read_text()) if f.exists() else []


def save_run(entry, runs_dir=None):
    runs = load_runs(runs_dir)
    runs.append(entry)
    _runs_file(runs_dir).write_text(json.dumps(runs, indent=1))
    return entry


def wet_run_allowed(program, machine, runs_dir=None, commissioning_record=None, test_panel=False, wall_id=None,
                    today=None, accept_sim=False):
    """Reasons a wet run of ``program`` must not start (empty list = allowed). ``test_panel``: the commissioning
    test sheet, which needs everything but its own result."""
    import commissioning
    reasons = []
    today = today or time.strftime("%Y-%m-%d")
    env = program_envelope(program)
    cuts = [r for r in load_runs(runs_dir) if r.get("dry_run") and r.get("completed") and (accept_sim or not r.get("sim"))
            and r.get("machine") == machine.name]
    same = any(r.get("hash") == program["hash"] for r in cuts)
    # A touch-up is a new program every round; it may run without its own air-cut only if all of its motion lies
    # inside what an air-cut of the same wall, with the same tool (another tool sticks out differently), on the same
    # day (the base has not moved off it), already swept.
    covered = wall_id is not None and any(r.get("wall_id") == wall_id and str(r.get("date", "")).startswith(today)
                                          and r.get("tool") == program.get("tool")
                                          and _inside(env, r.get("envelope") or {}) for r in cuts)
    if not (same or covered):
        reasons.append(f"no completed air-cut of this program ({program['hash']}) on {machine.name}, and none today of "
                       f"wall {wall_id!r} that contains its motion: run it with --dry-run first")
    ready = commissioning.ready(machine.name, for_="test_panel" if test_panel else "wall", record=commissioning_record,
                                platform=machine.platform)
    if not ready["ready"]:
        reasons += [f"commissioning: {r}" for r in ready["missing"]]
    return reasons


def as_run_job(job, run, machine):
    """The job the machine actually executed, from its status samples, in wall-plane mm for ``hardware.rasterize``.

    A sample counts as tool-on when the controller reported the tool's output (A:F / A:S) or, for contact tools, when
    Y was within 1 mm of working depth; samples taken in an air-cut never paint.
    """
    reach = hardware.PLATFORMS[job["platform"]].reach_x_mm
    contact = job["tool"]["contact"]
    y_contact = (machine.y_wall_mm or 0.0) + (roller_press_mm(job["tool"]) if job["tool"]["name"] == "roller"
                                              else machine.press_mm.get(job["tool"]["name"], 2.0))
    commands = []
    for s in run["samples"]:
        if s.get("station") is None:
            continue
        on = (s["y"] >= y_contact - 1.0) if contact else bool(s.get("outputs"))
        commands.append({"x": s["x"] - machine.x_origin_mm + s["station"] * reach, "y": s["z"] + machine.z_offset_mm, "t": s["t"],
                         "tool": job["tool"]["name"] if on else "travel", "paint": on and not run.get("dry_run")})
    return {**job, "commands": commands, "as_run": True}


def main():
    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
    sub = ap.add_subparsers(dest="cmd", required=True)
    for name in ("gcode", "stream"):
        p = sub.add_parser(name)
        p.add_argument("--job", required=True, help="hardware.py --out job JSON")
        p.add_argument("--machine", help="machine JSON (Machine fields); commissioning values are merged in")
        p.add_argument("--dry-run", action="store_true", help="air-cut: all outputs off, contact tools held off the wall")
    sub.choices["gcode"].add_argument("--out")
    s = sub.choices["stream"]
    s.add_argument("--port")
    s.add_argument("--sim", action="store_true", help="stream to the simulated GRBL")
    s.add_argument("--gate", help="safety_gate decision JSON for the wall (required to arm)")
    s.add_argument("--runs-dir")
    s.add_argument("--test-panel", action="store_true", help="this job is the commissioning test sheet")
    s.add_argument("--watchdog-port", help="serial port of the pump watchdog board (firmware/pump_watchdog)")
    s.add_argument("--operator", help="who is at the machine (arms the pump watchdog)")
    a = ap.parse_args()

    import commissioning
    job = json.loads(Path(a.job).read_text())
    machine = commissioning.machine_with_record(Machine.load(a.machine))
    program = build_program(job, machine, dry_run=a.dry_run)
    for w in program["warnings"]:
        print("WARNING:", w)
    if a.cmd == "gcode":
        text = to_gcode(program)
        if a.out:
            Path(a.out).write_text(text)
            print(f"{len(program['lines'])} lines -> {a.out} (program {program['hash']}{', air-cut' if a.dry_run else ''})")
        else:
            print(text)
        return
    wall_id = job["commands"][0].get("wall_id") if job["commands"] else None
    if not a.dry_run:
        reasons = wet_run_allowed(program, machine, a.runs_dir, test_panel=a.test_panel, wall_id=wall_id)
        if not (a.watchdog_port or a.sim):
            reasons.append("a wet run needs the pump watchdog board (--watchdog-port)")
        if reasons:
            print("refused:\n  " + "\n  ".join(reasons))
            raise SystemExit(2)
    decision = json.loads(Path(a.gate).read_text()) if a.gate else {"decision": "none", "reasons": ["no --gate decision"]}
    width = max(c["x"] for c in job["commands"]) if job["commands"] else 0.0
    height = max(c["y"] for c in job["commands"]) if job["commands"] else 0.0
    sup = hardware.supervisor_for(width, height, job["platform"], limits={"heartbeat_timeout_s": machine.heartbeat_timeout_s})
    sup.arm(decision, wall_id=job["commands"][0].get("wall_id") if job["commands"] else None, now=time.monotonic())
    transport = (SimGrbl(home=(machine.x_travel_mm[0], machine.y_retract_mm, machine.z_travel_mm[0])) if a.sim
                 else SerialTransport(a.port))

    def confirm(k, text):
        return input(f"{text}\nType 'ok' when the base is parked and registered: ").strip().lower() == "ok"

    import pump_watchdog
    watchdog = None
    if a.watchdog_port or (a.sim and not a.dry_run):
        wd = pump_watchdog.SimWatchdog() if a.sim else SerialTransport(a.watchdog_port)
        watchdog = pump_watchdog.WatchdogLink(wd)
        watchdog.arm(a.operator or ("sim" if a.sim else ""))
    run = Streamer(transport, sup, machine, job, on_station=(lambda k, t: True) if a.sim else confirm,
                   watchdog=watchdog, operator=a.operator or "sim").run(program)
    if watchdog is not None and run["completed"]:
        watchdog.pause()
    entry = {"machine": machine.name, "hash": program["hash"], "dry_run": a.dry_run, "completed": run["completed"],
             "wall_id": wall_id, "envelope": program_envelope(program), "tool": program["tool"],
             "reason": run["reason"], "date": time.strftime("%Y-%m-%dT%H:%M:%S"), "sim": a.sim, "job": str(a.job),
             "samples": len(run["samples"])}
    save_run(entry, a.runs_dir)
    print(json.dumps(entry, indent=1))
    raise SystemExit(0 if run["completed"] else 1)


if __name__ == "__main__":
    main()