BMW_E8x_EPS/eps-comms/findings.md

# EPS communication findings — running log

Newest entries at the top. Each entry: date, what was tested, what we saw,
what it means, what's still open.

For the consolidated result — how to run the EPS standalone — see
[EPS_PROTOCOL.md](EPS_PROTOCOL.md). This file is the working history.

---

## 2026-08-29 (evening) — minimum message set found: 0x130 + 0x1A0

**Result**: the EPS needs only **two** car messages, not the 69 the bus
carries: `0x130` CAS terminal status brings it up, `0x1A0` DSC road speed
sets the assist level. Plus the 12V enable on a discrete wire. Confirmed on
the bench by narrowing the transmit set.

**Two bugs found along the way, both mine, both looked like EPS faults:**

1. *Frozen counters.* A captured payload retransmitted verbatim leaves its
   alive counter stuck, which every consumer treats as a stale sender. This
   is why replay worked and static transmission didn't. Fixed by cycling the
   captured payload **sequence** per ID rather than one frozen frame.
2. *Transmit rate starvation.* `read_frame()` used a blocking
   `serial.read(64)` that could stall the transmit loop for up to 100 ms,
   so the bench managed 264 frames/s against the ~1330 the full set needs -
   every message arriving late, indistinguishable from a faulty sender.
   Fixed by reading only buffered bytes and batching writes: 264 -> 1282
   frames/s standalone. The two-message set needs just 60 frames/s, which
   removes the problem entirely.

**Checksums remain unsolved.** Brute-forced sum/one's-complement/XOR over
every byte range, all constants, with and without counter contribution:
`0x1A0` b7 best fit 87.5% (fold(b2..b6) + 0x22), `0x130` b4 high nibble best
fit 64%. Not good enough to generate frames blind. Workaround in use:
replay genuine captured frames per terminal state, and patch only the speed
field of `0x1A0` (bytes 0-1, which the best-fitting checksum range excludes).
See EPS_PROTOCOL.md §6.

**EPS output messages decoded**: `0x1FB` is a pure alive counter (b0 low
nibble, 0-14, high nibble 0xF) - the best health signal available; `0x4B0`
is a status heartbeat with a likely checksum in b0; `0x5B0` has exactly two
payloads, `01 03 80…` (initialising) and `40 81 01 15…` (ready).

**Open**: assist has only been observed at 0 km/h, so the speed response
curve is unverified; behaviour on CAN loss at speed is untested; whether
`0x45` is needed or `0x41` suffices is unknown. Full list in
EPS_PROTOCOL.md §9.

---

## 2026-08-29 — clean isolated-bus test (post-fix), EPS's own IDs identified, first ruleset

**Test conditions**: same sequence as before, firmware TX-echo bug fixed
first. Engine actually had to be started this time to get the EPS working;
the dashboard EPS fault was intermittent even then. Nothing blocked yet -
fully transparent relay. Log: `captures/gateway_20260829_164540.csv` (84s,
60643 `car->eps` frames, only **37** `eps->car` frames - the fix worked).

**EPS's own traffic, isolated and confirmed for the first time**:
- `0x100` - our own CAN-IO board status (not the EPS - a test artifact, drop
  before any real vehicle use).
- `0x1FB` (2 bytes) - genuinely the EPS module's own alive/counter frame.
  Previously mislabeled "counter + checksum, unidentified module" in
  `files/PTCAN_protocol.md`, which was written from a whole-bus capture -
  now confirmed it's the EPS.
- `0x4B0` (8 bytes, `XX 01 FE FF FF FF FF FF`) - the EPS's own module-status
  heartbeat. Byte 0 toggled between `0x00` and `0x30` - possibly a state or
  fault-flag nibble worth watching once we can correlate it with the
  cluster's fault indicator turning on/off.

**Intermittent fault - two live hypotheses, not yet distinguished**:
1. *Missing/incorrect input*: the EPS needs something we're not feeding it,
   or feeding it in a state that only appears with the engine actually
   running (e.g. a DME "engine actually running" flag vs. just "cranking
   allowed", or a voltage/RPM value only valid once running).
2. *Relay latency/jitter*: this is a software (PC + 2x USB-serial) gateway,
   which adds tens of ms of jitter per hop on top of whatever the real bus
   had. 10ms-cycle messages like `0x0C4` (steering angle) arriving late
   often enough could plausibly trip an EPS freshness/plausibility check
   intermittently. **This is a real architectural limit of a PC-mediated
   gateway** - if it turns out to be the cause, the fix is a dedicated
   hardware relay/generator (no PC round-trip) once we know the required
   message set, not more filtering.

Both are testable via replay (see below): replaying the known-working
`car->eps` sequence with **no live car** removes any car-side variability
and isolates the test to the gateway/EPS interaction. If the same
intermittent fault reproduces on replay, that points at (2) rather than (1).

**Digital IO was also recorded, indirectly**: the CAN-IO board reports its
own IN1-4/OUT1-2 state in its `0x100` status frame (on every change, plus a
1s heartbeat), which got logged like any other frame. Extracted with
`tools/extract_dio_timeline.py` - in this session, `IN1`/`OUT1` (the car's
12V signal and its repeat to the EPS) was high `3.539s -> 4.083s`, then
`21.794s -> 67.123s`, then off. Sparse (only 12 status samples in 84s) but
enough to know the on/off windows. `gateway/replay_to_bus.py --dio-log`
now replays this alongside the CAN traffic - **replaying CAN frames alone
would leave the EPS never seeing the 12V signal at all**, since a bench
replay has no real car to drive that input.

**Ruleset**: first best-guess filter saved to `gateway/rules.suggested.json`
(app's combined format) and `gateway/rules.suggested_car_to_eps.json` (plain
format for `replay_to_bus.py`/`FilterRules`). Blocks only what's almost
certainly irrelevant to EPS assist function - see table below. Verified by
dry-run replay: skips exactly the expected 3084 of 60643 frames.

| Blocked ID | Name | Why |
|---|---|---|
| 0x380 | VIN tail | Not needed for operation, and shouldn't be replayed/shared anyway |
| 0x1D6 | MFL steering wheel buttons | Unrelated function (audio/cruise control) |
| 0x1D0 | DME temps + fuel | Unrelated to steering assist |
| 0x1B4 | Instrument cluster (warning lamps) | Cluster's own broadcast; EPS is a lamp *source*, not consumer, of this one |
| 0x480, 0x492, 0x497, 0x4A9 | Other modules' generic status heartbeats | Not `0x4B0` - that one is the EPS's own, kept out of this list on purpose |
| 0x580, 0x592, 0x5A9, 0x5C0 | Diagnostic / ISO-TP session frames | Not part of normal operation |

Left allowed (either likely needed, or not confident enough to block yet):
steering angle/rate (`0x0C4`/`0x0C8`), road speed (`0x1A0`), wheel speeds
(`0x0CE`), DME engine speed/voltage/torque (`0xAA`/`0xA9`/`0xA8`), CAS
terminal (`0x130`), DSC status/counter/accumulator (`0x19E`/`0x0B6`/`0x1A6`),
`0x1B6` (flagged in the protocol notes as a possible steering-torque
signal - do not block without testing specifically), and everything still
unidentified.

**Next steps**:
1. Replay `captures/replay_car_to_eps_20260829.csv` (the exact car->eps
   traffic from this working session) onto the EPS bus alone, car
   disconnected, with `gateway/rules.suggested_car_to_eps.json` applied and
   `--dio-log captures/dio_20260829.csv` so the 12V signal repeat is
   reproduced too. Confirm the EPS still comes up the same way from replay
   as it did live.
2. If the intermittent fault reproduces identically on replay, that's
   evidence for the jitter hypothesis over a missing-message hypothesis.
3. Iterate the ruleset: block one more plausible-non-essential group at a
   time (start with `0x1B4`'s replacement candidates, `0x0B6`, `0x1A6`),
   re-replay, watch for the fault changing character.
4. Once a minimal set is confirmed stable, start the synthesis plan (see
   README.md) - a generator that produces just those IDs from scratch.

---

## 2026-08-29 — first gateway sessions, discovered a firmware bug (not a bus-sharing issue)


**Test conditions**: car + EPS both wired through the gateway (CANdapter on
car PT-CAN, CAN-IO board on the EPS's own bus). Two sessions:
- `gateway_20260829_162119.csv` (~21s) - no `eps->car` traffic at all.
- `gateway_20260829_162235.csv` (~227s) - ignition on, engine off, then car
  fully off. EPS "started working" with ignition on (but showed its usual
  dashboard error - apparently normal without the engine running) and did
  not start with the car fully off.

**What the data showed**: in the second session, 69 unique arbitration IDs
appeared in the `eps->car` direction; 64 of them were IDs the gateway had
*just* relayed `car->eps` moments earlier (recognisable car ECU messages:
`0xA8/0xA9/0xAA` DME, `0xC4/0xC8` SZL steering, `0xCE/0x19E/0x1A0` DSC,
`0x130` CAS). Only `0x100` (the board's own status frame) was genuinely
native to the EPS bus.

**Initial hypothesis (wrong)**: that the car and EPS buses were still
electrically the same bus. Ruled out - confirmed the EPS bus is physically
isolated, and the car/EPS ports used in the test were correct and distinct
(`/dev/cu.usbserial-DNBJV4F5` vs `/dev/cu.usbmodem101`).

**Actual cause (fixed)**: a firmware bug. The CAN-IO board mirrors its own
CAN transmissions back over the USB bridge so the PC can see frames the
board originates locally (status/heartbeat), since TWAI has no RX loopback.
That mirroring was too broad - it also fired for frames the *gateway* asked
the board to relay from the car bus, so every relayed ID echoed straight
back over USB looking exactly like new incoming EPS-bus traffic, which the
Python gateway then dutifully relayed back to the car bus (a real feedback
loop, not just a logging artifact). Fixed by tagging TX-queue entries with
whether they originated on the board itself (mirror) or came from the
bridge (don't mirror) - see `can-io/firmware/src/can_bus.{h,cpp}` and
`usb_bridge.cpp`, `can_send(msg, mirror_on_success)`.

**Conclusion**: both sessions captured before this fix are unreliable for
answering "what does the EPS bus actually carry" - re-test needed.

**Next test to run**: repeat the ignition-on / engine-off / car-off sequence
now that the firmware is fixed, and check whether `eps->car` traffic now
shows only genuinely EPS-native messages (plausibly very little beyond
`0x100`, until the EPS module itself responds to something).

**Open questions**:
- What does the EPS module transmit on its own bus, if anything, once it's
  awake? (Could not tell from data so far - contaminated by the bug above.)
- Is a wake/terminal signal (e.g. the 12V line into the CAN-IO board's
  input) required in addition to CAN traffic, or does CAN alone bring the
  EPS up? Not yet tested independently.
- No messages have been blocked yet in `gateway_app.py` - full transparent
  relay only so far.