chatter/doc/Chatter_phase1_findings.md

# Chatter — Phase 1 On-Device Findings

**Date:** 2026-06-25
**Device:** reMarkable Paper Pro Move (RM03A, codename "Chiappa")
**Method:** Read-only SSH probing from Jatke (`ssh chatter`). No changes made to
the device except enabling developer mode / SSH (Phase 0) and adding our SSH key.

This document records what we learned and the implementation decisions those
findings support. It feeds the phase plan in
[`Chatter_implementation.md`](Chatter_implementation.md).

---

## 1. Hardware & OS

- **SoC / arch:** NXP **i.MX93**, **aarch64** (Arm Cortex-A55). Kernel
  `6.12.49+git-imx93-chiappa`.
- **OS:** Codex Linux 5.7.121 (Yocto **scarthgap**), image `3.27.1.0`. glibc
  (`/lib/ld-linux-aarch64.so.1`, `libc.so.6`).
- **RAM:** 2 GB (~1.4 GB free) — ample for our app.
- **Storage (matters for deployment):**
  - `/` (rootfs): ~435 MB, **only ~88 MB free (78% used)** — also an
    OS-managed overlay, likely reset by firmware updates. **Do not install here.**
  - `/home`: **46 GB encrypted volume, ~45.8 GB free.** Install Chatter under
    `/home` (e.g. `/home/root/chatter`).

## 2. Input devices

Pen and finger are **separate evdev devices** — finger-vs-stylus discrimination
is free, exactly what Chatter's gesture design needs.

| Node | Name | Role | Notes |
|---|---|---|---|
| `event0` | `bbnsm:pwrkey` | Power button | |
| `event1` | Hall effect sensors | Folio open/close | switch events |
| **`event2`** | **Elan marker input** | **Stylus / Marker** | on SPI |
| **`event3`** | **Elan touch input** | **Capacitive multitouch (finger)** | `INPUT_PROP_DIRECT` |

**Pen (`event2`)** — digitizer space **6760 × 11960**:
- Buttons: `BTN_TOOL_PEN`, **`BTN_TOOL_RUBBER`** (Marker Plus eraser end —
  flip-to-erase is detectable), `BTN_TOUCH`, `BTN_STYLUS`, `BTN_STYLUS2`.
- `ABS_PRESSURE` 0–4096, `ABS_DISTANCE` (hover) 0–65535, `ABS_TILT_X/Y` ±9000.

**Touch (`event3`)** — grid **1248 × 2208**:
- Up to **10 contacts** (`ABS_MT_SLOT` 0–9) with `ABS_MT_POSITION_X/Y`,
  `ABS_MT_PRESSURE` (0–255), `ABS_MT_TRACKING_ID`, `ABS_MT_TOOL_TYPE`.

All three coordinate spaces share the screen's portrait aspect (~0.5625):
panel 954×1696, pen 6760×11960, touch 1248×2208.

## 3. Display pipeline

- **DRM/KMS only — there is NO `/dev/fb*`** and `/sys/class/graphics` is empty.
  Driver is **`imx-drm`**; device `/dev/dri/card0`; connector
  **`card0-LVDS-1`** ("connected", "disabled" at rest — normal for e-ink, which
  holds its image without continuous scanout).
- The connector advertises a **packed `365×1700` mode** (hardware buffer for the
  E Ink Gallery 3 color subpixel layout); the logical screen is 954×1696. The
  mapping is handled by the epaper plugin (below) — we don't touch it directly.
- **Implication:** the `mmap /dev/fb0` approach used by rmkit / libreMarkable on
  older grayscale models **does not apply** to this device.

## 4. Graphics / UI stack — the key finding

- The stock app **`xochitl`** is a **Qt 6.8.2 / Qt Quick (QML)** application.
- It renders through a **custom Qt platform (QPA) plugin: `epaper`**
  (`/usr/lib/plugins/platforms/libepaper.so`, links `libdrm`). Available QPA
  plugins on-device: **`epaper`**, `minimal`, `offscreen`, `vnc`.
- Qt 6.8.2 runtime libraries and QML modules (`QtQuick`, `QtQuickControls2`,
  etc.) are all present under `/usr/lib` and `/usr/lib/qml`. No dev headers
  on-device (expected — we cross-build).

**This means the native path is wide open:** build Chatter as a **Qt 6 app** and
run it with **`-platform epaper`** to inherit working e-ink display + refresh,
instead of reverse-engineering DRM/KMS and waveform handling. This resolves the
project's highest-risk unknown in our favor.

## 5. Stock app & services

- `xochitl.service` — the main UI app (`/usr/bin/xochitl`, pid varies).
- `marker-manager.service` — "Remarkable CSL Marker Manager" (pen support).
- `rm-sync.service` — document sync to the reMarkable cloud.
- Plus metrics / MDM / crash-uploader services.

**Coexistence / toggle implication:** on a DRM/KMS device only one process owns
the display (DRM master) at a time, and xochitl holds it (and likely grabs the
input devices) while running. So the Chatter ↔ standard-GUI **toggle is a
hand-off** (stop or pause `xochitl` ↔ run Chatter, and a return path), **not an
overlay**. The exact mechanism (systemctl stop/start vs. SIGSTOP/SIGCONT) is to
be validated in the Phase 2 spike.

## 6. Document store (for the Save phase)

Location: **`/home/root/.local/share/remarkable/xochitl/`**. Per document:

- `UUID.metadata` — JSON: `visibleName`, `parent` (empty = top level; a folder
  is a `CollectionType` document whose UUID is used as children's `parent`),
  `type` (`DocumentType`), timestamps, `pinned`, etc.
- `UUID.content` — JSON: page list under `cPages.pages[]` (each page has an `id`,
  ordering `idx`, `template`, scroll position, CRDT-style `timestamp` fields).
- `UUID/` — directory of per-page **`<pageUUID>.rm`** files.
- `UUID.thumbnails/` — page thumbnails.
- Store-root extras (Codex additions): `.tree` (binary "rM sync tree" index) and
  `rm-search-index.db` (SQLite **search index**, not the canonical store).

**`.rm` page format:** header confirmed as **`reMarkable .lines file,
version=6`** — the v6 binary scene-tree format (documented by the community,
e.g. `rmscene`). Authoring valid v6 is possible but non-trivial.

**Save options (decide in Phase 5):**
1. **PDF/PNG-backed document** — render Chatter's canvas to PDF/PNG and create a
   document that references it (xochitl already supports PDF documents). Far
   simpler and robust; still a real, renamable/syncable library item.
2. **Native v6 `.rm`** — author the binary page format so the transcript is an
   editable notebook. More work; revisit only if option 1 proves insufficient.

Either way, transcripts go under a **`Chatter`** `CollectionType` folder, with
filenames derived from date/time.

---

## 7. Decisions supported by Phase 1

1. **Stack: C++ / Qt 6.8.2**, rendered via the on-device **`epaper` QPA plugin**.
   Cross-build with a Yocto **scarthgap** aarch64 SDK matching Qt 6.8.2 / the
   device glibc.
2. **Input:** pen from `event2` (including `BTN_TOOL_RUBBER` for flip-to-erase),
   finger from `event3` (10-pt). Confirm in Phase 2 whether Qt+epaper already
   delivers these or we read evdev directly for custom gestures.
3. **Toggle = display/input hand-off** with xochitl, not an overlay.
4. **Save = PDF-backed document** (leaning), in a `Chatter` folder.
5. **Deploy under `/home`,** never the rootfs.

## 8. Open items for the Phase 2 spike

- Stand up the cross-SDK and build a trivial Qt app; run it with
  `-platform epaper` **after stopping xochitl**; confirm it draws and refreshes.
- Verify how input arrives in a Qt app under epaper (Qt event stream vs. raw
  evdev) and how reliably finger and pen separate at the Qt layer.
- Validate the toggle hand-off (stop/start vs. stop/cont) and a return path.