Skip to content

Device: run inside PyUI's process instead of launch.sh #29

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
abalakh merged 1 commit into from
Oct 30, 2026
Merged

Device: run inside PyUI's process instead of launch.sh #29

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 16 additions & 12 deletions .agents/code.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ in an upstream PR.
│ core/ models · storage · ra_client · sync · settings ·
│ credentials · screenshots · proxy · local_games
├─────────────────────────────────────────────────────────────┤
│ platform/ paths · env detection · device bootstrap · desktop runner (dev only)
│ platform/ paths · env detection · ROM identification · desktop runner (dev only)
└─────────────────────────────────────────────────────────────┘
```

Expand All @@ -44,7 +44,8 @@ in an upstream PR.

```
src/cheevos/
__main__.py device entry point: python -m cheevos
native.py the entry point PyUI calls (config.json "pyuiEntry"): one session
inside PyUI's process, and everything it puts into PyUI taken out again
app.py composition root: credentials, caches, sync, media, then home
core/
models.py dataclasses: UserProfile, GameProgress, Achievement, Award, Unlock, ...
Expand Down Expand Up @@ -72,8 +73,10 @@ src/cheevos/
net.py connectivity check through the shared transport
clock.py shared RA Date + monotonic app clock (system clock unchanged)
ui/
pyui/ bridge: bootstrap, views (lists, grids, carousel, popups), primitives
(drawing, input), text (fitting, glyph fallbacks), status_bar,
pyui/ bridge: host (what a session borrows from PyUI: textures,
messages, Spruce's settings), views (lists, grids, carousel,
popups), primitives (drawing, input), text (fitting, glyph
fallbacks), status_bar,
bar_layout, glyphs, generated (PNG swatches, button glyphs),
award_images (static awards), row_bars, bar_colors, image_frames
(shared outlines), badge_grid (compact badge grid), carousel (the
Expand All @@ -87,23 +90,23 @@ src/cheevos/
strings.py every user-facing string (ready for PyUI Language later)
platform/
paths.py every path, from CHEEVOS_SDCARD_ROOT (default /mnt/SDCARD)
arguments.py the --game-id argument shared by the launcher and desktop runner
identify_rom.py the identification helper's entry point (identify-rom.sh runs it)
curl_transport.py the helper's RA requests through Spruce's curl, Python's HTTPS as
the fallback
proxy_hasher.py adapter over RAOfflineProxy's rom_hashing wrapper (its only import
from the proxy)
desktop/ dev only: runner CLI on PyUI's DESKTOP device, fixture/live
environments, drills (showcase_awards: the showcase's event and site
badges)
desktop/ dev only: runner CLI on PyUI's DESKTOP device (pyui_setup brings
PyUI up), fixture/live environments, drills (showcase_awards: the
showcase's event and site badges)
res/icons/ original outlines: 24/48 px status, 96/144 px lists and fallbacks
res/awards/ pre-rendered awards: dark/light and black/white contrast variants
app/ packaging: config.json, launch.sh, identify-rom.sh, cheevos.png
app/ packaging: config.json, identify-rom.sh, cheevos.png
assets/ icon sources (SVG)
tests/ unit and screen tests, fixtures/
scripts/ check_conventions, record_fixtures, fetch_dev_media, fetch_pyui.sh,
build_package, release_notes, render_icons, make_synthetic_cache,
doc_screens, device.sh, fetch_proxy.sh, check_proxy_compat (with
doc_screens, device.sh (with device_session: make launch's
dev-only session), fetch_proxy.sh, check_proxy_compat (with
proxy_native: ELF exports, a simulated native hasher)
docs/ the user guide, published as the GitHub wiki (images/ from doc_screens)
.agents/ docs for coding agents, by area (AGENTS.md is the entry point)
Expand Down Expand Up @@ -141,7 +144,8 @@ On the device, everything is assembled into `/mnt/SDCARD/App/Cheevos/` ([device.
`make check` runs all of these locally; `make fmt` formats and auto-fixes. Per-file ignores worth
knowing about:
- `PLC0415` (import outside top level) is allowed in `cheevos/ui/pyui/**` and the desktop
`__main__`, because PyUI is importable only after `add_pyui_to_path()`.
`__main__`, because PyUI is importable only inside PyUI's process or after the desktop
runner's `add_pyui_to_path()`.
- `T20` (print) is allowed in `scripts/`.
- Tests skip docstrings and annotations, and allow `assert`, magic numbers, subprocesses, many
arguments and boolean flags.
Expand Down Expand Up @@ -247,7 +251,7 @@ Tooling gotchas:

How to run all of this: [TESTING.md](../TESTING.md).

## Shell (`launch.sh`, scripts)
## Shell (`identify-rom.sh`, scripts)
- POSIX `sh`, compatible with busybox ash, checked with `shellcheck -s sh`.
- Start by sourcing `/mnt/SDCARD/spruce/scripts/helperFunctions.sh` and `appEnv.sh`. Read only
`SPRUCE_*` / `PLATFORM` variables from them.
Expand Down
105 changes: 47 additions & 58 deletions .agents/device.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,9 @@ always apply. The device commands are in [TESTING.md](../TESTING.md).

```
/mnt/SDCARD/App/Cheevos/
config.json {"label": "Cheevos", "icon": "cheevos.png", "launch": "launch.sh",
"description": "RetroAchievements hub"} (no "devices": every device)
launch.sh sources helperFunctions.sh, sets the per-platform SDL env (mirroring PyUI's
launcher), runs Spruce's Python with -m cheevos; stderr to /tmp
config.json {"label": "Cheevos", "icon": "cheevos.png", "pyuiEntry": "cheevos.native:run",
"description": "RetroAchievements hub", "gameProgress": 2}
(no "launch": PyUI runs it in its own process; no "devices": every device)
identify-rom.sh Spruce's game menu runs it to identify a ROM: sources appEnv.sh and execs
-m cheevos.platform.identify_rom, headless ([integration.md](integration.md))
cheevos.png app icon
Expand All @@ -32,22 +31,21 @@ always apply. The device commands are in [TESTING.md](../TESTING.md).
```

- `make package` builds it into `dist/App/Cheevos/` (no desktop runner, no tests) and zips that
as `dist/Cheevos-<version>.zip`, with the `Cheevos/` prefix and no `App/` folder. Both shell
scripts are executable: Spruce runs them directly.
- `config.json`'s `"gameProgress"` tells Spruce what the package supports: 1 for
`launch.sh --game-id N`, 2 when `identify-rom.sh` ships too. Raise it only with a new contract
as `dist/Cheevos-<version>.zip`, with the `Cheevos/` prefix and no `App/` folder.
`identify-rom.sh` is executable: Spruce runs it directly.
- The Apps launcher discovers any `App/*/config.json`. With `"pyuiEntry"`, PyUI runs Cheevos
inside its own process and unloads it when it closes ([pyui.md](pyui.md), "In-process
sessions"); it needs a SpruceOS whose PyUI has `apps/native_app.py`. An app with no `"launch"`
is known to PyUI's hidden-apps list by its folder.
- `config.json`'s `"gameProgress"` tells Spruce what the package supports: 1 for opening one
game (`run(game_id=N)`), 2 when `identify-rom.sh` ships too. Raise it only with a new contract
Spruce checks for.
- The Apps launcher discovers any `App/*/config.json`. PyUI exits fully while the app runs and
restarts when it quits.
- **No `devices` list:** a `devices` list in an app's `config.json` hides it on every device not
named there. Cheevos ships without one, so the Apps menu shows it on every device.
`launch.sh` covers every platform PyUI's launcher does.
- **Shared UI across devices:** use the same screens and application logic on every platform,
with platform setup handled by the launcher and PyUI bridge.
- **Start-up failures:** when the app exits non-zero, `launch.sh` appends its stderr to the app
log and shows a message ([product.md](product.md), "Offline and error behaviour").
`/mnt/SDCARD/App/PyUI/launch.sh -msgDisplay "text" -msgDisplayTimeMs 5000` shows a message on
any platform.
named there. Cheevos ships without one, so the Apps menu shows it on every device: it runs
wherever PyUI does, with no platform set-up of its own.
- **Failures:** `cheevos.native.run` logs an error and shows a message; PyUI does the same for
one it raises before Cheevos's code runs ([product.md](product.md), "Offline and error
behaviour").
- The release is that zip; users copy its `Cheevos` folder into `App` on the SD card. Pushing a
`v<version>` tag publishes it as a GitHub release (the `release` job in `ci.yml`; steps in
[RELEASING.md](../RELEASING.md), under "Releases"). The description links merged PR titles
Expand All @@ -60,11 +58,14 @@ always apply. The device commands are in [TESTING.md](../TESTING.md).
that neither Cheevos nor a game is running first.
- **Stray taps:** once Cheevos exits (B on its home screen), further taps reach Spruce's menu,
where A starts a game. A blind tap loop once launched a game this way. `device.sh press` now
refuses on the device unless Cheevos is running, and stops at the first refusal. Still,
take a screenshot between screens rather than sending long sequences.
refuses on the device unless Cheevos is running, and stops at the first refusal. Opened from
the menus, Cheevos has no process of its own: `device.sh running` (which `press` uses) looks
for PyUI holding the app's log open, or for a `make launch` session. In PyUI's own menus, A
on Settings' first row powers off. Still, take a screenshot between screens rather than
sending long sequences.
- **`pgrep -f` self-match:** a plain `pgrep -f` pattern also matches the remote shell's own
command line. Use the `[x]` trick (`'[c]heevos'`), and keep the plain text out of the rest of
the same remote script too: a later `grep "python3.10 -m cheevos..."` made an idle check
the same remote script too: a later `grep "python3.10 -m cheevos..."` once made an idle check
find "Cheevos" (the shell itself).
- **Backups must stay off device `/tmp`.** It is RAM-backed tmpfs. An 18 MB app backup
during integration testing caused memory pressure, thread creation failures and a device
Expand All @@ -80,16 +81,17 @@ always apply. The device commands are in [TESTING.md](../TESTING.md).
- **Input injection:** `send_event /dev/input/event0 CODE:1|0`. Codes: A=57, B=29, X=42,
Y=56, Start=28, Select=97, L1=18, R1=20, D-pad 103/108/105/106. Never send MENU (1), power
or combos.
- **Launching an app remotely:** write `/tmp/cmd_to_run.sh` (PyUI's own format), then send
SIGTERM to PyUI; `principal.sh` runs the command and restarts PyUI afterwards. This also
records autoresume, exactly as launching from the Apps menu does.
- **Running a command remotely:** write `/tmp/cmd_to_run.sh` (PyUI's own format), then send
SIGTERM to PyUI; `principal.sh` runs the command and restarts PyUI afterwards. It also
records the command for autoresume. `make launch` uses it ([pyui.md](pyui.md), "In-process
sessions").
- **Graphics memory:** SDL textures come from the MMA pool (`mma_heap=…sz=0x1500000`, 21 MB),
not from the RAM `MemAvailable` reports. The framebuffer and PyUI's screen-sized canvases
take about 11.5 MB of it, leaving about 9 MB for textures. Free space:
`/proc/mi_modules/mi_sys_mma/mma_heap_name0` (`chunk_mgr_avail`, hex). Big textures need
one contiguous block. See [pyui.md](pyui.md), "Texture caches never evict".
- **Log times are UTC** (marked `Z`): PyUI applies the saved time zone partway through
start-up, so local times would jump within one run.
- **Log times are UTC** (marked `Z`), comparable with logs from before the app ran inside
PyUI (which applies the saved time zone partway through its start-up).
- **PIDs wrap at 4096**, so a low PID doesn't mean an old process. Check
`/proc/<pid>/stat` against `/proc/uptime` for its age.
- **Network services:** Spruce restarts Samba and SFTPGo after every app exits; SSH (dropbear)
Expand All @@ -102,6 +104,10 @@ always apply. The device commands are in [TESTING.md](../TESTING.md).

The hardware: a dual-core Cortex-A7 at 1.2 GHz, 128 MB RAM (103 MB for Linux, the rest for the
graphics pool), slow SD writes. Big lists and image-heavy screens need a check on it.
- **Inside PyUI** (2026-10-30): the same memory as the separate process it used to be while
open, and ~6-7 MB kept by PyUI after it closes, flat from one session to the next; opens in
0.5 s ([pyui.md](pyui.md), "In-process sessions"). The figures below were taken when Cheevos
was a separate process (with PyUI exited).
- **Memory headroom is small.** Measured 2026-10-14 on Spruce 4.5.5 nightly with a 2,948-game
account: about 17 MB available on Home, with RAOfflineProxy's service at 30–36 MB beside
Cheevos (33–36 MB RSS). Browsing images fills the 4 MB `/tmp` image scratch, and the heap
Expand All @@ -112,43 +118,26 @@ graphics pool), slow SD writes. Big lists and image-heavy screens need a check o
- **CPU:** measure per thread from `/proc/<pid>/task/*/stat` (fields 14 + 15, 1/100 s ticks);
100 ticks a second is one core. Busybox `top` shows % of both cores together.

### Start-up
Measured 2026-10-14 (Spruce 4.5.5 nightly, PyUI `346b4092`, warm, 2,948-game account), before
[SpruceOS #1857](https://github.com/spruceUI/spruceOS/pull/1857): Home's first frame appeared
4.1–4.5 s after Python started, down from 4.6–4.9 s before Cheevos deferred its screens and
sync imports. The first start after an update also compiled every module, about 1.5 s more.
Python's imports took most of it (3.4 s): finding and reading each module from the SD card
costs a few ms before its code even runs, and each dataclass 5–7 ms to build. The upstream
factory and game-menu fixes have not yet been timed on a device.
### Opening
Inside PyUI (2026-10-30, Mini+, warm): A to Cheevos running 0.5 s, nearly all of it Python
importing the package from the SD card; Home's first frame 0.05 s later. The first open after an
update also compiles every module (3 s). As a separate process (2026-10-14, 2,948-game
account) Home appeared 4.1–4.5 s after Python started, 3.4 s of it imports: finding and reading
each module from the SD card costs a few ms before its code even runs, and each dataclass
5–7 ms to build.
- **Rule: import what Home's first frame doesn't need on first use.** Every screen but Home
(`home._open`), the badge grid and carousel (`views`), the RA client and transport (`app`'s
factories), the sync engine (on the sync thread, `background`) and `email.utils` (`clock`)
load when first used; `ra_client/__init__.py` imports nothing. Names the UI needs without
the engine live in `sync/options.py`; `Credentials` lives in `core/credentials.py`.
`tests/unit/test_startup_imports.py` fails if any of them loads with `cheevos.app`: add a
new screen or worker to its list. The modules that import on first use are listed in
`pyproject.toml`'s per-file `PLC0415` ignores. The cost moves to each screen's first
opening (Games +0.3 s, Profile +0.2 s, Awards +0.1 s the first time).
`tests/unit/test_startup_imports.py` fails if any of them loads with `cheevos.native` (what
PyUI imports): add a new screen or worker to its list. The modules that import on first use
are listed in `pyproject.toml`'s per-file `PLC0415` ignores. The cost moves to each screen's
first opening (Games +0.3 s, Profile +0.2 s, Awards +0.1 s the first time). Inside PyUI every
open imports the package again, since closing unloads it.
- **The auto-sync starts after Home's first frame** (`Home(sync_when_shown=...)`, on its
first tick): started before, the sync thread's imports competed with the first frame
(+0.3 s).
- **PyUI's upstream device factory:** `bootstrap.pyui_device` imports `initialize_device`
from `devices.device_factory` for every device, including `DESKTOP`. This avoids the
launcher imports without copying any device branches; errors inside the factory surface
as start-up errors. `tests/unit/test_ui_bootstrap.py` checks that importing it leaves the
launcher and its game menus unloaded.
- **PyUI's game menus load on first activity-icon lookup:** SpruceOS #1857 (merge commit
`a3c494b1`) moved the two menu imports into `MiyooTrimCommon.get_image_for_activity`, which
Cheevos never calls. On a Mac, after importing PyUI's shared UI modules, the Mini device
adds 48 modules instead of 141, and the Brick 43 instead of 137. The pre-fix device timing
put about 0.36 s in these menu imports; the new hardware saving remains unmeasured.
- **PyUI's Wi-Fi status is read on a thread during the display's start-up**
(`bootstrap._warm_wifi_status`): its first read runs `wifiCmd status` (0.4 s) and the top
bar draws it from the first frame.
- **PyUI's remaining share** in the pre-fix measurements: `display.display` imports pysdl2's
`sdl2.ext`, whose `resources` module imports `urllib.request` with `http.client`, `email`
and `ssl` (0.2 s). `Display.init` took about 1 s (0.63 s re-initialising, 0.32 s loading
fonts). Don't plant stub modules to skip them: Spruce updates PyUI without a Cheevos release.
- **ROM identification helper** (`identify-rom.sh`), measured 2026-10-18 on the Mini+ with
Spruce 4.5.5 and RAOfflineProxy 2.2.0-alpha1 nightly 54, warm, from the shell to the JSON:
a saved path 0.6-0.7 s (1.3 s on the first run, compiling bytecode): `appEnv.sh` 0.05 s,
Expand All @@ -173,6 +162,6 @@ factory and game-menu fixes have not yet been timed on a device.
(1.9-2.1 s either way). The first lookup after a reboot spent 11 s hashing and reading the
proxy's cache, against 0.25 s afterwards.
- **How it was measured:** a patched build (never shipped) logs the first frame's time from
the process start (`/proc/self/stat`), and can run `cProfile` up to it or start Python with
`-X importtime`. A launch with `-X` doesn't match `pgrep -f 'python3[.]10 -m [c]heevos'`
(`device.sh` misses it): stop it by PID.
the process start (`/proc/self/stat`), and can run `cProfile` up to it. `make launch` passes
`DEVICE_PYTHON_FLAGS` to PyUI's Python (e.g. `-X importtime`, written to
`/tmp/cheevos-dev/stderr.log`).
Loading
Loading