Skip to content

feat: rewrite integration (3.0.0) - #31

Draft
g4bri3lDev wants to merge 123 commits into
mainfrom
feat/clean-port
Draft

feat: rewrite integration (3.0.0)#31
g4bri3lDev wants to merge 123 commits into
mainfrom
feat/clean-port

Conversation

@g4bri3lDev

@g4bri3lDev g4bri3lDev commented May 24, 2026

Copy link
Copy Markdown
Member

Summary

3.0.0 is a ground-up rewrite. 2.x was built around the OpenEPaperLink access-point/tag and ATC protocols with the internal imagegen engine; 3.0 talks directly to OpenDisplay BLE devices via py-opendisplay and renders with odl-renderer. Many 2.x services carry over (reworked onto the new stack); OEPL and ATC support is removed. Breaking changes below.

New in 3.0

  • BLE OTA firmware install: in 2.x the firmware update entity could only show that an update was available; 3.0 adds installing it. Silabs EFR32BG22 devices are flashed over a direct BLE connection. nRF / ESP32 devices still show the available version but have no Install button (proxy-unreliable / no BLE OTA path)
  • upload_image service: uploads a media-source image or a direct image URL
  • Touch event entities for OpenDisplay touch controllers
  • activate_buzzer service (Flex)
  • write_nfc service: writes an NDEF record (URL, text, MIME, or Home Assistant tag) to a display's NFC chip
  • 7-color and dual-controller panel support: SEVEN_COLOR (Spectra/ACeP) panels and the reTerminal E1004 (BWGBRY_SPLIT, dual-controller Spectra 6)
  • Per-device BLE RSSI diagnostic sensor (disabled by default)
  • Per-device landing link for the "Visit device" button
  • Automatic re-sync on reboot: when a device reboots (e.g. after a firmware install or config change), the integration detects it from the advertised reboot flag and reloads to re-read the firmware version and configuration
  • Partial refresh: upload_image/drawcustom support flicker-free diff updates via the device's partial-region protocol, with automatic fallback to a full refresh
  • Deep-sleep delivery: content sent to a sleeping battery-powered device is queued and delivered on its next wake instead of being dropped; entry setup can run entirely from cache while the device is dark

Ported from 2.x (reworked onto the new stack)

  • drawcustom: now renders via odl-renderer with voluptuous schema validation at the service boundary
  • activate_led (was setled): redesigned with up to 3 independent RGB color steps, each with its own flash count and timing (Flex)
  • Firmware update entity: shows installed vs. latest GitHub-release version with release notes (display-only in 2.x; the new install capability is listed above)
  • Display content image entity: shows the last image rendered to each device, updated after every upload_image/drawcustom (including dry-run)
  • Font directory search: .ttf files in /config/www/fonts, /config/media/fonts, or /media/fonts, referenced by name in drawcustom
  • Diagnostic sensors: battery voltage/percentage, temperature, last-seen

Fixes

  • Upload error messages now include the underlying cause
  • BLE END ACK timeout raised to 90s, resolving false upload failures on slow displays
  • Buzzer command ID corrected (py-opendisplay 7.3.2)
  • Icon rendering uses Pillow native anchors (odl-renderer 0.5.9)
  • Accept legacy drawcustom field values from pre-3.0 configs
  • Defer BLE connect until discovery is confirmed
  • Drop board revision from the reported hardware version

Breaking changes

  • All drawcustom element types and field names follow the odl-renderer schema. See the odl-renderer docs
  • Removes OpenEPaperLink (OEPL) and ATC support

Requirements

  • Home Assistant 2026.4.0 or later
  • py-opendisplay[silabs-ota] 7.14.0, odl-renderer 0.5.12

Closes

Closes #27
Closes #28
Closes #29
Closes #40
Closes #43

Changelog by pre-release

3.0.0-beta.1

  • Replaced imagegen with odl-renderer
  • drawcustom service + service-boundary validation; font_dirs wired up
  • upload_image service
  • Touch event entities
  • Requires Home Assistant 2026.4.0+

3.0.0-beta.2

  • Firmware update entity (installed vs. latest GitHub release)
  • activate_led and activate_buzzer services (Flex)
  • RSSI and last-seen diagnostic sensors (all devices, disabled by default)
  • Fix: upload error messages include underlying details
  • Fix: BLE END ACK timeout raised to 90s

3.0.0-beta.3

  • Display content image entity (last rendered image, updates on dry-run too)
  • activate_led redesigned with 3 independent RGB color steps
  • Fix: buzzer command ID (py-opendisplay 7.3.2)
  • Fix: icon rendering via Pillow native anchors (odl-renderer 0.5.9)

3.0.0-beta.4

  • BLE OTA firmware install (Silabs EFR32BG22, direct-connection only; nRF intentionally not offered over a Bluetooth proxy)
  • upload_image accepts a direct image URL
  • Accept legacy drawcustom field values from pre-3.0 configs
  • Defer BLE connect until discovery is confirmed
  • Drop board revision from hardware version
  • Per-device landing link for the "Visit device" button
  • py-opendisplay bumped to 7.8.0

3.0.0-beta.5

3.0.0-beta.6

3.0.0-beta.7

  • drawcustom gains rotation, mirror (h/v/hv) and pivot on every element, plus lenient string handling for visible (odl-renderer 0.5.10)
  • use_measured_palette exposed to drawcustom; default dithering changed to Burkes
  • Partial refresh support (refresh_type: partial) for upload_image/drawcustom, with automatic fallback to full refresh
  • Deep-sleep delivery: content sent to a sleeping device is queued and delivered flicker-free on its next wake instead of being dropped; entry setup can run entirely from cache for a dark device; new sleep_mode/missed_cycles/queue_timeout options and "Update pending" sensor
  • Probe-before-queue: image sends to a probably-asleep device try one short connect before falling back to the queue
  • OTA install fails fast with a clear error when requested on a provably-asleep device
  • Per-MAC BLE lock serializes image/LED/buzzer/OTA operations on a tag
  • Image prepare (dither/encode/compress) now runs in the executor instead of blocking the event loop
  • Wake-drain deadline raised 30s → 600s; timeouts fail loudly instead of retrying silently; delivery gives up after 5 attempts
  • All BLE connect paths now bounded by a wall-clock deadline
  • OTA/config-flow/setup auth failures now trigger reauth instead of raising
  • Fix: last_seen sourced from the Bluetooth stack directly (was freezing)
  • New sliding-window transfer tuning options (blocks_per_ack, max_queue_size)
  • py-opendisplay bumped to 7.12.0, odl-renderer bumped to 0.5.12
  • Docs: supported_types.md links to the odl-renderer README instead of duplicating it; gray levels and YAML hex-quoting documented

3.0.0-beta.8

  • New write_nfc action: writes an NDEF record to a display's NFC chip. Record types: URL, plain text, MIME data (e.g. vCards, text/vcard default), and ha_tag, which writes a Home Assistant tag URL that the companion app auto-registers on first scan and that fires tag_scanned for automations
  • Requires NFC-capable Silabs firmware (2.2+, NFC endpoint 0x83) and NFC configured on the device; content up to 512 bytes; friendly errors for devices without NFC or with pre-NFC firmware
  • 7-color panel support: devices reporting the new SEVEN_COLOR scheme (Spectra/ACeP) now dither and upload correctly (colors uncalibrated until a panel is measured)
  • reTerminal E1004 support: BWGBRY_SPLIT scheme for dual-controller Spectra 6 panels
  • Large-upload reliability: PIPE_WRITE retransmit budget now scales with transfer size, fixing MAX_RETX aborts on big panels (e.g. ~960 KB E1004 uploads)
  • Fix: BLE notification-watcher leaks closed on all paths (py-opendisplay 7.13.1)
  • py-opendisplay bumped to 7.14.0 (NFC write, protocol-v2 color schemes, scaled retransmit budget)

3.0.0-beta.9

g4bri3lDev added 29 commits May 24, 2026 14:12
… odl-renderer, bump py-opendisplay to 7.2.5)
Pass HA font search directories (/config/www/fonts, /config/media/fonts,
/media/fonts) to generate_image() so users can reference custom fonts by
name without absolute paths. Requires odl-renderer 0.5.8 which adds the
font_dirs parameter to FontManager and generate_image().
- SCHEMA_DRAWCUSTOM validates payload, background, rotate, dither,
  refresh_type and dry-run at the service boundary rather than at
  runtime inside the renderer
- _async_send_image centralises BLE connection, encryption key parsing
  and error handling; used by both upload_image and drawcustom
ImageGen was replaced by odl-renderer; rendering logic is now tested
in the odl-renderer package itself.
Delete empty tests/ dir, requirements_test.txt, tests workflow, and
pytest config now that all drawcustom tests have been removed.
Remove .claude/, .run/, ha_data_provider.md, and uv.lock from
version control; add them to .gitignore.
The upload_error exception swallowed the original OpenDisplayError message,
making it hard to diagnose failures. Pass str(err) as a translation placeholder
so the underlying cause is shown in the HA notification.
Both sensors are diagnostic and disabled by default. All devices now
load the sensor platform (moved Platform.SENSOR to _BASE_PLATFORMS),
so non-flex devices get temperature, RSSI, and last-seen; battery
sensors remain gated on power mode as before.

The sensor value_fn signature is widened from AdvertisementData to
OpenDisplayUpdate to give access to coordinator-level fields (RSSI,
last_seen) alongside advertisement payload fields.

Also syncs en.json with strings.json (adds translations for the new
sensors, update entity, no_leds/no_buzzers exceptions, and the
activate_led/activate_buzzer/drawcustom services that were missing).
- Replace single flat color/flash_count/loop_delay/inter_delay fields
  with three step groups (color1-3, flash_count1-3, loop_delay1-3,
  inter_delay1-3); steps 2 and 3 are skipped when flash_count is 0
- Use RGB color picker selector; voluptuous converts [R,G,B] to the
  firmware's packed 3R-3G-2B byte at validation time
- Expose loop_delay and inter_delay in milliseconds (×100ms units)
  with slider selectors; voluptuous converts to firmware units
- Brightness and repeats now use sliders
Adds a Display content image entity that updates via dispatcher signal
after every successful upload_image or drawcustom call (including dry-run).
name: Draw Custom Image
description: Draws a custom image on one or more E-Paper displays
target:
upload_image:

@LordMike LordMike May 29, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense for upload_image to (also) take a url?
Or perhaps thats a new service.

My use case is that I have puppet set up with the ability to snapshot HA pages, so I have a URL that can produce the image I want. I want to push that URL.

My current approach is an ODL with a single image entity that covers the whole screen.. which is also still an option.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be implemented now in (ef91895)

action: opendisplay.upload_image
  data:
    device_id: <your device>
    image: "http://homeassistant.local:10000/home?viewport=1000x1000&format=jpeg"

One requirement: the URL host has to be in allowlist_external_urls (this keeps the service from being tricked into fetching arbitrary/internal targets). Since puppet serves on a direct port (10000, no ingress) and handles its own HA auth, a one-time allowlist entry is all it needs:

homeassistant:
  allowlist_external_urls:
    - "http://homeassistant.local:10000"

g4bri3lDev and others added 5 commits July 17, 2026 00:21
Adds the services.yaml field definitions, strings.json/en.json service
descriptions, exception messages, and record_type selector options, and
the icons.json entry for the write_nfc service added in the previous task.
The ha_tag branch's empty-tag_id check was unreachable: the general
content.strip() check above already raises nfc_content_empty first.
Also add tests pinning the 512-byte exact-limit accept path and
proving the size check measures UTF-8 bytes, not characters.
…mage

_async_send_image accepted use_measured_palettes but never passed it to
prepare_image(), whose default (True) silently overrode the caller's choice —
'measured_palette: false' in drawcustom/upload_image was a no-op and every
frame was quantized against the measured panel palette.

Adds a regression test asserting the flag reaches prepare_image for both
values; before the fix the captured kwargs contain no use_measured_palettes
key at all.
g4bri3lDev and others added 24 commits July 23, 2026 14:54
The firmware parses BUILD_VERSION with a plain int conversion on the
digits after the dot, so the minor byte already equals the literal
tag digits (2.20 -> 20, 1.71 -> 71, 1.6 -> 6). Dividing by 10 for
minor >= 10 corrupted two-digit versions, e.g. showing 2.20 as 2.2.
Fixes #62.
stop scaling firmware minor version by 10
Add a new opendisplay.play_melody sibling service that plays a multi-note
melody on a device's buzzer via BuzzerActivateConfig.melody(). activate_buzzer
stays a single-tone service; its docs gain a pointer to the new service.

- services.py: _valid_melody validator, SCHEMA_PLAY_MELODY, _async_play_melody
  handler (mirrors _async_activate_buzzer), and registration.
- services.yaml / strings.json / translations/en.json: play_melody definition,
  field docs (grammar summary), and the invalid_melody exception, in lockstep.
- tests/test_services.py: wire-byte, plumbing, guard, and schema tests.

manifest.json pin bump deferred until the py-opendisplay release.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The firmware version response carries a trailing patch byte after the
SHA; once py-opendisplay parses it, format installed_version as
major.minor.patch so a device on a patch release (e.g. 2.25.1) matches
its release tag instead of reporting a phantom pending update forever.
Falls back to the two-part form for older py-opendisplay or cached
firmware dicts that lack the field.
Adds German, French, Dutch, Spanish, Italian, Polish, Brazilian
Portuguese and Czech translations, generated from translations/en.json
using GitHub Models and validated automatically before being written.

scripts/translate.py only sends keys that are missing or whose English
source was reworded, so a run after adding a few strings costs a few
strings' worth of tokens rather than the whole file. It records a
fingerprint of both the English source and the translation it wrote in
.github/translation-state.json, which lets it distinguish its own output
from a manual correction: hand-edited translations are never
overwritten, and are reported for review instead when the English behind
them changes.

scripts/verify_translations.py re-checks the files on disk as a CI gate.
Placeholder mismatches, empty values and unknown keys fail the build.
Translations use impersonal phrasing to stay neutral about the
familiar/polite distinction (German du/Sie, French tu/vous), and a
heuristic warns when one slips through.

Also rewords the url_not_allowed message to drop "your", which was
propagating a second-person possessive into every translation.

Uses only the Python standard library, so requirements.txt is unchanged,
and the built-in GITHUB_TOKEN with models: read, so there is no API key
to manage and no billing.
Home Assistant select entities use their option values as translation
keys, and those legitimately contain dots. The existing de.json on main
has entity.select.subghzchannel.state with keys like
"100 - 864.000 Mhz (Europe, etc)", which a dot separator silently
mis-nests on the way back out.

The current en.json happens to contain no such entity, so this never
surfaced. Switch the separator to NUL, which cannot appear in a key that
came from a real translation file, and add show() to render paths with
dots for logs and PR summaries.

Keys sent to the model are now plain indices rather than full paths.
They no longer carry the separator, and dropping the path text from the
response saves output tokens on every request.
Home Assistant treats pt and pt-BR as separate locales, and they differ
in everyday interface vocabulary: 73 of the 181 strings came out
differently (ligacao/conectar, encriptacao/criptografia, em progresso/em
andamento). main already shipped a pt.json, so covering both avoids
regressing European Portuguese users when the rewrite lands.

Documents in the README that translations other than English are machine
generated and that corrections are welcome, including the guarantee that
a hand-corrected string is never overwritten. Contributors have little
reason to submit a fix if they expect a bot to revert it.

Also reports pending review items in the per-language status line, which
previously read "up to date" even when a string was flagged because its
English source changed under a manual correction.
add AI-generated translations for 9 languages
add play_melody service for buzzer melodies
The run after PR 76 merged failed with a bare "HTTP Error 403: Forbidden"
traceback, which says nothing about why or what to do. The endpoint
explains itself in the response body, so read it and print it.

For 401 and 403 specifically, spell out the actual cause: a repository
GITHUB_TOKEN inherits the organisation's GitHub Models access, not that
of whoever triggered the run, so 'models: read' in the workflow is
necessary but not sufficient and an org owner has to enable Models. A
personal token working locally is not evidence that CI will work, which
is exactly how this got missed.
The scheduled run failed with 403 because GitHub Models access is a
Copilot entitlement, and a repository GITHUB_TOKEN carries the
organisation's entitlement rather than the triggering user's. This org
is on the free plan with no Copilot seats, so the built-in token cannot
reach GitHub Models however the workflow permissions are set, and the
org settings offer nothing to enable. A personal token working locally
was never evidence that CI would work.

Both GitHub Models and OpenRouter speak the same chat/completions
shape, so the endpoint, key and model are now a small provider table.
Whichever API key is present decides which is used, OPENROUTER_API_KEY
first; TRANSLATE_PROVIDER and TRANSLATE_MODEL override for one-off runs.
OpenRouter needs no Copilot entitlement, has no 4000-token output cap,
and costs roughly twelve cents for a full nine-language backfill, which
is already done.

Drops the now-pointless 'models: read' permission so the next reader
does not assume it is load-bearing.
make translation workflow work without a Copilot plan
The workflow regenerated FALLBACK_TAG_DEFINITIONS in const.py from the
OpenEPaperLink tag type definitions, using scripts/fetch_tag_types.py
and scripts/generate_tag_types.py.

The 3.0 rewrite removed all three: both scripts and the
FALLBACK_TAG_DEFINITIONS table they edit are gone from this branch. The
workflow still succeeds on main only because main is still the old
architecture; on this branch it fails at its first step, and it would
start failing on main the moment this branch lands.
…ions

chore: remove obsolete sync-tag-definitions workflow
forward measured_palette to prepare_image in drawcustom/upload_image
The patch component reached the update entity but not the device
registry, so the device page still showed "2.25" for a device running
2.25.1, and diagnostics omitted the field entirely.

The device registry now shares the update entity's formatter rather than
building its own string, so the two cannot disagree — a mismatch between
them is exactly what makes an update look permanently pending. The import
is deferred because update.py imports from __init__.

Bumps py-opendisplay to 7.14.1, the first release that parses the
trailing patch byte. Without it the version dict has no patch key at all,
the formatter falls back to the two-part form on every path, and a device
on the newest firmware keeps offering an update it cannot clear.

Adds a regression test comparing installed against latest the way the
update entity does: 2.25.1 against tag 2.25.1 must report no update,
while a genuinely newer release still does. Legacy two-part tags are
covered too — x.y.0 ranks equal to a bare x.y, so devices on pre-SemVer
releases do not gain a phantom update.

Fixes #62
report the firmware patch version everywhere HA shows it
…e baseline

The Create Release workflow was added three weeks before release-please
and has never run once. It could not have: it writes to
custom_components/open_display/manifest.json, an underscored path that
does not exist, so its bump step fails immediately. Two further faults
had it ever been used — its version regex allows no dot in the
prerelease part, rejecting the beta.N form every pre-release uses, and
it calls `gh release create` without --prerelease, which would publish a
beta as stable and push it to every HACS user rather than only those
opted in to betas.

Releases on main are cut by release-please, and pre-releases are cut by
hand from the dev branch. Neither uses this workflow, so it is removed
rather than repaired.

Also corrects the release-please baseline. It recorded 2.0.2, the last
version release-please itself released, while the project has since
published 3.0.0-beta.9 through the manual pre-release path that
release-please never sees. Left stale, the first release after the dev
branch merges anchors to 2.0.2 and can propose a version below what
users already have.

Tags are untouched: release-please's own releases are bare per
include-v-in-tag, and the hand-cut betas keep their v prefix. The two
series are each internally consistent.
drop the unused Create Release workflow and correct the release baseline
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

6 participants