Round scaled config values instead of truncating on cast - #1256
Open
gskjold wants to merge 1 commit into
Open
Conversation
User-entered decimals are stored as scaled integers, but the cast from
double to integer truncates. atof("0.0464") is 0.046399999999999996, so
* 10000.0 yields 463.99999999999994 and the price modifier is stored as
463 instead of 464.
68866 of the 999999 possible four-decimal price modifier values are
stored one unit too low this way. Switching from float to double in
a733365 barely moved that count (68886 before, 68866 after); it only
changed which values land just below the integer, and 0.0464 moved into
the broken set.
Round with lround() at every scaling site instead: price modifiers,
meter multipliers, Wi-Fi power, Vcc offset/multiplier/boot limit and the
day/month plot editor, both in the web form handler and in the config
file parser. The config file parser also still used toFloat() for the
Vcc values and the price modifier, so it kept the old truncation on
backup restore.
Reported in #1251
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
🔧 PR Build ArtifactsVersion: All environments built successfully. Download the zip files:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reported in #1251: a price modifier of
0.0464is stored as0.0463, while0.0463and0.0465are kept.Cause
User-entered decimals are stored as scaled integers, and the cast to integer truncates:
This is not a float-vs-double problem. Sweeping all 999999 four-decimal values through the price modifier conversion:
toFloat() * 10000(≤ v2.5.5)toDouble() * 10000.0(v2.5.6 → main)lround(toDouble() * 10000.0)a733365 (#1133) barely moved the count — it only changed which values land just below the integer.
0.0464happened to be correct withfloatand moved into the broken set in v2.5.6, which is why this surfaces now on v2.5.7.Change
lround()at every scaling site rather than relying on the cast:AmsWebServer::handleSave()— price modifier value, meter wattage/voltage/amperage/accumulated multipliers, Wi-Fi power, Vcc offset/multiplier/boot limitAmsWebServer::modifyDayPlot()/modifyMonthPlot()— the day/month data editorconfigFileParse()— the same fields on config-file restore. This path also still usedtoFloat()for the Vcc values and the price modifier, so a backup/restore re-applied the old truncation; those are nowtoDouble()too.lround()rounds half away from zero, which is correct for the one signed field here (vccOffset).Verification
0.0464→464; the full 999999-value sweep is exact for all of them.pio run -e esp32andpio run -e esp8266both build. ESP8266 grows 208 bytes and stays at 98.0% flash (20973 bytes free).🤖 Generated with Claude Code