diff --git a/docs/commands.md b/docs/commands.md index 8271439..0af1540 100644 --- a/docs/commands.md +++ b/docs/commands.md @@ -680,8 +680,9 @@ release time, jacket, Gekisou missions, score ranks, the whole `MasterLiveMusic` (read from the cue sheet's ACB, without decoding audio); per difficulty the chart facts (level and display level, full combo count, note counts, BPM, note times, the live's music length, skill event times, fever ranges) and the chart's deck statistics: the no-skill score and the weight of every score-up skill kind at every performance -position, measured by the deck model ournotes-deck (built into nnnotes as `nnnotes._deck`) on its whole-live -simulation and checked against the chart facts. `--full` also writes the deck model's input: every +position, with Gekisou on (a Gekisou live at rank 1, with what every other rank and the Perfect play need) and off (a +solo live), measured by the deck model ournotes-deck (built into nnnotes as `nnnotes._deck`) on its whole-live +simulation and checked against the chart facts and the master data. `--full` also writes the deck model's input: every `MasterLiveMusicScore` row's chart as the client builds it at runtime (notes, skill events, fever ranges) and the master data tables about cards, skills, bonuses, scores and events. `--no-deck` skips the deck model (every chart's `deck` is null); `--seeds N` (default 8) and `--workers N` (default: every processor) set its seeds on charts with a diff --git a/docs/music-data.md b/docs/music-data.md index 3fa9c84..ddbb563 100644 --- a/docs/music-data.md +++ b/docs/music-data.md @@ -3,8 +3,8 @@ `nnnotes music-data` writes one JSON file with every live song and chart of one master data version: what a song listing shows (titles and credits in every language, bands, vocal characters, category, tags, release time, score ranks, jacket, BGM length), per difficulty the chart facts (level, note counts, BPM, chart times, skill events, -fevers), and per chart its **deck statistics**: what the chart contributes to the live score whatever the deck, -measured by the deck model [ournotes-deck](https://github.com/empty-sekai/ournotes-deck), which nnnotes carries as +fevers), and per chart its **deck statistics**: what the chart contributes to the live score whatever the deck, in a +solo live (Gekisou off) and in a Gekisou live at every rank, measured by the deck model [ournotes-deck](https://github.com/empty-sekai/ournotes-deck), which nnnotes carries as its extension module `nnnotes._deck`. The format is `nnnotes.music-data/1`; its JSON Schema is [schema/music-data.schema.json](schema/music-data.schema.json). @@ -39,7 +39,8 @@ The command writes the file only when every table, chart and cue sheet was read or mismatching master data file, a table without a column the file exports, a text id that `MasterText` does not have, a score id that `MasterLiveMusicScore` does not have, a missing or unreadable chart asset, a note id that occurs twice in a chart, a cue sheet without the song's cue, (with `--jackets`) a missing jacket texture, a chart the -deck model cannot measure or whose check deck fails, or deck statistics that disagree with the chart facts stops it +deck model cannot measure or whose check deck fails, or deck statistics that disagree with the chart facts or the +master data stops it with exit status 1 and a line naming the input. The file is written through a temporary file and a rename. An installation without the extension module (a source checkout that was not built) runs only with `--no-deck`; @@ -138,12 +139,18 @@ plays, `musicLengthMs` is the length the score code uses; a listing chooses the ## Deck statistics -The deck model plays every chart of a song on its whole-live simulation, with Gekisou on as the game plays every -live, solo (rank 1), in the theoretical best play: every judged note at its exact time, Just inside the Just-count -ranges and Perfect elsewhere, the frame times of the game's default schedule. It measures, at deck power -`model.power` (300000): +The deck model plays every chart of a song on its whole-live simulation in the theoretical best play, the frame +times of the game's default schedule, in two scenarios: -- `score`: the exact score without skills, the rank bonuses of the Gekisou ranges included; +- **Gekisou on** (`seeds`), as a Gekisou live (Battle Live, up to five players) plays: every judged note at its exact + time, Just inside the Just-count ranges and Perfect elsewhere, rank 1 in every Gekisou range. Other ranks follow + from the same numbers (below). +- **Gekisou off** (`offSeeds`), as a solo live (Free Live, Challenge Live) plays: every judged note Perfect at its + exact time, seed 0, no Just, luck, Gekisou combo or rank bonus. A chart with more than three fevers plays here too. + +In each it measures, at deck power `model.power` (300000): + +- `score`: the exact score without skills (with Gekisou on, the rank 1 bonuses of the Gekisou ranges included); - for every **score-up kind** (`deck.kinds`) and every performance position `k`, `weights[kind][k]`: the exact score a deck gains when its position-`k` member has one effect of that kind at factor 1, divided by the deck power. The effect runs through the simulation's own updaters, conditions, frames and appliers, so the weight carries every rule @@ -163,15 +170,42 @@ master values at another power (`model.checkPower`), and the command fails when the formula. Effects of other types (cumulative score 2001 / 2003, life, judgement conversion, Gekisou and snap skills) are not linear in the chart alone and have no weights: a deck's score with them comes from the simulation. -A luck range draws lottery results from the play's random seed, so the measurements are given **per seed**: one seed -(0) when no range is a luck range, else the first `--seeds` seeds of the deck model's published seed set. The seed -set is not the game's seed law (which is unknown); a mean over it is not the game's expectation. +A luck range draws lottery results from the play's random seed, so the Gekisou on measurements are given **per +seed**: one seed (0) when no range is a luck range, else the first `--seeds` seeds of the deck model's published seed +set. The seed set is not the game's seed law (which is unknown); a mean over it is not the game's expectation. + +### Ranks + +In a Gekisou live, range `i` takes a rank `r_i` from 1 to 5 among the room's players, and its rank bonus is +`trunc(rangeScore_i * p_i(r_i) / 100)` with `p_i(r) = ranges[i].rankBonusPercents[r - 1]`. The bonus is a fixed score +at the range's end: it changes no factor and no note score, and a later range's score holds it at both ends, so the +range scores do not depend on the ranks. At ranks `r` a seed's numbers are therefore + +``` +score_r = score - sum_i rankBonus_i + sum_i trunc(rangeScore_i * p_i(r_i) / 100) (exact) +weights_r[kind][k] = weights[kind][k] + sum_i (p_i(r_i) - p_i(1)) / 100 * rangeWeights[kind][k][i] +``` + +(`rankBonus_i`, `rangeScore_i`: `seeds[].ranges[i]`), and a deck scores `P * (score_r / power + sum_k factor_k * +weights_r[kind_k][k])` as above; `weights_r` is within `2 * ranges / power` per unit of factor of the exact weight at +those ranks. At rank 1 everywhere these are `score` and `weights`. `rangeWeights` is null when a range's bonus can fall +inside another range's score frames (overlapping ranges), and a kind's entry is null when its conditions read the +confirmed rank (condition 7012): the ranks do not follow linearly there. `rankCheck` plays the seed's check deck at +random ranks through the simulation's explicit rank confirmations and bounds it against these formulas. + +### Just rate + +`scorePerfect` and `ranges[i].rangeScorePerfect` are the no-skill score (rank 1 bonuses included) and the range +scores of the same play with every Just judged Perfect (the Just judgement is enabled only inside the Just-count +ranges, so nothing else changes). Between a Just rate of 1 (`score`, `rangeScore`) and 0 (`scorePerfect`, +`rangeScorePerfect`) a page can interpolate; the rank bonus of the Perfect play is `trunc(rangeScorePerfect_i * +p_i(r_i) / 100)`. A chart without Just notes (`justNotes` 0) has `scorePerfect == score`. ### deck | Field | Content | |---|---| -| `model` | the deck model's description of the measurement: `engine`, `play`, `score` (the formula), `power`, `checkPower`, `unitValue` (the effect value of factor 1, 10000), `seeds` | +| `model` | the deck model's description of the measurement: `engine`, `play`, `score` (the formula), `power`, `checkPower`, `unitValue` (the effect value of factor 1, 10000), `seeds`, `ranks` (the rank formulas), `perfect` (the Perfect play), `off` (the Gekisou off scenario) | | `kinds[]` | the score-up kinds of the master data: `MasterLiveSkillEffect` rows of type 2000, 2002, 2004 or 2005 without a cumulative condition, grouped by what shapes their score. `id` (the index in `weights`), `effectType`, `activationTimeSecond`, `durationMs` (`ceil(activationTimeSecond * 1000f)`), `skillTargetIds`, `skillConditionGroup`, `skillReleaseConditionGroup`, `effectLimitCount`, `effectExecuteLimitCount`, `effectExecuteLimitResetConditionGroup`; `rows` (master rows of the kind) and `values` (their distinct `_effectValue`s, ascending) | The kind of a card's live skill is found by matching its `MasterLiveSkillEffect` row (at the skill level) on these @@ -185,14 +219,19 @@ fields; `values` lists what the master data uses. | `skip` | score per unit of deck power of a skipped live (every note Great, combo 0, no skills) | | `events` | `[[position, timeMs], ...]`: the skill events in chart order with the performance position each fires | | `positions` | the performance positions the events fire (the largest position + 1): the length of every `weights[kind]` | -| `ranges[]` | the Gekisou ranges: `index`, `mission` (1 combo, 2 luck, 3 Just count), `startMs`, `endMs`, `rankBonusPercent` (the solo rank bonus percentage of the song's mission pattern) | -| `justNotes` | notes judged Just on the play | -| `seeds[]` | per seed: `seed`; `score`; `ranges[]` (`rangeScore`: the score gained inside the range, `rankBonus`, `maxCombo`, `justCount`, `lotResults`: lottery results Miss, Hit, Super Hit, Critical); `weights[kind][position]`; `check` (`deck`: `[kind, value]` or null per position, `exact`, `predicted`, `bound`) | -| `unplayable` | null, or why the game cannot play the chart (more than three fevers: the game fails when the fourth starts); `seeds` is then empty | +| `ranges[]` | the Gekisou ranges: `index`, `mission` (1 combo, 2 luck, 3 Just count), `startMs`, `endMs`, `rankBonusPercents` (the rank bonus percentages of ranks 1..5 of the song's mission pattern, `MasterLiveGekisouRankingScoreBonus`), `rankBonusPercent` (the rank 1 percentage, `rankBonusPercents[0]`) | +| `justNotes` | notes judged Just on the Gekisou on play | +| `seeds[]` | Gekisou on, per seed: `seed`; `score` (points at `model.power`, rank 1 bonuses included); `ranges[]` (`rangeScore`: the points gained inside the range, `rankBonus`: its rank 1 bonus in points, `maxCombo`, `justCount`, `lotResults`: lottery results Miss, Hit, Super Hit, Critical, `rangeScorePerfect`: `rangeScore` on the Perfect play); `weights[kind][position]` (points per unit of deck power and of factor); `check` (`deck`: `[kind, value]` or null per position, `exact`, `predicted`, `bound`: points at `model.checkPower`); `scorePerfect` (`score` on the Perfect play); `rangeWeights[kind][position][range]` (range points per unit of deck power and of factor, or null; a kind null); `rankCheck` (`ranks`: 1..5 per range, `exact`, `predicted`, `bound`; null without ranges or range weights) | +| `offSeeds[]` | Gekisou off, one seed: `seed` (0), `score`, `weights[kind][position]` (a kind null when its conditions read the Gekisou state, which a solo live does not have) and `check`, as in `seeds[]` | +| `unplayable` | null, or why the game cannot play the chart with Gekisou (more than three fevers: the game fails when the fourth starts); `seeds` is then empty, `offSeeds` is not | The deck model's chart facts are checked against the file's: the song, difficulty, level, judged note count, last note time, music length, Gekisou missions, skill event times and fever ranges must agree, and are not repeated in -`deck`. +`deck`. Its numbers are checked against the master data and themselves: every range's `rankBonusPercents` are the +`MasterLiveGekisouRankingScoreBonus` rows of the song's mission pattern (0 without a row), every `rankBonus` is +`trunc(rangeScore * rankBonusPercent / 100)`, a chart without Just notes has the same scores on the Perfect play, +every array has its shape (`[kind][position]`, `[kind][position][range]`, one range result per range, one Gekisou off +seed), and every check and rank check is within its bound. ## The deck input (`--full`) diff --git a/docs/schema/music-data.schema.json b/docs/schema/music-data.schema.json index 18dcba2..42e8eda 100644 --- a/docs/schema/music-data.schema.json +++ b/docs/schema/music-data.schema.json @@ -546,7 +546,17 @@ "type": "integer" }, "rankBonusPercent": { - "type": "integer" + "type": "integer", + "description": "the rank 1 bonus percentage (rankBonusPercents[0])" + }, + "rankBonusPercents": { + "type": "array", + "items": { + "type": "integer" + }, + "minItems": 5, + "maxItems": 5, + "description": "the rank bonus percentages of ranks 1..5 of the song's mission pattern" } } } @@ -604,6 +614,10 @@ }, "minItems": 4, "maxItems": 4 + }, + "rangeScorePerfect": { + "type": "integer", + "description": "the range score on the Perfect play (every Just judged Perfect)" } } } @@ -618,6 +632,125 @@ }, "description": "weights[kind][position]: score per unit of deck power and of factor" }, + "check": { + "type": "object", + "required": [ + "deck", + "exact", + "predicted", + "bound" + ], + "properties": { + "deck": { + "type": "array", + "items": { + "type": [ + "array", + "null" + ] + } + }, + "exact": { + "type": "integer" + }, + "predicted": { + "type": "number" + }, + "bound": { + "type": "number" + } + } + }, + "scorePerfect": { + "type": "integer", + "description": "the exact no-skill score at model.power on the Perfect play (every Just judged Perfect), its rank 1 bonuses included" + }, + "rangeWeights": { + "type": [ + "array", + "null" + ], + "items": { + "type": [ + "array", + "null" + ], + "items": { + "type": "array", + "items": { + "type": "number" + } + } + }, + "description": "rangeWeights[kind][position][range]: range score per unit of deck power and of factor; null when the ranks do not follow linearly on the chart, a kind null when its conditions read the confirmed rank" + }, + "rankCheck": { + "type": [ + "object", + "null" + ], + "required": [ + "ranks", + "exact", + "predicted", + "bound" + ], + "description": "the check deck played at random ranks; null without ranges or range weights", + "properties": { + "ranks": { + "type": "array", + "items": { + "type": "integer", + "minimum": 1, + "maximum": 5 + } + }, + "exact": { + "type": "integer" + }, + "predicted": { + "type": "number" + }, + "bound": { + "type": "number" + } + } + } + } + } + }, + "offSeeds": { + "type": "array", + "description": "the measurements with Gekisou off (a solo live), one seed", + "items": { + "type": "object", + "required": [ + "seed", + "score", + "weights", + "check" + ], + "properties": { + "seed": { + "type": "integer" + }, + "score": { + "type": "integer", + "description": "the exact no-skill score at model.power, Gekisou off" + }, + "weights": { + "type": "array", + "items": { + "type": [ + "array", + "null" + ], + "items": { + "type": "number" + } + }, + "description": "weights[kind][position], Gekisou off; a kind null when its conditions read the Gekisou state" + }, "check": { "type": "object", "required": [ @@ -654,7 +787,8 @@ "type": [ "string", "null" - ] + ], + "description": "why the game cannot play the chart with Gekisou (seeds is then empty; offSeeds is not)" } } }, diff --git a/rust/Cargo.lock b/rust/Cargo.lock index 9213264..9a76560 100644 --- a/rust/Cargo.lock +++ b/rust/Cargo.lock @@ -44,7 +44,7 @@ checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" [[package]] name = "ournotes-deck" version = "0.0.1" -source = "git+https://github.com/empty-sekai/ournotes-deck?rev=7e5d84b5998d28c21541ce3f2e0a3dfb1439f4f6#7e5d84b5998d28c21541ce3f2e0a3dfb1439f4f6" +source = "git+https://github.com/empty-sekai/ournotes-deck?rev=d4abb52ff75059a13e99a88fd6812ec1bdaa5dd2#d4abb52ff75059a13e99a88fd6812ec1bdaa5dd2" dependencies = [ "serde", "serde_json", diff --git a/rust/Cargo.toml b/rust/Cargo.toml index 3a28e84..92a1db1 100644 --- a/rust/Cargo.toml +++ b/rust/Cargo.toml @@ -15,7 +15,7 @@ crate-type = ["cdylib"] [dependencies] pyo3 = { version = "0.29", features = ["abi3-py311"] } # pinned by commit; .github/workflows/deck.yml opens a pull request when ournotes-deck moves -ournotes-deck = { git = "https://github.com/empty-sekai/ournotes-deck", rev = "7e5d84b5998d28c21541ce3f2e0a3dfb1439f4f6" } +ournotes-deck = { git = "https://github.com/empty-sekai/ournotes-deck", rev = "d4abb52ff75059a13e99a88fd6812ec1bdaa5dd2" } serde_json = { version = "1", features = ["raw_value", "arbitrary_precision"] } [profile.release] diff --git a/src/nnnotes/musicdata.py b/src/nnnotes/musicdata.py index 04ffa64..24157dd 100644 --- a/src/nnnotes/musicdata.py +++ b/src/nnnotes/musicdata.py @@ -2,7 +2,8 @@ `nnnotes.music-data/1`, docs/music-data.md): titles and credits in every language, bands, vocal characters, category, tags, release time, score ranks, the live BGM's length, per difficulty the chart facts (level, note counts, BPM, chart times, skill events, fever ranges) and the chart's deck statistics, what the chart contributes to the live score -whatever the deck, measured by the deck model ournotes-deck (the extension module nnnotes._deck). +whatever the deck, with Gekisou on (a Gekisou live, every rank) and off (a solo live), measured by the deck model +ournotes-deck (the extension module nnnotes._deck). Master data is read from the files as served (deckdata.master_files / apk_master: SHA-256 checked against the manifest, decoded with master.decode). Charts are the TextAssets `Live/MusicScore/` converted by @@ -255,12 +256,89 @@ def stats(self, deck_input: dict) -> dict: # the keys of a chart's statistics carried by `deck` (the others are checked against the chart facts) -DECK_CHART_KEYS = ("convertedNoteCount", "skip", "events", "positions", "ranges", "justNotes", "seeds", +DECK_CHART_KEYS = ("convertedNoteCount", "skip", "events", "positions", "ranges", "justNotes", "seeds", "offSeeds", "unplayable") +RANKS = 5 # the ranks of a Gekisou range (a Gekisou live has up to five players) +GEKISOU_RANGES = 3 # the Gekisou ranges of a live (its first three fevers) + + +def mission_pattern(missions) -> int: + """The mission pattern of a song's three Gekisou missions, the `_missionPattern` of its rank bonus rows: 0 when a + mission is missing, 1 all the same, 2 all different, 3 otherwise.""" + a, b, c = missions + if not (a and b and c): + return 0 + if a == b: + return 1 if a == c else 3 + return 2 if b != c and a != c else 3 + + +def rank_bonus_percents(rows: list[dict], missions) -> list[list]: + """The rank bonus percentages [range][rank - 1] of a song's missions from the MasterLiveGekisouRankingScoreBonus + rows of its mission pattern (`_count`: the range 1..3, `_rank` 1..5; a later row wins; 0 without a row).""" + pattern = mission_pattern(missions) + out = [[0] * RANKS for _ in range(GEKISOU_RANGES)] + for r in rows: + c, k = (r.get("_count") or 0) - 1, (r.get("_rank") or 0) - 1 + if r.get("_missionPattern") == pattern and 0 <= c < GEKISOU_RANGES and 0 <= k < RANKS: + out[c][k] = r.get("_scoreBonusPercent") + return out -def chart_deck(song: dict, chart: dict, stats: dict) -> dict: - """A chart's `deck` from its statistics, after checking them against the song and the chart facts.""" +def _trunc_percent(score: int, percent: int) -> int: + """trunc(score * percent / 100), the rank bonus of a range score.""" + p = score * percent + return p // 100 if p >= 0 else -(-p // 100) + + +def _numbers(v, n: int) -> bool: + """Whether v is a list of n numbers (as the deck model writes them).""" + return isinstance(v, list) and len(v) == n and all( + isinstance(x, (int, deckdata._Num)) and not isinstance(x, bool) for x in v) + + +def _check_seed_shapes(stats: dict, kinds: int, where: str) -> None: + """The array shapes and the checks of a chart's statistics: `weights[kind][position]` (a kind null only with + Gekisou off), `rangeWeights[kind][position][range]` (null, or a kind null), one Gekisou off seed, the seeds' + ranges one per range, and every check (and rank check) within its bound.""" + positions, n = stats["positions"], len(stats["ranges"]) + + def check(c, what): + if abs(c["exact"] - float(c["predicted"])) > float(c["bound"]): + raise MusicDataError(f"{where}: {what} scores {c['exact']}, predicted {c['predicted']} beyond the bound " + f"{c['bound']}") + + for seed in stats["seeds"]: + s = f"seed {seed['seed']}" + w = seed.get("weights") + if not (isinstance(w, list) and len(w) == kinds and all(_numbers(x, positions) for x in w)): + raise MusicDataError(f"{where}: {s}: weights are not [kind][position]") + if len(seed["ranges"]) != n: + raise MusicDataError(f"{where}: {s}: {len(seed['ranges'])} range results for {n} ranges") + rw = seed.get("rangeWeights") + if rw is not None and not (isinstance(rw, list) and len(rw) == kinds and all( + k is None or (isinstance(k, list) and len(k) == positions and all(_numbers(x, n) for x in k)) + for k in rw)): + raise MusicDataError(f"{where}: {s}: rangeWeights are not [kind][position][range]") + check(seed["check"], f"{s}: the check deck") + rc = seed.get("rankCheck") + if rc is not None: + if len(rc["ranks"]) != n or not all(1 <= r <= RANKS for r in rc["ranks"]): + raise MusicDataError(f"{where}: {s}: rank check ranks {rc['ranks']!r}") + check(rc, f"{s}: the check deck at ranks {rc['ranks']!r}") + off = stats.get("offSeeds") + if not isinstance(off, list) or len(off) != 1: + raise MusicDataError(f"{where}: the deck model gives no Gekisou off statistics (offSeeds)") + for seed in off: + w = seed.get("weights") + if not (isinstance(w, list) and len(w) == kinds and all(x is None or _numbers(x, positions) for x in w)): + raise MusicDataError(f"{where}: Gekisou off: weights are not [kind][position]") + check(seed["check"], "Gekisou off: the check deck") + + +def chart_deck(song: dict, chart: dict, stats: dict, kinds: int, percents: list[list]) -> dict: + """A chart's `deck` from its statistics, after checking them against the song, the chart facts and the song's + rank bonus percentages (`percents`: rank_bonus_percents), `kinds` the number of score-up kinds.""" where = f"chart {chart['scoreId']} ({song['id']} {chart['difficulty']})" checks = ( ("music id", stats["musicId"], song["id"]), @@ -276,6 +354,24 @@ def chart_deck(song: dict, chart: dict, stats: dict) -> dict: for name, deck, facts in checks: if deck != facts: raise MusicDataError(f"{where}: the deck model's {name} {deck!r} differs from the chart's {facts!r}") + ranges = stats["ranges"] + for i, r in enumerate(ranges): + want = percents[i] if i < len(percents) else [0] * RANKS + got = r.get("rankBonusPercents") + if got != want or r["rankBonusPercent"] != want[0]: + raise MusicDataError(f"{where}: range {i}: the deck model's rank bonus percentages {got!r} " + f"({r['rankBonusPercent']!r}) differ from MasterLiveGekisouRankingScoreBonus {want!r}") + _check_seed_shapes(stats, kinds, where) + for seed in stats["seeds"]: + for i, (r, info) in enumerate(zip(seed["ranges"], ranges)): + bonus = _trunc_percent(r["rangeScore"], info["rankBonusPercent"]) + if r["rankBonus"] != bonus: + raise MusicDataError(f"{where}: seed {seed['seed']} range {i}: rank bonus {r['rankBonus']} is not " + f"trunc({r['rangeScore']} * {info['rankBonusPercent']} / 100) = {bonus}") + if stats["justNotes"] == 0 and (seed.get("scorePerfect") != seed["score"] or any( + r.get("rangeScorePerfect") != r["rangeScore"] for r in seed["ranges"])): + raise MusicDataError(f"{where}: seed {seed['seed']}: a chart without Just notes scores otherwise on the " + f"Perfect play ({seed.get('scorePerfect')} for {seed['score']})") return {k: stats.get(k) for k in DECK_CHART_KEYS} @@ -387,9 +483,11 @@ def record(sid: int) -> dict: by_score = {c["scoreId"]: c for c in stats["charts"]} if sorted(by_score) != measured: raise MusicDataError("deck model: the charts measured differ from the songs' charts") + bonus_rows = tables.get("MasterLiveGekisouRankingScoreBonus", []) for s in songs: + percents = rank_bonus_percents(bonus_rows, s["gekisouMissions"]) for c in s["charts"]: - c["deck"] = chart_deck(s, c, by_score[c["scoreId"]]) + c["deck"] = chart_deck(s, c, by_score[c["scoreId"]], len(stats["kinds"]), percents) deck_doc = {"model": stats["model"], "kinds": stats["kinds"]} read = [t for t in tables_of(deck is not None, full) if t in tables] diff --git a/tests/test_musicdata.py b/tests/test_musicdata.py index b292ba6..4e9cd20 100644 --- a/tests/test_musicdata.py +++ b/tests/test_musicdata.py @@ -60,6 +60,11 @@ def text(tid, stem): {"_id": 3, "_group": 9, "_liveScoreRank": 7, "_requiredScore": 900, "_battleLiveRequiredScore": 1800}, {"_id": 1, "_group": 9, "_liveScoreRank": 2, "_requiredScore": 0, "_battleLiveRequiredScore": 0}, {"_id": 2, "_group": 8, "_liveScoreRank": 6, "_requiredScore": 5, "_battleLiveRequiredScore": 6}], + # the songs' missions [1, 3, 3] are pattern 3; pattern 2 rows are another song's + "MasterLiveGekisouRankingScoreBonus": [ + {"_id": 100 * p + 10 * c + k, "_missionPattern": p, "_count": c, "_rank": k, + "_scoreBonusPercent": (6 - k) * c + 10 * p} + for p in (2, 3) for c in (1, 2, 3) for k in (1, 2, 3, 4, 5)], } CHARTS = {"c/c_01": gzip.compress(json.dumps(CHART).encode("utf-8"), mtime=0), "c/c_02": json.dumps(SMALL).encode("utf-8"), @@ -96,6 +101,10 @@ def bgm(sheet, cue): WEIGHT = "0.30000000000000004" # a binary64 value: kept as the deck model writes it +def trunc_percent(score, percent): + return int(score * percent / 100) + + class FakeDeck: """The interface of nnnotes._deck: statistics made from the deck input as the model reports them.""" COMMIT = "7e5d84b5998d28c21541ce3f2e0a3dfb1439f4f6" @@ -118,6 +127,8 @@ def chart_stats(self, data, seeds, workers): def table(name): return [dict(zip(m[name]["columns"], r)) for r in m[name]["rows"]] levels = {r["_id"]: r["_musicScoreLevel"] for r in table("MasterLiveMusicScore")} + bonus = {(r["_missionPattern"], r["_count"], r["_rank"]): r["_scoreBonusPercent"] + for r in table("MasterLiveGekisouRankingScoreBonus")} songs = {} for r in table("MasterLiveMusic"): for d in musicdata.DIFFICULTIES: @@ -129,16 +140,28 @@ def table(name): music, d, missions = songs[c["scoreId"]] last = max(c["notes"]["timeMs"]) fevers = list(zip(c["fevers"]["startMs"], c["fevers"]["endMs"]))[:3] + positions = min(len(c["skillEvents"]["timeMs"]), 5) + pattern = 3 if missions == [1, 3, 3] else 0 + percents = [[bonus.get((pattern, i + 1, k), 0) for k in range(1, 6)] for i in range(len(fevers))] + check = {"deck": [[0, 5000]], "exact": 2000, "predicted": 2000.25, "bound": 7.0} s = {"scoreId": c["scoreId"], "musicId": music, "difficulty": d, "level": levels[c["scoreId"]], "judgedNotes": sum(score.is_judgement_note(op) for op in c["notes"]["op"]), "convertedNoteCount": len(c["notes"]["id"]), "lastNoteMs": last, "musicLengthMs": last + 1000, "skip": 1.5, "events": [[i % 5, t] for i, t in enumerate(c["skillEvents"]["timeMs"])], - "positions": min(len(c["skillEvents"]["timeMs"]), 5), "missions": missions, - "ranges": [{"index": i, "mission": missions[i], "startMs": a, "endMs": b, "rankBonusPercent": 10} + "positions": positions, "missions": missions, + "ranges": [{"index": i, "mission": missions[i], "startMs": a, "endMs": b, + "rankBonusPercent": percents[i][0], "rankBonusPercents": percents[i]} for i, (a, b) in enumerate(fevers)], "justNotes": 0, - "seeds": [{"seed": 0, "score": 1234, "ranges": [], "weights": [["W"]], - "check": {"deck": [[0, 5000]], "exact": 2000, "predicted": 2000.25, "bound": 7.0}}]} + "seeds": [{"seed": 0, "score": 1234, + "ranges": [{"rangeScore": 101 * (i + 1), "rankBonus": trunc_percent(101 * (i + 1), p[0]), + "maxCombo": 1, "justCount": 0, "lotResults": [0, 0, 0, 0], + "rangeScorePerfect": 101 * (i + 1)} for i, p in enumerate(percents)], + "weights": [["W"] * positions], "check": check, "scorePerfect": 1234, + "rangeWeights": [[["W"] * len(fevers)] * positions], + "rankCheck": {"ranks": [2] * len(fevers), "exact": 2001, "predicted": 2000.5, + "bound": 7.0} if fevers else None}], + "offSeeds": [{"seed": 0, "score": 1000, "weights": [["W"] * positions], "check": dict(check)}]} if self.change: self.change(s) charts.append(s) @@ -251,8 +274,14 @@ def test_deck(tmp_path): assert list(c["deck"]) == list(musicdata.DECK_CHART_KEYS) assert c["deck"]["events"] == [[0, 2500], [1, 500]] and c["deck"]["ranges"][1]["startMs"] == 2000 assert c["deck"]["unplayable"] is None and c["deck"]["skip"] == 1.5 + # the ranks: the song's mission pattern (3) rows of every range + assert [r["rankBonusPercents"] for r in c["deck"]["ranges"]] == [[35, 34, 33, 32, 31], [40, 38, 36, 34, 32]] + seed = c["deck"]["seeds"][0] + assert seed["rankCheck"]["ranks"] == [2, 2] and seed["scorePerfect"] == seed["score"] + assert [r["rankBonus"] for r in seed["ranges"]] == [35, 80] + assert c["deck"]["offSeeds"][0]["score"] == 1000 # the deck model's numbers as it writes them: a binary64 value is not narrowed to binary32 - assert b'"weights":[[' + WEIGHT.encode() + b']]' in raw and b'"predicted":2000.25' in raw + assert b'"weights":[[' + WEIGHT.encode() + b',' + WEIGHT.encode() + b']]' in raw and b'"predicted":2000.25' in raw assert "master" not in doc and "charts" not in doc schema_validator().validate(doc) (tmp_path / "m").rename(tmp_path / "m0") @@ -268,6 +297,21 @@ def test_deck(tmp_path): (lambda s: s["events"].pop(), "skill event times"), (lambda s: s["ranges"] and s["ranges"][0].update(endMs=1), "fevers"), (lambda s: s.update(difficulty="hard"), "difficulty 'hard' differs"), + (lambda s: s["ranges"] and s["ranges"][0]["rankBonusPercents"].__setitem__(2, 99), "range 0: .*percentages"), + (lambda s: s["ranges"] and s["ranges"][0].update(rankBonusPercent=1), "range 0: .*percentages"), + (lambda s: s["ranges"] and s["seeds"][0]["ranges"][0].update(rankBonus=-1), "rank bonus -1 is not trunc"), + (lambda s: s["seeds"][0]["ranges"].pop(), "range results for"), + (lambda s: s.pop("offSeeds"), "no Gekisou off statistics"), + (lambda s: s["offSeeds"].append(s["offSeeds"][0]), "no Gekisou off statistics"), + (lambda s: s["offSeeds"][0]["weights"].append(None), "Gekisou off: weights"), + (lambda s: s["offSeeds"][0]["check"].update(exact=0), "Gekisou off: the check deck scores 0"), + (lambda s: s["seeds"][0].update(scorePerfect=1), "otherwise on the Perfect play"), + (lambda s: s["seeds"][0]["weights"][0].pop(), "weights are not"), + (lambda s: s["seeds"][0]["weights"].append(None), "weights are not"), + (lambda s: s["seeds"][0].update(rangeWeights=[[]]), "rangeWeights are not"), + (lambda s: s["seeds"][0]["check"].update(bound=0.1), "the check deck scores 2000"), + (lambda s: s["ranges"] and s["seeds"][0]["rankCheck"].update(exact=0), "at ranks"), + (lambda s: s["ranges"] and s["seeds"][0]["rankCheck"].update(ranks=[6, 1]), "rank check ranks"), ]) def test_deck_checks(tmp_path, change, match): with pytest.raises(musicdata.MusicDataError, match=match): @@ -275,6 +319,33 @@ def test_deck_checks(tmp_path, change, match): assert not (tmp_path / "music.json").exists() +def test_deck_without_range_weights(tmp_path): + """A chart without range weights, a kind without them and a kind without Gekisou off weights are carried.""" + def change(s): + s["seeds"][0]["rangeWeights"] = None if s["scoreId"] == 10 else [None] + s["seeds"][0]["rankCheck"] = None + s["offSeeds"][0]["weights"] = [None] + export(tmp_path, deck=FakeDeck(change)) + doc = json.loads((tmp_path / "music.json").read_bytes()) + decks = {c["scoreId"]: c["deck"] for s in doc["songs"] for c in s["charts"]} + assert decks[10]["seeds"][0]["rangeWeights"] is None and decks[20]["seeds"][0]["rangeWeights"] == [None] + assert decks[30]["offSeeds"][0]["weights"] == [None] + schema_validator().validate(doc) + + +def test_rank_bonus_percents(): + assert [musicdata.mission_pattern(m) for m in ([1, 2, 0], [2, 2, 2], [1, 2, 3], [1, 1, 3], [1, 3, 3], + [3, 1, 3])] == [0, 1, 2, 3, 3, 3] + rows = [{"_missionPattern": 2, "_count": 1, "_rank": 1, "_scoreBonusPercent": 30}, + {"_missionPattern": 2, "_count": 3, "_rank": 5, "_scoreBonusPercent": 4}, + {"_missionPattern": 2, "_count": 3, "_rank": 5, "_scoreBonusPercent": 5}, # a later row wins + {"_missionPattern": 2, "_count": 4, "_rank": 1, "_scoreBonusPercent": 9}, # no fourth range + {"_missionPattern": 2, "_count": 1, "_rank": 6, "_scoreBonusPercent": 9}, # no sixth rank + {"_missionPattern": 1, "_count": 2, "_rank": 1, "_scoreBonusPercent": 9}] # another pattern + assert musicdata.rank_bonus_percents(rows, [1, 2, 3]) == [[30, 0, 0, 0, 0], [0] * 5, [0, 0, 0, 0, 5]] + assert musicdata.rank_bonus_percents(rows, [0, 0, 0]) == [[0] * 5] * 3 + + def test_deck_errors(tmp_path, monkeypatch): with pytest.raises(musicdata.MusicDataError, match="deck model: chart 30: the check deck scores"): export(tmp_path, deck=FakeDeck(fail="chart 30: the check deck scores 1, the chart statistics predict 2"))