fix: uncomment and fix CalculateTopRanks, optimize core functions

[lenisko]

CalculateTopRanks:

• Fix lastRank tracking: populate lastRank[i] so duplicate detection works across level caps
• Fix Capped mutation: use index-based tracking (lastRankIdx) to modify rankings directly instead of local copies
• Pass comparator to calculateRanksCompact; pass stats as pointer
• Replace O(n²) Capped propagation with O(n) index lookup

Optimizations:

• Simplify RankingComparatorPreferHigherCp/PreferLowerCp with early return
• Replace reflect.DeepEqual with sha256 hash comparison in WatchPokemonData

Cleanup:

• Remove "Current State" section from README
• Remove commented-out CalculateAllRanks test/benchmark

[jfberry]

Do the masterfile changes align with the PR for golbat to combine the masterfiles?
I'd really like there to be one lol ;-)

[lenisko]

@jfberry point me to it please

[jfberry]

We already merged something (by mygod) that causes golbat to own the masterfile

[lenisko]

In this case I don't expect any issues. haven't touched this flow except ability to override mastefile url.

[jfberry]

@lenisko — looking for your guidance on an overlap. Golbat hit the §1.6 races in production (fatal error: concurrent map read and map write in QueryPvPRank during a masterfile reload — details in UnownHash/Golbat#403), and I opened #19 as a narrow fix before spotting that this PR had already diagnosed and fixed the same thing in February. Sorry for the parallel work — the fault analysis in both PRs agrees completely; they differ only in mechanism, and they conflict textually in every shared file, so we should pick one vehicle.

On the concurrency designs specifically:

• This PR: mu sync.RWMutex with short read-locked sections per query, atomic.Pointer[sync.Map] for the cache, atomic.Bool for safetyCheck. Correct, and the critical sections are tiny. The cost is that every query pays an RLock/RUnlock pair — two atomic read-modify-writes on a shared cache line. Under heavily parallel query load (Golbat runs ~100 decode workers calling QueryPvPRank concurrently, plus API scans) that line bounces between cores even when there's never a writer.
• fix: make masterfile reload safe under concurrent queries #19: masterfile data + rank cache in one immutable bundle behind a single atomic.Pointer; queries do one plain atomic load (no RMW, no shared-line writes), and a reload is one pointer swap covering data and cache together. Same correctness, cheaper read path, and reload can't interleave between the data swap and the cache clear.

The difference only matters at scanner-scale query concurrency — but that's gohbem's main audience, so I'd lean toward the pointer-swap read path whichever PR carries it.

One more thing worth merging regardless of direction: #19 has a dedicated -race regression test (TestReloadConcurrentWithQueries) that reloads and clears cache concurrently with QueryPvPRank/CalculateCp. It fails on master, and it's portable to either design — this PR's CI -race addition wouldn't currently catch the original bug without a test like it. Happy to contribute it here.

How would you like to proceed? Options as I see them:

1. Land fix: make masterfile reload safe under concurrent queries #19 first as the minimal crash fix, then rebase this PR's remaining work (CalculateTopRanks revival, perf, HTTP hardening, CI) on top, dropping its locking layer.
2. Fold the bundle approach + race test into this PR and close fix: make masterfile reload safe under concurrent queries #19.
3. Keep this PR's mutex design as-is; I close fix: make masterfile reload safe under concurrent queries #19 and just PR the regression test.

No strong attachment to my code landing — the goal is reload safety merged for consumers. Whatever's easiest for you.

[jfberry]

Follow-up with numbers, to make the concurrency-design discussion concrete. I ran the same benchmark file (public API only, so it compiles unmodified) against three heads: master, this PR, and #19. This PR is based on current master, so the comparison is apples-to-apples. Apple M3 Pro, -count=6 (uncached: -count=4 -benchtime=300x), medians:

Workload	master (racy)	#18 (RWMutex)	#19 (atomic bundle)
QueryPvPRank cached, serial	1.69 µs	1.70 µs	1.77 µs
QueryPvPRank cached, parallel (12)	647 ns	858 ns	648 ns
CalculateCp, parallel (12)	2.7 ns	88 ns	3.1 ns
QueryPvPRank uncached compute, serial	3.91 ms	3.87 ms	3.91 ms

Two observations, offered neutrally:

1. The lock cost is visible under parallelism. The RLock/RUnlock pair is two atomic RMWs on a shared cache line, and at 12 threads that's +33% on cached QueryPvPRank and ~30× on CalculateCp (where the lock swamps a ~3ns function). The bundle's single atomic load tracks racy master within noise in the parallel cases — which are the ones that map to scanner deployments (Golbat runs ~100 decode workers through QueryPvPRank concurrently). fix: make masterfile reload safe under concurrent queries #19's one visible cost is ~5% on serial cached queries (extra pointer indirection), which disappears in the parallel case.

2. On this particular query shape the compute-path optimizations don't show — serial cached and uncached are within noise of master. That's one shape (661, 4 leagues, 2 caps); they may well pay off elsewhere, e.g. in the revived CalculateTopRanks, which nothing here measures.

Offer: if you like the direction, I'm happy to do the merge work mechanically — #19's bundle as the concurrency layer, with everything else in this PR (CalculateTopRanks, perf work, HTTP hardening, CI) rebased on top, under whichever PR number you prefer. Equally fine to hand you the bundle + the -race regression test to fold in yourself. Your call.

Benchmark source (drop into the repo root as bench_compare_test.go on any of the three heads)

package gohbem

import "testing"

var benchCompareLeagues = map[string]League{
	"little": {Cap: 500, LittleCupRules: true},
	"great":  {Cap: 1500, LittleCupRules: false},
	"ultra":  {Cap: 2500, LittleCupRules: false},
	"master": {Cap: 0, LittleCupRules: false},
}

var benchCompareLevelCaps = []int{50, 51}

func benchCompareSetup(b *testing.B) *Ohbem {
	b.Helper()
	ohbem := &Ohbem{Leagues: benchCompareLeagues, LevelCaps: benchCompareLevelCaps}
	if err := ohbem.LoadPokemonData("./test/master-test.json"); err != nil {
		b.Fatalf("LoadPokemonData: %v", err)
	}
	if pvp, err := ohbem.QueryPvPRank(661, 0, 0, 1, 15, 15, 14, 1); err != nil || len(pvp) == 0 {
		b.Fatalf("warmup: pvp=%v err=%v", pvp, err)
	}
	return ohbem
}

func BenchmarkCompareQueryCachedSerial(b *testing.B) {
	ohbem := benchCompareSetup(b)
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		_, _ = ohbem.QueryPvPRank(661, 0, 0, 1, 15, 15, 14, 1)
	}
}

func BenchmarkCompareQueryCachedParallel(b *testing.B) {
	ohbem := benchCompareSetup(b)
	b.ResetTimer()
	b.RunParallel(func(pb *testing.PB) {
		for pb.Next() {
			_, _ = ohbem.QueryPvPRank(661, 0, 0, 1, 15, 15, 14, 1)
		}
	})
}

func BenchmarkCompareQueryUncachedSerial(b *testing.B) {
	ohbem := &Ohbem{Leagues: benchCompareLeagues, LevelCaps: benchCompareLevelCaps, DisableCache: true}
	if err := ohbem.LoadPokemonData("./test/master-test.json"); err != nil {
		b.Fatalf("LoadPokemonData: %v", err)
	}
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		_, _ = ohbem.QueryPvPRank(661, 0, 0, 1, 15, 15, 14, 1)
	}
}

func BenchmarkCompareCalculateCpParallel(b *testing.B) {
	ohbem := benchCompareSetup(b)
	b.ResetTimer()
	b.RunParallel(func(pb *testing.PB) {
		for pb.Next() {
			_, _ = ohbem.CalculateCp(661, 0, 0, 15, 15, 14, 40)
		}
	})
}

Run with: go test -run XXX -bench BenchmarkCompare -count=6 -benchmem .