Extract and attach context once at runtime once per run and remove ambiguous kwargs

[dtronmans]

Purpose

• refactors evaluation-time context handling to make parser and metric interfaces cleaner and less dependent on arbitrary kwargs
• Introduced a shared EvalContext which holds run-level metadata like model shape, class mappings, category ids, and target conversion helpers
• Now builds that context once during setup and attaches it to parsers

Before vs. after for parsing and metric updates:

  # Before

  # eval loop
  predictions = parser.parse(
      output,
      model_spec=model_spec,
      class_map=class_map,
      **parser_cfg.params,
  )

  metric.update(
      predictions=predictions,
      target=target,
      **metric_ctx,
  )

  # Extension points
  class MyParser(BaseParser):
      def parse(self, output, model_spec, **kwargs) -> Prediction: ...

  class MyMetric(BaseMetric):
      def update(self, predictions, target, **kwargs) -> None: ...

  # After

  # eval setup
  parser.attach_context(eval_context)
  metric.attach_context(eval_context)

  # eval loop
  predictions = parser.parse(output)
  metric.update(predictions=predictions, target=target)

  # Extension points
  class MyParser(BaseParser):
      def parse(self, output) -> Prediction:
          context = self.require_context()
          ...

  class MyMetric(BaseMetric):
      def update(self, predictions, target) -> None:
          context = self.require_context()
          ...

Specification

None / not applicable

Dependencies & Potential Impact

None / not applicable

Deployment Plan

None / not applicable

Testing & Validation

None / not applicable

AI Usage

Assisted-by: AGENT_NAME:MODEL_VERSION [TOOL1] [TOOL2]

Submitted code was reviewed by a human: YES/NO

The author is taking the responsibility for the contribution: YES/NO

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e9407ac1-a174-4112-adad-d252395ec1f5

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}