Skip to content

fix(format): compare JSON numbers exactly - #240

Merged
helloml0326 merged 1 commit into
agentscope-ai:mainfrom
Elon-Jiang:fix/json-number-comparison
Nov 16, 2026
Merged

helloml0326 merged 1 commit into
agentscope-ai:mainfrom
Elon-Jiang:fix/json-number-comparison

Conversation

@Elon-Jiang

Copy link
Copy Markdown
Contributor

OpenJudge Version

0.2.5, main 5d737c9.

Description

JsonMatchGrader gives a perfect match for 1.0000000000000001 versus 1.0, or 1e400 versus 2e400, because json.loads rounds fractional/exponent values to binary floats before comparison. Conversely, the equivalent values 9007199254740993 and 9007199254740993.0 do not match.

Parse fractional/exponent tokens with standard-library Decimal in both inputs. Keep integers, recursive comparison, unordered matching, boolean distinction and rejection of non-JSON NaN/Infinity literals unchanged. Parsed numbers remain internal; the returned score and metadata stay JSON-serializable.

Validation:

  • Public aevaluate regressions on unchanged main: 12 failed, 85 passed.
  • Format grader suite after fix: 128 passed.
  • pre-commit run --all-files --verbose: all hooks pass, including configured pytest 1263 passed, 139 skipped, five existing NumPy zero-variance warnings.
  • git diff --check: passed.

Tests cover nested ordered/unordered values, precision loss, overflow/underflow, integer/decimal equivalence, signed zero and boolean controls. No models, providers or service substitutes were used. Configured exclusions are multimodal/generator/data/examples; skipped provider tests did not run. Existing #232 rejects nonstandard literals; this fixes valid JSON numbers. #238 concerns the separate text NumberAccuracyGrader.

AI assistance: OpenAI Codex prepared the patch and ran validation.

Checklist

  • Code has been formatted with pre-commit run --all-files.
  • All configured tests pass; exclusions/skips described above.
  • Docstrings follow project style.
  • Related documentation has been updated.
  • Code is ready for review.

@helloml0326
helloml0326 merged commit d731453 into agentscope-ai:main Nov 16, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants