Skip to content

fix: tokenize numeric separators in JavaScript exponents - #6018

Open
CloudCorpRecords wants to merge 1 commit into
ajaxorg:masterfrom
CloudCorpRecords:js-numeric-separator-exponent-5256
Open

CloudCorpRecords wants to merge 1 commit into
ajaxorg:masterfrom
CloudCorpRecords:js-numeric-separator-exponent-5256

Conversation

@CloudCorpRecords

@CloudCorpRecords CloudCorpRecords commented Oct 26, 2026 •

Copy link
Copy Markdown

Issue #, if available: #5256

Description of changes:
The JavaScript highlight rules already tokenized digit separators in the mantissa and in hex, octal, and binary literals, but the exponent part of the decimal rule only accepted plain digits. That split literals like 1e1_0 into a numeric token plus an identifier. This change allows separators in the exponent digits, using the same digit-plus-separator pattern the mantissa already uses, so 1e1_0, 1_000e1_0, .5_5e1_0, and 1E+1_0 each tokenize as one constant.numeric token. The PHP rules already handled separators, including exponents, so no PHP change was needed.

Tests run:

  • src/mode/javascript_test.js: 20 passed, 1 pre-existing skip, including a new tokenizer test covering the exponent cases plus mantissa and plain-exponent regressions. The new test fails on the base commit and passes with the fix.
  • src/mode/php_test.js: 1 passed.
  • src/mode/html_test.js and src/mode/coldfusion_test.js (both import the JavaScript rules): 2 passed each.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Pull Request Checklist:

  • No backwards incompatible changes were made to Ace's public interface which is defined through the main typings file (ace.d.ts) and its references: no typings files were changed.

Open kitchen-sink @ 3498664f50183e41276c22d8451c11931615c5be

The JavaScript highlight rules already tokenized digit separators in the mantissa and in hex, octal, and binary literals, but the exponent part of the decimal rule only accepted plain digits, so literals like 1e1_0 were split into a numeric token plus an identifier. Allow separators in the exponent digits so the whole literal tokenizes as one constant.numeric token. The PHP rules already handled separators, including exponents, so no PHP change was needed. Adds a tokenizer test covering exponent separators plus mantissa and plain exponent regressions.

Signed-off-by: Rene Turcios <95189389+CloudCorpRecords@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant