The lexer was not allowing underscores between leading zeros in decimal integer
literals and float literals (e.g. 0_0 or 0_0.0).
library/lexer.m:
Allow underscores in the above cases.
tests/hard_coded/parse_number_from_io.{m,exp,exp2,exp3}:
tests/hard_coded/parse_number_from_string.{m,exp,exp2,exp3}:
Extend these tests to cover the above cases.
tests/invalid/invalid_binary_literal.err_exp:
tests/invalid/invalid_octal_literal.err_exp:
tests/invalid/invalid_hex_literal.err_exp:
Conform to the above change.
Allow the optional use of underscores in numeric literals for the purpose of
improving their readability (e.g. by grouping digits etc). We allow any number
of underscores between digits and also between the radix prefix (if present) and
the initial digit. (When integer type suffixes are supported we will also
allow them to be preceded by any number of underscores.) The following are
*not* allowed:
1. Leading underscores.
2. Trailing underscores.
3. Underscores inside the components of a radix prefix (e.g.
0_xffff or 0__b101010.)
4. Underscores immediately adjacent to the decimal point in a float
literal (e.g. 123_._123.)
5. Underscores immediately adjacent to the exponent ('e' or 'E) in
a float literal (e.g. 123_e12 or 123E_12.)
6. Underscores immediately adjacent to the optional sign of an exponent
in a float literal (e.g. 123_+e12 or 123-_E12.)
7. Underscores between the optional sign of an exponent and the exponent
indicator (e.g. 123+_e12.)
library/lexer.m:
Modify the scanner to account of underscores in numeric literals according
to the scheme above.
library/string.m:
library/integer.m:
Export undocumented functions for converting strings containing underscores
into ints or integers respectively.
tests/hard_coded/parse_number_from_io.{m,exp}:
Test parsing of valid numeric literals from file streams.
tests/hard_coed/parse_number_from_string.{m,exp}:
Test parsing of valid and invalid numeric literal from string.
tests/invalid/invalid_binary_literal.{m,err_exp}:
tests/invalid/invalid_decimal_literal.{m,err_exp}:
tests/invalid/invalid_octal_literal.{m,err_exp}:
tests/invalid/invalid_hex_literal.{m,err_exp}:
tests/invalid/invalid_float_literal.{m,err_exp}:
Test parsing of invalid numeric literals from file streams.
tests/hard_coded/parse_number_from_{io,string}.m:
tests/hard_coded/parse_number_from_{io,string}.exp:
Test parsing of valid numeric literals.
tests/hard_coded/Mmakefile:
tests/invalid/Mmakefile:
Add the new test cases.
