Skip to content

Increased String::num default decimal precision

Rafał Mikrut requested to merge github/fork/Heikki00/34541_to_json_precision into master

Created by: Heikki00

Fixes #34541

Increased MAX_DIGITS to 16, and made String::num use MAX_DIGITS consistently. If -1 is passed as decimal precision to String::num, it now gets changed to MAX_DIGITS instead of using printf default(which is 6)

Also made the types used in floating-point math more consistent in a few places.

Code from the original issue works correctly with these changes:

# To check actual precision of floats:
print("%.16f" % PI)
print("%.16f" % (1.0 / 3.0))

# Results after going through `to_json`
print(to_json(PI))
print(to_json(1.0 / 3.0))

# Round-trip test
print("%.16f" % (PI - parse_json(to_json(PI))))

Now outputs:

3.1415926535897931
0.3333333333333333
3.1415926535897931
0.3333333333333333
0.0000000000000000

Merge request reports