Fix toPrecision #16
Open
Fix toPrecision #16
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
amoshydra
force-pushed
the
fix-rounding-and-to-precision
branch
5 times, most recently
from
263cdbf to
0306510
Compare
amoshydra
force-pushed
the
fix-rounding-and-to-precision
branch
from
0306510 to
8f29b7d
Compare
|
turns out Marking this PR as ready for review |
this change the expectation of exponent returned from the original test
amoshydra
force-pushed
the
fix-rounding-and-to-precision
branch
from
8f29b7d to
2aada21
Compare
|
Disclaimer I am having a hard time understanding what Also the implementation slightly differ from the spec, as I couldn't get the implementation to work by following these highlighted part: red annotation reflects the changes I've applied |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
toPrecision, implementtoPrecisionaccording to specificationnew Decimal128("123.456").toPrecision({ digits: 2 })"12e+2""12e+1"new Decimal128("0.0").toPrecision({ digits: 1 })"0.0""0"toPrecisioncloser to specification.some variables are renamed and created to match the text in the spec
Related to
#15
How is this tested?
See updated tests for
toPrecision