Anonymous user
Talk:Convert decimal number to rational: Difference between revisions
Talk:Convert decimal number to rational (view source)
Revision as of 10:22, 8 October 2011
, 12 years ago→J and best fit: superduperscripts
m (→Name change?) |
m (→J and best fit: superduperscripts) |
||
Line 41:
:I believe J floats are bog standard ieee floats which means just under 16 decimal digits of precision. And, yes, r means ratio -- it's the first integer divided by the second integer. As for 866492568306 divided by 1671094481399, when you divide that out and compare 866492568306 with 866492568305 and 866492568307, you have to drop down to 11 digits of precision to before the +1 and -1 versions look identical. That means that 866492568306 is roughly accurate to 12 digits of precision. In other words, we might be ignoring up to 16 trailing binary digits with that answer (and apparently the algorithm being used in that version of J apparently requires that we keep over 10 digits of precision).
:I have not studied this matter very closely, but this seems like it is related to the way J checks for equality on floating point numbers:
:J uses an epsilon which is scaled by the value with the largest magnitude when checking for equality . The idea is that floating point numbers that are meant to be equal are usually quite close. In other words, when the difference between the two floating point values is less than (in the default case) 2
:: I see. 2
|