Talk:Convert decimal number to rational: Difference between revisions

m
→‎J and best fit: superduperscripts
m (→‎J and best fit: superduperscripts)
Line 41:
:I believe J floats are bog standard ieee floats which means just under 16 decimal digits of precision. And, yes, r means ratio -- it's the first integer divided by the second integer. As for 866492568306 divided by 1671094481399, when you divide that out and compare 866492568306 with 866492568305 and 866492568307, you have to drop down to 11 digits of precision to before the +1 and -1 versions look identical. That means that 866492568306 is roughly accurate to 12 digits of precision. In other words, we might be ignoring up to 16 trailing binary digits with that answer (and apparently the algorithm being used in that version of J apparently requires that we keep over 10 digits of precision).
:I have not studied this matter very closely, but this seems like it is related to the way J checks for equality on floating point numbers:
:J uses an epsilon which is scaled by the value with the largest magnitude when checking for equality . The idea is that floating point numbers that are meant to be equal are usually quite close. In other words, when the difference between the two floating point values is less than (in the default case) 2^<sup>-44</sup> times the larger magnitude value they are "equal". This does not always do the right thing (floating point subtraction can get awful), and can be turned off when the programmer is prepared to deal with proper numerical analysis, but it's usually the right thing to do when using an algorithm that cares about equality in the context of floating point numbers. --[[User:Rdm|Rdm]] 20:30, 14 June 2011 (UTC)
:: I see. 2^<sup>-44</sup> epsilon makes it sound like J's doulbe is the typical 8 byte IEEE float, so if J were just taking the bits out of the float, the denominator would be a power of 2 or power of 10. It probably did something similar to the continued fraction, but with a rather large denominator limit. For the record, for .518518 my perl code also gives goofy numbers like 30117673588/58084142861 from 10^<sup>11</sup> on, so it's just a matter of different rounding then. --[[User:Ledrug|Ledrug]] 21:15, 14 June 2011 (UTC)
Anonymous user