Anonymous user
Talk:Sequence of non-squares: Difference between revisions
Numeric issues
(New page: I chose this as a way to show how easy it is to investigate functions in a programming language. --~~~~) |
(Numeric issues) |
||
Line 1:
I chose this as a way to show how easy it is to investigate functions in a programming language. --[[User:Paddy3118|Paddy3118]] 08:42, 24 August 2008 (UTC)
: The formula need to be investigated for numeric stability. Calculation of sqrt is inexact, it need to be shown that for all n in question, floor(1/2 + sqrt(n)) yields the exact result. A minimal requirement for this (though insufficient, I guess) is that sqrt has an error below 0.5. Note that the addition following to squaring will normalize for big n. Also conversion of those to floating point becomes quickly inexact when 32-bit floats are used. --[[User:Dmitry-kazakov|Dmitry-kazakov]] 09:51, 24 August 2008 (UTC)
| |||