baffled by wrong result

Just starting out? Need help? Post your questions and find answers here.
jack
Addict
Addict
Posts: 1336
Joined: Fri Apr 25, 2003 11:10 pm

baffled by wrong result

Post by jack »

hello
some time ago I wrote some basic mp arithmetic routines in FreeBasic and it works ok though not very fast, now I am trying to convert the code to PB and for the most part it works but for the fpinv function which calculates 1/x via newton-raphson
perhaps the problem is due to no unsigned integer type?
Last edited by jack on Sun May 22, 2022 1:06 am, edited 2 times in total.
User avatar
idle
Always Here
Always Here
Posts: 5042
Joined: Fri Sep 21, 2007 5:52 am
Location: New Zealand

Re: baffled by wrong result

Post by idle »

what's the result supposed to be?
You can use a quad for longs and mask off with $FFFFFFFF to get unsigned when you need it.
Post Reply