UInt64 gets iffy when doing a lot of math?

I have a strange result coming from some great math, and don't know why I am getting a different answer from vb.net vs python.

Below are quick snippets and results:

VB.NET Dim MSB As UInt32 = 3067297518 Dim LSB As UInt32 = 1439785590 Dim sqln As UInt64 = MSB * (2 ^ 32) + LSB Python: sqln = msb * (2 ** 32) + lsb Python Result: 13173942528351756918 VB RESULT: 13173942528351756288 

Note. I am also trying to declare sqln as ULong and Double (same answer) MSB and LSB are a match in both debuggers - !! ??

Any ideas? = + My thanks

Great John is very eloquent and it works! One small continuation, could you suggest a fix for the final part? I believe the same thing happens even if you got my sqln straightened up :)

 python says: = bdntyxtax2smq vb.net says: = bfpuzytbx3s00 VB.NET Dim sqlid As String = "" Dim alphabet As String = "0123456789abcdfghjkmnpqrstuvwxyz" For iCount = 0 To 12 sqlid = alphabet((sqln / (32 ^ iCount)) Mod 32) + sqlid Next Python: for i in range(0, 13): sqlid = alphabet[(sqln / (32 ** i)) % 32] + sqlid 
+4
source share
1 answer

I decompiled your VB code as C # and it looks like this:

 uint MSB = 0xb6d33eee; uint LSB = 0x55d16276; ulong sqln = (ulong) Math.Round((double) ((num2 * 4294967296) + num)); 

This is because the statement in VB always returns Double :

As a result, a number raised to an exponent is always a double value.

I would suggest using the shift operator and make all your UInt64 variables, so the shift is done with a 64-bit integer:

 Dim MSB As UInt64 = 3067297518 Dim LSB As UInt64 = 1439785590 Dim sqln As UInt64 = (MSB << 32) + LSB 

This gives the correct answer. (Actually, you do not need LSB be UInt64 , but you can do everything with 64-bit integers.)

+4
source

Source: https://habr.com/ru/post/1399257/


All Articles