Add 64-bit offset to pointer

Question

F # has a NativePtr module, but it seems to only support 32-bit offsets for its add / get / set functions, as System.IntPtr does.

Is there a way to add a 64-bit offset to the internal pointer (nativeptr <'a>) in F #? Of course, I could convert all addresses to 64-bit integers, perform normal integer operations, and then convert the result to nativeptr <'a> again, but it would be worth the extra add and imul instructions. I really want the AGUs to do address calculations.

For example, using insecure in C #, you can do something like

void* ptr = Marshal.AllocHGlobal(...).ToPointer(); int64 offset = ...; T* newAddr = (T*)ptr + offset; // T has to be an unmanaged type 

Well, actually you cannot, because there is no "uncontrolled" restriction for type parameters, but at least you can do general pointer arithmetic in a non-standard way.

In F #, we finally got an unmanaged constraint; but how can I do pointer arithmetic?

Answer 1

I am not an expert in this field, but I looked at the F # implementation of the NativePtr module, and I think that there is no performance associated with converting nativeptr<'a> to nativeint and vice versa.

The implementation uses inline IL, and the embedded IL code does not contain any code - it should just make the F # compiler assume that the value on the stack is of a different type:

 let inline ofNativeInt (x:nativeint) = (# "" x : nativeptr<_> #) let inline toNativeInt (x:nativeptr<_>) = (# "" x : nativeint #) 

In fact, the NativePtr.add method also uses these two methods - it converts the pointer to nativeint , and then adds a 32-bit integer (times the size of type 'a ).

So, the following function should be in order:

 let inline addNativeInt (x:nativeptr<'a>) (n:nativeint) : nativeptr<'a> = (NativePtr.toNativeInt x) + n |> NativePtr.ofNativeInt 

All functions used in the code must be built-in, so in the end you will get only one instruction to add (although I have not confirmed this yet). You don’t even have to worry about using the function several times in your code (you can work with nativeptr<'a> all the time and use this function to add).

However, partitioning data can also be an option - as far as I know, the MSR command, which used F # to process some large (> 2 GB) data sets, used this very approach - they split the data into 2 GB blocks (stored in arrays).

Answer 2

Visualization data can easily exceed 4 gigabytes;)

It cannot, the x64 code has an offset limit of +/- 2 GB. One of the reasons you cannot allocate arrays larger than 2 GB in .NET. This limitation also exists in unmanaged 64-bit C / C ++ code. There are libraries that work around this limitation, for example WIC, but accessing all bits of the bitmap directly does not make sense when you use them.

However, you can create such an address by running IntPtr in C #:

 IntPtr addr = SomeCall(); long offset = blah; addr = (IntPtr)((long)addr + offset);