C # joins and copy array

Question

I am trying to copy an array of Struct1 to an array of Struct2 (same binary representation) as quickly as possible. I have defined a union for the conversion between Struct1[] and Struct2[] , but when I call Array.Copy, I get an exception indicating that the array is the wrong type. How can I get around this? Buffer.BlockCopy accepts only primitive types. Here is the code:

 [StructLayout(LayoutKind.Explicit)] public struct Struct12Converter { [FieldOffset(0)] public Struct1[] S1Array; [FieldOffset(0)] public Struct2[] S2Array; } public void ConversionTest() { var s1Array = new{new Struct1()} var converter = new Struct12Converter{S1Array = s1Array}; var s2Array = new Struct2[1]; Array.Copy(converter.S2Array,0,s2Array,0,1) //throws here //if you check with the debugger, it says converter.S2Array is a Struct1[], //although the compiler lets you use it like a Struct2[] //this has me baffled as well. } 

To provide more detailed information: I wanted to experiment to see if different performance characteristics work with a variable structure and change the value of their fields compared to working all the time using the same immutable structure. I think it should be similar, but I thought it was worth it to measure. The main application will be a low latency socket library in which I am currently using the ArraySegment<byte> socket ArraySegment<byte> . It so happened that in the SocketAsyncEventArgs api setting the BufferList property starts a copy of the array in which my “experiment” is taking place (I have an array MutableArraySegment , which I cannot convert to ArraySegment[] using the same method as before, which makes my comparison meaningless).

Answer 1

Warning: This can be a dangerous trick, as it bypasses the type system and the assembly is not subject to verification. Nevertheless, some surface tests did not cause any obvious problems, and for your "experiment" it may be worth it. Check out @usr warnings in the comments though ...

According to your assumptions (if you can tolerate an uncontrolled output collection), you do not need Marshal.XXX , Array.Copy or memcpy at all. You can read values ​​from a union type as an array of Struct1 or an array of Struct2 . My guess, although I have no evidence to support it, is that the runtime and the GC will not notice a mismatch between the type of the array and how you use these elements.

Here is a separate example that will be launched in LinqPad. The default packaging means that you really don't need the LayoutKind and FieldOffset in Struct1 and Struct2 (although, of course, you do this in the union type Struct12Converter ), but it helps to show it explicitly.

 [StructLayout(LayoutKind.Explicit)] public struct Struct1 { [FieldOffset(0)] public int Int1; [FieldOffset(4)] public int Int2; } [StructLayout(LayoutKind.Explicit)] public struct Struct2 { [FieldOffset(0)] public long Long; } [StructLayout(LayoutKind.Explicit)] public struct Struct12Converter { [FieldOffset(0)] public Struct1[] S1Array; [FieldOffset(0)] public Struct2[] S2Array; } public void ConversionTest() { var int1 = 987; var int2 = 456; var int3 = 123456; var int4 = 789123; var s1Array = new[] { new Struct1 {Int1 = int1, Int2 = int2}, new Struct1 {Int1 = int3, Int2 = int4}, }; // Write as Struct1s var converter = new Struct12Converter { S1Array = s1Array }; // Read as Struct2s var s2Array = converter.S2Array; // Check: Int2 is the high part, so that must shift up var check0 = ((long)int2 << 32) + int1; Debug.Assert(check0 == s2Array[0].Long); // And check the second element var check1 = ((long)int4 << 32) + int3; Debug.Assert(check1 == s2Array[1].Long); // Using LinqPad Dump: check0.Dump(); s2Array[0].Dump(); check1.Dump(); s2Array[1].Dump(); } void Main() { ConversionTest(); } 

Answer 2

If the Structs match exactly, you can accomplish this using the Marshal.PtrToStructure method.

You need to get a pointer to your structure, and then you can "deserialize" it back to another structure (which should have the EXACT same layout).

Here you can see an example.

Hope this helps, Ophir.

Answer 3

Do you know that you violated the type system by inadvertently viewing Struct1[] as Struct2[] ? This puts the CLR in undefined state. He may suggest that variables of type Struct1[] do indeed point to an instance of Struct1[] . Now you can see almost any strange behavior. (This is not a security issue. This code is not verifiable and requires full trust.)

In other words, you did not convert the contents of the array, but received a reference to the converted object.

Copying an array of blittable objects is usually done in the fastest way using memcpy . A manual copy loop is equivalent to this, but I will not trust JIT to optimize it in memcpy . JIT performs only basic optimizations from the current version.

Answer 4

This code is intentionally unsafe (since what you want to do is unsafe, AFAIK CLR / JIT can reorder structures for performance reasons)

Also note that the signature for MemCpy may vary depending on the version of the framework (it is still internal)

For performance reasons, you should cache this delegate correctly.

The idea from this question is here.

  unsafe delegate void MemCpyImpl(byte* src, byte* dest, int len); static MemCpyImpl memcpyimpl; public unsafe static void Copy(void* src, void* dst, int count) { byte* source = (byte*)src; byte* dest = (byte*)dst; memcpyimpl(source, dest, count); } 

Then forcing your arrays to be byte arrays (actually invalid *, but ignore the details)

  public static void ConversionTest() { var bufferType = typeof(Buffer); unsafe { var paramList = new Type[3] { typeof(byte*), typeof(byte*), typeof(int) }; var memcpyimplMethod = bufferType.GetMethod("Memcpy", BindingFlags.Static | BindingFlags.NonPublic, null, paramList, null); memcpyimpl = (MemCpyImpl)Delegate.CreateDelegate(typeof(MemCpyImpl), memcpyimplMethod); } Struct1[] s1Array = { new Struct1() { value = 123456789 } }; var converter = new Struct12Converter { S1Array = s1Array }; var s2Array = new Struct2[1]; unsafe { fixed (void* bad = s2Array) { fixed (void* idea = converter.S2Array) { Copy(bad, idea, 4); } } } }