Marshal struct with dynamic size array

Question

I am having trouble marshalling a struct with a dynamic size array. The fixed length arrays are simple; just add

[MarshalAs(UnmanagedType.ByValArray, SizeConst = TheSizeOfTheArray)]

However, I'm at a loss when it comes to dynamic size arrays. For simplicity, I will omit everything that's not relevant in my code.

The device I am sending this serialized struct to expects an ushort to inform about the length of the array, followed by the array itself, and a CRC at the end.

[StructLayout(LayoutKind.Sequential, Pack = 1, CharSet = CharSet.Ansi)]
public struct MyNetworkMessage
{
    public ushort Length { get; }

    // This attribute does not work as I had hoped
    [MarshalAs(UnmanagedType.ByValArray, SizeParamIndex = 1)]
    private byte[] _messageData;

    public ushort Crc { get; private set; }

    public byte[] MessageData
    {
        get { return _data; }
        private set { _data = value; }
    }

    public MyNetworkMessage(byte[] data)
    {
        MessageData = data;
        Length = (ushort)data.Length;
        Crc = Helper.CalcCrc(MessageData);
    }
}

This struct needs to be serialized to a byte array which is sent over the wire to another device, where the first two bytes is the length of the array, and the last two bytes is the CRC of the MessageData:

Byte 0..1       Length of Data-field, N
Byte 2..N+2     Data
Byte N+3..N+4   CRC

I have many different structs like this that need to be serialized and sent over the wire as byte arrays, so a generic way of handling this is what I am after. Creating a correct byte array for this one example is simple enough, but I don't want to have to write serialization/deserialization for each and every struct, when they all only contain simple data.

I have seen similar questions asked earlier here, marked as duplicates, without seeing any satisfactory answers.

@MatthewWatson how are attributes going to help with a dynamic size array? Arguments you pass to attribute should be known at comile-time. — Spo1ler
– Spo1ler, Commented Mar 3, 2015 at 9:55
@Spo1ler I was thinking he could use it in the method call (where it is used to indicate an argument to dynamically specify the array size), but actually it looks like he's serializing it manually. — Matthew Watson
– Matthew Watson, Commented Mar 3, 2015 at 10:44
@MatthewWatson: No, just this one. However, the contents of other structs may be placed in the MessageData field in this struct, and those structs may also contain dynamic size arrays.. — Walkingsteak
– Walkingsteak, Commented Mar 3, 2015 at 11:27

Matthew Watson · Accepted Answer · 2015-03-03 13:57:24Z

You could write your own simple serialization logic. You could also write your own attribute with which to decorate the fields that you want to be serialised.

Here's a complete compilable console application which demonstrates this idea.

Note how it creates a new attribute class, NetworkSerialisationAttribute, and uses it to decorate the serialisable fields. It also uses reflection to determine what fields to serialise.

This sample code only supports byte arrays and primitive types, but it should be enough to get you started. (It also only supports serialising fields, not properties, and it doesn't do deserialisation - but from what you said I think that would be enough.)

The idea here is to avoid having to write lots of repetitive serialisation code. Instead, you just use the [NetworkSerialisation] attribute to tell it what to serialise.

Note that most of the code here is only written once; then you can put it in a library and use it for all your data transfer types. For example, MyNetworkMessage and MyOtherNetworkMessage in the code below represent the data transfer types.

using System;
using System.ComponentModel.Composition;
using System.IO;
using System.Linq;
using System.Reflection;

namespace ConsoleApplication2
{
    public enum SerialisationKind
    {
        Scalar,
        Array
    }

    [MetadataAttribute]
    public sealed class NetworkSerialisationAttribute: Attribute
    {
        public NetworkSerialisationAttribute(int ordinal, SerialisationKind kind = SerialisationKind.Scalar)
        {
            _ordinal = ordinal;
            _kind = kind;
        }

        public SerialisationKind Kind // Array or scalar?
        {
            get
            {
                return _kind;
            }
        }

        public int Ordinal // Defines the order in which fields should be serialized.
        {
            get
            {
                return _ordinal;
            }
        }

        private readonly int _ordinal;
        private readonly SerialisationKind _kind;
    }

    public static class NetworkSerialiser
    {
        public static byte[] Serialise<T>(T item)
        {
            using (var mem = new MemoryStream())
            {
                serialise(item, mem);
                mem.Flush();
                return mem.ToArray();
            }
        }

        private static void serialise<T>(T item, Stream output)
        {
            var fields = item.GetType().GetFields(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance);

            var orderedFields = 
                from    field in fields
                let     attr = field.GetCustomAttribute<NetworkSerialisationAttribute>()
                where   attr != null
                orderby attr.Ordinal
                select  new { field, attr.Kind };

            foreach (var info in orderedFields)
            {
                if (info.Kind == SerialisationKind.Array)
                    serialiseArray(info.field.GetValue(item), output);
                else
                    serialiseScalar(info.field.GetValue(item), output);
            }
        }

        private static void serialiseArray(object value, Stream output)
        {
            var array = (byte[])value; // Only byte arrays are supported. This throws otherwise.

            ushort length = (ushort) array.Length;
            output.Write(BitConverter.GetBytes(length), 0, sizeof(ushort));
            output.Write(array, 0, array.Length);
        }

        private static void serialiseScalar(object value, Stream output)
        {
            if (value is byte) // Byte is a special case; there is no BitConverter.GetBytes(byte value)
            {
                output.WriteByte((byte)value);
                return;
            }

            // Hacky: Relies on the underlying type being a primitive type supported by one
            // of the BitConverter.GetBytes() overloads.

            var bytes = (byte[])
                typeof (BitConverter)
                .GetMethod("GetBytes", new [] {value.GetType()})
                .Invoke(null, new[] {value});

            output.Write(bytes, 0, bytes.Length);
        }
    }

    // In this class, note the use of the [NetworkSerialization] attribute to indicate
    // which fields should be serialised.

    public sealed class MyNetworkMessage
    {
        public MyNetworkMessage(byte[] data)
        {
            _data = data;
            _crc = 12345; // You should use Helper.CalcCrc(data);
        }

        public ushort Length
        {
            get
            {
                return (ushort)_data.Length;
            }
        }

        public ushort Crc
        {
            get
            {
                return _crc;
            }
        }

        public byte[] MessageData
        {
            get
            {
                return _data;
            }
        }

        [NetworkSerialisation(0, SerialisationKind.Array)]
        private readonly byte[] _data;

        [NetworkSerialisation(1)]
        private readonly ushort _crc;
    }

    // In this struct, note how the [NetworkSerialization] attribute is used to indicate the
    // order in which the fields should be serialised.

    public struct MyOtherNetworkMessage
    {
        [NetworkSerialisation(5)]  public int Int1;
        [NetworkSerialisation(6)]  public int Int2;

        [NetworkSerialisation(7)]  public long Long1;
        [NetworkSerialisation(8)]  public long Long2;

        [NetworkSerialisation(3)]  public byte Byte1;
        [NetworkSerialisation(4)]  public byte Byte2;

        [NetworkSerialisation(9)]  public double Double1;
        [NetworkSerialisation(10)] public double Double2;

        [NetworkSerialisation(1)]  public short Short1;
        [NetworkSerialisation(2)]  public short Short2;

        public float ThisFieldWillNotBeSerialised;

        public string AndNeitherWillThisOne;
    }

    class Program
    {
        private static void Main(string[] args)
        {
            var test1 = new MyNetworkMessage(new byte[10]);

            var bytes1 = NetworkSerialiser.Serialise(test1);

            Console.WriteLine(bytes1.Length + "\n");

            var test2 = new MyOtherNetworkMessage
            {
                Short1  = 1,
                Short2  = 2,
                Byte1   = 3,
                Byte2   = 4,
                Int1    = 5,
                Int2    = 6,
                Long1   = 7,
                Long2   = 8,
                Double1 = 9,
                Double2 = 10
            };

            var bytes2 = NetworkSerialiser.Serialise(test2);
            Console.WriteLine(bytes2.Length);

            foreach (byte b in bytes2)
            {
                Console.WriteLine(b);
            }
        }
    }
}

Collectives™ on Stack Overflow

Marshal struct with dynamic size array

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related