How to pass an array from C# to unmanaged COM by reference (VT_BYREF)

Question

I have a COM component written in C++ whose source I can't change, and one of the arguments to one of its methods is VARIANT *pParamArray. Using tlbimp I can create a managed stub for it and pass it an array from C#.

Unfortunately, the COM component is expecting its array to be passed by reference - there's an explicit check for pParamArray->vt != (VT_BYREF | VT_ARRAY | VT_VARIANT) and it returns an error if it doesn't pass that check.

I have the PDB and source for the COM component, so I'm debugging both the C# and the unmanaged code in tandem. I can see that my C# array of object[] is being passed as VT_ARRAY | VT_VARIANT, which is essentially a SAFEARRAY as far as I understand it.

How can I explicitly tell C# that I want to pass it by reference, so that the type on the far end has the VT_BYREF mask?

I've tried putting it in a VariantWrapper - I get an ArgumentException with the message "VariantWrappers cannot be stored in Variants."
I've tried doing a Marshal.AllocHGlobal and using Marshal.GetNativeVariantForObject() but I only get an int on the COM end.

tlbimp by default marshals the parameter in question as UnmanagedType.Struct. I'm not sure how to make tlbimp marshal it as IntPtr, or even if this would make a difference (I also tried using the enhanced tlbimp2 from CodePlex, but it doesn't seem to recognize my request for IntPtr in its config file).

I'm by no means an Interop expert so feel free to suggest something which may appear obvious to you.

Update 1

As requested by @ZdeslavVojkovic, here are relevant pieces of the IDL:

[
    uuid(01234567-89AB-CDEF-0123-3456789ABCDE),
    version(1.0),
    helpstring("XXX")
]
library LAbc
{
    [
        object,
        uuid(01234567-89AB-CDEF-0123-3456789ABCDE),
        dual,
        helpstring("XXX"),
        pointer_default(unique)
    ]
    interface IAbc : IDispatch
    {
            [id(1), helpstring("XXX")]
            HRESULT CallFunction([in] myEnum Function, [in, out] VARIANT* pParamArray, [out, retval] long* pVal);
    };

    [
        uuid(01234567-89AB-CDEF-0123-3456789ABCDE),
        helpstring("XXXs")
    ]
    coclass Abc
    {
        [default] interface IAbc;
    };
};

And here is the method signature itself and the internal check for the parameter's type:

STDMETHODIMP XAbc::CallFunction(myEnum Function, VARIANT *pParamArray, long *pVal)
{
    ...

    // we must get a pointer to an array of variants
    if(!pParamArray ||
        (pParamArray->vt != (VT_BYREF | VT_ARRAY | VT_VARIANT)) ||
        !(psa = *pParamArray->pparray))
        return E_INVALIDARG;

    ...
}

can you show the exact definition of the interface and signature of the method? — Zdeslav Vojkovic
– Zdeslav Vojkovic, Commented Mar 5, 2013 at 8:36
@ZdeslavVojkovic Added relevant pieces of the IDL, the signature, and how I'm calling it in my question above. — user533676
– user533676, Commented Mar 5, 2013 at 13:11
I believe that this is not possible without rewriting the marshaling info in the generated interop dll. If you would rewrite to declare the array as IntPtr instead of ref object, then it can be made to work — Zdeslav Vojkovic
– Zdeslav Vojkovic, Commented Mar 5, 2013 at 14:45
I ditched TLBIMP and wrote the interop code myself. Even when I declare the array as IntPtr and manually marshaled the array into an IntPtr before calling the method, it still comes out the far end as VT_ARRAY | VT_VARIANT, not VT_BYREF | VT_ARRAY | VT_VARIANT. Totally stumped here - next step is to try making a C++ wrapper and calling that from C# instead (as suggested by @jacob-seleznev below. — user533676
– user533676, Commented Mar 5, 2013 at 15:57

Zdeslav Vojkovic · Accepted Answer · 2013-03-06 08:18:11Z

2

Here's how to make it work without rewriting the IL.

Please note that for simplicity I have skipped the enum param so IDL definition of the method is like this:

[
    object,
    uuid(E2375DCC-8B5B-4BD3-9F6A-A9C1F8BD8300),
    dual,
    helpstring("IDummy Interface"),
    pointer_default(unique)
]
interface IDummy : IDispatch
{
    [id(1)] HRESULT Fn([in, out] VARIANT *pParamArray, [out, retval]long *pVal);
};

You can call it by late binding call like this:

INTEROPXLib.IDummy d = new INTEROPXLib.DummyClass();

object data = new object[3]; // method argument, i.e. pParamArray value

var t = typeof(INTEROPXLib.IDummy);
object[] args = new object[1]; // array which will contain all method arguments
args[0] = data; // data is the first argument, i.e. first element of args array

ParameterModifier[] pms = new ParameterModifier[1];
ParameterModifier pm = new ParameterModifier(1);
pm[0] = true; // pass the 1st argument by reference
pms[0] = pm;  // add pm to the array of modifiers 

// invoke Fn by name via IDispatch interface
var ret = t.InvokeMember("Fn", System.Reflection.BindingFlags.InvokeMethod, null, d, args, pms, null, null);
Console.Out.WriteLine("Result = " + ret);

For convenience, it would be better to wrap this into an extension method on the interface.

edited Mar 6, 2013 at 8:18

answered Mar 5, 2013 at 16:54

Zdeslav Vojkovic

14.7k37 silver badges45 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

user533676 Over a year ago

Excellent stuff - I had already made an intermediate proxy COM object but this appears to work really well and is cleaner. I wrote an extension method on the interface as you suggested - makes the API very clean.

Collectives™ on Stack Overflow

How to pass an array from C# to unmanaged COM by reference (VT_BYREF)

Update 1

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

Update 1

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related