Use OpenCV C++ codes in a VC# project — solution of creating a managed CLR wrapper

Tags

CLR wrapper, opencv 2.4.5, visual c++ 2010, visual c-sharp 2010

After all learning and practicing about DLL, LIB, CLR, P/Invoke in previous blogs, finally I got back to my main mission — to use VC++ OpenCV codes (with barely no UIs) in a VC# project (rich UIs). No concern about converting IplImage or CvMat images of OpenCV to formats supported by C# in this blog.

There were a few errors before I could get this done but it went well anyway. Mostly, I followed the same steps as written in Use C++ codes in a C# project — solution of wrapping native C++ with a managed CLR wrapper. Hence, there will be 3 projects involved: one VC++ OpenCV project for main computing, one VC++ CLR project for wrapping (creating DLL), and one VC# project for testing DLL call.

::::::: STEP 1: Create a VC++ OpenCV project for main computing :::::::

1 ) Create a new project, named cppOpencv, by choosing File > New > Project > Visual C++ >Win32 Console Application > Application Settings > Application type: Console Application, Additional options: Empty project > click Finish button.

2 ) Add a header file of your computing class to the project. For example, cppOpencv.h containing:

#pragma once
#include <opencv2/opencv.hpp>

class MYcppGui {
public:
    MYcppGui();
    ~MYcppGui();

    int myCppLoadAndShowRGB(char* url, int maxWidth, int maxHeight, int nPicPerRow);

private:
    bool win0; // status of an OpenCV window
};

3 ) Add an OpenCV C++ source file of your computing class to the project. For example, cppOpencv.cpp containing:

#pragma once
#include "cppOpencv.h"

MYcppGui::MYcppGui()
{
    win0 = false;
}

MYcppGui::~MYcppGui()
{
    cvDestroyAllWindows();
}

int MYcppGui::myCppLoadAndShowRGB( char* url, int maxWidth, int maxHeight, int nPicPerRow )
{       
    // my OpenCV C++ codes uses only cxcore, highgui and imgproc 
}

Note: I wrote all OpenCV codes in conventional unmanaged C++ styles. 

4 ) Add another C++ source file to the project. This file is just for testing the created class and will not be used in the further steps. For example, main.cpp containing:

#include <stdio.h>
#include "cppOpencv.h"

int main() {
    MYcppGui *myGui = new MYcppGui();
    myGui->myCppLoadAndShowRGB("C:\\Users\\Public\\Pictures\\Sample Pictures\\Desert.jpg", 800, 700, 3);
    free(myGui);

    printf("End of the test program\n");
    system("pause");
    return 0;
}

5 ) Before building the project, make sure to set Project Properties following instructions in Install OpenCV 2.4.5 and use it with Microsoft Visual C++ 2010. Otherwise, a number of errors will be reported. I recommend creating Property Sheets as suggested in the instructions as it can be reused in STEP 2.

6 ) Build and run the project to ensure that no error exists and your class works as desired.

::::::: STEP 2: Create a VC++ CLR project (DLL) for wrapping :::::::

1 ) Create a new CLR project, named clrOpencv, by choosing File > New > Project > Visual C++ >CLR > Class Library. And you will find many files created, including clrOpencv.h and clrOpencv.cpp in Solution Explorer panel.

2 ) In clrOpencv.h, add the following codes: (the codes printed in blue, red and orange are those added or edited by me; the orange ones refer to conversion between managed and unmanaged format)

• Alternative 1: Create a wrapper straightforward. The problem for this alternative is the parameter char* url (required by cvLoadImage command used inside myLoadAndShowRGB) which is an unmanaged format and not supported in C#. In order to convert System.String used in C# to char* and pass it to this wrapper function, unsafe directive is required in C# codes (will be explained later in STEP 3.3).

#pragma once

#include "C:\Projects\cppOpencv\cppOpencv\cppOpencv.h"
#include "C:\Projects\cppOpencv\cppOpencv\cppOpencv.cpp"

using namespace System;

namespace MYopencv {
    public ref class MYGui
    {
    public:
       MYGui();
       int myLoadAndShowRGB(char* url, int maxWidth, int maxHeight, int nPicPerRow);

    private:
       MYcppGui *myGui;
    };
}

• Alternative 2: Do the conversion (from unmanaged char* to managed String^) inside the wrapper. This way, the wrapper acts as a complete bridge between unmanaged C++ codes (STEP 1) and managed C# codes (STEP 3). No need to add codes for interop (inter-operate) in those two projects.

#pragma once

#include "C:\Projects\cppOpencv\cppOpencv\cppOpencv.h"
#include "C:\Projects\cppOpencv\cppOpencv\cppOpencv.cpp"

using namespace System;
using namespace System::Runtime::InteropServices;

namespace MYopencv {
    public ref class MYConversion
    {
    public:
       MYConversion();
       char* myStringToChar ( String^ str );
    };

    public ref class MYGui
    {
    public:
       MYGui();
       int myLoadAndShowRGB(String^ url, int maxWidth, int maxHeight, int nPicPerRow);

    private:
       MYcppGui *myGui;
    };
}

In the above example, I created a separated class named MYConversion as I planed to reuse it many times. However, you can just put the conversion codes directly in the body of myLoadAndShowRGB (STEP 2.3).

3 ) Continue from the previous step, in clrOpencv.cpp, add the following codes: (the codes printed in blue and orange are those added or edited by me; the orange ones refer to conversion between unmanaged and managed format)

• Alternative 1: Create a wrapping function by simply calling the instance of MYcppGui *myGui.

// This is the main DLL file.
#include "stdafx.h"
#include "clrOpencv.h"

#include "C:\Projects\cppOpencv\cppOpencv\cppOpencv.h"
#include "C:\Projects\cppOpencv\cppOpencv\cppOpencv.cpp"

using namespace MYopencv;

MYGui::MYGui()
{
    myGui = new MYcppGui();
}

int MYGui::myLoadAndShowRGB( char *url, int maxWidth, int maxHeight, int nPicPerRow)
{
    return myGui->myCppLoadAndShowRGB( url, maxWidth, maxHeight, nPicPerRow);
}

• Alternative 2: Create a wrapping function which includes conversion between unmanaged and managed format.

// This is the main DLL file.
#include "stdafx.h"
#include "clrOpencv.h"

#include "C:\Projects\cppOpencv\cppOpencv\cppOpencv.h"
#include "C:\Projects\cppOpencv\cppOpencv\cppOpencv.cpp"

using namespace MYopencv;

// ++++++++++++ MYGui Class ++++++++++++

MYGui::MYGui()
{
    myGui = new MYcppGui();
}

int MYGui::myLoadAndShowRGB( String^ url, int maxWidth, int maxHeight, int nPicPerRow)
{
    MYConversion ^convert = gcnew MYConversion();
    char* str = convert->myStringToChar(url);

    int result = myGui->myCppLoadAndShowRGB( str, maxWidth, maxHeight, nPicPerRow);

    // Add the line below will cause runtime assertion error (sth about heap error) in C#
    //free(str);   

    return result; 
} 

// ++++++++++++ MYConversion Class ++++++++++++ 

MYConversion::MYConversion() {} 

char* MYConversion::myStringToChar ( String^ str ) 
{ 
   IntPtr ptr = Marshal::StringToHGlobalAnsi(str); 
   return static_cast<char*>(ptr.ToPointer()) ; 
}

4 ) Your codes are ready for building DLL now. But don’t forget to set Project Properties regarding OpenCV. Otherwise, there are errors saying fatal error C1083: Cannot open include file: ‘opencv2/opencv.hpp’: No such file or directory. The easiest way to set Project Properties is to use Add Existing Property Sheet and choose the property files (*.props) created in STEP 1.

5 ) Build either Debug or Release version, or both, of the project and you should get the DLL(s) ready to be used in other .NET environment.

Note: Although I properly added Property Sheets to my CLR project, I still got many link errors saying something like

error LNK2028: unresolved token (0A0004C6) "extern "C" void __cdecl cvDestroyWindow(char const *)" ....

error LNK2019 unresolved external symbol "extern "C" void __cdecl cvDestroyWindow ....

error LNK2001 unresolved external symbol "public: virtual void __thiscall cv::HOGDescriptor::setSVMDetector ....

I double checked the Property Sheets and they already included $(OPENCV_BUILD)\x86\vc10\lib as an additional library directory; there was not supposed to have link errors at this point. Finally, in addition to including Property Sheets, I had to manually add $(OPENCV_BUILD)\x86\vc10\lib to Project Properties > Common Properties > VC++ directories > Library Directories again and those link errors just disappeared.

::::::: STEP 3: Create a VC# project for testing DLL call :::::::

1 ) Create a new VC# project and add your preferred UIs. Or create a simple console application.

2 ) Solution Explorer panel > References > right click and choose Add Reference > in Browse tab, browse for the DLL created in STEP 2 > click OK button. Now the name clrOpencv should be listed under References section in Solution Explorer panel.

3 ) Add your namespace to the C# project (by using directive) and try calling your wrapper functions. In the example below, I used OpenFileDialog to get System.String of image’s url from a user and pass it to my wrapper function: (codes related to this article are written in either blue or orange; the orange ones refer to managed-to-unmanaged conversion and vice versa)

• Alternative 1: As the wrapper does not perform any conversion from System.String to char*, C# must do it.

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using MYopencv;

namespace WindowsFormsApplication1
{
   public partial class Form1 : Form
   {
      MYGui cvGui;

      public Form1()
      {
         InitializeComponent();
         cvGui = new MYGui();
      }

      private void Form1_Load(object sender, EventArgs e)
      { }

      private void button2_Click(object sender, EventArgs e)
      {
         OpenFileDialog fdlg = new OpenFileDialog();
         fdlg.Title = "Choose image";
         fdlg.InitialDirectory = @"C:\Users\Public\Pictures\Sample Pictures";
         fdlg.Filter = "Image files (*.*)|*.*";
         fdlg.FilterIndex = 2;
         fdlg.RestoreDirectory = true;

         if (fdlg.ShowDialog() == DialogResult.OK)
            textBox1.Text = fdlg.FileName;
      }

      private void button1_Click(object sender, EventArgs e)
      {
          string url = textBox1.Text.Trim();
          byte[] urlByte = Encoding.ASCII.GetBytes(url);
          unsafe
          {
             fixed (byte* p =urlByte)
             {
                sbyte* sp = (sbyte*)p;
                if (cvGui.myLoadAndShowRGB( sp, 750, 750, 3) < 0)
                    MessageBox.Show("Invalid image's directory.", "Error", MessageBoxButtons.OK, MessageBoxIcon.Error);
             }
          } 

      } // end of button1_click

   } // end of class Form1

} // end of namespace WindowsFormsApplication1

Note: To allow unsafe codes in C#, go to Solution Explorer panel > right click at your project name (e.g., WindowsFormsApplication1) > choose Properties > choose All Configurations in Configuration drop-down-list > under General section, check Allow unsafe code.

• Alternative 2: All conversions were done inside the wrapper so no need to do anything special in C# codes.

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using MYopencv;

namespace WindowsFormsApplication1
{
   public partial class Form1 : Form
   {
      MYGui cvGui;

      public Form1()
      {
         InitializeComponent();
         cvGui = new MYGui();
      }

      private void Form1_Load(object sender, EventArgs e)
      { }

      private void button2_Click(object sender, EventArgs e)
      {
         OpenFileDialog fdlg = new OpenFileDialog();
         fdlg.Title = "Choose image";
         fdlg.InitialDirectory = @"C:\Users\Public\Pictures\Sample Pictures";
         fdlg.Filter = "Image files (*.*)|*.*";
         fdlg.FilterIndex = 2;
         fdlg.RestoreDirectory = true;

         if (fdlg.ShowDialog() == DialogResult.OK)
             textBox1.Text = fdlg.FileName;
      }

      private void button1_Click(object sender, EventArgs e)
      {
         int result = cvGui.myLoadAndShowRGB(textBox1.Text.Trim(), 750, 750, 3);
         if ( result < 0)
             MessageBox.Show("Invalid image's directory.", "Error", MessageBoxButtons.OK, MessageBoxIcon.Error);

      } // end of button1_click

   } // end of class Form1

} // end of namespace WindowsFormsApplication1

4 ) Build and run the project to ensure that no errors exist and everything works as expected.

::::::: Conclusion :::::::

For now, I think that wrapping OpenCV C++ codes with CLR is not much different from wrapping non-OpenCV C++ codes. Personally, I prefer doing all conversions in the wrapper class; do the conversion just once (in the wrapper) and it will keep all my future C# projects involving that DLL nice and clean.