Dealing with C++ smart pointers in C#

[Dave Dribin]

Hi fellow SWIG users,

I'm trying to wrap a C++ API that uses Boost shared pointers heavily
for use in C#. Let's say I have two C++ classes Foo and Bar:

class Foo { // ...
};

typedef boost::shared_ptr<Foo> FooPtr;

class Bar {
public:
FooPtr createFoo(...);

int useFoo(FooPtr foo);
};

Now, in my C# code, it would get used like:

Bar bar = new Bar();
FooPtr foo = createFoo();
bar.useFoo(foo);

This works, but I have a couple problems with it. First, FooPtr is
just an ugly name in C#. It would be nice to be able to deal with
classes named Foo and Bar in C#, as FooPtr is just a side effect of
using shared_ptr. I can do some fancy SWIG renaming, but that doesn't
help the second problem. And that problem is that while Foo objects
can be created from Bar, that's only a convenience. A user could just
as easily create a Foo object manually. In C++, it's not so bad:

FooPtr foo(new Foo());
bar->useFoo(foo);

But in C#, this requires:

FooPtr foo = new FooPtr(new Foo());
bar.useFoo(foo);

Even if I did some fancy renaming, it would be:

Foo foo = new Foo(new FooImpl());
bar.useFoo(foo);

That whole double-new thing really is ugly and cumbersome. And it also
introduces some memory management issues. C# will try and delete the
Foo object, even though the pointer is now owned by FooPtr, and I get a
double-free at some point.

I have complete control over the Foo and Bar APIs and I'm willing to
make changes to make this more SWIG-able. I still want to use
shared_ptr as it greatly simplifies memory management. Has anyone else
come across this issue and if so, what was their solutions?

The only solution I've come up with so far is to not expose shared_ptr
objects to end users by doing a PIMPL implementation, with the private
implementation pointer being a shared_ptr. This works beautifully, and
is easily SWIG-able. But it's somewhat of a pain to maintain because
the main and Impl class have common methods that need to be repeated.
In the interest of the Don't Repeat Yourself (DRY) concept, I'd like to
avoid that. But barring any other suggestions, the PIMPL route is the
way I'm going to go. It works for SWIG and yet keeps C++ memory
management sane and exception safe.

-Dave

_______________________________________________
Swig maillist - ***@cs.uchicago.edu
http://mailman.cs.uchicago.edu/mailman/listinfo/swig

[William S Fulton]

Post by Dave Dribin
Hi fellow SWIG users,
I'm trying to wrap a C++ API that uses Boost shared pointers heavily for
class Foo { // ...
};
typedef boost::shared_ptr<Foo> FooPtr;
class Bar {
FooPtr createFoo(...);
int useFoo(FooPtr foo);
};
Bar bar = new Bar();
FooPtr foo = createFoo();
bar.useFoo(foo);
This works, but I have a couple problems with it. First, FooPtr is just
an ugly name in C#. It would be nice to be able to deal with classes
named Foo and Bar in C#, as FooPtr is just a side effect of using
shared_ptr. I can do some fancy SWIG renaming, but that doesn't help
the second problem. And that problem is that while Foo objects can be
created from Bar, that's only a convenience. A user could just as
FooPtr foo(new Foo());
bar->useFoo(foo);
FooPtr foo = new FooPtr(new Foo());
bar.useFoo(foo);
Foo foo = new Foo(new FooImpl());
bar.useFoo(foo);
That whole double-new thing really is ugly and cumbersome. And it also
introduces some memory management issues. C# will try and delete the
Foo object, even though the pointer is now owned by FooPtr, and I get a
double-free at some point.
I have complete control over the Foo and Bar APIs and I'm willing to
make changes to make this more SWIG-able. I still want to use
shared_ptr as it greatly simplifies memory management. Has anyone else
come across this issue and if so, what was their solutions?
The only solution I've come up with so far is to not expose shared_ptr
objects to end users by doing a PIMPL implementation, with the private
implementation pointer being a shared_ptr. This works beautifully, and
is easily SWIG-able. But it's somewhat of a pain to maintain because
the main and Impl class have common methods that need to be repeated.
In the interest of the Don't Repeat Yourself (DRY) concept, I'd like to
avoid that. But barring any other suggestions, the PIMPL route is the
way I'm going to go. It works for SWIG and yet keeps C++ memory
management sane and exception safe.

I havn't yet found an ideal way of solving this problem either. auto_ptr
can be wrapped nicely though as the raw pointer can be accessed and the
auto_ptr told not to manage the memory by calling release() -- you pass
memory management over to the C# side. However, there is no release()
equivalent in shared_ptr. If you can find another smart pointer that is
better than auto_ptr, but has a release() type method, I'd use that. I
can't think of any though, but havn't really looked. One day, we'll have
to improve shared_ptr handling though...

William
_______________________________________________
Swig maillist - ***@cs.uchicago.edu
http://mailman.cs.uchicago.edu/mailman/listinfo/swig

[Dave Dribin]

Post by William S Fulton
I havn't yet found an ideal way of solving this problem either.
auto_ptr can be wrapped nicely though as the raw pointer can be
accessed and the auto_ptr told not to manage the memory by calling
release() -- you pass memory management over to the C# side. However,
there is no release() equivalent in shared_ptr. If you can find
another smart pointer that is better than auto_ptr, but has a
release() type method, I'd use that. I can't think of any though, but
havn't really looked. One day, we'll have to improve shared_ptr
handling though...

Thanks, William. I think I may just pass real pointers to the user,
and use auto_ptr internally to keep things exception safe. This puts a
burden on the user to clean up, however, but I think it'll be worth it
in this case. I started going down the PIMPL path, but that has some
issues that I'm not sure I can live with.

-Dave

_______________________________________________
Swig maillist - ***@cs.uchicago.edu
http://mailman.cs.uchicago.edu/mailman/listinfo/swig

[Dave Dribin]

Post by William S Fulton
I havn't yet found an ideal way of solving this problem either.
auto_ptr can be wrapped nicely though as the raw pointer can be
accessed and the auto_ptr told not to manage the memory by calling
release() -- you pass memory management over to the C# side. However,
there is no release() equivalent in shared_ptr. If you can find
another smart pointer that is better than auto_ptr, but has a
release() type method, I'd use that. I can't think of any though, but
havn't really looked. One day, we'll have to improve shared_ptr
handling though...

Okay, so say I return an auto_ptr to Foo instead of shared_ptr, like
such:

class Bar {
public:
std::auto_ptr<Foo> createFoo(...);
int useFoo(Foo * foo);
};

useFoo() must not take an auto_ptr in this case because it does not
assume ownership (it's not a sink). If I add this template
instantiation:

%template(FooAutoPtr) std::auto_ptr<Foo>;

then pipe the above right into SWIG, I get this error:

Test.cs(41) error CS0029: Cannot convert implicitly from `FooAutoPtr'
to `Foo'

However, I did try this little trick to give ownership to SWIG that
works:

%extend {
Foo * createFoo() {
std::auto_ptr<Foo> foo = self->createFoo();
return foo.release();
}
}

Is there anyway to automate this %extend with some sort of typemap?

BTW, the rationale for returning an auto_ptr instead of just a straight
pointer is to guard against C++ code that ignores the return value,
which would result in a memory leak [1].

-Dave

[1] http://www.gotw.ca/publications/using_auto_ptr_effectively.htm

_______________________________________________
Swig maillist - ***@cs.uchicago.edu
http://mailman.cs.uchicago.edu/mailman/listinfo/swig

[Dave Dribin]

Hi Dave
I've put together my auto_ptr typemaps into an example. Actually, I've
only ever used out typemaps, so they are tried and tested, however,
I've also put together the in typemaps too. They allow one to use the
same proxy class for methods that take an auto_ptr, plain pointer,
reference or passed by value.

Hi William,

I am only going to use out typemaps, as well. I've played around with
your code, and it looks like it'll work. My only concern is that it
was tied to the particular language we're swigging to. If I add Ruby,
Perl, or Python, I need a separate typemap for each language. Where as
if use an %extend, I don't have to worry about it. The problem with
%extend is one has to be written for each method. I may play around
with macros to see if there's a way to automate that.

BTW, is there any way to have the PROXYCLASS be the same as TYPE, by
default? I am not using auto_ptr's of templates, so TYPE and
PROXYCLASS end up being the same:

SWIG_AUTO_PTR_TYPEMAPS(Foo, Foo)

There's no such thing as default arguments for SWIG macros, is there?

Thanks for you help!

-Dave

_______________________________________________
Swig maillist - ***@cs.uchicago.edu
http://mailman.cs.uchicago.edu/mailman/listinfo/swig

[mark gossage]

Hello,
Here a few things to consider.

A few years back I worked on updating the python bindings for crystalspace (crystal.sf.net)
They also used smart pointers (reference counted version, not auto_ptr). The first version of the code wrappered the smart_ptr class.
So if we had:
ptr=CREATE_SCENE(...)
ptr was actually the smart pointer. This worked ok, but it was a bit ugly.
What we did in the end was instead of getting python to manage the smart pointer we just gave it the bare pointer instead.
So :
ptr=CREATE_SCENE(...)
has ptr as the real pointer.
To make this work we did a couple of clever things. We used a typemap, which converted the smart pointer back to the real pointer. It looked a little like this.
%typemap(out) SmartPointer<SomeObj> (SomeObj* temp) %{
temp=$1.disown(); // get the raw pointer out of the smart pointer
SWIG_NewPointerObj($1,$descriptor,$owner); // return the object
%}
Then we extended the SomeObj class to make it call decref() when the interpeter releases it
%extend SomeObj %{
~SomeObj() {self->decref();}
%}

It seems that you have not finalised your API that you are wrappering (snce you talk about changing it). The easiest would be just to return a raw pointer to the memory, and tell SWIG that it needs to be managed. It will set the correct flags & then C# will manage it for you.
something like:
%newobject Bar::createFoo();
class Bar {
public:
Foo* createFoo(...);
int useFoo(Foo * foo);
};
This should work fine, and look neat.

If you have to return the auto_ptr, you could look into using the typemap I described above, as a basis for your converting your code.

Hope this might help.
Mark Gossage



_______________________________________________
Swig maillist - ***@cs.uchicago.edu
http://mailman.cs.uchicago.edu/mailman/listinfo/swig

[William S Fulton]

Post by mark gossage
Hello,
Here a few things to consider.
A few years back I worked on updating the python bindings for crystalspace (crystal.sf.net)
They also used smart pointers (reference counted version, not auto_ptr). The first version of the code wrappered the smart_ptr class.
ptr=CREATE_SCENE(...)
ptr was actually the smart pointer. This worked ok, but it was a bit ugly.
What we did in the end was instead of getting python to manage the smart pointer we just gave it the bare pointer instead.
ptr=CREATE_SCENE(...)
has ptr as the real pointer.
To make this work we did a couple of clever things. We used a typemap, which converted the smart pointer back to the real pointer. It looked a little like this.
%typemap(out) SmartPointer<SomeObj> (SomeObj* temp) %{
temp=$1.disown(); // get the raw pointer out of the smart pointer
SWIG_NewPointerObj($1,$descriptor,$owner); // return the object
%}
Then we extended the SomeObj class to make it call decref() when the interpeter releases it
%extend SomeObj %{
~SomeObj() {self->decref();}
%}

Mark,

Which smart pointer were you using BTW? The boost::shared_ptr doesn't
have a disown()/release() type method. auto_ptr isn't a very good smart
pointer and I'd like to find a better one but one where you can still
take control of the underlying memory away from the smart pointer.

The decref() method as you've shown it indicates that every type (like
SomeObj) has a decref() method, ie every object derives from some sort
of smart pointer base class?

William
_______________________________________________
Swig maillist - ***@cs.uchicago.edu
http://mailman.cs.uchicago.edu/mailman/listinfo/swig