Director in Ruby hangs Ruby VM

[Soren Dreijer]

Hi folks!

I'm using directors in my Ruby SWIG project to allow C++ code to call back
in to Ruby. I have an interface defined like so:
%feature("director") IHostEventHandler;

In Ruby, I instantiate a class that derives from IHostEventHandler and then
pass that to my C++ code via a SWIGified function.

For testing purposes, when the IHostEventHandler is registered, I create a
C++ thread which calls one of the methods on the IHostEventHandler
interface every 2 seconds.

The call correctly comes in to Ruby. However, once the call returns, the
Ruby VM appears to be in a funky state and I'm no longer able to execute
other commands on the console. Interestingly, the C++ code is still able to
call in to Ruby via the interface; it's just like everything else in the
Ruby VM has stopped working.

Am I misunderstanding how directors work in Ruby? Is it perhaps wrong for
me to execute the callback on a non-Ruby thread?

Any help greatly appreciated,
Soren

[Klaus Kaempf]

Post by Soren Dreijer
The call correctly comes in to Ruby. However, once the call returns, the
Ruby VM appears to be in a funky state and I'm no longer able to execute
other commands on the console. Interestingly, the C++ code is still able to
call in to Ruby via the interface; it's just like everything else in the
Ruby VM has stopped working.

Which Ruby version are you using ?

Post by Soren Dreijer
Am I misunderstanding how directors work in Ruby? Is it perhaps wrong for
me to execute the callback on a non-Ruby thread?

Internal thread handling changed between Ruby 1.8 and 1.9. Ruby 2.0
changed the interface once more.

For Ruby 1.9 'rb_thread_blocking_region' is the magic function to call.
See e.g.
http://www.spacevatican.org/2012/7/5/whos-afraid-of-the-big-bad-lock/
for more information.

See https://bugs.ruby-lang.org/issues/8660 for Ruby 2.0

Hth,

Klaus

[Soren Dreijer]

I stumbled upon the following article:
http://burgestrand.se/articles/asynchronous-callbacks-in-ruby-c-extensions.html

I'm wondering if that isn't the issue here. I took a quick look through the
generated SWIG code and I don't see any Ruby threads being used for the
callback.

---
(It looks like I accidentally replied directly to Klaus rather than the
list earlier, so here's a quick catch up:)

I'm using Ruby 1.9.3p448.

I'm already using rb_thread_blocking_region() when making calls from Ruby
into native code for long operations. However,I'm not sure why
rb_thread_blocking_region() is relevant in this particular case. I'm
calling from native code *into* Ruby, not the other way around. That is, I
don't have a GIL to release at all.

I've created a sample here:
https://dl.dropboxusercontent.com/u/2406449/Swig/SwigRubyHang.7z

It includes a test.rb script, which creates an instance of the SWIG Hang
class and then sets a director via set_event_handler().

set_event_handler() creates a native thread (using C++11) that calls back
in to Ruby every 2 seconds. As soon as the first callback has been
executed, however, the Ruby VM hangs and takes no further user input.
Interestingly, SWIG is still able to continue executing callbacks; it's
just that all other Ruby execution has stopped (I can't even Ctrl+C the
process)

Post by Klaus Kaempf

Post by Soren Dreijer
The call correctly comes in to Ruby. However, once the call returns, the
Ruby VM appears to be in a funky state and I'm no longer able to execute
other commands on the console. Interestingly, the C++ code is still able

to

Post by Soren Dreijer
call in to Ruby via the interface; it's just like everything else in the
Ruby VM has stopped working.

Which Ruby version are you using ?

Post by Soren Dreijer
Am I misunderstanding how directors work in Ruby? Is it perhaps wrong for
me to execute the callback on a non-Ruby thread?

Internal thread handling changed between Ruby 1.8 and 1.9. Ruby 2.0
changed the interface once more.
For Ruby 1.9 'rb_thread_blocking_region' is the magic function to call.
See e.g.
http://www.spacevatican.org/2012/7/5/whos-afraid-of-the-big-bad-lock/
for more information.
See https://bugs.ruby-lang.org/issues/8660 for Ruby 2.0
Hth,
Klaus
--
SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix
Imendörffer, HRB 16746 (AG Nürnberg)
Maxfeldstraße 5, 90409 Nürnberg, Germany

[Soren Dreijer]

Alright, turns out it *was* because SWIG isn't calling back in to Ruby on a
native Ruby thread. Below, I've rewritten the original test code to instead
spawn a Ruby thread and then perform the SWIG callback on that thread. That
seems to fix the issue.

I've submitted a bug report: https://sourceforge.net/p/swig/bugs/1344/

Working code:

static VALUE DoSleep(void* pData)
{
sleep(2);
return 0;
}

static void CancelSleep(void* pData)
{
}

static VALUE SomeRubyThread(void* pData)
{
IEventHandler* pEventHandler = reinterpret_cast<IEventHandler*>(pData);

while (true)
{
rb_thread_blocking_region(&DoSleep, nullptr, &CancelSleep, nullptr);

pEventHandler->authenticate("moo", "moo");
}

return Qnil;
}

void Hang::set_event_handler(IEventHandler* pEventHandler)
{
rb_thread_create(&SomeRubyThread,
reinterpret_cast<void*>(pEventHandler));
}

Post by Soren Dreijer
http://burgestrand.se/articles/asynchronous-callbacks-in-ruby-c-extensions.html
I'm wondering if that isn't the issue here. I took a quick look through
the generated SWIG code and I don't see any Ruby threads being used for the
callback.
---
(It looks like I accidentally replied directly to Klaus rather than the
list earlier, so here's a quick catch up:)
I'm using Ruby 1.9.3p448.
I'm already using rb_thread_blocking_region() when making calls from Ruby
into native code for long operations. However,I'm not sure why
rb_thread_blocking_region() is relevant in this particular case. I'm
calling from native code *into* Ruby, not the other way around. That is, I
don't have a GIL to release at all.
https://dl.dropboxusercontent.com/u/2406449/Swig/SwigRubyHang.7z
It includes a test.rb script, which creates an instance of the SWIG Hang
class and then sets a director via set_event_handler().
set_event_handler() creates a native thread (using C++11) that calls back
in to Ruby every 2 seconds. As soon as the first callback has been
executed, however, the Ruby VM hangs and takes no further user input.
Interestingly, SWIG is still able to continue executing callbacks; it's
just that all other Ruby execution has stopped (I can't even Ctrl+C the
process)

Post by Soren Dreijer

Post by Soren Dreijer
The call correctly comes in to Ruby. However, once the call returns, the
Ruby VM appears to be in a funky state and I'm no longer able to execute
other commands on the console. Interestingly, the C++ code is still

able to

Post by Soren Dreijer
call in to Ruby via the interface; it's just like everything else in the
Ruby VM has stopped working.

Which Ruby version are you using ?

Post by Soren Dreijer
Am I misunderstanding how directors work in Ruby? Is it perhaps wrong

for

Post by Soren Dreijer
me to execute the callback on a non-Ruby thread?

Internal thread handling changed between Ruby 1.8 and 1.9. Ruby 2.0
changed the interface once more.
For Ruby 1.9 'rb_thread_blocking_region' is the magic function to call.
See e.g.
http://www.spacevatican.org/2012/7/5/whos-afraid-of-the-big-bad-lock/
for more information.
See https://bugs.ruby-lang.org/issues/8660 for Ruby 2.0
Hth,
Klaus
--
SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix
Imendörffer, HRB 16746 (AG Nürnberg)
Maxfeldstraße 5, 90409 Nürnberg, Germany