Re: [reSIProcate] IntrruptableThread dead-lock

[Kennard White <kennard_white@xxxxxxxxxxxx>]

Hi Scott,

Here is patch for select interruptor. I think it is a good safety patch in general, even if this specific issue may have a better fix.

The specific issue here is in DnsResult::lookup() where the async handler gets invoked. I doubt this is invoked from non-SipStack thread, but I guess it could be.

Kennard

On Sun, Nov 21, 2010 at 8:05 AM, Scott Godin <sgodin@xxxxxxxxxxxxxxx> wrote:

The non-blocking solution sounds reasonable to me.

>Is there a better (or complementary) way to address this? In particular, does the asyncHandler need to be invoked within DnsStub? When is DnsStub invoked outside the SipStack thread?

The following DnsStub fns are typically invoked by applications outside of the stack thread:

      void clearDnsCache();

      void logDnsCache();

      void setEnumSuffixes(const std::vector<Data>& suffixes);

Scott

On Sun, Nov 21, 2010 at 1:19 AM, Kennard White <kennard_white@xxxxxxxxxxxx> wrote:

Hi,

There is an interesting dead-lock case with InterruptableStackThread under linux. The interrupter can be invoked from within the SipStack thread itself (one particular case is DnsResult -- see stack trace below). Under heavy load the pipe can "fill-up" and the write blocks, causing dead-lock (e.g., the same thread that drains the pipe is blocked writing to the pipe).

For now, I've made the writes to the pipe non-blocking -- looks like equiv change has already been made on the Windows side. That seems to solve the problem.

Is there a better (or complementary) way to address this? In particular, does the asyncHandler need to be invoked within DnsStub? When is DnsStub invoked outside the SipStack thread?

One caveat: this occurs with epoll, and it is possible that it is some side effect of how I changed the call structure. But I more likely I think it is just the increase amount of activity allowed by using epoll enables the pipe to fill up and expose this bug.

Thanks,
Kennard

(gdb) where
#0  0x00007ffff798204b in write () from /lib/libpthread.so.0
#1  0x00000000004838a6 in resip::SelectInterruptor::interrupt (this=<value optimized out>) at SelectInterruptor.cxx:91
#2  resip::SelectInterruptor::handleProcessNotification (this=<value optimized out>) at SelectInterruptor.cxx:51
#3  0x00000000004efb82 in resip::DnsResult::lookupInternal (this=0x7fffe8837740, uri=<value optimized out>) at DnsResult.cxx:345
#4  0x00000000004f0042 in resip::DnsResult::lookup (this=0x7fffe8837740, uri=..., enumSuffixes=...) at DnsResult.cxx:224
#5  0x00000000004b96d9 in resip::TransportSelector::dnsResolve (this=0x7ffff7fc0d30, result=0x7fffe8837740, msg=0x7fffec7f9ce0)
    at TransportSelector.cxx:375
#6  0x00000000004af5e5 in resip::TransactionState::sendToWire (this=0x7fffe879d220, msg=0x7fffec7f9ce0, resend=false)
    at TransactionState.cxx:2073
#7  0x00000000004b2818 in resip::TransactionState::processClientNonInvite (this=0x7fffe879d220, msg=<value optimized out>)
    at TransactionState.cxx:710
#8  0x00000000004b3e9b in resip::TransactionState::processSipMessageAsNew (sip=0x7fffec7f9ce0, controller=..., tid=...)
    at TransactionState.cxx:298
#9  0x00000000004b49c6 in resip::TransactionState::process (controller=...) at TransactionState.cxx:579
#10 0x00000000004a4140 in resip::TransactionController::processEverything (this=0x7ffff7fc0c48, fdset=<value optimized out>)
    at TransactionController.cxx:107
#11 0x0000000000497cfd in resip::SipStack::process (this=0x7ffff7f9d010, fdset=...) at SipStack.cxx:589
#12 0x000000000049c194 in resip::InterruptableStackThread::thread (this=0x842de0) at InterruptableStackThread.cxx:41
#13 0x000000000053249f in threadIfThreadWrapper (threadParm=0x6) at ThreadIf.cxx:51
#14 0x00007ffff797aa04 in start_thread () from /lib/libpthread.so.0
#15 0x00007ffff6b26d4d in clone () from /lib/libc.so.6
#16 0x0000000000000000 in ?? ()

_______________________________________________
resiprocate-devel mailing list
resiprocate-devel@xxxxxxxxxxxxxxx
https://list.resiprocate.org/mailman/listinfo/resiprocate-devel

Attachment: resip-selintr-nonblock.patch
Description: Binary data