[reSIProcate] IntrruptableThread dead-lock
Hi,
There is an interesting dead-lock case with InterruptableStackThread under linux. The interrupter can be invoked from within the SipStack thread itself (one particular case is DnsResult -- see stack trace below). Under heavy load the pipe can "fill-up" and the write blocks, causing dead-lock (e.g., the same thread that drains the pipe is blocked writing to the pipe).
For now, I've made the writes to the pipe non-blocking -- looks like equiv change has already been made on the Windows side. That seems to solve the problem.
Is there a better (or complementary) way to address this? In particular, does the asyncHandler need to be invoked within DnsStub? When is DnsStub invoked outside the SipStack thread?
One caveat: this occurs with epoll, and it is possible that it is some side effect of how I changed the call structure. But I more likely I think it is just the increase amount of activity allowed by using epoll enables the pipe to fill up and expose this bug.
Thanks,
Kennard
(gdb) where
#0 0x00007ffff798204b in write () from /lib/libpthread.so.0
#1 0x00000000004838a6 in resip::SelectInterruptor::interrupt (this=<value optimized out>) at SelectInterruptor.cxx:91
#2 resip::SelectInterruptor::handleProcessNotification (this=<value optimized out>) at SelectInterruptor.cxx:51
#3 0x00000000004efb82 in resip::DnsResult::lookupInternal (this=0x7fffe8837740, uri=<value optimized out>) at DnsResult.cxx:345
#4 0x00000000004f0042 in resip::DnsResult::lookup (this=0x7fffe8837740, uri=..., enumSuffixes=...) at DnsResult.cxx:224
#5 0x00000000004b96d9 in resip::TransportSelector::dnsResolve (this=0x7ffff7fc0d30, result=0x7fffe8837740, msg=0x7fffec7f9ce0)
at TransportSelector.cxx:375
#6 0x00000000004af5e5 in resip::TransactionState::sendToWire (this=0x7fffe879d220, msg=0x7fffec7f9ce0, resend=false)
at TransactionState.cxx:2073
#7 0x00000000004b2818 in resip::TransactionState::processClientNonInvite (this=0x7fffe879d220, msg=<value optimized out>)
at TransactionState.cxx:710
#8 0x00000000004b3e9b in resip::TransactionState::processSipMessageAsNew (sip=0x7fffec7f9ce0, controller=..., tid=...)
at TransactionState.cxx:298
#9 0x00000000004b49c6 in resip::TransactionState::process (controller=...) at TransactionState.cxx:579
#10 0x00000000004a4140 in resip::TransactionController::processEverything (this=0x7ffff7fc0c48, fdset=<value optimized out>)
at TransactionController.cxx:107
#11 0x0000000000497cfd in resip::SipStack::process (this=0x7ffff7f9d010, fdset=...) at SipStack.cxx:589
#12 0x000000000049c194 in resip::InterruptableStackThread::thread (this=0x842de0) at InterruptableStackThread.cxx:41
#13 0x000000000053249f in threadIfThreadWrapper (threadParm=0x6) at ThreadIf.cxx:51
#14 0x00007ffff797aa04 in start_thread () from /lib/libpthread.so.0
#15 0x00007ffff6b26d4d in clone () from /lib/libc.so.6
#16 0x0000000000000000 in ?? ()