Re: [reSIProcate] Issue with getRandomHex()

[Yannick Guay <yannick.guay@xxxxxxxxx>]

Hi,

I spent a number of hours trying to come up with a code snippet that reproduce the issue without success. I could not put my finger on the exact cause of this issue nor could I easily demonstrate there truly is an issue with either our code or the stack.

I've put this task aside for now as it's not gating our work but we will resume work on it as time permits. We really need to get to the bottom of this before we release our product. We will keep you posted.

regards,

-Yannick Guay

2014-05-12 15:11 GMT-04:00 Yannick Guay <yannick.guay@xxxxxxxxx>:

While reading your answer, I realized that I may have mis-described the use-case. By multiple processes I meant a separate app (not resiprocate based) sending multiple invites simultaneously to a resiprocated based B2BUA which SIP stuff gets all handled from a single thread.

Sorry for the confusion.

2014-05-12 15:00 GMT-04:00 Scott Godin <sgodin@xxxxxxxxxxxxxxx>:

Oh multiple processes - interesting - we use the pid to seed the random number generator that you would think would avoid that issue.

Scott

On Mon, May 12, 2014 at 2:28 PM, Yannick Guay <yannick.guay@xxxxxxxxx> wrote:

Hi Scott,

it seems all 18 rutil/test/test* tests get executed successfully. This is no surprised to me as I had previously put similar tests code in my project in order to check for this.

The way the API gets hammered in my project is really different than what the unit tests is doing in which a call generator triggers multiple processes which simultaneously generate an INVITE to a single B2BUA. This, as a consequence, makes Random::getRandomHex(8) method get hit multiple times. This issue can be reproduced 100% of the time.

I will come up with a snippet of code that clearly demonstrates the issue and post it here. In the mean time feel free to comment on the issue.

best regards,

-Yannick Guay

2014-05-10 10:41 GMT-04:00 Scott Godin <sgodin@xxxxxxxxxxxxxxx>:

Hi Yannick,

Thanks for keeping us posted about your findings.  I'm still baffled why others aren't seeing this, when your test platforms are pretty standard.  Out of curiosity - does rutil/test/testRandomHex show that there is a uniqueness issue?

Daniel Pocock - Have you seen anything like this on your *nix platforms?

Scott

On Fri, May 9, 2014 at 2:17 PM, Yannick Guay <yannick.guay@xxxxxxxxx> wrote:

Hi Scott,

I made more testing recently, trying to isolating the problem a little more. The first thing I tried was to execute the same set of tests on a different platform (Debian 6.0.6 64bit) and I'm seeing the exact same problem.

I will conduct more testing next week and, as well, I will try the patch I suggested last week on Debian. In the interim please let me know if you have any comments.

FWIW: our system will be deployed on ESXI so all tests are conducted on VMWare, not bare metal.

best regards,

-Yannick Guay

2014-05-01 10:45 GMT-04:00 Yannick Guay <yannick.guay@xxxxxxxxx>:

Hi again,

A colleague here pointed me at another API which I tried (srand48 and lrand48, also part of standard library) and that seems to work better for me. The replacements exist on all modern Unix platforms (Solaris, Mac OS X, FreeBSD, Linux).

Although Evan Jones describes this as not being thread safe (read this post here), this could be a good starting point to work around the issue. I have only tested my scenario where the DUM is generating new transaction id and I've had success with it. I believe that's because all request for random numbers is coming from a single thread.

I'm aware that not everybody is suffering of this but I'm confident it's solving a real issue so I'm posting the patch for review. Perhaps we should confirm whether it is safe to use by running a wider range of tests?

thanks,

-Yannick

2014-05-01 8:48 GMT-04:00 Yannick Guay <yannick.guay@xxxxxxxxx>:

Hi Scott,

We've started using 1.9 as of  last week on Centos 6.3. I've tested the Random class separately and made it issue 20 8 byte long values without a problem, I can hardly identify what could cause this? I will keep investigating on my side and would be very thankful if anybody can give any sort of recommendation on how to get that fixed.

thanks,

-Yannick 

2014-04-30 17:02 GMT-04:00 Scott Godin <sgodin@xxxxxxxxxxxxxxx>:

Hi Yannick,

What resip release are you using?  There was quite a bit of work put into the Random.cxx class a couple years ago to ensure random number generator is seeded automatically for each thread (see Random::initialize()).  If you are on an older release you may want to compare your version Random.cxx with the latest one from SVN.  Note also - there is a lot of platform depend code branches in this init logic, so your platform may be playing a role here.  What platform are you using?

Scott

On Wed, Apr 30, 2014 at 3:50 PM, Yannick Guay <yannick.guay@xxxxxxxxx> wrote:

Hi,

I ran into an issue with DUM generating same transaction id several times when my b2bua is handling multiple simultaneous calls.

I'm using DialogUsageManager::makeInviteSession() which generates a proper branch parameter for the outgoing call but when it's time to send them down to the stack the branch parameter is set again by DialogUsageManager::send() (line 1036 or so), but same value gets reused for all calls. As a result, only the first call gets out of my gateway, all other subsequent calls are considered bad and eventually get dropped by TransactionState.

Here is the line that I'm finding in log files.

2014-04-30T15:12:27.854-04:00 Warning - B2BUA TU sent us a duplicate INVITE: fix this! 

My knowledge of srandom and random is rather limited but I trust after giving it a seed, then it should always hand off unique values?

Any Ideas what I'm doing wrong?

Best Regards,

-Yannick Guay

_______________________________________________
resiprocate-devel mailing list
resiprocate-devel@xxxxxxxxxxxxxxx
https://list.resiprocate.org/mailman/listinfo/resiprocate-devel