Re: [reSIProcate] reSIProcate speed is much slower now
I added an option to testStack to be able to do INVITE/200/ACK/BYE/200
testing as an alternative. Just run with -i option to enable. Here are
my results:
[jason@halifax test]% bin.opt.Linux.i686/testStack -l cout -v WARNING
-r 10000 -w 100 -b 127.0.0.1 -s 0 -i
Performing 10000 runs.
10000 calls peformed in 11859 ms, a rate of 843.241 calls per second.]
[jason@halifax test]% bin.opt.Linux.i686/testStack -l cout -v WARNING
-r 10000 -w 100 -b 127.0.0.1 -s 0
Performing 10000 runs.
10000 registrations peformed in 3622 ms, a rate of 2760.91
transactions per second.]
On 7/28/05, Andy Agarwal <Andy@xxxxxxxxxxx> wrote:
>
> The explanation below doesn't completely explain the difference in
> numbers. If so, then you should get 1572 * 2 / 5 = 629 calls per second
> if you run testSpeed on you laptop (with IPHLPAPI turned on). I doubt
> you would get that high a figure since I am getting 395 calls per second
> on a faster machine.
>
> It would make sense that the remaining difference is due to the extra
> time taken to construct these different messages - INVITE, ACK, BYE. I'm
> guessing INVITE is the most time consuming among these three.
>
> Anyway, sorry for generating some confusion by using the deprecated
> testSpeed instead of testStack (although, I still feel there is value to
> benchmarking reSIP using testSpeed)
>
> andy
>
>
>
> -----Original Message-----
> From: resiprocate-devel-bounces@xxxxxxxxxxxxxxxxxxx
> [mailto:resiprocate-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Andy
> Agarwal
> Sent: Thursday, July 28, 2005 11:06 AM
> To: Scott Godin; jason@xxxxxx
> Cc: resiprocate-devel@xxxxxxxxxxxxxxxxxxx
> Subject: RE: [reSIProcate] reSIProcate speed is much slower now
>
>
> testStack is calculating the time taken to execute transactions
> (Transaction = send REGISTER, receive 200) testSpeed is calculating the
> time taken to execute calls (Call = send INVITE, receive 200, send ACK,
> send BYE, receive 200)
>
> Hence the big difference in numbers :)
>
>
>
> -----Original Message-----
> From: Scott Godin [mailto:slgodin@xxxxxxxxxxxx]
> Sent: Thursday, July 28, 2005 10:59 AM
> To: 'jason@xxxxxx'; Scott Godin
> Cc: Robert Sparks; Andy Agarwal; resiprocate-devel@xxxxxxxxxxxxxxxxxxx
> Subject: RE: [reSIProcate] reSIProcate speed is much slower now
>
> On my Dell Latitude D810 laptop (PIV 1.73GHz) WinXP and using the same
> settings as Jason, I can get 1572 transactions per second (using
> IPHLPAPI) and 2084 transactions per second (with IPHLPAPI disabled).
> These results are a least a little closer than the testSpeed results.
> :)
>
> Oddly enough my desktop machine PIV 2.6 Ghz Win2k3 only gets 1361
> transactions per second.
>
> Scott
>
> -----Original Message-----
> From: Fischl jason [mailto:jason.fischl@xxxxxxxxx]
> Sent: Thursday, July 28, 2005 12:56 PM
> To: Scott Godin
> Cc: Robert Sparks; Andy Agarwal; resiprocate-devel@xxxxxxxxxxxxxxxxxxx
> Subject: Re: [reSIProcate] reSIProcate speed is much slower now
>
> Here's my output from testStack on Linux box.
> [jason@halifax test]% bin.opt.Linux.i686/testStack -r 10000 -w 100 -l
> cout -v ERR -b 127.0.0.1 Performing 10000 runs.
> 10000 registrations peformed in 3841 ms, a rate of 2603.49 transactions
> per second.]
>
> Note that this was compiled with -O3 on linux and uses the google
> malloc. https://sourceforge.net/projects/goog-perftools/
>
> Thanks,
> Jason
>
>
>
> On 7/27/05, Scott Godin <slgodin@xxxxxxxxxxxx> wrote:
> > If you use 127.0.0.1:<port> (the defaults in testSpeed.cxx that Andy
> posted)
> > it doesn't do any DNS lookups. I can see how localhost would have
> > caused problems though.
> >
> > I tried removing the IPHLPAPI library for determineSourceInterface
> (ie.
> > define NO_IPHLPAPI), so that windows uses the same mechanism as *nix
> > for this. I got an extra 100 calls per second - but still no where
> > near the
> > 2000+ calls per seconds Jason is seeing on Linux. :(
> >
> > -----Original Message-----
> > From: Robert Sparks [mailto:rjsparks@xxxxxxxxxxx]
> > Sent: Wednesday, July 27, 2005 2:08 PM
> > To: Scott Godin
> > Cc: Andy Agarwal; resiprocate-devel@xxxxxxxxxxxxxxxxxxx; jason@xxxxxx
> > Subject: Re: [reSIProcate] reSIProcate speed is much slower now
> >
> > TestStack gives a much more realistic measure of the stack performance
>
> > _and_ it didn't make assumptions about DNS availability the way
> > testSpeed did (people were getting horrid numbers because "localhost"
> > wouldn't resolve for example...)
> >
> > RjS
> >
> > On Jul 27, 2005, at 12:43 PM, Scott Godin wrote:
> >
> > > I was using the testSpeed.cxx that Andy posted from testing. I
> > > can't remember why testSpeed was replaced by testStack - can you
> > > refresh my memory?
> > >
> > > Thanks,
> > >
> > > Scott
> > >
> > > -----Original Message-----
> > > From: Robert Sparks [mailto:rjsparks@xxxxxxxxxxx]
> > > Sent: Wednesday, July 27, 2005 1:09 PM
> > > To: jason@xxxxxx
> > > Cc: resiprocate-devel@xxxxxxxxxxxxxxxxxxx; Andy Agarwal
> > > Subject: Re: [reSIProcate] reSIProcate speed is much slower now
> > >
> > > One bit of a sanity check -
> > >
> > > The discussion mentions testSpeed - we replaced that with testStack
> > > quite awhile back.
> > > testSpeed is not in the repository anymore. Was that a typo remember
>
> > > things past?
> > >
> > > RjS
> > >
> > > On Jul 25, 2005, at 6:09 PM, Fischl jason wrote:
> > >
> > >> Thanks Scott for fixing this.
> > >>
> > >> I am curious why we see much lower performance on win32 than on
> Linux.
> > >> On a 3GHz P4, I see > 2k calls per second under linux.
> > >>
> > >> Hmmm.
> > >>
> > >> Jason
> > >>
> > >>
> > >> On 7/25/05, Andy Agarwal <Andy@xxxxxxxxxxx> wrote:
> > >>> Great. The problem seems to have been fixed. Now the testSpeed
> > >>> program is generating 395 calls per second (even more than the
> > >>> results obtained from reSIP 0.9).
> > >>>
> > >>> Thanks for taking care of this...
> > >>>
> > >>>
> > >>>
> > >>> -----Original Message-----
> > >>> From: Scott Godin [mailto:slgodin@xxxxxxxxxxxx]
> > >>> Sent: Monday, July 25, 2005 11:17 AM
> > >>> To: Andy Agarwal; jason@xxxxxx
> > >>> Cc: resiprocate-devel@xxxxxxxxxxxxxxxxxxx
> > >>> Subject: RE: [reSIProcate] reSIProcate speed is much slower now
> > >>>
> > >>> I did some investigation into this issue and found that a
> > >>> relatively recent addition to the WinCompat code is causing the
> > >>> slowness. If you build with USE_IPV6, then WinCompat uses the
> > >>> determineSourceInterfaceWithIPV6 function, as opposed to the
> > >>> determineSourceInterfaceWithoutIPV6 function. The
> > >>> WithIPV6 version can end up taking 100-200ms to return - thus
> > >>> slowing down the entire test.
> > >>>
> > >>> This function needs to be fixed - but in the meantime, I've
> > >>> modified the code so that even if the USE_IPV6 flag is turned on,
> > >>> if the IP Address is a
> > >>> V4 address then it will use the "WithoutIPV6" version instead.
> > >>> This means that the performance is restored for IPV4 addresses,
> > >>> and only
> > >>> IPV6
> > >>> addresses will be slow.
> > >>>
> > >>> Thanks,
> > >>>
> > >>> Scott
> > >>>
> > >>> -----Original Message-----
> > >>> From: Andy Agarwal [mailto:Andy@xxxxxxxxxxx]
> > >>> Sent: Thursday, July 21, 2005 12:08 PM
> > >>> To: jason@xxxxxx
> > >>> Cc: resiprocate-devel@xxxxxxxxxxxxxxxxxxx
> > >>> Subject: RE: [reSIProcate] reSIProcate speed is much slower now
> > >>>
> > >>>
> > >>> I set the logging to "Error". Nothing was printed by the program
> > >>> while the test ran.
> > >>>
> > >>> Log::initialize(Log::Cout, Log::Err, argv[0]);
> > >>>
> > >>> I also built the stack and test program in Win32-Release mode. Did
>
> > >>> not change the reSIP settings. I see that it is set to - Full
> > >>> Optimization
> > >>> (Ox) and Favor Fast Code (/Oy)
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> -----Original Message-----
> > >>> From: Fischl jason [mailto:jason.fischl@xxxxxxxxx]
> > >>> Sent: Thursday, July 21, 2005 8:57 AM
> > >>> To: Andy Agarwal
> > >>> Cc: jiangjinke@xxxxxxx; resiprocate-devel@xxxxxxxxxxxxxxxxxxx
> > >>> Subject: Re: [reSIProcate] reSIProcate speed is much slower now
> > >>>
> > >>> Did you build with optimization turned on? Also, is any logging
> > >>> enabled?
> > >>> What type of logging (i.e. file-based)?
> > >>>
> > >>> Jason
> > >>>
> > >>>
> > >>> On 7/21/05, Andy Agarwal <Andy@xxxxxxxxxxx> wrote:
> > >>>> Yes, I am running the stack on Win32. Should have mentioned that,
> > >>> sorry.
> > >>>> The machine I'm running it on is a 3GHz P4 with 1GB RAM running
> > >>>> Windows
> > >>>> 2003 server
> > >>>>
> > >>>>
> > >>>>
> > >>>> -----Original Message-----
> > >>>> From: resiprocate-devel-bounces@xxxxxxxxxxxxxxxxxxx
> > >>>> [mailto:resiprocate-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf
> > >>>> Of jiangjinke@xxxxxxx
> > >>>> Sent: Wednesday, July 20, 2005 10:07 PM
> > >>>> To: resiprocate-devel@xxxxxxxxxxxxxxxxxxx
> > >>>> Subject: Re: [reSIProcate] reSIProcate speed is much slower now
> > >>>>
> > >>>> Hi Andy,
> > >>>>
> > >>>> I'm using the rev. 5129 now, and the average (about 10 times)
> > >>>> call rate is 340 cps.
> > >>>> The platform I'm using:
> > >>>> Redhat EL3,
> > >>>> P4 CPU 2.40GHz,
> > >>>> 1G Memory
> > >>>>
> > >>>> I've try the program on another win32 machine:
> > >>>> 500 calls peformed in 13859 ms, a rate of 36.0776 calls per
> > >>>> second.]
> > >>>> CPU: P4 2.4G
> > >>>> MEM: 512M
> > >>>> The performance drops a lot under win32 It seems the problem only
>
> > >>>> exists in the win32 related code.
> > >>>>
> > >>>>
> > >>>> Regards
> > >>>> Jinke Jiang
> > >>>>
> > >>>> ----- Original Message -----
> > >>>> From: "Andy Agarwal" <Andy@xxxxxxxxxxx>
> > >>>> To: <resiprocate-devel@xxxxxxxxxxxxxxxxxxx>
> > >>>> Sent: Wednesday, July 20, 2005 8:23 AM
> > >>>> Subject: [reSIProcate] reSIProcate speed is much slower now
> > >>>>
> > >>>>
> > >>>> I was using reSIP 0.9.5019 until yesterday. I switched to
> > >>>> revision
> > >>>> 5096 from the main branch because of a bug in 0.9 where the
> > >>>> duplicate transaction ids were being created in my multi-threaded
>
> > >>>> app (see -
> > >>>>
> http://list.sipfoundry.org/archive/resiprocate-devel/msg02835.html).
> > >>>>
> > >>>> I was trying to see if there is a performance difference between
> > >>>> 0.9 and rev. 5096 from the main branch and found a significant
> change.
> > >>>> In 0.9 the testSpeed program generates 359 calls per second.
> > >>>> In rev. 5096 it generates 72 calls per second.
> > >>>>
> > >>>> Can anyone explain the reason behind this huge drop ? I have
> > >>>> attached the testSpeed program for those interested.
> > >>>>
> > >>>> thanks,
> > >>>> andy
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> -----------------------------------------------------------------
> > >>>> ---
> > >>>> -
> > >>>> -
> > >>>> --
> > >>>> --------
> > >>>>
> > >>>>
> > >>>>> _______________________________________________
> > >>>>> resiprocate-devel mailing list
> > >>>>> resiprocate-devel@xxxxxxxxxxxxxxxxxxx
> > >>>>> https://list.sipfoundry.org/mailman/listinfo/resiprocate-devel
> > >>>>
> > >>>>
> > >>>>
> > >>>> _______________________________________________
> > >>>> resiprocate-devel mailing list
> > >>>> resiprocate-devel@xxxxxxxxxxxxxxxxxxx
> > >>>> https://list.sipfoundry.org/mailman/listinfo/resiprocate-devel
> > >>>> _______________________________________________
> > >>>> resiprocate-devel mailing list
> > >>>> resiprocate-devel@xxxxxxxxxxxxxxxxxxx
> > >>>> https://list.sipfoundry.org/mailman/listinfo/resiprocate-devel
> > >>>>
> > >>>
> > >> _______________________________________________
> > >> resiprocate-devel mailing list
> > >> resiprocate-devel@xxxxxxxxxxxxxxxxxxx
> > >> https://list.sipfoundry.org/mailman/listinfo/resiprocate-devel
> > >
> > > _______________________________________________
> > > resiprocate-devel mailing list
> > > resiprocate-devel@xxxxxxxxxxxxxxxxxxx
> > > https://list.sipfoundry.org/mailman/listinfo/resiprocate-devel
> >
> _______________________________________________
> resiprocate-devel mailing list
> resiprocate-devel@xxxxxxxxxxxxxxxxxxx
> https://list.sipfoundry.org/mailman/listinfo/resiprocate-devel
>