Client side stats SSL workload 4.3.2

[Alex Rousskov]

On 05/16/2013 03:22 AM, Dmitry Kurochkin wrote:
> Hi Michael.
> 
> Michael Hendrie <pmix at hendrie.id.au> writes:
> 
>> Hi List,
>>
>> I recently updated from Web-Polygraph 4.0.10 running on FreeBSD to
>> Web-Polygraph 4.3.2 running on Linux (RHEL 6.1 clone) and now have some
>> inconsistency being reported in the client side runtime stats.
>>
>> The issue only seems to occur when polygraph-client is configured to send
>> requests via a proxy.  In the client side stats, I'm seeing a significantly
>> higher request rate than what was configured, on the server side stats they
>> report as expected.  e.g. on a test configured for 250req/sec:
>>
>> Client Side Stats:
>> 030.41| i-top1 786670 421.83   1357   0.00   0 1671
>>            ssl 448598 421.83   1360   0.00   0 1671
>> 030.50| i-top1 788934 452.78   1379   0.00   0 1668
>>            ssl 449857 452.78   1379   0.00   0 1668
>> 030.58| i-top1 791229 458.98   1465   0.00   0 1660
>>            ssl 451193 458.98   1465   0.00   0 1660
>> 030.66| i-top1 793401 434.40   1461   0.00   0 1687
>>            ssl 452449 434.40   1463   0.00   0 1687
>> 030.75| i-top1 795476 415.02   1401   0.00   0 1723
>>            ssl 453636 415.02   1402   0.00   0 1721
>>
>> Server Side Stats:
>> 030.45| i-top1 426331 227.03   2518   0.00   0 2200
>>            ssl 426331 227.03   2518   0.00   0 1637
>> 030.54| i-top1 427519 237.59   2539   0.00   0 2201
>>            ssl 427519 237.59   2539   0.00   0 1638
>> 030.62| i-top1 428710 238.19   2467   0.00   0 2171
>>            ssl 428710 238.19   2468   0.00   0 1608
>> 030.70| i-top1 429962 250.40   2503   0.00   0 2300
>>            ssl 429962 250.40   2502   0.00   0 1737
>>
>>
>> I didn't see this inconsistency in 4.0.10 and if I have a direct
>> client->server test (no proxy) then stats are also reported correctly.
>>
>> The proxy itself (squid) is reporting around 190 client req/sec:
>> client_http.requests = 188.674032/sec
>>
>> Having just read the changelog (after typing the above), now wondering if
>> the fix in the non-public 4.6.0 would resolve this issue?
>>
>> *
>>
>> - Make retried transactions obey request inter-arrival
>>   distribution and, hence, Robot::req_rate setting.
>>
>> *
>>
>> Any thoughts/idea's?
>>
> 
> Since v4.2.0 CONNECT tunnel establishment is treated as a separate
> transaction:
> 
> 	- Treat an HTTP CONNECT response as the end of the transaction.
> 	  Collect and report stats dedicated to these new CONNECT-only
> 	  transactions. These CONNECT transactions are often rather
> 	  different from the in-tunnel transactions that follow.
> 	  Isolating them also prevents CONNECT overheads from polluting
> 	  the first in-tunnel transaction stats.
> 
> So an HTTPS GET transaction over proxy is counted as two transactions on
> the client side: CONNECT and GET.
> 
> Detailed stats are available in HTML report or can be extracted from
> binary log with the lx tool.
> 
> The behavior is the same on all newer Polygraph versions, so upgrading
> would not help.

.... but there are plans to make baseline reporting match configuration
better when dealing with what we call "compound" transactions. The
overall principle behind the planned changes is to have reported
baseline request rate (i-lines in the console output) match the
configured Bench::peak_req_rate (or equivalent), regardless of whether
the workload involves CONNECT requests and/or authentication.

We will also report CONNECT (and authentication) activity separately,
just like we report ssl activity separately today.


We hope the above changes will reduce surprises caused by compound
transactions involving multiple HTTP requests.


Cheers,

Alex.