<div dir="ltr">Thanks for all the help! You can really see a lot of hard work and love has gone into Web-Polygraph.<div><br></div><div>I've written a rails front end to handle the launching, gather all the files and host the finished report so your instructions fit perfectly.</div>
</div><div class="gmail_extra"><br clear="all"><div><div dir="ltr"><span style="background-color:rgb(255,255,255);border-collapse:separate;color:rgb(34,34,34);font-size:medium;font-family:Helvetica"><div style="word-wrap:break-word">
<span style="border-collapse:separate"><div style="word-wrap:break-word"><span style="border-collapse:separate"><div style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span lang="EN-US" style="font-size:8pt;font-family:Verdana,sans-serif;color:rgb(54,95,145)"><b>Alberto Klocker</b> | Software Developer</span><span style="font-size:8pt;font-family:Verdana,sans-serif;color:rgb(54,95,145)"> | </span><span lang="EN-US"><a href="http://www.bluereef.com.au/" style="color:blue" target="_blank"><span lang="EN-AU" style="font-size:8pt;font-family:Verdana,sans-serif;color:rgb(54,95,145);text-decoration:none">www.bluereef.com.au</span></a></span><span lang="EN-US" style="font-size:8pt;color:rgb(54,95,145)"> </span><span style="font-size:8pt;color:rgb(54,95,145)"></span></div>
<div style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><u><span style="font-size:8pt;color:rgb(54,95,145)">_________________________________________________________________</span></u></div></span></div>
</span></div></span><span style="background-color:rgb(255,255,255);color:rgb(34,34,34);font-size:15px;font-family:Calibri,sans-serif"><span lang="EN-US" style="font-size:8pt;font-family:Verdana,sans-serif;color:rgb(54,95,145)"><b>D: </b><a value="+61398958020" style="color:rgb(17,85,204)">+61 3 9895 8006</a> | </span></span><span style="background-color:rgb(255,255,255);border-collapse:separate;color:rgb(34,34,34);font-size:medium;font-family:Helvetica"><span style="font-size:15px;font-family:Calibri,sans-serif"><b><span lang="EN-US" style="font-size:8pt;font-family:Verdana,sans-serif;color:rgb(54,95,145)">T</span></b></span><span style="font-size:15px;font-family:Calibri,sans-serif"><span style="font-size:8pt;font-family:Verdana,sans-serif;color:rgb(54,95,145)">: <a value="+61398988000" style="color:rgb(17,85,204)">+61 3 9898 8000</a> </span></span><span style="font-size:15px;font-family:Calibri,sans-serif"><span style="font-size:8pt;font-family:Verdana,sans-serif;color:rgb(54,95,145)">| </span></span><a value="+61398988000" style="font-family:Verdana,sans-serif;font-size:11px;color:rgb(17,85,204)">@AlbertoKlocker</a><div style="word-wrap:break-word">
<span style="border-collapse:separate"><div style="word-wrap:break-word"><span style="border-collapse:separate"><span style="border-collapse:separate"><span style="font-family:'Lucida Grande'"><div style="margin:0cm 0cm 0.0001pt;font-size:12pt;font-family:'Times New Roman',serif">
<span style="border-collapse:separate;font-size:medium;font-family:Calibri"><span style="font-family:'Times New Roman',serif;font-size:16px"><span style="border-collapse:separate;font-size:medium;font-family:Calibri"><div style="margin:0cm 0cm 0.0001pt;font-size:12pt;font-family:'Times New Roman',serif">
<span lang="EN-US" style="font-size:8pt;font-family:Verdana,sans-serif;color:rgb(30,147,102)">Please consider the environment before printing this email</span></div></span></span></span></div></span></span></span></div></span></div>
</span></div></div>
<br><br><div class="gmail_quote">On Tue, Sep 10, 2013 at 2:53 AM, Pavel Kazlenka <span dir="ltr"><<a href="mailto:pavel.kazlenka@measurement-factory.com" target="_blank">pavel.kazlenka@measurement-factory.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><div class="im">
<div>Hi Alberto,<br>
<br>
On 09/09/2013 04:40 AM, Alberto Klocker wrote:<br>
</div>
</div><div class="im"><blockquote type="cite">
<div dir="ltr">
<pre style>Hi,</pre>
<pre style>We run tests on a 10 Gb network environment with Polygraph as one of the benchmarking tools but have found we are hitting 100% CPU usage on both the poly-client and poly-server which would indicate we need to spread the processes and systems out.</pre>
<pre style>How well does the reporting engine handle log files from multiple sources? </pre>
<pre style><span style="font-family:arial">Say I were to script a system that launches two or three instances of the client and server, will the reporting engine be capable of combining the results into one report?</span>
</pre>
</div>
</blockquote></div><div class="im">
All the polygraph reporting tools (like polygraph-lx,
polygraph-ltrace, polygraph-reporter) support multiple logs
aggregation. E.g. if you have logs from three client workers named
clt.1.log, clt.2.log, clt.3.log you can extract statistics from them
simply using <br>
$ polygraph-lx clt.1.log clt.2.log clt.3.log. Same for others.
However, there is a minor bug that prevents from ltrace tool
extracting logs from both server and client sides in one run: <a href="https://bugs.launchpad.net/polygraph/+bug/1204983" target="_blank">https://bugs.launchpad.net/polygraph/+bug/1204983</a>
</div><blockquote type="cite">
<div dir="ltr">
<pre style><span style="font-family:arial">Would it be better I write something that "merges" the results together?</span></pre>
</div>
</blockquote>
As a result of written above, you don't have to write anything.
<blockquote type="cite">
<div dir="ltr">
<pre style>I'm curious to see what others have done to ramp Polygraph up above its single core limits.</pre><div class="im">
<pre style>We can get 10 Gb throughput with large files but my goal is to find maximum, stable concurrent connections.</pre>
<pre style>Any help is much appreciated!</pre>
</div></div>
</blockquote>
Sure Alberto. There is a number of actions that could help you
increase polygraph performance:<div class="im"><br>
<br>
1) Run multiple instances of workers and bind them to different cpu
cores. E.g if you have 2 cpu cores on machine that runs
polygraph-client, run two clients like:<br>
taskset -c 0 polygraph-client --log clt.1.log ...<br>
taskset -c 1 polygraph-client --log clt.2.log ...<br>
<br>
2) Bind tune cpu affinity for NIC interrupts to cores different from
the ones that already occupied by polygraph workers (clients or
servers). E.g. if you have 4 cpu cores, consider to run three
polygraph-client instances bound to cores 0-2 and bind NIC
interrupts to core 3. This article should help you with SMP affinity
in linux: <a href="http://www.alexonlinux.com/smp-affinity-and-proper-interrupt-handling-in-linux" target="_blank">http://www.alexonlinux.com/smp-affinity-and-proper-interrupt-handling-in-linux</a>
. There is no good formula to say what should be ratio between
workers' and NIC's cores, but top output could give you the clues.<br>
<br>
3) If you have 'virtual' (hyper-threading provided) cores, it's not
a good idea to bind polygraph workers to both 'real' and 'virtual'
instances of the same core.<br>
<br>
4) If you have enough RAM, consider writing test results on
in-memory FS like tmpfs (<a href="https://www.kernel.org/doc/Documentation/filesystems/tmpfs.txt" target="_blank">https://www.kernel.org/doc/Documentation/filesystems/tmpfs.txt</a>).<br>
<br></div>
5) Tuning linux TCP stack for 10G networking is important part too.
I'd use this document as a guide:
<a href="http://landley.net/kdocs/ols/2009/ols2009-pages-169-184.pdf" target="_blank">http://landley.net/kdocs/ols/2009/ols2009-pages-169-184.pdf</a> .<br>
<br>
Best wishes,<br>
Pavel
<blockquote type="cite"><div><div class="h5">
<div dir="ltr">
<pre style>
</pre>
<pre style>Kevin Xie1 <<a href="http://www.web-polygraph.org/mailman/listinfo/users" target="_blank">Kevin_Xie1 at symantec.com</a>> writes:
><i> Thanks for your quick response, Dmitry.
</i>><i>
</i>><i> However, I don't understand how a single thread can simulate multiple
</i>><i> concurrent "users" submitting request to web servers. Is it that the
</i>><i> single thread iterates all open connections one by one (submitting a
</i>><i> request or processing response one at a time)?
</i>
Sort of, though details are more complex than that. If you are
interested, search for I/O multiplexing.
><i> bound by CPU since just 1 core is used at a time, or by slow
</i>><i> connections ..., do I miss something here, or my understanding is
</i>><i> fundamentally wrong?
</i>><i>
</i>
The fact that a single polygraph process can use only a single core may
be a limitation indeed. But moving each simulated agent to a separate
OS thread is not the solution.
You can run multiple Polygraph client or server processes on a single
system. But that makes it harder to manage the test and usually
requires some auxiliary scripts to start all processes.
It would be nice to have SMP support in Polygraph and we have some ideas
about it. But so far there has not been enough interest in such
project. Patches or sponsorships are welcome.
In any case, please do not make conclusions about Polygraph performance
from the fact that it is single-threaded. Polygraph is successfully
used for high performance tests in 10Gbit networks (though it may
require multiple Polygraph processes and/or systems).
Regards,
Dmitry
><i> Appreciated for any light shed!
</i>><i>
</i>><i> Kevin
</i>><i>
</i>><i> -----Original Message-----
</i>><i> From: Dmitry Kurochkin [mailto:<a href="http://www.web-polygraph.org/mailman/listinfo/users" target="_blank">dmitry.kurochkin at measurement-factory.com</a>]
</i>><i> Sent: May-03-12 3:19 PM
</i>><i> To: Kevin Xie1
</i>><i> Cc: <a href="http://www.web-polygraph.org/mailman/listinfo/users" target="_blank">users at web-polygraph.org</a>
</i>><i> Subject: Re: Is a client robot an OS thread?
</i>><i>
</i>><i> Hi Kevin.
</i>><i>
</i>><i> Kevin Xie1 <<a href="http://www.web-polygraph.org/mailman/listinfo/users" target="_blank">Kevin_Xie1 at symantec.com</a>> writes:
</i>><i>
</i>>><i> Hi,
</i>>><i>
</i>>><i> Is a client robot a real OS thread? I'm using web polygrah in Linux,
</i>>><i> but I doesn't see multiple threads in the OS during a test with 500
</i>>><i> robots.
</i>>><i>
</i>><i>
</i>><i> Polygraph client and server processes are single-threaded. All Robot
</i>><i> and Server agents are running in a single thread.
</i>><i>
</i>><i> Regards,
</i>><i> Dmitry
</i>><i>
</i>>><i> Thanks,
</i>>><i> Kevin</i></pre>
<div>
<div dir="ltr"><span style="border-collapse:separate;color:rgb(34,34,34);font-size:medium;font-family:Helvetica">
<div style="word-wrap:break-word"><span style="border-collapse:separate">
<div style="word-wrap:break-word">
<span style="border-collapse:separate">
<div style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-size:8pt;font-family:Verdana,sans-serif;color:rgb(54,95,145)" lang="EN-US"><b>Alberto Klocker</b> | Software
Developer</span><span style="font-size:8pt;font-family:Verdana,sans-serif;color:rgb(54,95,145)"> | </span><span lang="EN-US"><a href="http://www.bluereef.com.au/" style="color:blue" target="_blank"><span style="font-size:8pt;font-family:Verdana,sans-serif;color:rgb(54,95,145);text-decoration:none" lang="EN-AU">www.bluereef.com.au</span></a></span><span style="font-size:8pt;color:rgb(54,95,145)" lang="EN-US"> </span><span style="font-size:8pt;color:rgb(54,95,145)"></span></div>
<div style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><u><span style="font-size:8pt;color:rgb(54,95,145)">_________________________________________________________________</span></u></div>
</span></div>
</span></div>
</span><span style="color:rgb(34,34,34);font-size:15px;font-family:Calibri,sans-serif"><span style="font-size:8pt;font-family:Verdana,sans-serif;color:rgb(54,95,145)" lang="EN-US"><b>D: </b><a value="+61398958020" style="color:rgb(17,85,204)">+61
3 9895 8006</a> | </span></span><span style="border-collapse:separate;color:rgb(34,34,34);font-size:medium;font-family:Helvetica"><span style="font-size:15px;font-family:Calibri,sans-serif"><b><span style="font-size:8pt;font-family:Verdana,sans-serif;color:rgb(54,95,145)" lang="EN-US">T</span></b></span><span style="font-size:15px;font-family:Calibri,sans-serif"><span style="font-size:8pt;font-family:Verdana,sans-serif;color:rgb(54,95,145)">: <a value="+61398988000" style="color:rgb(17,85,204)">+61 3 9898 8000</a> </span></span><span style="font-size:15px;font-family:Calibri,sans-serif"><span style="font-size:8pt;font-family:Verdana,sans-serif;color:rgb(54,95,145)">| </span></span><a value="+61398988000" style="font-family:Verdana,sans-serif;font-size:11px;color:rgb(17,85,204)">@AlbertoKlocker</a>
<div style="word-wrap:break-word">
<span style="border-collapse:separate">
<div style="word-wrap:break-word"><span style="border-collapse:separate"><span style="border-collapse:separate"><span style="font-family:'Lucida Grande'">
<div>
<span style="border-collapse:separate;font-size:medium;font-family:Calibri"><span><span style="border-collapse:separate;font-size:medium;font-family:Calibri">
<div>
<span style="font-size:8pt;font-family:Verdana,sans-serif;color:rgb(30,147,102)" lang="EN-US">Please consider the
environment before printing this
email</span></div>
</span></span></span></div>
</span></span></span></div>
</span></div>
</span></div>
</div>
</div>
<br>
<fieldset></fieldset>
<br>
</div></div><div class="im"><pre>_______________________________________________
Users mailing list
<a href="mailto:Users@web-polygraph.org" target="_blank">Users@web-polygraph.org</a>
<a href="http://www.web-polygraph.org/mailman/listinfo/users" target="_blank">http://www.web-polygraph.org/mailman/listinfo/users</a>
</pre>
</div></blockquote>
<br>
</div>
</blockquote></div><br></div>