From chengy.fan at gmail.com Thu Nov 3 22:57:58 2011 From: chengy.fan at gmail.com (Chengyu Fan) Date: Thu, 3 Nov 2011 16:57:58 -0600 Subject: Polygraph-reporter generation Message-ID: Hi~ I am a new guy to use web-polygraph. I have a question about report generation. It seems that when I do report generation, I have to put client's log file and server's log file together. But in my scenario, there are lots of clients connect to different servers simultaneously. What I want to do is generating the report for each server. Can I do that and how to do it? Thanks. -- Regards, Chengyu Fan -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitry.kurochkin at measurement-factory.com Thu Nov 3 23:33:44 2011 From: dmitry.kurochkin at measurement-factory.com (Dmitry Kurochkin) Date: Fri, 04 Nov 2011 03:33:44 +0400 Subject: Polygraph-reporter generation In-Reply-To: References: Message-ID: <87mxccvl0n.fsf@gmail.com> Hello. On Thu, 3 Nov 2011 16:57:58 -0600, Chengyu Fan wrote: > Hi~ > I am a new guy to use web-polygraph. > I have a question about report generation. > It seems that when I do report generation, I have to put client's log file > and server's log file together. > But in my scenario, there are lots of clients connect to different > servers simultaneously. What I want to do is generating the report for > each server. Can I do that and how to do it? You can run polygraph-reporter over the logs which you want. It may be multiple logs or a single log. The reporter does not care. But usually you want to generate a report that includes stats from all your clients and servers. Many tests include multiple Polygraph client and server processes, reporter can generate a single report for all of them. Also, in most cases a single polygraph-server (polygraph-client) process simulates multiple Server (Robot) agents. And binary logs from each process contain stats for all agents simulated by this process. You can not extract stats for a single simulated agent from it. Regards, Dmitry > Thanks. > > -- > Regards, > Chengyu Fan > _______________________________________________ > Users mailing list > Users at web-polygraph.org > http://www.web-polygraph.org/mailman/listinfo/users From rousskov at measurement-factory.com Fri Nov 4 15:05:38 2011 From: rousskov at measurement-factory.com (Alex Rousskov) Date: Fri, 04 Nov 2011 09:05:38 -0600 Subject: Polygraph-reporter generation In-Reply-To: References: Message-ID: <4EB3FF42.1020903@measurement-factory.com> On 11/03/2011 04:57 PM, Chengyu Fan wrote: > I have a question about report generation. > It seems that when I do report generation, I have to put client's log > file and server's log file together. > But in my scenario, there are lots of clients connect to different > servers simultaneously. What I want to do is generating the report for > each server. Can I do that and how to do it? Hello Chengyu Fan, Dmitry has already answered that what you want is impossible without placing each Server in its own polygraph-server process, but can you detail _why_ you need per-server statistics or what kind of Server-specific statistics you are looking for? With more information, me might be able to find other ways to do what you need or at least keep your needs in mind when adding new reporting features. Thank you, Alex. From chengy.fan at gmail.com Tue Nov 15 23:55:35 2011 From: chengy.fan at gmail.com (Chengyu Fan) Date: Tue, 15 Nov 2011 16:55:35 -0700 Subject: questions about report and poly-server Message-ID: Hi~ When I try the test, I use downtime-2.pg in the workloads.And I use the log files to generate the report. But I find that some phrase don't have the all the data. For example, I generate report just for phrase "meas", but in the hit ratios, I can only find "offered DHR" in the graph, No "measured DHR". Also, I have "first byte read" in "client byte latency histogram", but no "last byte written". Is this normal? Another question is I find when the client finished all the phases, it will quit automatically. But why Server is still running? Can I stop it using some options? If I missed something apparently, I apologize first. Thanks. -- Regards, Chengyu Fan -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitry.kurochkin at measurement-factory.com Wed Nov 16 16:05:21 2011 From: dmitry.kurochkin at measurement-factory.com (Dmitry Kurochkin) Date: Wed, 16 Nov 2011 20:05:21 +0400 Subject: questions about report and poly-server In-Reply-To: References: Message-ID: <87ty64giim.fsf@gmail.com> Hello. On Tue, 15 Nov 2011 16:55:35 -0700, Chengyu Fan wrote: > Hi~ > When I try the test, I use downtime-2.pg in the workloads.And I use the log > files to generate the report. > But I find that some phrase don't have the all the data. > For example, I generate report just for phrase "meas", You run the reporter with "--phase meas" parameter, right? > but in the hit > ratios, I can only find "offered DHR" in the graph, No "measured DHR". What about the table above the graphs on the "hit ratios" page? Does it have 0 for measured DHR? Or does it match the offered DHR? Do BHR graphs look good? Sometime you do not see the line on a graph because it is covered by another line or is 0. Note that if you are running without proxy, the measured DHR is 0. FWIW I have run the downtime-2 workload with a Squid proxy and I see both measured and offered DHR on the graphs. > Also, I have "first byte read" in "client byte latency histogram", but no > "last byte written". That is ok. It means that Robots send requests without delays i.e. basically at the same moment a transaction is started. Note that there is a table with aggregated byte latency stats above the graphs on the response times page. You should have all zeros in Last request byte written cells (except for Max, perhaps). > Is this normal? > In general it is ok that some stats or graphs are missing. Some workloads may not generate particular types of traffic so some stats are missing. Also, some stats are collected on server- or client-side only (e.g. hit ratios are collected on client-side). So if a report is generated using logs from only one side, corresponding stats would be missing. Of course, it does not mean there can not be bugs related to report generation or stats recording. So please let us know if you find anything suspicious. > Another question is I find when the client finished all the phases, it will > quit automatically. But why Server is still running? A workload may use multiple polygraph-server and polygraph-client processes. Some of these processes may be "faster" than other. Because of that, there may still be clients running (and sending requests) even after a server finishes the last phase. So if the server quits, those clients would start getting errors when sending requests to this server. This is why all servers have implicit "cold" phase at the end. > Can I stop it using > some options? > If you know that all clients have finished, you can stop the server (e.g. with a keyboard interrupt). Otherwise, it would quit on idle timeout. Idle timeout is 5min on server-side by default and can be changed with --idle_tout option. > If I missed something apparently, I apologize first. > I hope the above helps. If you still have troubles understanding why some stats are missing, please consider sending your binary logs and/or HTML report (you can send it in private if you prefer). And note which version of Web Polygraph you are using. Regards, Dmitry > Thanks. > > -- > Regards, > Chengyu Fan > _______________________________________________ > Users mailing list > Users at web-polygraph.org > http://www.web-polygraph.org/mailman/listinfo/users From chengy.fan at gmail.com Wed Nov 16 21:36:21 2011 From: chengy.fan at gmail.com (Chengyu Fan) Date: Wed, 16 Nov 2011 14:36:21 -0700 Subject: questions about report and poly-server In-Reply-To: <87ty64giim.fsf@gmail.com> References: <87ty64giim.fsf@gmail.com> Message-ID: Thanks. You help me a lot! On Wed, Nov 16, 2011 at 9:05 AM, Dmitry Kurochkin < dmitry.kurochkin at measurement-factory.com> wrote: > Hello. > > On Tue, 15 Nov 2011 16:55:35 -0700, Chengyu Fan > wrote: > > Hi~ > > When I try the test, I use downtime-2.pg in the workloads.And I use the > log > > files to generate the report. > > But I find that some phrase don't have the all the data. > > For example, I generate report just for phrase "meas", > > You run the reporter with "--phase meas" parameter, right? > > > but in the hit > > ratios, I can only find "offered DHR" in the graph, No "measured DHR". > > What about the table above the graphs on the "hit ratios" page? Does it > have 0 for measured DHR? Or does it match the offered DHR? Do BHR > graphs look good? > > Sometime you do not see the line on a graph because it is covered by > another line or is 0. > > Note that if you are running without proxy, the measured DHR is 0. > > FWIW I have run the downtime-2 workload with a Squid proxy and I see > both measured and offered DHR on the graphs. > > > Also, I have "first byte read" in "client byte latency histogram", but no > > "last byte written". > > That is ok. It means that Robots send requests without delays > i.e. basically at the same moment a transaction is started. Note that > there is a table with aggregated byte latency stats above the graphs on > the response times page. You should have all zeros in Last request byte > written cells (except for Max, perhaps). > > > Is this normal? > > > > In general it is ok that some stats or graphs are missing. Some > workloads may not generate particular types of traffic so some stats are > missing. Also, some stats are collected on server- or client-side > only (e.g. hit ratios are collected on client-side). So if a report is > generated using logs from only one side, corresponding stats would be > missing. > > Of course, it does not mean there can not be bugs related to report > generation or stats recording. So please let us know if you find > anything suspicious. > > > Another question is I find when the client finished all the phases, it > will > > quit automatically. But why Server is still running? > > A workload may use multiple polygraph-server and polygraph-client > processes. Some of these processes may be "faster" than other. Because > of that, there may still be clients running (and sending requests) even > after a server finishes the last phase. So if the server quits, those > clients would start getting errors when sending requests to this > server. This is why all servers have implicit "cold" phase at the end. > > > Can I stop it using > > some options? > > > > If you know that all clients have finished, you can stop the server > (e.g. with a keyboard interrupt). Otherwise, it would quit on idle > timeout. Idle timeout is 5min on server-side by default and can be > changed with --idle_tout option. > > > If I missed something apparently, I apologize first. > > > > I hope the above helps. If you still have troubles understanding why > some stats are missing, please consider sending your binary logs and/or > HTML report (you can send it in private if you prefer). And note which > version of Web Polygraph you are using. > > Regards, > Dmitry > > > Thanks. > > > > -- > > Regards, > > Chengyu Fan > > _______________________________________________ > > Users mailing list > > Users at web-polygraph.org > > http://www.web-polygraph.org/mailman/listinfo/users > -- Regards, Chengyu Fan -------------- next part -------------- An HTML attachment was scrubbed... URL: From computer_masi at 126.com Thu Nov 24 17:24:05 2011 From: computer_masi at 126.com (computermasi) Date: Fri, 25 Nov 2011 01:24:05 +0800 (CST) Subject: Could polygraph give us the hit ratio of inverse proxy ! Message-ID: <4614132f.c4.133d69b0e28.Coremail.computer_masi@126.com> Hello , everyon~ I'm a newer for polygraph and recently my boss give me the task to configure the enviroment of inverse proxy with "Nginx". So I installed nginx on one machine as the inverse proxy server and put polygraph on another machine to test the performance of "Nginx-inverse-proxy". But the hit ratio is just zero no matter how I adjust the parameters ! Therefore, I wonder if "Polygraph" itself just doesn't (or say cann't) compute the hit ratio of inverse proxy for us! Thanks so much! -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitry.kurochkin at measurement-factory.com Fri Nov 25 02:42:53 2011 From: dmitry.kurochkin at measurement-factory.com (Dmitry Kurochkin) Date: Fri, 25 Nov 2011 06:42:53 +0400 Subject: Could polygraph give us the hit ratio of inverse proxy ! In-Reply-To: <4614132f.c4.133d69b0e28.Coremail.computer_masi@126.com> References: <4614132f.c4.133d69b0e28.Coremail.computer_masi@126.com> Message-ID: <87r50wncr6.fsf@gmail.com> Hello. Polygraph should be able to calculate hit ratio for reverse proxies. Some things that may be worth checking: * Verify that Polygraph generates cachable contents. There should be non-zero offered hit ratio. You can check this in polygraph-lx(1) output or HTML report generated by polygraph-reporter(1). * Verify nginx configuration. Does it say that there were cache hits in logs? You may want to check that caching works by making few requests with curl(1). * Another easy way to check if there were any hits is to compare the number of replies on Polygraph server and client side. You can do this by running polygraph-lx(1) tool on client and server logs: $ polygraph-lx --objects rep.rptm.count log-file If the numbers from server and client logs are about equal, that would means there were no hits. Regards, Dmitry From g.hohl at aurenz.de Wed Nov 30 14:36:02 2011 From: g.hohl at aurenz.de (Hohl, Gerrit) Date: Wed, 30 Nov 2011 15:36:02 +0100 Subject: polygraph-pgl2acl doesn't recognize credentials Message-ID: <3B7A7BDFFD286F49A19A5B35D26E08FB8A457C@srvmail2.aurenz.local> Hello everyone, my project manager introduced web-polygraph to me. We want to test our own proxy with it. I'm playing with the test suite since last week. I set up a Ubuntu 10.04.3 server 64-bit in a virtual machine, downloaded the latest web-polygraph stable version 4.3.2 and compiled it. Currently I'm using a modified polymix-4 test case. Instead of using a few hardware machines I modified the test case in that way that it runs on one machine by using a lot of aliases on the loopback adapter - for the servers as well as for the clients. In the documentation there are also used aliases, but they are distributed on the hardware machines. I've successfully tested the polymix-4 with server and clients without any proxy. I also set up an squid proxy on the same machine and performed a test including it. This also ran without any problems. But now I'm trying to include authentication into the test. I added the following line in the robot: credentials = credentials(count(addresses), "authmix"); and started polygraph-pgl2acl. But it only shows that it couldn't find any authentication rules: polygraph-pgl2acl: parsing... polygraph-pgl2acl: collecting rules... polygraph-pgl2acl: pruning 0 rules ... polygraph-pgl2acl: sorting 0 rules ... polygraph-pgl2acl: symplifying... polygraph-pgl2acl: printing 0 rules ... Normally there should be around 125 IP adresses for the robots. So shouldn't there be also 125 credentials? Why doesn't it find anything? I don't know what I did wrong. Maybe one of you can help me finding my mistake. Regards Gerrit From rousskov at measurement-factory.com Wed Nov 30 16:25:23 2011 From: rousskov at measurement-factory.com (Alex Rousskov) Date: Wed, 30 Nov 2011 09:25:23 -0700 Subject: polygraph-pgl2acl doesn't recognize credentials In-Reply-To: <3B7A7BDFFD286F49A19A5B35D26E08FB8A457C@srvmail2.aurenz.local> References: <3B7A7BDFFD286F49A19A5B35D26E08FB8A457C@srvmail2.aurenz.local> Message-ID: <4ED658F3.70007@measurement-factory.com> On 11/30/2011 07:36 AM, Hohl, Gerrit wrote: > my project manager introduced web-polygraph to me. We want to test our > own proxy with it. Hello Gerrit, Welcome to the list. > I'm trying to include authentication into the test. I added the > following line in the robot: > > credentials = credentials(count(addresses), "authmix"); > > and started polygraph-pgl2acl. But it only shows that it couldn't find > any authentication rules: > > polygraph-pgl2acl: parsing... > polygraph-pgl2acl: collecting rules... > polygraph-pgl2acl: pruning 0 rules ... > polygraph-pgl2acl: sorting 0 rules ... > polygraph-pgl2acl: symplifying... > polygraph-pgl2acl: printing 0 rules ... > > Normally there should be around 125 IP adresses for the robots. So > shouldn't there be also 125 credentials? Why doesn't it find anything? I > don't know what I did wrong. Maybe one of you can help me finding my > mistake. The pgl2acl tool produces Access Control Lists (ACLs) from Robot.acl fields and is not directly related to Robot IP addresses or Robot credentials. I suspect your PGL Robots had no acl rules so the tool produced nothing. You probably do not need those rules if all you are interested in is authentication. What you probably want is the polygraph-pgl2ldif tool. See http://www.web-polygraph.org/docs/userman/auth.html#Sect:3.2 and search for "human-friendly" to find a non-LDIF example that may be useful for generating rules to configure your proxy. HTH, Alex.