From g.hohl at aurenz.de Mon Feb 13 12:24:34 2012 From: g.hohl at aurenz.de (Hohl, Gerrit) Date: Mon, 13 Feb 2012 13:24:34 +0100 Subject: Scaling the number of users instead of the hit rate Message-ID: <3B7A7BDFFD286F49A19A5B35D26E08FB9B3FD5@srvmail2.aurenz.local> Hello everyone, after a longer pause I'm now at our stress testing again. One thing is that I have to include NTLM authentification. Dimtry already wrote a message her and will try it now. But my problem now is that I should be able to scale the number of users (robots) instead of the hit rate. Our product is more sensitive to this than the hit rate as our focus is on authentification. We don't do any caching or things like that. So is there a possibility to scale users? Or maybe I should ask the question in a different way: Is there a way I can calculate the number of expected robots from other values like Bench.client_side.peak_req_rate, Bench.client_side.max_host_load or Bench.client_side.max_agent_load? Regards Gerrit From g.hohl at aurenz.de Tue Feb 14 15:25:00 2012 From: g.hohl at aurenz.de (Hohl, Gerrit) Date: Tue, 14 Feb 2012 16:25:00 +0100 Subject: AW: AW: Problems using robots with authentication In-Reply-To: <8762hhtx18.fsf@gmail.com> References: <3B7A7BDFFD286F49A19A5B35D26E08FB8F1860@srvmail2.aurenz.local> <3B7A7BDFFD286F49A19A5B35D26E08FB8F1868@srvmail2.aurenz.local> <8762hhtx18.fsf@gmail.com> Message-ID: <3B7A7BDFFD286F49A19A5B35D26E08FB9B4167@srvmail2.aurenz.local> Hello Dmitry, okay, the OpenSSL package was installed on my system, but the libssl-dev not. I executed ./configure and everything went fine: All SSL test were green. I execute make and make install. But I still facing some problems. I have credentials in my polygraph test case like this one: string[] cred = [ "TEST/000000poly at client1.windows.local:p1FXn2S165", [...] ]; I've running a squid with NTLM support. But the polygraph-client can't establish a connection to the server using this proxy. It always runs into "407 Proxy Authentication Required" messages. So I used Ethereal Version 0.10.14 on that squid machine - one time I analyzed the communication with the polygraph-client, the second time I used a Firefox on a Window machine. polygraph-client communication ------------------------------ 1st request: GET http://w1141.h1128o1005s1010.bench.tst/w18d91ae1.2d680f1b:00000120/t06/_00000001 HTTP/1.1 Accept: */* Host: w1141.h1128o1005s1010.bench.tst X-Xact: 18d91ae1.2d680f1b:00000002 18d91ae1.2d680f1b:0000042c 0 X-Loc-World: 18d91ae1.2d680f1b:00000120 -1/1 0 X-Rem-World: 18d91ae1.2d680f1b:00000120 -1/1 0 X-Target: w1141.h1128o1005s1010.bench.tst:80 X-Abort: 1412400744 2082554117 X-Phase-Sync-Pos: 0 1st response: HTTP/1.0 407 Proxy Authentication Required Server: squid/2.7.STABLE6 Date: Tue, 14 Feb 2012 13:44:50 GMT Content-Type: text/html Content-Length: 1550 X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0 Proxy-Authenticate: NTLM Proxy-Authenticate: Basic realm="squid server" X-Cache: MISS from polygraph-squid.windows.local X-Cache-Lookup: NONE from polygraph-squid.windows.local:3128 Via: 1.0 polygraph-squid.windows.local:3128 (squid/2.7.STABLE6) Connection: close 2nd request: GET http://w1141.h1128o1005s1010.bench.tst/w18d91ae1.2d680f1b:00000120/t06/_00000001 HTTP/1.1 Accept: */* Host: w1141.h1128o1005s1010.bench.tst X-Xact: 18d91ae1.2d680f1b:00000002 18d91ae1.2d680f1b:0000042e 0 X-Loc-World: 18d91ae1.2d680f1b:00000120 -1/1 0 X-Rem-World: 18d91ae1.2d680f1b:00000120 -1/1 0 X-Target: w1141.h1128o1005s1010.bench.tst:80 X-Abort: 1798565613 512442519 X-Phase-Sync-Pos: 0 Proxy-Authorization: NTLM TlRMTVNTUAABAAAABoIIAAAAAAAAAAAAAAAAAAAAAAA= Ethereal shows the following information in Proxy-Authorization > NTLMSSP Flags: 0x00088206 Calling workstation domain: NULL Calling workstation name: NULL 2nd response: HTTP/1.0 407 Proxy Authentication Required Server: squid/2.7.STABLE6 Date: Tue, 14 Feb 2012 13:44:51 GMT Content-Type: text/html Content-Length: 1550 X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0 Proxy-Authenticate: Basic realm="squid server" X-Cache: MISS from polygraph-squid.windows.local X-Cache-Lookup: NONE from polygraph-squid.windows.local:3128 Via: 1.0 polygraph-squid.windows.local:3128 (squid/2.7.STABLE6) Connection: close Windows / Firefox communication ------------------------------- 1st request: GET http://download.mozilla.org/?product=firefox-10.0.1-complete&os=win&lang=de HTTP/1.1 Host: download.mozilla.org User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0.1) Gecko/20100101 Firefox/8.0.1 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3 Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Proxy-Connection: keep-alive Range: bytes=300000-599999 Cookie: dmo=10.8.84.211.1329128259926354 1st response: HTTP/1.0 407 Proxy Authentication Required Server: squid/2.7.STABLE6 Date: Tue, 14 Feb 2012 14:16:05 GMT Content-Type: text/html Content-Length: 1397 X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0 Proxy-Authenticate: NTLM Proxy-Authenticate: Basic realm="squid server" X-Cache: MISS from polygraph-squid.windows.local X-Cache-Lookup: NONE from polygraph-squid.windows.local:3128 Via: 1.0 polygraph-squid.windows.local:3128 (squid/2.7.STABLE6) Connection: close 2nd request: GET http://download.mozilla.org/?product=firefox-10.0.1-complete&os=win&lang=de HTTP/1.1 Host: download.mozilla.org User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0.1) Gecko/20100101 Firefox/8.0.1 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3 Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Proxy-Connection: keep-alive Range: bytes=300000-599999 Cookie: dmo=10.8.84.211.1329128259926354 Proxy-Authorization: NTLM TlRMTVNTUAABAAAAB4IIogAAAAAAAAAAAAAAAAAAAAAGAbAdAAAADw== Ethereal shows the following information in Proxy-Authorization > NTLMSSP Flags: 0xA2088207 Calling workstation domain: NULL Calling workstation name: NULL 2nd response: HTTP/1.0 407 Proxy Authentication Required Server: squid/2.7.STABLE6 Date: Tue, 14 Feb 2012 14:16:05 GMT Content-Type: text/html Content-Length: 1397 X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0 Proxy-Authenticate: NTLM TlRMTVNTUAACAAAACAAIADgAAAAFgomiCEQRwpFGL1oAAAAAAAAAAMgAyABAAAAABgGwHQAAAA9UAEUAUwBUAAIACABUAEUAUwBUAAEAGgBXADIASwA4AFIAMgBTAFIAVgAtAEQATwBNAAQAIgB0AGUAcwB0AC4AYQB1AHIAZQBuAHoALgBsAG8AYwBhAGwAAwA+AFcAMgBLADgAUgAyAFMAUgBWAC0ARABPAE0ALgB0AGUAcwB0AC4AYQB1AHIAZQBuAHoALgBsAG8AYwBhAGwABQAiAHQAZQBzAHQALgBhAHUAcgBlAG4AegAuAGwAbwBjAGEAbAAHAAgAcxVmHSPrzAEAAAAA X-Cache: MISS from polygraph-squid.windows.local X-Cache-Lookup: NONE from polygraph-squid.windows.local:3128 Via: 1.0 polygraph-squid.windows.local:3128 (squid/2.7.STABLE6) Connection: keep-alive Proxy-Connection: keep-alive Ethereal shows the following information in Proxy-Authorization > NTLMSSP Flags: 0xA2908205 3rd request: GET http://download.mozilla.org/?product=firefox-10.0.1-complete&os=win&lang=de HTTP/1.1 Host: download.mozilla.org User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0.1) Gecko/20100101 Firefox/8.0.1 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3 Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Proxy-Connection: keep-alive Range: bytes=300000-599999 Cookie: dmo=10.8.84.211.1329128259926354 Proxy-Authorization: NTLM TlRMTVNTUAADAAAAGAAYAH4AAAAYABgAlgAAAAgACABYAAAADgAOAGAAAAAQABAAbgAAAAAAAACuAAAABYKIogYBsB0AAAAPn8qbQDsUUb8Odt0FrfLvDVQARQBTAFQAdABlAHMAdABlAHIANQBDAEwASQBFAE4AVAAtADMAoHqz/a+76EoAAAAAAAAAAAAAAAAAAAAAwOMG9tdnijCslk8x46O5Jk5+0GXpoiPd Ethereal shows the following information in Proxy-Authorization > NTLMSSP Flags: 0xA2888205 3rd response: HTTP/1.0 302 Moved Temporarily Date: Tue, 14 Feb 2012 14:15:34 GMT Server: Apache X-Backend-Server: pp-app-dist01 X-Powered-By: PHP/5.1.6 Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0, private Pragma: no-cache Location: http://mirror01.th.ifl.net/mozilla-releases/firefox/releases/10.0.1/update/win32/de/firefox-10.0.1.complete.mar Content-Length: 0 Content-Type: text/html; charset=UTF-8 X-Cache: MISS from polygraph-squid.windows.local X-Cache-Lookup: MISS from polygraph-squid.windows.local:3128 Via: 1.1 polygraph-squid.windows.local:3128 (squid/2.7.STABLE6) Connection: keep-alive Proxy-Connection: keep-alive Because of the NULL values I first thought that this was the problem. But in the second test there are also NULL values. The most significant difference are the NTLM flags. Can this be the cause of the problem? I made a table which compares the different flags. And it shows that Firefox supports 56-bit as well as 128-bit encryption, but WebPolgraph doesn't. Maybe we (mean squid) need this? WP2Q -> WebPolygraph 2nd reQuest WF2Q -> Windows Firefox 2nd reQuest WF2S -> Windows Firefox 2nd reSponse WF3Q -> Windows Firefox 3rd reQuest Value (from highest to lowest bit) | WP2Q | WF2Q | WF2S | WF3Q ---------------------------------------|------|------|------|------ Negotiate 56 | - | x | x | x Negotiate Key Exchange | - | - | - | - Negotiate 128 | - | x | x | x Negotiate 0x10000000 | - | - | - | - Negotiate 0x08000000 | - | - | - | - Negotiate 0x04000000 | - | - | - | - Negotiate 0x02000000 | - | x | x | x Negotiate 0x01000000 | - | - | - | - Negotiate Target Info | - | - | x | x Negotiate 0x00400000 | - | - | - | - Negotiate 0x00200000 | - | - | - | - Negotiate 0x00100000 | - | - | x | - Negotiate NTLM2 key | x | x | - | x Negotiate Challenge Non NT session Key | - | - | - | - Negotiate Challenge Accept Response | - | - | - | - Negotiate Challenge Init Reponse | - | - | - | - Negotiate Always Sign | x | x | x | x Negotiate This is Local Call | - | - | - | - Negotiate Workstation Supplied | - | - | - | - Negotiate Domain Supplied | - | - | - | - Negotiate 0x00000800 | - | - | - | - Negotiate 0x00000400 | - | - | - | - Negotiate NTLM key | x | x | x | x Negotiate Netware | - | - | - | - Negotiate Lan Manager Key | - | - | - | - Negotiate Datagramm Style | - | - | - | - Negotiate Seal | - | - | - | - Negotiate Sign | - | - | - | - Request 0x00000008 | - | - | - | - Request Target | x | x | x | x Negotiate OEM | x | x | - | - Negotiate UNICODE | - | x | x | x I hope my mail didn't grow too long. But I wanted it be as detailed as possible. Maybe it will help to find my mistake. Regards, Gerrit -----Urspr?ngliche Nachricht----- Von: Dmitry Kurochkin [mailto:dmitry.kurochkin at measurement-factory.com] Gesendet: Donnerstag, 15. Dezember 2011 21:12 An: Hohl, Gerrit; users at web-polygraph.org Betreff: Re: AW: Problems using robots with authentication Hi Gerrit. On Thu, 15 Dec 2011 17:36:34 +0100, "Hohl, Gerrit" wrote: > Hello everyone, > > I read the article "Prerequisites" in the documentation: > http://www.web-polygraph.org/docs/reference/models/ssl.html#Sect:2 > > Polygraph SSL support is based on the OpenSSL library. A recent version of the library is required to compile Polygraph. We have tested with OpenSSL versions 0.9.6g and 0.9.7b. The presence of OpenSSL is determined at ./configure time. Please check that ./configure actually found SSL library and headers if you install Polygraph and want SSL support: > > ... > checking for CRYPTO_lock in -lcrypto... yes > checking for SSL_connect in -lssl... yes > checking for openssl/ssl.h... yes > checking for openssl/err.h... yes > checking for openssl/rand.h... yes > > Eh, I don't get these lines when I call the script. I assume that the script was modified, but the documentation not. > The exact messages may have change, but they are still there: $ ./configure | grep -i ssl checking for SSL_connect in -lssl... yes checking openssl/ssl.h usability... yes checking openssl/ssl.h presence... yes checking for openssl/ssl.h... yes checking openssl/err.h usability... yes checking openssl/err.h presence... yes checking for openssl/err.h... yes checking openssl/rand.h usability... yes checking openssl/rand.h presence... yes checking for openssl/rand.h... yes Assertion at NtlmAuth.cc:798 means that you built without OpenSSL support. You should install it and rebuild. On Debian-based systems you should install libssl-dev package. The assertion is always a bug. Web Polygraph should print a proper error here. There is an open bug #878881 [1] for this issue. > One remark: I only want NTLM authentication and not HTTPS benchmarking. > NTLM needs some crypto functions (MD5, at least). That is why OpenSSL is needed for it. Regards, Dmitry [1] https://bugs.launchpad.net/polygraph/+bug/878881 > Regards, > Gerrit > > -----Urspr?ngliche Nachricht----- > Von: users-bounces at web-polygraph.org > [mailto:users-bounces at web-polygraph.org] Im Auftrag von Hohl, Gerrit > Gesendet: Donnerstag, 15. Dezember 2011 17:08 > An: users at web-polygraph.org > Betreff: Problems using robots with authentication > > Hello everyone, > > I have the same problem that ufa faced at the end of October: > I use NTLM authentication and get the message > > NtlmAuth.cc:798: assertion failed: 'false' > Aborted > > from the polygraph-client program. After the first time I received that message I included the following line in my PGL file: > > Robot robot = { > [...] > pconn_use_lmt = const(2147483647); > [...] > }; > > But I still get the same message. I read something in Dmitry response about OpenSSL. The OpenSSL package ('openssl') is installed on that Ubuntu machine I'm currently using. But I'm not sure if polygraph was compiled with or without SSL. Is there a way to test it? Or what do I have to do to make sure that polygraph compiles with SSL support? > > Regards, > Gerrit > _______________________________________________ > Users mailing list > Users at web-polygraph.org > http://www.web-polygraph.org/mailman/listinfo/users > _______________________________________________ > Users mailing list > Users at web-polygraph.org > http://www.web-polygraph.org/mailman/listinfo/users From eacg at cin.ufpe.br Wed Feb 15 02:17:09 2012 From: eacg at cin.ufpe.br (Erico Augusto Cavalcanti Guedes) Date: Tue, 14 Feb 2012 23:17:09 -0300 Subject: About "mean response time" Message-ID: Dears, on PG docs about Run-time stats: (1) (2)(3) (4) (5) (6) (7) (8) (9) 000.09| i-dflt 23 4.52 196 0.00 0 1 000.17| i-dflt 48 5.00 199 0.00 0 1 000.25| i-dflt 73 5.00 199 0.00 0 1 000.34| i-dflt 98 5.00 198 0.00 0 1 000.42| i-dflt 125 5.40 185 0.00 0 1 Here is a brief explanation for each column of the run-time stats output. 1. Minutes since start 2. "i-" and "p-" stand for interval and phase-based stats 3. current phase name 4. number of replies received so far (across all phases) 5. reply rate (replies per second) 6. mean response time in milliseconds 7. hit ratio in percents (client side); 8. nothing interesting on the server side yet 9. number of transaction errors during that interval or phase (note: not all errors are transaction errors) 10. number of open sockets, including UDP and other housekeeping sockets if any; 11. this number is usually close to the number of pending transactions; (interval stats only!) The question: Is number 6 (mean response time) the mean time of all responses or is it related with number 7 (hit ratio in percents) and it is the mean response time of requests that were on cache? Am I been clear? Thanks in advance. Erico. -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.hohl at aurenz.de Wed Feb 15 09:09:12 2012 From: g.hohl at aurenz.de (Hohl, Gerrit) Date: Wed, 15 Feb 2012 10:09:12 +0100 Subject: AW: Scaling the number of users instead of the hit rate In-Reply-To: <3B7A7BDFFD286F49A19A5B35D26E08FB9B3FD5@srvmail2.aurenz.local> References: <3B7A7BDFFD286F49A19A5B35D26E08FB9B3FD5@srvmail2.aurenz.local> Message-ID: <3B7A7BDFFD286F49A19A5B35D26E08FB9B41ED@srvmail2.aurenz.local> Good morning everyone, maybe I should change my question a little bit: In the following article I found a formular for calculating the expected numbers of robots: http://polygraph.ircache.net/Workloads/PolyMix-2/ "[...] 6.2 Number of clients and servers Given the total request rate RR req/sec, we allocate (R = RR/0.4 = 2.5*RR) robot IP addresses (0.4 is request rate for an individual robot). The number of server IP addresses is then 0.1*R+500. Robot and server IPs are distributed evenly and sequentially across Polygraph machines (see example). We place a limit of 1000 robots per client machine. The number of server machines is equal to the number of client machines. When the calculations produce non-integer value V, we round towards the closest integer greater than V. [...]" Does that also apply if I use the PolyMix4As class? Bench theBench = { peak_req_rate = 500/sec; client_side = { max_host_load = 20000/sec; max_agent_load = 6/min; addr_space = ... ; hosts = ... ; }; server_side = { max_host_load = client_side.max_host_load; max_agent_load = undef(); addr_space = ... ; hosts = ... ; }; }; PolyMix4As asPolyMix4 = { agents_per_addr = 2; }; Will this configuration result in 5000 robots, 2 robots on each of 2500 IP addresses? robots = peak_req_rate / max_agent_load = 500/sec / 6/min = 30,000/min / 6/min = 30,000 / 6 = 5000 Regards Gerrit -----Urspr?ngliche Nachricht----- Von: users-bounces at web-polygraph.org [mailto:users-bounces at web-polygraph.org] Im Auftrag von Hohl, Gerrit Gesendet: Montag, 13. Februar 2012 13:25 An: users at web-polygraph.org Betreff: Scaling the number of users instead of the hit rate Hello everyone, after a longer pause I'm now at our stress testing again. One thing is that I have to include NTLM authentification. Dimtry already wrote a message her and will try it now. But my problem now is that I should be able to scale the number of users (robots) instead of the hit rate. Our product is more sensitive to this than the hit rate as our focus is on authentification. We don't do any caching or things like that. So is there a possibility to scale users? Or maybe I should ask the question in a different way: Is there a way I can calculate the number of expected robots from other values like Bench.client_side.peak_req_rate, Bench.client_side.max_host_load or Bench.client_side.max_agent_load? Regards Gerrit _______________________________________________ Users mailing list Users at web-polygraph.org http://www.web-polygraph.org/mailman/listinfo/users From g.hohl at aurenz.de Wed Feb 15 10:46:44 2012 From: g.hohl at aurenz.de (Hohl, Gerrit) Date: Wed, 15 Feb 2012 11:46:44 +0100 Subject: WG: Scaling the number of users instead of the hit rate Message-ID: <3B7A7BDFFD286F49A19A5B35D26E08FB9B421D@srvmail2.aurenz.local> Hello everyone, I went through the code of web polygraph 4.3.2. And I guess I found the parts which do the calculation of the numbers for robots and servers. I wrote a small Java class that does this calculation so I can play with the values without starting the polygraph-client and polygraph-server over and over again: public class Test { /** * @param args */ public static void main(String[] args) { double max_host_load, max_agent_load, peak_req_rate; int agents_per_addr, robotCnt, serverCnt; peak_req_rate = 500; max_host_load = 20000; max_agent_load = 6 / 60d; agents_per_addr = 2; System.out.println("Parameters"); System.out.println(" Peak req. rate (per sec.) : " + peak_req_rate); System.out.println(" Max host load (per sec.) : " + max_host_load); System.out.println(" Max agent load (per sec.) : " + max_agent_load); System.out.println(" Agents per addr. : " + agents_per_addr); robotCnt = calcRobots(peak_req_rate, max_host_load, max_agent_load, agents_per_addr); serverCnt = calcServers(peak_req_rate, max_host_load, robotCnt); System.out.println("Result"); System.out.println(" No# of robots : " + robotCnt); System.out.println(" No# of servers : " + serverCnt); } public static int calcServers(double peak_req_rate, double max_host_load, int robotCnt) { /* * /pgl/PolyMix4AsSym * String servers(ArraySym *&addrs) */ int minSrvCount, hostCnt, agentCnt; minSrvCount = (int)(500.5 + (0.1 * robotCnt)); hostCnt = (int) xceil(peak_req_rate, max_host_load); agentCnt = singleDiv(hostCnt, minSrvCount); System.out.println("Servers"); System.out.println(" No# of hosts : " + hostCnt); System.out.println(" No# of agents : " + agentCnt); return agentCnt; } public static int calcRobots(double peak_req_rate, double max_host_load, double max_agent_load, int agents_per_addr) { /* * /pgl/VerFourAsSym * String addresses(const BenchSideSym *side, ArraySym *&addrs) */ int maxAgentsPerHost, maxAddrPerHost, hostCnt, agentCnt; maxAgentsPerHost = (int) xceil(max_host_load, max_agent_load); maxAddrPerHost = (int) xceil(maxAgentsPerHost, agents_per_addr); hostCnt = (int) xceil(peak_req_rate, max_host_load); agentCnt = singleDiv(hostCnt * agents_per_addr, doubleDiv(hostCnt, peak_req_rate, max_agent_load)); if (agentCnt > agents_per_addr * maxAddrPerHost * hostCnt) System.out.println("too many agents?"); System.out.println("Robots"); System.out.println(" No# of hosts : " + hostCnt); System.out.println(" No# of agents : " + agentCnt); return agentCnt; } public static double xceil(double nom, double denom) { /* * /xstd/gadgets.cc */ double cex, cm1, cp1, dm1, dex, dp1, result = 0; cex = Math.ceil(nom / denom); cm1 = cex - 1; cp1 = cex + 1; dm1 = Math.abs(nom - (cm1 * denom)); dex = Math.abs(nom - (cex * denom)); dp1 = Math.abs(nom - (cp1 * denom)); if ((dm1 <= dex) && (nom <= cm1 * denom)) result = ((dm1 <= dp1) || !(nom <= cp1 * denom)) ? cm1 : cp1; else result = ((dex <= dp1) || !(nom <= cp1 * denom)) ? cex : cp1; return result; } public static int singleDiv(int factor, double n) { /* * /pgl/AddrSchemeSym */ return (int)(factor * xceil(n, factor)); } public static int doubleDiv(int factor, double n, double d) { /* * /pgl/AddrSchemeSym */ double apx; apx = xceil(n, d); return singleDiv(factor, apx); } } And it seems that the formula of my last post was right: I get the expected values for the given values. One remark about my Java code: It only implements PolyMix4. You can't calculate other schemes like PolyMix3As and so on as they may have a different formula. That means if I want to scale the number of robots but not the request rate, I have to transform the formula like this: robots = peak_req_rate / max_agent_load -> peak_req_rate = robots * max_agent_load In my case the following is constant max_agent_load = 6/min; So when I want a different number of robots, I have to calculate the resulting peak_req_rate. robots | peak_req_rate --------+--------------- 1000 | 100/sec 2500 | 250/sec 5000 | 500/sec 7500 | 750/sec 10000 | 1000/sec 15000 | 1500/sec 20000 | 2000/sec 30000 | 3000/sec 50000 | 5000/sec ... | ... Can someone please confirm my theory? Regards, Gerrit -----Urspr?ngliche Nachricht----- Von: Hohl, Gerrit Gesendet: Mittwoch, 15. Februar 2012 10:09 An: 'users at web-polygraph.org' Betreff: AW: Scaling the number of users instead of the hit rate Good morning everyone, maybe I should change my question a little bit: In the following article I found a formular for calculating the expected numbers of robots: http://polygraph.ircache.net/Workloads/PolyMix-2/ "[...] 6.2 Number of clients and servers Given the total request rate RR req/sec, we allocate (R = RR/0.4 = 2.5*RR) robot IP addresses (0.4 is request rate for an individual robot). The number of server IP addresses is then 0.1*R+500. Robot and server IPs are distributed evenly and sequentially across Polygraph machines (see example). We place a limit of 1000 robots per client machine. The number of server machines is equal to the number of client machines. When the calculations produce non-integer value V, we round towards the closest integer greater than V. [...]" Does that also apply if I use the PolyMix4As class? Bench theBench = { peak_req_rate = 500/sec; client_side = { max_host_load = 20000/sec; max_agent_load = 6/min; addr_space = ... ; hosts = ... ; }; server_side = { max_host_load = client_side.max_host_load; max_agent_load = undef(); addr_space = ... ; hosts = ... ; }; }; PolyMix4As asPolyMix4 = { agents_per_addr = 2; }; Will this configuration result in 5000 robots, 2 robots on each of 2500 IP addresses? robots = peak_req_rate / max_agent_load = 500/sec / 6/min = 30,000/min / 6/min = 30,000 / 6 = 5000 Regards Gerrit -----Urspr?ngliche Nachricht----- Von: users-bounces at web-polygraph.org [mailto:users-bounces at web-polygraph.org] Im Auftrag von Hohl, Gerrit Gesendet: Montag, 13. Februar 2012 13:25 An: users at web-polygraph.org Betreff: Scaling the number of users instead of the hit rate Hello everyone, after a longer pause I'm now at our stress testing again. One thing is that I have to include NTLM authentification. Dimtry already wrote a message her and will try it now. But my problem now is that I should be able to scale the number of users (robots) instead of the hit rate. Our product is more sensitive to this than the hit rate as our focus is on authentification. We don't do any caching or things like that. So is there a possibility to scale users? Or maybe I should ask the question in a different way: Is there a way I can calculate the number of expected robots from other values like Bench.client_side.peak_req_rate, Bench.client_side.max_host_load or Bench.client_side.max_agent_load? Regards Gerrit _______________________________________________ Users mailing list Users at web-polygraph.org http://www.web-polygraph.org/mailman/listinfo/users From dmitry.kurochkin at measurement-factory.com Wed Feb 15 16:13:40 2012 From: dmitry.kurochkin at measurement-factory.com (Dmitry Kurochkin) Date: Wed, 15 Feb 2012 20:13:40 +0400 Subject: About "mean response time" In-Reply-To: References: Message-ID: <878vk4gjm3.fsf@gmail.com> Hi Erico. On Tue, 14 Feb 2012 23:17:09 -0300, Erico Augusto Cavalcanti Guedes wrote: > Dears, > > on PG docs about Run-time stats: > (1) (2)(3) (4) (5) (6) (7) (8) (9) > 000.09| i-dflt 23 4.52 196 0.00 0 1 > 000.17| i-dflt 48 5.00 199 0.00 0 1 > 000.25| i-dflt 73 5.00 199 0.00 0 1 > 000.34| i-dflt 98 5.00 198 0.00 0 1 > 000.42| i-dflt 125 5.40 185 0.00 0 1 > > Here is a brief explanation for each column of the run-time stats output. > 1. Minutes since start > 2. "i-" and "p-" stand for interval and phase-based stats > 3. current phase name > 4. number of replies received so far (across all phases) > 5. reply rate (replies per second) > 6. mean response time in milliseconds > 7. hit ratio in percents (client side); > 8. nothing interesting on the server side yet > 9. number of transaction errors during that interval or phase (note: not > all errors are transaction errors) > 10. number of open sockets, including UDP and other housekeeping sockets if > any; > 11. this number is usually close to the number of pending transactions; > (interval stats only!) > > The question: > > Is number 6 (mean response time) the mean time of all responses or is it > related with number 7 (hit ratio in percents) and it is the mean response > time of requests that were on cache? > It is the mean time of all responses received by the client, including both hits and misses. If you are interested in detailed response time stats for different kind of replies (hits, misses and more), they are available in binary logs. You can read binary logs with polygraph-lx(1) tool or generate an HTML report with polygraph-reporter(1). Regards, Dmitry > Am I been clear? > > Thanks in advance. > > Erico. > _______________________________________________ > Users mailing list > Users at web-polygraph.org > http://www.web-polygraph.org/mailman/listinfo/users From dmitry.kurochkin at measurement-factory.com Wed Feb 15 19:08:40 2012 From: dmitry.kurochkin at measurement-factory.com (Dmitry Kurochkin) Date: Wed, 15 Feb 2012 23:08:40 +0400 Subject: AW: AW: Problems using robots with authentication In-Reply-To: <3B7A7BDFFD286F49A19A5B35D26E08FB9B4167@srvmail2.aurenz.local> References: <3B7A7BDFFD286F49A19A5B35D26E08FB8F1860@srvmail2.aurenz.local> <3B7A7BDFFD286F49A19A5B35D26E08FB8F1868@srvmail2.aurenz.local> <8762hhtx18.fsf@gmail.com> <3B7A7BDFFD286F49A19A5B35D26E08FB9B4167@srvmail2.aurenz.local> Message-ID: <87zkcjgbif.fsf@gmail.com> Hi Gerrit. On Tue, 14 Feb 2012 16:25:00 +0100, "Hohl, Gerrit" wrote: > Hello Dmitry, > > okay, the OpenSSL package was installed on my system, but the libssl-dev not. I executed ./configure and everything went fine: All SSL test were green. I execute make and make install. But I still facing some problems. > > I have credentials in my polygraph test case like this one: > > string[] cred = [ "TEST/000000poly at client1.windows.local:p1FXn2S165", [...] ]; > > I've running a squid with NTLM support. But the polygraph-client can't establish a connection to the server using this proxy. It always runs into "407 Proxy Authentication Required" messages. So I used Ethereal Version 0.10.14 on that squid machine - one time I analyzed the communication with the polygraph-client, the second time I used a Firefox on a Window machine. > > polygraph-client communication > ------------------------------ > 1st request: > > GET http://w1141.h1128o1005s1010.bench.tst/w18d91ae1.2d680f1b:00000120/t06/_00000001 HTTP/1.1 > Accept: */* > Host: w1141.h1128o1005s1010.bench.tst > X-Xact: 18d91ae1.2d680f1b:00000002 18d91ae1.2d680f1b:0000042c 0 > X-Loc-World: 18d91ae1.2d680f1b:00000120 -1/1 0 > X-Rem-World: 18d91ae1.2d680f1b:00000120 -1/1 0 > X-Target: w1141.h1128o1005s1010.bench.tst:80 > X-Abort: 1412400744 2082554117 > X-Phase-Sync-Pos: 0 > > 1st response: > > HTTP/1.0 407 Proxy Authentication Required > Server: squid/2.7.STABLE6 > Date: Tue, 14 Feb 2012 13:44:50 GMT > Content-Type: text/html > Content-Length: 1550 > X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0 > Proxy-Authenticate: NTLM > Proxy-Authenticate: Basic realm="squid server" > X-Cache: MISS from polygraph-squid.windows.local > X-Cache-Lookup: NONE from polygraph-squid.windows.local:3128 > Via: 1.0 polygraph-squid.windows.local:3128 (squid/2.7.STABLE6) > Connection: close > > 2nd request: > > GET http://w1141.h1128o1005s1010.bench.tst/w18d91ae1.2d680f1b:00000120/t06/_00000001 HTTP/1.1 > Accept: */* > Host: w1141.h1128o1005s1010.bench.tst > X-Xact: 18d91ae1.2d680f1b:00000002 18d91ae1.2d680f1b:0000042e 0 > X-Loc-World: 18d91ae1.2d680f1b:00000120 -1/1 0 > X-Rem-World: 18d91ae1.2d680f1b:00000120 -1/1 0 > X-Target: w1141.h1128o1005s1010.bench.tst:80 > X-Abort: 1798565613 512442519 > X-Phase-Sync-Pos: 0 > Proxy-Authorization: NTLM TlRMTVNTUAABAAAABoIIAAAAAAAAAAAAAAAAAAAAAAA= > > Ethereal shows the following information in Proxy-Authorization > NTLMSSP > Flags: 0x00088206 > Calling workstation domain: NULL > Calling workstation name: NULL > > 2nd response: > > HTTP/1.0 407 Proxy Authentication Required > Server: squid/2.7.STABLE6 > Date: Tue, 14 Feb 2012 13:44:51 GMT > Content-Type: text/html > Content-Length: 1550 > X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0 > Proxy-Authenticate: Basic realm="squid server" > X-Cache: MISS from polygraph-squid.windows.local > X-Cache-Lookup: NONE from polygraph-squid.windows.local:3128 > Via: 1.0 polygraph-squid.windows.local:3128 (squid/2.7.STABLE6) > Connection: close > > Windows / Firefox communication > ------------------------------- > 1st request: > > GET http://download.mozilla.org/?product=firefox-10.0.1-complete&os=win&lang=de HTTP/1.1 > Host: download.mozilla.org > User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0.1) Gecko/20100101 Firefox/8.0.1 > Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3 > Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 > Proxy-Connection: keep-alive > Range: bytes=300000-599999 > Cookie: dmo=10.8.84.211.1329128259926354 > > 1st response: > > HTTP/1.0 407 Proxy Authentication Required > Server: squid/2.7.STABLE6 > Date: Tue, 14 Feb 2012 14:16:05 GMT > Content-Type: text/html > Content-Length: 1397 > X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0 > Proxy-Authenticate: NTLM > Proxy-Authenticate: Basic realm="squid server" > X-Cache: MISS from polygraph-squid.windows.local > X-Cache-Lookup: NONE from polygraph-squid.windows.local:3128 > Via: 1.0 polygraph-squid.windows.local:3128 (squid/2.7.STABLE6) > Connection: close > > 2nd request: > > GET http://download.mozilla.org/?product=firefox-10.0.1-complete&os=win&lang=de HTTP/1.1 > Host: download.mozilla.org > User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0.1) Gecko/20100101 Firefox/8.0.1 > Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3 > Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 > Proxy-Connection: keep-alive > Range: bytes=300000-599999 > Cookie: dmo=10.8.84.211.1329128259926354 > Proxy-Authorization: NTLM TlRMTVNTUAABAAAAB4IIogAAAAAAAAAAAAAAAAAAAAAGAbAdAAAADw== > > Ethereal shows the following information in Proxy-Authorization > NTLMSSP > Flags: 0xA2088207 > Calling workstation domain: NULL > Calling workstation name: NULL > > 2nd response: > > HTTP/1.0 407 Proxy Authentication Required > Server: squid/2.7.STABLE6 > Date: Tue, 14 Feb 2012 14:16:05 GMT > Content-Type: text/html > Content-Length: 1397 > X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0 > Proxy-Authenticate: NTLM TlRMTVNTUAACAAAACAAIADgAAAAFgomiCEQRwpFGL1oAAAAAAAAAAMgAyABAAAAABgGwHQAAAA9UAEUAUwBUAAIACABUAEUAUwBUAAEAGgBXADIASwA4AFIAMgBTAFIAVgAtAEQATwBNAAQAIgB0AGUAcwB0AC4AYQB1AHIAZQBuAHoALgBsAG8AYwBhAGwAAwA+AFcAMgBLADgAUgAyAFMAUgBWAC0ARABPAE0ALgB0AGUAcwB0AC4AYQB1AHIAZQBuAHoALgBsAG8AYwBhAGwABQAiAHQAZQBzAHQALgBhAHUAcgBlAG4AegAuAGwAbwBjAGEAbAAHAAgAcxVmHSPrzAEAAAAA > X-Cache: MISS from polygraph-squid.windows.local > X-Cache-Lookup: NONE from polygraph-squid.windows.local:3128 > Via: 1.0 polygraph-squid.windows.local:3128 (squid/2.7.STABLE6) > Connection: keep-alive > Proxy-Connection: keep-alive > > Ethereal shows the following information in Proxy-Authorization > NTLMSSP > Flags: 0xA2908205 > > 3rd request: > > GET http://download.mozilla.org/?product=firefox-10.0.1-complete&os=win&lang=de HTTP/1.1 > Host: download.mozilla.org > User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0.1) Gecko/20100101 Firefox/8.0.1 > Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3 > Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 > Proxy-Connection: keep-alive > Range: bytes=300000-599999 > Cookie: dmo=10.8.84.211.1329128259926354 > Proxy-Authorization: NTLM TlRMTVNTUAADAAAAGAAYAH4AAAAYABgAlgAAAAgACABYAAAADgAOAGAAAAAQABAAbgAAAAAAAACuAAAABYKIogYBsB0AAAAPn8qbQDsUUb8Odt0FrfLvDVQARQBTAFQAdABlAHMAdABlAHIANQBDAEwASQBFAE4AVAAtADMAoHqz/a+76EoAAAAAAAAAAAAAAAAAAAAAwOMG9tdnijCslk8x46O5Jk5+0GXpoiPd > > Ethereal shows the following information in Proxy-Authorization > NTLMSSP > Flags: 0xA2888205 > > 3rd response: > > HTTP/1.0 302 Moved Temporarily > Date: Tue, 14 Feb 2012 14:15:34 GMT > Server: Apache > X-Backend-Server: pp-app-dist01 > X-Powered-By: PHP/5.1.6 > Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0, private > Pragma: no-cache > Location: http://mirror01.th.ifl.net/mozilla-releases/firefox/releases/10.0.1/update/win32/de/firefox-10.0.1.complete.mar > Content-Length: 0 > Content-Type: text/html; charset=UTF-8 > X-Cache: MISS from polygraph-squid.windows.local > X-Cache-Lookup: MISS from polygraph-squid.windows.local:3128 > Via: 1.1 polygraph-squid.windows.local:3128 (squid/2.7.STABLE6) > Connection: keep-alive > Proxy-Connection: keep-alive > > Because of the NULL values I first thought that this was the problem. But in the second test there are also NULL values. The most significant difference are the NTLM flags. Can this be the cause of the problem? > I made a table which compares the different flags. And it shows that Firefox supports 56-bit as well as 128-bit encryption, but WebPolgraph doesn't. Maybe we (mean squid) need this? > Web Polygraph NTLM support has room for improvement. I suspect it does not work with some proxy configurations. It looks like in your particular case, flags in the NTLM T1 message is the (first) problem. You may want to look at the (debug) logs on the proxy side. It may give more information on why the authentication request is rejected. It may be possible to solve the issue by changing the proxy configuration (e.g. enabling 56bit encryption). You may also try modifying Web Polygraph sources. To change the flags sent in the NTLM T1 message you should modify NtlmAuthPrintT1() function in src/client/NtlmAuth.cc file. Here is the relevant snippet: ... LONGQUARTET( NTLMFLAG_NEGOTIATE_OEM| NTLMFLAG_REQUEST_TARGET| NTLMFLAG_NEGOTIATE_NTLM_KEY| (useNTLM2Session ? NTLMFLAG_NEGOTIATE_NTLM2_KEY : 0) | NTLMFLAG_NEGOTIATE_ALWAYS_SIGN ), ... You can try adding NTLMFLAG_NEGOTIATE_128 flag to the list above. I do not know if Polygraph NTLM implementation actually supports 128bit encryption. So keep in mind that changing the above flags may work around the immediate issue with rejected T1 message, but break later on. Regards, Dmitry > WP2Q -> WebPolygraph 2nd reQuest > WF2Q -> Windows Firefox 2nd reQuest > WF2S -> Windows Firefox 2nd reSponse > WF3Q -> Windows Firefox 3rd reQuest > > Value (from highest to lowest bit) | WP2Q | WF2Q | WF2S | WF3Q > ---------------------------------------|------|------|------|------ > Negotiate 56 | - | x | x | x > Negotiate Key Exchange | - | - | - | - > Negotiate 128 | - | x | x | x > Negotiate 0x10000000 | - | - | - | - > Negotiate 0x08000000 | - | - | - | - > Negotiate 0x04000000 | - | - | - | - > Negotiate 0x02000000 | - | x | x | x > Negotiate 0x01000000 | - | - | - | - > Negotiate Target Info | - | - | x | x > Negotiate 0x00400000 | - | - | - | - > Negotiate 0x00200000 | - | - | - | - > Negotiate 0x00100000 | - | - | x | - > Negotiate NTLM2 key | x | x | - | x > Negotiate Challenge Non NT session Key | - | - | - | - > Negotiate Challenge Accept Response | - | - | - | - > Negotiate Challenge Init Reponse | - | - | - | - > Negotiate Always Sign | x | x | x | x > Negotiate This is Local Call | - | - | - | - > Negotiate Workstation Supplied | - | - | - | - > Negotiate Domain Supplied | - | - | - | - > Negotiate 0x00000800 | - | - | - | - > Negotiate 0x00000400 | - | - | - | - > Negotiate NTLM key | x | x | x | x > Negotiate Netware | - | - | - | - > Negotiate Lan Manager Key | - | - | - | - > Negotiate Datagramm Style | - | - | - | - > Negotiate Seal | - | - | - | - > Negotiate Sign | - | - | - | - > Request 0x00000008 | - | - | - | - > Request Target | x | x | x | x > Negotiate OEM | x | x | - | - > Negotiate UNICODE | - | x | x | x > > I hope my mail didn't grow too long. But I wanted it be as detailed as possible. Maybe it will help to find my mistake. > > Regards, > Gerrit > > > -----Urspr?ngliche Nachricht----- > Von: Dmitry Kurochkin [mailto:dmitry.kurochkin at measurement-factory.com] > Gesendet: Donnerstag, 15. Dezember 2011 21:12 > An: Hohl, Gerrit; users at web-polygraph.org > Betreff: Re: AW: Problems using robots with authentication > > Hi Gerrit. > > On Thu, 15 Dec 2011 17:36:34 +0100, "Hohl, Gerrit" wrote: > > Hello everyone, > > > > I read the article "Prerequisites" in the documentation: > > http://www.web-polygraph.org/docs/reference/models/ssl.html#Sect:2 > > > > Polygraph SSL support is based on the OpenSSL library. A recent version of the library is required to compile Polygraph. We have tested with OpenSSL versions 0.9.6g and 0.9.7b. The presence of OpenSSL is determined at ./configure time. Please check that ./configure actually found SSL library and headers if you install Polygraph and want SSL support: > > > > ... > > checking for CRYPTO_lock in -lcrypto... yes > > checking for SSL_connect in -lssl... yes > > checking for openssl/ssl.h... yes > > checking for openssl/err.h... yes > > checking for openssl/rand.h... yes > > > > Eh, I don't get these lines when I call the script. I assume that the script was modified, but the documentation not. > > > > The exact messages may have change, but they are still there: > > $ ./configure | grep -i ssl > checking for SSL_connect in -lssl... yes > checking openssl/ssl.h usability... yes > checking openssl/ssl.h presence... yes > checking for openssl/ssl.h... yes > checking openssl/err.h usability... yes > checking openssl/err.h presence... yes > checking for openssl/err.h... yes > checking openssl/rand.h usability... yes > checking openssl/rand.h presence... yes > checking for openssl/rand.h... yes > > Assertion at NtlmAuth.cc:798 means that you built without OpenSSL support. You should install it and rebuild. On Debian-based systems you should install libssl-dev package. > > The assertion is always a bug. Web Polygraph should print a proper error here. There is an open bug #878881 [1] for this issue. > > > One remark: I only want NTLM authentication and not HTTPS benchmarking. > > > > NTLM needs some crypto functions (MD5, at least). That is why OpenSSL is needed for it. > > Regards, > Dmitry > > [1] https://bugs.launchpad.net/polygraph/+bug/878881 > > > Regards, > > Gerrit > > > > -----Urspr?ngliche Nachricht----- > > Von: users-bounces at web-polygraph.org > > [mailto:users-bounces at web-polygraph.org] Im Auftrag von Hohl, Gerrit > > Gesendet: Donnerstag, 15. Dezember 2011 17:08 > > An: users at web-polygraph.org > > Betreff: Problems using robots with authentication > > > > Hello everyone, > > > > I have the same problem that ufa faced at the end of October: > > I use NTLM authentication and get the message > > > > NtlmAuth.cc:798: assertion failed: 'false' > > Aborted > > > > from the polygraph-client program. After the first time I received that message I included the following line in my PGL file: > > > > Robot robot = { > > [...] > > pconn_use_lmt = const(2147483647); > > [...] > > }; > > > > But I still get the same message. I read something in Dmitry response about OpenSSL. The OpenSSL package ('openssl') is installed on that Ubuntu machine I'm currently using. But I'm not sure if polygraph was compiled with or without SSL. Is there a way to test it? Or what do I have to do to make sure that polygraph compiles with SSL support? > > > > Regards, > > Gerrit > > _______________________________________________ > > Users mailing list > > Users at web-polygraph.org > > http://www.web-polygraph.org/mailman/listinfo/users > > _______________________________________________ > > Users mailing list > > Users at web-polygraph.org > > http://www.web-polygraph.org/mailman/listinfo/users From g.hohl at aurenz.de Thu Feb 16 11:18:32 2012 From: g.hohl at aurenz.de (Hohl, Gerrit) Date: Thu, 16 Feb 2012 12:18:32 +0100 Subject: AW: Scaling the number of users instead of the hit rate In-Reply-To: <3B7A7BDFFD286F49A19A5B35D26E08FB9B421D@srvmail2.aurenz.local> References: <3B7A7BDFFD286F49A19A5B35D26E08FB9B421D@srvmail2.aurenz.local> Message-ID: <3B7A7BDFFD286F49A19A5B35D26E08FB9B42D3@srvmail2.aurenz.local> Hello everyone, okay, maybe I wrote a little bit too early that my formula is working. The number of host and robots is correct. But something different is odd: If I run a test with the configuration beneath, I would expect a request rate of 500/sec in sum. But I get a request of about 2000/sec at each of my two polygraph-servers, means about 4000/sec in total. What did I wrong? Is the peak_req_rate per robot? Or does the max_host_load have any influence on that? I thought it is only the maximum rate on which the simulated servers should respond. Should I lower it also to 500/sec? Or does it apply for each simulated server, or maybe for each polygraph-server? Or maybe there is a bug and it ignores 'min' and still thinks that is "6/sec"? Bench theBench = { peak_req_rate = 500/sec; client_side = { max_host_load = 20000/sec; max_agent_load = 6/min; addr_space = ... ; hosts = ... ; }; server_side = { max_host_load = client_side.max_host_load; max_agent_load = undef(); addr_space = ... ; hosts = ... ; }; }; PolyMix4As asPolyMix4 = { agents_per_addr = 2; }; Regards Gerrit From rousskov at measurement-factory.com Thu Feb 16 16:55:20 2012 From: rousskov at measurement-factory.com (Alex Rousskov) Date: Thu, 16 Feb 2012 09:55:20 -0700 Subject: Scaling the number of users instead of the hit rate In-Reply-To: <3B7A7BDFFD286F49A19A5B35D26E08FB9B3FD5@srvmail2.aurenz.local> References: <3B7A7BDFFD286F49A19A5B35D26E08FB9B3FD5@srvmail2.aurenz.local> Message-ID: <4F3D34F8.904@measurement-factory.com> On 02/13/2012 05:24 AM, Hohl, Gerrit wrote: > But my problem now is that I should be able to scale the number of users > (robots) instead of the hit rate. Hi Gerrit, I assume that by "hit rate" you mean "request rate". And by "scale" you mean "change during a test". Yes, Polygraph can vary the number of active robots during a test. This is accomplished using "populus factors" in test Phase configurations: http://www.web-polygraph.org/docs/reference/pgl/types.html#type:docs/reference/pgl/types/Phase Please note that in simple configurations more robots means higher request rate if you vary the number of robots during the test (just like in most real-world situations more users means more incoming requests from those users). You _can_ vary the number of robots while keeping the request rate constant, but that requires more complex Phase configurations. > Our product is more sensitive to this It may be important to define what _traffic properties_ the product is sensitive to: total request rate, total authentication rate, number of active concurrent connections, number of idle persistent connections, number of unique IP source addresses, number of unique server addresses, etc. Most of these traffic properties can be modeled with just one robot or with 10000 robots. It is tempting to say "I want to see if my proxy can serve a population of one million potential users" and then try to create one million robots representing those potential users, but you will run into hardware, OS, and Polygraph limitations with that approach because of the amount of waste it implies. A better strategy is to focus on the important traffic properties (number of users is not a traffic property!) and use the minimum/simplest configuration that can model those properties. > than the hit rate as our focus is on authentification. We don't do any > caching or things like that. So is there a possibility to scale users? > Or maybe I should ask the question in a different way: Is there a way I > can calculate the number of expected robots from other values like > Bench.client_side.peak_req_rate, Bench.client_side.max_host_load or > Bench.client_side.max_agent_load? If your configuration is reasonable, and you are using Bench and address scheme objects to configure your test, then the number of robots will be approximately bench.peak_req_rate / bench.client_side.max_agent_load Depending on the address scheme, Polygraph will allocate robot (and server) IP addresses a little differently, but all address schemes are designed to more-or-less spread the load among all your hosts. When it is not possible, the configuration is usually not reasonable. It is difficult to define "reasonable" precisely, but you can think of it as "not self-contradictory" or "not allowing unrealistic interpretations". > peak_req_rate = 500/sec; > > client_side = { > max_host_load = 20000/sec; > max_agent_load = 6/min; > addr_space = ... ; > hosts = ... ; > }; > For example, the above configuration tells me that you want a single client-side host to do up to 20000 requests per second while each robot on that host will do up to 6 requests per minute. This means you want Polygraph to create up to 200000 robots on a single host. That may already exceed hardware and/or socket descriptor/port limits. And then you tell Polygraph to use 2 robots per IP address: > PolyMix4As asPolyMix4 = { > agents_per_addr = 2; > }; which would result in up to 100000 IP addresses per host. This is probably not reasonable for today's operating systems. Now, your peak_req_rate is just 500/sec so, in the actual test, there should be fewer robots (about 5000) and fewer IP addresses (about 2500), but Polygraph should refuse to use more than one client-side host if you have several configured and that could cause other problems. I recommend the following as a starting point: 0. Read the following to understand how Polygraph interprets your configuration when it comes to allocation of IP addresses and agents: http://www.web-polygraph.org/docs/userman/address.html 1a. Define theBench object so that all values are true and reasonable. 1b. Do not forget to use() theBench object. 2. When calculating robot and server addresses, use SpreadAs address scheme. It has a more straightforward address allocation algorithm: http://www.web-polygraph.org/docs/reference/pgl/types.html#type:docs/reference/pgl/types/SpreadAs 3a. If you want to know the details about PolyMix-4 address allocation scheme, please see: http://www.web-polygraph.org/docs/workloads/polymix-4/ http://www.measurement-factory.com/docs/PolyMix-4/#Sect:3 3b. You may also use the pmix3-ips.pl script distributed with Polygraph to see what PolyMix-3 or PolyMix-4 IPs will be needed for a given request rate. If you get strange results, attach your entire workload and console logs (compress if needed) because the bug could be in how you use Bench and address scheme objects or in host addresses, and that may not be visible in a PGL snippet. HTH, Alex. From g.hohl at aurenz.de Thu Feb 16 16:34:47 2012 From: g.hohl at aurenz.de (Hohl, Gerrit) Date: Thu, 16 Feb 2012 17:34:47 +0100 Subject: AW: Scaling the number of users instead of the hit rate In-Reply-To: <3B7A7BDFFD286F49A19A5B35D26E08FB9B42D3@srvmail2.aurenz.local> References: <3B7A7BDFFD286F49A19A5B35D26E08FB9B421D@srvmail2.aurenz.local> <3B7A7BDFFD286F49A19A5B35D26E08FB9B42D3@srvmail2.aurenz.local> Message-ID: <3B7A7BDFFD286F49A19A5B35D26E08FB9B4349@srvmail2.aurenz.local> Hello everyone, is it possible that I also have to set the req_rate of the robots? Robot robot = { [...] peak_rate = 6/min; [...] }; If I do that and run the test the overall request rate is far more near to the values I would expect. Regards Gerrit -----Urspr?ngliche Nachricht----- Von: users-bounces at web-polygraph.org [mailto:users-bounces at web-polygraph.org] Im Auftrag von Hohl, Gerrit Gesendet: Donnerstag, 16. Februar 2012 12:19 An: users at web-polygraph.org Betreff: AW: Scaling the number of users instead of the hit rate Hello everyone, okay, maybe I wrote a little bit too early that my formula is working. The number of host and robots is correct. But something different is odd: If I run a test with the configuration beneath, I would expect a request rate of 500/sec in sum. But I get a request of about 2000/sec at each of my two polygraph-servers, means about 4000/sec in total. What did I wrong? Is the peak_req_rate per robot? Or does the max_host_load have any influence on that? I thought it is only the maximum rate on which the simulated servers should respond. Should I lower it also to 500/sec? Or does it apply for each simulated server, or maybe for each polygraph-server? Or maybe there is a bug and it ignores 'min' and still thinks that is "6/sec"? Bench theBench = { peak_req_rate = 500/sec; client_side = { max_host_load = 20000/sec; max_agent_load = 6/min; addr_space = ... ; hosts = ... ; }; server_side = { max_host_load = client_side.max_host_load; max_agent_load = undef(); addr_space = ... ; hosts = ... ; }; }; PolyMix4As asPolyMix4 = { agents_per_addr = 2; }; Regards Gerrit _______________________________________________ Users mailing list Users at web-polygraph.org http://www.web-polygraph.org/mailman/listinfo/users From dmitry.kurochkin at measurement-factory.com Thu Feb 16 20:02:26 2012 From: dmitry.kurochkin at measurement-factory.com (Dmitry Kurochkin) Date: Fri, 17 Feb 2012 00:02:26 +0400 Subject: Scaling the number of users instead of the hit rate In-Reply-To: <4F3D34F8.904@measurement-factory.com> References: <3B7A7BDFFD286F49A19A5B35D26E08FB9B3FD5@srvmail2.aurenz.local> <4F3D34F8.904@measurement-factory.com> Message-ID: <87obsyfsx9.fsf@gmail.com> Hi Gerrit. On Thu, 16 Feb 2012 09:55:20 -0700, Alex Rousskov wrote: > On 02/13/2012 05:24 AM, Hohl, Gerrit wrote: > > > But my problem now is that I should be able to scale the number of users > > (robots) instead of the hit rate. > > Hi Gerrit, > > I assume that by "hit rate" you mean "request rate". And by "scale" > you mean "change during a test". > > Yes, Polygraph can vary the number of active robots during a test. This > is accomplished using "populus factors" in test Phase configurations: > http://www.web-polygraph.org/docs/reference/pgl/types.html#type:docs/reference/pgl/types/Phase > > Please note that in simple configurations more robots means higher > request rate if you vary the number of robots during the test (just like > in most real-world situations more users means more incoming requests > from those users). You _can_ vary the number of robots while keeping the > request rate constant, but that requires more complex Phase configurations. > > > > Our product is more sensitive to this > > > It may be important to define what _traffic properties_ the product is > sensitive to: total request rate, total authentication rate, number of > active concurrent connections, number of idle persistent connections, > number of unique IP source addresses, number of unique server addresses, > etc. Most of these traffic properties can be modeled with just one robot > or with 10000 robots. > > It is tempting to say "I want to see if my proxy can serve a population > of one million potential users" and then try to create one million > robots representing those potential users, but you will run into > hardware, OS, and Polygraph limitations with that approach because of > the amount of waste it implies. A better strategy is to focus on the > important traffic properties (number of users is not a traffic > property!) and use the minimum/simplest configuration that can model > those properties. > > > > than the hit rate as our focus is on authentification. We don't do any > > caching or things like that. So is there a possibility to scale users? > > Or maybe I should ask the question in a different way: Is there a way I > > can calculate the number of expected robots from other values like > > Bench.client_side.peak_req_rate, Bench.client_side.max_host_load or > > Bench.client_side.max_agent_load? > > > If your configuration is reasonable, and you are using Bench and address > scheme objects to configure your test, then the number of robots will be > approximately > > bench.peak_req_rate / bench.client_side.max_agent_load > > Depending on the address scheme, Polygraph will allocate robot (and > server) IP addresses a little differently, but all address schemes are > designed to more-or-less spread the load among all your hosts. When it > is not possible, the configuration is usually not reasonable. > > It is difficult to define "reasonable" precisely, but you can think of > it as "not self-contradictory" or "not allowing unrealistic > interpretations". > > > > peak_req_rate = 500/sec; > > > > client_side = { > > max_host_load = 20000/sec; > > max_agent_load = 6/min; > > addr_space = ... ; > > hosts = ... ; > > }; > > > > > For example, the above configuration tells me that you want a single > client-side host to do up to 20000 requests per second while each robot > on that host will do up to 6 requests per minute. This means you want > Polygraph to create up to 200000 robots on a single host. That may > already exceed hardware and/or socket descriptor/port limits. And then > you tell Polygraph to use 2 robots per IP address: > > > PolyMix4As asPolyMix4 = { > > agents_per_addr = 2; > > }; > > > which would result in up to 100000 IP addresses per host. This is > probably not reasonable for today's operating systems. > > Now, your peak_req_rate is just 500/sec so, in the actual test, there > should be fewer robots (about 5000) and fewer IP addresses (about 2500), > but Polygraph should refuse to use more than one client-side host if you > have several configured and that could cause other problems. > > > I recommend the following as a starting point: > > 0. Read the following to understand how Polygraph interprets your > configuration when it comes to allocation of IP addresses and agents: > http://www.web-polygraph.org/docs/userman/address.html > > 1a. Define theBench object so that all values are true and reasonable. > 1b. Do not forget to use() theBench object. > > 2. When calculating robot and server addresses, use SpreadAs address > scheme. It has a more straightforward address allocation algorithm: > http://www.web-polygraph.org/docs/reference/pgl/types.html#type:docs/reference/pgl/types/SpreadAs > > 3a. If you want to know the details about PolyMix-4 address allocation > scheme, please see: > http://www.web-polygraph.org/docs/workloads/polymix-4/ > http://www.measurement-factory.com/docs/PolyMix-4/#Sect:3 > > 3b. You may also use the pmix3-ips.pl script distributed with Polygraph > to see what PolyMix-3 or PolyMix-4 IPs will be needed for a given > request rate. > I would recommend using polygraph-pgl2ips(1) tool instead of pmix2-ips.pl, pmix3-ips.pl and webaxe4-ips.pl scripts. Pgl2ips is a more recent tool which would calculate per-host and per-agent IP addresses based on a PGL workload. Pgl2ips uses the same code as the client and server, so it supports any valid Bench configuration and uses the right formula for address calculations. I hope the pgl2ips would cover all your needs wrt Polygraph address calculation. If it does not, we should improve it. We definitely should do better than forcing users to dig into source code and reimplement non-trivial calculations in Java! Regards, Dmitry > > If you get strange results, attach your entire workload and console logs > (compress if needed) because the bug could be in how you use Bench and > address scheme objects or in host addresses, and that may not be visible > in a PGL snippet. > > > HTH, > > Alex. > _______________________________________________ > Users mailing list > Users at web-polygraph.org > http://www.web-polygraph.org/mailman/listinfo/users