Bench SMP mode

[William Law]

Hi All,
Some info, bug and progress update for you.

> >
> > Here is a better (but untested) version of your Bench, using 4 cores per
> > drone, numbered 2 through 5.
> >
> >   Bench sslBench = {
> >     client_side = {
> >       max_host_load = 100/sec;
> >       max_agent_load = 1/sec;
> >       addr_space = [ 'lo::198.18.24-27.10-249/32' ];
> >       hosts =  [ '198.18.24.2' ** 4 ];
> >       cpu_cores = [ [2], [3], [4], [5] ];
> >     };
> >     server_side = {
> >       max_host_load = client_side.max_host_load;
> >       max_agent_load = client_side.max_agent_load;
> >       addr_space = [ 'lo::198.18.28-29.10-249:443/32' ];
> >       hosts = [ '198.18.28.2' ** 4 ];
> >       cpu_cores = client_side.cpu_cores;
> >     };
> >   };
> >
> > Avoid sharing physical cores among virtual cores: Two busy virtual cores
> > can do _less_ than one real core they share.
>

Here's the Bench config I've managed to get working.  The strange core
progression is to utilise the primary logical core ID of each physical
core - didn't want to go altering behaviours of other previous tests too
much by disabling hyperthreading as yet.

Bench sslBench = {
  client_side = {
    max_host_load = 10000/sec;
    max_agent_load = 50/sec;
    addr_space = [ 'em1::198.18.24-27.10-249/22' ];
    hosts =  [ '198.18.24.2' ** 16 ] ;
    cpu_cores = [
[0,1],[2,3],[8,9],[10,11],[16,17],[18,19],[24,25],[26,27],[32,33],[34,35],[40,41],[42,43],[48,49],[50,51],[56,57],[58,59]
 ];
    // cpu_cores = [
[0,1],[2,3],[8,9],[10,11],[16,17],[18,19],[24,25],[26,27],[32,33],[34,35],[40,41],[42,43],[48,49],[50,51],[56,57],[58,59],[64,65],[66,67],[72,73],[74,75]
 ];
   };
  server_side = {
    max_host_load = client_side.max_host_load;
    max_agent_load = client_side.max_agent_load;
    addr_space = [ 'em1::198.18.28-29.10-249:443/23' ];
    hosts = [ '198.18.28.2' ** 16 ] ;
    cpu_cores = client_side.cpu_cores;
  };
};

I'm not getting the same latency spikes that I was before, but I have run
into a new problem - I can only configure a maximum of 16 core sets before
polygraph-server segfaults or bails with the following:

*** glibc detected *** /usr/local/bin/polygraph-server: double free or
corruption (fasttop): 0x000000000285ac00 ***
======= Backtrace: =========
/lib64/libc.so.6[0x31d6a75e5e]
/lib64/libc.so.6[0x31d6a78cad]
/usr/local/bin/polygraph-server[0x40cbe1]
/usr/local/bin/polygraph-server[0x41ad92]
/usr/local/bin/polygraph-server[0x41baec]
/usr/local/bin/polygraph-server[0x4077ba]
/lib64/libc.so.6(__libc_start_main+0x100)[0x31d6a1ed20]
/usr/local/bin/polygraph-server[0x407409]

To get this result, I enter in more than 16 cores, and the associated number
of entries in cpu_cores.

If you can let me know what other debug information you want from me and I
can DM the details to you.

This is a shame as the test starts to max out all 16 server processes at
about 50% client test load & 5.4Gbit/s of traffic.  Although I did leave it
running and saw it hit 8Gbit/s with 30+ second latencies as the phase
started to back down the load amount.  Now the interesting this is that if
the processes are running with 2 cores allocated then they should have been
showing 200% cpu usage per process (2 cores consumed), not just 100% (1
core).  Are the processes not multithreaded in that way?

> script. We should add PGL function(s) to generate typical CPU affinity
> map(s) for a given number of cores. Quality patches or sponsorships
> welcome.

Not much of a C coder myself, I could probably cobble something together but
it may not necessarily be quality :D  Might see if we can help another way.

>
> >
> > HTH,
> >
> > Alex.

Kind Regards,

William Law