Bench SMP mode

[Alex Rousskov]

On 08/06/2018 11:55 PM, William Law wrote:

> Here's the Bench config I've managed to get working.  The strange core
> progression is to utilise the primary logical core ID of each physical
> core - didn't want to go altering behaviours of other previous tests too
> much by disabling hyperthreading as yet.

I would list just one virtual core (e.g. odd ones) per worker. The
workload will look simpler and you might save a few CPU cycles by not
moving processes across virtual cores. No need to disable hyperthreading
(although doing that may improve overall performance a bit if you have
enough physical cores).

Another useful trick for high-load tests is to confine NIC interrupts to
one (or, if necessary, more) dedicated CPU cores as well. Just like
Polygraph, the interrupt processing code in the kernel probably benefits
from CPU affinity and, more importantly, confining it prevents
CPU-grabbing conflicts between it and Polygraph workers.

In summary, you want to give each CPU-hungry program (worker, kernel,
etc.) a dedicated CPU core (to the extent possible).


> I'm not getting the same latency spikes that I was before, but I have run
> into a new problem - I can only configure a maximum of 16 core sets before
> polygraph-server segfaults

We will try to find the time to reproduce and fix.


> If you can let me know what other debug information you want from me and I
> can DM the details to you.

A stack trace from the crash ("bt full" in gdb) would be a good start if
we cannot reproduce this.


> if the processes are running with 2 cores allocated then they should have been
> showing 200% cpu usage per process (2 cores consumed), not just 100% (1
> core).  Are the processes not multithreaded in that way?

You have one worker process per physical core. One process cannot
consume more than 100% of anything. Workers have no threads (for this
discussion, you can view each worker as a thread if you wish). And two
virtual cores are a red herring -- in a context of a single busy
process, they only add overheads.


Cheers,

Alex.