HN Reader

The author disables SMT (hyperthreading) like this:

    disabled_cpus=(1 3 5 7 9 11 13 15)
    for cpu_no in $disabled_cpus ; do
      echo 0 | sudo tee /sys/devices/system/cpu/cpu$cpu_no/online
    done

But there is an easier way on Linux that doesn't require parsing /sys/devices/system/cpu/cpu*/topology/thread_siblings_list:

    sudo tee /sys/devices/system/cpu/smt/control <<< off

These workarounds might be good enough to detect ~1-5% changes from a baseline of a native, pre-compiled application. However this won't be sufficient for many dynamic, JIT-compiled languages which will usually still have large amounts of inter-run variance due to timing-sensitive compilation choices of the runtime. A statically significant ~10% change can be hard to detect in these circumstances from a single run.

In my experience multi-run benchmarking frameworks which use non-parametric statistics should be the default tool of choice unless you know the particular benchmark is exceptionally well behaved.

Windows has been driving me crackers the last few days trying to benchmark hard to measure optimisations, and tend to end up on long runs, looking for the minimum time.. Usually closing chrome results in an immediate ~10% performance boost even minimised.. I’d love to see an option for a Developer to lock off some cores totally, nothing runs on them unless its in an approved list.. At least then I can profile on those cores and get a reasonable result..