This class helps with distributing threads to the different CPU cores. More...

#include <numa.h>

Public Member Functions
	ThreadDistributor (long num_threads, std::shared_ptr< spdlog::logger >={})

void	pin_this_thread (thread_id_t thread_id)

Private Attributes
std::vector< cpu_id_t >	m_TargetCPUs
	List of CPUs to which the threads will be assigned. More...

std::shared_ptr< spdlog::logger >	m_Logger

Detailed Description

This class helps with distributing threads to the different CPU cores.

For best performance, we want to pin the different threads to different CPU cores.

This works as follows. The constructor of ThreadDistributor is given the number of threads it shall distribute, and produces a list of CPU cores to which threads will be assigned. Subsequent calls to pin_this_thread() will then look up the core in this list and pin the current thread to it. We also set the memory strategy to allocate on the local node.

The way the threads are distributed is as follows: We choose NUMA nodes in a round-robin fashion, and assign cores in the node in increasing order. For example for two nodes with four cores each, the allocation order would be 0:0, 1:4, 0:1, 1:5, 0:2, 1:6, .... This strategy may be suboptimal if the number of threads is of the same order as the number of NUMA nodes (e.g. running 4 threads on mahti).

TODO enable spread-out allocation within a NUMA node TODO figure out balancing of HT

Definition at line 118 of file numa.h.

Constructor & Destructor Documentation

◆ ThreadDistributor()

ThreadDistributor::ThreadDistributor	(	long	num_threads,
		std::shared_ptr< spdlog::logger >	logger = `{}`
	)

Definition at line 240 of file numa.cpp.

References anonymous_namespace{numa.cpp}::get_available_nodes(), m_Logger, m_TargetCPUs, and THROW_EXCEPTION.

Member Function Documentation

◆ pin_this_thread()

void ThreadDistributor::pin_this_thread ( thread_id_t thread_id )

Definition at line 265 of file numa.cpp.

References anonymous_namespace{numa.cpp}::current_numa_node(), m_Logger, m_TargetCPUs, anonymous_namespace{numa.cpp}::set_thread_affinity(), and dismec::opaque_int_type< Tag, T >::to_index().

Referenced by dismec::parallel::ParallelRunner::run().

Member Data Documentation

◆ m_Logger

std::shared_ptr<spdlog::logger> dismec::parallel::ThreadDistributor::m_Logger

private

Definition at line 125 of file numa.h.

Referenced by pin_this_thread(), and ThreadDistributor().

◆ m_TargetCPUs

std::vector<cpu_id_t> dismec::parallel::ThreadDistributor::m_TargetCPUs

private

List of CPUs to which the threads will be assigned.

Definition at line 123 of file numa.h.