Improve thread pinning:

+ Now pins threads intelligently on SMT systems by balancing threads among
      cores.
    + performs fewer migrations when pinning threads (If a thread's current
      CPU is a valid CPU for pinning, then it is not migrated).
    + Handle cases where the user requests more threads than available CPUs.
    + Handle cases where the process is restricted to a subset of CPUs by its
      environment.  (for instance, Linux cpuset(7))

This is largely made possible by the hwloc library
http://www.open-mpi.org/projects/hwloc/  However, hwloc is not required and the
runtime system will fall back to sched_setaffinity(), it will simply be less
intelligent WRT SMT.

runtime/mercury_context.h:
runtime/mercury_context.c:
    Do thread pinning either via hwloc or sched_setaffinity.  Previously only
    sched_setaffinity was used.

    Update thread-pinning algorithm, this:

    Include the general thread pinning code only if MR_HAVE_THREAD_PINNING is
    defined.

    Use a combination of sysconf and sched_getaffinity to detect the number of
    processors when hwloc isn't available.  This makes the runtime compatible
    with Linux cpuset(7) when hwloc isn't available.

configure.in:
Mmake.common.in:
    Detect presence of the hwloc library.

configure.in:
    Detect sched_getaffinity()

aclocal.m4:
acinclude.m4:
    Move aclocal.m4 to acinclude.m4, the aclocal program will build aclocal.m4
    and retrieve macros from the system and the contents of acinclude.m4.

Mmakefile:
    Create a make target for aclocal.m4.

runtime/Mmakefile:
    Link the runtime with libhwloc in low-level C parallel grades.

    Include CFLAGS for libhwloc.

scripts/ml.in:
    Link programs and libraries with libhwloc in low-level C parallel grades.

runtime/mercury_conf.h.in:
    Define MR_HAVE_HWLOC when it is available.

    Define MR_HAVE_SCHED_GETAFFINITY when it is available.

runtime/mercury_conf_param.h:
    Define MR_HAVE_THREAD_PINNING if either hwloc or [sched_setaffinity and
    sched_getaffinity] are available.

runtime/mercury_thread.c:
runtime/mercury_wrapper.c:
    Only call MR_pin_thread and MR_pin_primordial_thread if
    MR_HAVE_THREAD_PINNING is defined.

runtime/mercury_thread.h:
runtime/mercury_context.h:
    Move the declaration of MR_pin_primordial_thread to mercury_context.h from
    mercury_thead.h since it's definition is in mercury_context.c.

    Require MR_HAVE_THREAD_PINNING for the declaration of
    MR_pin_primordial_thread.

runtime/mercury_wrapper.c:
    Conform to changes in mercury_context.h

INSTALL_CVS:
tools/test_mercury
    Run aclocal at the right times while testing Mercury.
This commit is contained in:
Paul Bone
2011-10-13 02:42:21 +00:00
parent 1d0cd8d04f
commit a071eaba53
15 changed files with 454 additions and 129 deletions

View File

@@ -370,6 +370,9 @@ cleanint:
#-----------------------------------------------------------------------------#
aclocal.m4: configure.in acinclude.m4
aclocal
configure: configure.in aclocal.m4
autoconf