configure.ac:
Drop support for OSF/1.
Delete more IRIX stuff.
configure.ac:
runtime/mercury_conf.h.in:
runtime/mercury_thread.h:
Delete workarounds required when using the Digital Unix pthreads
library.
Mmake.common.in:
runtime/mercury_wrapper.c:
Delete stuff for IRIX.
README.OSF1:
Delete this file.
.README.in:
Delete the pointers to the readme files for IRIX and OSF/1.
NEWS:
Announce the dropping of support for OSF/1.
The pthreads-win32 has worked with 64-bit compilers since version 2.9.
This diff adds the initial support for the .par grades with MinGW64 and
pthreads-win32.
configure.ac:
Set C compiler and linker flags for x86_64-w64-mingw32.
(This is provisional; on my system the library appears
to have a different name, but I'm not sure how normal
this is.)
runtime/mercury_thread.h:
Adjust the definition of the MR_SELF_THREAD_ID macro
so that the integer it expands to is at least as big
as a pointer. (Needed for pthreads-win32 since thread
ids are pointer values, not integers with that.)
runtime/mercury_thread.c:
Avoid warnings in some debugging code.
+ Now pins threads intelligently on SMT systems by balancing threads among
cores.
+ performs fewer migrations when pinning threads (If a thread's current
CPU is a valid CPU for pinning, then it is not migrated).
+ Handle cases where the user requests more threads than available CPUs.
+ Handle cases where the process is restricted to a subset of CPUs by its
environment. (for instance, Linux cpuset(7))
This is largely made possible by the hwloc library
http://www.open-mpi.org/projects/hwloc/ However, hwloc is not required and the
runtime system will fall back to sched_setaffinity(), it will simply be less
intelligent WRT SMT.
runtime/mercury_context.h:
runtime/mercury_context.c:
Do thread pinning either via hwloc or sched_setaffinity. Previously only
sched_setaffinity was used.
Update thread-pinning algorithm, this:
Include the general thread pinning code only if MR_HAVE_THREAD_PINNING is
defined.
Use a combination of sysconf and sched_getaffinity to detect the number of
processors when hwloc isn't available. This makes the runtime compatible
with Linux cpuset(7) when hwloc isn't available.
configure.in:
Mmake.common.in:
Detect presence of the hwloc library.
configure.in:
Detect sched_getaffinity()
aclocal.m4:
acinclude.m4:
Move aclocal.m4 to acinclude.m4, the aclocal program will build aclocal.m4
and retrieve macros from the system and the contents of acinclude.m4.
Mmakefile:
Create a make target for aclocal.m4.
runtime/Mmakefile:
Link the runtime with libhwloc in low-level C parallel grades.
Include CFLAGS for libhwloc.
scripts/ml.in:
Link programs and libraries with libhwloc in low-level C parallel grades.
runtime/mercury_conf.h.in:
Define MR_HAVE_HWLOC when it is available.
Define MR_HAVE_SCHED_GETAFFINITY when it is available.
runtime/mercury_conf_param.h:
Define MR_HAVE_THREAD_PINNING if either hwloc or [sched_setaffinity and
sched_getaffinity] are available.
runtime/mercury_thread.c:
runtime/mercury_wrapper.c:
Only call MR_pin_thread and MR_pin_primordial_thread if
MR_HAVE_THREAD_PINNING is defined.
runtime/mercury_thread.h:
runtime/mercury_context.h:
Move the declaration of MR_pin_primordial_thread to mercury_context.h from
mercury_thead.h since it's definition is in mercury_context.c.
Require MR_HAVE_THREAD_PINNING for the declaration of
MR_pin_primordial_thread.
runtime/mercury_wrapper.c:
Conform to changes in mercury_context.h
INSTALL_CVS:
tools/test_mercury
Run aclocal at the right times while testing Mercury.
This is a problem for the following reasons:
The work stealing code must take a lock to access the resizeable array of
work stealing dequeues. This adds global contention that can be avoided if
this array has a fixed size.
If a context is blocked on a future then that engine cannot execute the
sparks from that context, instead it tries to find global work, this is
more expensive than necessary.
If there are a few dozen contexts then there may be just as many work
stealing queues to take work from, the density of these queues will be
higher if they are fewer. Therefore work stealing will be more successful
on average.
This change associates spark deques with Mercury Engines rather than Contexts
to avoid these problems.
This has invalidated some invariants that allowed the runtime system to make
some worth-while optimisations. These optimisations have been maintained.
Mercury's idle loop has been reimplemented to allow for this. This
re-implementation has allowed for a number of other improvements:
Polling was used to check for new global sparks. This has been removed and
each engine now sleeps using it's own semaphore.
Checks for work can be done in different orders depending on how an engine
joins the idle loop.
When global work becomes available a particular engine can be woken up
rather than any arbitrary engine. We take advantage of this when making
contexts runnable, we try to schedule them on the engine that last executed
them.
When an engine is woken up it can be instructed with what it should do upon
waking up.
When a engine looks for a context to run, it will try to pick a context
that was last executed on it. This may avoid cache misses when the context
begins to run.
In the future we should consider:
Experiment with telling engines which context to run.
Improve the selection of which engine work should be scheduled on to be
hardware and memory-hierarchy aware.
Things that need doing next (probably next week):
./configure should check for POSIX semaphore support.
Profiling times have been broken by this change, they will need fixing.
The threadscope event long now breaks an invariants that the threadscope
graphical tool requires.
Semaphores are setup but never released, this is not a big problem but the
manual page says that some implementations may leak resources.
runtime/mercury_context.h:
runtime/mercury_context.c:
Remove the spark deque field from the MR_Context structure.
Export the new array of spark deques so that other modules may fill in
elements as engines are setup.
Modify the resume_owner_thread field of the MR_Context structure, this was
used to ensure that a context returning through C code would be resumed on
the engine with the correct C stack and depth. This field is now an engine
id and has been renamed to resume_owner_engine, it is advisory unless
resume_engine_required is also set. This way it is used to advise which
engine most recently executed this context and therefore may have a warm
cache.
Remove code that dynamically resized the array of spark deques. Including
the lock that protected against updating this array while it was being read
from other thread.
Introduce code that initialises the statically sized array of spark deques.
Reimplement the idle loop. This replaces MR_runnext and MR_do_runnext with
MR_idle and MR_do_idle respectively. There are also two new entry points
into the idle loop. Which one to use depends on the state of the engine.
Introduce new mechanisms for waking a particular engine. For example the
engine that last executed a context that is now runnable.
Change the algorithm for selecting which context to run, try to select
contexts that where last used on the current engine to avoid cache misses.
Use an engine's victim counter rather than a global victim counter when
trying to steal work.
Introduce some conditionally-compiled code that can be used to profile how
quickly new contexts can be created.
Rename MR_init_thread_stuff and MR_finalize_thread_stuff. The term thread
has been replaced with context since they're in mercury_context.c. This
allows the creation of a new function MR_init_thread_stuff() in
mercury_thread.c I also found the mismatch between the function names and
file name confusing. Move some of the code from MR_init_context_stuff to
the new MR_init_thread_stuff function where it belongs.
Refactor the thread pinning code so that even when thread pinning is
disabled it can be used to allocate each thread to a CPU but not actually
pin them.
Fix some whitespace errors.
runtime/mercury_thread.h:
runtime/mercury_thread.c:
In MR_init_engine():
Allocate an engine id for each engine.
A number of arrays had one slot per engine and where setup using a
lock. Now engine ids are used to index each array and setup is done
without a lock, each engine simply sets up its own slot.
Setup the new per-engine work stealing deques.
The MR_all_engine_bases array has been moved to this file.
Implement a new MR_init_thread_stuff function which initialises some global
variables and locks. Some of MR_init_thread_stuff has been moved from
mercury_context.c
Pin threads as part of MR_init_thread, excluding the primordial thread
which must be pinned before threadscope is initialised.
Add functions for debugging the use of semaphores.
Add corresponding macros that can be used to redirect semaphore calls to
debugging functions as above.
Improved thread debugging code, ensured that stderr is flushed after every
use, and that logging is done after calls return as well as before they're
called.
Conform to changes in mercury_context.h
runtime/mercury_engine.h:
runtime/mercury_engine.c:
Add spark deque and victim counter fields to the MercuryEngine structure.
Make the MR_eng_id field of the MercuryEngine structure available in all
thread safe grades, formerly it was used in only threadscope grades.
Move the MR_all_engine_bases variable to mercury_thread.[ch]
Put a reference to the engine's spark queue into the global array. This is
done here, so that it is after thread pinning because the original plan was
to have this array sorted by CPU rather then engine - we may yet do this in
the future.
Initialise an engine's spark deque when an engine is initialised.
Setup the engine specific threadscope data in mercury_thread.c
Conform to changes in mercury_context.h
runtime/mercury_wrapper.c:
The engine base array is no longer setup here, that code has been moved to
mercury_thread.c
Conform to changes in mercury_context.h and mercury_thread.h
runtime/mercury_wsdeque.h:
runtime/mercury_wsdeque.c:
The original implementation allocated an array for a spark queue only if
one wasn't already allocated, which could happen when a context was reused.
Now that spark queues are associated with engines arrays are always
allocated.
Replaced two macros with a single macro since there's no-longer a
distinction between global and local work queues, all work queues are
local.
runtime/mercury_wsdeque.c:
runtime/mercury_wsdeque.h:
Remove the --worksteal-max-attempts and --worksteal-sleep-msecs options as
they are no-longer used.
runtime/mercury_threadscope.h:
runtime/mercury_threadscope.c:
The MR_EngineId type has been moved to mercury_types.h
Engine IDs are no-longer allocated here, this is done in mercury_thread.c
The run spark and steal spark messages now write 0xFFFFFFFF for the context
id if there is no current context. Previously this would dereference a
null pointer.
runtime/mercury_memory_zones.c:
When checking for an existing memory zone check the free_zones_list
variable before taking a lock. This can prevent taking the lock in cases
where there are no free zones.
Introduce some conditionally-compiled code that can be used to profile how
quickly new contexts can be created.
runtime/mercury_bootstrap.h:
Remove macros that no-longer resolve to functions due to changes in the
runtime system.
runtime/mercury_types.h:
Move the MR_EngineId type from mercury_threadscope.h to mercury_types.h
runtime/mercury_grade.h:
Introduce a parallel grade version number, this change brakes binary
compatibility with existing parallel code.
runtime/mercury_backjump.c:
runtime/mercury_par_builtin.c:
runtime/mercury_mm_own_stacks.c:
library/stm_builtin.m:
library/thread.m:
library/thread.semaphore.m:
Conform to changes in mercury_context.h.
library/io.m:
Make this module compatible with MR_debug_threads.
doc/user_guide.texi
Remove the documentation for the --worksteal-max-attempts and
--worksteal-sleep-msecs options. The documentation was already commented
out.
Branches: main, 10.04
Support the use of the pthreads-win32 library on MinGW systems.
(This is based on the patch provided by Sergey Khorev.)
The main change is to remove the assumption in the runtime code
that POSIX thread handles are integers; in the pthreads-win32 library
they are not.
With this change the hlc.par.gc grade will work on Windows / MinGW.
(The low-level C parallel grades will require further work.)
configure.in:
Configure the Boehm GC to use pthreads-win32 if that is being used
to provide threads for the runtime on MinGW.
Delete the --with-pthreads-win32 option; it is no longer needed.
Add a new option --with-gc-pthreads-win32 that forces the Boehm GC
to use pthreads-win32. This is the default for MinGW anyway, the option
is intended for use by developers using pthreads-win32 in other ways,
e.g. with MSVC.
runtime/mercury_thread.h:
Add a new macro / function (depending on the implementation of pthreads)
that returns the "null" thread.
Add a new macro MR_thread_equal() that tests two thread handles
for equality.
runtime/mercury_thread.c:
Provide implementations of MR_null_thread().
Add a macro, for use within this module, that returns the id
of a thread in a form suitable for use in debugging messages.
runtime/mercury_engine.c:
runtime/mercury_context.c:
runtime/mercury_wrapper.c:
runtime/mercury_thread.c:
Use MR_null_thread() instead of NULL or 0.
Use MR_thread_equal() instead of directly comparing thread handles.
The hlc.gc.par grade was broken after committing my fix for the stack segment
parallel grades. The problem is that hlc grades use Mercury engines but don't
follow the same code paths as the low-level C grades. This means that they
expect that the MR_all_engine_bases array isn't allocated by the time the
engine structures are being created.
This change set makes the MR_all_engine_bases array only available in low-level
C parallel grades making problems involving this array impossible in high level
C grades.
runtime/mercury_engine.h:
runtime/mercury_engine.c:
Only make MR_all_engine_bases available in thread safe low-level C grades.
runtime/mercury_thread.h:
runtime/mercury_thread.c:
Only make MR_init_engine_array_lock available in thread safe low-level C
grades.
Only try to populate MR_all_engine_bases in thread safe low-level C grades.
runtime/mercury_context.c:
Only initialise MR_init_engine_array_lock in thread safe low-level C
grades.
MR_destroy_context will cache contexts in case the runtime needs a context in
the near future. Because the context no-longer represents an on-going
computation MR_destroy_context did not copy context-data out of the
MercuryEngine and real machine registers into the context before caching it.
However in a stack segments grade a one of these values is the current stack
segment, this means that a context may be cached and later re-used which refers
to a stack segment that another context is now using, the cached context will
then trash the other context's stack.
The solution is to save the context before caching it.
This change also contains code that was helpful in diagnosing this problem.
runtime/mercury_context.c:
Fix the bug (as above).
Initialise the MR_init_engine_array_lock in MR_setup_thread_stuff.
Print out logging messages if MR_DEBUG_STACK_SEGMENTS is defined in various
places.
runtime/mercury_debug.h:
runtime/mercury_debug.c:
Write code for printing out debug log messages in a grade appropriate-way.
runtime/mercury_memory_handlers.c:
When exiting a signal handler flush the threadscope buffers of all engines
before re-raising the signal.
runtime/mercury_engine.h:
runtime/mercury_engine.c:
In thread safe grades provide an array of pointers to Mercury engines in
the runtime. This is used to flush the threadscope buffers in the signal
handlers. It may be used to improve work stealing in the future.
runtime/mercury_thread.h:
runtime/mercury_thread.c:
When creating threads add each one's engine to the array of engine pointers.
runtime/mercury_wrapper.c:
Allocate the array of engine pointers when the runtime starts up.
This patch is heavily based on earlier, uncommitted work by Peter Wang. It
has been updated so that it applies against the current version of the source.
A number of other changes have been made. Peter's original ChangeLog
follows:
Implement work stealing for parallel conjunctions. This builds on an
older patch which introduced work-stealing deques to the runtime but
didn't perform work stealing.
Previously when we came across a parallel conjunct, we would place a spark
into either the _global spark queue_ or the _local spark stack_ of the
Mercury context. A spark on the global spark queue may be picked up for
parallel execution by an idle Mercury engine, whereas a spark on a local
spark stack is confined to execution in the context that originated it.
The problem is that we have to decide, ahead of time, where to put a
spark. Ideally, we should have just enough sparks in the global queue to
keep the available Mercury engines busy, and leave the rest of the sparks
to execute in their original contexts since that is more efficient. But
we can't predict the future so have to make do with guesses using simple
heuristics. A bad decision, once made, cannot be reversed. An engine may
sit idle due to an empty global spark queue, even while there are sparks
available in some local spark stacks.
In the work stealing scheme, sparks are always placed into each context's
_local spark deque_. Idle engines actively try to steal sparks from
random spark deques. We don't need to make irreversible and potentially
suboptimal decisions about where to put sparks. Making a spark available
for parallel execution is cheap and happens by default because of the
work-stealing deques; putting a spark on a global queue implies
synchronisation with other threads. The downside is that idle engines
need to expend more time and effort to find the work from multiple places
instead of just one place.
Practically, the new scheme seems to work as well as the old scheme and
vice versa, except that the old scheme often required
`--max-context-per-threads' to be set "correctly" to get good results.
Only tested on x86-64, which has a relatively constrained memory model.
My modifications include:
The difference between 'shared' and 'private' synchronisation terms has
been removed. All sync terms are assumed to be shared and thread-safe
operations are used everywhere. This allows us to remove complicated code
used when a private synchronisation term became shared. This may change
the performance of thread stealing, in particular it may become slower due
to the assumption that all sync terms are shared and therefore atomic
operations must always be used when decrementing their count field.
I've re-factored MR_do_join_and_continue, It is now much simpler as the
conditional code in it enumerates the possible cases clearly.
This change bootchecks and successfully runs the test suite in asm_fast.gc
asm_fast.gc.par hlc.gc and hlc.par, no other grades where tested. I have not
yet tested performance.
runtime/mercury_context.c:
runtime/mercury_context.h:
Keep pointers to all spark deques in a flat array, so we have access
to them for stealing.
Added functions to manage the global array of spark deques.
Modify MR_do_run_next, it now attempts to steal work from other context's
spark queues. Threads sleeping on the condition variable in
MR_do_run_next now use a timed wait so they can wakeup and try to steal
sparks.
Re-factored MR_do_join_and_continue.
MR_num_idle_engines is used by atomic operations, it has been made an
MR_Integer so that it's size matches the expectations of the atomic
operations we have defined.
Modified the MR_SyncTerm and MR_Spark structures. Sparks now point to
their sync terms. The perant stack pointer has been moved into the
SyncTerm structure. The MR_st_is_shared field in the MR_SyncTerm
structure has been removed.
runtime/mercury_atomic_ops.c:
runtime/mercury_atomic_ops.h:
Implement a new atomic operation: decrement integer and is zero. On the
x86/x86_64 one can't atomically decrement an integer and fetch the result
in a single instruction, a loop with a 'compare and exchange' instruction
is necessary. However since we only want to test if the value has become
zero after the decrement we can use the processor's flags. This can be
done in two instructions, but more importantly a loop is not required and
only one instruction is atomic.
runtime/mercury_wrapper.c:
runtime/mercury_wrapper.h:
Added runtime tunable options for work stealing. These control the number
of attempts an idle engine will make when looking for work, and the
duration to sleep after failing to find any work.
runtime/mercury_thread.c:
runtime/mercury_thread.h:
Added MR_COND_TIMED_WAIT, which waits on condition variables like
MR_COND_WAIT except that it may time out.
runtime/mercury_wsdeque.h:
runtime/mercury_wsdeque.c:
MR_wsdeque_pop_bottom now uses it's second argument to return the code
address to jump to rather the whole spark.
runtime/mercury_conf.h.in:
configure.in:
Test for sched_yield()
Change the synchronisation term structure.
doc/user_guide.texi:
Add commented out documentation for two new tunable parameters,
`--worksteal-max-attempts' and `--worksteal-sleep-msecs'.
Implementors may want to experiment with different values but end
users shouldn't need to know about them.
This change replaces the MR_join_and_continue macro with a C procedure. A
smaller macro named MR_join_and_continue wraps the new C procedure and provides
a trampoline to prevent C stack leaks. MR_join_and_continue will now have the
additional cost of a C procedure call rather than always being inlined. This
code is only used in the implementation of parallel conjunctions in the low
level C grades, it does not affect other grades.
An earlier revision of this code was causing deadlocks, to debug them support
was added to the MR_SIGNAL MR_BROADCAST and MR_WAIT macros to enable better
logging of the use of condition variables when MR_DEBUG_THREADS is defined at
compile time.
This change passes bootcheck and the test suite in the asm_fast.gc.par grade.
runtime/mercury_context.h:
runtime/mercury_context.c:
Created MR_do_join_and_continue procedure from old MR_join_and_continue
macro.
Added additional comments to this procedure, describing how it works.
Created a new macro MR_join_and_continue that wraps the new procedure.
Conform to changes in the MR_WAIT, MR_SIGNAL and MR_BROADCAST macros.
runtime/mercury_thread.h:
Added a from parameter to the MR_WAIT, MR_SIGNAL and MR_BROADCAST macros.
Added a from parameter to the C procedures' declarations that implement the
debugging versions of the condition operations above.
Adjusted the formatting of these declarations to match the C style used in
the project.
runtime/mercury_thread.c:
The C procedures implementing the debugging versions of the condition
operations now print out their from parameter.
MR_cond_broadcast now uses "broadcast" in it's log message rather than
"signal"
MR_cond_wait's log message now more clearly specifies which argument is the
lock and which is the condition variable.
Estimated hours taken: 0.1
Branches: main
runtime/mercury_context.[ch]:
library/par_builtin.m:
Fix spelling error in Paul's change, and fix some old bad indentation.
calls that are made when thread debugging is not enabled.
runtime/mercury_thread.c
runtime/mercury_thread.h
Update the functions to which the debug versions of MR_LOCK,
MR_UNLOCK, MR_SIGNAL, etc, expand. Make them all return int error
codes, as do the underlying pthread_* functions, make
MR_cond_signal call pthread_cond_signal instead of
pthread_cond_broadcast, and create an analogous MR_cond_broadcast
function.
Estimated hours taken: 7
Branches: main
Add support for thread-local backjumping in grades that support concurrency.
library/backjump.m:
Shift the C definition of backjump handlers and choice ids into the
runtime. This is needed because MR_Context now refers to them and
the runtime should not depend on the library.
Delete the XXX comment regarding pred_is_external/3. (This has been
fixed below.)
runtime/mercury_backjump.h:
runtime/mercury_backjump.c:
New module that defines those parts of the backjumping support that
the runtime requires access to.
In high-level C .par grades make the global state required by
backjumping thread-specific.
Conform to the usual coding conventions in the runtime in the new
versions of the data structures that were originally in backjump.m.
Rename ML_Choice_Id to MR_BackJumpChoiceId, the latter is less
ambiguous.
runtime/mercury_context.h:
runtime/mercury_context.c:
In low-level C grades add two extra fields to the MR_Context structure
to hold the global state required by backjumping.
In high-level C .par grades initialise the the thread-specific data that
is used to store the backjump global state at program startup.
Reformat a function prototype.
runtime/mercury_thread.h:
Reformat a function prototype.
runtime/Mmakefile:
Include the new files.
mdbcomp/program_representation.m:
Update pred_is_external/3 to include backjump.builtin_choice_id/1
and backjump.builtin_backjump/1.
tests/hard_coded/Mmakefile:
tests/hard_coded/tl_backjump_test.m:
tests/hard_coded/tl_backjump_test.exp:
Test thread-local backjumping.
tests/hard_coded/tl_backjump_test.exp2:
Expected output for the above test case for grades in which spawn/3
does not work.
Branches: main
Prevent multi-threaded programs from terminating as soon as the main thread
terminates, i.e. the process should not terminate until all threads started by
thread.spawn/3 terminate.
This is done by maintaining a a global count of the number of threads started
by thread.spawn. In low-level C grades the main context will suspend if it
reaches the global_success label and finds there are other contexts still
outstanding. The last context to terminate then reschedules the main context
to resume.
Similarly, in high-level C grades the main thread waits on a condition
variable, which is signalled by the last thread to terminate.
library/thread.m:
runtime/mercury_context.c:
runtime/mercury_thread.c:
runtime/mercury_thread.h:
runtime/mercury_wrapper.c:
As above.
Add some extra assertions related to this.
tests/par_conj/Mmakefile:
tests/par_conj/thread_barrier.exp:
tests/par_conj/thread_barrier.m:
Add test case
NEWS:
Announce the change.
Estimated hours taken: 6
Branches: main
Fix some software rot that prevented I/O operations from working in mmos
grades. The problem was the change to the I/O module to make it use thread
local storage via a new field of the MR_Context structure which was accessed
via the MR_eng_this_context field of the engine, instead of via the
MR_eng_context field. The new field was not set by the code for initializing
the contexts used by own stack minimal model tabling.
runtime/mercury_context.h:
runtime/mercury_engine.h:
Add significant new documentation about how fields of the MR_Context
structure are accessed, both because the documentation is useful and to
make similar mistakes less likely in future.
Add a macro for use by own stack minimal model tabling.
runtime/mercury_thread.c:
Add a comment about a link to mercury_engine.h.
runtime/mercury_thread.h:
Convert to four-space indentation, and fix some formatting.
runtime/mercury_mm_own_stacks.c:
Add code for filling in the missing fields of newly created contexts.
runtime/mercury_wrapper.c:
In own stack minimal model grades, set up the main context properly.
The previous code was based on a flawed understanding of the
relationalship between MR_eng_context and MR_eng_this_context.
tests/debugger/mmos_print.{m,inp,exp}:
Add a new test case (which we don't yet pass due to a problem with
formatting of mdb output) to test the fix. The old versions of the
compiler don't pass this test case, because the "p *" commands of the
debugger invoke I/O code in the Mercury standard library, which fails
with a segfault due to the thread local fields of generators' contexts
being uninitialized.
Note that the .inp aborts execution, because without the abort the
execution would go into an infinite loop since mmos grades don't yet
have code for detecting completion.
tests/debugger/Mmakefile:
Enable the new test case in mmos grades.
Fix inconsistent indentation.
tests/tabling/Mmakefile:
Do not try to execute minimal tests in mmos grades, since we don't pass
them yet, and the symptom is in many cases an infinite loop.
Branches: main
Move more of the concurrency related modules from extras into the standard
library.
library/Mercury.options:
library/library.m:
library/thread.m:
library/thread.channel.m:
library/thread.mvar.m:
library/thread.semaphore.m:
Move the concurrency-related modules `channel', `mvar' and
`semaphore' from extras/concurrency into the standard library.
Make thread.mvar use the standard library module mutvar instead of
providing its own implementation of the same thing.
Replace "ME_" prefixes by "ML_".
library/mutvar.m:
Add predicate `new_mutvar0' which is like `new_mutvar' but does not
require an initial value for the mutvar. This is needed for
thread.mvar.
Define `new_mutvar' in terms of `new_mutvar0' and `set_mutvar'.
runtime/mercury_thread.h:
Make the MR_WAIT macro expand to "(0)" when MR_THREAD_SAFE is not
defined, so it can be used in an expression context. Zero is the
success code for pthread_cond_wait.
extras/concurrency/channel.m:
extras/concurrency/mvar.m:
extras/concurrency/semaphore.m:
Remove these modules.
extras/concurrency/Mercury.options:
extras/concurrency/concurrency.m:
Delete lines pertaining to removed modules.
extras/concurrency/philo.m:
extras/concurrency/philo2.m:
extras/concurrency/philo3.m:
Update to use the standard library modules.
NEWS:
Announce the change.
Estimated hours taken: 15
Branches: main
Add support for thread-local mutables. These can take on a different value for
each Mercury thread. Child threads automatically inherit the thread-local
values of the parent thread that spawned it.
compiler/make_hlds_passes.m:
compiler/prog_io.m:
compiler/prog_item.m:
compiler/prog_mutable.m:
Accept a `thread_local' attribute for mutables and update the
source-to-source transformation.
doc/reference_manual.texi:
Document the `thread_local' attribute as a Melbourne Mercury compiler
extension.
runtime/mercury_context.c:
runtime/mercury_context.h:
Add a `thread_local_mutables' field to MR_Context, which points to an
array which holds all the values of thread-local mutables in the
program. Each thread-local mutable has an associated index into the
array, which is allocated during initialisation. A child thread
inherits the parent's thread-locals simply by copying the array.
Add a `thread_local_mutables' field to MR_Spark and update the parallel
conjunction implementation to take into account thread-locals.
runtime/mercury_thread.c:
runtime/mercury_thread.h:
Add the functions and macros which are used by the code generated for
thread-local mutables.
runtime/mercury_wrapper.c:
Allocate a thread-local mutable array for the initial context at
startup.
extras/concurrency/spawn.m:
Update the spawn/3 implementation to make child threads inherit the
thread-local values of the parent.
Make different threads in high-level C grades use different
MR_Contexts. This makes it possible to use the same implementation of
thread-local mutables as in the low-level C grades.
tests/hard_coded/mutable_decl.exp:
tests/hard_coded/mutable_decl.m:
tests/hard_coded/pure_mutable.exp:
tests/hard_coded/pure_mutable.m:
tests/invalid/bad_mutable.err_exp:
tests/invalid/bad_mutable.m:
Add some thread-local mutables to these test cases.
NEWS:
Announce the addition.
Estimated hours taken: 20
Branches: main
Add coroutining support for dependent parallel conjunctions in lowlevel
parallel grades.
library/par_builtin.m:
Change definitions of synchronisation primitives so that waiting on a
future causes the current context to be suspended. Signalling a
future causes all the contexts waiting on the future to be scheduled.
runtime/mercury_context.c:
runtime/mercury_thread.c:
runtime/mercury_thread.h:
runtime/mercury_wrapper.c:
Add a global `MR_primordial_thread' to hold the thread id of the
primordial thread.
Add sanity checks, in particular that the primordial thread does not
exit like other threads as it needs to clean up the Mercury runtime.
tests/par_conj/Mmakefile:
Actually run dependent parallel conjunction tests since they should
no longer deadlock.
tests/par_conj/*.exp:
Add expected outputs for test cases which didn't have them.
Estimated hours taken: 6
Branches: main, release
This patch fixes a bug with lowlevel parallel grades. A program built in such
a grade could hang when running with multiple threads.
runtime/mercury_context.c:
After scheduling a Mercury context, use MR_BROADCAST to wake up all
idle threads instead of MR_SIGNAL, if the newly scheduled context might
not be accepted for execution by any single woken thread. The hang
used to occur when a context was scheduled but the wrong idle thread
was woken up to execute it (because the context is 'owned' by another
thread) and promptly went back to idling.
runtime/mercury_thread.h:
Add MR_BROADCAST macros.
tests/par_conj/Mmakefile:
tests/par_conj/threads_hang.exp:
tests/par_conj/threads_hang.m:
Add a test case.
Estimated hours taken: 0.2
Branches: main
runtime/mercury_calls.h:
runtime/mercury_prof.h:
runtime/mercury_signal.h:
runtime/mercury_string.h:
runtime/mercury_thread.c:
runtime/mercury_thread.h:
Fix some layout issues in these files. There are no algorithmic
changes.
Estimated hours taken: 0.1
Branches: main
Fix a bug in my last attempt to fix bugs in petdr's last change.
runtime/mercury_thread.h:
Use `MR_bool' rather than `bool' for MR_debug_threads.
Estimated hours taken: 0.25
Branches: main
Fix another bug in petdr's last change.
runtime/mercury_thread.h:
Declare MR_debug_threads.
This needs to be declared here since it is referenced from
mercury_wrapper.c.
Estimated hours taken: 0.1
Branches: main
Fix bugs in Pete's last change.
runtime/mercury_wrapper.c:
#include "mercury_thread.h", since this file references
MR_debug_threads which is declared there.
runtime/mercury_thread.h:
Move the declarations of MR_mutex_* and MR_cond_*
inside `#if MR_THREAD_SAFE', since they refer to
the MercuryLock and MercuryCond types which are only
defined in the MR_THREAD_SAFE case.
Estimated hours taken: 1
Branches: main, release
Allow one to turn thread debugging on at runtime.
However only modules which are compiled with MR_DEBUG_THREADS will
have debugging messages output.
runtime/mercury_thread.c:
Add MR_debug_threads global variable which is used to control
whether debugging messages are output.
Always include the debug version of lock, unlock, signal and
wait in the runtime library.
runtime/mercury_thread.h:
When MR_DEBUG_THREADS is defined, conditionally choose using
the MR_debug_threads global variable between the debug and the
pthread library versions of lock, unlock, signal and wait.
runtime/mercury_wrapper.c:
Parse the --debug-threads option in the MERCURY_OPTIONS
environment variable.
doc/user_guide.texi:
Document --debug-threads.
Estimated hours taken: 4
Branches: main, release
Get exception handling working in the parallel grades.
library/exception.m:
Define the macros ML_GET_EXCEPTION_HANDLER and
ML_SET_EXCEPTION_HANDLER which either save the exception
handler into thread local storage for the parallel grades or
save it into a global variable.
runtime/mercury_context.c:
Initialise the thread local storage for holding the exception
handler.
runtime/mercury_thread.c:
runtime/mercury_thread.h:
Define the key used to access the thread local storage for the
exception handler.
Estimated hours taken: 4
Branches: main
Add MR_ prefixes to the remaining non-prefixed symbols.
This change will require all workspaces to be updated
The compiler will start generating references to MR_TRUE,
MR_bool, etc., which are not defined in the old runtime
header files.
runtime/mercury_std.h:
Add MR_ prefixes to bool, TRUE, FALSE, max, min,
streq, strdiff, strtest, strntest, strneq, strndiff,
strntest, NO_RETURN.
Delete a commented out definition of `reg'.
runtime/mercury_tags.h:
Add an MR_ prefix to TAGBITS.
configure.in:
runtime/mercury_goto.h:
runtime/machdeps/i386_regs.h/mercury_goto.h:
Add an MR_ prefix to PIC.
runtime/mercury_conf_param.h:
Allow non-prefixed PIC and HIGHTAGS to be defined on
the command line.
runtime/mercury_bootstrap.h:
Add backwards compatibility definitions.
RESERVED_MACRO_NAMES:
Remove the renamed macros.
compiler/export.m:
compiler/ml_code_gen.m:
Use MR_bool rather than MR_Bool (MR_Bool is
meant to be for references to the Mercury type
bool__bool).
runtime/mercury_types.h:
Add a comment the MR_Bool is for references to
bool__bool.
*/*.c:
*/*.h:
*/*.m:
Add MR_ prefixes.
Estimated hours taken: 16
Add some Mmake rules to the runtime and some code to tools/bootcheck
so that we automatically check that the namespace remains clean.
Also add some `MR_' prefixes that Zoltan missed in his earlier change.
tools/bootcheck:
Add an option, which is enabled by default, to
build the check_namespace target in the runtime.
runtime/RESERVED_MACRO_NAMES:
New file. Contains a list of the macros names that
don't start with `MR_' or the like.
runtime/Mmakefile:
Change the rule for `check_headers' so that it checks for macros
that don't occur in the RESERVED_MACRO_NAMES files, as well as not
starting with `MR_' prefixes, and reports errors for such macros.
Also add a rule for check_objs that checks whether the object
files define any global symbols that don't have the right prefixes,
and a rule `check_namespace' that does both of the above.
runtime/mercury_bootstrap.h:
#include "mercury_types.h" and "mercury_float.h",
to ensure that this header file is self-contained.
Also make sure that all the old names are disabled if you
compile with `-DMR_NO_BACKWARDS_COMPAT'.
runtime/mercury_context.c:
runtime/mercury_thread.h:
runtime/mercury_thread.c:
Use `bool' rather than `MR_Bool' for the argument to
MR_check_pending_contexts() and the return type of
MR_init_thread(), since these are C bools, not Mercury
bools, and there's no requirement that they have the
same size as MR_Integer.
runtime/mercury_type_info.h:
runtime/mercury_deep_copy_body.h:
runtime/mercury_tabling.c:
library/std_util.m:
trace/mercury_trace_declarative.c:
trace/mercury_trace_external.c:
trace/mercury_trace_internal.c:
tests/hard_coded/existential_types_test.m:
Add MR_ prefixes to UNIV_OFFSET_FOR_TYPEINFO and UNIV_OFFSET_FOR_VALUE.
trace/mercury_trace_external.c:
trace/mercury_trace_internal.c:
trace/mercury_trace_tables.c:
Add MR_ prefixes to do_init_modules().
runtime/mercury_tabling.h:
runtime/mercury_tabling.c:
runtime/mercury_overflow.h:
runtime/mercury_debug.h:
Add MR_ prefixes to table_*.
runtime/mercury_overflow.h:
runtime/mercury_debug.h:
Add MR_ prefixes to IF().
runtime/mercury_context.h:
Add MR_ prefixes to IF_MR_THREAD_SAFE().
runtime/mercury_engine.h:
Add MR_ prefixes to IF_NOT_CONSERVATIVE_GC().
runtime/mercury_engine.h:
runtime/mercury_engine.c:
extras/aditi/aditi.m:
Add MR_ prefixs to do_fail, do_redo, do_not_reached, etc.
compiler/trace.m:
compiler/fact_table.m:
compiler/llds_out.m:
Add MR_ prefixes to the generated code.
Estimated hours taken: 10
Make everything in the runtime use MR_ prefixes, and make the compiler
bootstrap with -DMR_NO_BACKWARDS_COMPAT.
runtime/mercury_*.[ch]
Add MR_ prefixes to all functions, global variables and almost all
macros that could pollute the namespace. The (intentional) exceptions
are
1. some function, variable, type and label names that already start
with MR_, mercury_, Mercury or _entry;
2. some standard C macros in mercury_std.h;
3. the macros used in autoconfiguration (since they are used in scripts
as well as the runtime, the MR_ prefix may not be appropriate for
those).
In some cases, I deleted things instead of adding prefixes
if the "things" were obsolete and not user visible.
runtime/mercury_bootstrap.h:
Provide MR_-less forms of the macros for bootstrapping and for
backward compatibility for user code.
runtime/mercury_debug.[ch]:
Add a FILE * parameter to a function that needs it.
compiler/code_info.m:
compiler/export.m:
compiler/fact_table.m:
compiler/llds.m:
compiler/llds_out.m:
compiler/pragma_c_gen.m:
compiler/trace.m:
Add MR_ prefixes to the C code generated by the compiler.
library/*.m:
Add MR_ prefixes to handwritten code.
trace/mercury_trace_*.c:
util/mkinit.c:
Add MR_ prefixes as necessary.
extras/concurrency/semaphore.m:
Add MR_ prefixes as necessary.
Estimated hours taken: 2.5
Fix a bug with :- export and threads.
Each time we called from C to Mercury, we initialized the thread engine
(if necessary).
This allocated a new context every time we entered Mercury.
Unfortunately, these contexts were never released, so every entry into
Mercury from C cost about 4Mb of memory (almost all of which is the
deterministic stack). In a busy CORBA application you would run out of
memory really quickly.
compiler/export.m:
When initializing threads, remember whether we are responsible
for finalizing the engine (e.g. if we are the first C->Mercury
call to create the engine, we'll be the last to exit it and should
clean up afterwards).
runtime/mercury_engine.c:
Finalize engines by destroying the context (this will put the
memory zones onto a free list).
runtime/mercury_thread.c:
runtime/mercury_thread.h:
Make init_thread return TRUE if an engine has been allocated and
it is the caller's responsibility to finialize it.
Estimated hours taken: 4
When exposing a Mercury predicate to C, you need to make sure there is
an engine for it to run in before calling it.
For single-threaded programs, we simply make sure that the engine is run
soon after main(). But for multi-threaded programs, code written in C
could create new threads and call Mercury using one of these threads.
Of course the same code could be called from a thread where there is
already a Mercury engine.
This change fixes this problem by creating a Mercury engine if and only
if it is required.
compiler/export.m:
Run init_engine when calling into Mercury. It is important to
run this after we save the C registers, but before we try to use
the Mercury registers.
runtime/mercury_thread.c:
runtime/mercury_thread.h:
If an existing Mercury engine is already in the current thread,
don't create a new one.
Estimated hours taken: 0.25
Fix a bootstrapping problem with the changes to MR_OBTAIN_GLOBAL_C_LOCK()
that Tom Conway recently committed.
runtime/mercury_thread.h:
Rename the remaining occurrences of MR_{OBTAIN,RELEASE}_GLOBAL_C_LOCK()
as MR_{OBTAIN,RELEASE}_GLOBAL_LOCK(), and add the new argument.
(Tom and I both forgot these additional occurrences in our previous
changes.)
Estimated hours taken: 0.5
Fix a bootstrapping problem with the changes to MR_OBTAIN_GLOBAL_C_LOCK()
that Tom Conway recently committed.
runtime/mercury_thread.h:
compiler/pragma_c_gen.m:
Rename MR_{OBTAIN,RELEASE}_GLOBAL_C_LOCK()
as MR_{OBTAIN,RELEASE}_GLOBAL_LOCK().
Also delete the unnecessary trailing semicolons
from the definitions of those macros.
runtime/mercury_bootstrap.h:
Add a definition of the old zero-arity MR_*_GLOBAL_C_LOCK() macros
in terms of the new unary MR_*_GLOBAL_LOCK() macros.
Estimated hours taken: 20
Improvements to coroutining support. These changes allow us to do
provide io primatives that cause the Mercury context to suspend rather
than causing the engine to block.
configure.in:
Test to see if we can handle contexts that block on IO
using select().
compiler/pragma_c_gen.m:
Include the predicate name in the calls to MR_OBTAIN_GLOBAL_C_LOCK
and MR_RELEASE_GLOBAL_C_LOCK for improved debugging.
Fix a bug where the global lock was not being released when
semidet pragma c code failed.
runtime/mercury_thread.h:
Change the global lock macros to include the message generated
by the changes to pragma_c_gen.
library/char.m:
library/std_util.m:
include `thread_safe' in the flags for a couple of pragma c
definitions that seem to have missed out.
runtime/mercury_context.{c,h}:
Add a list of "pending" contexts that are blocked on a
file descriptor. When the runqueue becomes empty, we call
select on all the pending contexts.
Move schedule from the header file to the c file (changing
it from a macro to a function) for easier debugging at a
slight performance cost.
TODO: add a nonblocking call to select so that we can poll
for io from time to time rather than waiting till there is
nothing else to do.
runtime/mercury_reg_workarounds.{c,h}:
Make functions that forward to the FD_* macros, which on Linux
attempt to use registers that we've already grabbed. Aarrggh!
runtime/mercury_thread.c:
Tidy up some of the conditional compilation.
runtime/mercury_types.h:
Remove the definition of SpinLock since we're not using them
and are not likely to any time soon.
Estimated hours taken: 15
Fix several bugs in the runtime engine to do with thread-safe execution.
Add a new flag to pragma c_code/import to allow programmers to specify
whether or not the C code is thread-safe or not, and modify code generation
to put a lock around C code that isn't thread_safe.
runtime/mercury_thread.{c,h}:
Add a global lock.
Change the handling of thread creation. create_thread now takes
a "closure" (a C struct with a fn pointer and an argument to pass
the function) which it calls in the new thread. (The same mechanism
is used in the Boehm collector), or NULL which causes the thread
to wait for work to appear in the Mercury runqueue.
runtime/mercury_context.c:
initialize the global lock.
runtime/mercury_engine.{c,h}:
Add a new field to the MercuryEngine structre which is used to
store a list of saved thread ids. These were being saved in a
local variable in call_engine_inner which was a bug because
call_engine_inner's (C) stack frame gets scribbled on by Mercury
execution. For more detail see the comments in mercury_engine.h
runtime/mercury_wrapper.c:
Use the new interface to create_thread.
compiler/prog_io_pragma.m:
Parse either a single attribute or a list of attributes instead
of just 'may_call_mercury' in pragma c code and pragma import.
These are stored in an abstract type 'pragma_c_code_attributes'
that uses a bit array (aka int) to store the attributes.
compiler/pragma_c_gen.m:
Get the code generator to emit c code to obtain and release the
global lock for pragma c code that isn't thread_safe.
compiler/<various>.m:
Change may_call_mercury to pragma_c_code_attributes.
doc/reference_manual.m:
Document the change to pragma c code.
scripts/mgnuc.in:
Pass some extra C flags for thread-safe compilation for Linux.
Estimated hours taken: 2
Various minor cleanups to fix some things introduced in recent
changes to the runtime.
mercury_context.c:
mercury_context.h:
mercury_regorder.h:
mercury_regs.h:
mercury_thread.c:
mercury_thread.h:
mercury_type_info.c:
mercury_engine.h:
mercury_engine.c:
Fix up the layout in a few places. For multi-line macros
defined using `do { ... } while (0)', put the `do {' on a line
of its own.
Change some names to conform to our standard naming
convention and to avoid name clashes:
s/Context/MR_Context/g
s/CONTEXT/MR_context_struct/g
s/SYNCTERM/MR_sync_term/g
s/sp/MR_sp/g
s/curfr/MR_curfr/g
s/maxfr/MR_maxfr/g
mercury_engine.h:
mercury_engine.c:
Fix a couple of bugs that caused it to not compile in certain
grades.
Delete some obsolete comments. Add some new ones and rearrange
some of the code to make it more readable.
mercury_type_info.c:
Cast values used in switch statements to (int), to avoid
a warning from gcc.
