library/*.m:
Remove tabled_for_io attributes from C# and Java foreign_procs.
I/O tabling is not supported by those backends.
Remove will_not_modify_trail attributes from C# and Java foreign_procs,
and from predicates that do I/O. They have no effect in the former and
cannot affect anything with the latter.
Fix a spot a where will_not_modify_trail was given, but
will_not_call_mercury was meant.
The non .par LLDS grades currently "support" concurrency. However, this support
is really only usable with a small few benchmark programs, due to the inability
of the implementation to switch threads if the executing thread blocks. Disable
this capability by causing spawn/3 to abort if used in non .par LLDS grades.
library/thread.m:
Make the above change to spawn/3.
compiler/globals.m:
Update the code that checks if the current grade supports concurrency.
compiler/options.m:
Adjust documentation for the --parallel option.
tests/warnings/help_text.err_exp:
Conform to the above change.
library/bt_array.m:
Reorder the arguments of the resize and shrink predicates
to make them easier to use with state variables.
tests/general/array_test.m:
Update the test calls to these two predicates.
library/version_array2d.m:
Rewrite part of this module using a programming style that includes
variable names that are longer than one character :-(. In the absence
of comprehensive, or even basic, test cases, leave the rest using
the old style.
Add a non-field-syntax lookup operation.
Mark the site of a probable bug.
library/version_bitmap.m:
Add non-field-syntax versions of the get_bit and set_bit operations.
NEWS.md:
Announce the reordering and the new predicates above.
library/getopt.m:
library/getopt_io.m:
Apply to getopt_io.m an update that was previously applied to getopt.m,
even though getopt.m is now computed from getopt_io.m.
library/array2d.m:
library/bit_buffer.m:
library/bit_buffer.read.m:
library/bit_buffer.write.m:
library/bitmap.m:
library/cord.m:
library/edit_distance.m:
library/edit_seq.m:
library/fat_sparse_bitset.m:
library/fatter_sparse_bitset.m:
library/list.m:
library/one_or_more.m:
library/pair.m:
library/ra_list.m:
library/rbtree.m:
library/set_bbbtree.m:
library/set_ctree234.m:
library/set_ordlist.m:
library/set_tree234.m:
library/set_unordlist.m:
library/solutions.m:
library/sparse_bitset.m:
library/test_bitset.m:
library/thread.barrier.m:
library/thread.m:
library/thread.semaphore.m:
library/time.m:
library/tree_bitset.m:
library/version_array.m:
library/version_hash_table.m:
library/version_store.m:
General updates to style, the main ones being the following.
Using standard names for type variables:
- K and V for key and value types in map-like structures
- A to F for types of accumulators
- T for most everything else, possibly with a numeric suffix
to distinguish two separate types that nevertheless play
similar roles.
(I left descriptive type var names such as Stream and Store unchanged.)
Adding or expanding descriptions of exported predicates and functions.
Declaring and defining field syntax getters and setters without using ^,
while keeping the use of field syntax, including ^, in the operations'
descriptions.
Defining field syntax operations operations in terms of the
non-field-syntax versions, not vice versa.
Defining function versions of operations in terms of the predicate
versions, not vice versa.
The tabling transform aborted on a predicate in thread.m because
an argument has mode 'unused':
Software Error: predicate `transform_hlds.table_gen.get_input_output_vars'/7:
Unexpected: bad var
library/thread.m:
Change the mode of the argument from 'unused' to 'in'.
library/thread.m:
Implement spawn_native_joinable and join_thread for C# and Java.
Rename the existing Java helper class RunGoal to RunGoalDetached.
Add RunGoalJoinable.
Rename the C# helper MercuryThread to RunGoalDetached, to match the
Java backend. Add RunGoalJoinable.
java/runtime/MercuryThreadPool.java:
Replace submitExclusiveThread() method with createExclusiveThread(),
which returns the newly created thread, without starting it.
It is often necessary to be certain that a thread has terminated before
the rest of the program can continue. In some cases, signalling that a
thread is ABOUT to terminate (using any synchronisation device)
is insufficient: even after the last piece of Mercury code has run,
the thread still has additional cleanup code to run, which might include
arbitrary code in thread-specific data destructors, etc.
Introduce a predicate to create a joinable native thread,
and a predicate to wait for that thread to terminate (joining).
Joinable threads are only implemented for C backends for now.
library/thread.m:
Add new predicates spawn_native_joinable and join_thread.
Add a internal type thread_handle.
Rename the internal type thread_id type to thread_desc,
as thread_id is too similar to thread_handle.
tests/hard_coded/Mmakefile:
tests/hard_coded/spawn_native_joinable.exp:
tests/hard_coded/spawn_native_joinable.exp2:
tests/hard_coded/spawn_native_joinable.m:
Add test case.
NEWS.md:
Announce changes.
In the Mercury standard library, every exported predicate or function
has (or at least *should* have) a comment that documents it, including
the meanings of its arguments. About 35-40% of these modules put `'s
(left and right quotes) around the names of the variable representing
those arguments. Some tried to do it consistently (though there were spots
with unquoted or half quoted names), while some did it only a few places.
This is inconsistent: we should either do it everywhere, or nowhere.
This diff makes it nowhere, because
- this is what the majority of the standard library modules do;
- this is what virtually all of the modules in the compiler, profiler,
deep_profiler etc directories do;
- typing all those quotes when adding new predicates in modules that
follow this convention is a pain in the ass; and because
- on many modern terminals, `' looks non-symmetrical and weird.
Likewise, the comment explaining a predicate often started with
% `predname(arguments)' returns ...
This diff deletes these quotes as well, since they add nothing useful.
This diff does leave in place quotes around code fragments, both terms
and goals, where this helps delineate the boundaries of that fragment.
library/thread.m:
Add thread_options type, and a variant of the spawn_native predicate
that takes a thread_options argument.
The only thread option supported so far is to request a minimum
stack size for the newly created thread. This is only implemented
for POSIX threads so far.
NEWS:
Announce changes.
library/thread.closeable_channel.m
Add the new module. It omits two operations from thread.channel.m:
- untake, because it is deprecated
- duplicate, because I'm not sure what it's useful for in practice
library/MODULES_DOC:
library/library.m
library/thread.m:
NEWS:
Mention the new module.
browser/collect_lib.m:
browser/declarative_execution.m:
browser/dl.m:
browser/io_action.m:
compiler/make.util.m:
compiler/pickle.m:
compiler/process_util.m:
compiler/prog_event.m:
library/array.m:
library/benchmarking.m:
library/bit_buffer.m:
library/builtin.m:
library/char.m:
library/deconstruct.m:
library/dir.m:
library/erlang_rtti_implementation.m:
library/int.m:
library/int16.m:
library/int32.m:
library/int64.m:
library/int8.m:
library/io.m:
library/math.m:
library/mutvar.m:
library/private_builtin.m:
library/profiling_builtin.m:
library/rtti_implementation.m:
library/store.m:
library/string.format.m:
library/string.m:
library/table_builtin.m:
library/term_size_prof_builtin.m:
library/thread.m:
library/time.m:
library/type_desc.m:
library/uint16.m:
library/uint32.m:
library/uint64.m:
library/uint8.m:
ssdb/ssdb.m:
As above. This mostly involved two things.
The first was grouping foreign_procs by predicate instead of by language.
In a few cases, this revealed that some predicates *had* no foreign_proc
for a language, while related predicates did have one that just aborted
if called. This diff adds similar aborting foreign_procs to predicate/
language combinations that were missing them, when this seemed obviously
the right thing to do.
The second was moving pragmas about a predicate from the middle of the
block of clauses of that predicate to the start of that block.
The majority of calls to MR_fatal_error do not follow an operation that
sets errno, so printing out an error message unrelated to the reason for
the fatal error will lead to confusion. It can also cause test failures
if errno happens to be set to non-zero some time prior to an expected
call to MR_fatal_error. Fixes bug #464.
runtime/mercury_misc.c:
Don't print value of errno in MR_fatal_error.
runtime/mercury_context.c:
runtime/mercury_thread.c:
Pass strerror strings to MR_fatal_error where appropriate.
runtime/mercury_memory_zones.c:
runtime/mercury_memory_zones.h:
Pass strerror strings to MR_fatal_error following failures of
MR_protect_pages. Document that this assumes MR_protect_pages sets
errno on error.
Skip unnecessary call to sprintf before MR_fatal_error.
runtime/mercury_deep_profiling.c:
Skip unnecessary call to sprintf before MR_fatal_error.
Reduce size of some buffers.
runtime/mercury_overflow.c:
runtime/mercury_stack_trace.c:
Pass a fixed format string to MR_fatal_error just in case
the message string may contain percentage signs.
runtime/mercury_tabling.c:
Skip unnecessary call to sprintf before MR_fatal_error.
deep_profiler/timeout.m:
library/thread.m:
mdbcomp/shared_utilities.m:
Pass strerror strings to MR_fatal_error where appropriate.
trace/mercury_trace.c:
Skip unnecessary call to sprintf before MR_fatal_error.
trace/mercury_trace_external.c:
Pass a fixed format string to MR_fatal_error just in case.
Discussion of these changes can be found on the Mercury developers
mailing list archives from June 2018.
COPYING.LIB:
Add a special linking exception to the LGPL.
*:
Update references to COPYING.LIB.
Clean up some minor errors that have accumulated in copyright
messages.
library/thread.m:
Make num_processors/3 call a function in the runtime instead of
reading a global variable, so we can defer initialisation of the
variable.
Return `no' if unable to determine the number of CPUs instead of
defaulting to 1.
runtime/mercury_context.c:
runtime/mercury_context.h:
Add a function MR_get_num_processors().
Hide MR_num_processors_detected variable.
Rename MR_detect_num_processors() to
MR_init_available_cpus_and_detect_num_processors()
to reflect what it does.
Add a function MR_free_available_cpus(), currently only implemented
for the Linux CPU affinity API path.
Call MR_init_available_cpus_and_detect_num_processors() at startup
only in low-level C grades. The only reason to call it in high-level
C grades was to set MR_num_processors_detected. It will now be set
at the first call to MR_get_num_processors().
Free the available CPU data structures as soon as we have
MR_num_processors_detected, unless required for thread pinning.
Rename MR_setup_num_threads to MR_setup_num_ws_engines.
Pass the number of processors detected as an argument to make
the dependency explicit.
Make MR_pin_primordial_thread return a CPU number as its comment
suggests.
Delete the global variable MR_primordial_thread_cpu as it is
currently unused.
Add a function MR_done_thread_pinning().
runtime/mercury_wrapper.c:
Call MR_done_thread_pinning() to free up available CPU data
structures after thread pinning is done with them.
Add an unrelated XXX.
The main fix is to retry the sched_getaffinity() call in
MR_reset_available_cpus(). If that call failed because we provided a
cpuset that is too small for the kernel, MR_reset_available_cpus() would
set MR_available_cpus to NULL but leave MR_cpuset_size non-zero,
leading to a null pointer dereference soon after.
runtime/mercury_context.c:
runtime/mercury_context.h:
Use a macro MR_HAVE_LINUX_CPU_AFFINITY_API if we have all the pieces
we need for that API.
Rename MR_num_processors to MR_num_processors_detected for clarity.
Add a global variable MR_cpuset_num_cpus to record the size of the
cpuset that we allocated.
Change MR_cpuset_size to size_t to match the type in the underlying
interface.
Rename MR_available_cpus to MR_cpuset_available to clarify the
relationship with MR_cpuset_size and MR_cpuset_num_cpus.
In MR_reset_available_cpus(), retry sched_getaffinity() if it fails
with EINVAL. It may fail because the kernel uses a larger CPU
affinity mask than the cpuset we provided; then we must retry the
call with a larger cpuset.
Ensure MR_reset_available_cpus() does not leave MR_cpuset_*
variables in an inconsistent state.
Make MR_detect_num_processors() set MR_num_processors to 1 if the
MR_cpuset_available set is somehow empty after
MR_reset_available_cpus(). It should not happen, but the program
should be able to continue anyway.
In MR_pin_thread_no_locking(), loop over the entire
MR_cpuset_available set to find a target CPU to pin to.
The CPUs available to the thread are not necessarily numbered
consecutively from the initial CPU onwards mod MR_num_processors.
Fix cpuset allocation in MR_do_pin_thread to not assume that CPUs
are numbered from 0 to MR_num_processors-1. Fix a memory leak.
Clean up MR_current_cpu().
Prevent a null pointer dereference in the hwloc path:
hwloc_get_pu_obj_by_os_index always returns NULL on my system.
Change some types from 'unsigned' to 'int' to match underlying APIs.
library/thread.m:
Conform to variable renaming.
checkout sysconf(_SC_NPROCESSORS_ONLN); error
library/thread.m:
Return error message for errno when pthread_create() fails
in thread.spawn_native.
Return the exception message when thread creation fails in C#.
Catch System.SystemException as that is seen in practice with mono
(might not be documented).
Return the exception message when thread creation fails in Java.
Catch SecurityException (as documented) and OutOfMemoryError
(seen in practice).
Implement thread.yield/2 for the C# grade.
m4/mercury.m4:
Update the C# compiler check to not try the old aliases for the
Mono C# compiler; do not try the DotGNU (which is no longer a going
concern) C# compiler either.
Update the copyright notice for this file.
library/thread.m:
Uncomment and correct the C# implementation of thread.yield/2.
glibc prior to 2.21 is susceptible to a race between sem_post/sem_wait
https://sourceware.org/bugzilla/show_bug.cgi?id=12674
(I have not personally encountered this bug, and I hope never to.)
library/thread.m:
Use a pthread mutex and condition variable instead of a pthread
semaphore to synchronise the newly spawned thread with its parent in
ML_create_exclusive_thread.
sem_wait or sem_timedwait may be interrupted by signal handlers and
return an error. We must retry the call if errno is set to EINTR.
library/thread.m:
Retry MR_SEM_WAIT if interrupted by a signal handler in
ML_create_exclusive_thread.
Call MR_fatal_error if MR_SEM_WAIT fails for any other reason.
The only other documented error code is EINVAL if the semaphore is
invalid; that should not happen.
runtime/mercury_context.c:
Retry MR_SEM_WAIT if interrupted by a signal handler in
MR_shutdown_ws_engines.
runtime/mercury_thread.h:
Add the macro MR_SEM_IS_EINTR for checking if errno == EINTR
after calling MR_SEM_WAIT or MR_SEM_TIMED_WAIT.
runtime/mercury_thread.h:
Rename MR_WAIT to MR_COND_WAIT etc.
runtime/mercury_wrapper.c:
library/thread.m:
library/thread.semaphore.m:
Conform to the above change.
Isolate all dependencies on POSIX unnamed semaphores to the runtime's
mercury_thread module. This is a step towards fixing bug #357.
runtime/mercury_thread.{h,c}:
Provide wrappers for the functions sem_destroy and sem_timedwait.
The latter does not exist on OS X, so (for now) just abort if it
is called.
runtime/mercury_context.c:
library/thread.m:
Use the above wrappers instead of calling the sem_* functions
directly.
Add a predicate that retrieves the number of processors available to this
process if known.
library/thread.m:
As above.
runtime/mercury_context.c:
The existing code that determines the number of processors was only used
on thread safe low-level C grades. Make it also available for thread
safe high-level C grades.
Add a fall back (less accurate) method of determining the number of
processors.
Remove an out-of-date comment.
runtime/mercury_context.h:
Export the number of available processors.
scripts/ml.in:
Link to libhwloc (if configured) on thread safe high and low-level C
grades.
NEWS:
Announce new predicate.
configure.ac:
Update a --help message.
In low-level C grades spawn_context attempts to use the stack of the new
context to pass the closure and thread ID to the new context once it starts
running. The first stack slot used has a 0 offset from the stack pointer,
but the second used has a -1 offset. If the stack pointer is at the
beginning of the memory zone (which happens sometimes but not usually, see
MR_next_offset()) then this can create an invalid memory access. This
raises a segfault with the new version of Boehm GC.
Create a stack frame for the closure and thread ID in spawn_context.
library/thread.m:
As above.
If a module has two or more import_module or use_module declarations
for the same module, (typically, but not always, one being in its interface
and one in its implementation), generate an informational message about
each redundant declaration if --warn-unused-imports is enabled.
compiler/hlds_module.m:
We used to record the set of imported/used modules, and the set of
modules imported/used in the interface of the current module. However,
these sets
- did not record the distinction between imports and uses;
- did not allow distinction between single and multiple imports/uses;
- did not record the locations of the imports/uses.
The first distinction was needed only by module_qual.m, which *did*
pay attention to it; the other two were not needed at all.
To generate messages for imports/uses shadowing other imports/uses,
we need all three, so change the data structure storing such information
for *direct* imports to one that records all three of the above kinds
of information. (For imports made by read-in interface and optimization
files, the old set of modules approach is fine, and this diff leaves
the set of thus *indirectly* imported module names alone.)
compiler/unused_imports.m:
Use the extra information now available to generate a
severity_informational message about any import or use that is made
redundant by an earlier, more general import or use.
Fix two bugs in the code that generated warnings for just plain unused
modules.
(1) It did not consider that a use of the builtin type char justified
an import of char.m, but without that import, the type is not visible.
(2) It scanned cons_ids in goals in procedure bodies, but did not scan
cons_ids that have been put into the const_struct_db. (I did not update
the code here when I added the const_struct_db.)
Also, add a (hopefully temporary) workaround for a bug in
make_hlds_passes.m, which is noted below.
However, there are at least three problems that prevent us from enabling
--warn-unused-imports by default.
(1) In some places, the import of a module is used only by clauses for
a predicate that also has foreign procs. When compiled in a grade that
selects one of those foreign_procs as the implementation of the predicate,
the clauses are discarded *without* being added to the HLDS at all.
This leads unused_imports.m to generate an uncalled-for warning in such
cases. To fix this, we would need to preserve the Mercury clauses for
*all* predicates, even those with foreign procs, and do all the semantic
checks on them before throwing them away. (I tried to do this once, and
failed, but the task should be easier after the item list change.)
(2) We have two pieces of code to generate import warnings. The one in
unused_imports.m operates on the HLDS after type and mode checking,
while module_qual.m operates on the parse tree before the creation of
the HLDS. The former is more powerful, since it knows e.g. what types and
modes are used in the bodies of predicates, and hence can generate warnings
about an import being unused *anywhere* in a module, as opposed to just
unused in its interface.
If --warn-unused-imports is enabled, we will get two separate set of
reports about an interface import being unused in the interface,
*unless* we get a type or mode error, in which case unused_imports.m
won't be invoked. But in case we do get such errors, we don't want to
throw away the warnings from module_qual.m. We could store them and
throw them away only after we know we won't need them, or just get
the two modules to generate identical error_specs for each warning,
so that the sort_and_remove_dups of the error specs will do the
throwing away for us for free, if we get that far.
(3) The valid/bug100.m test case was added as a regression test for a bug
that was fixed in module_qual.m. However the bug is still present in
unused_imports.m.
compiler/make_hlds_passes.m:
Give hlds_module.m the extra information it now needs for each item_avail.
Add an XXX for a bug that cannot be fixed right now: the setting of
the status of abstract instances to abstract_imported. (The "abstract"
part is correct; the "imported" part may not be.)
compiler/intermod.m:
compiler/try_expand.m:
compiler/xml_documentation.m:
Conform to the change in hlds_module.m.
compiler/module_qual.m:
Update the documentation of the relationship of this module
with unused_imports.m.
compiler/hlds_data.m:
Document a problem with the status of instance definitions.
compiler/hlds_out_module.m:
Update the code that prints out the module_info to conform to the change
to hlds_module.m.
Print status information about instances, which was needed to diagnose
one of the bugs in unused_imports.m. Format the output for instances
nicer.
compiler/prog_item.m:
Add a convenience predicate.
compiler/prog_data.m:
Remove a type synonym that makes things harder to understand, not easier.
compiler/modules.m:
Delete an XXX that asks for the feature this diff implements.
Add another XXX about how that feature could be improved.
compiler/Mercury.options.m:
Add some more modules to the list of modules on which the compiler
should be invoked with --no-warn-unused-imports.
compiler/*.m:
library/*.m:
mdbcomp/*.m:
browser/*.m:
deep_profiler/*.m:
mfilterjavac/*.m:
Delete unneeded imports. Many of these shadow other imports, and some
are just plain unneeded, as shown by --warn-unused-imports. In a few
modules, there were a *lot* of unneeded imports, but most had just
one or two.
In a few cases, removing an import from a module, because it *itself*
does not need it, required adding that same import to those of its
submodules which *do* need it.
In a few cases, conform to other changes above.
tests/invalid/Mercury.options:
Test the generation of messages about import shadowing on the existing
import_in_parent.m test case (although it was also tested very thoroughly
when giving me the information needed for the deletion of all the
unneeded imports above).
tests/*/*.{m,*exp}:
Delete unneeded imports, and update any expected error messages
to expect the now-smaller line numbers.
configure.ac:
Check whether the installed compiler supports `:- pragma external_pred'.
library/backjump.m:
library/builtin.m:
library/exception.m:
library/par_builtin.m:
library/profiling_builtin.m:
library/table_builtin.m:
library/thread.m:
Replace `:- external' with `:- pragma external_pred'.
Replace (C->T;E) with (if C then T else E).
Fix indentation and white space.
library/library.m:
library/thread.future.m:
library/thread.m:
Add new future standard library module.
NEWS:
Announce the new addition.
library/thread.semaphore.m:
Add an impure interface to thread.semaphore.m. Semaphores are used to
implement our other concurrency primitives and an impure interface can
often be useful to implement things such as futures, which don't require
IO state threading. The impure interface predicate names are prefixed
with "impure_".
library/thread.semaphore.m:
NEWS:
Deprecate the impure init/1 function.
library/thread.mvar.m:
Conform to changes in semaphore.m.
benchmarks/progs/mandelbrot/mandelbrot.m:
Add future example to mandelbrot benchmark.
If threads are blocked while there is work in the queue extra threads may be
spawned to keep the processors busy.
Beginning now, tasks created with thread.spawn are use the thread pool.
(thread.spawn_native does not use the thread pool.)
java/runtime/Semaphore.java:
Wrap Java's Semaphore class which call the current thread's blocked()
and running() methods when a thread blocks and then runs after being
blocked.
library/thread.semaphore.m:
Use our own Semaphore class.
java/runtime/MercuryThread.java:
java/runtime/MercuryWorkerThread.java:
Define blocked() and running() on our threads.
java/runtime/NativeThread.java:
This class is used by spawn_native/4 and is required to define blocked()
and running(), however it implements them as no-ops as it isn't included
in the thread pool.
java/runtime/ThreadStatus.java:
Define the BLOCKED status.
java/runtime/MercuryThreadPool.java:
Count blocked threads seperatly and allow the creation of new threads
when existing threads become blocked.
Add some tracing code to help debug the thread management code. This is
disabled by default.
library/thread.m:
Implement spawn for Java using the thread pool. This was not enabled
earlier because without using java/runtime/Semaphore.java it was
possible to deadlock the system.
java/runtime/Task.java:
Add some tracing code to debug thread state changes, this is disabled by
default.
library/thread.m:
The C functions ML_{incr,decr}_thread_barrier_count are declared
with static linkage; their containing foreign_decl pragma should
be marked as local.
Thread pools are often used to reduce poor performance on embarrassingly
parallel work loads. This isn't immediately necessary on the Java backend
however I've implemented it because:
+ It will be required when we implement parallel conjunctions for the
Java backend.
+ I want to implement parallel profiling on the Java backend and don't
want to have to implement this once now, and then re-implement it after
introducing thread pools later.
We want the thread pool to generally restrict the number of threads that are
in use, this reduces overheads. However, when one or more tasks become
blocked then it can be desirable to create extra threads, this helps ensure
that all processors are kept busy and that thread pooling doesn't contribute
to any deadlocks itself. The implementation is a work in prograss and
currently does not implement this second feature.
Java's API provides several different thread pools, see
java.util.concurrent.Executors, none of which are suitable. Specifically
the fixed thread pool is unsuitable as we want to be able to temporarily
exceed the normal number of threads as explained above; and the cached
thread pools, which are also very similar to the ThreadPoolExecutor class,
do not implement the correct algorithm for determining when a new thread
should be created (they can still perform poorly for embarassingly parallel
workloads). Additionally we cannot instrument this code as easily for
parallel profiling.
These changes alter the behaviour of Mercury threads on the Java backend in
two ways, they now behave more correctly and more like threads on the C
backends.
+ If a thread throws an exception it is now reported and the program is
aborted. Previously it was ignored and let pass to the Java runtime
where I assume it was reported.
+ The program now exits only after all threads have exited.
The ThreadPool will automatically detect the number of threads to use, or if
the -P flag is given in the MERCURY_OPTIONS environment variable it will
honor that.
java/runtime/MercuryThread.java:
java/runtime/MercuryThreadPool.java:
java/runtime/MercuryWorkerThread.java:
java/runtime/Task.java:
java/runtime/ThreadStatus.java:
These new classes make up the thread pool. A MercuryThread is an
abstract class for Mercury threads, MercuryWorkerThread is a concrete
subclass of MercuryThread which includes the worker thread behaviour.
A Task is a computation/closure that has not yet been started, it
provides some methods not available in Java's generic Runnable and
Callable classes. The others should be self-explanatory and all files
contain documentation.
java/runtime/Getopt.java:
java/runtime/MercuryOptions.java:
Parse the MERCURY_OPTIONS environment variable for the -P flag.
java/runtime/JavaInternal.java:
Add support for handling Mercury exceptions, this is used in case a
worker thread's task (a Mercury thread) throws an exception.
compiler/mlds_to_java.m:
The main method of the main Java class of an application now starts and
uses the thread pool to execute main/2.
library/exception.m:
Export exception reporting code to the Java runtime system.
library/thread.m:
Use the thread pool for thread.spawn.
library/benchmarking.m:
library/bit_buffer.m:
library/exception.m:
library/gc.m:
library/io.m:
library/library.m:
library/thread.m:
library/time.m:
Add `thread_safe' or `not_thread_safe' to foreign procs
where it is clear which is correct.
The bug avoided by the preceding commit is that the Erlang backend can
produce code for cc_multi_equal(!IO) which refers to a non-existent
dummy IO variable.
library/thread.m:
Replace `cc_multi_equal' by a disjunction to convince the
compiler that `spawn_context_2' is non-deterministic.
Delete the Erlang foreign proc.
Most backends already mapped Mercury threads to "native" threads in spawn/3,
but it was and remains an implementation detail. spawn_native provides
that behaviour as a documented feature for programs which require it,
including for the low-level C backend.
While we are at it, add a `thread' handle type. It currently holds a
thread identifier (not yet formally exported), but it may also have
other uses such as a handle for a `thread.join' predicate, or a place to
hold result values or uncaught exceptions.
library/thread.m:
Add abstract type `thread'.
Add can_spawn_native.
Add spawn_native/4. It can report failure to start a thread,
which was missing from the spawn/3 interface.
Add spawn/4 to match spawn_native/4, without the native thread
requirement.
Make ML_create_exclusive_thread wait for a success code from
the new thread before continuing.
Reduce accessibility levels in C# and Java helper classes.
runtime/mercury_thread.c:
Make MR_init_thread_inner and MR_setup_engine_for_threads
return errors instead of aborting on failure.
tests/hard_coded/Mercury.options:
tests/hard_coded/Mmakefile:
tests/hard_coded/spawn_native.exp2:
tests/hard_coded/spawn_native.exp:
tests/hard_coded/spawn_native.m:
Add test case.
NEWS:
Announce change.
This change allows Mercury engines (each in a separate OS thread) to be
created and destroyed dynamically in low-level C grades.
We divide Mercury engines into two types:
"Shared" engines may execute code from any Mercury thread.
Shared engines may steal work from other shared engines, so are also
called work-stealing engines; we do not have shared engines that
refrain from work-stealing.
"Exclusive" engines execute code only for a single Mercury thread.
Only exclusive engines may be created and destroyed dynamically so far.
This assumption could be lifted when and if the need should arise.
Exclusive engines are a means for the user to map a Mercury thread directly
to an OS thread. Calls to blocking procedures on that thread will not block
progress in arbitrary other Mercury threads. Foreign code which depends on
the OS thread-local state is usable when called from that thread.
We do not yet allow shared engines to steal parallel work from exclusive
engines.
runtime/mercury_wrapper.c:
runtime/mercury_wrapper.h:
Rename MR_num_threads to MR_num_ws_engines. It counts only
work-stealing engines. Move comment to the header file.
Add MR_max_engines. The default value is arbitrary.
Add MERCURY_OPTIONS `--max-engines' option.
Define MR_num_ws_engines and MR_max_engines only with
MR_LL_PARALLEL_CONJ.
runtime/mercury_context.c:
runtime/mercury_context.h:
Rename MR_num_idle_engines to MR_num_idle_ws_engines.
It only counts idle work-stealing engines.
Extend MR_spark_deques to MR_max_engines length.
Extend engine_sleep_sync_data to MR_max_engines length.
Add function to index engine_sleep_sync_data with optional bounds
checking.
Replace instances of MR_num_threads by MR_num_ws_engines or
MR_max_engines as appropriate.
Add MR_ctxt_exclusive_engine field.
Rename existing MR_Context fields to remove the implication that the
engine "owns" the context. The new exclusive_engine field does
imply a kind of ownership, hence potential confusion.
Rename MR_SavedOwner, too.
Make MR_find_ready_context respect MR_ctxt_exclusive_engine.
Make MR_schedule_context respect MR_ctxt_exclusive_engine.
Rename MR_try_wake_an_engine to MR_try_wake_ws_engine
and restrict it to work-stealing engines.
Rename MR_shutdown_all_engines to MR_shutdown_ws_engines
and restrict it to work-stealing engines.
Make try_wake_engine and try_notify_engine decrement
MR_num_idle_ws_engines only for shared engines.
In MR_do_idle, make exclusive engines bypass work-stealing
and skip to the sleep state.
In MR_do_sleep, make exclusive engines ignore work-stealing advice
and abort the program if told to shut down.
Assert that a context with an exclusive_engine really is only loaded
by that engine.
In MR_fork_new_child, make exclusive engines not attempt to wake
work-stealing engines. Its sparks cannot be stolen anyway.
Make do_work_steal fail the attempt for exclusive engines.
There is one call where this might happen.
Add notes to MR_attempt_steal_spark. Its behaviour is unchanged.
Replace a call to MR_destroy_thread by MR_finalize_thread_engine.
Delete MR_num_exited_engines. It was unused.
runtime/mercury_thread.c:
runtime/mercury_thread.h:
Delete MR_next_engine_id and MR_next_engine_id_lock. We can no longer
allocate engine ids by incrementing a counter. Engine ids need to be
reused as they act as indices into fixed-sized arrays.
Extend MR_all_engine_bases to MR_max_engines entries.
Add MR_all_engine_bases_lock to protect MR_all_engine_bases.
Add MR_highest_engine_id.
Add MR_EngineType with the two options described.
Split the main part of MR_init_engine into a new function which
accepts an engine type. MR_init_engine is used by generated code so
maintain the interface.
Factor out setup/shutdown for thread support.
Make MR_finalize_thread_engine call the shutdown function.
Specialise MR_create_thread into MR_create_worksteal_thread.
The generic form was unused.
Move thread pinning into MR_create_worksteal_thread as other threads
do not require it.
Delete MR_destroy_thread. Its one caller can use
MR_finalize_thread_engine.
Delete declaration for non-existent variable
MR_init_engine_array_lock.
runtime/mercury_engine.c:
runtime/mercury_engine.h:
Add MR_eng_type field.
Make MR_eng_spark_deque a pointer to separately-allocated memory.
The reason is given in MR_attempt_steal_spark.
Add MR_ENGINE_ID_NONE, a dummy value for MR_ctxt_exclusive_engine.
Delete MR_eng_owner_thread which was obsoleted by engine ids
before.
Delete misplaced declaration of MR_all_engine_bases.
runtime/mercury_memory_zones.c:
Replace MR_num_threads by appropriate counters (I hope).
runtime/mercury_memory_handlers.c:
runtime/mercury_par_builtin.h:
Conform to changes.
runtime/mercury_threadscope.c:
Conform to renaming (but it might be wrong).
library/thread.m:
Add hidden predicate `spawn_native' for testing.
The interface is subject to change.
Share much of the code with the high-level C backend.
library/par_builtin.m:
Delete `num_os_threads' as it is unused.
doc/user_guide.texi:
Document MERCURY_OPTIONS `--max-engines' option.
Mission Critical IT has maintained a library of code for concurrent
programming. We're happy to contribute this upstream to the Mercury
project starting with this module implementing barriers.
library/thread.barrier.m:
Add the new module implementing barriers.
library/thread.m:
library/library.m:
Add new module.
NEWS:
Announce the new module.
library/thread.semaphore.m:
Add a comment.
These changes fix a couple of performance bugs and also modify the
algorithms in the RTS so that they match the ones in my thesis.
runtime/mercury_context.[ch]:
An engine with a dirty context will no-longer check for a runnable
context first. It first checks for a local spark, if the spark is not
compatible it puts the spark back on the local spark stack. Then it
saves the dirty context and jumps to MR_idle - the entry point for
engines with no contexts.
Remove the MR_MAYBE_TRAMPOLINE macro and expand out any case where it
was previously used.
We no longer execute a spark when an engine has a dirty incompatible
context. (previously we saved the old context then allocated a new
one). Therefore prepare_engine_for_spark() no-longer needs the
join_label parameter (which was used when saving a dirty context).
Consequently, the same parameter has been removed from
MR_do_steal_spark.
If a work stealing thief looses a race then it retries until it wins or
there is no work.
Use a mutex rather than a (binary) semaphore to protect the engine sleep
sync structure. This more directly reflects the intentions plus POSIX
mutexes don't always make kernel calls but semaphores do.
The MR_num_idle_engines global was not being updated correctly, in
particular it showed that there were idle engines even when there
weren't. This caused an engine creating a spark to always attempt to
notify other engines of the spark. Fixing the use of
MR_num_idle_engines improves performance by over a factor of 2x in the
naive fibs micro benchmark.
Refactor MR_join_and_continue to match the simplier structure in my
thesis.
Rename MR_destroy_context to MR_release_context, which gives a more
accurate impression.
Update some MR_assert calls that where incorrect.
runtime/mercury_engine.c:
runtime/mercury_par_builtin.c:
Conform to MR_release_context.
library/thread.m:
Conform to MR_release_context.
Add a missing MR_save_context.
Branches: main
Fix a memory leak in thread.spawn in high-level C grades.
library/thread.m:
Call MR_finalize_thread_engine after finishing the thread goal in
ML_thread_wrapper.
runtime/mercury_thread.c:
Call MR_destroy_engine in MR_finalize_thread_engine.
(The XXX is from year 2000 -- a lot of things have changed since then.
If the problem is still present, we should fix it.)
