Since the compiler now refuses to allow debugging in parallel grades,
tests that require debugging need to be skipped over in low-level
parallel grades or deleted.
tests/hard_coded/Mmakefile:
Disable 'parse' test in parallel grades as it happens to link with
debug libraries.
tests/par_conj/Mercury.options:
tests/par_conj/Mmakefile:
tests/par_conj/par_ddeath.exp:
tests/par_conj/par_ddeath.m:
tests/par_conj/par_ddeath_2.exp:
tests/par_conj/par_ddeath_2.m:
Delete par_ddeath and par_ddeath_2 tests that once triggered a
compiler abort with --trace deep in parallel grades.
I don't think there is much value in keeping them around.
tests/valid/Mmake.valid.common:
Skip LLDS_PROGS (i.e. test cases that required debugging) in
parallel grades.
tests/valid/Mercury.options:
tests/valid/Mmakefile:
tests/valid/untuple_bug.m:
Delete untuple_bug. It tests an old bug in the untupling
transformation, which was more a programming exercise than something
that should be used anyway.
library/io.m:
As above. They were already marked as obsolete.
NEWS:
Mention the change.
tests/hard_coded/remove_file.m:
tests/hard_coded/utf8_io.m:
tests/par_conj/dep_par_24.m:
tests/par_conj/dep_par_24b.m:
tests/tabling/mercury_java_parser_dead_proc_elim_bug.m:
tests/tabling/mercury_java_parser_dead_proc_elim_bug2.m:
tests/valid/mercury_java_parser_follow_code_bug.m:
Replace references to the deleted predicates.
This first step deals with the consequences of such removal.
The removal itself will happen in stage 2. That step will
add "is" to the prolog module in the library.
compiler/add_pred.m:
Prepare for "is" being in the prolog module.
compiler/options.m:
Add a way to test whether the change to add_pred.m is in the
installed compiler.
tests/accumulator/base.m:
tests/accumulator/call_in_base.m:
tests/accumulator/chain.m:
tests/accumulator/commutative.m:
tests/accumulator/construct_test.m:
tests/accumulator/dcg.m:
tests/accumulator/deconstruct_test.m:
tests/accumulator/disj.m:
tests/accumulator/func.m:
tests/accumulator/heuristic.m:
tests/accumulator/highorder.m:
tests/accumulator/identity.m:
tests/accumulator/inter.m:
tests/accumulator/nonrec.m:
tests/accumulator/out_to_in.m:
tests/accumulator/qsort.m:
tests/accumulator/simple.m:
tests/accumulator/split.m:
tests/accumulator/swap.m:
tests/benchmarks/cqueens.m:
tests/benchmarks/crypt.m:
tests/benchmarks/deriv.m:
tests/benchmarks/deriv2.m:
tests/benchmarks/nrev.m:
tests/benchmarks/poly.m:
tests/benchmarks/primes.m:
tests/benchmarks/qsort.m:
tests/benchmarks/query.m:
tests/benchmarks/tak.m:
tests/debugger/interactive.m:
tests/declarative_debugger/Mercury.options:
tests/declarative_debugger/io_read_bug.m:
tests/declarative_debugger/queens.exp:
tests/declarative_debugger/queens.m:
tests/dppd/imperative_solve_impl.m:
tests/dppd/map_impl.m:
tests/dppd/max_length_impl.m:
tests/dppd/sum.m:
tests/dppd/upto_sum_impl.m:
tests/par_conj/dep_par_21.m:
tests/tabling/seq.m:
tests/term/dds3_14.m:
tests/term/mmatrix.m:
tests/term/money.m:
tests/term/occur.m:
tests/term/pl4_5_2.m:
tests/term/queens.m:
tests/typeclasses/inference_test.m:
tests/typeclasses/inference_test_2.m:
tests/valid/lazy_list.m:
tests/warnings/duplicate_const.m:
Replace calls to "is" with unifications. In many places,
bring programming style up to date.
My commit afe2887882 broke the ability
to run the test suite outside of a bootcheck by executing "mmake runtests"
in the tests directory. This diff fixes that.
tests/Mmake.common:
Don't define "TESTS_DIR = ..". While every single tests/*/Mmakefile
defined it as such, I overlooked the fact that tests/Mmakefile itself
defined it ".", referring to the same directory from a different starting
point. Document this easily-overlooked fact.
Rename the old runtests target, which after afe2887 runs the tests
in a single directory, as runtests_dir, to leave the target name
"runtests" itself free for tests/Mmakefile to use.
tests/Mmakefile:
Define "TESTS_DIR = .", and add a target "runtests" which invokes
"mmake runtests_dir" in each test directory.
tools/bootcheck:
Invoke "mmake runtests_dir" instead of "mmake runtests" in each
test directory.
Initialize a variable just before it is used.
tests/*/Mmakefile:
Add back the definition "TESTS_DIR = .."
A long time ago, test directories such as hard_coded had subdirectories
such as hard_coded/typeclasses. These have since been flattened out
(e.g. hard_coded/typeclasses is now just typeclasses), but there were
still remnants of the old approach. This diff deletes those remnants.
tests/*/Mmakefile:
Delete the TESTS_DIR and the SUBDIRS mmake variables; TESTS_DIR
was always set to "..", and SUBDIRS to the empty string.
Delete any references to the make variable NOT_WORKING, since
it is never used.
tests/Mmake.common:
Document that Mmakefiles in test directories don't have to set
TESTS_DIR and SUBDIRS anymore. Fix the formatting of the documentation
of the make variables they do still have to set.
Delete the targets and actions for handling subdirectories of
test directories, since there aren't any.
tests/Mmakefile:
Simplify some code.
tests/benchmarks/Mmakefile:
Delete variables only used with the IL or GCC back-ends.
tests/debugger/Mmakefile:
tests/declarative_debugger/Mmakefile:
tests/par_conj/Mmakefile:
Delete references to the IL backend.
tests/par_conj/dep_par_10.m:
Add missing C# and Java foreign procs.
tests/dppd/Mmakefile:
Run these tests in the Java grade.
tests/general/Mmakefile:
Delete a workaround for OSF/1 -- we no longer support it.
tests/hard_coded/Mmakefile:
Delete the unused list of tests that will pass in the Java grade.
Delete a reference to the IL backend.
tests/Mmake.common:
Don't invoke any actions in the clean_local and realclean_local
targets, since if using mmc --make, the builtin mmake rules
have actions for those targets as well, and make can't handle
more than one action for a target having actions. Replace those
actions with dependencies on other, unique targets that have
the actions instead.
tests/*/Mmakefile:
Avoid actions in clean_local and realclean_local targets the same way.
Sort the test names in some directories that didn't already do so.
Delete some obsolete comments.
Fix style.
tests/valid/Mmake.valid.common:
As for the Mmakefiles above, and also move the definition of a make
variable before it is needed.
tests/Mmake.common:
Replace the -j1 in the runtests_local target used by all the test
directories with $(MAYBE_J1).
tests/*/Mmakefile:
Define MAYBE_J1 it as the empty string in test directories in which
different tests don't share source files.
Define MAYBE_J1 as -j1 in test directories in which
different tests do share source files.
tests/submodules/sub2_a.m:
Add this copy of sub_a.m to allow tests in the submodules directory
to be done in parallel.
tests/submodules/accessibility2.m:
Import sub2_a.m instead of sub_a.m.
tests/warnings/ambig_types_high_level.m:
Add this copy of ambig_types.m to allow tests in the warnings directory
to be done in parallel.
tests/warnings/ambig_high_level.m:
Import ambig_types_high_level.m instead of ambig_types.m.
tests/invalid/*.{m,err_exp}:
tests/misc_tests/*.m:
tests/mmc_make/*.m:
tests/par_conj/*.m:
tests/purity/*.m:
tests/stm/*.m:
tests/string_format/*.m:
tests/structure_reuse/*.m:
tests/submodules/*.m:
tests/tabling/*.m:
tests/term/*.m:
tests/trailing/*.m:
tests/typeclasses/*.m:
tests/valid/*.m:
tests/warnings/*.{m,exp}:
Make these tests use four-space indentation, and ensure that
each module is imported on its own line. (I intend to use the latter
to figure out which subdirectories' tests can be executed in parallel.)
These changes usually move code to different lines. For the tests
that check compiler error messages, expect the new line numbers.
browser/cterm.m:
browser/tree234_cc.m:
Import only one module per line.
tests/hard_coded/boyer.m:
Fix something I missed.
This file-specific setting will override a default setting of expandtabs
in $HOME/.vimrc.
*/Makefile:
*/Mmakefile:
As above.
tests/hard_coded/.gitignore:
Don't ignore the purity subdir. This ignore must have been left over
from when purity.m was a test in hard_coded, not hard_coded/purity,
and it ignored an executable, not a directory.
Branches: main, 11.07
Make the none.gc.memprof grade work with MSVC again.
Avoid more warnings when compiling with MSVC.
compiler/layout_out.m:
Avoid an incomplete type in the declaration of the alloc_sites
array. (Similar changes need to be made for the layout structures
related to debugging and deep profiling - I am in the process of
testing the former, the latter doesn't currently work on Windows
any way.)
browser/listing.m:
Use don't-care variables in some foreign_procs.
This avoids warnings about assignments from uninitialized
variables with MSVC.
*/.cvsignore:
Update cvsignore entries.
Ignore files generated by mprof.
Estimated hours taken: 3
Branches: main
compiler/par_loop_control.m:
Add draft support for copying shared variables to child stacks
in order to enable tail recursion.
conjunction transformation we may push a wait into a callee that doesn't
actually make use of it's argument on some paths. In these cases we insert
a wait signal at the end of the procedure so that the caller can assume that
any future passed in an argument is waited on (and therefore it doesn't need
to use the wait operation). This can be a problem with the wait that was
inserted is optimised away because it is a pure, deterministic goal that
does not bind any variables.
This change makes the future_wait/2 operation impure to prevent this.
compiler/dep_par_conj.m:
Make all calls to wait_future/2 impure, preventing them from being
optimized away.
library/par_builtin.m:
Make wait_future/2 impure.
tests/par_conj/impure_wait.m:
tests/par_conj/impure_wait.exp:
tests/par_conj/Mmakefile:
Add a test case for this problem.
Bug 144 is a pathological case where right-recursion is used in a parallel
conjunction and the conjuncts cannot be re-ordered. This can cause excess
stack allocation and abysmal performance. The --max-contexts-per-thread
runtime option is used to reduce the impact of these cases by reducing the
amount of parallelism gained at runtime.
Bug 171 is a simple case where the threadscope grade could not be compiled
without enabling the Boehm garbage collector.
runtime/mercury_threadscope.c:
Enclose boehm GC specific code within #ifdef MR_BOEHM_GC
runtime/mercury_context.[ch]:
Record the number of contexts running or suspended at any time in a new
variable, MR_num_outstanding_contexts
Remove counts of other in-use objects such as the sum of outstanding
contexts and sparks.
Remove two granularity control macros that haven't been used for some time.
compiler/granularity.m:
Ensure that the runtime granularity decision is updated for when it is
available.
library/par_builtin.m:
Remove granularity decisions for which support has been removed in the
runtime.
tests/par_conj/Mmakefile:
tests/par_conj/pathological_right_recursion.{m,exp}:
Add a test case for bug 144.
Branches: main
Add support for the `csharp' grade to `mmc --make', and make it possible to
install the `csharp' grade with `mmake install'.
Also some miscellaneous fixes.
configure.in:
Require a recent enough bootstrap compiler that recognises C# as a
language for `pragma foreign_type'.
Mmakefile:
Use `mmc --make' to install the standard library in csharp grade.
aclocal.m4:
Search for the Mono C# compiler `gmcs', which is required for generics
at this time. Prefer it over the DotGNU C# compiler, which I have not
tested.
Search for `mono'. If found, it will be used in shell scripts to
launch executables generated via the csharp backend.
Remove "MS_" prefixes on the variables MS_CSC and MS_ILASM, which are
not Microsoft-specific. More importantly, it should be less likely to
make the mistake of adding an extra underscore to CSCFLAGS and
ILASMFLAGS.
README.DotNet:
Conform to variable renamings.
compiler/compile_target_code.m:
Add new linked target types `csharp_executable', `java_launcher' and
`erlang_launcher', instead of overloading `executable'.
Link with `mer_std.dll' and other libraries when generating C#
executables. There is no `mer_rt.dll'.
Pass "/debug" to the C# compiler if `--target-debug' is set.
Create a shell script to launch the executable if necessary.
Delete an unused predicate `standard_library_directory_option'.
compiler/file_names.m:
`.cs' and `.cs_date' are grade-dependent.
compiler/handle_options.m:
Force `.exe' as the executable file extension in csharp grades.
Make the `erlang' grade component imply the same options as MLDS
grades.
compiler/make.m:
Classify executable target types based on the compilation target.
compiler/make.module_target.m:
Handle `mmc --grade csharp --make <target>.dll'.
compiler/make.program_target.m:
Install library DLLs in csharp grades.
Make clean targets remove files for csharp grades.
Conform to changes.
compiler/make.util.m:
Add a stub foreign type.
Conform to changes.
compiler/module_cmds.m:
Factor out code to generate the shell scripts which launch programs
compiled in Java, Erlang and C# grades.
compiler/options.m:
Add `cli_interpreter' option to remember the name of the program which
should be used to run CLI (.NET) programs.
Add C#-related options to the help message.
compiler/options_file.m:
Remove "MS_" prefixes on MS_ILASM_FLAGS and MS_CSC_FLAGS, and remove
the extra underscore before "FLAGS". In all uses of the variables,
they were spelt without the extra underscore.
doc/user_guide.texi:
Document options and file types related to the C# grade.
library/Mmakefile:
Pass `mercury_dotnet.cs' to the C# compiler when building the standard
library. Suppress some warnings.
Allow stubs in this directory for csharp grade.
Conform to variable renamings.
library/builtin.m:
Uncomment foreign language pragmas for C#.
Handle null values in C# implementation of `deep_copy'.
library/private_builtin.m:
library/string.m:
Compare strings by ordinals in C#, instead of culture-specific rules.
Although the latter is allowed according to the documentation, it is
likely to slower, and cause confusion when porting between backends.
Handle negative index in string.set_char.
library/rtti_implementation.m:
Uncomment foreign language pragmas for C#.
`System.Type.GetType' only searches the current executing assembly or
in mscorlib for a type. As we have to be able to find types in other
assemblies (e.g. mer_std.dll or user DLLs), explicitly search through
a list of assemblies.
library/thread.semaphore.m:
Uncomment foreign language pragmas for C#.
Fix missing class qualification.
library/array.m:
library/bitmap.m:
library/bool.m:
library/dir.m:
library/exception.m:
library/io.m:
library/mutvar.m:
library/par_builtin.m:
library/region_builtin.m:
library/store.m:
library/thread.m:
library/time.m:
library/univ.m:
library/version_array.m:
Uncomment foreign language pragmas for C#.
mdbcomp/rtti_access.m:
Add type and procedure stubs.
runtime/mercury_dotnet.cs.in:
Override `Equals(object)' methods in `TypeCtorInfo_Struct' and
`TypeInfo_Struct' classes. This requires we override `GetHashCode' as
well.
Handle nulls arguments to `Equals' methods as is the expected behaviour.
Override `ToString' in `TypeCtorInfo_Struct' to produce more useful
output during debugging.
scripts/Mercury.config.in:
Record the configured CLI_INTERPRETER and pass that to the compiler as
a flag.
Conform to variable renamings.
scripts/Mmake.vars.in:
Pass value of CSCFLAGS from Mmake through to `mmc --make'.
Conform to variable renamings.
scripts/Mercury.config.bootstrap.in:
scripts/Mmake.rules:
Conform to variable renaming.
scripts/canonical_grade.sh-subr:
scripts/final_grade_options.sh-subr:
scripts/init_grade_options.sh-subr:
scripts/parse_grade_options.sh-subr:
Canonicalise high-level code, high-level-data, C# target code to the
`csharp' grade.
Handle erlang grades like other grades.
scripts/prepare_install_dir.in:
Copy `.cs' files from the runtime directory when preparing an install
directory.
browser/Mmakefile:
compiler/Mmakefile:
deep_profiler/Mmakefile:
mdbcomp/Mmakefile:
profiler/Mmakefile:
runtime/Mmakefile:
slice/Mmakefile:
ssdb/Mmakefile:
trace/Mmakefile:
Do as other non-C grades in this directory.
Conform to variable renamings.
tests/hard_coded/foreign_enum_dummy.m:
tests/hard_coded/sub-modules/non_word_mutable.m:
tests/hard_coded/sub-modules/sm_exp_bug.m:
Make these tests work in C#.
tests/mmc_make/Mmakefile:
Update a regular expression to account for `mmc --make' writing
"Making rebuild.exe" on platforms where the .exe suffix is not normally
used.
tests/mmc_make/complex_test.exp2:
Add alternative output (minor difference in floating point precision).
tests/debugger/Mmakefile:
tests/debugger/declarative/Mmakefile:
tests/general/structure_reuse/Mmakefile:
tests/hard_coded/Mmakefile:
tests/hard_coded/sub-modules/Mmakefile:
tests/par_conj/Mmakefile:
tests/stm/Mmakefile:
Disable some tests in the csharp grade.
tests/invalid/Mmakefile:
Disable some tests in the csharp grade.
Enable a test which should work in java grades.
tests/valid/Mmakefile:
Do as other non-C grades in this directory.
When testing the csharp grade in this directory, produce only the C#
target files for now.
tests/run_one_test:
Don't compress a failing test case executable when the executable is
actually only a shell script.
compiler/par_conj_dep.m:
Check the determinism of goals before inserting a signal in them of testing
if we should insert a signal in them.
Create a new constructor symbol in the cost_after_signal type that is
returned by should_we_push_signal when the code that we would normally
insert a signal into is unreachable.
Use an instantiation sub type to list the values of cost_after_signal that
are valid inputs to the should_we_push_signal predicate.
In should_we_push_signal: Replace some if-then-else goals with switches,
this ensures that the determinism checker can alert us when there are
uncovered values in the switches.
Merge the results of should_we_push_signal after branching code.
tests/par_conj/bug_130_unreachable.m:
This was originally bug_130.m except that it doesn't test for bug 130 but a
similar bug with the same symptom.
Simplified this test case, this still triggers the bug in unpatched
compilers.
tests/par_conj/bug_130_unreachable.exp:
The expected output for bug_130_unreachable.
tests/par_conj/bug_130_should_push_signal.m:
tests/par_conj/bug_130_should_push_signal.exp:
Discovered another bug when should_we_push_signal/4 seems to get confused
around erroneous code.
tests/par_conj/bug_130.m:
tests/par_conj/bug_130.exp:
Removed this test case since it's a duplicate of bug_130_unreachable.
This change adds a test case for bug_130 and modifies the dependant parallel
conjunction transformation to ensure that the test case passes. This may have
not fixed the bug as the test case might be triggering a different bug than the
one reported.
compiler/dep_par_conj.m:
Don't perform the dependant parallelisation transformation when the instmap
after the conjunction is 'unreachable', make these sequential conjunctions.
Add an extra trace goal to dump the HLDS of the procedure before and after
the dependant parallel conjunction transformation.
Improve the output of the existing trace goal by making it also print the
pred and proc IDs of the procedure it is about to specialise and that this is
being printed out before specialisation.
tests/par_conj/Mmakefile:
tests/par_conj/bug_130.exp:
tests/par_conj/bug_130.m:
Add a regression test for bug 130.
Estimated hours taken: 0.2
Branches: main
tests/*/Mmakefile:
Sort the lists of tests in each directory. (We usually keep each list
sorted, but then we concatenate several lists, which loses the order.)
This gives you a better idea of how far a bootcheck still has to go.
tools/bootcheck:
Build the interface files before the rest of the files in the library
directory. Again, this gives you a better idea of how far a bootcheck
still has to go.
Make the dependencies for the slice subdirectory of the stage2 at the
same time as the other subdirs; don't let it be caught later.
library/Mmakefile:
Build the interface files in the right order.
Estimated hours taken: 1
Branches: main
compiler/dep_par_conj.m:
Fix a bug that caused us to insert signal operations after goals even
when those goals generated only *tentative*, not *final*, values for
shared variables.
tests/par_conj/produce_in_nondet_disj.{m,exp}:
New test case to check for the bug.
tests/par_conj/Mmakefile:
Enable the new test case.
Estimated hours taken: 32
Branches: main
Implement a new option --allow-some-paths-only-waits, that tells dep_par_conj.m
that it is ok to insert a wait for a variable along some but not all execution
paths in a branched control structure such as a switch or if-then-else. If this
happens, then we need to wait separate for any uses of the variable in later
code, and we rename the consumed variable in the branched control structure
to avoid defining it twice.
The effect of the option is shown by this partial HLDS dump from after
the dep_par_conj transformation for the consume_in_some_branches_and_after
test case:
(if
consume_in_some_branches_and_after.test(Y_5)
then
- A_7 = Y_5
+ ( % conjunction
+ A_7 = Y_5
+ ,
+ par_builtin.wait(FutureX_8, X_4)
+ )
else
( % conjunction
- par_builtin.wait(FutureX_8, X_9)
+ par_builtin.wait(FutureX_8, X_4)
,
- A_7 = X_9
+ A_7 = X_4
)
)
,
- par_builtin.wait(FutureX_8, X_4)
- ,
Z_6 = int.(X_4 + A_7)
The lines starting with + are with --no-allow-some-paths-only-waits, while
the lines starting with - are with --allow-some-paths-only-waits.
The advantage of --allow-some-paths-only-waits is that if execution takes
a path through the branched control structure in which the consumed variable
is NOT needed, then we can delay the wait until the next consumer of the
variable, which may be substantially later. The disadvantage is that if
execution takes a path through the branched control structure in which the
consumed variable IS needed, then we will wait for it more than once.
The disadvantage is a small constant factor while the advantage may be
substantial. In absence of further analysis (which could estimate the size
of both the advantage and the disadvantage), our best guess is that the
advantage wins.
compiler/options.m:
Add the new option.
compiler/dep_par_conj.m:
Implement the new option.
tests/par_conj/consume_in_some_branches_and_after.m:
Replace a unification with a call to allow the procedure containing
that code to serve as a simple test case of the option (the one
shown above). The computation being described remains unaffected.
Estimated hours taken: 20
Branches: main
Fix an old bug in the transformation that implements dependent parallel
conjunctions. The bug occurred when some arms of a branched control structure
consumed a shared variable, and some arms didn't. For code like this
q(X::in, Y::in, Z::out) :-
( Y = 2 ->
A = Y
;
A = X
),
Z = X + A
where X is a shared variable in a conjunction that calls q, the transformation
yielded code that waited for the future version of X only in the else branch,
not in the then branch. Since each wait was the producer of X from FutureX,
this led directly to a mode inconsistency between the branches, and hence
to a compiler crash.
The main part of the fix is to make sure that the code that inserts waits into
branched goals always inserts the wait either into every branch (if the
variable is consumed by at least one branch), or into no branch (if no branch
consumes the variable).
The other part of the fix is to replace the code which tried to ensure,
in a general sort of way, the invariant that the conjuncts of a parallel
conjunction should not have any of their shared variables in their nonlocal
set, with code that directly ensures that shared variables are always renamed
in their consuming conjuncts. Unlike the earlier approach, this approach
generates code that is obviously correct by construction. It also takes
much less code to implement, and should take less compilation time.
compiler/dep_par_conj.m:
Implement the fixes above.
Separate out the different parts of the transformation. For each
part, pass around only the information that part requires, document
which parts of this information are read-only. Give more descriptive
names to each function, including using a distinct set of prefixes
for different parts of the transformation.
Improve the documentation.
Remove the code that used to handle the case of grades that do not
support parallel conjunctions, since it cluttered up other code.
compiler/mercury_compile.m:
Check just once whether the grade supports parallel conjunctions.
If not, call the code in a new module parallel_to_plain_conj.
The dep_par_conj pass creates specialized versions of procedures
and redirects calls to them, so it may leave the original versions
without any calls and thus dead, so invoke the dead procedure
elimination pass immediately after it.
Move the final dead procedure elimination pass *after* the dep_par_conj
pass, and after some some other passes (that may also create dead
procedures).
compiler/parallel_to_plain_conj.m:
Downgrade all parallel conjunctions to plain conjunctions.
compiler/transform_hlds.m:
compiler/notes/compiler_design.m:
Add the new module.
compiler/dead_proc_elim.m:
Rename the parameter of the main procedure of this module to avoid
a false dichotomy (the invocation of dead_proc_elim after the
dep_par_conj pass is neither for warnings nor a final optimization).
compiler/goal_util.m:
Give a predicate a more meaningful name. Export a predicate for use by
dep_par_conj.
compiler/pd_util.m:
Rename some predicates to avoid ambiguity.
compiler/deforest.m:
compiler/delay_partial_inst.m:
compiler/inlining.m:
compiler/quantification.m:
Conform to the change to goal_util and/or pd_util.
compiler/simplify.m:
Rename some predicates to avoid ambiguity.
Use state variables to reduce the apparent complexity of some code.
tests/par_conj/consume_in_some_branches.{m,exp}:
tests/par_conj/consume_in_some_branches_and_after.{m,exp}:
tests/par_conj/consume_wait.{m,exp}:
New test cases to test for the bug.
tests/par_conj/Mmakefile:
Enable the new test cases.
Estimated hours taken: 0.2
Branches: main
tests/*/.cvsignore:
tests/*/.nocopyright:
Update or add these files. In particular, CVS should ingore
.mgnuc* files.
Estimated hours taken: 1.5
Branches: main
library/io.m:
Implement some more I/O primitives for Erlang.
tests/debugger/Mmakefile:
tests/debugger/declarative/Mmakefile:
tests/hard_coded/Mmakefile:
tests/invalid/Mmakefile:
tests/par_conj/Mmakefile:
tests/recompilation/Mmakefile:
tests/valid/Mmakefile:
Disable test cases which don't apply to the Erlang backend for
whatever reason.
tests/hard_coded/constraint_order.m:
tests/hard_coded/copy_pred.m:
tests/hard_coded/copy_pred_2.m:
tests/hard_coded/export_test.m:
tests/hard_coded/external_unification_pred.m:
tests/hard_coded/foreign_type3.m:
tests/hard_coded/ho_solns.m:
tests/hard_coded/ho_univ_to_type.m:
tests/hard_coded/intermod_multimode.m:
tests/hard_coded/intermod_poly_mode_2.m:
tests/hard_coded/lookup_disj.m:
tests/hard_coded/multimode.m:
tests/hard_coded/no_inline.m:
tests/hard_coded/pragma_foreign_export.m:
tests/hard_coded/redoip_clobber.m:
tests/hard_coded/rnd.m:
tests/hard_coded/sub-modules/sm_exp_bug.m:
tests/hard_coded/typeclasses/impure_methods.m:
tests/par_conj/dep_par_10.m:
tests/valid/flatten_conj_bug.m:
Fix test cases which were failing with the Erlang for simple reasons,
mostly missing foreign procs or foreign types.
tests/general/string_format_test_2.exp5:
tests/general/string_format_test_3.exp5:
tests/hard_coded/exceptions/test_uncaught_exception.exp5:
tests/hard_coded/no_fully_strict.exp5:
Add expected outputs with stack dumps for Erlang.
Branches: main
Prevent multi-threaded programs from terminating as soon as the main thread
terminates, i.e. the process should not terminate until all threads started by
thread.spawn/3 terminate.
This is done by maintaining a a global count of the number of threads started
by thread.spawn. In low-level C grades the main context will suspend if it
reaches the global_success label and finds there are other contexts still
outstanding. The last context to terminate then reschedules the main context
to resume.
Similarly, in high-level C grades the main thread waits on a condition
variable, which is signalled by the last thread to terminate.
library/thread.m:
runtime/mercury_context.c:
runtime/mercury_thread.c:
runtime/mercury_thread.h:
runtime/mercury_wrapper.c:
As above.
Add some extra assertions related to this.
tests/par_conj/Mmakefile:
tests/par_conj/thread_barrier.exp:
tests/par_conj/thread_barrier.m:
Add test case
NEWS:
Announce the change.
Estimated hours taken: 0.5
Branches: main
tests/par_conj/spawn_many.m:
Fix an implicit assumption in this test case that the child threads
will manage to write to the channel faster than the main thread can
read from it.
tests/par_conj/Mmakefile:
Reduce the peak memory usage of the spawn_many test case.
Estimated hours taken: 10
Branches: main
Change the implementation of thread.spawn in low-level C grades. Previously it
called the thread goal (a closure) via an exported Mercury procedure. The
problem with that is the call counts as a C -> Mercury call, and so goes via
MR_call_engine() which sets up a large stack frame on the Mercury engine's C
stack that won't be deallocated until the thread terminates.
This is unnecessary inefficient since we can call the thread goal directly
instead of via MR_call_engine(). The thread goal won't be returning to a C
foreign proc when it finishes, and we don't need MR_call_engine() reserving
C stack space for Mercury execution as any Mercury engine that is about to
execute the thread goal would have that reserved space already.
library/thread.m:
As above.
tests/par_conj/Mmakefile:
tests/par_conj/spawn_many.exp:
tests/par_conj/spawn_many.m:
Add test case.
Estimated hours taken: 80
Branches: main
This change adds two new passes to the compiler. The first one,
implicit_parallelism, uses deep profiling feedback information, generated by
mdprof_feedback, to introduce parallel conjunctions where it could be
worthwhile. It deals with both independent and dependent parallelism.
The second new pass, distance_granularity, applies a transformation that
controls the granularity of parallelism for recursive procedures using the
distance metric.
This change also fixes a bug in mdprof_feedback regarding the construction of
the list of CSSs.
compiler/implicit_parallelism.m:
New module which uses the profiling feedback file generated by
mdprof_feedback to introduce parallel conjunction where it could be
useful.
compiler/distance_granularity.m:
New module. A program transformation that implements granularity control
of parallel execution using the distance metric.
compiler/dep_par_conj.m:
Moved find_shared_variables into the interface (needed for
implicit_parallelism.m).
compiler/goal_util.m:
Add two new predicates: flatten_conj and create_conj.
compiler/hhf.m:
Delete flatten_conj and use the one of goal_util instead.
compiler/hlds_pred.m:
Add a predicate to set the arity of a predicate (needed for
distance_granularity).
compiler/mercury_compile.m:
Add the calls to apply implicit parallelism and to control granularity
using the distance metric.
compiler/options:
Add implicit-parallelism, feedback-file and distance-granularity options.
compiler/pred_table.m:
Add a predicate to get the next pred_id available (needed for
distance_granularity).
compiler/prog_util.m:
Extend the predicate make_pred_name and the type new_pred_id for
creating a predicate name for distance_granularity.
compiler/transform_hlds.m:
Include implicit_parallelism and distance_granularity.
deep_profiler/mdprof_feedback.m:
Rename distribution to measure.
Add handling of dump_stages and dump_options.
Insert elements into the list of CSSs in the correct order.
deep_profiler/dump.m:
Add "all" option to dump everything out of the Deep.data file.
doc/user_guide.texi:
Add the following options: --distance-granularity, --implicit-parallelism and
--feedback-file.
tests/par_conj/Mercury.options:
tests/par_conj/dg_fib.{m,exp}:
tests/par_conj/dg_fib_func.{m,exp}:
Add two test cases for the distance_granularity module:dg_fib and
dg_fib_func. As things are, we do not check whether the granularity
control transformation using the distance metric is applied correctly or
not. We only check the output of these test cases.
Estimated hours taken: 12
Branches: main
Add an optional pass that tries to avoid generating too many parallel goals.
The first transformation implemented by this pass is to transform parallel
conjunctions into goals of the form
( queues already contain lot of work ->
sequential version of parallel conjunction
;
parallel conjunction as before
)
if they contain recursive calls.
The effect of this transformation is to reduce the overhead of the new par_fib
test case from:
fib(35): sequential 189 vs parallel 5770
to
fib(35): sequential 189 vs parallel 1090
i.e. a speedup of more than a factor of five.
compiler/granularity.m:
New module that implements this transformation. I intend to add other,
more sophisticated transformations in the future.
compiler/transform_hlds.m:
Add granularity.m as one of the submodules of transform_hlds.m.
compiler/mercury_compile.m:
Invoke the new pass.
Invoke dep_par_conj only if needed.
Fix some stage numbers.
compiler/notes/compiler_design.html:
Document the new module.
Document some modules that should have been documented earlier.
Fix a hurried deletion of a reference to the Aditi backend.
compiler/goal_util.m:
Add some utility functions for use by the new module.
compiler/simplify.m:
Record the information mercury_compile.m needs in order to check
whether we have any parallelism for granularity.m and dep_par_conj.m
to process.
compiler/hlds_module.m:
Add a slot to the module_info to record the information from
simplify.m. Clean up some interfaces.
compiler/add_type.m:
Conform to the change to hlds_module.m.
compiler/dependency_graph.m:
Delete unnecessary module qualifications, and rename some predicates
to avoid potential ambiguities.
compiler/options.m:
Add the options required for controlling the new transformation.
Rename an option's internal name to avoid conflict with a language
keyword (the user-visible name remains unchanged).
Move some options around to put them in logical groups.
doc/user_guide.texi:
Document the new options.
Fix some omissions in some earlier options.
compiler/handle_options.m:
compiler/termination.m:
Conform to the option rename.
compiler/quantification.m:
Rename some predicates to avoid ambiguities.
library/par_builtin.m:
Add a predicate for use by the new transformation.
tests/par_conj/par_fib.{m,exp}:
A new test case: a version of fib for use in testing parallelism.
tests/par_conj/Mmakefile:
Enable the new test case.
Estimated hours taken: 0.25
Branches: main
When pushing `signal' calls into predicates we mustn't forget there may be
further references to a signalled variable after the `signal' call. So we
need to insert `get' calls, as we do when pushing `wait' calls.
p(X::out), signal(FutureX, X), q(X, Y), ...
=>
'Parallel__p__[1]'(FutureX), get(FutureX, X), q(X, Y), ...
compiler/dep_par_conj.m:
Fix the bug as above.
tests/par_conj/Mmakefile:
tests/par_conj/dep_par_29.exp:
tests/par_conj/dep_par_29.m:
Add a test case.
Estimated hours taken: 0.5
Branches: main
tests/par_conj/threads_hang.m:
tests/par_conj/Mmakefile:
Set an alarm that aborts this test case if it's taking too long
so as not to hold up the test suite.
Branches: main
Make the dependent parallel conjunction transformation insert calls to
`par_builtin.wait' as late as possible, and calls to `par_builtin.signal' as
soon as possible.
Next, combine contiguous sequences of waits, then a plain call to a
non-imported procedure, then signals (where there must be at least one wait or
signal) into a call to a specialised "parallel" procedure that takes futures
in place of the arguments for which waiting or signalling is required.
The waits and signals are then moved into the parallel procedures where they
can be shifted around more and combined into further calls.
compiler/dep_par_conj.m:
As above.
compiler/hlds_pred.m:
compiler/layout_out.m:
Add dependent_parallel_conjunction as a source of transformed
predicates.
compiler/prog_util.m:
Add parallel_args as a way to build names for "parallel" procedures.
compiler/simplify.m:
Don't let the simplification pass do simplifications of common
information between parallel conjuncts, as they would introduce
dependences between conjuncts which the programmer (and the dependent
parallel conjunction tranformation...) probably did not intend.
library/par_builtin.m:
mdbcomp/program_representation.m:
Add `get/2' which gets the value associated with a future, i.e. it is
like `wait/2' but assumes that the future has been signalled already.
Fix a wrong assertion that a future should only be signalled once.
tests/par_conj/Mercury.options:
tests/par_conj/Mmakefile:
tests/par_conj/dep_par_11d.exp:
tests/par_conj/dep_par_11d.m:
tests/par_conj/dep_par_24.exp:
tests/par_conj/dep_par_24.m:
tests/par_conj/dep_par_24b.exp:
tests/par_conj/dep_par_24b.m:
tests/par_conj/dep_par_25.exp:
tests/par_conj/dep_par_25.m:
tests/par_conj/dep_par_25b.exp:
tests/par_conj/dep_par_25b.m:
tests/par_conj/dep_par_26.exp:
tests/par_conj/dep_par_26.m:
tests/par_conj/dep_par_27.exp:
tests/par_conj/dep_par_27.m:
tests/par_conj/dep_par_28.exp:
tests/par_conj/dep_par_28.m:
tests/par_conj/dep_par_6b.exp:
tests/par_conj/dep_par_6b.m:
Add test cases.
Estimated hours taken: 0.1
Branches: main
tests/par_conj/.mgnuc_opts:
tests/par_conj/.mgnuc_copts:
Add these files as copies of the corresponding files from the other
test directories. Without these, all the test cases were failing
because they couldn't find the workspace's header files.
Estimated hours taken: 0.0
Branches: main
tests/par_conj/Mmakefile:
Disable testing of multiple Mercury engines until I figure out yet
another spurious abort.
Estimated hours taken: 20
Branches: main
Add coroutining support for dependent parallel conjunctions in lowlevel
parallel grades.
library/par_builtin.m:
Change definitions of synchronisation primitives so that waiting on a
future causes the current context to be suspended. Signalling a
future causes all the contexts waiting on the future to be scheduled.
runtime/mercury_context.c:
runtime/mercury_thread.c:
runtime/mercury_thread.h:
runtime/mercury_wrapper.c:
Add a global `MR_primordial_thread' to hold the thread id of the
primordial thread.
Add sanity checks, in particular that the primordial thread does not
exit like other threads as it needs to clean up the Mercury runtime.
tests/par_conj/Mmakefile:
Actually run dependent parallel conjunction tests since they should
no longer deadlock.
tests/par_conj/*.exp:
Add expected outputs for test cases which didn't have them.
Estimated hours taken: 6
Branches: main
compiler/liveness.m:
Fix the deadness pass's handling of parallel conjunctions.
The set of dead variables at the end of a parallel conjunction
should not necessarily include all the variables dead at the
end of the goal that the parallel conjunction is in.
This was causing problems with later passes (delay death and resume
point detection) because variables were appearing in the pre-death
and post-death sets that they shouldn't have.
Also make detect_deadness_in_par_conj look more similar to
detect_deadness_in_disj and detect_deadness_in_cases.
Revert the previous change to delay_death_par_conj which was
unnecessary and caused problems with nested parallel conjunctions
and --delay-death.
tests/par_conj/Mercury.options:
tests/par_conj/Mmakefile:
tests/par_conj/par_ddeath.exp:
tests/par_conj/par_ddeath.m:
tests/par_conj/par_ddeath_2.exp:
tests/par_conj/par_ddeath_2.m:
Add test cases.
Estimated hours taken: 0.5
Branches: main
tests/par_conj/Mmakefile:
tests/par_conj/threads_hang.exp:
tests/par_conj/threads_hang.m:
Make par_conj/threads_hang calculate smaller fibonacci numbers so as
not to use as much memory and fail nightly tests. Compensate by
running the test more times.
Estimated hours taken: 6
Branches: main, release
This patch fixes a bug with lowlevel parallel grades. A program built in such
a grade could hang when running with multiple threads.
runtime/mercury_context.c:
After scheduling a Mercury context, use MR_BROADCAST to wake up all
idle threads instead of MR_SIGNAL, if the newly scheduled context might
not be accepted for execution by any single woken thread. The hang
used to occur when a context was scheduled but the wrong idle thread
was woken up to execute it (because the context is 'owned' by another
thread) and promptly went back to idling.
runtime/mercury_thread.h:
Add MR_BROADCAST macros.
tests/par_conj/Mmakefile:
tests/par_conj/threads_hang.exp:
tests/par_conj/threads_hang.m:
Add a test case.
Estimated hours taken: 50
Branches: main
This patch adds preliminary support for deterministic, dependent
parallel conjunctions to the low-level backend. In other backends
dependent parallel conjunctions are converted into plain conjunctions.
For a parallel conjunction (A & B), if the goal B is dependent on a variable
X which is bound by goal A, we transform the conjunction such that
goal B must wait for the value to be produced by A before it begins
executing. This transformation is not yet useful in practice (you might as
well use a sequential conjunction). A later version of this transformation
will move the synchronisation deeper into the goals so that B can execute as
much as possible before it waits for the value from A.
There is no coroutining support yet so if there are not enough threads
available then dependent parallel conjunctions can cause the program to
deadlock.
configure.in:
runtime/mercury_conf.h.in:
Check for existence of semaphore.h and #define MR_HAVE_SEMAPHORE_H
if it does.
library/library.m:
library/par_builtin.m:
library/private_builtin.m:
mdbcomp/prim_data.m:
mdbcomp/program_representation.m:
Add a new module `par_builtin' to hold synchronisation primitives.
compiler/modules.m:
Import `par_builtin' module in parallel grades.
compiler/dep_par_conj.m:
compiler/transform_hlds.m:
compiler/mercury_compile.m:
Add a transformation to detect dependent parallel conjunctions
and insert the necessary synchronisation code.
compiler/hlds_goal.m:
Consider empty parallel conjunctions as atomic, the same as plain
conjunctions.
compiler/inlining.m:
Flatten parallel conjunctions when inlining, as for plain conjunctions.
compiler/live_vars.m:
Fix build_live_sets_in_par_conj to handle dependent parallel
conjunctions.
compiler/liveness.m:
Delay deaths in parallel conjunctions the same way as for plain
conjunctions.
Update detect_resume_points_in_par_conj for dependent parallel
conjunctions.
compiler/mode_util.m:
Treat parallel and plain conjunctions equally when recomputing instmap
deltas.
compiler/modes.m:
compiler/unique_modes.m:
Treat parallel and plain conjunctions equally when checking modes and
uniqueness. However, don't flatten parallel conjunctions into plain
conjunctions and vice versa.
compiler/simplify.m:
Don't reset instmaps and seen calls after each conjunct of a parallel
conjunction.
compiler/store_alloc.m:
Update allocation for dependent parallel conjunctions.
compiler/switch_detection.m:
Detect switches in parallel conjunctions the same as plain
conjunctions.
compiler/par_conj_gen.m:
Add a todo comment.
tests/Mmakefile:
tests/par_conj/.cvsignore:
tests/par_conj/Mmakefile:
tests/par_conj/dep_par_1.m:
tests/par_conj/dep_par_10.m:
tests/par_conj/dep_par_11.m:
tests/par_conj/dep_par_11b.m:
tests/par_conj/dep_par_11c.m:
tests/par_conj/dep_par_12.m:
tests/par_conj/dep_par_13.m:
tests/par_conj/dep_par_14.m:
tests/par_conj/dep_par_14b.m:
tests/par_conj/dep_par_14c.m:
tests/par_conj/dep_par_14d.m:
tests/par_conj/dep_par_16.m:
tests/par_conj/dep_par_17.m:
tests/par_conj/dep_par_18.m:
tests/par_conj/dep_par_2.m:
tests/par_conj/dep_par_20.m:
tests/par_conj/dep_par_21.m:
tests/par_conj/dep_par_22.m:
tests/par_conj/dep_par_23.m:
tests/par_conj/dep_par_3.m:
tests/par_conj/dep_par_3b.m:
tests/par_conj/dep_par_3c.m:
tests/par_conj/dep_par_4.m:
tests/par_conj/dep_par_5.m:
tests/par_conj/dep_par_5b.m:
tests/par_conj/dep_par_5c.m:
tests/par_conj/dep_par_5d.m:
tests/par_conj/dep_par_6.m:
tests/par_conj/dep_par_7.m:
tests/par_conj/dep_par_8.m:
tests/par_conj/dep_par_9.m:
tests/par_conj/indep_par_append.exp:
tests/par_conj/indep_par_append.m:
tests/par_conj/indep_par_nested.exp:
tests/par_conj/indep_par_nested.m:
Add some parallel conjunction test cases. The dependent parallel
conjunction tests are not yet executed as they can deadlock if there
are not enough threads available.
browser/RESERVED_MACRO_NAMES:
library/RESERVED_MACRO_NAMES:
runtime/RESERVED_MACRO_NAMES:
trace/RESERVED_MACRO_NAMES:
Add LINUX_THREADS, _REENTRANT and _THREAD_SAFE.
Estimated hours taken: 50
Branches: main
This patch adds preliminary support for deterministic, dependent
parallel conjunctions to the low-level backend. In other backends
dependent parallel conjunctions are converted into plain conjunctions.
For a parallel conjunction (A & B), if the goal B is dependent on a variable
X which is bound by goal A, we transform the conjunction such that
goal B must wait for the value to be produced by A before it begins
executing. This transformation is not yet useful in practice (you might as
well use a sequential conjunction). A later version of this transformation
will move the synchronisation deeper into the goals so that B can execute as
much as possible before it waits for the value from A.
There is no coroutining support yet so if there are not enough threads
available then dependent parallel conjunctions can cause the program to
deadlock.
configure.in:
runtime/mercury_conf.h.in:
Check for existence of semaphore.h and #define MR_HAVE_SEMAPHORE_H
if it does.
library/library.m:
library/par_builtin.m:
library/private_builtin.m:
mdbcomp/prim_data.m:
mdbcomp/program_representation.m:
Add a new module `par_builtin' to hold synchronisation primitives.
compiler/modules.m:
Import `par_builtin' module in parallel grades.
compiler/dep_par_conj.m:
compiler/transform_hlds.m:
compiler/mercury_compile.m:
Add a transformation to detect dependent parallel conjunctions
and insert the necessary synchronisation code.
compiler/hlds_goal.m:
Consider empty parallel conjunctions as atomic, the same as plain
conjunctions.
compiler/inlining.m:
Flatten parallel conjunctions when inlining, as for plain conjunctions.
compiler/live_vars.m:
Fix build_live_sets_in_par_conj to handle dependent parallel
conjunctions.
compiler/liveness.m:
Delay deaths in parallel conjunctions the same way as for plain
conjunctions.
Update detect_resume_points_in_par_conj for dependent parallel
conjunctions.
compiler/mode_util.m:
Treat parallel and plain conjunctions equally when recomputing instmap
deltas.
compiler/modes.m:
compiler/unique_modes.m:
Treat parallel and plain conjunctions equally when checking modes and
uniqueness. However, don't flatten parallel conjunctions into plain
conjunctions and vice versa.
compiler/simplify.m:
Don't reset instmaps and seen calls after each conjunct of a parallel
conjunction.
compiler/store_alloc.m:
Update allocation for dependent parallel conjunctions.
compiler/switch_detection.m:
Detect switches in parallel conjunctions the same as plain
conjunctions.
compiler/par_conj_gen.m:
Add a todo comment.
tests/Mmakefile:
tests/par_conj/.cvsignore:
tests/par_conj/Mmakefile:
tests/par_conj/dep_par_1.m:
tests/par_conj/dep_par_10.m:
tests/par_conj/dep_par_11.m:
tests/par_conj/dep_par_11b.m:
tests/par_conj/dep_par_11c.m:
tests/par_conj/dep_par_12.m:
tests/par_conj/dep_par_13.m:
tests/par_conj/dep_par_14.m:
tests/par_conj/dep_par_14b.m:
tests/par_conj/dep_par_14c.m:
tests/par_conj/dep_par_14d.m:
tests/par_conj/dep_par_16.m:
tests/par_conj/dep_par_17.m:
tests/par_conj/dep_par_18.m:
tests/par_conj/dep_par_2.m:
tests/par_conj/dep_par_20.m:
tests/par_conj/dep_par_21.m:
tests/par_conj/dep_par_22.m:
tests/par_conj/dep_par_23.m:
tests/par_conj/dep_par_3.m:
tests/par_conj/dep_par_3b.m:
tests/par_conj/dep_par_3c.m:
tests/par_conj/dep_par_4.m:
tests/par_conj/dep_par_5.m:
tests/par_conj/dep_par_5b.m:
tests/par_conj/dep_par_5c.m:
tests/par_conj/dep_par_5d.m:
tests/par_conj/dep_par_6.m:
tests/par_conj/dep_par_7.m:
tests/par_conj/dep_par_8.m:
tests/par_conj/dep_par_9.m:
tests/par_conj/indep_par_append.exp:
tests/par_conj/indep_par_append.m:
tests/par_conj/indep_par_nested.exp:
tests/par_conj/indep_par_nested.m:
Add some parallel conjunction test cases. The dependent parallel
conjunction tests are not yet executed as they can deadlock if there
are not enough threads available.
browser/RESERVED_MACRO_NAMES:
library/RESERVED_MACRO_NAMES:
runtime/RESERVED_MACRO_NAMES:
trace/RESERVED_MACRO_NAMES:
Add LINUX_THREADS, _REENTRANT and _THREAD_SAFE.
