The predicates that read a Mercury source, interface or optimization file
used to return four things:
- the name of the file (for callers who specified only a module name),
- the timestamp, if requested and available,
- a parse tree, and
- a representation of any errors (sets of error categories, and error_specs).
However, these four things were not independent. Some combinations of their
values were not allowed, but (a) there was no documentation of what these
combinations were, and (b) code that processed these four things had to be
prepared to handle illegal as well as legal combinations.
This diff makes these predicates return only one result, which contains
- all four of the above things, when the file could be opened, but
- only the file name and a representation of the error if the file
could not be opened,
- only the file name and a representation of *no* errors, if the caller
asked the predicate to read the file only if its timestamp did not match
a specified value, and it does match that value.
We use a somewhat modified version of an existing type, have_read_module,
for this. It is modified both by including information that its users
now need that they did not need before, and shortening the names of its
function symbols, which now occur in many more places than before.
compiler/read_modules.m:
Make the change to the output arguments described above.
Making this change requires having the affected predicates deal with
the case where we either cannot find or cannot open the relevant file.
(The two are effectively the same thing in this module, since we search
by attempting to open.) Passing that task off to parse_modules.m
was always inelegant.
Simplify the have_read_module_map type by making the key always
be a module_name. We deleted the only use of have_read_module_maps
with another kind of key a while ago.
Delete some no-longer-needed predicates.
compiler/parse_module.m:
Delete the code dealing with the absence of a file stream to parse.
Replace code that used to construct dummy parse trees with code
that simply returns *no* parse tree if the file could be opened
but could not be read, or if its timestamp indicated that reading it
would have been redundant.
Delete some utility predicates moved to parse_error.m, so that
read_modules.m could also use them.
Fix comment rot.
compiler/parse_error.m:
Add some utility predicates for use by read_modules.m and parse_module.m.
compiler/deps_map.m:
Conform to the changes above.
Document the dubious effects of an old design decision.
Fix a misleading predicate name.
Fix comment rot.
compiler/grab_modules.m:
Conform to the changes above.
Some predicates used to return parse trees that could be dummies.
Change them to return just the parts of the parse tree that the
caller was interested in, which was usually a tiny part, and which
can be constructed trivially even when we don't have a parse tree.
Delete an unneeded type.
compiler/recompilation.check.m:
Conform to the changes above.
Represent the operations we need to test version numbers in interface files
as a typeclass, and rewrite the checking operation in terms of that
typeclass, with one instance for each kind of interface file.
Move some repeated code into a predicate.
Shorten some long names.
compiler/mercury_compile_main.m:
Conform to the changes above.
Break up a large predicate.
compiler/generate_dep_d_files.m:
compiler/make.module_dep_file.m:
compiler/write_module_interface_files.m:
Conform to the changes above.
compiler/prog_item.m:
Delete the parse_tree_some_int type. The change in recompilation.check.m,
deleted its last few uses there, and allowed the deletion of the last
uses in read_modules.m.
tests/recompilation/two_module_debug:
Extend this script to help deal with problems in all stages of the
execution of a test case, since that was required while debugging
this diff.
Document which parts of this script correspond to which parts of
two_module_test.
tests/recompilation/test_functions:
tests/recompilation/two_module_test:
Simplify the logic of the main test function, by testing a condition
once instead of three times.
Specify whether a recompilation test is expected to succeed or fail
directly, using alternatives that model a bespoke type, instead
of alternatives that mimic a boolean answer to a question, which
does not help readers who don't remember the question.
Always put shell variable names in braces.
Note a problem that makes one of the shell functions ineffective.
tests/recompilation/Mmakefile:
Tell two_module_test what to do using its updated interface.
My commit afe2887882 broke the ability
to run the test suite outside of a bootcheck by executing "mmake runtests"
in the tests directory. This diff fixes that.
tests/Mmake.common:
Don't define "TESTS_DIR = ..". While every single tests/*/Mmakefile
defined it as such, I overlooked the fact that tests/Mmakefile itself
defined it ".", referring to the same directory from a different starting
point. Document this easily-overlooked fact.
Rename the old runtests target, which after afe2887 runs the tests
in a single directory, as runtests_dir, to leave the target name
"runtests" itself free for tests/Mmakefile to use.
tests/Mmakefile:
Define "TESTS_DIR = .", and add a target "runtests" which invokes
"mmake runtests_dir" in each test directory.
tools/bootcheck:
Invoke "mmake runtests_dir" instead of "mmake runtests" in each
test directory.
Initialize a variable just before it is used.
tests/*/Mmakefile:
Add back the definition "TESTS_DIR = .."
A long time ago, test directories such as hard_coded had subdirectories
such as hard_coded/typeclasses. These have since been flattened out
(e.g. hard_coded/typeclasses is now just typeclasses), but there were
still remnants of the old approach. This diff deletes those remnants.
tests/*/Mmakefile:
Delete the TESTS_DIR and the SUBDIRS mmake variables; TESTS_DIR
was always set to "..", and SUBDIRS to the empty string.
Delete any references to the make variable NOT_WORKING, since
it is never used.
tests/Mmake.common:
Document that Mmakefiles in test directories don't have to set
TESTS_DIR and SUBDIRS anymore. Fix the formatting of the documentation
of the make variables they do still have to set.
Delete the targets and actions for handling subdirectories of
test directories, since there aren't any.
tests/Mmakefile:
Simplify some code.
tests/Mmake.common:
Don't invoke any actions in the clean_local and realclean_local
targets, since if using mmc --make, the builtin mmake rules
have actions for those targets as well, and make can't handle
more than one action for a target having actions. Replace those
actions with dependencies on other, unique targets that have
the actions instead.
tests/*/Mmakefile:
Avoid actions in clean_local and realclean_local targets the same way.
Sort the test names in some directories that didn't already do so.
Delete some obsolete comments.
Fix style.
tests/valid/Mmake.valid.common:
As for the Mmakefiles above, and also move the definition of a make
variable before it is needed.
tests/Mmake.common:
Replace the -j1 in the runtests_local target used by all the test
directories with $(MAYBE_J1).
tests/*/Mmakefile:
Define MAYBE_J1 it as the empty string in test directories in which
different tests don't share source files.
Define MAYBE_J1 as -j1 in test directories in which
different tests do share source files.
tests/submodules/sub2_a.m:
Add this copy of sub_a.m to allow tests in the submodules directory
to be done in parallel.
tests/submodules/accessibility2.m:
Import sub2_a.m instead of sub_a.m.
tests/warnings/ambig_types_high_level.m:
Add this copy of ambig_types.m to allow tests in the warnings directory
to be done in parallel.
tests/warnings/ambig_high_level.m:
Import ambig_types_high_level.m instead of ambig_types.m.
This file-specific setting will override a default setting of expandtabs
in $HOME/.vimrc.
*/Makefile:
*/Mmakefile:
As above.
tests/hard_coded/.gitignore:
Don't ignore the purity subdir. This ignore must have been left over
from when purity.m was a test in hard_coded, not hard_coded/purity,
and it ignored an executable, not a directory.
Branches: main
Allow testing of java grade. Requires using `mmc --make' for now.
This patch does not attempt to fix test failures.
tests/Mmake.common:
Delete unneeded Java-specific rule, which was broken.
tests/benchmarks/Mmakefile:
tests/general/Mmakefile:
tests/general/string_format/Mmakefile:
tests/grade_subdirs/Mmakefile:
tests/hard_coded/Mmakefile:
tests/recompilation/Mmakefile:
tests/term/Mmakefile:
tests/valid/Mmakefile:
Don't deliberately disable tests in java grade.
tests/*.m:
Add Java foreign code.
Write dummy procedures instead of abusing `:- external'.
Estimated hours taken: 0.2
Branches: main
tests/*/Mmakefile:
Sort the lists of tests in each directory. (We usually keep each list
sorted, but then we concatenate several lists, which loses the order.)
This gives you a better idea of how far a bootcheck still has to go.
tools/bootcheck:
Build the interface files before the rest of the files in the library
directory. Again, this gives you a better idea of how far a bootcheck
still has to go.
Make the dependencies for the slice subdirectory of the stage2 at the
same time as the other subdirs; don't let it be caught later.
library/Mmakefile:
Build the interface files in the right order.
Estimated hours taken: 1.5
Branches: main
library/io.m:
Implement some more I/O primitives for Erlang.
tests/debugger/Mmakefile:
tests/debugger/declarative/Mmakefile:
tests/hard_coded/Mmakefile:
tests/invalid/Mmakefile:
tests/par_conj/Mmakefile:
tests/recompilation/Mmakefile:
tests/valid/Mmakefile:
Disable test cases which don't apply to the Erlang backend for
whatever reason.
tests/hard_coded/constraint_order.m:
tests/hard_coded/copy_pred.m:
tests/hard_coded/copy_pred_2.m:
tests/hard_coded/export_test.m:
tests/hard_coded/external_unification_pred.m:
tests/hard_coded/foreign_type3.m:
tests/hard_coded/ho_solns.m:
tests/hard_coded/ho_univ_to_type.m:
tests/hard_coded/intermod_multimode.m:
tests/hard_coded/intermod_poly_mode_2.m:
tests/hard_coded/lookup_disj.m:
tests/hard_coded/multimode.m:
tests/hard_coded/no_inline.m:
tests/hard_coded/pragma_foreign_export.m:
tests/hard_coded/redoip_clobber.m:
tests/hard_coded/rnd.m:
tests/hard_coded/sub-modules/sm_exp_bug.m:
tests/hard_coded/typeclasses/impure_methods.m:
tests/par_conj/dep_par_10.m:
tests/valid/flatten_conj_bug.m:
Fix test cases which were failing with the Erlang for simple reasons,
mostly missing foreign procs or foreign types.
tests/general/string_format_test_2.exp5:
tests/general/string_format_test_3.exp5:
tests/hard_coded/exceptions/test_uncaught_exception.exp5:
tests/hard_coded/no_fully_strict.exp5:
Add expected outputs with stack dumps for Erlang.
Estimated hours taken: 0.25
Branches: main
recompilation/Mmakefile:
Disable the recompilation tests for the il grade because it
currently doesn't support intermodule optimization.
Estimated hours taken: 14
Branches: main
Modifications to the test suite to allow testing in grade java.
mercury/tools/test_mercury:
Set-up the required CLASSPATH variable.
tests/Mmake.common:
Added a rule to build `.out' files when the grade is java.
tests/benchmarks/Mmakefile:
Enable only the working tests for the benckmarks directory.
tests/dppd/Mmakefile:
tests/general/Mmakefile:
tests/general/accumulator/Mmakefile:
tests/general/string_format/Mmakefile:
tests/general/structure_reuse/Mmakefile:a
tests/grade_subdirs/Mmakefile:
tests/hard_coded/Mmakefile:
tests/hard_coded/exceptions/Mmakefile:
tests/hard_coded/purity/Mmakefile:
tests/hard_coded/sub-modules/Mmakefile:
tests/hard_coded/typeclasses/Mmakefile:
tests/recompilation/Mmakefile
tests/term/Mmakefile:
tests/valid/Mmakefile:
Disable test cases for grade java.
tests/recompilation/test_functions:
Add framework needed to do recompilation testing for the java grade.
Estimated hours taken: 30
Branches: main
Improve the test framework to make it easier to find out which tests
failed and to reduce disk usage (important in debug grades).
Allow the tests to be run with `mmc --make' (still some failures).
Allow the user to run only the failing tests from a previous
run by using `mmake ERROR_FILE=runtests.errs', where runtests.errs
is the log file from the previous run.
tests/Mmake.common:
tests/*/Mmakefile:
Move common code (such as the code to deal with subdirectories)
to Mmake.common.
Run the tests using `mmake runtests' rather than using slightly
different runtests scripts in each directory.
Add to the output from `mmake runtests' to make it easier to
identify which tests failed in which grades.
Move per-module options into Mercury.options files so they
can be read by `mmc --make'.
Remove the last of the NU-Prolog support.
Consistently use the main module name when listing tests.
Some directories (e.g. invalid) were using the source file
name.
tests/process_log.awk:
Collect the parts of the output relating to failing tests.
tests/generate_exp:
tests/handle_options:
tests/subdir_runtests:
tests/startup:
tests/shutdown:
tests/*/runtests:
tests/recompilation/TESTS:
Removed.
tests/recompilation/test_functions:
Make sure the old result file is removed before starting
each test.
Put the mmake output for tests which are supposed to fail
into a different file for each test.
tests/warnings/Mmakefile:
Use %.c rather than $(cs_subdir)%.c in a rule.
The $(cs_subdir) part doesn't work with `mmc --make',
and isn't necessary any more (modules.m generates a rule
`module.c: $(cs_subdir)module.c').
tests/README:
Updated.
tools/bootcheck:
tools/test_mercury:
Use `mmake runtests' instead of the `runtests' script.
Add a `-f' (`--failing-tests') option to bootcheck which
runs only the failing tests from the last run.
tools/test_mercury:
tools/run_all_tests_from_cron:
Use the new framework to summarize test failures.
Estimated hours taken: 2.5
Branches: main
Add an option `--find-all-recompilation-reasons' which causes
smart recompilation to find all reasons why a module needs to
be recompiled, not just the first. This allows test cases in
tests/recompilation to contain multiple tests. Also,
tests/recompilation will now work with deep profiling.
compiler/options.m:
compiler/handle_options.m:
doc/user_guide.texi:
Add the new option
compiler/recompilation_check.m:
Collect all reasons for recompilation if
`--find-all-recompilation-reasons' is set.
tests/recompilation/Mmakefile:
Add `--find-all-recompilation-reasons' to EXTRA_MCFLAGS.
tests/recompilation/remove_type_re.err_exp.2:
Update expected output.
Estimated hours taken: 1
Branches: main
Fix bugs in the handling of nested sub-modules with --smart-recompilation.
compiler/recompilation_check.m:
If an error occurs before the nested sub-modules of the top-level
module are known, recompile all of the nested sub-modules.
If there is a syntax error in a `.used' file, recompile all
sub-modules. This makes the above change simpler, and is probably
a good idea anyway -- all outputs from a compilation which produces
output files containing syntax errors are suspect.
compiler/mercury_compile.m:
When working out the list of sub-modules of the main module
in a source file, use all the sub-modules, not just the ones
that are being recompiled.
tests/recompilation/Mmakefile:
tests/recompilation/TESTS:
tests/recompilation/nested_module_2_r*:
Test case.
Estimated hours taken: 4
Branches: main
Fix bugs in smart recompilation.
compiler/recompilation_version.m:
Avoid problems where splitting of combined predicate type
and mode declarations resulted in different varsets when
declarations were read back in from `.int' files for comparison
with the new declarations. The symptom was that the interface
file would change (due to changing timestamps) even if the
source module was just touched.
Pragmas can apply to typeclass methods, so consider a typeclass
to be changed if any pragmas which could apply to any of its
methods are changed.
compiler/hlds_out.m:
Export mode_to_term for use by recompilation_version.m in
testing whether two modes are identical with variable renaming.
tests/recompilation/change_type_nr*:
tests/recompilation/combined_type_mode_nr*:
tests/recompilation/typeclass_method_pragma_r*:
Test cases.
Estimated hours taken: 0.25
Branches: main
tests/recompilation/Mmakefile:
tests/recompilation/TESTS:
tests/recompilation/runtests:
Parallel mmake does not work with nested sub-modules,
so disable it for tests containing nested sub-modules.
Estimated hours taken: 0.2
tests/Mmake.common:
recompilation/Mmakefile:
recompilation/runtests:
Provide a way to disable the recompilation tests --
put `RUN_RECOMPILATION_TESTS=no' in tests/Mmake.params.
This is needed for compilers built in deep profiling grades,
because smart recompilation uses exception handling, which does
not yet work with deep profiling.
Estimated hours taken: 0.2
tests/recompilation/Mmakefile:
Use `override' when setting EXTRA_MCFLAGS, because otherwise
make ignores the assigment if EXTRA_MCFLAGS is set on the
command line, as it is for the nightly tests.
Estimated hours taken: 400
Smart recompilation. Record version numbers for each item
in interface files. Record which items are used in each compilation.
Only recompile a module if the output file does not exist or
nothing has changed.
There is still some work to do on this:
- it doesn't work with inter-module optimization.
- it doesn't work when the module name doesn't match the file name.
(this problem will go away when mmake functionality is moved into
the compiler.
I'll hold off documenting this change in the NEWS file and
on the web page for a month or so, until I've had a bit more
experience using it.
compiler/options.m:
compiler/handle_options.m:
doc/user_guide.texi:
Add an option `--smart-recompilation', currently off by default.
Add an internal option `--generate-version-numbers' to control
whether version numbers are written to the interface files. If
`--smart-recompilation' is disabled because the module
is being compiled with `--intermodule-optimization' (e.g. in the
standard library), we still want to write the version numbers
to the interface files.
Add an option `--verbose-recompilation' (default off)
to write messages describing why recompilation is needed.
Add an option `--warn-smart-recompilation' (default on)
to control warnings relating to the smart recompilation
system. Warn if smart recompilation will not work with
the output and inter-module optimization options given.
compiler/recompilation.m:
Type declarations for smart recompilation.
Predicates to record program items used by compilation.
compiler/recompilation_version.m:
Compute version numbers for program items in interface files.
compiler/recompilation_usage.m:
Find all items used by a compilation.
compiler/recompilation_check.m:
Check whether recompilation is necessary.
compiler/timestamp.m:
Timestamp ADT for smart recompilation.
compiler/mercury_compile.m:
Invoke the smart recompilation passes.
compiler/modules.m:
compiler/prog_io.m:
Return timestamps for modules read.
When reading a module make sure the current input stream
is reset to its old value, not stdin.
Handle version number items in interface files.
compiler/module_qual.m:
compiler/unify_proc.m:
compiler/make_hlds.m:
Record all items used by local items.
compiler/make_hlds.m:
Process `:- pragma type_spec' declarations in
add_item_list_clauses. The qual_info is needed
when processing `:- pragma type_spec' declarations
so that any equivalence types used by the declaration
can be recorded as used by the predicate or function to
which the `:- pragma type_spec' applies.
compiler/equiv_type.m:
For each imported item, record which equivalence types
are used by that item.
compiler/hlds_module.m:
Add a field to the module_info to store information about
items used during compilation of a module.
compiler/check_typeclass.m:
Make sure any items used in clauses for typeclass method
implementations are recorded in the `.used' file.
compiler/prog_data.m:
compiler/*.m:
Factor out some duplicated code by combining the
pred and func, and pred_mode and func_mode items.
Make it easier to extract the name of a type, inst or mode
from its declaration.
Add an item type to hold the version numbers for an interface file.
Allow warnings to be reported for `nothing' items (used for
reporting when version numbers are written using an
obsolete format).
compiler/prog_io.m:
compiler/prog_io_util.m:
compiler/typecheck.m:
compiler/type_util.m:
compiler/*.m:
Strip contexts from all types, not just those in class constraints.
This makes it possible to use ordinary unification to check
whether items have changed (with the exception of clauses).
Remove code to create types with contexts in typechecking.
Remove code scattered through the compiler to remove contexts
from types in class constraints.
compiler/hlds_pred.m:
compiler/prog_util.m:
Move hlds_pred__adjust_func_arity to prog_util, so that it
can be used by the pre-hlds passes.
compiler/typecheck.m:
compiler/hlds_module.m:
Move typecheck__visible_modules to hlds_module.m, so it can
be used by recompilation_usage.m.
compiler/typecheck.m:
Add a comment telling where updates may be required if the
code to typecheck a var-functor unification changes.
compiler/error_util.m:
Allow writing messages without contexts (used for the verbose
recompilation messages).
Add functions to format sym_name and sym_name_and_arity,
and to add punctuation to the end of an error message
without unwanted line breaks before the punctuation.
scripts/Mmake.rules:
compiler/modules.m:
Don't remove the output file before running the compiler. We need
to leave the old output file intact if smart recompilation detects
that recompilation is not needed.
compiler/notes/compiler_design.html:
Document the new modules.
library/io.m:
NEWS:
Add predicates to find the modification time of files
and input_streams.
library/set.m:
NEWS:
Add a predicate version of set__fold
Don't sort the output of set__filter, it's already sorted.
library/std_util.m:
NEWS:
Add a predicate `std_util__map_maybe/3' and a function
`std_util__map_maybe/2' to apply a predicate or a function to
a value stored in a term of type `std_util__maybe'.
configure.in:
runtime/mercury_conf.h.in:
runtime/RESERVED_MACRO_NAMES:
When checking whether the compiler is recent enough, check for
the --warn-smart-recompilation option.
Check for stat().
library/Mmakefile:
Disable warnings about smart recompilation not working with
`--intermodule-optimization'.
browser/Mmakefile:
Disable warnings about smart recompilation not working when
the module name doesn't match the file name.
runtime/mercury_string.h:
Add a macro MR_make_string_const() which automates computation
of the length of string argument to MR_string_const().
tests/recompilation/Mmakefile:
tests/recompilation/runtests:
tests/recompilation/test_functions:
tests/recompilation/TESTS:
tests/recompilation/README:
A framework for testing smart recompilation.
The option currently only works for the recompilation directory.
tests/recompilation/TEST.m.{1,2}:
tests/recompilation/TEST_2.m.{1,2}:
tests/recompilation/TEST.exp.{1,2}:
tests/recompilation/TEST.err_exp.2:
Test cases, where TEST is one of add_constructor_r, add_instance_r,
add_instance_2_r, add_type_nr, change_class_r, change_instance_r,
change_mode_r, field_r, func_overloading_nr, func_overloading_r,
lambda_mode_r, nested_module_r, no_version_numbers_r,
pragma_type_spec_r, pred_ctor_ambiguity_r, pred_overloading_r,
add_type_re, remove_type_re, type_qual_re.
tests/handle_options:
Add an option `-e' to generate any missing expected output files.
