My commit afe2887882 broke the ability
to run the test suite outside of a bootcheck by executing "mmake runtests"
in the tests directory. This diff fixes that.
tests/Mmake.common:
Don't define "TESTS_DIR = ..". While every single tests/*/Mmakefile
defined it as such, I overlooked the fact that tests/Mmakefile itself
defined it ".", referring to the same directory from a different starting
point. Document this easily-overlooked fact.
Rename the old runtests target, which after afe2887 runs the tests
in a single directory, as runtests_dir, to leave the target name
"runtests" itself free for tests/Mmakefile to use.
tests/Mmakefile:
Define "TESTS_DIR = .", and add a target "runtests" which invokes
"mmake runtests_dir" in each test directory.
tools/bootcheck:
Invoke "mmake runtests_dir" instead of "mmake runtests" in each
test directory.
Initialize a variable just before it is used.
tests/*/Mmakefile:
Add back the definition "TESTS_DIR = .."
A long time ago, test directories such as hard_coded had subdirectories
such as hard_coded/typeclasses. These have since been flattened out
(e.g. hard_coded/typeclasses is now just typeclasses), but there were
still remnants of the old approach. This diff deletes those remnants.
tests/*/Mmakefile:
Delete the TESTS_DIR and the SUBDIRS mmake variables; TESTS_DIR
was always set to "..", and SUBDIRS to the empty string.
Delete any references to the make variable NOT_WORKING, since
it is never used.
tests/Mmake.common:
Document that Mmakefiles in test directories don't have to set
TESTS_DIR and SUBDIRS anymore. Fix the formatting of the documentation
of the make variables they do still have to set.
Delete the targets and actions for handling subdirectories of
test directories, since there aren't any.
tests/Mmakefile:
Simplify some code.
tests/benchmarks/Mmakefile:
Delete variables only used with the IL or GCC back-ends.
tests/debugger/Mmakefile:
tests/declarative_debugger/Mmakefile:
tests/par_conj/Mmakefile:
Delete references to the IL backend.
tests/par_conj/dep_par_10.m:
Add missing C# and Java foreign procs.
tests/dppd/Mmakefile:
Run these tests in the Java grade.
tests/general/Mmakefile:
Delete a workaround for OSF/1 -- we no longer support it.
tests/hard_coded/Mmakefile:
Delete the unused list of tests that will pass in the Java grade.
Delete a reference to the IL backend.
tests/Mmake.common:
Don't invoke any actions in the clean_local and realclean_local
targets, since if using mmc --make, the builtin mmake rules
have actions for those targets as well, and make can't handle
more than one action for a target having actions. Replace those
actions with dependencies on other, unique targets that have
the actions instead.
tests/*/Mmakefile:
Avoid actions in clean_local and realclean_local targets the same way.
Sort the test names in some directories that didn't already do so.
Delete some obsolete comments.
Fix style.
tests/valid/Mmake.valid.common:
As for the Mmakefiles above, and also move the definition of a make
variable before it is needed.
tests/Mmake.common:
Replace the -j1 in the runtests_local target used by all the test
directories with $(MAYBE_J1).
tests/*/Mmakefile:
Define MAYBE_J1 it as the empty string in test directories in which
different tests don't share source files.
Define MAYBE_J1 as -j1 in test directories in which
different tests do share source files.
tests/submodules/sub2_a.m:
Add this copy of sub_a.m to allow tests in the submodules directory
to be done in parallel.
tests/submodules/accessibility2.m:
Import sub2_a.m instead of sub_a.m.
tests/warnings/ambig_types_high_level.m:
Add this copy of ambig_types.m to allow tests in the warnings directory
to be done in parallel.
tests/warnings/ambig_high_level.m:
Import ambig_types_high_level.m instead of ambig_types.m.
This file-specific setting will override a default setting of expandtabs
in $HOME/.vimrc.
*/Makefile:
*/Mmakefile:
As above.
tests/hard_coded/.gitignore:
Don't ignore the purity subdir. This ignore must have been left over
from when purity.m was a test in hard_coded, not hard_coded/purity,
and it ignored an executable, not a directory.
Branches: main
Allow testing of java grade. Requires using `mmc --make' for now.
This patch does not attempt to fix test failures.
tests/Mmake.common:
Delete unneeded Java-specific rule, which was broken.
tests/benchmarks/Mmakefile:
tests/general/Mmakefile:
tests/general/string_format/Mmakefile:
tests/grade_subdirs/Mmakefile:
tests/hard_coded/Mmakefile:
tests/recompilation/Mmakefile:
tests/term/Mmakefile:
tests/valid/Mmakefile:
Don't deliberately disable tests in java grade.
tests/*.m:
Add Java foreign code.
Write dummy procedures instead of abusing `:- external'.
Estimated hours taken: 0.2
Branches: main
tests/*/Mmakefile:
Sort the lists of tests in each directory. (We usually keep each list
sorted, but then we concatenate several lists, which loses the order.)
This gives you a better idea of how far a bootcheck still has to go.
tools/bootcheck:
Build the interface files before the rest of the files in the library
directory. Again, this gives you a better idea of how far a bootcheck
still has to go.
Make the dependencies for the slice subdirectory of the stage2 at the
same time as the other subdirs; don't let it be caught later.
library/Mmakefile:
Build the interface files in the right order.
Estimated hours taken: 0.1
Branches: main
tests/benchmarks/Mmakefile:
Improve the documentation: explain how to do performance testing for MLDS grades.
Estimated hours taken: 14
Branches: main
Modifications to the test suite to allow testing in grade java.
mercury/tools/test_mercury:
Set-up the required CLASSPATH variable.
tests/Mmake.common:
Added a rule to build `.out' files when the grade is java.
tests/benchmarks/Mmakefile:
Enable only the working tests for the benckmarks directory.
tests/dppd/Mmakefile:
tests/general/Mmakefile:
tests/general/accumulator/Mmakefile:
tests/general/string_format/Mmakefile:
tests/general/structure_reuse/Mmakefile:a
tests/grade_subdirs/Mmakefile:
tests/hard_coded/Mmakefile:
tests/hard_coded/exceptions/Mmakefile:
tests/hard_coded/purity/Mmakefile:
tests/hard_coded/sub-modules/Mmakefile:
tests/hard_coded/typeclasses/Mmakefile:
tests/recompilation/Mmakefile
tests/term/Mmakefile:
tests/valid/Mmakefile:
Disable test cases for grade java.
tests/recompilation/test_functions:
Add framework needed to do recompilation testing for the java grade.
Estimated hours taken: 30
Branches: main
Improve the test framework to make it easier to find out which tests
failed and to reduce disk usage (important in debug grades).
Allow the tests to be run with `mmc --make' (still some failures).
Allow the user to run only the failing tests from a previous
run by using `mmake ERROR_FILE=runtests.errs', where runtests.errs
is the log file from the previous run.
tests/Mmake.common:
tests/*/Mmakefile:
Move common code (such as the code to deal with subdirectories)
to Mmake.common.
Run the tests using `mmake runtests' rather than using slightly
different runtests scripts in each directory.
Add to the output from `mmake runtests' to make it easier to
identify which tests failed in which grades.
Move per-module options into Mercury.options files so they
can be read by `mmc --make'.
Remove the last of the NU-Prolog support.
Consistently use the main module name when listing tests.
Some directories (e.g. invalid) were using the source file
name.
tests/process_log.awk:
Collect the parts of the output relating to failing tests.
tests/generate_exp:
tests/handle_options:
tests/subdir_runtests:
tests/startup:
tests/shutdown:
tests/*/runtests:
tests/recompilation/TESTS:
Removed.
tests/recompilation/test_functions:
Make sure the old result file is removed before starting
each test.
Put the mmake output for tests which are supposed to fail
into a different file for each test.
tests/warnings/Mmakefile:
Use %.c rather than $(cs_subdir)%.c in a rule.
The $(cs_subdir) part doesn't work with `mmc --make',
and isn't necessary any more (modules.m generates a rule
`module.c: $(cs_subdir)module.c').
tests/README:
Updated.
tools/bootcheck:
tools/test_mercury:
Use `mmake runtests' instead of the `runtests' script.
Add a `-f' (`--failing-tests') option to bootcheck which
runs only the failing tests from the last run.
tools/test_mercury:
tools/run_all_tests_from_cron:
Use the new framework to summarize test failures.
Estimated hours taken: 1
Add some code for running timing tests on the
programs in the tests/benchmarks directory.
tests/benchmarks/Mmakefile:
Add new target `times'.
tests/benchmarks/BenchEntry:
tests/benchmarks/BenchEntry.hlc:
New files, for use by the `times' target in the Mmakefile.
These two files contain the names of the non-I/O entry points
for the various benchmarks, for LLDS and MLDS grades respectively.
tests/benchmarks/BenchRepeatCounts:
New file. The contents of this file specify how many times
to repeat each benchmark.
tests/benchmarks/repeat_time:
New file, for use by the `times' target in the Mmakefile.
This is a tiny (t)csh script for running a command repeatedly
and timing how long it takes.
Estimated hours taken: 1
tools/bootcheck:
If the new --test-params option is given, copy the stage 2 Mmake.params
file to the tests directory. This capability is required to test
changes such as compiling the library without special preds and
the test cases with special preds. You get a lot of spurious errors
if a test case assumes that since it was compiled with special
preds, all other modules it is linked with are also compiled with
special preds, and thus makes calls to non-existent special preds
for the types defined in the library.
tests/*/Mmakefile:
tests/*/*/Mmakefile:
Include Mmake.params from the top level tests directory if it exists.
