mercurylang/mercury
1
0
Fork 0
mirror of https://github.com/Mercury-Language/mercury.git synced 2026-04-19 03:13:40 +00:00
Code Issues Projects Releases Wiki Activity
22,765 Commits 24 Branches 141 Tags
083d376e6598628362ee91c2da170febd83590f4
Commit Graph

10 Commits

Author SHA1 Message Date
Zoltan Somogyi
0c6a4df1f5 If a test fails, print any .err file. 
Looking at the .err file of a failed test is easy after most bootchecks,
but for an outsider it is effectively impossible when the bootcheck is on
testing.mercurylang.org. Including the contents of the .err file in the
test's .log file will help.
 2022-01-10 14:46:41 +11:00
Zoltan Somogyi
66aa649378 Keep the .log files of successful tests if requested. 
Sometimes, when a test case fails in a workspace even though it has passed
before, it is not clear whether the cause of the failure is that the
updated code in the workspace is generating a different sequence of
mmc invocations, or whether the same invocations do something different.
The new option allows developers to answer that question by keeping the logs
from a bootcheck in an unchanged workspace, and comparing the logs
between the workspaces.

tools/bootcheck:
    If given the --keep-success-log-files option, set an environment variable
    that records this fact.

tests/run_one_test:
    If this environment variable is set, then rename the .log files of
    successful test cases as .kept_log files instead of deleting them.

    Note that we cannot keep the .log files around under their original name,
    because we currently interpret the presence of *any* .log file
    in a test directory as meaning "some tests failed in this test directory".
 2020-03-08 13:41:57 +11:00
Peter Wang
fd24b2e331 Don't compress failed test executables. 
Disk space isn't the concern that it used to be so compressing
failed test executables is mostly just a waste of time.

tests/run_one_test:
    Don't compress failed test executables.

tests/Mmake.common:
    Delete clean_zip target.

tests/*/.gitignore:
    Delete *.gz ignore patterns.
 2018-07-31 16:32:41 +10:00
Zoltan Somogyi
e0cdfc2fe2 Make the tests really work in parallel. 
There was a bug that prevented the tests in each directory from being run
in parallel, even when the mmake was invoked with e.g. -j4. The bug
was the absence of a '+' on an action that invoked tests/run_one_test.
Without that +, the make process inside the "mmake -j4 runtests_local"
command issued by bootcheck screwed up its connection with the recursive
make invoked by run_one_test, and apparently decided to stop using
parallelism. It was telling us this all this time but its messages,
which looked like this:

    make[2]: warning: -jN forced in submake: disabling jobserver mode.

were lost in the sea of other bootcheck output until my recent change
to run_one_test.

A single-character change fixes the bug; the rest of this big change is
dealing with the consequences of fixing the bug. Many test cases in
the hard_coded and valid directories contain nested modules, which
need to be compiled sequentially.

tests/Mmake.common:
    Fix the bug.

    Delete the duplicate "ALL TESTS SUCCEEDED" message from the runtests
    target, since the runtests_local target already prints a message
    to that effect. Make this message more emphatic.

tests/run_one_test:
    Make the output a bit easier to understand.

tests/hard_coded/*.{m,exp}:
tests/submodules/*.{m,exp}:
    Move the source files and expected output files of the test cases that
    use nested modules from hard_coded to submodules, since most of the tests
    in the hard_coded can be done in parallel, while the ones in submodules
    are already done in sequence.

tests/hard_coded/Mmakefile:
tests/hard_coded/Mercury.options:
tests/submodules/Mmakefile:
tests/submodules/Mercury.options:
    Move the Mmakefile and Mercury.options entries of the moved test cases
    from hard_coded to submodules.

tests/valid_seq:
    A new test directory, to hold the test cases originally in tests/valid
    that contain nested modules. Moving these to a new directory, which
    forces -j1, allows us to execute the remaining ones in parallel.

tests/valid/*.m:
tests/valid_seq/*.m:
    Move the source files of the test cases that use nested modules
    from valid to valid_seq.

    In a few cases, clean up the test cases a bit.

tests/valid/Mmakefile:
tests/valid/Mercury.options:
tests/valid_seq/Mmakefile:
tests/valid_seq/Mercury.options:
tests/valid/Mmake.valid.common:
    Add valid_seq/Mmakefile to list the test cases now in valid_seq,
    and add valid_seq/Mercury.options to hold their option values.
    Delete the entries of those tests from valid/Mmakefile, and their option
    values from valid/Mercury.options. Unlike valid/Mmakefile,
    valid_seq/Mmakefile forces -j1.

    To avoid unnecessary duplication between the two Makefiles, put
    all the rules that both need into valid/Mmake.valid.common, and
    include this in both Mmakefiles.

tests/string_format/Mmakefile:
    Force -j1, since these tests share a library module.

tests/Mmakefile:
    List the new valid_seq directory among the others.

tests/bootcheck:
    Execute the tests in the new valid_seq directory as well as the others.

    Record when the execution of the test suite is started.

    Comment out invocations of set -x, since they add no useful information
    in the vast majority of cases. The comment sign can be removed if and
    when the information *would* be useful.

    Don't try to copy a nonexistent file. (The error message about this
    was also lost in the noise.)
 2015-08-26 01:01:54 +10:00
Zoltan Somogyi
51d8c1903e Print details only for failing test cases. 
tests/run_one_test:
    Put all the output for each test case into a file, and print the file
    (the test case log) *only* if the test fails.
 2015-08-25 12:40:50 +10:00
Zoltan Somogyi
c5b2faf4ec Improve tests/run_one_test. 
Replace $x references to shell variables with ${x} references to protect
against mistakes in the extent of the variable name, and put those references
into double quotes when possible to protect against mistakes involving
spaces in variable values or empty strings as variable values.

Initialize local shell variables just before their first use.

Remove the "set -x", since the shell commands that it causes to echo
are just about never interesting, because they are boilerplate code,
the same code being executed for just about every test case, the only
variation being between test cases with and without a -nodepend suffix.
(The interesting shell commands, those which are sometimes needed to
track down the causes of test case failures, are the commands issued by mmake,
which are unaffected by set -x.)

For me, this reduced the size of the output of a bootcheck from 73k+ lines
to 48k+ lines.
 2015-08-23 01:21:14 +10:00
Peter Wang
fd76da59ff Add support for the csharp' grade to mmc --make', and make it possible to 
Branches: main

Add support for the `csharp' grade to `mmc --make', and make it possible to
install the `csharp' grade with `mmake install'.

Also some miscellaneous fixes.


configure.in:
        Require a recent enough bootstrap compiler that recognises C# as a
        language for `pragma foreign_type'.

Mmakefile:
        Use `mmc --make' to install the standard library in csharp grade.

aclocal.m4:
        Search for the Mono C# compiler `gmcs', which is required for generics
        at this time.  Prefer it over the DotGNU C# compiler, which I have not
        tested.

        Search for `mono'.  If found, it will be used in shell scripts to
        launch executables generated via the csharp backend.

        Remove "MS_" prefixes on the variables MS_CSC and MS_ILASM, which are
        not Microsoft-specific.  More importantly, it should be less likely to
        make the mistake of adding an extra underscore to CSCFLAGS and
        ILASMFLAGS.

README.DotNet:
        Conform to variable renamings.

compiler/compile_target_code.m:
        Add new linked target types `csharp_executable', `java_launcher' and
        `erlang_launcher', instead of overloading `executable'.

        Link with `mer_std.dll' and other libraries when generating C#
        executables.  There is no `mer_rt.dll'.

        Pass "/debug" to the C# compiler if `--target-debug' is set.

        Create a shell script to launch the executable if necessary.

        Delete an unused predicate `standard_library_directory_option'.

compiler/file_names.m:
        `.cs' and `.cs_date' are grade-dependent.

compiler/handle_options.m:
        Force `.exe' as the executable file extension in csharp grades.

        Make the `erlang' grade component imply the same options as MLDS
        grades.

compiler/make.m:
        Classify executable target types based on the compilation target.

compiler/make.module_target.m:
        Handle `mmc --grade csharp --make <target>.dll'.

compiler/make.program_target.m:
        Install library DLLs in csharp grades.

        Make clean targets remove files for csharp grades.

        Conform to changes.

compiler/make.util.m:
        Add a stub foreign type.

        Conform to changes.

compiler/module_cmds.m:
        Factor out code to generate the shell scripts which launch programs
        compiled in Java, Erlang and C# grades.

compiler/options.m:
        Add `cli_interpreter' option to remember the name of the program which
        should be used to run CLI (.NET) programs.

        Add C#-related options to the help message.

compiler/options_file.m:
        Remove "MS_" prefixes on MS_ILASM_FLAGS and MS_CSC_FLAGS, and remove
        the extra underscore before "FLAGS".  In all uses of the variables,
        they were spelt without the extra underscore.

doc/user_guide.texi:
        Document options and file types related to the C# grade.

library/Mmakefile:
        Pass `mercury_dotnet.cs' to the C# compiler when building the standard
        library.  Suppress some warnings.

        Allow stubs in this directory for csharp grade.

        Conform to variable renamings.

library/builtin.m:
        Uncomment foreign language pragmas for C#.

        Handle null values in C# implementation of `deep_copy'.

library/private_builtin.m:
library/string.m:
        Compare strings by ordinals in C#, instead of culture-specific rules.
        Although the latter is allowed according to the documentation, it is
        likely to slower, and cause confusion when porting between backends.

        Handle negative index in string.set_char.

library/rtti_implementation.m:
        Uncomment foreign language pragmas for C#.

        `System.Type.GetType' only searches the current executing assembly or
        in mscorlib for a type.  As we have to be able to find types in other
        assemblies (e.g. mer_std.dll or user DLLs), explicitly search through
        a list of assemblies.

library/thread.semaphore.m:
        Uncomment foreign language pragmas for C#.

        Fix missing class qualification.

library/array.m:
library/bitmap.m:
library/bool.m:
library/dir.m:
library/exception.m:
library/io.m:
library/mutvar.m:
library/par_builtin.m:
library/region_builtin.m:
library/store.m:
library/thread.m:
library/time.m:
library/univ.m:
library/version_array.m:
        Uncomment foreign language pragmas for C#.

mdbcomp/rtti_access.m:
        Add type and procedure stubs.

runtime/mercury_dotnet.cs.in:
        Override `Equals(object)' methods in `TypeCtorInfo_Struct' and
        `TypeInfo_Struct' classes.  This requires we override `GetHashCode' as
        well.

        Handle nulls arguments to `Equals' methods as is the expected behaviour.

        Override `ToString' in `TypeCtorInfo_Struct' to produce more useful
        output during debugging.

scripts/Mercury.config.in:
        Record the configured CLI_INTERPRETER and pass that to the compiler as
        a flag.

        Conform to variable renamings.

scripts/Mmake.vars.in:
        Pass value of CSCFLAGS from Mmake through to `mmc --make'.

        Conform to variable renamings.

scripts/Mercury.config.bootstrap.in:
scripts/Mmake.rules:
        Conform to variable renaming.

scripts/canonical_grade.sh-subr:
scripts/final_grade_options.sh-subr:
scripts/init_grade_options.sh-subr:
scripts/parse_grade_options.sh-subr:
        Canonicalise high-level code, high-level-data, C# target code to the
        `csharp' grade.

        Handle erlang grades like other grades.

scripts/prepare_install_dir.in:
        Copy `.cs' files from the runtime directory when preparing an install
        directory.

browser/Mmakefile:
compiler/Mmakefile:
deep_profiler/Mmakefile:
mdbcomp/Mmakefile:
profiler/Mmakefile:
runtime/Mmakefile:
slice/Mmakefile:
ssdb/Mmakefile:
trace/Mmakefile:
        Do as other non-C grades in this directory.

        Conform to variable renamings.

tests/hard_coded/foreign_enum_dummy.m:
tests/hard_coded/sub-modules/non_word_mutable.m:
tests/hard_coded/sub-modules/sm_exp_bug.m:
        Make these tests work in C#.

tests/mmc_make/Mmakefile:
        Update a regular expression to account for `mmc --make' writing
        "Making rebuild.exe" on platforms where the .exe suffix is not normally
        used.

tests/mmc_make/complex_test.exp2:
        Add alternative output (minor difference in floating point precision).

tests/debugger/Mmakefile:
tests/debugger/declarative/Mmakefile:
tests/general/structure_reuse/Mmakefile:
tests/hard_coded/Mmakefile:
tests/hard_coded/sub-modules/Mmakefile:
tests/par_conj/Mmakefile:
tests/stm/Mmakefile:
        Disable some tests in the csharp grade.

tests/invalid/Mmakefile:
        Disable some tests in the csharp grade.

        Enable a test which should work in java grades.

tests/valid/Mmakefile:
        Do as other non-C grades in this directory.

        When testing the csharp grade in this directory, produce only the C#
        target files for now.

tests/run_one_test:
        Don't compress a failing test case executable when the executable is
        actually only a shell script.
 2010-09-30 07:23:36 +00:00
Zoltan Somogyi
863874df85 Document my recent change implementing coverage testing. 
Estimated hours taken: 6
Branches: main

Document my recent change implementing coverage testing. At the same time,
eliminate the old hack that allowed a file containing a list of file names to
be considered a trace count file. We haven't needed it since the addition of
mtc_union, and it can lead to incomprensible error messages. (The presence
of the old hack made documenting coverage testing harder.)

In the process, fix the tools code for rerunning failed test cases only.

doc/user_guide.texi:
	Document my recent change implementing coverage testing, and the
	elimination of the old hack.

mdbcomp/trace_counts.m:
	Modify the predicates for reading in trace count files along the lines
	above.

mdbcomp/slice_and_dice.m:
	Modify the predicates for reading in slices and dices along the lines
	above.

	Rename some function symbols to avoid ambiguities.

compiler/tupling.m:
slice/mcov.m:
slice/mtc_diff.m:
slice/mtc_union.m:
trace/mercury_trace_declarative.c:
	Conform to the changes above.

slice/mcov.m:
	Fix the usage message, which referred to this program by its old name
	mct.

	Allow the output to be restricted to a set of named modules only.
	This is to make testing easier.

slice/mtc_diff.m:
	Rename the long form of the -o option from --out to --output-file,
	to make it consistent with the other programs.

tests/run_one_test:
tools/bootcheck:
	Modify the algorithm we use to gather trace counts for the Mercury
	compiler from both passed and failed test cases to run mtc_union
	periodically instead of gathering all the trace counts file and keeping
	them to the end (which takes far too much disk space).

	Fix an old bug: gather trace counts from executions of the Mercury
	compiler only.

tests/debugger/Mmakefile:
tests/debugger/dice.passes:
	Modify the dice test case to compute the union of the trace counts for
	the passed versions of this test case to use mtc_union to create
	dice.passes, instead of having dice.passes statically contain the list
	of the names of the passed trace count files (since that capability
	is deleted by this diff).

tools/bootcheck:
tests/Mmake.common:
	Fix the code for rerunning failed tests only.

mdbcomp/prim_data.m:
	Eliminate some ambiguities in predicate names.

compiler/*.m:
	Conform to the change to prim_data.m.

compiler/error_util.m:
	Add reading files as a phase in error messages.

compiler/mercury_compile.m:
	Use the new facilities in error_util for printing an error message.
 2006-10-02 05:21:44 +00:00
Zoltan Somogyi
83f8deb354 Add two new command line tools "mslice" and "mdice", which manipulate slices 
Estimated hours taken: 24
Branches: main

Add two new command line tools "mslice" and "mdice", which manipulate slices
and dices respectively. The functionality of "mdice" duplicates the
functionality of mdb's current "dice" command, while the functionality of
"mslice" is new.

runtime/mercury_trace_base.[ch]:
	When generating trace count files, include information about the
	context of each execution count. While mdb has access to this
	information in the running program, mslice and mdice do not.
	In any case, the code in mdb for looking up this information was
	terribly inefficient.

	Provide a mechanism for recording all execution counts, even the zero
	ones, for use in coverage testing. This mechanism is not used yet.

	Put a header on trace counts files, to make them recognizable as such.
	Make this header indicate whether we are including zero trace counts.

runtime/mercury_wrapper.c:
	Provide a flag in MERCURY_OPTIONS for turning on recording of
	zero execution counts.

browser/dice.m:
mdbcomp/slice_and_dice.m:
	Move most of the code of browser/dice.m to mdbcomp/slice_and_dice.m,
	much modified; browser/dice.m is now empty. The modifications are
	as follows.

	Factor out the code for reading dices, since dices are useful
	independent of mdb's dice command.

	Generalize the code for computing dices to allow either or both
	of the slices being subtracted to be specified as the union of
	one or more trace counts files.

	Add two more sort conditions for comparing the execution counts
	in two slices: ascending and descending versions of simple execution
	count differences.

	For each operation we have so far performed on dices, add code
	for performing that operation on slices.

browser/mdb.m:
	Delete the include of dice.m, since it is now empty.

	Delete a duplicate include_module of term_rep, and delete the now
	unnecessary include_modules of set_cc and tree234_cc.

mdbcomp/mdbcomp.m:
	Add slice_and_dice.m as a submodule.

mdbcomp/trace_counts.m:
	Provide a mechanism to allow a slice to be specified as coming not
	from a single trace count file but from the union operation on a set
	of trace count files.

	Convert to four-space indentation to eliminate some excessively
	indented lines.

slice/mdice.m:
slice/mslice.m:
	The main modules of two new commands. Their functionality is almost
	entirely in mdbcomp/slice_and_dice.m and mdbcomp/trace_counts.m.

slice/Mmakefile:
	New Mmakefile for building mslice and mdice. It is modelled on
	profiler/Mmakefile.

slice/Mercury.options:
	Empty file, for use in the future.

slice/.nocopyright:
	Don't require copyright notice in Mercury.options.

Mmakefile:
	Process the slice directory at appropriate points when processing
	other directories.

compiler/tupling.m:
	Conform to the updated interface of mdbcomp/trace_counts.

library/list.m:
	Add versions of list__map for some more arities, for use in the code
	above.

trace/mercury_trace_internal.c:
	Generalize the code for specifying dices to allow either or both
	of the slices being subtracted to be specified as the union of
	one or more trace counts files.

	Fix several places where we weren't checking the return value of
	malloc. Fix two places where we could conceivably free strings that
	were still alive. Fix some places where we could pass NULL strings
	to Mercury code, and some places where we could free NULL pointers
	(which, once upon a time, was not guaranteed to work on all platforms).

	Use existing functions such as MR_copy_string where appropriate.

tests/run_one_test:
	Fix two small bugs in this script: make the filenames more user
	friendly, and make sure that gzip isn't asked to overwrite an
	existing file, since that causes it to ask a question on stdout
	and to wait for an answer.

tools/bootcheck:
	Copy or link the slice directory into stage 2. Rename the
	--copy-profiler option as --copy-profilers, since it now copies three
	directories containing performance engineering tools: profiler,
	deep_profiler and slice.
 2005-04-29 01:03:29 +00:00
Zoltan Somogyi
6ab59f41ab Provide a mechanism for gathering sets of trace counts from both passed and 
Estimated hours taken: 6
Branches: main

Provide a mechanism for gathering sets of trace counts from both passed and
failed test cases.

runtime/mercury_trace_base.[ch]:
	Instead of writing out trace counts to a file with a fixed name,
	write it out to a file whose name includes the name of the executable
	and the process id.

	Export only the functions that need to be exported.

	Add sanity checking code that looks for and reports occurrences
	of a situation reported by Peter Wang: positive counts for internal
	events in a procedure with a missing (and hence zero) count for the
	CALL event of that same procedure. Make this sanity check conditional
	on the new configuration macro MR_TRACE_COUNT_DEBUG. The sanity check
	did not find any anomalies during any of my bootchecks.

runtime/mercury_conf_param.h:
	Document MR_TRACE_COUNT_DEBUG.

tests/Mmake.common:
tests/run_one_test:
	Move the code for executing one test from a very long action in
	Mmake.common to the new shell script run_one_test, to make it possible
	to edit it without having to worry about line continuations.

tests/run_one_test:
	Make the code for executing one test move the trace count files
	generated by the test, if any, to the directory for passed or failed
	test cases, and record their names.

	Add a mechanism for creating a timestamp file for each test.

tools/bootcheck:
	Clean up the files and directories containing trace counts information
	from a previous bootcheck before running the tests this time.

	Add an option --progress, that creates a timestamp file for each test.
	This allows users to gauge the rate at which bootcheck executes test
	cases, and provides the information required for detecting performance
	problems that manifest themselves as slow test case execution.

	Allow the --trace-count option to be spelt --trace-counts.

	Since run_one_test now gathers a list of failed test in a much cleaner
	format than before (a simple list of the names of failed test cases,
	complete with their directories), print this list at the end of the
	output. This saves having to grep for FAILED TEST in the output file.

	The existing --failing-tests option did "dumpster diving" to discover
	the list of failing test cases, and rerun only them. This behavior
	is still available as the --error-file option, but --failing-tests
	now uses the much simpler record of which tests failed to repeat
	only failed tests.

	The new --specified-test option allows the rerunning of only a single
	test. This way, "tools/bootcheck -b- -s debugger/queens" will rerun
	only the queens test in the debugger directory. While you can often
	compile and run test cases by hand, it is difficult to compile and
	run them the same way as bootcheck does, especially for multi-module
	programs. This way, one can use the bootcheck script as a scalpel,
	not a club.

	Make this file use four-space indentation.

tests/Mmake.common:
	Clean up any stray trace count files.

	Implement the infrastructure required for --specified-test.

tests/debugger/Mmakefile:
	Conform to the new naming scheme for trace counts files.
 2005-03-24 01:58:12 +00:00