... using an approach proposed by Peter, with an extra twist from Julien.
Instead of having two modules, getopt.m and getopt_io.m, with the former
defining predicates that do not take an I/O state pair, and the latter
defining predicates that do take an I/O state pair, put both kinds of
predicates into a single module. The versions with an I/O state pair
have an "_io" suffix added to their names for disambiguation.
Both versions are a veneer on top of a common infrastructure,
which relies on a simple type class to implement the operation
"give the contents of the file with this name". The predicate versions
with I/O state pairs have a normal implementation of this typeclass,
while the predicate versions that do not have I/O state pairs
have an implementation that always returns an error indication.
The above change just about doubles the number of exported predicates.
We already had two versions of most exported predicates that differed
in whether we returned errors in the form of a string, or in the form
of a structured representation, with names of the latter having
an "_se" suffix. Since we agreed that the structured representation
is the form we want to encourage, this diff deletes the string versions,
and deletes the "_se" suffix from the predicate names that used to have them.
(It still remains at the end of the name of a type.) This "undoubling"
should offset the effect of the doubling in the previous paragraph.
Eventually, we want to have just one module, getopt.m, containing
the updated code described above, but for now, we put the same code
into both getopt_io.m and getopt.m to prevent too big a shock to
people with existing code that uses getopt_io.m.
library/getopt.m:
library/getopt_io.m:
Make the changes described above.
library/Mmakefile:
Instead of building both getopt_io.m and getopt.m from getopt_template,
build getopt.m from getopt_io.m.
tools/bootcheck:
Delete references to getopt_template.
compiler/typecheck_errors.m:
When a type error involves one of the getopt/getopt_io predicates
whose interfaces are changed by this diff, tell the user about
how these changes could have caused the error, and thus what the
probable fix is.
compiler/handle_options.m:
browser/parse.m:
deep_profiler/mdprof_cgi.m:
deep_profiler/mdprof_create_feedback.m:
deep_profiler/mdprof_dump.m:
deep_profiler/mdprof_procrep.m:
deep_profiler/mdprof_report_feedback.m:
deep_profiler/mdprof_test.m:
profiler/mercury_profile.m:
slice/mcov.m:
slice/mdice.m:
slice/mslice.m:
slice/mtc_diff.m:
slice/mtc_union.m:
tests/hard_coded/space.m:
Use the updated getopt interface.
compiler/compile_target_code.m:
compiler/compute_grade.m:
compiler/deforest.m:
compiler/det_report.m:
compiler/format_call.m:
compiler/globals.m:
compiler/goal_expr_to_goal.m:
compiler/make.build.m:
compiler/make.m:
compiler/make.module_dep_file.m:
compiler/make.program_target.m:
compiler/make.util.m:
compiler/mercury_compile_main.m:
compiler/ml_top_gen.m:
compiler/module_cmds.m:
compiler/op_mode.m:
compiler/optimization_options.m:
compiler/options.m:
compiler/write_module_interface_files.m:
tools/make_optimization_options_middle:
tools/make_optimization_options_start:
Replace references to getopt_io.m with references to getopt.m.
tests/invalid/getopt_io_old.{m,err_exp}:
tests/invalid/getopt_old.{m,err_exp}:
tests/invalid/getopt_old_se.{m, err_exp}:
New test cases for the extra help
tests/invalid/Mmakefile:
Enable the new test cases.
library/LIB_FLAGS.in:
mfilterjavac/MFILTERJAVAC_FLAGS.in:
profiler/PROF_FLAGS.in:
slice/SLICE_FLAGS.in:
ssdb/SSDB_FLAGS.in:
Specify --warn-inconsistent-pred-order-clauses as a default flag
in these directories as well.
library/string.m:
Reorder code to make this module compile cleanly with
--warn-inconsistent-pred-order-clauses.
Don't export base_string_to_int_underscore to lexer.m, since
lexer.m does not use it anymore, and neither does anything else.
(The export was not publicly documented.)
library/benchmarking.m:
library/bitmap.m:
library/builtin.m:
library/map.m:
library/profiling_builtin.m:
library/rbtree.m:
library/table_builtin.m:
library/uint.m:
profiler/globals.m:
profiler/options.m:
profiler/prof_info.m:
slice/mdice.m:
slice/mslice.m:
slice/mtc_diff.m:
slice/mtc_union.m:
ssdb/ssdb.m:
Reorder code to make these modules compile cleanly with
--warn-inconsistent-pred-order-clauses.
library/Mercury.options:
Specify --no-warn-inconsistent-pred-order-clauses for the modules
that would still get warnings without this option.
Estimated hours taken: 1
Branches: main
Prevent not just the deep profiler, but also the other programs in the
Mercury system from running out of stack in hlc grades.
mdbcomp/shared_utilities.m:
A new module to contain the predicate that increases OS limits
on stack usage as much as possible.
mdbcomp/mdbcomp.m:
Include the new module in this package.
deep_profiler/startup.m:
Remove the stack unlimit predicate that is now in
mdbcomp/shared_utilities.m.
deep_profiler/Mmakefile:
slice/Mmakefile:
Copy the new file over from mdbcomp, along with the rest of the files
in the mdbcomp package.
compiler/mercury_compile.m:
slice/mcov.m:
slice/mdice.m:
slice/mslice.m:
slice/mtc_diff.m:
slice/mtc_union.m:
Invoke the stack unlimit predicate.
mdbcomp/prim_data.m:
Fix a missing divider.
Estimated hours taken: 3
Branches: main
Minor improvements in mslice/mdice output formats.
library/string.m:
Add format_table_max, a version of format_table that allows the caller
to specify a maximum width for each column. If a column exceeds that
maximum, format_table_max adds a newline, and starts the line again
at the correct position for the next column.
mdbcomp/slice_and_dice.m:
Use format_table_max instead of format_table to generate the tables
containing slice data.
slice/mslice.m:
slice/mdice.m:
Add some options that callers can use to specify the max widths of
the predicate name, path/port and filename/linenumber fields. Pass
the values of these options on to slice_and_dice.m.
Make the default sort string print the most frequently executed events,
not the ones from the predicates that come first in the lexicographic
sequence.
slice/Mmakefile:
Fix some missing dependencies.
doc/user_guide.texi:
Document the new options.
Estimated hours taken: 60
Branches: main
Implement coverage testing. The output format is a bit crude, but people
have been asking for this capability.
The main problem tackled in this diff is that coverage testing requires
gathering information from a lot of program executions, and the execution count
files for all these executions require a huge amount of disk space. We now
therefore put a limit on the number of files we keep; when this limit is
exceeded, the program execution that reaches the limit will automatically
summarize all these files back into a single file before it exits.
This diff also tackles the same problem along a different axis by changing
the format of execution count files to make them smaller. One way is to factor
out and represent just once some information that is common to many procedures:
the file name and the module name. Another is to abbreviate some keywords,
e.g. "fproc" instead of "proc function". The third is not to write out the
defining module's name unless it differs from the declaring module's name,
which it almost never does. (The two differ only when the compiler is invoked
with intermodule optimization, and creates a specialized version of a predicate
in a module other than its home module.)
Since we are changing the trace count file format anyway, make another change
useful for coverage testing: record the entire provenance of the trace counts
in the file, including the name of the program and what files went into unions
and diffs of trace count files.
When doing coverage testing of the compiler, the compiler *must* be in a debug
grade. However, the tools for summarizing trace files, invoked from the
compiler executable when the compiler is being coverage tested, *cannot* be
in debug grade, because debug grade disables tail recursion, and without tail
recursion the summarization program runs out of stack space. This diff
therefore arranges for the slice directory to not be affected by the parameters
applying to the rest of the workspace (including the top level Mmake.params).
Mmakefile:
Don't apply the top level mmake's parameters to recursive mmakes in
the slice directory.
Factor out some common code.
configure.in:
Require that the installed compiler contain the renamed standard
library function names installed by my diff on Sep 20, since the
slice directory needs them, and cannot get them from the workspace.
mdbcomp/trace_counts.m:
Update the parsing code to parse the new format for trace count files,
and update the code for writing out trace counts to generate the new
format.
Replace the proc_label_and_filename type with the proc_label_in_context
type, which makes it easier to keep track of the current module as well
as the current file (this is required by the new, more compact format
for trace count files).
When considering the union of multiple trace counts files, keep track
of whether they contained all counts or just the nonzero counts. This
requires keeping track of this info for single files as well.
Provide ways to represent and to compute differences between trace
count files, to support the new program in slice/mtc_diff.m.
mdbcomp/slice_and_dice.m:
Reformat to conform to our Mercury style guide.
Conform to the change to trace_counts.m.
compiler/tupling.m:
Conform to the change to mdbcomp.
runtime/mercury_wrapper.c:
Implement the new option values used to implement coverage testing.
These allow control of the limit on the number of execution count
files, and collecting execution counts only from a specified
executable.
Add MR_ prefixes.
runtime/mercury_trace_base.[ch]:
Provide the mechanism for summarizing execution counts when we reach
the limit on the number of execution counts files.
Update the code that writes out trace counts files to generate
the new format for trace counts files. Make this code take the boolean
that says whether to include labels with zero counts in the output
as an explicit parameter, not as a global variable.
Break up an excessively large function.
scripts/mtc:
Add the options needed to control the process of automatic
summarization of trace counts files.
slice/.mgnuc_copts:
slice/.mgnuc_opts:
slice/SLICE_FLAGS.in:
Make these files empty, since we don't want to refer to the rest of the
workspace. (We could delete them as well, but CVS doesn't handle
resurrection of deleted files very well, and we don't want to burn any
bridges.)
slice/Mmakefile:
Add the new executables, and make the code in this directory
independent of the other directories in the workspace.
Since we need the code of the modules in the mdbcomp directory
but don't want to link to the object files in that directory (since
the grades may differ), make copies of those modules in this directory.
slice/mcov.m:
Add this module, the code for the Mercury coverage test tool.
slice/mtc_diff.m:
Add this module, the code for computing the diff between two trace
counts files. The intended use is to compare two trace counts files
dumped at different stages of execution. (Since foreign_procs can be
used to invoke the C functions in the runtime that write out the trace
counts files in the middle of a program's execution, not just the end.)
slice/mdice.m:
slice/mslice.m:
slice/mtc_union.m:
Convert to four space indentation.
tools/bootcheck:
Since the slice directory's grade is independent of the grade of the
other directories, don't copy it to the stage2 and stage3 by default.
If it is copied, then still compile it (and otherwise handle it)
separate from the other directories.
Add an option for gathering coverage test data during bootchecking.
Estimated hours taken: 0.1
Branches: main
Add newline characters to the usage messages of mslice and mdice.
slice/mdice.m:
slice/mslice.m:
Add newline characters.
Estimated hours taken: 24
Branches: main
Add two new command line tools "mslice" and "mdice", which manipulate slices
and dices respectively. The functionality of "mdice" duplicates the
functionality of mdb's current "dice" command, while the functionality of
"mslice" is new.
runtime/mercury_trace_base.[ch]:
When generating trace count files, include information about the
context of each execution count. While mdb has access to this
information in the running program, mslice and mdice do not.
In any case, the code in mdb for looking up this information was
terribly inefficient.
Provide a mechanism for recording all execution counts, even the zero
ones, for use in coverage testing. This mechanism is not used yet.
Put a header on trace counts files, to make them recognizable as such.
Make this header indicate whether we are including zero trace counts.
runtime/mercury_wrapper.c:
Provide a flag in MERCURY_OPTIONS for turning on recording of
zero execution counts.
browser/dice.m:
mdbcomp/slice_and_dice.m:
Move most of the code of browser/dice.m to mdbcomp/slice_and_dice.m,
much modified; browser/dice.m is now empty. The modifications are
as follows.
Factor out the code for reading dices, since dices are useful
independent of mdb's dice command.
Generalize the code for computing dices to allow either or both
of the slices being subtracted to be specified as the union of
one or more trace counts files.
Add two more sort conditions for comparing the execution counts
in two slices: ascending and descending versions of simple execution
count differences.
For each operation we have so far performed on dices, add code
for performing that operation on slices.
browser/mdb.m:
Delete the include of dice.m, since it is now empty.
Delete a duplicate include_module of term_rep, and delete the now
unnecessary include_modules of set_cc and tree234_cc.
mdbcomp/mdbcomp.m:
Add slice_and_dice.m as a submodule.
mdbcomp/trace_counts.m:
Provide a mechanism to allow a slice to be specified as coming not
from a single trace count file but from the union operation on a set
of trace count files.
Convert to four-space indentation to eliminate some excessively
indented lines.
slice/mdice.m:
slice/mslice.m:
The main modules of two new commands. Their functionality is almost
entirely in mdbcomp/slice_and_dice.m and mdbcomp/trace_counts.m.
slice/Mmakefile:
New Mmakefile for building mslice and mdice. It is modelled on
profiler/Mmakefile.
slice/Mercury.options:
Empty file, for use in the future.
slice/.nocopyright:
Don't require copyright notice in Mercury.options.
Mmakefile:
Process the slice directory at appropriate points when processing
other directories.
compiler/tupling.m:
Conform to the updated interface of mdbcomp/trace_counts.
library/list.m:
Add versions of list__map for some more arities, for use in the code
above.
trace/mercury_trace_internal.c:
Generalize the code for specifying dices to allow either or both
of the slices being subtracted to be specified as the union of
one or more trace counts files.
Fix several places where we weren't checking the return value of
malloc. Fix two places where we could conceivably free strings that
were still alive. Fix some places where we could pass NULL strings
to Mercury code, and some places where we could free NULL pointers
(which, once upon a time, was not guaranteed to work on all platforms).
Use existing functions such as MR_copy_string where appropriate.
tests/run_one_test:
Fix two small bugs in this script: make the filenames more user
friendly, and make sure that gzip isn't asked to overwrite an
existing file, since that causes it to ask a question on stdout
and to wait for an answer.
tools/bootcheck:
Copy or link the slice directory into stage 2. Rename the
--copy-profiler option as --copy-profilers, since it now copies three
directories containing performance engineering tools: profiler,
deep_profiler and slice.
