Estimated hours taken: 8
Branches: main
Implement a more cache-friendly translation of lookup switches. Previously,
for a switch such as the one in
:- pred p(foo::in, string::out, bar::out, float::out) is semidet.
p(d, "four", f1, 4.4).
p(e, "five", f2, 5.5).
p(f, "six", f4("hex"), 6.6).
p(g, "seven", f5(77.7), 7.7).
we generated three static cells, one for each argument, and then indexed
into each one in turn to get the values of HeadVar__2, HeadVar__3 and
HeadVar__4. The different static cells each represent a column here.
Each of the loads accessing the columns will access a different cache block,
so with this technique we expect to get as many cache misses as there are
output variables.
This diff changes the code we generate to use a vector of static cells
where each cell represents a row. The assignments to the output variables
will now access the different fields of a row, which will be next to each
other. We thus expect only one cache miss irrespective of the number of output
variables, at least up to the number of variables that actually fit into one
cache block.
compiler/global_data.m:
Provide a mechanism for creating not just single (scalar) static cells,
but arrays (vectors) of them.
compiler/lookup_switch.m:
Use the new mechanism to generate code along the lines described above.
Put the information passed between the two halves of the lookup switch
implementation (detection and code generation) into an opaque data
structure.
compiler/switch_gen.m:
Conform to the new interface of lookup_switch.m.
compiler/ll_pseudo_type_info.m:
compiler/stack_layout.m:
compiler/string_switch.m:
compiler/unify_gen.m:
compiler/var_locn.m:
Conform to the change to global_data.m.
compiler/llds.m:
Define the data structures for holding vectors of static cells. Rename
the function symbols we used to use to refer to static cells to make
clear that they apply to scalar cells only. Provide similar mechanisms
for representing static cell vectors and references to them.
Generalize heap_ref heap references to allow the index to be computed
at runtime, not compile time. For symmetry's sake, do likewise
for stack references.
compiler/llds_out.m:
Add the code required to write out static cell vectors.
Rename decl_ids to increase clarity and avoid ambiguity.
compiler/code_util.m:
compiler/exprn_aux.m:
Modify code that traverses rvals to now also traverse the new rvals
inside memory references.
compiler/name_mangle.m:
Provide the prefix for static cell vectors.
compiler/layout_out.m:
compiler/rtti_out.m:
compiler/opt_debug.m:
Conform to the change to data_addrs and decl_ids.
compiler/code_info.m:
Provide access to the new functionality in global_data.m, and conform
to the change to llds.m.
Provide a utility predicate needed by lookup_switch.m.
compiler/hlds_llds.m:
Fix the formatting of some comments.
tools/binary:
tools/binary_step:
Fix the bit rot that has set in since they were last used (the rest
of the system has changed quite a lot since then). I had to do so
to debug one part of this change.
tests/hard_coded/dense_lookup_switch2.{m,exp}:
tests/hard_coded/dense_lookup_switch3.{m,exp}:
New test cases to exercise the new algorithm.
tests/hard_coded/Mmakefile:
Enable the new test cases, as well as an old one (from 1997!)
that seems never to have been enabled.
Estimated hours taken: 12
Branches: main
Move the mdbcomp library to its own directory. To make this change less painful
to test, improve the way we handle installs.
browser/mdbcomp.m:
browser/mer_mdbcomp.m:
browser/prim_data.m:
browser/program_representation.m:
browser/trace_counts.m:
Move these files to the mdbcomp directory.
browser/Mmakefile:
browser/Mercury.options:
mdbcomp/Mmakefile:
mdbcomp/Mercury.options:
Split the contents of the old Mmakefile and Mercury.options file
in the browser directory between these files as appropriate.
Simplify away the stuff not needed now that there is only one library
per directory. Make the browser directory see the relevant files
from the mdbcomp directory.
Mmake.common.in:
Separate out the prefixes allowed in the browser and the mdbcomp
directories.
Mmake.workspace:
Set up a make variable to refer to the mdbcomp directory.
Adjust references to the mdbcomp library to point to its new location.
Mmakefile:
Make invocations visit the mdbcomp library as necessary.
Improve the way we install grades. Making temporary backups of the
directories modified by the install process is unsatisfactory for two
reasons. First, if the install fails, the cleanup script, which is
necessary for user friendliness, destroys any evidence of the cause.
Second, the restore of the backup wasn't perfect, e.g. it left the
.d files modified to depend on .mih files, which don't exist in
LLDS grades, and also left altered timestamps.
This diff changes the install process to make a single tmp_dir
subdirectory of the workspace, with all the work of install_grade
being done inside tmp_dir. The original directories aren't touched
at all.
*/Mmakefile:
Adjust references to the browser directory to refer to the mdbcomp
directory instead or as well.
scripts/Mmake.rules:
*/Mmakefile:
Make it easier to debug Mmakefiles. Previously, creating a
Mmake.makefile with mmake -s and invoking "make -d" ignored the
most fundamental rules of mmake, because Mmake.rules was treating
an unset MMAKE_USE_MMC_MAKE as if it were set to "yes", simply because
it was different from "no". This diff changes it to treat an unset
MMAKE_USE_MMC_MAKE as if it were set to "no", which is a more
sensible default.
scripts/prepare_tmp_dir_fixed_part.in:
scripts/scripts/prepare_tmp_dir_grade_part:
Two new scripts that each do half the work of preparing tmp_dir for
the real work of the install_grade make target. The fixed_part script
prepares the parts of tmp_dir that are grade-independent, while the
grade_part scripts prepares the parts that are grade-dependent.
configure.in:
Test C files in the mdbcomp directory to see whether they need to
be recompiled after reconfiguration.
Create prepare_tmp_dir_fixed_part from prepare_tmp_dir_fixed_part.in.
compiler/*.m:
runtime/mercury_wrapper.c:
Update the references to the moved files.
compiler/notes/overall_design.html:
Mention the new directory.
Estimated hours taken: 10
Branches: main
Split the existing browser library into two libraries, by making the
program_representation module into its own library. This is useful because
the compiler refers to program_representation.m, whose code thus needs to be
linked into compiler executables even if the compiler isn't compiled with
debugging enabled. By creating a new library for this module, we avoid any
chance of the linker dragging in the rest of the modules in the browser
library. (This is a problem with an upcoming diff.).
The name of the new library is "mdbcomp", because the intention is that it
contain code that is shared between the debugger and the compiler. This means
mostly the definitions of data structures that the compiler generates for the
debugger, and the predicates that operate on them.
Mmake.common.in:
Allow MDB_COMP_ as a prefix for symbol names in the browser directory.
Mmake.workspace:
Add a make variable holding for the name of the new library, and
add the name to the relevant lists of libraries.
Avoid duplicating the lists of filenames that need to be updated
when adding new libraries or changing their names.
Mmakefile:
Use make variables to refer to library names.
browser/mdbcomp.m:
browser/mer_mdbcomp.m:
Add these files as the top modules of the new library.
browser/program_representation.m:
Make program_representation.m a submodule of mdbcomp, not mdb.
browser/program_representation.m:
browser/browser_info.m:
Move a predicate from program_representation.m to browser_info.m
to avoid the mdbcomp library depend on the browser library, since
this would negate the point of the exercise.
browser/mdb.m:
Delete program_representation.m from the list of submodules.
browser/Mmakefile:
Update this file to handle the new module.
browser/Mercury.options:
Mention the new module.
browser/*.m:
Update the lists of imported modules. Import only one browser module
per line.
compiler/notes/overall_design.html:
Document the new library.
compiler/compile_target_code.m:
Add the mdbcomp library to the list of libraries we need to link with.
compiler/prog_rep.m:
trace/mercury_trace_internal.c:
Import program_representation.m by its new name.
scripts/c2init.in:
Centralize knowledge about which files need to be updated when the list
of libraries changes here.
scripts/c2init.in:
scripts/ml.in:
tools/binary:
tools/binary_step:
tools/bootcheck:
tools/linear:
tools/lml:
Update the list of libraries programs are linked with.
Estimated hours taken: 8
Branches: main
Factor out the common Mmake code to set up the mmc, mgnuc
and ml flags when building the Mercury compiler and libraries.
Use options, not environment variables. This will make it simpler
to use `mmc --make' to compile the compiler.
Mmake.workspace:
Similar to tools/lmc. Sets up Mmake variables to
use the library files in a workspace rather than
an installed library.
configure.in:
Check for the `--no-mercury-stdlib-dir' mmc option.
Bootstrap CVS tag: bootstrap_20020429_stdlib_dir
Mmake.common.in:
*/Mmakefile:
Move common code into Mmake.workspace.
browser/Mmakefile:
library/Mmakefile:
Avoid invoking the linker explicitly when creating
libraries of Mercury code. That won't work well
with `mmc --make'.
tools/bootcheck:
tests/Mmake.common:
Use Mmake.workspace instead of setting up environment
variables in bootcheck.
scripts/Mmake.vars.in:
mmc compiles split C files to object code itself,
so pass `--cflags "$(ALL_CFLAGS)"' to mmc when
compiling with `--split-c-files'.
browser/interactive_query.m:
Use `mmc --make' when compiling the query. This is needed
to make tests/debugger/interactive_query.m work when linking
against a workspace using options rather than environment
variables. This also fixes a bug -- mmc options were being
passed to ml.
Clean up after the query.
tests/debugger/Mmakefile:
tests/debugger/interactive.inp:
tests/debugger/interactive.inp.subdirs:
tests/debugger/interactive.inp.nosubdirs:
tests/debugger/interactive.exp:
tests/debugger/interactive.exp2:
Generate the input file to this test so that MCFLAGS
and MC_MAKE_FLAGS (from Mmake.workspace) are used when
compiling queries.
tests/debugger/Mmakefile now sets SHELL to /usr/local/bash
to allow the use of $(...) style command substitution
(`...` style command substitution can't be nested).
tests/warnings/Mmakefile:
tests/dppd/Mmakefile:
Include tests/Mmake.common.
tools/*:
scripts/c2init.in:
scripts/ml.in:
Update the lists of files containing the library names.
Estimated hours taken: 0.5
Branches: main
tools/bootcheck:
tools/binary:
tools/binary_step:
tools/linear:
tools/makebatch:
tools/test_mercury:
Change the options for these tools so that the short form of
negative options, such as `--no-test-suite', is `-t-' rather
than `-t', for consistency with the option handling convention
used by other Mercury programs such as `mmc' and `ml'.
I made this change because I found it very confusing
that `-t' *disabled* the tests in the `tests' directory,
whereas `-e' *enabled* the tests in the `extras' directory.
With this change, all options that disable things
are now of the form `-<letter>-' or `--no-<long-name>'.
Estimated hours taken: 0.2
tools/binary:
tools/divide:
tools/linear:
Fix some software rot in these scripts: we now need main.[co] in
the top level directory, and END_MODULE is now spelt MR_END_MODULE.
Estimated hours taken: 5
Improvements to the infrastructure for debugging code generator changes.
tools/binary:
If either stage2.ok or stage2.bad is missing object files, then do not
complain: instead, recreate the missing object files.
Fix a bug: copy the library's .pic_o files together with its .o files.
Fix a bug: make sure that we link *all* the possible relevant .init
files from the runtime to the stage2 directory.
If the search narrows down to one file, and the script is trying to
find out which part of the file is in error, then consider all the
parts that are identical between the stage2.ok and stage2.bad to be
known good from the start. This can reduce the number of bootchecks
needed by one or two.
tools/binary:
tools/binary_step:
Allow the test to be the successful compilation of the stage 3 library
directory. (In almost all cases, bad stage 2 compilers fail while
compiling the stage 3 library. By not compiling the stage 3 compiler
if the compilation of the stage 3 library succeeds, we can save a lot
of time.)
If the search narrows down to one file, and the script is trying to
find out which part of the file is in error, watch out for errors that
prevent the stage 2 executable from being built. If such an error
occurs, then stop right then and there. In such cases, there is no
point in further binary search, since each further invocation of
binary_step will just indicate that the source files in stage2.bad and
stage2.ok are not compatible (e.g. because they do not use the same
mapping from static term numbers to static term contents.)
tools/binary_step:
Reduce the time for declaring a program to be in an infinite loop,
since the slowest machine we may want to use is faster now.
tools/makebatch:
Fix some glaring bugs: e.g. test uses -lt, not < for comparisons.
Add an option, --stop-at-failure, that stops makebatch if a bootcheck
fails and thus preserves the stage[23] directories involved in the
failure. This allows one to verify that a change works in many grades
without sacrificing the ability to debug any problems.
Add another option --c-files, that gathers the C files created
in each bootcheck. This allows the C files to be compared, e.g.
for efficiency problems.
Estimated hours taken: 0.5
tools/binary:
tools/binary_step:
Update the binary script and its helper to accommodate the changes
in the compilation environment in the last year or so.
Estimated hours taken: 0.5
Fix some scripts in the tools directory to work with the new
directories and library names.
tools/binary:
tools/binary_step:
tools/linear:
Update these scripts to work properly with the new browser and
trace directories and the renamed libraries.
Mmake.common.in:
scripts/c2init.in:
scripts/ml.in:
Add tools/binary, tools/binary_step and tools/linear to the
list of files that know about library names.
Estimated hours taken: 0.05
tools/binary:
Disable command echo in some more parts of the script where the
output is not informative but quite voluminous.
Estimated hours taken: 0.2
tools/binary:
Update to handle the changes in the development environment
over the last few months, e.g. the default removal of object
files from stage 2 directories, and the requirement for
the presence of the VERSION and library/debugger_interface.h
files.
Do not set up the stage 3 directory if it is not required.
Disable command echo in some parts of the script where the
output is not informative but quite voluminous.
Simplify the conditions of some if-then-elses.
clashing with the default rule for creating a `.init' file
in the automatically-generated `library/library.dep' file.
The initialization for the library is different to the
standard automatically-generated initialization, because
the library contains C code that needs explicit initialization.
(because it has BEGIN_MODULE ... END_MODULE macros in it); this
C code hence has `INIT blah' comments in it naming functions that
need to be inserted in the `.init' file so that they will be called
at initialization time.
Mmakefile:
library/Mmakefile:
compiler/Mmakefile:
profiler/Mmakefile:
tools/bootcheck:
tools/linear:
tools/binary:
s/library.init/libmercury.init/g
Estimated hours taken: 1
binary_step:
Add a capability to compare stage3 to stage2.bad, not just to stage.ok,
for use when the stage3 compilation uses the Mmake parameters from
stage2.bad.
Fix a bug: when cleaning out the stage3 directories, remove not
just the source and interface files but also the date stamp and
optimization files.
binary:
Add a capability to generate stage3 using the Mmake parameters from
stage2.bad, with success being judged by comparing stage3 to
stage2.bad.
Use $RMSTAGECMD to remove the old stage2 if it is defined.
Estimated hours taken: 0.5
binary:
Remove some redundant code.
half:
Use /usr/ucb/echo if it exists, since on such machines (e.g. kryten)
the standard echo may not recognize -n.
Estimated hours taken: 1.5
binary, bootcheck:
If the environment variable RMSTAGECMD is defined, use it as a prefix
on commands to remove stages. This enables those with munkora accounts,
i.e. me :-) and noone else :-( to execute the rm command on munkora,
which is faster than removing large numbers of files across NFS.
bootcheck:
Exit with a non-zero status if the tests directory can't be found.
This should encourage people to run the tests.
optstages:
A script to break up the output of mc --debug-opt into several files,
each containing the output of one optimization stage. This makes it
easier to run diff and see what has changed.
Estimated hours taken: 1
binary:
Construct a stage3 before starting the search for the bug.
We used to be able to rely on a stage3 being already present,
but this reliance has been misplaced since Fergus's modification
to bootcheck that deletes stage3 on success.
binary_step:
Impose a limit of 10 CPU-minutes on every process, to catch
malfunctioning compilers that don't crash but go into infinite loops.
Estimated hours taken: 0.5
binary, binary_step:
Add a option, -d. When used, the successful making of the stage3
dependencies is taken to indicate the absence of the bug being searched
for.
Estimated hours taken: 1
Users of the "binary" script so far could choose between testing the
correctness of the cobbled-together stage2 compiler by attempting to
create a stage3 compiler, or by running all the tests in some directories.
This change adds a third alternative: checking whether a command executes
successfully. This is useful if the symptom of failure of the stage2
compiler is that it aborts when compiling some file.
Estimated hours taken: 2
bootcheck, binary, linear:
Link to absolute, not relative path names, since relative path names
do not work back across symbolic links.
binary, linear:
Allow the user to give an initial list of suspects.
assemble, binary, linear:
Allow negative searches, i.e. searches in which the base version is
the bad version and we want to see which module "fixes" the problem
caused by the other modules, as opposed to normal searches in which
we are looking for the module that introduces the problem.
Estimated hours taken: 0.5
I moved the scripts bootcheck, binary, binary_step, divide and assemble
into the tools subdirectory from the main mercury directory.
I also added the auxiliary scripts appears, cleanint and half.
I modified binary and binary_step to compare stage3 with stage2.ok,
and only pass a stage2 compiler if the comparison finds no differences.
This requires that the Mmake.params files be copied, not linked, from
the top level's Mmake.stage2.params. The old behavior (no comparison)
is still available; just specify -c (compile-only).
Estimated hours taken: 6
configure.in:
Rename BRANCH_DELAY_SLOT to HAVE_DELAY_SLOT.
bootcheck:
Add a new option, -r. If given, this makes a copy of the runtime
directory instead of linking to it in stage[23]. This allows the
stage[23] versions to use a different grade than stage1.
Another change is that we now remake only the library and compiler
dependencies, but not the profiler dependencies.
binary:
A shell script to find code generation and optimization bugs by
performing binary search. It depends on the existence of two
directories, stage2.ok and stage2.bad, containing correct and buggy
versions of stage2, and tries different mixes of .c code from each
until it locates the offending part of the offending .c file.
Note that this script has so far been tested only in pieces.
binary_step:
Check whether a particular mix of .c files from the ok and bad
directories is able to build a stage3 compiler. It doesn't check
for differences from stage2, since stage2 is a patchwork.
divide:
Divide a .c file into its parts.
assemble:
Assemble a .c file, with the specified parts coming from the .c file
in stage2.bad and the others from stage2.ok.
NOTE: these scripts require some other auxiliary scripts. I will put these
into /usr/local/contrib when it is created on cyclone. In the meantime,
they are in ~zs/bin/script.
