Estimated hours taken: 16
Branches: main
Allow the MLDS backend to generate indexing switches (switches implemented
more efficiently than just a if-then-else chain) for strings even if the target
language does not support gotos.
Previously, we use always used gotos to break out of search loops
after we found a match:
do {
if (we have a match) {
... handle the match ...
goto end
} else {
... handle nonmatches ...
}
} while (loop should continue);
maybe some code to handle the failure of the search
end:
Now, if the "maybe some code" is empty, we prefer to use break statements
if the target language supports this:
do {
if (we have a match) {
... handle the match ...
break;
} else {
... handle nonmatches ...
}
} while (loop should continue)
If we cannot use either gotos or break statements, we instead use
a boolean variable named "stop_loop":
stop_loop = 0;
do {
if (we have a match) {
... handle the match ...
stop_loop = 1;
} else {
... handle nonmatches ...
}
} while (stop_loop == 0 && loop should continue)
if (stop_loop == 0) {
maybe some code to handle the failure of the search
}
We omit the final if statement if the then-part would be empty.
The break method generates the smallest code, followed by the goto code.
I don't have information on speed, since we don't have a benchmark that
runs long enough, and the compiler itself does not spend any significant
amount of time on string switches. Probably the break method is also the
fastest, simply because it leaves the code looking most like normal C code.
(Some optimizations are harder to apply to code containing gotos, and some
optimizer writers do not bother.)
For C, we now normally prefer to generate code using the second method
(breaks), if we can, though normally "maybe some code" is not empty,
in which case we use the first method (goto).
However, if the value of the --experiment option is set to "use_stop_loop",
we always use the third method, and if it is set to "use_end_label", we always
use the first, even when we could use the second. This allow us to test all
three approaches using the C back end.
With backends that support neither gotos nor break, we always use the third
method (stop_loop).
With backends that don't support gotos but do support breaks, we also always
use the third method. This is because trying to use the second method would
require us to commit to not creating the stop_loop variable BEFORE we know
that the "maybe some code to handle the failure of the search" is empty,
and if it isn't empty, then we don't have the goto method to fall back on.
compiler/ml_string_switch.m:
Make the change described above. Where possible, make the required
change not to the original code, but to a version in which common
parts have been factored out. (Previously, the duplicated code was
small; now, it would be big.)
compiler/ml_target_util.m:
A new module containing existing functions that test various properties
of the target language. Keeping some of those functions in their
original modules would have introduced a circular dependency.
compiler/ml_switch_gen.m:
Enable the new functionality by removing the tests that previously
prevented the compiler from using indexing switches on strings
if the target language did not support gotos.
Remove the code moved to ml_target_util.m.
compiler/ml_optimize.m:
compiler/ml_unify_gen.m:
Remove the code moved to ml_target_util.m.
compiler/ml_backend.m:
compiler/notes/compiler_design.m:
Add the new module.
compiler/ml_proc_gen.m:
Delete a predicate that hasn't been used for a long time.
tools/makebatch:
Fix an old pair of typos.
Estimated hours taken: 0.1
Branches: main
tools/makebatch:
Provide a mechanism that allows users to add an arbitrary Mmakefile
fragment for each version in the batch.
Estimated hours taken: 6
Branches: main
Optimize higher order calls by providing variants of the relevant code that
are specialized to a given number of explicitly given arguments.
runtime/mercury_ho_call.[ch]:
Define variants of do_call_closure and do_call_class_method
specialized to 0, 1, 2 or 3 explicit input arguments. Apart from
not needing to be passed the number of explicit input arguments
in a register, these avoid some runtime tests and unroll loops.
Harmonize the variable names used in the do_call_closure and
do_call_class_method variants. Since they are near-copies of each
other, factor out their documentation. (Factoring out the code itself
would be possible, but would not make maintenance easier and would make
the code harder to read.)
Provide a mechanism to gather statistics about the numbers of hidden
and explicit arguments if the macro MR_DO_CALL_STATS is set.
compiler/options.m:
Add options that specify how many of these variants exist. These
provide the necessary synchronization between the runtime and the
compiler. They are not meant to be set from the command line, even by
implementors.
runtime/mercury_conf_params.h:
Document MR_DO_CALL_STATS.
runtime/mercury_wrapper.c:
If MR_DO_CALL_STATS is set, print the gathered statistics when
execution ends.
runtime/mecury_mm_own_stack.c:
Fix a typo that prevented the stage2 library from linking in jump.gc
grade.
compiler/llds.m:
Provide a way to represent the labels of the new specialized variants.
compiler/llds__out.m:
Output the labels of the new specialized variants if required.
Convert to four-space indentation.
compiler/call_gen.m:
Call the specialized variants of do_call_closure or
do_call_class_method if they are applicable.
code_info/follow_vars.m:
code_info/interval.m:
code_info/tupling.m:
Conform to the change in call_gen.m.
code_info/dupproc.m:
code_info/exprn_aux.m:
code_info/livemap.m:
code_info/opt_util.m:
Conform to the change in llds.m.
compiler/code_info.m:
Minor style cleanups.
tools/bootcheck:
Enable the collection of statistics from the compilation of stage 3
and the test cases, for use when the stage 2 is built with
MR_DO_CALL_STATS enabled.
tools/ho_call_stats:
A new script to analyze the statistics collected.
tools/makebatch:
Add a new option --save-stage2-on-no-compiler, which is a variant of
the existing option --save-stage2-on-error.
Estimated hours taken: 0.1
Branches: main
tools/makebatch:
Do not generate unnecessary error messages when saving C files for the
first time, and reduce the peak disk requirement by gzipping files
*before* copying them, not after.
Estimated hours taken: 0.1
Branches: main
tools/makebatch:
Make the checkpointing process more regular by making it work
the same way for all iterations, even the first.
Estimated hours taken: weeks
Branches: main
Implement a mechanism to generate the information required to determine the
algorithmic complexity of selected procedures.
The basis of the mechanism is a program transformation that wraps up the body
of each selected procedure in code that detects top-level (non-recursive)
calls, and for each top-level call, records the sizes of the input arguments
and information about the cost of the call. For now, the cost information
consists only of the number of cells and words allocated during the call,
but there is provision for later adding information from a real-time clock.
compiler/complexity.m:
A new module containing the new transformation.
compiler/transform_hlds.m:
Add complexity.m to the list of submodules.
compiler/mercury_compile.m:
Invoke the new module.
compiler/notes/compiler_design.html:
Mention the new module.
compiler/options.m:
Add an option, --experimental-complexity. Its argument is a filename
that specifies the list of procedures to transform.
Add an option, --no-allow-inlining, to disallow all inlining.
This is simpler to use than specifying several options to turn off
each potential reason to inline procedures.
doc/user_guide.texi:
Document the new options. The documentation present now is only a
shell; it will be expanded later.
compiler/table_gen.m:
compiler/goal_util.m:
Move the predicate for creating renamings from table_gen.m to
goal_util.m, since complexity.m also needs it now. In the process,
make it more general by allowing outputs to have more complex modes
than simply `out'.
compiler/goal_util.m:
Fix a bug exposed by the new transformation: when renaming goals
(e.g. for quantification), rename the variables holding information
about term sizes.
compiler/handle_options.m:
Disable inlining if experimental complexity analysis is enabled.
compiler/compile_target_code.m:
Pass the --experimental-complexity option on to the linker.
library/term_size_prof_builtin.m:
Add the Mercury predicates that serve as interfaces to the primitives
needed by the experimental complexity transformation.
runtime/mercury_term_size.[ch]:
Add the implementations of the primitives needed by the experimental
complexity transformation.
runtime/mercury_wrapper.[ch]:
Add global variables holding counters of the numbers of words and cells
allocated so far.
runtime/mercury_heap.h:
Update these global variables when allocating memory.
runtime/mercury_complexity.h:
New file that contains the definition of the data structures holding
the data collected by the experimental complexity transformation.
This is separate from mercury_term_size.h, because it needs to be
#included in mercury_init.h, the header file of the mkinit-generated
<program>_init.c files.
runtime/mercury_init.h:
runtime/mercury_imp.h:
#include mercury_complexity.h.
util/mkinit.c:
Define and initialize the data structures holding complexity
information when given the -X option (mkinit doesn't have long
options).
Fix some deviations from our coding style.
scripts/parse_ml_options.sh-subr.in:
Accept the --experiment-complexity option.
scripts/c2init.in:
Pass the --experiment-complexity option on to mkinit.c.
tools/bootcheck:
Preserve the files containing the results of complexity analysis,
if they exist.
tools/makebatch:
Allow the specification of EXTRA_MLFLAGS in the generated
Mmake.stage.params files.
Estimated hours taken: 16
Branches: main
Get Mercury to work with gcc 3.4. This required fixing several problems.
One problem that caused errors is that gcc 3.4 is smart enough to figure out
that in LLDS grades with gcc gotos, the C functions containing our code are
not referred to, and so it optimizes them away. The fix is to ensure that
mercury_<module>_init is defined always to call those functions, even if
the macro that usually controls this, MR_MAY_NEED_INITIALIZATION, is not
defined. The mercury_<module>_init won't be called from the init_modules
function in the program's _init.c file, so there is no impact on initialization
time, but gcc doesn't know this when compiling a module's .c file, so
it doesn't optimize away the code we need. The cost of this change is thus
only a small amount of code space. It is worth paying this cost even with
compilers other than gcc 3.4 for simplicity. Actually, this size increase seems
to be slightly smaller than the size reduction due to the better optimization
capabilities of gcc 3.4 compared to gcc 3.2.2.
A second problem is that gcc 3.4 warns about casts in lvalues being a
deprecated feature. This gave lots of warnings, since we used to define
several Mercury abstract machine registers, including MR_succip, MR_hp, MR_sp,
MR_maxfr and MR_curfr using lvalue casts. The fix is to have two macros
for each of these abstract machine registers, one of type MR_Word that you can
assign to (e.g. MR_sp_word), and one of the original type that is of the right
type but not an lvalue (e.g. MR_sp). The lvalue itself can't be made the right
type, because MR_sp isn't a variable in its own right, but possibly defined
to be a machine register. The machine register could made the right type,
but only at the cost of a lot of complexity.
This problem doesn't apply to the special-purpose Mercury abstract machine
registers that can't be allocated to machine registers. Instead of #defining
these to slots in MR_fake_reg, we make them global variables of the natural
type. This should also make it easier to debug code using these registers.
We treat these global variables as if they were machine registers in that
MR_save_registers copies values from these global variables to slots reserved
for them in the MR_fake_reg array, to allow code to loop over all Mercury
abstract machine registers. These saved slots must of course be of type
MR_Word, so we again need two macros to refer to them, a lvalue of type
MR_Word and an rvalue with the right type.
A third problem is that gcc 3.4 warns about conditionals in lvalues being a
deprecated feature. This gave a few warnings, since we used to define
MR_virtual_reg and MR_saved_reg using lvalues using conditionals. The fix
is to have one macro (MR_virtual_reg_value) for use in rvalues and a
separate macro which uses an if-then-else instead of a conditional
expression (MR_virtual_reg_assign), for assignments.
A fourth problem is that gcc 3.4 warns about comma operators in lvalues
being a deprecated feature. This gave warnings in the few places where
we refer to MR_r(N) for values of N that can map to fake registers directly,
since in those cases we preceded the reference to the fake_reg array with
a range check of the array index. The fix to this is to move the test to
compile time for compiler-generated code. Hand-written code never refers
to MR_r(N) for these values, and is very unlikely to do so in the future;
instead, it refers to the underlying fake_reg array directly, since that way
it doesn't have to worry about which fake registers have their own MR_rN macro
and which don't. Therefore no check mechanism for hand-written code is
necessary. This change mean that changing the number of MR_rN registers
now requires change to the compiler as well as to the runtime system.
A fifth problem is that gcc 3.4 by default assumes -fstrict-aliasing at -O2.
Since we cast between integers and pointers of different types all the time,
and changing that is not practical, at least in the short term, we need to
disable -fstrict-aliasing when we enable -O2.
NEWS:
Note that Mercury now works with gcc 3.4.
configure.in:
scripts/mgnuc.in:
Detect whether the compiler supports -fstrict-aliasing, and if so,
whether it assumes it by default with -O2. If the answer is yes to
both, make mgnuc specify -fno-strict-aliasing when it specifies -O2.
By including it in CFLAGS_FOR_OPT, which gets put into Mercury.config,
we also get -f-no-strict-aliasing when mmc invokes the C compiler
directly.
compiler/llds_out.m:
Don't generate #ifdef MR_MAY_NEED_INITIALIZATION around the definitions
and calls to the bunch functions, which call the functions we don't
want the C compiler to optimize away.
Generate the newly required lvalues on the left sides of assignments.
We still have code to generate LVALUE_CASTs in some cases, but I don't
think those cases ever arise.
Add a compile-time check of register numbers. Ideally, the code
generator should use stack slots instead of registers beyond the max
number, but I don't recall us ever bumping into this limit by accident.
compiler/fact_table.m:
Use the newly required lvalues on the left sides of assignments
in some hand-written C code included in generated .c files.
runtime/mercury_regs.h:
Make the changes described above to fix the second, third and fourth
problems. We still use comma operators in lvalues when counting
references to registers, but it is OK to require anyone who wants
to enable this feature to use a compiler version that supports comma
operators in lvalues or to ignore the warnings.
Use the same mapping from Mercury abstract machine registers to
the register count array as to the MR_fake_reg array.
Have this mapping depend as little as possible on whether we need a
real machine register to store MR_engine base, even if it costs a
wasted slot in MR_fake_reg.
Fix an old inconsistency: treat the Mercury abstract machine registers
used for trailing the same way as the other Mercury abstract machine
registers, by making MR_save_registers/MR_restore_registers copy them
to and from their global variable homes.
Document the requirement for the match between the runtime's and the
compiler's notions of the maximum MR_rN register number. This
requirement makes it harder for users to increase the number of
virtual registers, but as far as I know noone has wanted to do this.
Change the names of some of the macros to make them clearer.
Reorder some parts of this file, and add some documentation, also
in the interest of clarity.
runtime/mercury_regorder.h:
Delete this file after moving its contents, in much modified form,
to mercury_regs.h. mercury_regorder.h was always logically part of
mercury_regs.h, but was separated out to make it easier to change
the mapping from Mercury abstract machine registers to machine
registers. However, the cost of incompatibility caused by any such
changes would be much greater that any likely performance benefit.
runtime/Mmakefile:
Remove the reference to mercury_regorder.h.
runtime/mercury_regs.[ch]:
runtime/mercury_memory_zones.[ch]:
Move some functionality dealing with registers from
mercury_memory_zones to mercury_regs, since it belongs there.
runtime/mercury_regs.[ch]:
Add a function to make it easiler to debug changes to map from
Mercury abstract machine to MR_fake_reg slots.
runtime/mercury_regs.[ch]:
runtime/mercury_wrapper.c:
Move the code to print counts of register uses from mercury_wrapper.c
to mercury_regs.c.
Make mercury_wrapper.c call the debugging function in mercury_regs.c
if -X is specified in MERCURY_OPTIONS.
runtime/mercury_bootstrap.h:
Move the old MR_saved_reg and MR_virtual_reg macros from mercury_regs.h
to mercury_bootstrap.h to prevent their accidental use. Since
they shouldn't be used by user code, move them to the section
that is not enabled by default.
runtime/mercury_stacks.[ch]:
Add _word versions of the macros for stack slots, for the same reason
why we need them for Mercury abstract machine registers, and use them.
Add global variables for the Mercury abstract machine registers
for the gen, cut and pneg stacks.
runtime/mercury_heap.h:
Change the macros for allocating memory to assign to MR_hp_word instead
of MR_hp.
runtime/mercury_string.h:
Change the macros for allocating strings to accomodate the updates to
mercury_heap.h. Also change the expected type of the target to make it
MR_String instead of MR_ConstString, since the latter requires casts in
the caller.
runtime/mercury_trail.h:
runtime/mercury_types.h:
Move the definition of the type MR_TrailEntry from mercury_trail.h
to mercury_types.h, since it is now used in mercury_regs.h.
runtime/mercury_accurate_gc.c:
runtime/mercury_agc_debug.c:
runtime/mercury_calls.h:
runtime/mercury_context.[ch]:
runtime/mercury_deconstruct_macros.h:
runtime/mercury_deep_copy_body.h:
runtime/mercury_engine.[ch]:
runtime/mercury_hand_compare_body.h:
runtime/mercury_hand_unify_body.h:
runtime/mercury_ho_call.c:
runtime/mercury_layout_util.c:
runtime/mercury_make_type_info_body.h:
runtime/mercury_minimal_model.c:
runtime/mercury_ml_deconstruct_body.h:
runtime/mercury_ml_functor_body.h:
runtime/mercury_stack_layout.h:
runtime/mercury_type_desc.c:
runtime/mercury_type_info.c:
runtime/mercury_unify_compare_body.h:
runtime/mercury_wrapper.c:
Conform to the changes in the rest of the runtime.
In some cases, fix inconsistencies in indentation.
runtime/mercury_stack_trace.c:
Add some conditionally compiled debugging code controlled by the macro
MR_ADDR_DEBUG, to help debug some problems with stored stack pointers.
runtime/mercury_grade.h:
Increment the binary compatibility version number. This is needed to
avoid potential problems when a Mercury module and the debugger are
compiled with different versions of the macros in mercury_regs.h.
library/exception.m:
Update the code that assigns to abstract machine registers.
library/array.m:
library/construct.m:
library/dir.m:
library/io.m:
library/string.m:
Conform to the new definitions of allocation macros.
library/time.m:
Delete an unnecessary #include.
trace/mercury_trace.c:
trace/mercury_trace_declarative.c:
trace/mercury_trace_util.c:
Conform to the changes in the rest of the runtime.
tests/hard_coded/qual_test_is_imported.m:
tests/hard_coded/aditi_private_builtin.m:
Remove an unnecessary import to avoid a warning.
tools/makebatch:
Add an option --save-stage2-on-error, that saves the stage2 directory
if a bootcheck fails.
scripts/ml.in:
Make ml more robust in the face of garbage files.
Estimated hours taken: 0.1
Branches: main
tools/makebatch:
Preserve all output files in their entirety. This allows us to look for warnings in the
output, and we now have enough disk space.
Estimated hours taken: 0.2
Branches: main
tools/makebatch:
Fix a bug: initialize a shell variable that could otherwise be tested
without being assigned.
Estimated hours taken: 0.1
Branches: main
tools/makebatch:
Add an option to preserve the entire stage2 of each run in the batch.
This can be useful e.g. when one uses makebatch to find out which
optimization levels work and which do not, prior to applying
tools/binary.
Estimated hours taken: 0.1
Branches: main
tools/makebatch:
Copy the stage2 executables instead of linking them. Gzip refuses to
compress files with more than one link, and the
stage2/compiler/mercury_compile now has a link (in top_level).
Estimated hours taken: 0.5
Branches: main
tools/bootcheck:
tools/binary:
tools/binary_step:
tools/linear:
tools/makebatch:
tools/test_mercury:
Change the options for these tools so that the short form of
negative options, such as `--no-test-suite', is `-t-' rather
than `-t', for consistency with the option handling convention
used by other Mercury programs such as `mmc' and `ml'.
I made this change because I found it very confusing
that `-t' *disabled* the tests in the `tests' directory,
whereas `-e' *enabled* the tests in the `extras' directory.
With this change, all options that disable things
are now of the form `-<letter>-' or `--no-<long-name>'.
Estimated hours taken: 0.5
Branches: main
tools/makebatch:
Add a new option, --test-params, which cause that same option to
be passed on to the bootchecks invoked by makebatch.
Add a fourth control file that allows the specification of MGNUCFLAGS
in the bootcheck.
Estimated hours taken: 0.1
Branches: main
tools/makebatch:
Record the grade of each compiler version in the corresponding
parameter file, together with the options used in creating it.
Estimated hours taken: 0.2
Branches: main
tools/makebatch:
Add an option, --compile-times, for invoking bootcheck with that
option. Make the preservation of stage 2 compilers the default.
Estimated hours taken: 5
Improvements to the infrastructure for debugging code generator changes.
tools/binary:
If either stage2.ok or stage2.bad is missing object files, then do not
complain: instead, recreate the missing object files.
Fix a bug: copy the library's .pic_o files together with its .o files.
Fix a bug: make sure that we link *all* the possible relevant .init
files from the runtime to the stage2 directory.
If the search narrows down to one file, and the script is trying to
find out which part of the file is in error, then consider all the
parts that are identical between the stage2.ok and stage2.bad to be
known good from the start. This can reduce the number of bootchecks
needed by one or two.
tools/binary:
tools/binary_step:
Allow the test to be the successful compilation of the stage 3 library
directory. (In almost all cases, bad stage 2 compilers fail while
compiling the stage 3 library. By not compiling the stage 3 compiler
if the compilation of the stage 3 library succeeds, we can save a lot
of time.)
If the search narrows down to one file, and the script is trying to
find out which part of the file is in error, watch out for errors that
prevent the stage 2 executable from being built. If such an error
occurs, then stop right then and there. In such cases, there is no
point in further binary search, since each further invocation of
binary_step will just indicate that the source files in stage2.bad and
stage2.ok are not compatible (e.g. because they do not use the same
mapping from static term numbers to static term contents.)
tools/binary_step:
Reduce the time for declaring a program to be in an infinite loop,
since the slowest machine we may want to use is faster now.
tools/makebatch:
Fix some glaring bugs: e.g. test uses -lt, not < for comparisons.
Add an option, --stop-at-failure, that stops makebatch if a bootcheck
fails and thus preserves the stage[23] directories involved in the
failure. This allows one to verify that a change works in many grades
without sacrificing the ability to debug any problems.
Add another option --c-files, that gathers the C files created
in each bootcheck. This allows the C files to be compared, e.g.
for efficiency problems.
Estimated hours taken: 1
tools/makebatch:
Conform to an old change to bootcheck: grades must be passed on the
command line, not in Mmake.params.
Generate files that sort the same way alphabetically as well as
numerically.
Estimated hours taken: 0.3
tools/makebatch:
Add an option that allows the object files from failed bootchecks to be
saved, for space measurement purposes (e.g. for the RTTI paper).
Estimated hours taken: 0.2
makebatch:
Add a new option, -f, which tells makebatch that it should collect
stage2 compilers even if the bootcheck fails. This is intended to
make it easier to make compilers in grades without garbage collection.
Such compilers should be able to compile small modules on large
memory machines, even though they run out of memory when compiling
large modules, such as those in the compiler itself.
Estimated hours taken: 0.1
makebatch:
Instead of letting the -c and -g options to decide whether
to use the .CFLAGS and .GRADE files, use them they exist.
Remove the -c and -g options.
Estimated hours taken: 2
bootcheck:
By default, execute all the tests after verifying that the compiler
compiles itself to a fix point. Don't execute the tests if called
with the -t option.
makebatch:
Pass the -t option to bootcheck if makebatch is itself called with -t.
test_mercury:
Run the tests on the bootstrapped compiler *before* installing it,
and don't install it unless it passes all the tests.
Cycle through a list of different parameter settings, so that
in general different nightly runs use different sets of options.
Call bootcheck with -r, since some of these options may cause
a compilation model difference between the current and new compiler.
list.*:
Lists of parameter settings for various machines.
expand_params:
Expand the paramater settings from one line of a list.x file
into a Mmake.param file
cur_param:
Report which line in a list.x file is the current one.
next_param:
Advance the current line in a list.x file. When we reach the end,
we start again at the start.
