Files
mercury/compiler/notes/compiler_design.html
Zoltan Somogyi f007b45df8 Implement the infrastructure for term size profiling.
Estimated hours taken: 400
Branches: main

Implement the infrastructure for term size profiling. This means adding two
new grade components, tsw and tsc, and implementing them in the LLDS code
generator. In grades including tsw (term size words), each term is augmented
with an extra word giving the number of heap words it contains; in grades
including tsc (term size cells), each term is augmented with an extra word
giving the number of heap cells it contains. The extra word is at the start,
at offset -1, to leave almost all of the machinery for accessing the heap
unchanged.

For now, the only way to access term sizes is with a new mdb command,
"term_size <varspec>". Later, we will use term sizes in conjunction with
deep profiling to do experimental complexity analysis, but that requires
a lot more research. This diff is a necessary first step.

The implementation of term size profiling consists of three main parts:

- a source-to-source transform that computes the size of each heap cell
  when it is constructed (and increments it in the rare cases when a free
  argument of an existing heap cell is bound),

- a relatively small change to the code generator that reserves the extra
  slot in new heap cells, and

- extensions to the facilities for creating cells from C code to record
  the extra information we now need.

The diff overhauls polymorphism.m to make the source-to-source transform
possible. This overhaul includes separating type_ctor_infos and type_infos
as strictly as possible from each other, converting type_ctor_infos into
type_infos only as necessary. It also includes separating type_ctor_infos,
type_infos, base_typeclass_infos and typeclass_infos (as well as voids,
for clarity) from plain user-defined type constructors in type categorizations.
This change needs this separation because values of those four types do not
have size slots, but they ought to be treated specially in other situations
as well (e.g. by tabling).

The diff adds a new mdb command, term_size. It also replaces the proc_body
mdb command with new ways of using the existing print and browse commands
("print proc_body" and "browse proc_body") in order to make looking at
procedure bodies more controllable. This was useful in debugging the effect
of term size profiling on some test case outputs. It is not strictly tied
to term size profiling, but turns out to be difficult to disentangle.

compiler/size_prof.m:
	A new module implementing the source-to-source transform.

compiler/notes/compiler_design.html:
	Mention the new module.

compiler/transform_hlds.m:
	Include size_prof as a submodule of transform_hlds.

compiler/mercury_compile.m:
	If term size profiling is enabled, invoke its source-to-source
	transform.

compiler/hlds_goal.m:
	Extend construction unifications with an optional slot for recording
	the size of the term if the size is a constant, or the identity of the
	variable holding the size, if the size is not constant. This is
	needed by the source-to-source transform.

compiler/quantification.m:
	Treat the variable reference that may be in this slot as a nonlocal
	variable of construction unifications, since the code generator needs
	this.

compiler/compile_target_code.m:
	Handle the new grade components.

compiler/options.m:
	Implement the options that control term size profiling.

doc/user_guide.texi:
	Document the options and grade components that control term size
	profiling, and the term_size mdb command. The documentation is
	commented out for now.

	Modify the wording of the 'u' HLDS dump flag to include other details
	of unifications (e.g. term size info) rather than just unification
	categories.

	Document the new alternatives of the print and browse commands. Since
	they are for developers only, the documentation is commented out.

compiler/handle_options.m:
	Handle the implications of term size profiling grades.

	Add a -D flag value to print HLDS components relevant to HLDS
	transformations.

compiler/modules.m:
	Import the new builtin library module that implements the operations
	needed by term size profiling automatically in term size profiling
	grades.

	Switch the predicate involved to use state var syntax.

compiler/prog_util.m:
	Add predicates and functions that return the sym_names of the modules
	needed by term size profiling.

compiler/code_info.m:
compiler/unify_gen.m:
compiler/var_locn.m:
 	Reserve an extra slot in heap cells and fill them in in unifications
	marked by size_prof.

compiler/builtin_ops.m:
	Add term_size_prof_builtin.term_size_plus as a builtin, with the same
	implementation as int.+.

compiler/make_hlds.m:
	Disable warnings about clauses for builtins while the change to
	builtin_ops is bootstrapped.

compiler/polymorphism.m:
	Export predicates that generate goals to create type_infos and
	type_ctor_infos to add_to_construct.m. Rewrite their documentation
	to make it more detailed.

	Make orders of arguments amenable to the use of state variable syntax.

	Consolidate knowledge of which type categories have builtin unify and
	compare predicates in one place.

	Add code to leave the types of type_ctor_infos alone: instead of
	changing their types to type_info when used as arguments of other
	type_infos, create a new variable of type type_info instead, and
	use an unsafe_cast. This would make the HLDS closer to being type
	correct, but this new code is currently commented out, for two
	reasons. First, common.m is currently not smart enough to figure out
	that if X and Y are equal, then similar unsafe_casts of X and Y
	are also equal, and this causes the compiler do not detect some
	duplicate calls it used to detect. Second, the code generators
	are also not smart enough to know that if Z is an unsafe_cast of X,
	then X and Z do not need separate stack slots, but can use the same
	slot.

compiler/type_util.m:
	Add utility predicates for returning the types of type_infos and
	type_ctor_infos, for use by new code in polymorphism.m.

	Move some utility predicates here from other modules, since they
	are now used by more than one module.

	Rename the type `builtin_type' as `type_category', to better reflect
	what it does. Extend it to put the type_info, type_ctor_info,
	typeclass_info, base_typeclass_info and void types into categories
	of their own: treating these types as if they were a user-defined
	type (which is how they used to be classified) is not always correct.
	Rename the functor polymorphic_type to variable_type, since types
	such as list(T) are polymorphic, but they fall into the user-defined
	category. Rename user_type as user_ctor_type, since list(int) is not
	wholly user-defined but falls into this category. Rename pred_type
	as higher_order_type, since it also encompasses functions.

	Replace code that used to check for a few of the alternatives
	of this type with code that does a full switch on the type,
	to ensure that they are updated if the type definition ever
	changes again.

compiler/pseudo_type_info.m:
	Delete a predicate whose updated implementation is now in type_util.m.

compiler/mlds_to_c.m:
compiler/mlds_to_gcc.m:
compiler/mlds_to_il.m:
compiler/mlds_to_java.m:
	Still treat type_infos, type_ctor_infos, typeclass_infos and
	base_typeclass_infos as user-defined types, but prepare for when
	they won't be.

compiler/hlds_pred.m:
	Require interface typeinfo liveness when term size profiling is
	enabled.

	Add term_size_profiling_builtin.increase_size as a
	no_type_info_builtin.

compiler/hlds_out.m:
	Print the size annotations on unifications if HLDS dump flags call
	for unification details. (The flag test is in the caller of the
	modified predicate.)

compiler/llds.m:
	Extend incr_hp instructions and data_addr_consts with optional fields
	that allow the code generator to refer to N words past the start of
	a static or dynamic cell. Term size profiling uses this with N=1.

compiler/llds_out.m:
	When allocating memory on the heap, use the macro variants that
	specify an optional offset, and specify the offset when required.

compiler/bytecode_gen.m:
compiler/dense_switch.m:
compiler/dupelim.m:
compiler/exprn_aux.m:
compiler/goal_form.m:
compiler/goal_util.m:
compiler/higher_order.m:
compiler/inst_match.m:
compiler/intermod.m:
compiler/jumpopt.m:
compiler/lambda.m:
compiler/livemap.m:
compiler/ll_pseudo_type_info.m:
compiler/lookup_switch.m:
compiler/magic_util.m:
compiler/middle_rec.m:
compiler/ml_code_util.m:
compiler/ml_switch_gen.m:
compiler/ml_unify_gen.m:
compiler/mlds.m:
compiler/mlds_to_c.m:
compiler/mlds_to_gcc.m:
compiler/mlds_to_il.m:
compiler/mlds_to_java.m:
compiler/modecheck_unify.m:
compiler/opt_debug.m:
compiler/opt_util.m:
compiler/par_conj_gen.m:
compiler/post_typecheck.m:
compiler/reassign.m:
compiler/rl.m:
compiler/rl_key.m:
compiler/special_pred.m:
compiler/stack_layout.m:
compiler/static_term.m:
compiler/string_switch.m:
compiler/switch_gen.m:
compiler/switch_util.m:
compiler/table_gen.m:
compiler/term_util.m:
compiler/type_ctor_info.m:
compiler/unused_args.m:
compiler/use_local_vars.m:
	Minor updates to conform to the changes above.

library/term_size_prof_builtin.m:
	New module containing helper predicates for term size profiling.
	size_prof.m generates call to these predicates.

library/library.m:
	Include the new module in the library.

doc/Mmakefile:
	Do not include the term_size_prof_builtin module in the library
	documentation.

library/array.m:
library/benchmarking.m:
library/construct.m:
library/deconstruct.m:
library/io.m:
library/sparse_bitset.m:
library/store.m:
library/string.m:
	Replace all uses of MR_incr_hp with MR_offset_incr_hp, to ensure
	that we haven't overlooked any places where offsets may need to be
	specified.

	Fix formatting of foreign_procs.

	Use new macros defined by the runtime system when constructing
	terms (which all happen to be lists) in C code. These new macros
	specify the types of the cell arguments, allowing the implementation
	to figure out the size of the new cell based on the sizes of its
	fields.

library/private_builtin.m:
	Define some constant type_info structures for use by these macros.
	They cannot be defined in the runtime, since they refer to types
	defined in the library (list.list and std_util.univ).

util/mkinit.c:
	Make the addresses of these type_info structures available to the
	runtime.

runtime/mercury_init.h:
	Declare these type_info structures, for use in mkinit-generated
	*_init.c files.

runtime/mercury_wrapper.[ch]:
	Declare and define the variables that hold these addresses, for use
	in the new macros for constructing typed lists.

	Since term size profiling can refer to a memory cell by a pointer
	that is offset by one word, register the extra offsets with the Boehm
	collector if is being used.

	Document the incompatibility of MR_HIGHTAGS and the Boehm collector.

runtime/mercury_tags.h:
	Define new macros for constructing typed lists.

	Provide macros for preserving the old interface presented by this file
	to the extent possible. Uses of the old MR_list_cons macro will
	continue to work in grades without term size profiling. In term
	size profiling grades, their use will get a C compiler error.

	Fix a bug caused by a missing backslash.

runtime/mercury_heap.h:
	Change the basic macros for allocating new heap cells to take
	an optional offset argument. If this is nonzero, the macros
	increment the returned address by the given number of words.
	Term size profiling specifies offset=1, reserving the extra
	word at the start (which is ignored by all components of the
	system except term size profiling) for holding the size of the term.

	Provide macros for preserving the old interface presented by this file
	to the extent possible. Since the old MR_create[123] and MR_list_cons
	macros did not specify type information, they had to be changed
	to take additional arguments. This affects only hand-written C code.

	Call new diagnostic macros that can help debug heap allocations.

	Document why the macros in this files must expand to expressions
	instead of statements, evn though the latter would be preferable
	(e.g. by allowing them to declare and use local variables without
	depending on gcc extensions).

runtime/mercury_debug.[ch]:
	Add diagnostic macros to debug heap allocations, and the functions
	behind them if MR_DEBUG_HEAP_ALLOC is defined.

	Update the debugging routines for hand-allocated cells to print the
	values of the term size slot as well as the other slots in the relevant
	grades.

runtime/mercury_string.h:
	Provide some needed variants of the macro for copying strings.

runtime/mercury_deconstruct_macros.h:
runtime/mercury_type_info.c:
	Supply type information when constructing terms.

runtime/mercury_deep_copy_body.h:
	Preserve the term size slot when copying terms.

runtime/mercury_deep_copy_body.h:
runtime/mercury_ho_call.c:
runtime/mercury_ml_expand_body.h:
	Use MR_offset_incr_hp instead of MR_incr_hp to ensure that all places
	that allocate cells also allocate space for the term size slot if
	necessary.

	Reduce code duplication by using a now standard macro for copying
	strings.

runtime/mercury_grade.h:
	Handle the two new grade components.

runtime/mercury_conf_param.h:
	Document the C macros used to control the two new grade components,
	as well as MR_DEBUG_HEAP_ALLOC.

	Detect incompatibilities between high level code and profiling.

runtime/mercury_term_size.[ch]:
	A new module to house a function to find and return term sizes
	stored in heap cells.

runtime/mercury_proc_id.h:
runtime/mercury_univ.h:
	New header files. mercury_proc_id.h contains the (unchanged)
	definition of MR_Proc_Id, while mercury_univ.h contains the
	definitions of the macros for manipulating univs that used to be
	in mercury_type_info.h, updated to use the new macros for allocating
	memory.

	In the absence of these header files, the following circularity
	would ensue:

	mercury_deep_profiling.h includes mercury_stack_layout.h
		- needs definition of MR_Proc_Id
	mercury_stack_layout.h needs mercury_type_info.h
		- needs definition of MR_PseudoTypeInfo
	mercury_type_info.h needs mercury_heap.h
		- needs heap allocation macros for MR_new_univ_on_hp
	mercury_heap.h includes mercury_deep_profiling.h
		- needs MR_current_call_site_dynamic for recording allocations

	Breaking the circular dependency in two places, not just one, is to
	minimize similar problems in the future.

runtime/mercury_stack_layout.h:
	Delete the definition of MR_Proc_Id, which is now in mercury_proc_id.h.

runtime/mercury_type_info.h:
	Delete the macros for manipulating univs, which are now in
	mercury_univ.h.

runtime/Mmakefile:
	Mention the new files.

runtime/mercury_imp.h:
runtime/mercury.h:
runtime/mercury_construct.c:
runtime/mercury_deep_profiling.h:
	Include the new files at appropriate points.

runtime/mercury.c:
	Change the names of the functions that create heap cells for
	hand-written code, since the interface to hand-written code has
	changed to include type information.

runtime/mercury_tabling.h:
	Delete some unused macros.

runtime/mercury_trace_base.c:
runtime/mercury_type_info.c:
	Use the new macros supplying type information when constructing lists.

scripts/canonical_grade_options.sh-subr:
	Fix an undefined sh variable bug that could cause error messages
	to come out without identifying the program they were from.

scripts/init_grade_options.sh-subr:
scripts/parse_grade_options.sh-subr:
scripts/canonical_grade_options.sh-subr:
scripts/mgnuc.in:
	Handle the new grade components and the options controlling them.

trace/mercury_trace_internal.c:
	Implement the mdb command "term_size <varspec>", which is like
	"print <varspec>", but prints the size of a term instead of its value.
	In non-term-size-profiling grades, it prints an error message.

	Replace the "proc_body" command with optional arguments to the "print"
	and "browse" commands.

doc/user_guide.tex:
	Add documentation of the term_size mdb command. Since the command is
	for implementors only, and works only in grades that are not yet ready
	for public consumption, the documentation is commented out.

	Add documentation of the new arguments of the print and browse mdb
	commands. Since they are for implementors only, the documentation
	is commented out.

trace/mercury_trace_vars.[ch]:
	Add the functions needed to implement the term_size command, and
	factor out the code common to the "size" and "print"/"browse" commands.

	Decide whether to print the name of a variable before invoking the
	supplied print or browse predicate on it based on a flag design for
	this purpose, instead of overloading the meaning of the output FILE *
	variable. This arrangement is much clearer.

trace/mercury_trace_browse.c:
trace/mercury_trace_external.c:
trace/mercury_trace_help.c:
	Supply type information when constructing terms.

browser/program_representation.m:
	Since the new library module term_size_prof_builtin never generates
	any events, mark it as such, so that the declarative debugger doesn't
	expect it to generate any.

	Do the same for the deep profiling builtin module.

tests/debugger/term_size_words.{m,inp,exp}:
tests/debugger/term_size_cells.{m,inp,exp}:
	Two new test cases, each testing one of the new grades.

tests/debugger/Mmakefile:
	Enable the two new test cases in their grades.

	Disable the tests sensitive to stack frame sizes in term size profiling
	grades.

tests/debugger/completion.exp:
	Add the new "term_size" mdb command to the list of command completions,
	and delete "proc_body".

tests/debugger/declarative/dependency.{inp,exp}:
	Use "print proc_body" instead of "proc_body".

tests/hard_coded/nondet_c.m:
tests/hard_coded/pragma_inline.m:
	Use MR_offset_incr_hp instead of MR_incr_hp to ensure that all places
	that allocate cells also allocate space for the term size slot if
	necessary.

tests/valid/Mmakefile:
	Disable the IL tests in term size profiling grades, since the term size
	profiling primitives haven't been (and probably won't be) implemented
	for the MLDS backends, and handle_options causes a compiler abort
	for grades that combine term size profiling and any one of IL, Java
	and high level C.
2003-10-20 07:29:59 +00:00

1507 lines
52 KiB
HTML

<html>
<head>
<title>
Notes On The Design Of The Mercury Compiler
</title>
</head>
<body bgcolor="#ffffff" text="#000000">
<hr>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<p>
This file contains an overview of the design of the compiler.
<p>
See also <a href="overall_design.html">overall_design.html</a>
for an overview of how the different sub-systems (compiler,
library, runtime, etc.) fit together.
<hr>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<h2> OUTLINE </h2>
<p>
The main job of the compiler is to translate Mercury into C, although it
can also translate (subsets of) Mercury to some other languages:
Mercury bytecode (for a planned bytecode interpreter), MSIL (for the
Microsoft .NET platform) and RL (the Aditi Relational Language).
<p>
The top-level of the compiler is in the file mercury_compile.m,
which is a sub-module of the top_level.m package.
The basic design is that compilation is broken into the following
stages:
<ul>
<li> 1. parsing (source files -&gt; HLDS)
<li> 2. semantic analysis and error checking (HLDS -&gt; annotated HLDS)
<li> 3. high-level transformations (annotated HLDS -&gt; annotated HLDS)
<li> 4. code generation (annotated HLDS -&gt; target representation)
<li> 5. low-level optimizations
(target representation -&gt; target representation)
<li> 6. output code (target representation -&gt; target code)
</ul>
<p>
Note that in reality the separation is not quite as simple as that.
Although parsing is listed as step 1 and semantic analysis is listed
as step 2, the last stage of parsing actually includes some semantic checks.
And although optimization is listed as steps 3 and 5, it also occurs in
steps 2, 4, and 6. For example, elimination of assignments to dead
variables is done in mode analysis; middle-recursion optimization and
the use of static constants for ground terms is done in code
generation; and a few low-level optimizations are done in llds_out.m
as we are spitting out the C code.
<p>
In addition, the compiler is actually a multi-targeted compiler
with several different back-ends.
<p>
The modules in the compiler are structured by being grouped into
"packages". A "package" is just a meta-module,
i.e. a module that contains other modules as sub-modules.
(The sub-modules are almost always stored in separate files,
which are named only for their final module name.)
We have a package for the top-level, a package for each main pass, and
finally there are also some packages for library modules that are used
by more than one pass.
<p>
Taking all this into account, the structure looks like this:
<ul type=disc>
<li> At the top of the dependency graph is the top_level.m package,
which currently contains only the one module mercury_compile.m
which invokes all the different passes in the compiler.
<li> The next level down is all of the different passes of the compiler.
In general, we try to stick by the principle that later passes can
depend on data structures defined in earlier passes, but not vice
versa.
<ul type=disc>
<li> front-end
<ul type=disc>
<li> 1. parsing (source files -&gt; HLDS)
<br> Packages: parse_tree.m and hlds.m
<li> 2. semantic analysis and error checking (HLDS -&gt; annotated HLDS)
<br> Package: check_hlds.m
<li> 3. high-level transformations (annotated HLDS -&gt; annotated HLDS)
<br> Package: transform_hlds.m
</ul>
<li> back-ends
<ul type=disc>
<li> a. LLDS back-end
<br> Package: ll_backend.m
<ul type=disc>
<li> 3a. LLDS-back-end-specific HLDS-&gt;HLDS transformations
<li> 4a. code generation (annotated HLDS -&gt; LLDS)
<li> 5a. low-level optimizations (LLDS -&gt; LLDS)
<li> 6a. output code (LLDS -&gt; C)
</ul>
<li> b. MLDS back-end
<br> Package: ml_backend.m
<ul type=disc>
<li> 4b. code generation (annotated HLDS -&gt; MLDS)
<li> 5b. MLDS transformations (MLDS -&gt; MLDS)
<li> 6b. output code
(MLDS -&gt; C or MLDS -&gt; MSIL or MLDS -&gt; Java, etc.)
</ul>
<li> c. RL back-end
<br> Package: aditi_backend.m
<ul type=disc>
<li> 3c. RL-back-end-specific HLDS-&gt;HLDS transformations
<li> 4c. code generation (annotated HLDS -&gt; RL)
<li> 5c. low-level optimizations (RL -&gt; RL)
<li> 6c. output code (RL -&gt; RL-bytecode)
</ul>
<li> d. bytecode back-end
<br> Package: bytecode_backend.m
<ul type=disc>
<li> 4d. code generation (annotated HLDS -&gt; bytecode)
</ul>
<li> There's also a package backend_libs.m which contains
modules which are shared between several different back-ends.
</ul>
</ul>
<li> Finally, at the bottom of the dependency graph there is the package
libs.m. libs.m contains the option handling code, and also library
modules which are not sufficiently general or sufficiently useful to
go in the Mercury standard library.
</ul>
<p>
In addition to the packages mentioned above, there are also packages
for the build system: make.m contains the support for the `--make' option,
and recompilation.m contains the support for the `--smart-recompilation'
option.
<hr>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<h2> DETAILED DESIGN </h2>
<p>
This section describes the role of each module in the compiler.
For more information about the design of a particular module,
see the documentation at the start of that module's source code.
<hr>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<p>
The action is co-ordinated from mercury_compile.m or make.m (if `--make'
was specified on the command line).
<h3> Option handling </h3>
<p>
Option handling is part of the libs.m package.
<p>
The command-line options are defined in the module options.m.
mercury_compile.m calls library/getopt.m, passing the predicates
defined in options.m as arguments, to parse them. It then invokes
handle_options.m to postprocess the option set. The results are
stored in the io__state, using the type globals defined in globals.m.
<h3> Build system </h3>
<p>
Support for `--make' is in the make.m package,
which contains the following modules:
<dl>
<dt> make.m
<dd>
Categorizes targets passed on the command line and passes
them to the appropriate module to be built.
<dt> make.program_target.m
<dd>
Handles whole program `mmc --make' targets, including
executables, libraries and cleanup.
<dt> make.module_target.m
<dd>
Handles targets built by a compilation action associated
with a single module, for example making interface files,
<dt> make.dependencies.m
<dd>
Compute dependencies between targets and between modules.
<dt> make.module_dep_file.m
<dd>
Record the dependency information for each module between
compilations.
<dt> make.util.m
<dd>
Utility predicates.
<dt> options_file.m
<dd>
Read the options files specified by the `--options-file'
option. Also used by mercury_compile.m to collect the value
of DEFAULT_MCFLAGS, which contains the auto-configured flags
passed to the compiler.
</dl>
The build process also invokes routines in compile_target_code.m,
which is part of the backend_libs.m package (see below).
<p>
<hr>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<h3> FRONT END </h3>
<h4> 1. Parsing </h4>
<h5> The parse_tree.m package </h5>
<p>
The first part of parsing is in the parse_tree.m package,
which contains the modules listed below
(except for the library/*.m modules,
which are in the standard library).
This part produces the parse_tree.m data structure,
which is intended to match up as closely as possible
with the source code, so that it is suitable for tasks
such as pretty-printing.
<p>
<ul>
<li> <p> lexical analysis (library/lexer.m)
<li> <p> stage 1 parsing - convert strings to terms. <p>
library/parser.m contains the code to do this, while
library/term.m and library/varset.m contain the term and varset
data structures that result, and predicates for manipulating them.
<li> <p> stage 2 parsing - convert terms to `items'
(declarations, clauses, etc.)
<p>
The result of this stage is a parse tree that has a one-to-one
correspondence with the source code. The parse tree data structure
definition is in prog_data.m, while the code to create it is in
prog_io.m and its submodules prog_io_dcg.m (which handles clauses
using Definite Clause Grammar notation), prog_io_goal.m (which handles
goals), prog_io_pragma.m (which handles pragma declarations),
prog_io_typeclass.m (which handles typeclass and instance declarations)
and prog_io_util.m (which defines predicates and types needed by the
other prog_io*.m modules. The data structure for insts is stored in
its own module, inst.m.
<p>
The modules prog_out.m and mercury_to_mercury.m contain predicates
for printing the parse tree. prog_util.m contains some utility
predicates for manipulating the parse tree.
<li><p> imports and exports are handled at this point (modules.m)
<p>
modules.m has the code to write out `.int', `.int2', `.int3',
`.d' and `.dep' files.
<p>
source_file_map.m contains code to read, write and search
the mapping between module names and file names.
<li><p> module qualification of types, insts and modes
<p>
module_qual.m - <br>
Adds module qualifiers to all types insts and modes,
checking that a given type, inst or mode exists and that
there is only possible match. This is done here because
it must be done before the `.int' and `.int2' interface files
are written. This also checks whether imports are really needed
in the interface.
<p>
Notes on module qualification:
<ul>
<li> all types, typeclasses, insts and modes occuring in pred, func,
type, typeclass and mode declarations are module qualified by
module_qual.m.
<li> all types, insts and modes occuring in lambda expressions,
explicit type qualifications, and clause mode annotations
are module qualified in make_hlds.m.
<li> constructors occuring in predicate and function mode declarations
are module qualified during type checking.
<li> predicate and function calls and constructors within goals
are module qualified during mode analysis.
</ul>
<li><p> reading and writing of optimization interfaces
(intermod.m and trans_opt.m -- these are part of the
hlds.m package, not the parse_tree.m package).
<p>
&lt;module&gt;.opt contains clauses for exported preds suitable for
inlining or higher-order specialization. The `.opt' file for the
current module is written after type-checking. `.opt' files
for imported modules are read here.
&lt;module&gt;.opt contains termination analysis information
for exported preds (eventually it ought to contain other
"transitive" information too, e.g. for optimization, but
currently it is only used for termination analysis).
`.trans_opt' files for imported modules are read here.
The `.trans_opt' file for the current module is written
after the end of semantic analysis.
<li><p> expansion of equivalence types (equiv_type.m)
<p>
`with_type` and `with_inst` annotations on predicate
and function type and mode declarations are also expanded.
<p>
Expansion of equivalence types is really part of type-checking,
but is done on the item_list rather than on the HLDS because it
turned out to be much easier to implement that way.
</ul>
<p>
That's all the modules in the parse_tree.m package.
<h5> The hlds.m package </h5>
<p>
Once the stages listed above are complete, we then convert from the parse_tree
data structure to a simplified data structure, which no longer attempts
to maintain a one-to-one correspondence with the source code.
This simplified data structure is called the High Level Data Structure (HLDS),
which is defined in the hlds.m package.
<p>
The last stage of parsing is this conversion to HLDS:
<ul>
<li><p> conversion to superhomogeneous form and into HLDS
<p>
make_hlds.m transforms the clauses into superhomogeneous form,
and at the same time converts the parse tree into the HLDS.
It expands away state variable syntax, universal quantification
(using `all [Vs] G' ===&gt; `not (some [Vs] (not G))')
and implication (using `A =&gt; B' ===&gt; `not(A, not B)').
It converts `pragma import', `pragma c_code' and `pragma fact_table'
declarations into clauses with HLDS `pragma_c_code'
instructions for bodies.
The `pragma fact_table' conversion is done by calling fact_table.m
(which is part of the ll_backend.m package); fact_table.m also
reads the facts from the declared file and compiles them
into a separate C file for which the `pragma_c_code' contains lookup
code.
make_hlds.m also calls make_tags.m which chooses the data
representation for each discriminated union type by
assigning tags to each functor.
make_hlds.m also performs a number of semantic checks,
such as checking for circular insts and modes
and warning about singleton variables.
</ul>
<p>
The HLDS data structure itself is spread over four modules:
<ol>
<li> hlds_data.m defines the parts of the HLDS concerned with
function symbols, types, insts, modes and determinisms;
<li> hlds_goal.m defines the part of the HLDS concerned with the
structure of goals, including the annotations on goals;
<li> hlds_pred.m defines the part of the HLDS concerning
predicates and procedures;
<li> hlds_module.m defines the top-level parts of the HLDS,
including the type module_info.
</ol>
<p>
The module hlds_out.m contains predicates to dump the HLDS to a file.
<p>
The hlds.m package also contains some utility modules that contain
various library routines which are used by other modules that manipulate
the HLDS:
<dl>
<dt> hlds_code_util.m
<dd> Utility routines for use during HLDS generation.
<dt> goal_form.m
<dd> Contains predicates for determining whether
HLDS goals match various criteria.
<dt> goal_util.m
<dd> Contains various miscellaneous utility predicates for
manipulating HLDS goals, e.g. for renaming variables.
<dt> passes_aux.m
<dd> Contains code to write progress messages, and higher-order
code to traverse all the predicates defined in the current
module and do something with each one.
<dt> error_util.m:
<dd> Utility routines for printing nicely formatted error messages.
</dl>
<h4> 2. Semantic analysis and error checking </h4>
<p>
This is the check_hlds.m package.
<p>
Any pass which can report errors or warnings must be part of this stage,
so that the compiler does the right thing for options such as
`--halt-at-warn' (which turns warnings into errors) and
`--error-check-only' (which makes the compiler only compile up to this stage).
<dl>
<dt> implicit quantification
<dd>
quantification.m (XXX which for some reason is part of the hlds.m
package rather than the check_hlds.m package)
handles implicit quantification and computes
the set of non-local variables for each sub-goal.
It also expands away bi-implication (unlike the expansion
of implication and universal quantification, this expansion
cannot be done until after quantification).
This pass is called from the `transform' predicate in make_hlds.m.
<dt> checking typeclass instances (check_typeclass.m)
<dd>
check_typeclass.m both checks that instance declarations satisfy all
the appropriate superclass constraints and
performs a source-to-source transformation on the
methods from the instance declarations.
The transformed code is checked for type, mode, uniqueness, purity
and determinism correctness by the later passes, which has the effect
of checking the correctness of the instance methods themselves
(ie. that the instance methods match those expected by the typeclass
declaration).
During the transformation,
pred_ids and proc_ids are assigned to the methods for each instance.
<p>
In
addition, while checking that the superclasses of a class are satisfied
by the instance declaration, a set of constraint_proofs are built up
for the superclass constraints. These are used by polymorphism.m when
generating the base_typeclass_info for the instance.
<dt> type checking
<dd>
<ul>
<li> typecheck.m handles type checking, overloading resolution &amp;
module name resolution, and almost fully qualifies all predicate
and functor names. It sets the map(var, type) field in the
pred_info. However, typecheck.m doesn't figure out the pred_id
for function calls or calls to overloaded predicates; that can't
be done in a single pass of typechecking, and so it is done
later on (in post_typecheck.m, for both preds and function calls)
Typeclass constraints are checked here, and
any redundant constraints that are eliminated are recorded (as
constraint_proofs) in the pred_info for future reference.
<li> type_util.m contains utility predicates dealing with types
that are used in a variety of different places within the compiler
<li> post_typecheck.m may also be considered to logically be a part
of typechecking, but it is actually called from purity
analysis (see below). It contains the stuff related to
type checking that can't be done in the main type checking pass.
It also removes assertions from further processing.
post_typecheck.m reports errors for unbound type and inst variables,
for unsatisified type class constraints, for indistinguishable
predicate or function modes, and for invalid Aditi calls and updates.
</ul>
<dt> assertions
<dd>
assertion.m (XXX in the hlds.m package)
is the abstract interface to the assertion table.
Currently all the compiler does is type check the assertions and
record for each predicate that is used in an assertion, which
assertion it is used in. The set up of the assertion table occurs
in post_typecheck__finish_assertion.
<dt> purity analysis
<dd>
purity.m is responsible for purity checking, as well as
defining the <CODE>purity</CODE> type and a few public
operations on it. It also calls post_typecheck.m to
complete the handling of predicate
overloading for cases which typecheck.m is unable to handle,
and to check for unbound type variables.
Elimination of double negation is also done here; that needs to
be done after quantification analysis and before mode analysis.
Calls to `private_builtin__unsafe_type_cast/2' are converted
into `generic_call(unsafe_cast, ...)' goals here.
<dt> polymorphism transformation
<dd>
polymorphism.m handles introduction of type_info arguments for
polymorphic predicates and introduction of typeclass_info arguments
for typeclass-constrained predicates.
This phase needs to come before mode analysis so that mode analysis
can properly reorder code involving existential types.
(It also needs to come before simplification so that simplify.m's
optimization of goals with no output variables doesn't do the
wrong thing for goals whose only output is the type_info for
an existentially quantified type parameter.)
<p>
This phase also
converts higher-order predicate terms into lambda expressions,
and copies the clauses to the proc_infos in preparation for
mode analysis.
<p>
The polymorphism.m module also exports some utility routines that
are used by other modules. These include some routines for generating
code to create type_infos, which are used by simplify.m and magic.m
when those modules introduce new calls to polymorphic procedures.
<p>
When it has finished, polymorphism.m calls clause_to_proc.m to
make duplicate copies of the clauses for each different mode of
a predicate; all later stages work on procedures, not predicates.
<dt> mode analysis
<dd>
<ul>
<li> modes.m is the main mode analysis module.
It checks that the code is mode-correct, reordering it
if necessary, and annotates each goal with a delta-instmap
that specifies the changes in instantiatedness of each
variable over that goal.
<li> modecheck_unify.m is the sub-module which analyses
unification goals.
It also module qualifies data constructors.
<li> modecheck_call.m is the sub-module which analyses calls.
<p>
The following sub-modules are used:
<dl>
<dt> mode_info.m
<dd>
(the main data structure for mode analysis)
<dt> delay_info.m
<dd>
(a sub-component of the mode_info data
structure used for storing the information
for scheduling: which goals are currently
delayed, what variables they are delayed on, etc.)
<dt> instmap.m (XXX in the hlds.m package)
<dd>
Defines the instmap and instmap_delta ADTs
which store information on what instantiations
a set of variables may be bound to.
<dt> inst_match.m
<dd>
This contains the code for examining insts and
checking whether they match.
<dt> inst_util.m
<dd>
This contains the code for creating new insts from
old ones: unifying them, merging them and so on.
<dt> mode_errors.m
<dd>
This module contains all the code to
print error messages for mode errors
</dl>
<li> mode_util.m contains miscellaneous useful predicates dealing
with modes (many of these are used by lots of later stages
of the compiler)
<li> mode_debug.m contains utility code for tracing the actions
of the mode checker.
</ul>
<dt> indexing and determinism analysis
<dd>
<ul>
<li> switch_detection.m transforms into switches those disjunctions
in which several disjuncts test the same variable against different
function symbols.
<li> cse_detection.m looks for disjunctions in which each disjunct tests
the same variable against the same function symbols, and hoists any
such unifications out of the disjunction.
If cse_detection.m modifies the code,
it will re-run mode analysis and switch detection.
<li> det_analysis.m annotates each goal with its determinism;
it inserts cuts in the form of "some" goals wherever the determinisms
and delta instantiations of the goals involved make it necessary.
Any errors found during determinism analysis are reported by
det_report.m.
det_util.m contains utility predicates used in several modules.
</ul>
<dt> checking of unique modes (unique_modes.m)
<dd>
unique_modes.m checks that non-backtrackable unique modes were
not used in a context which might require backtracking.
Note that what unique_modes.m does is quite similar to
what modes.m does, and unique_modes calls lots of predicates
defined in modes.m to do it.
<dt> stratification checking
<dd>
The module stratify.m implements the `--warn-non-stratification'
warning, which is an optional warning that checks for loops
through negation.
<dt> simplification (simplify.m)
<dd>
simplify.m finds and exploits opportunities for simplifying the
internal form of the program, both to optimize the code and to
massage the code into a form the code generator will accept.
It also warns the programmer about any constructs that are so simple
that they should not have been included in the program in the first
place. (That's why this pass needs to be part of semantic analysis:
because it can report warnings.)
simplify.m converts complicated unifications into procedure calls.
simplify.m calls common.m which looks for (a) construction unifications
that construct a term that is the same as one that already exists,
or (b) repeated calls to a predicate with the same inputs, and replaces
them with assignment unifications.
simplify.m also attempts to partially evaluate calls to builtin
procedures if the inputs are all constants (this is const_prop.m
in the transform_hlds.m package).
</dl>
<h4> 3. High-level transformations </h4>
<p>
This is the transform_hlds.m package.
<p>
The first pass of this stage does tabling transformations (table_gen.m).
This involves the insertion of several calls to tabling predicates
defined in mercury_builtin.m and the addition of some scaffolding structure.
Note that this pass can change the evaluation methods of some procedures to
eval_table_io, so it should come before any passes that require definitive
evaluation methods (e.g. inlining).
<p>
The next pass of this stage is a code simplification, namely
removal of lambda expressions (lambda.m):
<ul>
<li>
lambda.m converts lambda expressions into higher-order predicate
terms referring to freshly introduced separate predicates.
This pass needs to come after unique_modes.m to ensure that
the modes we give to the introduced predicates are correct.
It also needs to come after polymorphism.m since polymorphism.m
doesn't handle higher-order predicate constants.
</ul>
(Is there any good reason why lambda.m comes after table_gen.m?)
<p>
The next pass is termination analysis. The various modules involved are:
<ul>
<li>
termination.m is the control module. It sets the argument size and
termination properties of builtin and compiler generated procedures,
invokes term_pass1.m and term_pass2.m
and writes .trans_opt files and error messages as appropriate.
<li>
term_pass1.m analyzes the argument size properties of user-defined procedures,
<li>
term_pass2.m analyzes the termination properties of user-defined procedures.
<li>
term_traversal.m contains code common to the two passes.
<li>
term_errors.m defines the various kinds of termination errors
and prints the messages appropriate for each.
<li>
term_util.m defines the main types used in termination analysis
and contains utility predicates.
<li>
lp.m is used by term_pass1.m. It implements the Linear Programming
algorithm for optimizing a set of linear constraints with respect to
a linear cost function. (XXX Perhaps this should be put in the libs.m
package rather than the check_hlds.m package.)
</ul>
<p>
Most of the remaining HLDS-to-HLDS transformations are optimizations:
<ul>
<li> specialization of higher-order and polymorphic predicates where the
value of the higher-order/type_info/typeclass_info arguments are known
(higher_order.m)
<li> attempt to introduce accumulators (accumulator.m). This optimizes
procedures whose tail consists of independent associative computations
or independant chains of commutative computations into a tail
recursive form by the introduction of accumulators. If lco is turned
on it can also transform some procedures so that only construction
unifications are after the recursive call. This pass must come before
lco, unused_args (eliminating arguments makes it hard to relate the
code back to the assertion) and inlining (can make the associative
call disappear).
<p>
This pass makes use of the goal_store.m module, which is a dictionary-like
data structure for storing HLDS goals.
<li> inlining (i.e. unfolding) of simple procedures (inlining.m)
<li> loop_inv.m: loop invariant hoisting. This transformation moves
computations within loops that are the same on every iteration to the outside
of the loop so that the invariant computations are only computed once. The
transformation turns a single looping predicate containing invariant
computations into two: one that computes the invariants on the first
iteration and then loops by calling the second predicate with extra arguments
for the invariant values. This pass should come after inlining, since
inlining can expose important opportunities for loop invariant hoisting.
Such opportunities might not be visible before inlining because only
*part* of the body of a called procedure is loop-invariant.
<li> deforestation and partial evaluation (deforest.m). This optimizes
multiple traversals of data structures within a conjunction, and
avoids creating intermediate data structures. It also performs
loop unrolling where the clause used is known at compile time.
deforest.m makes use of the following sub-modules (`pd_' stands for
"partial deduction"):
<ul>
<li> constraint.m transforms goals so that goals which can fail are
executed earlier.
<li> pd_cost.m contains some predicates to estimate the improvement
caused by deforest.m.
<li> pd_debug.m produces debugging output.
<li> pd_info.m contains a state type for deforestation.
<li> pd_term.m contains predicates to check that the deforestation
algorithm terminates.
<li> pd_util.m contains various utility predicates.
</ul>
<li> issue warnings about unused arguments from predicates, and create
specialized versions without them (unused_args.m); type_infos are often unused.
<li> delay_construct.m pushes construction unifications to the right in
semidet conjunctions, in an effort to reduce the probability that it will
need to be executed.
<li> unneeded_code.m looks for goals whose results are either not needed
at all, or needed in some branches of computation but not others. Provided
that the goal in question satisfies some requirements (e.g. it is pure,
it cannot fail etc), it either deletes the goal or moves it to the
computation branches where its output is needed.
<li> elimination of dead procedures (dead_proc_elim.m). Inlining, higher-order
specialization and the elimination of unused args can make procedures dead
even the user doesn't, and automatically constructed unification and
comparison predicates are often dead as well.
</ul>
<p>
The module transform.m contains stuff that is supposed to be useful
for high-level optimizations (but which is not yet used).
<p>
The last two HLDS-to-HLDS transformations implement
term size profiling (size_prof.m) and
deep profiling (deep_profiling.m, in the ll_backend.m package).
Both passes insert into procedure bodies, among other things,
calls to procedures (some of which are impure)
that record profiling information.
<hr>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<h3> a. LLDS BACK-END </h3>
<p>
This is the ll_backend.m package.
<h4> 3a. LLDS-specific HLDS -&gt; HLDS transformations </h4>
Before LLDS code generation, there are a few more passes which
annotate the HLDS with information used for LLDS code generation,
or perform LLDS-specific transformations on the HLDS:
<dl>
<dt> choosing registers for procedure arguments (arg_info.m)
<dd>
Currently uses one of two simple algorithms, but
we may add other algorithms later.
<dt> reducing the number of variables that have to be
saved across procedure calls (saved_vars.m)
<dd>
We do this by putting the code that generates
the value of a variable just before the use of
that variable, duplicating the variable and the
code that produces it if necessary, provided
the cost of doing so is smaller than the cost
of saving and restoring the variable would be.
<dt> transforming procedure definitions to reduce the number
of variables that need their own stack slots
(stack_opt.m)
<dd>
The main algorithm in stack_opt.m figures out when
variable A can be reached from a cell pointed to by
variable B, so that storing variable B on the stack
obviates the need to store variable A on the stack
as well.
This algorithm relies on an implementation of
the maximal matching algorithm in matching.m.
<dt> migration of builtins following branched structures
(follow_code.m)
<dd>
This transformation the results of follow_vars.m
(see below)
<dt> simplification again (simplify.m, in the check_hlds.m
package)
<dd>
We run this pass a second time in case the intervening
transformations have created new opportunities for
simplification. It needs to be run immediately
before code generation, because it enforces some
invariants tha the LLDS code generator relies on.
<dt> annotation of goals with liveness information (liveness.m)
<dd>
This records the birth and death of each variable
in the HLDS goal_info.
<dt> allocation of stack slots
<dd>
This is done by stack_alloc.m, with the assistance of
the following modules:
<ul>
<li> live_vars.m works out which variables need
to be saved on the stack when.
<li> graph_colour.m (in the libs.m package)
contains the algorithm that
stack_alloc.m calls to convert sets of variables
that must be saved on the stack at the same time
to an assignment of a stack slot to each such variable.
</ul>
<dt> allocating the follow vars (follow_vars.m)
<dd>
Traverses backwards over the HLDS, annotating some
goals with information about what locations variables
will be needed in next. This allows us to generate
more efficient code by putting variables in the right
spot directly. This module is not called from
mercury_compile.m; it is called from store_alloc.m.
<dt> allocating the store map (store_alloc.m)
<dd>
Annotates each branched goal with variable location
information so that we can generate correct code
by putting variables in the same spot at the end
of each branch.
<dt> computing goal paths (goal_path.m
in the check_hlds.m package)
<dd>
The goal path of a goal defines its position in
the procedure body. This transformation attaches
its goal path to every goal, for use by the debugger.
</dl>
<h4> 4a. Code generation. </h4>
<dl>
<dt> code generation
<dd>
Code generation converts HLDS into LLDS.
For the LLDS back-end, this is also the point at which we
insert code to handle debugging and trailing, and to do
heap reclamation on failure.
The main code generation module is code_gen.m.
It handles conjunctions and negations, but calls sub-modules
to do most of the other work:
<ul>
<li> ite_gen.m (if-then-elses)
<li> call_gen.m (predicate calls and also calls to
out-of-line unification procedures)
<li> disj_gen.m (disjunctions)
<li> par_conj.m (parallel conjunctions)
<li> unify_gen.m (unifications)
<li> switch_gen.m (switches), which has sub-modules
<ul>
<li> dense_switch.m
<li> lookup_switch.m
<li> string_switch.m
<li> tag_switch.m
<li> switch_util.m -- this is in the backend_libs.m
package, since it is also used by MLDS back-end
</ul>
<li> commit_gen.m (commits)
<li> pragma_c_gen.m (embedded C code)
</ul>
<p>
code_gen.m also calls middle_rec.m to do middle recursion optimization,
which is implemented during code generation.
<p>
The code generation modules make use of
<dl>
<dt> code_info.m
<dd>
The main data structure for the code generator.
<dt> var_locn.m
<dd>
This defines the var_locn type, which is a
sub-component of the code_info data structure;
it keeps track of the values and locations of variables.
It implements eager code generation.
<dt> exprn_aux.m
<dd>
Various utility predicates.
<dt> code_util.m
<dd>
Some miscellaneous preds used for code generation.
<dt> code_aux.m
<dd>
Some miscellaneous preds which, unlike those in
code_util, use code_info.
<dt> continuation_info.m
<dd>
For accurate garbage collection, collects
information about each live value after calls,
and saves information about procedures.
<dt> trace.m
<dd>
Inserts calls to the runtime debugger.
<dt> trace_params.m (in the libs.m package, since it
is considered part of option handling)
<dd>
Holds the parameter settings controlling
the handling of execution tracing.
</dl>
<dt> code generation for `pragma export' declarations (export.m)
<dd> This is handled seperately from the other parts of code generation.
mercury_compile.m calls the procedures `export__produce_header_file'
and `export__get_pragma_exported_procs' to produce C code fragments
which declare/define the C functions which are the interface stubs
for procedures exported to C.
<dt> generation of constants for RTTI data structures
<dd> This could also be considered a part of code generation,
but for the LLDS back-end this is currently done as part
of the output phase (see below).
</dl>
<p>
The result of code generation is the Low Level Data Structure (llds.m),
which may also contains some data structures whose types are defined in rtti.m.
The code for each procedure is generated as a tree of code fragments
which is then flattened (tree.m).
<h4> 5a. Low-level optimization (LLDS). </h4>
<p>
The various LLDS-to-LLDS optimizations are invoked from optimize.m.
They are:
<ul>
<li> optimization of jumps to jumps (jumpopt.m)
<li> elimination of duplicate code sequences (dupelim.m)
<li> optimization of stack frame allocation/deallocation (frameopt.m)
<li> filling branch delay slots (delay_slot.m)
<li> dead code and dead label removal (labelopt.m)
<li> peephole optimization (peephole.m)
<li> introduction of local C variables (use_local_vars.m) <br>
<li> removal of redundant assignments, i.e. assignments that assign a value
that the target location already holds (reassign.m) <br>
</ul>
<p>
The module opt_debug.m contains utility routines used for debugging
these LLDS-to-LLDS optimizations.
<p>
Several of these optimizations (frameopt and use_local_vars) also
use livemap.m, a module that finds the set of locations live at each label.
<p>
Use_local_vars numbering also introduces
references to temporary variables in extended basic blocks
in the LLDS representation of the C code.
The transformation to insert the block scopes
and declare the temporary variables is performed by wrap_blocks.m.
<p>
Depending on which optimization flags are enabled,
optimize.m may invoke many of these passes multiple times.
<p>
Some of the low-level optimization passes use basic_block.m,
which defines predicates for converting sequences of instructions to
basic block format and back, as well as opt_util.m, which contains
miscellaneous predicates for LLDS-to-LLDS optimization.
<h4> 6a. Output C code </h4>
<ul>
<li> type_ctor_info.m
(in the backend_libs.m package, since it is shared with the MLDS back-end)
generates the type_ctor_gen_info structures that list
items of information (including unification, index and compare predicates)
associated with each declared type constructor that go into the static
type_ctor_info data structure. If the type_ctor_gen_info structure is not
eliminated as inaccessible, this module adds the corresponding type_ctor_info
structure to the RTTI data structures defined in rtti.m,
which are part of the LLDS.
<li> base_typeclass_info.m
(in the backend_libs.m package, since it is shared with the MLDS back-end)
generates the base_typeclass_info structures that
list the methods of a class for each instance declaration. These are added to
the RTTI data structures, which are part of the LLDS.
<li> stack_layout.m generates the stack_layout structures for
accurate garbage collection. Tables are created from the data
collected in continuation_info.m.
Stack_layout.m uses prog_rep.m and static_term.m to generate representations
of procedure bodies for use by the declarative debugger.
<li> Type_ctor_info structures and stack_layout structures both contain
pseudo_type_infos, which are type_infos with holes for type variables;
these are generated by pseudo_type_info.m
(in the backend_libs.m package, since it is shared with the MLDS back-end).
<li> llds_common.m extracts static terms from the main body of the LLDS, and
puts them at the front. If a static term originally appeared several times,
it will now appear as a single static term with multiple references to it.
[XXX FIXME this module has now been replaced by global_data.m]
<li> transform_llds.m is responsible for doing any source to source
transformations on the llds which are required to make the C output
acceptable to various C compilers. Currently computed gotos can have
their maximum size limited to avoid a fixed limit in lcc.
<li> Final generation of C code is done in llds_out.m, which subcontracts the
output of RTTI structures to rtti_out.m and of other static
compiler-generated data structures (such as those used by the debugger,
the deep profiler, and in the future by the garbage collector)
to layout_out.m.
</ul>
<hr>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<h3> b. MLDS BACK-END </h3>
<p>
This is the ml_backend.m package.
<p>
The original LLDS code generator generates very low-level code,
since the LLDS was designed to map easily to RISC architectures.
We have developed a new back-end that generates much higher-level
code, suitable for generating Java, high-level C, etc.
This back-end uses the Medium Level Data Structure (mlds.m) as its
intermediate representation.
<h4> 3b. pre-passes to annotate/transform the HLDS </h4>
<p>
Before code generation there is a pass which annotates the HLDS with
information used for code generation:
<ul>
<li> mark_static_terms.m marks construction unifications
which can be implemented using static constants rather
than heap allocation.
</ul>
<p>
For the MLDS back-end, we've tried to keep the code generator simple.
So we prefer to do things as HLDS to HLDS transformations where possible,
rather than complicating the HLDS to MLDS code generator.
Thus we have a pass which transforms the HLDS to handle trailing:
<ul>
<li> add_trail_ops.m inserts code to manipulate the trail,
in particular ensuring that we apply the appropriate
trail operations before each choice point, when execution
resumes after backtracking, and whenever we do a commit.
The trail operations are represented as (and implemented as)
calls to impure procedures defined in library/private_builtin.m.
<li> add_heap_ops.m is very similar to add_trail_ops.m;
it inserts code to do heap reclamation on backtracking.
</ul>
<h4> 4b. MLDS code generation </h4>
<ul>
<li> ml_code_gen.m converts HLDS code to MLDS.
The following sub-modules are used to handle different constructs:
<ul>
<li> ml_unify_gen.m
<li> ml_closure_gen.m
<li> ml_call_gen.m
<li> ml_switch_gen.m, which in turn has sub-modules
<ul>
<li> ml_dense_switch.m
<li> ml_string_switch.m
<li> ml_tag_switch.m
<li> switch_util.m (in the backend_libs.m package,
since it is also used by LLDS back-end)
</ul>
</ul>
The module ml_code_util.m provides utility routines for
MLDS code generation. The module ml_util.m provides some
general utility routines for the MLDS.
<li> ml_type_gen.m converts HLDS types to MLDS.
<li> type_ctor_info.m and base_typeclass_info.m generate
the RTTI data structures defined in rtti.m and pseudo_type_info.m
(those four modules are in the backend_libs.m package, since they
are shared with the LLDS back-end)
and then mlds_to_rtti.m converts these to MLDS.
</ul>
<h4> 5b. MLDS transformations </h4>
<ul>
<li> ml_tailcall.m annotates the MLDS with information about tailcalls.
It also has a pass to implement the `--warn-non-tail-recursion' option.
<li> ml_optimize.m does MLDS-&gt;MLDS optimizations
<li> ml_elim_nested.m does two MLDS transformations that happen
to have a lot in common: (1) eliminating nested functions
and (2) adding code to handle accurate garbage collection.
</ul>
<h4> 6b. MLDS output </h4>
<p>
There are currently four backends that generate code from MLDS:
one generates C/C++ code,
one generates assembler (by interfacing with the GCC back-end),
one generates Microsoft's Intermediate Language (MSIL or IL),
and one generates Java.
<ul>
<li>mlds_to_c.m converts MLDS to C/C++ code.
</ul>
<p>
The MLDS-&gt;asm backend is logically part of the MLDS back-ends,
but it is in a module of its own (mlds_to_gcc.m), rather than being
part of the ml_backend package, so that we can distribute a version
of the Mercury compiler which does not include it. There is a wrapper
module called maybe_mlds_to_gcc.m which is generated at configuration time
so that mlds_to_gcc.m will be linked in iff the GCC back-end is available.
<p>
The MLDS-&gt;IL backend is broken into several submodules.
<ul>
<li> mlds_to_ilasm.m converts MLDS to IL assembler and writes it to a .il file.
<li> mlds_to_il.m converts MLDS to IL
<li> ilds.m contains representations of IL
<li> ilasm.m contains output routines for writing IL to assembler.
<li> il_peephole.m performs peephole optimization on IL instructions.
</ul>
After IL assembler has been emitted, ILASM in invoked to turn the .il
file into a .dll or .exe.
<p>
The MLDS-&gt;Java backend is broken into two submodules.
<ul>
<li> mlds_to_java.m converts MLDS to Java and writes it to a .java file.
<li> java_util.m contains some utility routines.
</ul>
After the Java code has been emitted, a Java compiler (normally javac)
is invoked to turn the .java file into a .class file containing Java bytecodes.
<hr>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<h3> c. Aditi-RL BACK-END </h3>
<p>
This is the aditi_backend.m package.
<h4> 3c. Aditi-specific HLDS -&gt; HLDS transformations</h4>
The Aditi back-end first performs some HLDS-to-HLDS transformations
that are specific to the Aditi back-end:
<ul>
<li> conversion of Aditi procedures into disjunctive normal form (dnf.m).
The supplementary magic sets and context transformations are only defined
for predicates in DNF.
<li> transformation of Aditi builtins into normal calls to predicates
in extras/aditi/aditi_private_builtin.m (aditi_builtin_ops.m).
<li> supplementary magic sets or supplementary context transformation of
Aditi procedures (magic.m, magic_util.m, context.m).
The magic sets or context transformations must be applied to convert the
program to a form for which Aditi-RL bytecode can be generated.
</ul>
<h4> 4c. Aditi-RL generation </h4>
<ul>
<li> rl_gen.m converts HLDS to RL.
<li> rl_relops.m generates RL for relational operations such as joins.
<li> rl_info.m defines a state type.
<li> rl.m defines the representation of Aditi-RL used within the
Mercury compiler. There are some slight differences between rl.m
and Aditi-RL to make optimization easier.
<li> rl_dump.m contains predicates to write the types defined in rl.m.
</ul>
<h4> 5c. Aditi-RL optimization </h4>
<ul>
<li> rl_opt.m invokes the RL optimization passes.
<li> rl_block.m converts an RL procedure into basic blocks, and performs
other tasks such as detecting the loops in those basic blocks.
<li> rl_analyse.m contains a generic data-flow analysis procedure for
RL procedures.
<li> rl_liveness.m uses rl_analyse.m to insert code to initialise relations
and clear references to them when they are no longer needed.
<li> rl_loop.m moves invariants out of loops.
<li> rl_block_opt.m performs common subexpression elimination and instruction
merging on basic blocks.
<li> rl_key.m computes upper and lower bounds on the non-locals of a goal
for which the goal could succeed, which is useful for determining when
an index could be used for a relational operation.
<li> rl_sort.m introduces sort-merge and indexed versions of relational
operations where possible, and optimizes away unnecessary sorting and
indexing.
<li> rl_stream.m detects relations which do not need to be materialised.
</ul>
<h4> 6c. Output Aditi-RL code </h4>
<ul>
<li> rl_out.m converts from the instructions defined in rl.m
to bytecode either as data in the &lt;module&gt;.c file or
to &lt;module&gt;.rlo and outputs a text representation to
&lt;module&gt;.rla.
<li> rl_exprn.m is called from rl_out.m to convert top down Mercury
code (in HLDS form) to Aditi bytecode.
<li> rl_code.m contains the definition of the bytecodes interpreted
by Aditi. This file is automatically generated from the Aditi sources.
<li> rl_file.m contains routines to output the bytecodes defined in rl_code.m.
</ul>
<hr>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<h3> d. BYTECODE BACK-END </h3>
<p>
This is the bytecode_backend.m package.
<p>
The Mercury compiler can translate Mercury programs into bytecode for
interpretation by a bytecode interpreter. The intent of this is to
achieve faster turn-around time during development. However, the
bytecode interpreter has not yet been written.
<ul>
<li> bytecode.m defines the internal representation of bytecodes, and contains
the predicates to emit them in two forms. The raw bytecode form is emitted
into &lt;filename&gt;.bytecode for interpretation, while a human-readable
form is emitted into &lt;filename&gt;.bytedebug for visual inspection.
<li> bytecode_gen.m contains the predicates that translate HLDS into bytecode.
<li> bytecode_data.m contains the predicates that translate ints, strings
and floats into bytecode. This is also used by rl_code.m.
</ul>
<hr>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<h3> SMART RECOMPILATION </h3>
<p>
This is the recompilation.m package.
<p>
The Mercury compiler can record program dependency information
to avoid unnecessary recompilations when an imported module's
interface changes in a way which does not invalidate previously
compiled code.
<ul>
<li> recompilation.m contains types used by the other smart
recompilation modules.
<li> recompilation_version.m generates version numbers for program items
in interface files.
<li> recompilation_usage.m works out which program items were used
during a compilation.
<li> recompilation_check.m is called before recompiling a module.
It uses the information written by recompilation_version.m and
recompilation_usage.m to work out whether the recompilation is
actually needed.
</ul>
<hr>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<h3> MISCELLANEOUS </h3>
The modules special_pred.m (in the hlds.m package) and unify_proc.m
(in the check_hlds.m package) contain stuff for handling the special
compiler-generated predicates which are generated for
each type: unify/2, compare/3, and index/1 (used in the
implementation of compare/3).
<p>
This module is part of the transform_hlds.m package.
<dl>
<dt> dependency_graph.m:
<dd>
This contains predicates to compute the call graph for a
module, and to print it out to a file.
(The call graph file is used by the profiler.)
The call graph may eventually also be used by det_analysis.m,
inlining.m, and other parts of the compiler which could benefit
from traversing the predicates in a module in a bottom-up or
top-down fashion with respect to the call graph.
</dl>
<p>
The following modules are part of the backend_libs.m package.
<dl>
<dt> builtin_ops:
<dd>
This module defines the types unary_op and binary_op
which are used by several of the different back-ends:
bytecode.m, llds.m, and mlds.m.
<dt> c_util:
<dd>
This module defines utility routines useful for generating
C code. It is used by both llds_out.m and mlds_to_c.m.
<dt> compile_target_code.m
<dd>
Invoke C, C#, IL, Java, etc. compilers and linkers to compile
and link the generated code.
<dt> det_util:
<dd>
This module contains utility predicates needed by the parts
of the semantic analyzer and optimizer concerned with
determinism.
</dl>
<p>
The following modules are part of the libs.m package.
<dl>
<dt> process_util.m:
<dd>
Predicates to deal with process creation and signal handling.
This module is mainly used by make.m and its sub-modules.
<dt> timestamp.m
<dd>
Contains an ADT representing timestamps used by smart
recompilation and `mmc --make'.
<dt> graph_color.m
<dd>
Graph colouring. <br>
This is used by the LLDS back-end for register allocation
<dt> tree.m
A simple tree data type. <br>
Used by the LLDS, RL, and IL back-ends for collecting
together the different fragments of the generated code.
<dd> Graph colouring.
</dl>
<hr>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<h3> CURRENTLY UNDOCUMENTED </h3>
<ul>
<li> mmc_analysis.m
</ul>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<h3> CURRENTLY USELESS </h3>
<dl>
<dt> atsort.m (in the libs.m package)
<dd>
Approximate topological sort.
This was once used for traversing the call graph,
but nowadays we use relation__atsort from library/relation.m.
<dt> lco.m (in the transform_hlds.m package):
<dd>
This finds predicates whose implementations would benefit
from last call optimization modulo constructor application.
It does not apply the optimization and will not until the
mode system is capable of expressing definite aliasing.
</dl>
<hr>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
Last update was $Date: 2003-10-20 07:29:18 $ by $Author: zs $@cs.mu.oz.au. <br>
</body>
</html>