Discussion of these changes can be found on the Mercury developers
mailing list archives from June 2018.
COPYING.LIB:
Add a special linking exception to the LGPL.
*:
Update references to COPYING.LIB.
Clean up some minor errors that have accumulated in copyright
messages.
Branches: main
Pack consecutive enumeration arguments in discriminated union types into a
single word to reduce cell sizes. Argument packing is only enabled on C
back-ends with low-level data, and reordering arguments to improve
opportunities for packing is not yet attempted. The RTTI implementations for
other back-ends will need to be updated, but that is best left until after any
argument reordering change.
Modules which import abstract enumeration types are notified so by writing
declarations of the form:
:- type foo where type_is_abstract_enum(NumBits).
into the interface file for the module which defines the type.
compiler/prog_data.m:
Add an `arg_width' argument to constructor arguments.
Replace `is_solver_type' by `abstract_type_details', with an extra
option for abstract exported enumeration types.
compiler/handle_options.m:
compiler/options.m:
Add an internal option `--allow-argument-packing'.
compiler/make_hlds_passes.m:
Determine whether and how to pack enumeration arguments, updating the
`arg_width' fields of constructor arguments before constructors are
added to the HLDS.
compiler/mercury_to_mercury.m:
compiler/modules.m:
Write `where type_is_abstract_enum(NumBits)' to interface files
for abstract exported enumeration types.
compiler/prog_io_type_defn.m:
Parse `where type_is_abstract_enum(NumBits)' attributes on type
definitions.
compiler/arg_pack.m:
compiler/backend_libs.m:
Add a new module. This mainly contains a predicate which packs rvals
according to arg_widths, which is used by both LLDS and MLDS back-ends.
compiler/ml_unify_gen.m:
compiler/unify_gen.m:
Take argument packing into account when generating code for
constructions and deconstructions. Only a relatively small part of the
compiler actually needs to understand argument packing. The rest works
at the HLDS level with constructor arguments and variables, or at the
LLDS and MLDS levels with structure fields.
compiler/code_info.m:
compiler/var_locn.m:
Add assign_field_lval_expr_to_var and
var_locn_assign_field_lval_expr_to_var.
Allow more kinds of rvals in assign_cell_arg. I do not know why it was
previously restricted, except that the other kinds of rvals were not
encountered as cell arguments before.
compiler/mlds.m:
We can now rely on the compiler to pack arguments in the
mlds_decl_flags type instead of doing it manually. A slight downside
is that though the type is packed down to a single word cell, it will
still incur a memory allocation per cell. However, I did not notice
any difference in compiler speed.
compiler/rtti.m:
compiler/rtti_out.m:
Add and output a new field for MR_DuFunctorDesc instances, which, if
any arguments are packed, points to an array of MR_DuArgLocn. Each
array element describes the offset in the cell at which the argument's
value is held, and which bits of the word it occupies. In the more
common case where no arguments are packed, the new field is simply
null.
compiler/rtti_to_mlds.m:
Generate the new field to MR_DuFunctorDesc.
compiler/structure_reuse.direct.choose_reuse.m:
For now, prevent structure reuse reusing a dead cell which has a
different constructor to the new cell. The code to determine whether a
dead cell will hold the arguments of a new cell with a different
constructor will need to be updated to account for argument packing.
compiler/type_ctor_info.m:
Bump RTTI version number.
Conform to changes.
compiler/add_type.m:
compiler/check_typeclass.m:
compiler/equiv_type.m:
compiler/equiv_type_hlds.m:
compiler/erl_rtti.m:
compiler/hlds_data.m:
compiler/hlds_out_module.m:
compiler/intermod.m:
compiler/make_tags.m:
compiler/mlds_to_gcc.m:
compiler/opt_debug.m:
compiler/prog_type.m:
compiler/recompilation.check.m:
compiler/recompilation.version.m:
compiler/special_pred.m:
compiler/type_constraints.m:
compiler/type_util.m:
compiler/unify_proc.m:
compiler/xml_documentation.m:
Conform to changes.
Reduce code duplication in classify_type_defn.
compiler/hlds_goal.m:
Clarify a comment.
library/construct.m:
Make `construct' pack arguments when necessary.
Remove an old RTTI version number check as recommended in
mercury_grade.h.
library/store.m:
Deal with packed arguments in this module.
runtime/mercury_grade.h:
Bump binary compatibility version number.
runtime/mercury_type_info.c:
runtime/mercury_type_info.h:
Bump RTTI version number.
Add MR_DuArgLocn structure definition.
Add a macro to unpack an argument as described by MR_DuArgLocn.
Add a function to determine a cell's size, since the number of
arguments is no longer correct.
runtime/mercury_deconstruct.c:
runtime/mercury_deconstruct.h:
runtime/mercury_deconstruct_macros.h:
runtime/mercury_ml_arg_body.h:
runtime/mercury_ml_expand_body.h:
Deal with packed arguments when deconstructing.
Remove an old RTTI version number check as recommended in
mercury_grade.h.
runtime/mercury_deep_copy_body.h:
Deal with packed arguments when copying.
runtime/mercury_table_type_body.h:
Deal with packed arguments in tabling.
runtime/mercury_dotnet.cs.in:
Add DuArgLocn field to DuFunctorDesc. Argument packing is not enabled
for the C# back-end yet so this is unused.
trace/mercury_trace_vars.c:
Deal with packed arguments in MR_select_specified_subterm,
use for the `hold' command.
java/runtime/DuArgLocn.java:
java/runtime/DuFunctorDesc.java:
Add DuArgLocn field to DuFunctorDesc. Argument packing is not enabled
for the Java back-end yet so this is unused.
extras/trailed_update/tr_store.m:
Deal with packed arguments in this module (untested).
extras/trailed_update/samples/interpreter.m:
extras/trailed_update/tr_array.m:
Conform to argument reordering in the array, map and other modules in
previous changes.
tests/hard_coded/Mercury.options:
tests/hard_coded/Mmakefile:
tests/hard_coded/lco_pack_args.exp:
tests/hard_coded/lco_pack_args.m:
tests/hard_coded/pack_args.exp:
tests/hard_coded/pack_args.m:
tests/hard_coded/pack_args_copy.exp:
tests/hard_coded/pack_args_copy.m:
tests/hard_coded/pack_args_intermod1.exp:
tests/hard_coded/pack_args_intermod1.m:
tests/hard_coded/pack_args_intermod2.m:
tests/hard_coded/pack_args_reuse.exp:
tests/hard_coded/pack_args_reuse.m:
tests/hard_coded/store_ref.exp:
tests/hard_coded/store_ref.m:
tests/invalid/Mmakefile:
tests/invalid/where_abstract_enum.err_exp:
tests/invalid/where_abstract_enum.m:
tests/tabling/Mmakefile:
tests/tabling/pack_args_memo.exp:
tests/tabling/pack_args_memo.m:
Add new test cases.
tests/hard_coded/deconstruct_arg.exp:
tests/hard_coded/deconstruct_arg.exp2:
tests/hard_coded/deconstruct_arg.m:
Add constructors with packed arguments to these cases.
tests/invalid/where_direct_arg.err_exp:
Update expected output.
Branches: main
Implement a new form of memory profiling, which tells the user what memory
is being retained during a program run. This is done by allocating an extra
word before each cell, which is used to "attribute" the cell to an
allocation site. The attribution, or "allocation id", is an address to an
MR_AllocSiteInfo structure generated by the Mercury compiler, giving the
procedure, filename and line number of the allocation, and the type
constructor and arity of the cell that it allocates.
The user must manually instrument the program with calls to
`benchmarking.report_memory_attribution', which forces a GC and summarises
the live objects on the heap using the attributions. The mprof tool is
extended with a new mode to parse and present that data.
Objects which are unattributed (e.g. by hand-written C code which hasn't
been updated) are still accounted for, but show up in profiles as "unknown".
Currently this profiling mode only works in conjunction with the Boehm
garbage collector, though in principle it can work with any memory allocator
for which we can access a list of the live objects. Since term size
profiling relies on the same technique of using an extra word per memory
cell, the two profiling modes are incompatible.
The output from `mprof -s' looks like this:
------ [1] some label ------
cells words cumul procedure / type (location)
14150 38872 total
* 1949/ 13.8% 4872/ 12.5% 12.5% <predicate `parser.parse_rest/7' mode 0>
975/ 6.9% 1950/ 5.0% list.list/1 (parser.m:502)
487/ 3.4% 1948/ 5.0% term.term/1 (parser.m:501)
487/ 3.4% 974/ 2.5% term.const/0 (parser.m:501)
* 1424/ 10.1% 4272/ 11.0% 23.5% <predicate `parser.parse_simple_term_2/6' mode 0>
708/ 5.0% 2832/ 7.3% term.term/1 (parser.m:643)
708/ 5.0% 1416/ 3.6% term.const/0 (parser.m:643)
...
boehm_gc/alloc.c:
boehm_gc/include/gc.h:
boehm_gc/misc.c:
boehm_gc/reclaim.c:
Add a callback function to be called for every live object after a GC.
Add a function to write out the GC_size_map array.
compiler/layout.m:
Define the alloc_site_info type which is equivalent to the
MR_AllocSiteInfo C structure.
Add alloc_site_array as a kind of "layout" array.
compiler/llds.m:
Add allocation sites to `cfile' structure.
Replace TypeMsg argument (which was also for profiling) on `incr_hp'
instructions by an allocation site identifier.
Add a new foreign_proc_component for allocation site ids.
compiler/code_info.m:
compiler/global_data.m:
compiler/proc_gen.m:
Keep the set of allocation sites in the code_info and global_data
structures.
compiler/unify_gen.m:
Add allocation sites to LLDS allocation instructions.
compiler/layout_out.m:
compiler/llds_out_file.m:
compiler/llds_out_instr.m:
Output MR_AllocSiteInfo arrays in generated C files.
Output code to register the MR_AllocSiteInfo array with the Mercury
runtime.
Output allocation site ids for memory allocation instructions.
compiler/llds_out_util.m:
Add allocation sites to llds_out_info.
compiler/pragma_c_gen.m:
compiler/ml_foreign_proc_gen.m:
Generate a macro MR_ALLOC_ID which resolves to an allocation site
structure, for every foreign_proc whose C code contains the string
"MR_ALLOC_ID". This is to be used by hand-written C code which
allocates memory.
MR_PROC_LABELs are retained for backwards compatibility. Though
they were introduced for profiling, they seem to have been co-opted
for printf-debugging since then.
compiler/ml_global_data.m:
Add allocation site structures to the MLDS global data.
compiler/mlds.m:
compiler/ml_unify_gen.m:
Add allocation site id to `new_object' instruction.
compiler/mlds_to_c.m:
Output allocation site arrays and allocation ids in high-level C code.
Output a call to register the allocation site array with the Mercury
runtime.
Delete an unused predicate.
compiler/exprn_aux.m:
compiler/jumpopt.m:
compiler/livemap.m:
compiler/mercury_compile_llds_back_end.m:
compiler/middle_rec.m:
compiler/ml_accurate_gc.m:
compiler/ml_elim_nested.m:
compiler/ml_optimize.m:
compiler/ml_util.m:
compiler/mlds_to_cs.m:
compiler/mlds_to_gcc.m:
compiler/mlds_to_il.m:
compiler/mlds_to_java.m:
compiler/mlds_to_managed.m:
compiler/opt_debug.m:
compiler/opt_util.m:
compiler/use_local_vars.m:
compiler/var_locn.m:
Conform to changes.
compiler/pickle.m:
compiler/prog_event.m:
compiler/timestamp.m:
Conform to changes in memory allocation macros.
library/benchmarking.m:
Add the `report_memory_attribution' instrumentation predicates.
Conform to changes to MR_memprof_record.
library/array.m:
library/bit_buffer.m:
library/bitmap.m:
library/construct.m:
library/deconstruct.m:
library/dir.m:
library/io.m:
library/mutvar.m:
library/store.m:
library/string.m:
library/thread.semaphore.m:
library/version_array.m:
Use attributed memory allocation throughout the standard library so
that objects don't show up in the memory profile as "unknown".
Replace MR_PROC_LABEL by MR_ALLOC_ID.
mdbcomp/program_representation.m:
mdbcomp/rtti_access.m:
Replace MR_PROC_LABEL by MR_ALLOC_ID.
profiler/Mercury.options:
profiler/globals.m:
profiler/mercury_profile.m:
profiler/options.m:
profiler/output.m:
profiler/snapshots.m:
Add a new mode to `mprof' to parse and present the data from
`Prof.Snapshots' files.
Add options for the new profiling mode.
profiler/process_file.m:
Fix a typo.
runtime/mercury_conf_param.h:
#define MR_MPROF_PROFILE_MEMORY_ATTRIBUTION if memory profiling
is enabled and we are using Boehm GC.
runtime/mercury.h:
Make MR_new_object take an allocation id argument.
Conform to changes in memory allocation macros.
runtime/mercury_memory.c:
runtime/mercury_memory.h:
runtime/mercury_types.h:
Define MR_AllocSiteInfo.
Add memory allocation functions and macros which take into the
account the additional word necessary for the new profiling mode.
These should be used in preferences to the raw memory allocation
functions wherever possible so that objects do not show up in the
profile as "unknown".
Add analogues of realloc/free which take into account the offset
introduced by the attribution word.
Add function versions of the MR_new_object macros, which can't be
written in standard C. They are only used when necessary.
Add built-in allocation site ids, to be used in the runtime and
other hand-written code when context-specific ids are unavailable.
runtime/mercury_heap.h:
Make MR_tag_offset_incr_hp_msg and MR_tag_offset_incr_hp_atomic_msg
allocate an extra word when memory attribution is desired, and store
the allocation id there.
Similarly for MR_create{1,2,3}_msg.
Replace proclabel arguments in allocation macros by alloc_id
arguments.
Replace MR_hp_alloc_atomic by MR_hp_alloc_atomic_msg. It was only
used for boxing floats.
Conform to change to MR_new_object macro.
runtime/mercury_bootstrap.h:
Delete obsolete macro hp_alloc_atomic.
runtime/mercury_heap_profile.c:
runtime/mercury_heap_profile.h:
Add the code to summarise the live objects on the Boehm GC heap and
writes out the data to `Prof.Snapshots', for display by mprof.
Don't store the procedure name in MR_memprof_record: the procedure
address is enough and faster to compare.
runtime/mercury_prof.c:
Finish and close the `Prof.Snapshots' file when the program
terminates.
Conform to changes in MR_memprof_record.
runtime/mercury_misc.h:
Add a macro to expand to the name of the allocation sites array
in LLDS grades.
runtime/mercury_bitmap.c:
runtime/mercury_bitmap.h:
Pass allocation id through bitmap allocation functions.
Delete unused function MR_string_to_bitmap.
runtime/mercury_string.h:
Add MR_make_aligned_string_copy_msg.
Make string allocation macros take allocation id arguments.
runtime/mercury.c:
runtime/mercury_array_macros.h:
runtime/mercury_context.c:
runtime/mercury_deconstruct.c:
runtime/mercury_deconstruct_macros.h:
runtime/mercury_dlist.c:
runtime/mercury_engine.c:
runtime/mercury_float.h:
runtime/mercury_hash_table.c:
runtime/mercury_ho_call.c:
runtime/mercury_label.c:
runtime/mercury_prof_mem.c:
runtime/mercury_stacks.c:
runtime/mercury_stm.c:
runtime/mercury_string.c:
runtime/mercury_thread.c:
runtime/mercury_trace_base.c:
runtime/mercury_trail.c:
runtime/mercury_type_desc.c:
runtime/mercury_type_info.c:
runtime/mercury_wsdeque.c:
Use attributed memory allocation throughout the runtime so that
objects don't show up in the profile as "unknown".
runtime/mercury_memory_zones.c:
Attribute memory zones to the Mercury runtime.
runtime/mercury_tabling.c:
runtime/mercury_tabling.h:
Use attributed memory allocation macros for tabling structures.
Delete unused MR_table_realloc_* and MR_table_copy_bytes macros.
runtime/mercury_deep_copy_body.h:
Try to retain the original attribution word when copying values.
runtime/mercury_ml_expand_body.h:
Conform to changes in memory allocation macros.
runtime/mercury_tags.h:
Replace proclabel arguments by alloc_id arguments in allocation macros.
runtime/mercury_wrapper.c:
If memory attribution is enabled, tell Boehm GC that pointers may be
displaced by an extra word.
trace/mercury_trace.c:
trace/mercury_trace_tables.c:
Conform to changes in memory allocation macros.
extras/net/tcp.m:
extras/solver_types/library/any_array.m:
extras/trailed_update/tr_array.m:
Conform to changes in memory allocation macros.
doc/user_guide.texi:
Document the new profiling mode.
doc/reference_manual.texi:
Update a commented out example.
Branches: main, 11.01
Merge changes from G12's copy of the tr_array module into the one in the
trailed_update package in the extras distribution.
extras/trailed_update/tr_array.m:
As above, the main changes are:
- some problems that caused C compiler warnings are fixed
- argument orderings for a few predicates have been altered
in order to work better with state variables.
- will_not_modify_trail annotations have been added to
foreign_procs where appropriate.
Branches: main
Fix bitrot in extras/trailed_update.
extras/trailed_updated/var.m:
Don't specify an initialisation predicated for the solver type var/1.
extras/trailed_updated/samples/interpreter.m:
Conform with the change that added an extra argument to the variable/2
constructor of the standard library type term/1.
extras/trailed_update/tests/var_test.m:
Use up-to-date syntax for modes.
extras/trailed_update/Mmakefile:
extras/trailed_update/samples/Mmakefile:
extras/trailed_update/tests/var_test.m:
Use trail segments since that's what you get in an installation by
default.
extras/trailed_update/*.m:
Update syntax and formatting to conform more closely to our
current coding standards.
Add require_feature_set pragmas for trailing.
Estimated hours taken: 0
Branches: main
Fix compilation of extras/trailed_update/tr_array.m.
extras/trailed_update/tr_array.m:
Comment out duplicate modes.
Estimated hours taken: 1
Branches: main
Use `pragma foreign_export' in preference to `pragma export' throughout
the Mercury distribution.
Convert more of the extras distribution to four-space indentation and
other minor cleanups.
browser/*.m:
library/*.m:
samples/*:
extras/*:
As above.
Estimated hours taken: 0.5
Branches: main
Fix compilation problems in the extras distribution caused by recent
changes.
extras/stream/stream.m:
Provide a definition for the type stream/1.
extras/*/*.m:
Conform to the recent changes to the standard library.
Estimated hours taken: 2.5
Branches: main, release
extras/trailed_update/tr_array.m:
extras/trailed_update/tr_store.m:
extras/trailed_update/var.m:
extras/trailed_update/unsafe.m:
extras/trailed_update/samples/vqueens.m:
Use more recent syntax and make these modules
conform more closely to our coding standard.
Use the new foreign language interface.
Replace deprecated mode and inst syntax.
extras/trailed_update/samples/interpreter.m:
extras/trailed_update/samples/interpreter.exp:
s/mutvar/generic_mutvar/
Estimated hours taken: 8
Branches: main
library/array.m:
extras/trailed_update/tr_array.m:
runtime/mercury_deep_copy_body.h:
runtime/mercury_library_types.h:
Allocate arrays on the Mercury heap, using MR_incr_hp_msg(),
rather than using MR_GC_malloc(). This is needed for accurate
GC, to ensure that the objects pointed to by the array elements
will get traced by the collector.
Estimated hours taken: 16
Implement trailing for the MLDS back-end.
library/private_builtin.m:
Define impure Mercury procedures corresponding to the
trailing operations in runtime/mercury_trail.h.
compiler/add_trail_ops.m:
A new HLDS->HLDS pass.
This implements trailing by inserting calls to the
trailing primitives declared in library/private_builtin.m.
compiler/notes/compiler_design.html:
Document the new pass.
compiler/mercury_compile.m:
Change the code for mercury_compile__mlds_backend so that
if trailing is enabled, it invokes the new add_trail_ops pass.
runtime/mercury_wrapper.c:
For high-level C grades, call MR_init_memory() (which sets up
the redzone signal handlers, etc.) and if trailing is enabled,
initialize the trail.
library/benchmarking.m:
Change the code in report_stats for printing out the size of
the trail so that it gets the trail zone from MR_trail_zone
rather than via the MercuryEngine, so that it works in
high-level C grades (which don't have any MercuryEngine).
extras/trailed_update/tr_array.m:
extras/trailed_update/var.m:
Add missing `#includes' to make these files work with the
MLDS back-end.
Estimated hours taken: 0.25
extras/trailed_update/tr_array.m:
Update to reflect the new names of the memory allocation
macros: s/make_many/MR_GC_NEW_ARRAY/
Estimated hours taken: 6
Various bug fixes and enhancements to var.m and other stuff in
extras/trailed_update.
extras/trailed_update/var.m:
Add code to detect floundering.
Change the code for debug_freeze/3 to avoid a floundering warning.
Add new predicates debug_freeze/4 (to go with debug_freeze/3)
and dump_var/3.
Fix a bug in freeze/3 (higher-order terms are monomorphic, so
there is no need to explicitly pass type-infos when calling them).
Fix a bug in wakeup_delayed_goals/2 (we need to set the binding
to ground _before_ waking up the goals).
extras/trailed_update/Mmakefile:
Add `ML = ml --mercury-libs shared', so that it will work on Linux.
extras/trailed_update/samples/Mmakefile:
Make it work with shared libraries on Linux.
extras/trailed_update/samples/tests.m:
extras/trailed_update/samples/tests.exp:
Some test cases for var.m.
extras/trailed_update/tr_store.m:
Add a prototype for ML_compare_typeinfo().
extras/trailed_update/unsafe.m:
Change the C code for unsafe_perform_io to match the new
interface for exported Mercury procedures that do I/O --
there is no longer any need to pass dummy io__state arguments
in the C code.
Estimated hours taken: 1
extras/trailed_update/tr_store.m:
New file. A backtrackable version of library/store.m.
extras/trailed_update/tr_array.m:
Delete the `mui_' and `mdi_' prefixes on predicate names.
This was done for consistency with tr_store.m.
extras/trailed_update/trailed_update.m:
Add tr_store.m to the list of modules in the `trailed_update'
library.
extras/trailed_update/.cvsignore:
Ignore .c and .h files.
