Discussion of these changes can be found on the Mercury developers
mailing list archives from June 2018.
COPYING.LIB:
Add a special linking exception to the LGPL.
*:
Update references to COPYING.LIB.
Clean up some minor errors that have accumulated in copyright
messages.
runtime/*.[ch]:
trace/*.[chyl]:
As above. In some places, improve comments, e.g. by expanding contractions
such as "we've". Add #ifndef guards against double inclusion around
the trace/*.h files that did not already have them.
tools/*:
Make the corresponding changes in shell scripts that generate .[ch] files
in the runtime.
tests/*:
Conform to a slight change in the text of a message.
Move all .h and .c files to four-space indentation without tabs,
if they weren't there already.
Use the same vim line for all .h and .c files.
Align all backslashes at the ends of lines in macro definitions.
Align close comment signs.
In some places, fix inconsistent indentation.
Fix a bunch of comments. Add XXXs to a few of them.
The old version_array rewind code used linear stack space in order to
perform it's updates in the right order (newest to oldest) by following the
structure's next pointers (which are a oldest to newest list). I previously
introduced week prev pointers so that walking over this structure newest to
oldest could be done with constant stack space. However that is less
efficient than the method suggested by Peter Wang.
Peter suggested using a bitmap and traversing oldest-to-newest, marking
each update in the bitmap and checking the bitmap before making any update.
Thus preserving older values over newer ones (which is good, this code
_rewinds_ updates). This patch implements Peter's suggestion and removes
the prev pointers from the structure for C and Java backends.
Thanks to Peter Wang for giving me a hand with C#.
library/version_array.m:
As above.
runtime/mercury_bitmap.h:
Add some macros for initialising bitmaps and testing, setting and clearing
their bits.
library/bitmap.m:
java/runtime/MercuryBitmap.java:
Move the Java MercuryBitmap class into the runtime library. This makes
it easier for other standard library modules to use this Java class.
library/bitmap.m:
runtime/mercury_dotnet.cs.in:
Move C# MercuryBitmap class into runtime/mercury_dotnet.cs
library/bitmap.m:
Add extra methods to the C# MercuryBitmap class.
tests/hard_coded/version_array_test.exp:
tests/hard_coded/version_array_test.m:
Ensure that the test verifies that rolling back updates to a version
array rolls back changes in the correct order: If two updates modify the
same cell, then the older one should be visible in the result.
Use longer arrays so that the bitmap used in the rollback code is more
than one byte in length.
Branches: main, 11.07
Avoid using the __GNUC__ macro in the runtime as a test for the presence of
gcc, since clang also defines that macro. Since clang doesn't support all
of the GNU C extensions, we can't actually use __GNUC__ without also checking
whether we are actually using clang.
runtime/mercury_conf_param.h:
Add three new macros, MR_CLANG, MR_GNUC and MR_MSVC that are defined
only when the C compiler is clang, gcc, or Visual C respectively.
(In particular, MR_GNUC will _not_ be defined when the C compiler
is clang.)
runtime/mercury.c:
runtime/mercury.h:
runtime/mercury_atomic_ops.c:
runtime/mercury_atomic_ops.h
runtime/mercury_bitmap.h:
runtime/mercury_float.h:
runtime/mercury_getopt.c:
runtime/mercury_goto.h:
runtime/mercury_heap.h:
runtime/mercury_std.h:
Replace uses of the __GNUC__ and __clang__ macros with the above.
runtime/mercury_regs.h:
As above, also #include mercury_conf_param.h directly since
this file is #included by some of the tests in the configure
script.
Branches: main
Implement a new form of memory profiling, which tells the user what memory
is being retained during a program run. This is done by allocating an extra
word before each cell, which is used to "attribute" the cell to an
allocation site. The attribution, or "allocation id", is an address to an
MR_AllocSiteInfo structure generated by the Mercury compiler, giving the
procedure, filename and line number of the allocation, and the type
constructor and arity of the cell that it allocates.
The user must manually instrument the program with calls to
`benchmarking.report_memory_attribution', which forces a GC and summarises
the live objects on the heap using the attributions. The mprof tool is
extended with a new mode to parse and present that data.
Objects which are unattributed (e.g. by hand-written C code which hasn't
been updated) are still accounted for, but show up in profiles as "unknown".
Currently this profiling mode only works in conjunction with the Boehm
garbage collector, though in principle it can work with any memory allocator
for which we can access a list of the live objects. Since term size
profiling relies on the same technique of using an extra word per memory
cell, the two profiling modes are incompatible.
The output from `mprof -s' looks like this:
------ [1] some label ------
cells words cumul procedure / type (location)
14150 38872 total
* 1949/ 13.8% 4872/ 12.5% 12.5% <predicate `parser.parse_rest/7' mode 0>
975/ 6.9% 1950/ 5.0% list.list/1 (parser.m:502)
487/ 3.4% 1948/ 5.0% term.term/1 (parser.m:501)
487/ 3.4% 974/ 2.5% term.const/0 (parser.m:501)
* 1424/ 10.1% 4272/ 11.0% 23.5% <predicate `parser.parse_simple_term_2/6' mode 0>
708/ 5.0% 2832/ 7.3% term.term/1 (parser.m:643)
708/ 5.0% 1416/ 3.6% term.const/0 (parser.m:643)
...
boehm_gc/alloc.c:
boehm_gc/include/gc.h:
boehm_gc/misc.c:
boehm_gc/reclaim.c:
Add a callback function to be called for every live object after a GC.
Add a function to write out the GC_size_map array.
compiler/layout.m:
Define the alloc_site_info type which is equivalent to the
MR_AllocSiteInfo C structure.
Add alloc_site_array as a kind of "layout" array.
compiler/llds.m:
Add allocation sites to `cfile' structure.
Replace TypeMsg argument (which was also for profiling) on `incr_hp'
instructions by an allocation site identifier.
Add a new foreign_proc_component for allocation site ids.
compiler/code_info.m:
compiler/global_data.m:
compiler/proc_gen.m:
Keep the set of allocation sites in the code_info and global_data
structures.
compiler/unify_gen.m:
Add allocation sites to LLDS allocation instructions.
compiler/layout_out.m:
compiler/llds_out_file.m:
compiler/llds_out_instr.m:
Output MR_AllocSiteInfo arrays in generated C files.
Output code to register the MR_AllocSiteInfo array with the Mercury
runtime.
Output allocation site ids for memory allocation instructions.
compiler/llds_out_util.m:
Add allocation sites to llds_out_info.
compiler/pragma_c_gen.m:
compiler/ml_foreign_proc_gen.m:
Generate a macro MR_ALLOC_ID which resolves to an allocation site
structure, for every foreign_proc whose C code contains the string
"MR_ALLOC_ID". This is to be used by hand-written C code which
allocates memory.
MR_PROC_LABELs are retained for backwards compatibility. Though
they were introduced for profiling, they seem to have been co-opted
for printf-debugging since then.
compiler/ml_global_data.m:
Add allocation site structures to the MLDS global data.
compiler/mlds.m:
compiler/ml_unify_gen.m:
Add allocation site id to `new_object' instruction.
compiler/mlds_to_c.m:
Output allocation site arrays and allocation ids in high-level C code.
Output a call to register the allocation site array with the Mercury
runtime.
Delete an unused predicate.
compiler/exprn_aux.m:
compiler/jumpopt.m:
compiler/livemap.m:
compiler/mercury_compile_llds_back_end.m:
compiler/middle_rec.m:
compiler/ml_accurate_gc.m:
compiler/ml_elim_nested.m:
compiler/ml_optimize.m:
compiler/ml_util.m:
compiler/mlds_to_cs.m:
compiler/mlds_to_gcc.m:
compiler/mlds_to_il.m:
compiler/mlds_to_java.m:
compiler/mlds_to_managed.m:
compiler/opt_debug.m:
compiler/opt_util.m:
compiler/use_local_vars.m:
compiler/var_locn.m:
Conform to changes.
compiler/pickle.m:
compiler/prog_event.m:
compiler/timestamp.m:
Conform to changes in memory allocation macros.
library/benchmarking.m:
Add the `report_memory_attribution' instrumentation predicates.
Conform to changes to MR_memprof_record.
library/array.m:
library/bit_buffer.m:
library/bitmap.m:
library/construct.m:
library/deconstruct.m:
library/dir.m:
library/io.m:
library/mutvar.m:
library/store.m:
library/string.m:
library/thread.semaphore.m:
library/version_array.m:
Use attributed memory allocation throughout the standard library so
that objects don't show up in the memory profile as "unknown".
Replace MR_PROC_LABEL by MR_ALLOC_ID.
mdbcomp/program_representation.m:
mdbcomp/rtti_access.m:
Replace MR_PROC_LABEL by MR_ALLOC_ID.
profiler/Mercury.options:
profiler/globals.m:
profiler/mercury_profile.m:
profiler/options.m:
profiler/output.m:
profiler/snapshots.m:
Add a new mode to `mprof' to parse and present the data from
`Prof.Snapshots' files.
Add options for the new profiling mode.
profiler/process_file.m:
Fix a typo.
runtime/mercury_conf_param.h:
#define MR_MPROF_PROFILE_MEMORY_ATTRIBUTION if memory profiling
is enabled and we are using Boehm GC.
runtime/mercury.h:
Make MR_new_object take an allocation id argument.
Conform to changes in memory allocation macros.
runtime/mercury_memory.c:
runtime/mercury_memory.h:
runtime/mercury_types.h:
Define MR_AllocSiteInfo.
Add memory allocation functions and macros which take into the
account the additional word necessary for the new profiling mode.
These should be used in preferences to the raw memory allocation
functions wherever possible so that objects do not show up in the
profile as "unknown".
Add analogues of realloc/free which take into account the offset
introduced by the attribution word.
Add function versions of the MR_new_object macros, which can't be
written in standard C. They are only used when necessary.
Add built-in allocation site ids, to be used in the runtime and
other hand-written code when context-specific ids are unavailable.
runtime/mercury_heap.h:
Make MR_tag_offset_incr_hp_msg and MR_tag_offset_incr_hp_atomic_msg
allocate an extra word when memory attribution is desired, and store
the allocation id there.
Similarly for MR_create{1,2,3}_msg.
Replace proclabel arguments in allocation macros by alloc_id
arguments.
Replace MR_hp_alloc_atomic by MR_hp_alloc_atomic_msg. It was only
used for boxing floats.
Conform to change to MR_new_object macro.
runtime/mercury_bootstrap.h:
Delete obsolete macro hp_alloc_atomic.
runtime/mercury_heap_profile.c:
runtime/mercury_heap_profile.h:
Add the code to summarise the live objects on the Boehm GC heap and
writes out the data to `Prof.Snapshots', for display by mprof.
Don't store the procedure name in MR_memprof_record: the procedure
address is enough and faster to compare.
runtime/mercury_prof.c:
Finish and close the `Prof.Snapshots' file when the program
terminates.
Conform to changes in MR_memprof_record.
runtime/mercury_misc.h:
Add a macro to expand to the name of the allocation sites array
in LLDS grades.
runtime/mercury_bitmap.c:
runtime/mercury_bitmap.h:
Pass allocation id through bitmap allocation functions.
Delete unused function MR_string_to_bitmap.
runtime/mercury_string.h:
Add MR_make_aligned_string_copy_msg.
Make string allocation macros take allocation id arguments.
runtime/mercury.c:
runtime/mercury_array_macros.h:
runtime/mercury_context.c:
runtime/mercury_deconstruct.c:
runtime/mercury_deconstruct_macros.h:
runtime/mercury_dlist.c:
runtime/mercury_engine.c:
runtime/mercury_float.h:
runtime/mercury_hash_table.c:
runtime/mercury_ho_call.c:
runtime/mercury_label.c:
runtime/mercury_prof_mem.c:
runtime/mercury_stacks.c:
runtime/mercury_stm.c:
runtime/mercury_string.c:
runtime/mercury_thread.c:
runtime/mercury_trace_base.c:
runtime/mercury_trail.c:
runtime/mercury_type_desc.c:
runtime/mercury_type_info.c:
runtime/mercury_wsdeque.c:
Use attributed memory allocation throughout the runtime so that
objects don't show up in the profile as "unknown".
runtime/mercury_memory_zones.c:
Attribute memory zones to the Mercury runtime.
runtime/mercury_tabling.c:
runtime/mercury_tabling.h:
Use attributed memory allocation macros for tabling structures.
Delete unused MR_table_realloc_* and MR_table_copy_bytes macros.
runtime/mercury_deep_copy_body.h:
Try to retain the original attribution word when copying values.
runtime/mercury_ml_expand_body.h:
Conform to changes in memory allocation macros.
runtime/mercury_tags.h:
Replace proclabel arguments by alloc_id arguments in allocation macros.
runtime/mercury_wrapper.c:
If memory attribution is enabled, tell Boehm GC that pointers may be
displaced by an extra word.
trace/mercury_trace.c:
trace/mercury_trace_tables.c:
Conform to changes in memory allocation macros.
extras/net/tcp.m:
extras/solver_types/library/any_array.m:
extras/trailed_update/tr_array.m:
Conform to changes in memory allocation macros.
doc/user_guide.texi:
Document the new profiling mode.
doc/reference_manual.texi:
Update a commented out example.
Estimated hours taken: 0.25
Branches: main
runtime/mercury_bitmap.h:
Fix comparison of bitmaps: it was calling memcmp on the
entire bitmap, not just the elements.
Estimated hours taken: 0
Branches: main
runtime/mercury_bitmap.h:
Fix a problem with Simon's recent change that broke compilation
of the runtime on 64-bit machines.
Estimated hours taken: 80
Branches: main
Improvements for bitmap.m, to make it useable as a general container
for binary data.
library/bitmap.m:
runtime/mercury_bitmap.c:
runtime/mercury_bitmap.h:
Specialize the representation of bitmaps to an array of unsigned
bytes defined as a foreign type.
This is better than building on top of array(int) because it:
- is better for interfacing with foreign code
- has a more sensible machine-independent comparison order
(same as array(bool))
- avoids storing the size twice
- has more efficient copying, unification, comparison and tabling
(although we should probably specialize the handling of array(int)
and isomorphic types as well)
- uses GC_MALLOC_ATOMIC to avoid problems with bit patterns that look
like pointers (although we should do that for array(int) as well)
XXX The code for the Java and IL backends is untested.
Building the library in grade Java with Sun JDK 1.6 failed (but
at least passed error checking), and I don't have access to a
copy of MSVS.NET. The foreign code that needs to be tested is
trivial.
Add fields `bit', `bits' and `byte' to get/set a single bit,
multiple bits (from an int) or an 8 bit byte.
Add functions for converting bitmaps to hex strings and back,
for use by stream.string_writer.write and deconstruct.functor/4.
bitmap.intersect was buggy in the case where the input bitmaps
had a different size. Given that bitmaps are implemented with
a fixed domain (lookups out of range throw an exception), it
makes more sense to throw an exception in that case anyway,
so all of the set operations do that now.
The difference operation actually performed xor. Fix it and
add an xor function.
library/version_bitmap.m:
This hasn't been fully updated to be the same as bitmap.m.
The payoff would be much less because foreign code can't
really do anything with version_bitmaps.
Add a `bit' field.
Deprecate the `get/2' function in favour of the `bit' field.
Fix the union, difference, intersection and xor functions
as for bitmap.m.
Fix comparison of version_arrays so that it uses the same
method as array.m: compare size then elements in order.
The old code found version_arrays to be equal if one was
a suffix of the other.
library/char.m:
Add predicates for converting between hex digits and integers.
library/io.m:
library/stream.string_writer.m:
library/term.m:
Read and write bitmaps.
runtime/mercury_type_info.h:
runtime/mercury_deep_copy_body.h:
runtime/mercury_mcpp.h:
runtime/mercury_table_type_body.h:
runtime/mercury_tabling_macros.h:
runtime/mercury_unify_compare_body.h:
runtime/mercury_construct.c:
runtime/mercury_deconstruct.c:
runtime/mercury_term_size.c:
runtime/mercury_string.h:
library/construct.m:
library/deconstruct.m
compiler/prog_type.m:
compiler/mlds_to_gcc.m:
compiler/rtti.m:
Add a MR_TypeCtorRep for bitmaps, and handle it in the library
and runtinme.
library/Mercury.options:
Compile bitmap.m with `--no-warn-insts-without-matching-type'.
runtime/mercury_type_info.h:
Bump MR_RTTI_VERSION.
NEWS:
Document the changes.
tests/hard_coded/Mmakefile:
tests/hard_coded/bitmap_test.m:
tests/hard_coded/bitmap_simple.m:
tests/hard_coded/bitmap_tester.m:
tests/hard_coded/bitmap_test.exp:
tests/tabling/Mmakefile:
tests/tabling/expand_bitmap.m:
tests/tabling/expand_bitmap.exp:
tests/hard_coded/version_array_test.m:
tests/hard_coded/version_array_test.exp:
Test cases.
