mirror of
https://github.com/Mercury-Language/mercury.git
synced 2025-12-12 20:34:19 +00:00
8151ec691c517bb8881f7020fd4ace3a6d0c9abd
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Julien Fischer
|
dc79a9a412 |
Fix a problem that was causing the namespace check to fail.
Branches: main Fix a problem that was causing the namespace check to fail. runtime/mercury_heap_profile.h: Make sure that MR_STATIC_CODE_CONST is defined when doing the namespace check. Fix some formatting issues. |
||
|
Peter Wang
|
7e26b55e74 |
Implement a new form of memory profiling, which tells the user what memory
Branches: main
Implement a new form of memory profiling, which tells the user what memory
is being retained during a program run. This is done by allocating an extra
word before each cell, which is used to "attribute" the cell to an
allocation site. The attribution, or "allocation id", is an address to an
MR_AllocSiteInfo structure generated by the Mercury compiler, giving the
procedure, filename and line number of the allocation, and the type
constructor and arity of the cell that it allocates.
The user must manually instrument the program with calls to
`benchmarking.report_memory_attribution', which forces a GC and summarises
the live objects on the heap using the attributions. The mprof tool is
extended with a new mode to parse and present that data.
Objects which are unattributed (e.g. by hand-written C code which hasn't
been updated) are still accounted for, but show up in profiles as "unknown".
Currently this profiling mode only works in conjunction with the Boehm
garbage collector, though in principle it can work with any memory allocator
for which we can access a list of the live objects. Since term size
profiling relies on the same technique of using an extra word per memory
cell, the two profiling modes are incompatible.
The output from `mprof -s' looks like this:
------ [1] some label ------
cells words cumul procedure / type (location)
14150 38872 total
* 1949/ 13.8% 4872/ 12.5% 12.5% <predicate `parser.parse_rest/7' mode 0>
975/ 6.9% 1950/ 5.0% list.list/1 (parser.m:502)
487/ 3.4% 1948/ 5.0% term.term/1 (parser.m:501)
487/ 3.4% 974/ 2.5% term.const/0 (parser.m:501)
* 1424/ 10.1% 4272/ 11.0% 23.5% <predicate `parser.parse_simple_term_2/6' mode 0>
708/ 5.0% 2832/ 7.3% term.term/1 (parser.m:643)
708/ 5.0% 1416/ 3.6% term.const/0 (parser.m:643)
...
boehm_gc/alloc.c:
boehm_gc/include/gc.h:
boehm_gc/misc.c:
boehm_gc/reclaim.c:
Add a callback function to be called for every live object after a GC.
Add a function to write out the GC_size_map array.
compiler/layout.m:
Define the alloc_site_info type which is equivalent to the
MR_AllocSiteInfo C structure.
Add alloc_site_array as a kind of "layout" array.
compiler/llds.m:
Add allocation sites to `cfile' structure.
Replace TypeMsg argument (which was also for profiling) on `incr_hp'
instructions by an allocation site identifier.
Add a new foreign_proc_component for allocation site ids.
compiler/code_info.m:
compiler/global_data.m:
compiler/proc_gen.m:
Keep the set of allocation sites in the code_info and global_data
structures.
compiler/unify_gen.m:
Add allocation sites to LLDS allocation instructions.
compiler/layout_out.m:
compiler/llds_out_file.m:
compiler/llds_out_instr.m:
Output MR_AllocSiteInfo arrays in generated C files.
Output code to register the MR_AllocSiteInfo array with the Mercury
runtime.
Output allocation site ids for memory allocation instructions.
compiler/llds_out_util.m:
Add allocation sites to llds_out_info.
compiler/pragma_c_gen.m:
compiler/ml_foreign_proc_gen.m:
Generate a macro MR_ALLOC_ID which resolves to an allocation site
structure, for every foreign_proc whose C code contains the string
"MR_ALLOC_ID". This is to be used by hand-written C code which
allocates memory.
MR_PROC_LABELs are retained for backwards compatibility. Though
they were introduced for profiling, they seem to have been co-opted
for printf-debugging since then.
compiler/ml_global_data.m:
Add allocation site structures to the MLDS global data.
compiler/mlds.m:
compiler/ml_unify_gen.m:
Add allocation site id to `new_object' instruction.
compiler/mlds_to_c.m:
Output allocation site arrays and allocation ids in high-level C code.
Output a call to register the allocation site array with the Mercury
runtime.
Delete an unused predicate.
compiler/exprn_aux.m:
compiler/jumpopt.m:
compiler/livemap.m:
compiler/mercury_compile_llds_back_end.m:
compiler/middle_rec.m:
compiler/ml_accurate_gc.m:
compiler/ml_elim_nested.m:
compiler/ml_optimize.m:
compiler/ml_util.m:
compiler/mlds_to_cs.m:
compiler/mlds_to_gcc.m:
compiler/mlds_to_il.m:
compiler/mlds_to_java.m:
compiler/mlds_to_managed.m:
compiler/opt_debug.m:
compiler/opt_util.m:
compiler/use_local_vars.m:
compiler/var_locn.m:
Conform to changes.
compiler/pickle.m:
compiler/prog_event.m:
compiler/timestamp.m:
Conform to changes in memory allocation macros.
library/benchmarking.m:
Add the `report_memory_attribution' instrumentation predicates.
Conform to changes to MR_memprof_record.
library/array.m:
library/bit_buffer.m:
library/bitmap.m:
library/construct.m:
library/deconstruct.m:
library/dir.m:
library/io.m:
library/mutvar.m:
library/store.m:
library/string.m:
library/thread.semaphore.m:
library/version_array.m:
Use attributed memory allocation throughout the standard library so
that objects don't show up in the memory profile as "unknown".
Replace MR_PROC_LABEL by MR_ALLOC_ID.
mdbcomp/program_representation.m:
mdbcomp/rtti_access.m:
Replace MR_PROC_LABEL by MR_ALLOC_ID.
profiler/Mercury.options:
profiler/globals.m:
profiler/mercury_profile.m:
profiler/options.m:
profiler/output.m:
profiler/snapshots.m:
Add a new mode to `mprof' to parse and present the data from
`Prof.Snapshots' files.
Add options for the new profiling mode.
profiler/process_file.m:
Fix a typo.
runtime/mercury_conf_param.h:
#define MR_MPROF_PROFILE_MEMORY_ATTRIBUTION if memory profiling
is enabled and we are using Boehm GC.
runtime/mercury.h:
Make MR_new_object take an allocation id argument.
Conform to changes in memory allocation macros.
runtime/mercury_memory.c:
runtime/mercury_memory.h:
runtime/mercury_types.h:
Define MR_AllocSiteInfo.
Add memory allocation functions and macros which take into the
account the additional word necessary for the new profiling mode.
These should be used in preferences to the raw memory allocation
functions wherever possible so that objects do not show up in the
profile as "unknown".
Add analogues of realloc/free which take into account the offset
introduced by the attribution word.
Add function versions of the MR_new_object macros, which can't be
written in standard C. They are only used when necessary.
Add built-in allocation site ids, to be used in the runtime and
other hand-written code when context-specific ids are unavailable.
runtime/mercury_heap.h:
Make MR_tag_offset_incr_hp_msg and MR_tag_offset_incr_hp_atomic_msg
allocate an extra word when memory attribution is desired, and store
the allocation id there.
Similarly for MR_create{1,2,3}_msg.
Replace proclabel arguments in allocation macros by alloc_id
arguments.
Replace MR_hp_alloc_atomic by MR_hp_alloc_atomic_msg. It was only
used for boxing floats.
Conform to change to MR_new_object macro.
runtime/mercury_bootstrap.h:
Delete obsolete macro hp_alloc_atomic.
runtime/mercury_heap_profile.c:
runtime/mercury_heap_profile.h:
Add the code to summarise the live objects on the Boehm GC heap and
writes out the data to `Prof.Snapshots', for display by mprof.
Don't store the procedure name in MR_memprof_record: the procedure
address is enough and faster to compare.
runtime/mercury_prof.c:
Finish and close the `Prof.Snapshots' file when the program
terminates.
Conform to changes in MR_memprof_record.
runtime/mercury_misc.h:
Add a macro to expand to the name of the allocation sites array
in LLDS grades.
runtime/mercury_bitmap.c:
runtime/mercury_bitmap.h:
Pass allocation id through bitmap allocation functions.
Delete unused function MR_string_to_bitmap.
runtime/mercury_string.h:
Add MR_make_aligned_string_copy_msg.
Make string allocation macros take allocation id arguments.
runtime/mercury.c:
runtime/mercury_array_macros.h:
runtime/mercury_context.c:
runtime/mercury_deconstruct.c:
runtime/mercury_deconstruct_macros.h:
runtime/mercury_dlist.c:
runtime/mercury_engine.c:
runtime/mercury_float.h:
runtime/mercury_hash_table.c:
runtime/mercury_ho_call.c:
runtime/mercury_label.c:
runtime/mercury_prof_mem.c:
runtime/mercury_stacks.c:
runtime/mercury_stm.c:
runtime/mercury_string.c:
runtime/mercury_thread.c:
runtime/mercury_trace_base.c:
runtime/mercury_trail.c:
runtime/mercury_type_desc.c:
runtime/mercury_type_info.c:
runtime/mercury_wsdeque.c:
Use attributed memory allocation throughout the runtime so that
objects don't show up in the profile as "unknown".
runtime/mercury_memory_zones.c:
Attribute memory zones to the Mercury runtime.
runtime/mercury_tabling.c:
runtime/mercury_tabling.h:
Use attributed memory allocation macros for tabling structures.
Delete unused MR_table_realloc_* and MR_table_copy_bytes macros.
runtime/mercury_deep_copy_body.h:
Try to retain the original attribution word when copying values.
runtime/mercury_ml_expand_body.h:
Conform to changes in memory allocation macros.
runtime/mercury_tags.h:
Replace proclabel arguments by alloc_id arguments in allocation macros.
runtime/mercury_wrapper.c:
If memory attribution is enabled, tell Boehm GC that pointers may be
displaced by an extra word.
trace/mercury_trace.c:
trace/mercury_trace_tables.c:
Conform to changes in memory allocation macros.
extras/net/tcp.m:
extras/solver_types/library/any_array.m:
extras/trailed_update/tr_array.m:
Conform to changes in memory allocation macros.
doc/user_guide.texi:
Document the new profiling mode.
doc/reference_manual.texi:
Update a commented out example.
|
||
|
Zoltan Somogyi
|
972d97c44f |
Fix three bugs in the runtime system's support for deep profiling.
Estimated hours taken: 8 Branches: main Fix three bugs in the runtime system's support for deep profiling. runtime/mercury_heap.h: One bug was that information about memory allocations wasn't being collected. The problem was that code fragment that incremented the relevant fields of the current call site dynamic structure was being invoked from a set of macros that the compiler doesn't emit in deep profiling grades. The fix is to invoke that code fragment from the basic macros that all the others in mercury_heap.h are built upon. Apply the same fix to the code fragment that gathers information for complexity analysis. To prevent such problems from arising in the future, document the layers of macros defined in this module. Move some macros around to yield a consistent structure (to make the documentation easier). Convert to four-space indentation. runtime/mercury_deep_*_port_body.h: library/exception.m: The second bug was that we collected the wrong sort of call sequence number information. We tried to define the call sequence number field (MR_own_call_seqs) of call dynamic structures to make them contain, for each call, a number proportional to the time from the start of the call to the end of the call. The problems with this is that all the other cost measure fields of the call site structure (time quanta, number of memory allocations, number of words allocated) were non-cumulative; e.g. the time quanta field was incremented only when a clock interrupt happened inside the code of the called predicate, not in its descendants. Propagating the costs of descendants to the ancestors is the deep profiler's job. By including the cost of descendants in the MR_own_call_seqs field, the propagation algorithm of the deep profiler actually screws up the call sequence number information, by including the cost of descendants twice. We could modify the propagation algorithm to avoid double-propagating call sequence number information, but it would be a significant coding job. The much simpler fix is to treat the increment of the call sequence number the same way we handle a profiling interrupt: we increment the relevant field of the current call site dynamic structure. runtime/mercury_deep_profiling.h: The third bug was that we were using the mask value of 0x10 to signify both the presence of a nonzero exception count and a nonzero number of allocations count. The former was a typo. Change the data format number, since the meaning of the call sequence number field has changed. runtime/mercury_context.h: Add a missing MR_ prefix in a comment. runtime/mercury_heap_profile.h: Fix some formatting. |
||
|
Peter Ross
|
66a99d33ea |
Add the ability to turn profiling off and on programatically.
Estimated hours taken: 1 Branches: main Add the ability to turn profiling off and on programatically. This allows one to ignore parts of a running program in the profile. runtime/mercury_heap_profile.c: runtime/mercury_heap_profile.h: Add the ability to turn off and on heap profiling. runtime/mercury_prof.c: runtime/mercury_prof.h: Add the ability to stop recording the call graph. Check if time profiling is on or off before turning it on or off as it seems to be problematic to turn on already turned on profiling. |
||
|
Zoltan Somogyi
|
678fa1075f |
Add a capability for measuring statistics about the stack frame usage
Estimated hours taken: 6
Branches: main
Add a capability for measuring statistics about the stack frame usage
of a program.
configure.in:
Find out the symbolic names of the max values for the 16 and 32 bit
integer types.
runtime/mercury_conf.h.in:
Include MR_{UINT,INT}_LEAST{16,32}_MAX among the macros whose values
are determined by autoconfiguration.
runtime/mercury_conf_param.h:
Document MR_STACK_FRAME_STATS as a macro whose definition causes the
program to collect statistics on stack frame sizes.
Fix an obsolete reference: PARALLEL was renamed MR_THREAD_SAFE a long
time ago.
Document the incompatibility between MR_THREAD_SAFE and the statistics
collection flags.
runtime/mercury_stacks.h:
If MR_STACK_FRAME_STATS is defined, then record statistics every time
we create a stack frame.
runtime/mercury_stacks.c:
Define the global variables and functions needed for stack frame
statistics.
runtime/mercury_heap_profile.h:
runtime/mercury_dword.h:
Move the macros for managing 64-bit counters from
mercury_heap_profile.h to a new header file, mercury_dword.h,
since mercury_stacks.h now needs such counters too.
Rewrite the macros to make fewer assumptions, using MR_int_least64_t
and MR_int_least32_t instead of "long long" and "int".
Add expression-like forms of some of the macros for use in
mercury_stacks.h.
Rename the type MR_dword as MR_Dword.
runtime/mercury_heap_profile.c:
#include the new header file.
runtime/mercury_prof.c:
Conform to the change from MR_dword to MR_Dword.
runtime/Mmakefile:
Add the new header file to the list of header files.
runtime/mercury_wrapper.c:
If MR_STACK_FRAME_STATS is defined, initialize the stack frame stats
before execution starts and write out the statistics when execution
ends.
tools/speedtest:
Add an option that when set, causes the script to report stack frame
stats for each variant being tested.
|
||
|
Tyson Dowd
|
db64a3588d |
Add MR_ prefixes to the types used when generating C code.
Estimated hours taken: 4 Add MR_ prefixes to the types used when generating C code. This means types such as Word, String, Bool, Float become MR_Word, MR_String, MR_Bool, MR_Float. Also define MR_Box for both the LLDS and MLDS backends so we can use it uniformly. This is very important in environments where String or Bool have already been used as system types (for example, managed C++). And besides, we should do it anyway as part of the grand namespace cleanup. I have fixed all of the uses of the non-prefixed types in the runtime and trace directories. I haven't done it for the library and compiler directories yet (no promises that I will do it in future either). But if you see a non-prefixed type in code from now on, please consider it a bug and fix it. mercury_bootstrap.h contains #defines to map the non-prefixed types into the prefixed ones. Like many of the other namespace cleaning backwards compatibility macros, this can be turned off with MR_NO_BACKWARDS_COMPAT. This shouldn't break any code, but this kind of change affects so many things that of course there could be problems lurking in there somewhere. If you start getting errors from the C compiler after this change is installed, you will want to make sure you at least have the runtime system updated so that you are getting the backwards compatibility definitions in mercury_bootstrap.h. Then if you continue to have problems you can bug me about it. compiler/export.m: compiler/llds_out.m: compiler/mlds_to_c.m: Use MR_Word, MR_Float, MR_Bool, etc when generating C. doc/reference_manual.texi: Update the reference manual to talk about MR_Word, MR_String, MR_Char, etc. runtime/mercury_bootstrap.h: Add bootstrapping typedefs. runtime/*: trace/*: Change Word, Float, Bool, Code, String, etc to MR_Word, MR_Float, MR_Bool, MR_Code, MR_String. |
||
|
Fergus Henderson
|
fbd55b8d7c |
Fix the code in the runtime so that `mmake check_headers' succeeds.
Estimated hours taken: 3
Fix the code in the runtime so that `mmake check_headers' succeeds.
runtime/*.h:
Add #include statements to make all the header files
self-contained.
runtime/mercury_table.h:
runtime/mercury_table.c:
Delete these empty files.
For some reason zs left these files as empty instead of deleting
them when he renamed the module as mercury_hash_table.{h,c}.
runtime/mercury_table_builtins.h:
runtime/mercury_table_enum.h:
runtime/mercury_table_any.h:
runtime/mercury_table_type_info.h:
runtime/mercury_table_builtins.c:
runtime/mercury_table_enum.c:
runtime/mercury_table_any.c:
runtime/mercury_table_type_info.c:
runtime/mercury_tabling.h:
runtime/mercury_tabling.c:
Avoid a circular dependency problem (mercury_tabling.h
depended on mercury_table_*.h which in turn depended on
mercury_tabling.h) by moving all the tabling code into
a single module mercury_tabling.{h,c}.
The new module is a total of 1100 lines of code,
which is not too large IMHO.
Also improve the documentation a little.
runtime/Mmakefile:
Update the reflect the added and removed files.
Also update the rule for `mmake check_headers' so that
it can make use of parallelism in parallel makes.
And make a start towards enforcing namespace cleanliness --
I've added code to check it, but not to enforce it.
|
||
|
Fergus Henderson
|
e6ac077bae |
Add support for memory profiling.
Estimated hours taken: 40 (+ unknown time by Zoltan)
Add support for memory profiling.
(A significant part of this change is actuallly Zoltan's work. Zoltan
did the changes to the compiler and a first go at the changes to the
runtime and library. I rewrote much of Zoltan's changes to the runtime
and library, added support for the new options/grades, added code to
interface with mprof, did the changes to the profiler, and wrote the
documentation.)
[TODO: add test cases.]
NEWS:
Mention support for memory profiling.
runtime/mercury_heap_profile.h:
runtime/mercury_heap_profile.c:
New files. These contain code to record heap profiling information.
runtime/mercury_heap.h:
Add new macros incr_hp_msg(), tag_incr_hp_msg(),
incr_hp_atomic_msg(), and tag_incr_hp_atomic_msg().
These are like the non-`msg' versions, except that if
PROFILE_MEMORY is defined, they also call MR_record_allocation()
from mercury_heap_profile.h to record heap profiling information.
Also, fix up the indentation in lots of places.
runtime/mercury_prof.h:
runtime/mercury_prof.c:
Added code to dump out memory profiling information to files
`Prof.MemoryWords' and `Prof.MemoryCells' (for use by mprof).
Change the format of the `Prof.Counts' file so that the
first line says what it is counting, the units, and a scale
factor. Prof.MemoryWords and Prof.MemoryCells can thus have
exactly the same format as Prof.Counts.
Also cleaned up the interface to mercury_prof.c a bit, and did
various other minor cleanups -- indentation changes, changes to
use MR_ prefixes, additional comments, etc.
runtime/mercury_prof_mem.h:
runtime/mercury_prof_mem.c:
Rename prof_malloc() as MR_prof_malloc().
Rename prof_make() as MR_PROF_NEW() and add MR_PROF_NEW_ARRAY().
runtime/mercury_wrapper.h:
Minor modifications to reflect the new interface to mercury_prof.c.
runtime/mercury_wrapper.c:
runtime/mercury_label.c:
Rename the old `-p' (primary cache size) option as `-C'.
Add a new `-p' option to disable profiling.
runtime/Mmakefile:
Add mercury_heap_profile.[ch].
Put the list of files in alphabetical order.
Delete some obsolete stuff for supporting `.mod' files.
Mention that libmer_dll.h and libmer_globals.h are
produced by Makefile.DLLs.
runtime/mercury_imp.h:
Mention that libmer_dll.h is produced by Makefile.DLLs.
runtime/mercury_dummy.c:
Change a comment to refer to libmer_dll.h rather than
libmer_globals.h.
compiler/llds.m:
Add a new field to `create' and `incr_hp' instructions
holding the name of the type, for heap profiling.
compiler/unify_gen.m:
Initialize the new field of `create' instructions with
the appropriate type name.
compiler/llds_out.m:
Output incr_hp_msg() / tag_incr_hp_msg() instead of
incr_hp() / tag_incr_hp().
compiler/*.m:
Minor changes to most files in the compiler back-end to
accomodate the new field in `incr_hp' and `create' instructions.
library/io.m:
Add `io__report_full_memory_stats'.
library/benchmarking.m:
Add `report_full_memory_stats'. This uses the information saved
by runtime/mercury_heap_profile.{c,h} to print out a report
of memory usage by procedures and by types.
Also modify `report_stats' to print out some of that information.
compiler/mercury_compile.m:
If `--statistics' is enabled, call io__report_full_memory_stats
at the end of main/2. This will print out full memory statistics,
if the compiler was compiled with memory profiling enabled.
compiler/options.m:
compiler/handle_options.m:
runtime/mercury_grade.h:
scripts/ml.in:
scripts/mgnuc.in:
scripts/init_grade_options.sh-subr:
scripts/parse_grade_options.sh-subr:
Add new option `--memory-profiling' and new grade `.memprof'.
Add `--time-profiling' as a new synonym for `--profiling'.
Also add `--profile-memory' for more fine-grained control:
`--memory-profiling' implies both `--profile-memory' and
`--profile-calls'.
scripts/mprof_merge_runs:
Update to handle the new format of Prof.Counts and to
also merge Prof.MemoryWords and Prof.MemoryCells.
profiler/options.m:
profiler/mercury_profile.m:
Add new options `--profile memory-words' (`-m'),
`--profile memory-cells' (`-M') and `--profile time' (`-t').
Thes options make the profiler select a different count file,
Prof.MemoryWords or Prof.MemoryCells instead of Prof.Counts.
specific to time profiling.
profiler/read.m:
profiler/process_file.m:
profiler/prof_info.m:
profiler/generate_output.m:
Update to handle the new format of the counts file.
When reading the counts file, look at the first line of
the file to determine what is being profiled.
profiler/globals.m:
Add a new global variable `what_to_profile' that records
what is being profiled.
profiler/output.m:
Change the headings to reflect what is being profiled.
doc/user_guide.texi:
Document memory profiling.
Document new options.
doc/user_guide.texi:
compiler/options.m:
Comment out the documentation for `.proftime'/`--profile-time',
since doing time and call profiling seperately doesn't work,
because the code addresses change when you recompile with a
different grade. Ditto for `.profmem'/`--profile-memory'.
Also comment out the documentation for
`.profcalls'/`--profile-calls', since it is redundant --
`.memprof' produces the same information and more.
configure.in:
Build a `.memprof' grade. (Hmm, should we do this only
if `--enable-all-grades' is specified?)
Don't ever build a `.profcalls' grade.
|