Estimated hours taken: 40 (+ unknown time by Zoltan)
Add support for memory profiling.
(A significant part of this change is actuallly Zoltan's work. Zoltan
did the changes to the compiler and a first go at the changes to the
runtime and library. I rewrote much of Zoltan's changes to the runtime
and library, added support for the new options/grades, added code to
interface with mprof, did the changes to the profiler, and wrote the
documentation.)
[TODO: add test cases.]
NEWS:
Mention support for memory profiling.
runtime/mercury_heap_profile.h:
runtime/mercury_heap_profile.c:
New files. These contain code to record heap profiling information.
runtime/mercury_heap.h:
Add new macros incr_hp_msg(), tag_incr_hp_msg(),
incr_hp_atomic_msg(), and tag_incr_hp_atomic_msg().
These are like the non-`msg' versions, except that if
PROFILE_MEMORY is defined, they also call MR_record_allocation()
from mercury_heap_profile.h to record heap profiling information.
Also, fix up the indentation in lots of places.
runtime/mercury_prof.h:
runtime/mercury_prof.c:
Added code to dump out memory profiling information to files
`Prof.MemoryWords' and `Prof.MemoryCells' (for use by mprof).
Change the format of the `Prof.Counts' file so that the
first line says what it is counting, the units, and a scale
factor. Prof.MemoryWords and Prof.MemoryCells can thus have
exactly the same format as Prof.Counts.
Also cleaned up the interface to mercury_prof.c a bit, and did
various other minor cleanups -- indentation changes, changes to
use MR_ prefixes, additional comments, etc.
runtime/mercury_prof_mem.h:
runtime/mercury_prof_mem.c:
Rename prof_malloc() as MR_prof_malloc().
Rename prof_make() as MR_PROF_NEW() and add MR_PROF_NEW_ARRAY().
runtime/mercury_wrapper.h:
Minor modifications to reflect the new interface to mercury_prof.c.
runtime/mercury_wrapper.c:
runtime/mercury_label.c:
Rename the old `-p' (primary cache size) option as `-C'.
Add a new `-p' option to disable profiling.
runtime/Mmakefile:
Add mercury_heap_profile.[ch].
Put the list of files in alphabetical order.
Delete some obsolete stuff for supporting `.mod' files.
Mention that libmer_dll.h and libmer_globals.h are
produced by Makefile.DLLs.
runtime/mercury_imp.h:
Mention that libmer_dll.h is produced by Makefile.DLLs.
runtime/mercury_dummy.c:
Change a comment to refer to libmer_dll.h rather than
libmer_globals.h.
compiler/llds.m:
Add a new field to `create' and `incr_hp' instructions
holding the name of the type, for heap profiling.
compiler/unify_gen.m:
Initialize the new field of `create' instructions with
the appropriate type name.
compiler/llds_out.m:
Output incr_hp_msg() / tag_incr_hp_msg() instead of
incr_hp() / tag_incr_hp().
compiler/*.m:
Minor changes to most files in the compiler back-end to
accomodate the new field in `incr_hp' and `create' instructions.
library/io.m:
Add `io__report_full_memory_stats'.
library/benchmarking.m:
Add `report_full_memory_stats'. This uses the information saved
by runtime/mercury_heap_profile.{c,h} to print out a report
of memory usage by procedures and by types.
Also modify `report_stats' to print out some of that information.
compiler/mercury_compile.m:
If `--statistics' is enabled, call io__report_full_memory_stats
at the end of main/2. This will print out full memory statistics,
if the compiler was compiled with memory profiling enabled.
compiler/options.m:
compiler/handle_options.m:
runtime/mercury_grade.h:
scripts/ml.in:
scripts/mgnuc.in:
scripts/init_grade_options.sh-subr:
scripts/parse_grade_options.sh-subr:
Add new option `--memory-profiling' and new grade `.memprof'.
Add `--time-profiling' as a new synonym for `--profiling'.
Also add `--profile-memory' for more fine-grained control:
`--memory-profiling' implies both `--profile-memory' and
`--profile-calls'.
scripts/mprof_merge_runs:
Update to handle the new format of Prof.Counts and to
also merge Prof.MemoryWords and Prof.MemoryCells.
profiler/options.m:
profiler/mercury_profile.m:
Add new options `--profile memory-words' (`-m'),
`--profile memory-cells' (`-M') and `--profile time' (`-t').
Thes options make the profiler select a different count file,
Prof.MemoryWords or Prof.MemoryCells instead of Prof.Counts.
specific to time profiling.
profiler/read.m:
profiler/process_file.m:
profiler/prof_info.m:
profiler/generate_output.m:
Update to handle the new format of the counts file.
When reading the counts file, look at the first line of
the file to determine what is being profiled.
profiler/globals.m:
Add a new global variable `what_to_profile' that records
what is being profiled.
profiler/output.m:
Change the headings to reflect what is being profiled.
doc/user_guide.texi:
Document memory profiling.
Document new options.
doc/user_guide.texi:
compiler/options.m:
Comment out the documentation for `.proftime'/`--profile-time',
since doing time and call profiling seperately doesn't work,
because the code addresses change when you recompile with a
different grade. Ditto for `.profmem'/`--profile-memory'.
Also comment out the documentation for
`.profcalls'/`--profile-calls', since it is redundant --
`.memprof' produces the same information and more.
configure.in:
Build a `.memprof' grade. (Hmm, should we do this only
if `--enable-all-grades' is specified?)
Don't ever build a `.profcalls' grade.
Estimated hours taken: 0.5
configure.in:
scripts/Mmake.rules:
Delete calls to `chmod -r'. Turning off read permission seems
to cause more problems than it solves. Amoung other things,
it causes problems on systems such as Windows where you can't
delete a read-only file. (GNU-win32 has some work-arounds for
that, but there are bugs in the work-arounds.)
We should probably also delete the various calls to `chmod +w',
since they are no longer needed, but for the moment I have left
them in for backwards compatibility. We can reconsider this
in a year or two's time.
Estimated hours taken: 1
Add support for a new `GRADEFLAGS' variable to Mmake,
so that you can use the more readable (and more portable)
`GRADEFLAGS = --profiling' rather than e.g. `GRADEFLAGS = asm_fast.gc.prof'.
scripts/Mmake.vars.in:
Add GRADEFLAGS variable, defaulting to `--grade $(GRADE)'.
scripts/Mmake.rules:
Change various rules to use `$(GRADEFLAGS)' instead of
`--grade $(GRADE)'.
compiler/modules.m:
Change various rules in the generated `.dep' files to
use `$(GRADEFLAGS)' instead of `--grade $(GRADE)'.
doc/user_guide.texi:
Modify the documentation to reflect the above changes.
Estimated hours taken: 0.5
Delete support for `.mod' files, since they are obsolete now.
scripts/mod2c:
Delete this script.
scripts/Mmakefile:
Delete reference to `mod2c'.
scripts/Mmake.rules:
Delete the `.mod' suffix and the rule for .mod -> .c,
scripts/Mmake.vars.in:
Delete the MOD2C, MOD2CFLAGS, MOD2H, and MOD2HFLAGS variables.
doc/user_guide.texi
Delete mention of `.mod' files.
Estimated hours taken: 4
scripts/Mmake.vars.in:
Set GPATH to the same value as VPATH.
This is necessary to make things work with GNU Make versions >= 3.76.
(Otherwise GNU Make tries to build the library interface files in
the current directory rather than the library directory,
if they are out-of-date with respect to the datestamp files,
and then since they don't exist in the current directory,
it thinks they have changed, even if they haven't.)
Estimated hours taken: 0.5
scripts/mprof_merge_runs:
Change the layout of the `--help' message so that
the formatting of the automatically-generated man page
comes out OK.
Estimated hours taken: 2
scripts/mprof_merge_runs:
New file. A script for merging multiple runs into a single
profile.
scripts/Mmakefile:
Add mprof_merge_runs to the list of scripts.
doc/Mmakefile:
Add mprof_merge_runs to the list of things which need manpages.
(The manpage will be created automatically by `make_manpage'.)
doc/user_guide.texi:
Document the use of mprof_merge_runs.
Estimated hours taken: 0.25
scripts/ml.in:
Make the user's `-R' (`-rpath') options precede the Mercury ones.
This allows the user to use their own local copies of the
Mercury shared libraries.
Estimated hours taken: 0.25
scripts/mmc.in:
Set the MERCURY_LPSOLVE environment variable to the full path
"@BINDIR@/lp_solve", if not already set, so that it won't matter
if users have another command called `lp_solve' in their path.
compilation model options as mmc.
Allow the user to mix `--grade foo' options with
other options that affect the grade such as `--profiling'.
Compute the final grade to link with from the options.
Also add a few new options and grade modifiers.
compiler/options.m:
compiler/mercury_compile.m:
Add `--profile-time' and `--profile-calls' options.
Change `--profiling' to now just imply both of those.
Add `--pic-reg' option (just implies `-DPIC_REG' in cflags).
compiler/handle_options.m:
Add support for grade modifiers `.proftime' and `.profcalls'.
Make `.debug' a grade modifier, rather than having a base
grade `debug'.
scripts/init_grade_options.sh-subr:
scripts/parse_grade_options.sh-subr:
New files containing sh subroutines for parsing grade-related
options.
configure.in:
bindist/bindist.configure.in:
Use AC_SUBST_FILE to allow #inclusion of the above-mentioned
`.sh-subr' files.
scripts/ml.in:
scripts/mgnuc.in:
Use the above-mentioned sh subroutines.
scripts/ml.in:
compiler/mercury_compile.m:
compiler/handle_options.m:
Compute the final grade to link with from the various
grade-related options.
scripts/mgnuc.in:
Support the `--inline-alloc' option.
doc/user_guide.texi:
README.Linux:
Document the above changes.
Estimated hours taken: 0.5
scripts/ml.in:
For Linux, make `--make-shared-lib' imply `--mercury-libs shared',
since on Linux the shared and non-shared grades are not compatible.
Estimated hours taken: 0.25
Fix a bug that broke things on SunOS 4.x.
scripts/ml.in:
Don't pass -rpath options unless the system supports shared
libraries.
Estimated hours taken: 0.25
scripts/c2init.in:
Use only libmercury.init and runtime.init in $MERCURY_MOD_LIB_DIR,
rather than using `$MERCURY_MOD_LIB_DIR/*'. This avoids problems
if other .init files (e.g. cfloat_lib.init) are installed in
the same directory.
Estimated hours taken: 8
Split up the startup interface so that there are seperate
initialization and termination functions, rather than just a single
mercury_runtime_main() function which does everything.
Also change things so that the io__state data is stored in global
variables.
runtime/init.h:
util/mkinit.c:
runtime/wrapper.h:
runtime/wrapper.mod:
Move declarations for stuff defined in wrapper.mod from init.h
to wrapper.h. Clean up the remainder of init.h so that it
is clear which parts are interface and which are just there
for use by *_init.c.
Change the automatically-generated *_init.c files
so that they call mercury_runtime_init(),
mercury_runtime_main(), and mercury_runtime_terminate(),
rather than just mercury_runtime_main().
Define these new two functions in wrapper.mod.
Delete the library_entry_point; change do_interpreter
to call program_entry_point directly, rather than via
library_entry_point.
runtime/engine.h:
runtime/engine.mod:
Add new function terminate_engine().
Delete the function start_mercury_engine();
move the code that used to be there to the end
of init_engine().
runtime/context.h:
runtime/context.mod:
Add new function shutdown_processes().
(The current implementation is just a stub.)
Add a call to debug_memory() in init_process_context().
runtime/memory.h:
runtime/memory.c:
Export the debug_memory() function, for use by context.c.
library/io.m:
library/io.nu.nl:
Change things so that the io__state data is stored in C global
variables (or, for Prolog, using assert/retract), rather than
passing around a data structure. (Actually we still pass the
data structure around, but it is just a dummy Word that never
gets used.)
Delete the old hand-coded io__run predicate, which was
the library entry point; instead export C functions
ML_io_init_state() and ML_io_finalize_state(). Move the
code for handling profiling from io__run to do_interpreter
in runtime/wrapper.mod.
scripts/c2init.in:
Change the default entry point from io__run_0_0 to main_2_0
Estimated hours taken: 0.5
scripts/mtags:
Add a `--vim' option to work-around limitations in vim;
if `--vim' is specified, it generates a dumbed-down tags
file that vim can understand.
Estimated hours: 1
This change is part one of a group of changes to generalize the
current support for trailing. This part just involves changing
the names from application-specific (--constraints) to more
general ones (--use-trail).
scripts/mgnuc.in:
compiler/handle_options.m:
Rename the `*.cnstr' grades as `*.tr', and use
`-DMR_USE_TRAIL' rather than `-DCONSTRAINTS'.
compiler/options.m:
Rename the `--constraints' option as `--use-trail'.
compiler/code_gen.m:
compiler/disj_gen.m:
compiler/ite_gen.m:
compiler/mercury_compile.m:
Trivial changes to handle the renaming of the `constraints'
option as `use_trail'.
Estimated hours taken: 0.25
scripts/mercury_update_interface.in:
If interface file has changed, print out a message, even
if `-v' (verbose) was not specified. Rationale: this is
useful information, and printing it out makes it easier to
understand why Mmake decides to remake things.
and document it.
compiler/modules.m:
Add rules for creating the `.a', `.so', and `.init' files.
The `.so' file is creating using ml's new `--make-shared-lib' option.
scripts/Mmake.vars.in:
Add definitions for AR, ARFLAGS, RANLIB, and RANLIBFLAGS,
for use by the new rules generated in `.dep' files by modules.m.
Mmake.common.in:
Remove RANLIB, since it's now defined in scripts/Mmake.vars.in.
doc/user_guide.texi:
Add a chapter on libraries.
Estimated hours taken: 2
Rename `mc' as `mmc'.
Note: we do not change the names of variables such as MCFLAGS, just the
`mc' executable.
configure.in:
Look for mmc not mc. If you can't find mmc to bootstrap, try for
mc.
bindist/Mmakefile:
bindist/bindist.configure.in:
doc/Mmakefile:
doc/user_guide.texi:
scripts/Mmake.vars.in:
scripts/Mmakefile:
scripts/msl.in:
tools/expand_params:
tools/optstages:
tools/speedtest:
tools/test_mercury:
Change references to mc into mmc.
Estimated hours taken: 2
Add support to `ml' for creating and using shared libraries.
scripts/ml.in:
- Change the `--shared' option so that it does not pass
`-shared' to $CC. This is because passing `-shared' to gcc
would have the effect of *creating* a shared library, but
ml's documentation for `--shared' says that it means the
generated executable should *use* shared libs. Also rename
the variable $SHARED_OR_STATIC_OPT as $MAYBE_STATIC_OPT,
since the previous name was now misleading.
- Add a new option `--make-shared-lib', for creating shared libraries.
This is to allow users to create their own shared libraries.
- Add a new option `-R<dir>' / `--shared-lib-directory', that
adds a directory to the RPATH used by the dynamic linker to
search for shared libraries. This is to allow users to link
in their own shared libraries.
They weren't working because Linux requires the `-rpath' option
be specified when linking the shared libraries, as well as when
linking executables. Previously we only passed `-rpath' option
when linking executables.
configure.in:
Mmake.common.in:
bindist/bindist.build_vars.in:
Add new configuration variables for handling `-rpath' or `-R' options:
EXE_RPATH_OPT, EXE_RPATH_SEP, SHLIB_RPATH_OPT, and SHLIB_RPATH_SEP.
scripts/ml.in:
Rewrite the system-specific code for handling rpaths
to instead use the new rpath configuration variables.
library/Mmakefile:
runtime/Mmakefile:
When linking shared libraries that depend on other shared libraries,
pass appropriate rpath options using the rpath configuration variables.
Estimated hours taken: 0.5
compiler/modules.m:
scripts/Mmake.rules:
Remove references to .garb files.
We are not going to use a separate link phase for accurate gc
anymore.
Estimated hours taken: 8
scripts/Mmake.rules:
Work around a new (mis-)feature present in GNU Make 3.74
but not in 3.71: recent versions will not update the timestamp
of a target if the commands for that target are just whitespace.
Instead, we need to use an explicit sh `:' command.
Estimated hours taken: 0.5
Added documentation of the new MERCURY_SP_OVERRIDING_LIB_OBJS (etc.)
environment variables, and improved documentation of other Prolog related
Mercury environment variables.
scripts/msl.in:
document MERCURY_SP_OVERRIDING_LIB_OBJS and add documentation for some
other variables
scripts/mnl.in:
document MERCURY_NU_OVERRIDING_LIB_OBJS, and add documentation for some
other variables
Estimated hours taken: 4
Improved the error messages for map__lookup failure by writing out the types
of the values and keys.
Also, since the Prologs cannot handle the type_of functions, a `normal'
version of map__lookup has been added to map.nu.nl
To accomodate this, this change implements a mechanism to override
definitions of predicates for particular Prologs. This mechanism makes
explicit which .no or .ql files contain preds overriding others, and so
makes things a bit more robust.
library/map.m:
Implement the better error messages.
library/map.nu.nl:
The Prolog version of map__lookup.
library/Mmakefile:
Set new environment variables, MERCURY_NU_OVERRIDING_LIB_OBJS or
MERCURY_SP_OVERRIDING_LIB_OBJS to be the files which are to be
linked, but contain definitions which override others.
scripts/mnl.in:
Link everything in MERCURY_NU_OVERRIDING_LIB_OBJS last.
scripts/msl.in:
Link everything in MERCURY_SP_OVERRIDING_LIB_OBJS last.
Estimated hours taken: 0.25
Work-around a bug in gcc 2.6.3 on mips that caused a gcc internal error
when compiling compiler/bytecode.c.
scripts/mgnuc.in:
Ensure that bytecode.c is compiled with `-O1' rather than `-O2'.
Estimated hours taken: 0.25
Fix a problem on Windows reported by Dominique de Waleffe.
scripts/mercury_update_interface.in:
Delete the hard-coded setting of `PATH=/bin:/usr/bin'.
The original intent was to avoid problems with people who have
done things like putting shell scripts that override the
standard commands such as `mv' and `cp' in their path.
However, since we haven't done this consistently for all of our
shell scripts, and since this causes problems for Windows
installations (for which the binaries might not be in /bin
or /usr/bin), it seems simplest to just remove it.
If we run into any problems with this, it should be put back in
as `PATH=/bin:/usr/bin:$PATH' rather than `PATH=/bin:/usr/bin',
and it should be done consistently for all shell scripts, not just
this one.
Estimated hours taken: 2
Use EXTRA_* environment variables for setting flags.
doc/user_guide.texi:
Document that EXTRA_* variables can be used for setting flags
when mmake is being used. Document the commonly used mmake
variables.
scripts/Mmake.vars.in:
Set *FLAGS variables according to EXTRA_*FLAGS.
Estimated hours taken: 0.25
scripts/mmake.in:
Undo my previous change to use `exec', since that causes
problems because the temporary files never get removed.
Instead, I've just added a comment saying why `exec' can't
be used.
Estimated hours taken: 0.1
scripts/ml.in:
Allow `*-cygwin32' as a another name for Windows running gnu-win32,
since that is the name that `config.guess' outputs.
Estimated hours taken: 0.25
scripts/mmake.in:
Change the code so that it uses `exec' to run make,
rather than just invoking it as a sub-process. This
should be a little more efficient.
in gc 4.11 meant that `ml -static' didn't work on Linux, because
you got unresolved references to _DYNAMIC in dyn_load.o.
scripts/ml.in:
If we're passed `-static' or `--static', then pass
`-static -Wl,-defsym,_DYNAMIC=0' to gcc, so that gcc
will pass `-defsym _DYNAMIC=0' to the linker.
Estimated hours taken: 0.25
scripts/ml.in:
Fix a bug in the code to parse the new -static, --static, -shared and
--shared options: add in some missing `shift' statements.
scripts/c2init.in:
scripts/mmake.in:
scripts/msc.in:
Small changes to the help messages to make them come out
better when converted to man pages.
scripts/mgnuc.in:
Add a `--help' option.
Estimated hours taken: 1
Some work towards making it easier to use shared libraries on Linux.
scripts/ml.in:
Add a `--mercury-libs {shared, static, none}' option
(`--mercury-libs none' replaces the old `--no-libs' option).
Add `--shared' and `--static' options.
Add a `--help' option.
Estimated hours taken: 0.25
scripts/Mmake.rules:
Add a rule for creating `.pic_s' files (which are the
assembler files corresponding to the `.pic_o' objects).
Estimated hours taken: 0.25
scripts/msc.in:
Change the default compile mode from `fastcode' to `compactcode',
because the SICStus 2.1#9 native code generator for Sparc
has some bugs in it.
Estimated hours taken: 0.5
One of several changes to add support for ELF shared libraries on Linux.
scripts/ml.in:
For i?86-*-linux*, pass the appropriate flags to the linker to
link in shared libraries, if there are any.
#endif
Estimated hours taken: 0.5
More stuff to support non-Mercury main() programs.
util/c2init.c:
Implement a new `-l' / `--libary' option.
If this option is enabled, we pass `-l' to mkinit,
which has the end result that we don't emit a main(),
but instead we emit a mercury_main(argc, argv, stack_bottom)
function that can be called from C.
