mirror of
https://github.com/Mercury-Language/mercury.git
synced 2026-04-18 19:03:45 +00:00
083d376e6598628362ee91c2da170febd83590f4
47 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Zoltan Somogyi
|
e35a09542e |
Print two kinds of ambiguities only if asked for.
trace/mercury_trace_cmd_developer.c:
trace/mercury_trace_tables.[ch]:
Add two options to the mdb command "ambiguity".
Print ambiguities between function and predicate forms of the same
operation, such as list.length, only if the new option -b, or
--both-pred-and-func, is given.
Print ambiguities involving procedures that were created by type
specialization only if the new option -s, or --typespec is given.
(The -t option name was already taken.)
These changes remove from the ambiguity command's output
(some of) the parts that are not useful when one wants to eliminate
ambiguities by renaming.
Clarify a heading.
doc/user_guide.texi:
Document the above changes.
runtime/mercury_proc_id.h:
Fix a field name that has become misleading.
MR_UserProcId_Structs have a field named MR_user_arity.
In this name, the "MR_user_" part is a prefix shared by the other
fields in that structure, to indicate that they are part of the id
of a user-defined procedure, as opposed to a compiler-created
unify, compare or index procedure. However, the arity it contains
is what the compiler now calls a pred_form_arity: it does not count
type_info and typeclass_info arguments added by polymorphism, but
it *does* count function return values for functions. This is now
misleading, because in the compiler, a user_arity does *not* count
function return values for functions.
Replace this field name with MR_user_pred_form_arity, which tells
readers that this arity is a pred_form_arity. The presence of the
"user" part of the name may still cause some confusion, but at least
that confusion should motivate readers to look up the field name,
whose comment should clarify things.
mdbcomp/rtti_access.m:
runtime/mercury_debug.c:
runtime/mercury_deep_profiling.c:
runtime/mercury_ho_call.c:
runtime/mercury_layout_util.c:
runtime/mercury_ml_expand_body.h:
runtime/mercury_stack_layout.h:
runtime/mercury_trace_base.c:
trace/mercury_trace.c:
trace/mercury_trace_external.c:
trace/mercury_trace_util.c:
Conform to the change in mercury_proc_id.h.
tests/debugger/ambiguity.{m,inp,exp}:
tests/debugger/ambiguity_helper.m:
Expand this test case to test the new functionality.
The type specialized predicates are in a new helper module,
because this is the simplest way to avoid dead procedure elimination
deleting any of the predicates whose names we want to test for ambiguities.
|
||
|
Julien Fischer
|
ee9aaa8c1b |
Always use 64-bit offsets with fseek() and ftell().
Most sane systems use 64-bit offsets with those functions; Windows does not.
runtime/mercury_file.h:
Define macros MR_fseek and MR_ftell that expand to the name of the
fseek or ftell function that uses 64-bit offsets.
runtime/mercury_deep_profiling.c:
library/io.m:
Use the new macros and always use 64-bit offsets.
|
||
|
Peter Wang
|
72f174b4e2 |
Don't print value of errno in MR_fatal_error.
The majority of calls to MR_fatal_error do not follow an operation that sets errno, so printing out an error message unrelated to the reason for the fatal error will lead to confusion. It can also cause test failures if errno happens to be set to non-zero some time prior to an expected call to MR_fatal_error. Fixes bug #464. runtime/mercury_misc.c: Don't print value of errno in MR_fatal_error. runtime/mercury_context.c: runtime/mercury_thread.c: Pass strerror strings to MR_fatal_error where appropriate. runtime/mercury_memory_zones.c: runtime/mercury_memory_zones.h: Pass strerror strings to MR_fatal_error following failures of MR_protect_pages. Document that this assumes MR_protect_pages sets errno on error. Skip unnecessary call to sprintf before MR_fatal_error. runtime/mercury_deep_profiling.c: Skip unnecessary call to sprintf before MR_fatal_error. Reduce size of some buffers. runtime/mercury_overflow.c: runtime/mercury_stack_trace.c: Pass a fixed format string to MR_fatal_error just in case the message string may contain percentage signs. runtime/mercury_tabling.c: Skip unnecessary call to sprintf before MR_fatal_error. deep_profiler/timeout.m: library/thread.m: mdbcomp/shared_utilities.m: Pass strerror strings to MR_fatal_error where appropriate. trace/mercury_trace.c: Skip unnecessary call to sprintf before MR_fatal_error. trace/mercury_trace_external.c: Pass a fixed format string to MR_fatal_error just in case. |
||
|
Mark Brown
|
d465fa53cb |
Update the COPYING.LIB file and references to it.
Discussion of these changes can be found on the Mercury developers
mailing list archives from June 2018.
COPYING.LIB:
Add a special linking exception to the LGPL.
*:
Update references to COPYING.LIB.
Clean up some minor errors that have accumulated in copyright
messages.
|
||
|
Zoltan Somogyi
|
53b573692a |
Convert C code to use // style comments.
runtime/*.[ch]:
trace/*.[chyl]:
As above. In some places, improve comments, e.g. by expanding contractions
such as "we've". Add #ifndef guards against double inclusion around
the trace/*.h files that did not already have them.
tools/*:
Make the corresponding changes in shell scripts that generate .[ch] files
in the runtime.
tests/*:
Conform to a slight change in the text of a message.
|
||
|
Zoltan Somogyi
|
67326f16e4 |
Fix style issues in the runtime.
Move all .h and .c files to four-space indentation without tabs, if they weren't there already. Use the same vim line for all .h and .c files. Align all backslashes at the ends of lines in macro definitions. Align close comment signs. In some places, fix inconsistent indentation. Fix a bunch of comments. Add XXXs to a few of them. |
||
|
Peter Wang
|
31d3897e22 |
Thread-safe alternative to strerror.
Add MR_strerror as a thread-safe alternative to strerror. The current implementation wraps strerror_r(), strerror_s() or sys_errlist as appropriate for the platform. Bug #340. configure.ac: runtime/mercury_conf.h.in: Check for strerror_r, strerror_s. Delete irrelevant code in the sockets test for the external debugger. runtime/mercury_runtime_util.c: runtime/mercury_runtime_util.h: Add MR_strerror and use it. library/io.m: Use MR_strerror. In particular, mercury_output_error was not thread-safe. Pass errno to mercury_output_error explicitly for clarity. Delete req_lock parameter in ML_maybe_make_err_msg macro which is not needed any more. compiler/prog_event.m: runtime/mercury_deep_profiling.c: runtime/mercury_misc.c: runtime/mercury_term_size.c: runtime/mercury_trace_base.c: trace/mercury_trace_cmd_developer.c: trace/mercury_trace_cmd_exp.c: trace/mercury_trace_cmd_misc.c: trace/mercury_trace_declarative.c: trace/mercury_trace_external.c: trace/mercury_trace_internal.c: Use MR_strerror. compiler/notes/coding_standards.html: Update coding standard. extras/net/sockets.m: extras/net/tcp.m: Use MR_strerror. NEWS: Announce change. |
||
|
Julien Fischer
|
e9d90b2891 |
Avoid warnings from clang in the runtime and util directories.
runtime/mercury_stack_trace.c: In the function MR_dump_stack_from_layout_clique initialise the variable lines_dumped_so_far. (This appears to be an actual bug.) runtime/mercury_prof.c: Only define the static global in_profiling_code if either of call count profiling or time profiling is enabled. runtime/mercury_context.c: runtime/mercury_deconstruct.c: runtime/mercury_deep_profiling.c: Only define some local variables in grades that require them. runtime/mercury_float.c: runtime/mercury_deep_copy_body.h: runtime/mercury_construct.c: runtime/mercury_memory_zones.c: runtime/mercury_stm.c: runtime/mercury_trace_base.c: runtime/mercury_type_info.c: Delete unused local variables. util/mdemangle.c: Delete an unused static global variable. Use fputs in place of fprintf in a couple of places in order to avoid warnings about format strings that are not string literals. Delete an unused function. util/mkinit.c: util/mkinit_erl.c: Delete unused local variables. |
||
|
Zoltan Somogyi
|
2d0bfc0674 |
The algorithm that decides whether the order independent state update
Estimated hours taken: 120 Branches: main The algorithm that decides whether the order independent state update transformation is applicable in a given module needs access to the list of oisu pragmas in that module, and to information about the types of variables in the procedures named in those pragmas. This diff puts this information in Deep.procrep files, to make them available to the autoparallelization feedback program, to which that algorithm will later be added. Compilers that have this diff will generate Deep.procrep files in a new, slightly different format, but the deep profiler will be able to read Deep.procrep files not just in the new format, but in the old format as well. runtime/mercury_stack_layout.h: Add to module layout structures the fields holding the new information we want to put into Deep.procrep files. This means three things: - a bytecode array in module layout structures encoding the list of oisu pragmas in the module; - additions to the bytecode arrays in procedure layout structures mapping the procedure's variables to their types; and - a bytecode array containing the encoded versions of those types themselves in the module layout structure. This allows us to represent each type used in the module just once. Since there is now information in module layout structures that is needed only for deep profiling, as well as information that is needed only for debugging, the old arrangement that split a module's information between two structures, MR_ModuleLayout (debug specific info) and MR_ModuleCommonLayout (info used by both debugging and profiling), is no longer approriate. We could add a third structure containing profiling-specific info, but it is simpler to move all the info into just one structure, some of whose fields may not be used. This wastes only a few words of memory per module, but allows the runtime system to avoid unnecessary indirections. runtime/mercury_types.h: Remove the type synonym for the deleted type. runtime/mercury_grade.h: The change in mercury_stack_layout.h destroys binary compatibility with previous versions of Mercury for debug and deep profiling grades, so bump their grade-component-specific version numbers. runtime/mercury_deep_profiling.c: Write out the information in the new fields in module layout structures, if they are filled in. Since this changes the format of the Deep.procrep file, bump its version number. runtime/mercury_deep_profiling.h: runtime/mercury_stack_layout.c: Conform to the change to mercury_stack_layout.h. mdbcomp/program_representation.m: Add to module representations information about the oisu pragmas defined in that module, and the type table of the module. Optionally add to procedure representations a map mapping the variables of the procedure to their types. Rename the old var_table type to be the var_name_table type, since it contains just names. Make the var to type map separate, since it will be there only for selected procedures. Modify the predicates reading in module and procedure representations to allow them to read in the new representation, while still accepting the old one. Use the version number in the Deep.procrep file to decide which format to expect. mdbcomp/rtti_access.m: Add functions to encode the data representations that this module also decodes. Conform to the changes above. mdbcomp/feedback.automatic_parallelism.m: Conform the changes above. mdbcomp/prim_data.m: Fix layout. compiler/layout.m: Update the compiler's representation of layout structures to conform to the change to runtime/mercury_stack_layout.h. compiler/layout_out.m: Output the new parts of module layout structures. compiler/opt_debug.m: Allow the debugging of code referring to the new parts of module layout structures. compiler/llds_out_file.m: Conform to the move to a single module layout structure. compiler/prog_rep_tables.m: This new module provided mechanisms for building the string table and the type table components of module layouts. The string table part is old (it is moved here from stack_layout.m); the type table part is new. Putting this code in a module of its own allows us to remove a circular dependency between prog_rep.m and stack_layout.m; instead, both now just depend on prog_rep_tables.m. compiler/ll_backend.m: Add the new module. compiler/notes/compiler_design.html: Describe the new module. compiler/prog_rep.m: When generating the representation of a module for deep profiling, include the information needed by the order independent state update analysis: the list of oisu pragmas in the module, if any, and information about the types of variables in selected procedures. To avoid having these additions increasing the size of the bytecode representation too much, convert some fixed 32 bit numbers in the bytecode to use variable sized numbers, which will usually be 8 or 16 bits. Do not use predicates from bytecode_gen.m to encode numbers, since there is nothing keeping these in sync with the code that reads them in mdbcomp/program_representation.m. Instead, use new predicates in program_representation.m itself. compiler/stack_layout.m: Generate the new parts of module layouts. Remove the code moved to prog_rep_tables.m. compiler/continuation_info.m: compiler/proc_gen.m: Make some more information available to stack_layout.m. compiler/prog_data.m: Fix some formatting. compiler/introduce_parallelism.m: Conform to the renaming of the var_table type. compiler/follow_code.m: Fix the bug that used to cause the failure of the hard_coded/mode_check_clauses test case in deep profiling grades. deep_profiler/program_representation_utils.m: Output the new parts of module and procedure representations, to allow the correctness of this change to be tested. deep_profiler/mdprof_create_feedback.m: If we cannot read the Deep.procrep file, print a single error message and exit, instead of continuing with an analysis that will generate a whole bunch of error messages, one for each attempt to access a procedure's representation. deep_profiler/mdprof_procrep.m: Give this program an option that specifies what file it is to look at; do not hardwire in "Deep.procrep" in the current directory. deep_profiler/report.m: Add a report type that just prints the representation of a module. It returns the same information as mdprof_procrep, but from within the deep profiler, which can be more convenient. deep_profiler/create_report.m: deep_profiler/display_report.m: Respectively create and display the new report type. deep_profiler/query.m: Recognize a query asking for the new report type. deep_profiler/autopar_calc_overlap.m: deep_profiler/autopar_find_best_par.m: deep_profiler/autopar_reports.m: deep_profiler/autopar_search_callgraph.m: deep_profiler/autopar_search_goals.m: deep_profiler/autopar_types.m: deep_profiler/branch_and_bound.m: deep_profiler/coverage.m: deep_profiler/display.m: deep_profiler/html_format.m: deep_profiler/mdprof_test.m: deep_profiler/measurements.m: deep_profiler/query.m: deep_profiler/read_profile.m: deep_profiler/recursion_patterns.m: deep_profiler/top_procs.m: deep_profiler/top_procs.m: Conform to the changes above. Fix layout. tests/debugger/declarative/dependency.exp2: Add this file as a possible expected output. It contains the new field added to module representations. |
||
|
Paul Bone
|
b023312f7a |
Various changes to make feedback analysis (for automatic parallelism) easier to
use.
These changes mean that a normal installation with a deep profiling and a
low-level C parallel grade can be used for automatic parallelism.
compiler/options.m:
Rename --profile-for-implicit-parallelism to --profile-for-feedback.
--profile-for-implicit-parallelism is now retained as a synonym.
Enable --coverage-profiling by default, it is only checked if
--deep-profiling is enabled.
Make corrections to the commented-out documentation for the developer-only
coverage profiling options.
compiler/handle_options.m:
Conform to changes in options.m
Remove the set of options implied by --coverage-profiling since they're
already set correctly in options.m.
doc/user_guide.texi:
Added extra concept index declarations to entries for automatic parallelism
and feedback analysis.
Conform to changes in options.m
library/Mmakefile:
Compile the standard library with --profile-optimized
runtime/mercury_deep_profiling.c:
Always write out the Procrep.data file when writing out the Deep.data file.
runtime/mercury_wrapper.[ch]:
Remove the --deep-procrep-file MERCURY_OPTIONS option, it is now always
enabled. (In the future the procrep data may be included inside the
Deep.data file).
tools/bootcheck:
Remove the --deep-procrep-file option from the bootcheck script.
|
||
|
Julien Fischer
|
8d0936946b |
Fix problems that prevent the runtime compiling in profdeep grades with MSVC.
Branches: main Fix problems that prevent the runtime compiling in profdeep grades with MSVC. runtime/mercury_deep_profiling.c Make sure automatic variable declarations precede any code since MSVC does allow them to be mixed in the C99 style. Fix a mismatch in a function prototype and its definition. |
||
|
Paul Bone
|
cef44de0bd |
Fix an error in the initialisation of MR_main_grandparent_call_site_dynamic.
The value did not correctly correspond to the type.
Thanks to Julien for spotting/diagnosing this.
runtime/mercury_deep_profiling.c:
As above.
|
||
|
Zoltan Somogyi
|
58211e2f2e |
Allow more than 2^15 vars in a procedure representation.
Estimated hours taken: 12 Branches: main Allow more than 2^15 vars in a procedure representation. mdbcomp/program_representation.m: Allow a variable number to be represented by four bytes as well as two and one. This means that we also have to represent the number of variables in a procedure using a four-byte number, not a two-byte number. Use four bytes to represent line numbers. Programs that overflow 16-bit var numbers may also overflow 16 bit line numbers. These requires a change in the deep profiler data's binary compatibility version number. compiler/prog_rep.m: Encode vars using four bytes if necessary. Be consistent in using only signed 8-bit as well as signed 16-bit numbers. compiler/implicit_parallelism.m: Conform to the change in program_representation.m. deep_profiler/profile.m: deep_profiler/read_profile.m: Add a compression flag to the set of flags read from the data file. Put the flags into the profile_stats as a group, not one-by-one. deep_profiler/canonical.m: deep_profiler/create_report.m: deep_profiler/dump.m: deep_profiler/mdprof_feedback.m: deep_profiler/old_html_format.m: deep_profiler/old_query.m: deep_profiler/query.m: Conform to the change in profile.m. runtime/mercury_deep_profiling.c: Prepare for compression of profiling data files being implemented. runtime/mercury_stack_layout.h: Fix some documentation rot. runtime/mercury_conf_param.h: Add an implication between debug flags to make debugging easier. |
||
|
Paul Bone
|
7e7d77e23f |
Make coverage profiling data 'deep'.
The deep profiler associates measurements with their context in the call graph
modulo recursion. This has been true for all measurements except for coverage
profiling data. This patch allows coverage data to be associated with
ProcDynamic structures so that it is keyed by this context not just the static
procedure. This new behaviour is the default the old option of static coverage
profiling is still available for testing, as is no coverage profiling. Note
that, as before, coverage profiling is supported by default however coverage
points are not inserted by default.
This change will be used to measure the depth of recursion, and therefore the
average cost of recursion as well as the likely times when variables are
produced in calls for the automatic parallelisation analysis.
runtime/mercury_conf_param.h:
Create three new preprocessor macros:
MR_DEEP_PROFILING_COVERAGE - defined when coverage profiling is
enabled.
MR_DEEP_PROFILING_COVERAGE_STATIC - defined when static coverage
profiling is being used.
MR_DEEP_PROFILING_COVERAGE_DYNAMIC - defined when dynamic coverage
profiling is being used.
runtime/mercury_deep_profiling.h:
Update data structures to support dynamic coverage profiling.
Use conditional compilation to allow us to test the deep profiler in three
different modes, without coverage profiling, with static coverage profiling
and with dynamic coverage profiling.
Rename MR_PROFILING_MALLOC, since it takes a type rather than a size in
bytes it should be called MR_PROFILING_NEW to conform with existing malloc
and new functions.
runtime/mercury_deep_profiling.c:
Avoid a C compiler warning.
MR_write_out_coverage_point has been removed, it's replaced with:
MR_write_out_coverage_points_static and
MR_write_out_coverage_points_dynamic.
These write out more than one coverage point and write out either static or
dynamic coverage points.
Write a 64bit flags value (a bitfield) to the header of the Deep.data file.
This replaces the canonical byte (with a bit). and the byte that describes
the word size. This value also includes two bits describing the whether no
coverage data, static coverage data or dynamic coverage data is present in
the file. A bit is reserved ti indicate if the data is compressed (which
is not yet supported).
MR_write_fixed_size_int now writes out 8 byte integers, this is only used
for some counts present at the beginning of the data file along with the
new flags value. It now takes a MR_uint_least64_t integer as it's
parameter. The assertion to test for negative numbers has been removed
since this type is unsigned.
Increment the Deep.data file format version number.
compiler/layout_out.m:
Conditionally compile the NULL pointer that represents the coverage points
list in proc statics. This is conditional on the
MR_DEEP_PROFILING_COVERAGE_STATIC macro being defined.
compiler/coverage_profiling.m:
Add support for generating dynamic coverage points.
compiler/options.m:
compiler/handle_options.m:
Implement the new developer options for controlling coverage profiling.
library/profiling_builtin.m:
Specialize increment_coverage_point_count for both static and dynamic
coverage profiling. This creates
increment_{static,dynamic}_coverage_point_count.
deep_profiler/profile.m:
Add an extra field to profile_stats, this tracks whether the file reader
should try to read none, static or dynamic coverage data.
Add an extra field to proc_dynamic, An array of coverage counts wrapped by
a maybe type. It's indexed the same as the array of coverage infos in
proc_static. This array is present if dynamic coverage profiling is being
done (the default).
Modify the coverage_points field in proc static, now there are two fields,
an array of coverage_point_info values which store compile-time data. And
an optional array of coverage points (present if static coverage profiling
was performed).
Updated the formatting of the proc static structure.
Moved the coverage_point type to coverage.m.
Created a new type, coverage_data_type which enumerates the possibilities
for coverage profiling: none, static and dynamic.
deep_profiler/coverage.m:
Move the coverage point type here from profile.m, as the profile data
structure no longer refers to it directly.
Create a predicate coverage_point_arrays_to_list/3 which merges coverage
point information and the counts themselves into coverage points. This can
be used to construct a list of coverage points regardless of whether static
or dynamic coverage points are being used.
deep_profiler/read_profile.m:
Conform to changes in runtime/mercury_deep_profiling.c.
Refactored reading of the file header, a new named predicate is now used
rather than a lambda expression.
Incremented the Deep.data version number.
deep_profiler/report.m:
Updated the proc dynamic dump report structure to include a list of
coverage points.
deep_profiler/create_report.m:
deep_profiler/display_report.m:
Conform to changes in profile.m.
The proc dynamic dump now shows coverage information that was contained in
that proc dynamic.
deep_profiler/canonical.m:
deep_profiler/dump.m:
Conform to changes in profile.m.
deep_profiler/io_combinator.m:
Add a 13-arg version of maybe_error_sequence.
deep_profiler/Mercury.options:
Documented another trace flag.
|
||
|
Paul Bone
|
feadc37bd4 |
Modify the program representation bytecode to include mode information for the
Estimated hours taken: 3
Branches: main
Modify the program representation bytecode to include mode information for the
head variables of a procedure.
Modes in the program representation bytecode and structure are represented as
an initial instantiation state, and a final instantiation state. These can
have one of the three values each: free, ground and other.
mdbcomp/program_representation.m:
Introduce new types head_var_rep, var_mode_rep and inst_rep.
Head variables of a proc_defn_rep structure now use the head_var_rep type.
Add head_var_to_var function.
Add code to read in mode information with head variables from progrep
bytecode.
Increment the Deep.procrep file format version number.
Use a higher order abstraction to remove duplicated code.
compiler/prog_rep.m:
Write out bytecodes for the modes of procedures' head variables.
deep_profiler/program_representation_utils.m:
Conform to changes in mdbcomp/program_representation.m
Print out the mode information for each variable in the head of a
procedure.
Use a higher order abstraction to remove duplicated code.
browser/declarative_tree.m:
Conform to changes in mdbcomp/program_representation.m
runtime/mercury_deep_profiling.c:
Increment the Deep.procrep file format version number.
|
||
|
Paul Bone
|
7a86a73ef2 |
Alter the program representation bytecode used by the the declartive debugger
Estimated hours taken: 0.5
Branches: main
Alter the program representation bytecode used by the the declartive debugger
and deep profiler. Modify the representation of a switch goal to include
information about whether it is complete.
mdbcomp/program_representation.m:
Create the new type 'switch_can_fail_rep'.
Include a field in the switch_rep alternative of the goal_expr_rep type.
Process an extra byte to give the completeness of a switch when reading in
switch goals.
Increment procrep file version number.
compiler/prog_rep.m:
Write out an extra byte for switches describing if the switch is complete.
runtime/mercury_deep_profiling.c:
Increment progrep file version number.
browser/declarative_tree.m:
deep_profiler/program_representation_utils.m:
tests/debugger/declarative/builtin_call_rep.exp:
tests/debugger/declarative/dependency.exp:
Conform to changes in mdbcomp/program_representation.m
|
||
|
Paul Bone
|
1de6e0747a |
Modify the bytecode used by the deep profiler and declarative debugger to
Estimated hours taken: Branches: main Modify the bytecode used by the deep profiler and declarative debugger to include the determinism of each goal, not just atomic goals. This increases the size of the compiler's bytecode by 0.8% and the compiler's bytecode by 0.1%. mdbcomp/program_representation.m: Introduce field names for members of the proc_defn_rep structure. Introduce determinism field to proc_defn_rep structure. Renamed goal_rep type to goal_expr_rep, created new goal_rep type containing a goal_expr_rep and a detism_rep. Removed detism_rep field from atomic_goal_rep function symbol. Read determinism information after each goal, for that goal. No-longer read determinism information within atomic goals. Incremented procrep bytecode version number. compiler/prog_rep.m: represent_proc_as_bytecodes now takes an extra argument, the determinism of the procedure that it's creating bytecode for. Now writes determinism bytecode after each goal, rather than just for atomic goals. compiler/stack_layout.m: Call represent_proc_as_bytecodes with the determinism or the procedure. runtime/mercury_deep_profiler.m: Incremented procrep bytecode version number. browser/declarative_tree.m: Conform to changes in mdbcomp/program_representation.m. Modify some code to use record syntax so it's not impacted in the future by unrelated changes to these program_representation data structures. deep_profiler/mdprof_procrep.m: Print determinism information for each procedure and most goals. Determinisms of conjunctions and negations are not printed to avoid cluttering the output. Often the determinism of these goals can be determined by inspecting the inner goal(s). Conform to changes in mdbcomp/program_representation.m |
||
|
Paul Bone
|
1de09830dc |
Deep profiling and program representation file format changes.
Estimated hours taken: 7.5 Branches: main Deep profiling and program representation file format changes. Include a table of variable names in the program representation used by the deep profiler (Deep.procrep). The table shouldn't contain variables introduced by the compiler. Include the name of the program from which the deep profile was generated in the header of the deep profiling data file (Deep.data). compiler/prog_rep.m: Optionally create the variable table within the bytecode generated for each procedure, The actual variable names are stored in the string table for the module. compiler/stack_layout.m: Enable the variable table only for deep profiling builds. mdbcomp/program_representation.m: Include variable table in each program representation structure. Create predicates to read in the variable table if it is present. Create predicates to lookup a variable name from a table or generate a name automatically. Incremented version number for the program representation file format. runtime/mercury_deep_profiling.c: Write out program name in header of deep profiling data. Increment version numbers of deep profiling and program representation file formats. browser/declarative_tree.m: deep_profiler/canonical.m: tests/debugger/declarative/builtin_call_rep.exp: tests/debugger/declarative/dependency.exp: Conform to changes in mdbcomp/program_representation.m deep_profiler/mdprof_procrep.m: Lookup and display variable names when displaying program representation. deep_profiler/profile.m: Make program name part of the profile_stats structure. deep_profiler/read_profile.m: deep_profiler/io_combinator.m: Read in program name from deep profiling data. Incremented version number for the deep profiling file format. deep_profiler/report.m: deep_profiler/create_report.m: deep_profiler/display_report.m: Display program name with the statistics on the menu report. deep_profiler/dump.m Conform to changes in deep_profiler/profile.m. |
||
|
Paul Bone
|
7593b61b70 |
Introduce coverage profiling. While regular profiling shows which procedures
Estimated Hours Taken: 100
Branches: main
Introduce coverage profiling. While regular profiling shows which procedures
are used the most, coverage profiling goes further, and also shows the most
common execution paths through each procedure. We intend coverage profiling
data to be used by a future automatic parallelization pass in the compiler.
However, it should also be useful for other purposes.
This diff adds a compiler option, --coverage-profiling, that, when specified,
adds program instrumentation to record execution counts at selected points
in procedure bodies. The implementation currently stores coverage profiling
information in ProcStatic structures. We will later investigate the impact
of storing this information in ProcDynamic structures instead.
For now coverage statistics can be viewed with the mdprof_dump tool.
compiler/deep_profiling.m:
Introduced coverage profiling transformation after deep profiling
transformation, it will be run if at least one coverage point type is
enabled.
compiler/goal_util.m:
Created create_conj_from_list/3, to create a conjunction from a list of
goals.
compiler/hlds_goal.m:
Added dp_goal_info structure that stores information relevant to the deep
profiling pass.
Added egi_maybe_dp field to extra_goal_info to store a dp_goal_info
structure when required by the deep profiler.
compiler/hlds_pred.m:
Added a list of coverage_point_info structures to the hlds_proc_static
structure.
compiler/hlds_out.m:
Added the ability to dump the dp_goal_info structure when the correct
option is given.
compiler/layout.m:
Added extra layout structures to store coverage point static and dynamic
data for each procedure.
compiler/layout_out.m:
Added code to write out new layout structures in the Low Level C
Backend.
Added code to write out references to coverage point data from the proc
static structures.
Conform to changes in layout.m.
compiler/opt_debug.m:
Conform to changes in layout.m,
compiler/options.m:
Added command line parameters to enable coverage profiling and different
coverage points, as well as options that can cause coverage points not
to be inserted in some circumstances.
compiler/handle_options.m:
Added hlds dump options to the 'all' aliases for dumping the new
dp_goal_info structure in the hlds_info.
deep_profiler/dump.m:
Modified to dump coverage profiling data read in from Deep.data file.
deep_profiler/profile.m:
Added coverage points to proc static structure.
deep_profiler/read_profile.m:
Incremented Deep.data format version number.
Read coverage points from Deep.data file.
Confirm to changes in profile.m.
library/profiling_builtin.m:
Added real declaration and dummy implementation of
increment_coverage_point_count/2. This represents the instrumentation
introduced by the coverage profiling transformation.
mdbcomp/program_representation.m:
Added types to support coverage profiling, including a foreign_enum for
cp_type.
Added coverage_point_type_c_value to convert a coverage point type to a
string representing it's C value.
runtime/mercury_deep_profiling.c:
Incremented Deep.data format version.
Implemented writing out of coverage points to the Deep.data file.
runtime/mercury_deep_profiling.h:
Modified runtime structures to support storing coverage point
information in proc static structures.
doc/user_guide.texi:
Documented coverage profiling options, however this is commented out since
it's experimental.
|
||
|
Zoltan Somogyi
|
9e1c06af71 |
Fix a bug that prevented the compiler from bootstrapping in deep profiling
Estimated hours taken: 6 Branches: main Fix a bug that prevented the compiler from bootstrapping in deep profiling grades. The symptom was the violation of a sanity check in mercury_deep_profiling.c, which lead to this message: Mercury runtime: UNWRITTEN nodes: Deep.data file corrupted Mmake.workspace: Don't include the ssdb library's init file unconditionally in every _init.c file we generate; instead include the ssdb library's init file only in ssdb grades. The reason why this is needed is that the referenced but unwritten nodes were the proc layouts of three predicates in the browser directory. They were not written because the browser library's init file is (correctly) included in the compiler's _init.c file only if the browser library is linked into the compiler, i.e. only if the compiler is compiled in a debug grade. They were referenced by call site static structures from ssdb/ssdb.m, whose init file was (incorrectly) being included in the compiler's _init.c file even if the compiler wasn't being compiled in an ssdb grade. scripts/parse_ml_options.sh-subr.in: Add the option needed by Mmake.workspace to add an init file only in an ssdb grade: --ssdb-init-file. scripts/c2init.in: Act on the value of that option. Delete a reference to $TRACE_INIT_FILES, since it should always be empty. Document a similar problem with $EXTRA_INIT_FILES. Mmakefile: Improve formatting. compiler/layout_out.m: Avoid inconsistent indentation in the generated .c code. tools/bootcheck: Add two options that helped track down the bug: --deep-all-write, which causes bootcheck to write out *every*deep profiling data file, and --deep-debug, which switches on the already existing deep profiling debug support. runtime/mercury_deep_profiling.c: Put the two lines of the error message we generate for unwritten nodes if deep profiling's debug support is enabled in the logical order. |
||
|
Julien Fischer
|
9d743da106 |
Avoid integer overflow on 64-bit machines in the runtime code that writes
Estimated hours taken: 0.1 Branches: main Avoid integer overflow on 64-bit machines in the runtime code that writes out deep profiles. runtime/mercury_deep_profiling.c: Update some assertions to work correctly on 64-bit machines. |
||
|
Zoltan Somogyi
|
2dc982cfe4 |
Make a representation of the program available to the deep profiler.
Estimated hours taken: 50 Branches: main Make a representation of the program available to the deep profiler. We do this by letting the user request, via the option "--deep-procrep-file" in MERCURY_OPTIONS, that when the Deep.data file is written, a Deep.procrep file should be written alongside it. The intended use of this information is the discovery of profitable parallelism. When a conjunction contains two expensive calls, e.g. p(...) and q(...) connected by some shared variables, the potential gain from executing them in parallel is limited by how early p produces those variables and how late q consumes them, and knowing this requires access to the code of p and q. Since the debugger and the deep profiler both need access to program representations, put the relevant data structures and the operations on them in mdbcomp. The data structures are significantly expanded, since the deep profiler deals with the whole program, while the debugger was interested only in one procedure at a time. The layout structures have to change as well. In a previous change, I changed proc layout structures to make room for the procedure representation even in non-debugging grades, but this isn't enough, since the procedure representation refers to the module's string table. This diff therefore makes some parts of the module layout structure, including of course the string table, also available in non-debugging grades. configure.in: Check whether the installed compiler can process switches on foreign enums correctly, since this diff depends on that. runtime/mercury_stack_layout.[ch]: runtime/mercury_types.h: Add a new structure, MR_ModuleCommonLayout, that holds the part of the module layout that is common to deep profiling and debugging. runtime/mercury_deep_profiling.[ch]: The old "deep profiling token" enum type was error prone, since at each point in the data file, only a subset was applicable. This diff breaks up the this enum into several enums, each consisting of the choice applicable at a given point. This also allows some of the resulting enums to be used in procrep files. Rename some enums and functions to avoid ambiguities, and in one case to conform to our naming scheme. Make write_out_proc_statics take a second argument. This is a FILE * that (if not NULL) asks write_out_proc_statics to write the representation of the current module to specified stream. These module representations go into the middle part of the program representation file. Add functions to write out the prologue and epilogue of this file. Write out procedure representations if this is requested. Factor out some code that is now used in more than one place. runtime/mercury_deep_profiling_hand.h: Conform to the changes to mercury_deep_profiling.h. runtime/mercury_builtin_types.c: Pass the extra argument in the argument lists of invocations of write_out_proc_statics. runtime/mercury_trace_base.[ch]: Conform to the name change from proc_rep to proc_defn_rep in mdbcomp. runtime/mercury_grade.h: Due to the change to layout structures, increment the binary compatibility version numbers for both debug and deep profiling grades. runtime/mercury_wrapper.[ch]: Provide two new MERCURY_OPTION options. The first --deep-procrep-file, allows the user to ask for the program representation to be generated. The second, --deep-random-write, allows tools/bootcheck to request that only a fraction of all program invocations should generate any deep profiling output. The first option will be documented once it is tested much more fully. The second option is deliberately not documented. Update the type of the variable that holds the address of the (mkinit-generated) write_out_proc_statics function to accept the second argument. util/mkinit.c: Pass the extra argument in the argument list of write_out_proc_statics. mdbcomp/program_representation.m: Extend the existing data structures for representing a procedure body to represent a procedure (complete with name), a module and a program. The name is implemented as string_proc_label, a form of proc_label that can be written out to files. This replaces the old proc_id type the deep profiler. Extend the representation of switches to record the identity of the variable being switched on, and the cons_ids of the arms. Without the former, we cannot be sure when a variable is first used, and the latter is needed for meaningful prettyprinting of procedure bodies. Add code for reading in files of bytecodes, and for making sense of the bytecodes themselves. (It is this code that uses foreign enums.) mdbcomp/prim_data.m: Note the relationship of proc_label with string_proc_label. mdbcomp/rtti_access.m: Add the access operations needed to find module string tables with the new organization of layout structures. Provide operations on bytecodes and string tables generally. trace/mercury_trace_cmd_browsing.c: Conform to the change to mdbcomp/program_representation.m. compiler/layout.m: Add support for a MR_ModuleCommonLayout. Rename some function symbols to avoid ambiguities. compiler/layout_out.m: Handle the new structure. compiler/stack_layout.m: Generate the new structure and the procedure representation bytecode in deep profiling grades. compiler/llds_out.m: Generate the code required to write out the prologue and epilogue of program representation files. Pass the extra argument in the argument lists of invocations of write_out_proc_statics that tells those invocations to write out the module representations between the prologue and the epilogue. compiler/prog_rep.m: When generating bytecodes, include the new information for switches. compiler/continuation_info.m: Replace a bool with a more expressive type. compiler/proc_rep.m: Conform to the change to continuation_info.m. compiler/opt_debug.m: Conform to the change to layout.m. deep_profiler/mdprof_procrep.m: A new test program to test the reading of program representations. deep_profiler/DEEP_FLAGS.in: deep_profiler/Mmakefile: Copy the contents of the mdbcomp module to this directory on demand, instead of linking to it. This is necessary now that the deep profiler depends directly on mdbcomp even if it is compiled in a non-debugging grade. The arrangements for doing this were copied from the slice directory, which has long done the same. Avoid a duplicate include of Mmake.deep.params. Add the new test program to the list of programs in this directory. Mmakefile: Go through deep_profiler/Mmakefile when deciding whether to do "mmake depend" in the deep_profiler directory. The old actions won't work correctly now that we need to copy some files from mdbcomp before we can run "mmake depend". deep_profiler/profile.m: Remove the code that was moved (in cleaned-up form) to mdbcomp. deep_profiler/dump.m: deep_profiler/profile.m: Conform to the changes above. browser/declarative_execution.m: browser/declarative_tree.m: Conform to the changes in mdbcomp. doc/user_guide.texi: Add commented out documentation of the two new options. slice/Mmakefile: Fix formatting, and a bug. library/exception.m: library/par_builtin.m: library/thread.m: library/thread.semaphore.m: Update all the handwritten modules to pass the extra argument now required by write_out_proc_statics. tests/debugger/declarative/dependency.exp: Conform to the change from proc_rep to proc_defn_rep. tools/bootcheck: Write out deep profiling data only from every 25th invocation, since otherwise the time for a bootcheck takes six times as long in deep profiling grades than in asm_fast.gc. However, do test the ability to write out program representations. Use the mkinit from the workspace, not the installed one. Don't disable line wrapping. |
||
|
Zoltan Somogyi
|
455e1eea75 |
The runtime had two different conventions for naming types.
Estimated hours taken: 2 Branches: main The runtime had two different conventions for naming types. One convention, used mostly in the debugger-related modules, added underscores between capitalized words; example: MR_Label_Layout. The other convention, used in most modules, used capitalized words without underscores (e.g. MR_TypeInfo). This diff standardizes on the second convention. It has no algorithmic changes, only renames of types. runtime/*.[ch]: trace/*.[ch]: compiler/*.m: library/*.m: mdbcomp/*.m: Effect the change described above. The only substantive change is that runtime/mercury_stack_layout.h used to define *two* types for trace levels: MR_TraceLevel and MR_Trace_Level, and this diff standardizes on just one (they had equivalent definitions). runtime/mercury_bootstrap.h: Add a #define from the old name to the new for all the changed type names that the installed compiler can put into .c files. We can delete these #defines some time after this diff has bootstrapped. slice/.mgnuc_opts: Restore the --no-mercury-stdlib-dir option, without which the slice directory won't compile after this change (because it looks for type names in the installed runtime header files, which define the old versions of type names). |
||
|
Zoltan Somogyi
|
2c7289a290 |
Add conditionally compiled code that can be used to generate a machine
Estimated hours taken: 2 Branches: main runtime/mercury_deep_call_port_body.h: runtime/mercury_deep_leave_port_body.h: runtime/mercury_deep_redo_port_body.h: Add conditionally compiled code that can be used to generate a machine processable log of the updates to the deep profiling tree. Convert to four-space indentation. runtime/mercury_deep_profiling.[ch]: Add support for the new code in the port bodies, and add a mechanism for writing out proc statics and call sites statics at the start of the same machine processable log. runtime/mercury_wrapper.c: Switch on the new debugging mechanism if the --deep-log-file or --deep-log-prog option is given in MERCURY_OPTIONS. runtime/mercury_conf_param.h: Document the macro controlling the new logging mechanism. |
||
|
Zoltan Somogyi
|
972d97c44f |
Fix three bugs in the runtime system's support for deep profiling.
Estimated hours taken: 8 Branches: main Fix three bugs in the runtime system's support for deep profiling. runtime/mercury_heap.h: One bug was that information about memory allocations wasn't being collected. The problem was that code fragment that incremented the relevant fields of the current call site dynamic structure was being invoked from a set of macros that the compiler doesn't emit in deep profiling grades. The fix is to invoke that code fragment from the basic macros that all the others in mercury_heap.h are built upon. Apply the same fix to the code fragment that gathers information for complexity analysis. To prevent such problems from arising in the future, document the layers of macros defined in this module. Move some macros around to yield a consistent structure (to make the documentation easier). Convert to four-space indentation. runtime/mercury_deep_*_port_body.h: library/exception.m: The second bug was that we collected the wrong sort of call sequence number information. We tried to define the call sequence number field (MR_own_call_seqs) of call dynamic structures to make them contain, for each call, a number proportional to the time from the start of the call to the end of the call. The problems with this is that all the other cost measure fields of the call site structure (time quanta, number of memory allocations, number of words allocated) were non-cumulative; e.g. the time quanta field was incremented only when a clock interrupt happened inside the code of the called predicate, not in its descendants. Propagating the costs of descendants to the ancestors is the deep profiler's job. By including the cost of descendants in the MR_own_call_seqs field, the propagation algorithm of the deep profiler actually screws up the call sequence number information, by including the cost of descendants twice. We could modify the propagation algorithm to avoid double-propagating call sequence number information, but it would be a significant coding job. The much simpler fix is to treat the increment of the call sequence number the same way we handle a profiling interrupt: we increment the relevant field of the current call site dynamic structure. runtime/mercury_deep_profiling.h: The third bug was that we were using the mask value of 0x10 to signify both the presence of a nonzero exception count and a nonzero number of allocations count. The former was a typo. Change the data format number, since the meaning of the call sequence number field has changed. runtime/mercury_context.h: Add a missing MR_ prefix in a comment. runtime/mercury_heap_profile.h: Fix some formatting. |
||
|
Zoltan Somogyi
|
35df0c4944 |
Include the count of call sequence numbers in the Deep.data file, since the
Estimated hours taken: 1 Branches: main Include the count of call sequence numbers in the Deep.data file, since the deep profiler needs to know this to generate percentages. runtime/mercury_deep_profiling.c: Write the count of call sequence numbers to the Deep.data file. deep_profiler/profile.m: Add a field to the profile for holding the count of call sequence numbers. deep_profiler/read_profile.m: Read in the count of call sequence numbers from the Deep.data file. deep_profiler/io_combinator.m: Add some functions now required by read_profile.m. Convert from DCGs to state variables. deep_profiler/canonical.m: deep_profiler/dump.m: Conform to the change to profile.m. |
||
|
Zoltan Somogyi
|
5343a6f616 |
Fix omission in my previous diff adding call sequence numbers: actually
Estimated hours taken: 0.1 Branches: main runtime/mercury_deep_profiling.c: Fix omission in my previous diff adding call sequence numbers: actually write them out. |
||
|
Zoltan Somogyi
|
e8415df212 |
Enhance the deep profiler by including in the output a measure of execution
Estimated hours taken: 0.5
Branches: main
Enhance the deep profiler by including in the output a measure of execution
time that has much finer granularity than clock ticks: a count of procedure
calls. The deep profiler does not yet display this data; that is for a later
diff.
runtime/mercury_deep_profiling.[ch]:
Add a field containing this counter if the appropriate macro is
defined. Initialize the new field when the profiling metrics structure
it is in is initialized.
Use masks that fit into one byte for the commonly needed fields
of the profiling metrics structure.
runtime/mercury_conf_params.h:
Define this macro in deep profiling grades unless its absence is
expressly requested (for performance tests of the deep profiler
itself).
runtime/mercury_grade.h:
Increment the binary compatibility number for the deep profiling
grades.
runtime/mercury_deep_{call,redo,leave}_port_body.h:
Update the call sequence number global at call sites.
Update the new field when control enters a procedure invocation
through the call or redo ports, and when it leaves the invocation
through the exit or fail ports.
library/exception.m:
Update the new field when control leaves a procedure invocation
through the excp port.
deep_profiler/measurements.m:
Include the new field in the deep profiler's version of the
profiling metric structure.
deep_profiler/read_profile.m:
Read in the updated format now generated by the runtime.
deep_profiler/dump.m:
Include the new field in any dumps of the deep profiler's
internal data structures.
|
||
|
Julien Fischer
|
fb7abe9e6d |
Conform to the recent change in the definition of MR_Long_Lval.
Estimated hours taken: 0.1 Branches: main runtime/mercury_deep_profiling.c: Conform to the recent change in the definition of MR_Long_Lval. Fix the initializer for MR_main_parent_proc_layout: some of the components were in the wrong order. |
||
|
Julien Fischer
|
0898dab6d9 |
Avoid a C compiler warning on x86_64.
Estimated hours taken: 0.1 Branches: main Avoid a C compiler warning on x86_64. runtime/mercury_deep_profiling.c: In the MR_hash_ptr macro, cast `void * ' to `MR_Unsigned' rather than `unsigned int'. |
||
|
Julien Fischer
|
28125db92a |
Fix a bug that causes general/read_line_as_string to seg fault
Estimated hours taken: 9 Branches: main, release runtime/mercury_deep_profiling.c: Fix a bug that causes general/read_line_as_string to seg fault in the .profdeep grades. The problem was that the slots in the hash tables were not being initialised to NULL. The above test case triggered this problem because the start of the MR_call_site_dynamic_table was reusing memory that io.read_line_as_string_2/6 had previously used as a temporary buffer. Consequently, the initial values of many of the slots in the hash table contained rubbish. The code that writes out the Deep.data treated these junk values as pointers. It seg faulted because these "pointers" were to memory that was inaccessible. library/io.m: library/string.m: A little bit of preventative maintenance: s/memcpy/MR_memcpy/ |
||
|
Julien Fischer
|
b1d348845c |
Fix the typo that has been causing the runtime to abort
Estimated hours taken: 6 Branches: main, release runtime/mercury_deep_profiling.c: Fix the typo that has been causing the runtime to abort when writing out the Deep.data files. |
||
|
Zoltan Somogyi
|
9d909aa8df |
Fix two bugs in the code that writes out Deep.data.
Estimated hours taken: at least 16, over some months Branches: main, release runtime/mercury_deep_profiling.c: Fix two bugs in the code that writes out Deep.data. This required improving the infrastructure for debugging this module. The first bug involved the management of the table that mapped the addresses of proc_layout structures (which point to proc statics) to numerical ids. Of the three places that inserted into this table, one inserted the address of the proc static structure directly, leaving dangling proc static ids (ids that don't refer to anything) in the generated Deep.data file. This caused mdprof to abort when processing the Deep.data file. To make such bugs less likely in the future, provide type-safe interface functions to the main functions for manipulating hash tables. The second bug was effectively invisible: the "written" flag wasn't set on call site dynamic structures as they were written out. Convert the module to four-space indentation to reduce the number of bad line breaks. |
||
|
Zoltan Somogyi
|
642c902c90 |
Reduce the sizes of .c and .o files in debug grades by 8%.
Estimated hours taken: 4
Branches: main
Reduce the sizes of .c and .o files in debug grades by 8%.
Add to mdb the capability of putting breakpoints on unify, compare and index
predicates.
compiler/code_gen.m:
Since compiler-generated unify, index and compare pred can be presumed
to be correct, compile them with shallow tracing instead of deep
tracing even if the user asks for deep tracing of the rest of the
module. This saves disk space, compile time and run time.
Move the code for setting the trace level to generate_proc_code,
where it is executed whether we generate code by predicates or by
phases.
compiler/trace_params.m:
Provide a function for use by code_gen.m.
Consider unify and compare predicates to behave as if their address
were taken, since they may be called from builtin.unify or
builtin.compare.
compiler/jumpopt.m:
Do not replace jumps with the block being jumped to if the block
contains an instruction that represents a call to the debugger.
In such cases, the jump is a minor cost in runtime compared to the
call, and the locality benefit from not increasing the code size is
quite likely to exceed that cost.
compiler/llds.m:
Add field names to pragma_c instructions to make the change to jumpopt
easier.
tests/debugger/uci.{inp,exp}:
Update this test case. An update is required since we now generate
fewer events from unify and compare preds due to their now being
shallow traced.
To make this the update easier, use the new capability of putting
breakpoints on unify and compare predicates.
trace/mercury_trace_tables.[ch]:
Expand procedure specifications to allow the specification of unify,
compare and index predicates, and expand the function that checks for
matches between a procedure layout and a procedure specification
accordingly.
Fix two instances of a bug: if a procedure specification has no name,
represent it as NULL, not as a non-NULL pointer to a null character,
since the match function expects only the former.
doc/user_guide.texi:
Document the new definition of procedure specifications.
NEWS:
Mention the new definition of procedure specifications, and expand
on some previous additions to the debugger.
runtime/mercury_stack_trace.c:
Allow mdb breakpoint specifications on uci predicates to be saved.
Factor out some common data structure accesses.
runtime/mercury_proc_id.h:
Change the name of a macro from MR_PROC_ID_COMPILER_GENERATED to
MR_PROC_ID_IS_UCI, for consistency with other names dealing with
unify, compare and index predicates.
runtime/*.[ch]:
trace/*.[ch]:
Conform to the change in mercury_proc_id.h.
|
||
|
Zoltan Somogyi
|
e0c212359f |
Fix problems in white space in the commit of the fix of the deep profiler.
Estimated hours taken: 0.2 Branches: main Fix problems in white space in the commit of the fix of the deep profiler. These were caused by the fact that the workspace in which the original commit took place was created by applying a patch to a fresh workspace, and the patch was created with diff -b. |
||
|
Zoltan Somogyi
|
c80d143cc8 |
The following change is only 98% complete, not 100%.
Estimated hours taken: 120
Branches: main
The following change is only 98% complete, not 100%. I am committing it in
this state because (1) we pass many more test cases in deep profiling grade
with it than without it, and (2) the double maintanance involved in fixing
CVS conflicts is preventing me from doing the last 2%.
Get the deep profiler to work for code that sets up to catch exceptions,
which for the last year or more has included the compiler, and to get it
almost working for code that actually catching exceptions.
The basic problem is that code that throws exceptions will in general cause
several calls to "return" without executing the exit or fail port codes that
the deep profiling transformation inserted into their bodies, and this leaves
the data structure being built by the deep profiling inconsistent. The solution
uses the same approach as we have adopted for the debugger: have the code that
handles the throwing of exceptions simulate a return from each call between
the throw and the catch, updating the deep profiling data structures as needed.
This requires us to be able to walk the stack at runtime not just in debugging
grades but also in deep profiling grades. Since the debugger already has the
mechanisms required for this, we reuse them. The procedure layouts used by the
debugger were designed to have three segments: the procedure stack walk
information, the procedure id, and the execution tracing information. We now
modify this design to make the third segment contain two pointers: to the
execution tracing information (for use by the debugger), and to the procedure's
proc_static structure (for use by deep profiling). Each pointer will be null
unless the pointed-to structure is required by compile-time options.
This common use by the debugger and by deep profiling of the stack-walk
structure and the procedure id structure (which deep profiling used to
generate independently and possibly redundantly) required some rearrangement
of the compiler's version of these data structures.
To make this rearrangement simpler, this diff removes a capability that
we theoretically supported but never used: turning on stack traces without
turning on execution tracing and vice versa. After this diff, stack tracing
is enabled if and only if either execution tracing or deep profiling is
enabled.
The diff also includes improvements in the debugging infrastructure for
debugging deep profiling, which were necessary for the implementation of the
rest of the changes.
compiler/deep_profiling.m:
The code in exception.m needs to know the locations of the variables
that we would pass to the exit or fail port code, so it can simulate
leaving the procedure invocation through the exception port. Without
this information, throwing an exception leaves the deep profiling
data structures of the procedure invocations between throw and catch
in an inconsistent state.
Deep_profiling.m creates these variables, but it doesn't know where
they will be at runtime, so it records their identities; the code
generator will allocate them stack slots and record the numbers of
these stack slots for placement in the now expanded proc layout
structures. Deep profiling used to generate static data structures
separately from the HLDS, but since the code generator now needs
access to them, we store their information in proc_infos in the HLDS.
Instead of passing the addresses of proc_static structures to the deep
profiling port procedures, pass the address of proc_layout structures,
since the information about the identities of procedures are now stored
not in the proc_static structure, but in the proc_layout structure
that points to the proc_static structure.
compiler/hlds_pred.m:
compiler/layout.m:
Move the definitions of the static data structures generated by deep
profiling from layout.m to hlds_pred.m, to allow deep_profiling.m
to store them in proc_infos.
compiler/hlds_pred.m:
compiler/rtti.m:
Move the definition of rtti_proc_label from rtti.m to hlds_pred.m,
since some of the new data structures in hlds_pred.m need it. Despite
its name, the rtti_proc_label type doesn't contain any info that
doesn't belong in the HLDS.
Add some information to the rtti_proc_label type that is now needed
by deep profiling, e.g. record determinisms instead of just code
models. Record explicitly the outcome of some tests that used to be
duplicated in more than one place in the compiler, e.g. for whether
the procedure (as opposed to the predicate) is imported. Change some
of the field names to be more precise about the field's meaning.
compiler/code_gen.m:
Transmit the contents of the deep profiling data structures stored in
the proc_info by deep_profiling.m to continuation_info.m, together
with the layout structures created for execution tracing and the
identities of the variables needed for handling exceptions,
when code generation for a procedure is complete.
After the goal that generates these variables, save them to stack
for use by the exception handler.
compiler/hlds_goal.m:
Add a feature to mark the goal that generates the deep profiling
variables needed by the exception handler.
compiler/hlds_llds.m:
Add a utility predicate for new code in code_gen.m
compiler/continuation_info.m:
Hold the deep profiling information computed by code_gen.m for use by
stack_layout.m.
compiler/layout.m:
compiler/layout_out.m:
Update the definitions of the data structures describing procedure
layouts, and the code writing them out, to reflect the use of some
parts of procedure layouts by deep profiling as well as debugging.
Change the layout structures generated by deep profiling to use
rtti_proc_labels, which are backend independent, instead of
proc_labels, which are specific to the LLDS backend.
Conform to the changes in runtime/mercury_stack_layout.h.
compiler/stack_layout.m:
Generate the updated version of proc_layout structures.
compiler/mercury_compile.m:
compiler/global_data.m:
Conform to the fact that deep profiling no longer generates layout
structures separate from proc_infos.
compiler/llds_out.m:
Register proc_layout structures instead of proc_static structures
for use by runtime/mercury_deep_profiling.c.
compiler/options.m:
compiler/handle_options.m:
Rename the require_tracing option as exec_trace, since this more
directly reflects its meaning.
Instead of having --debug set both require_tracing and stack_trace,
make it set (be the user-visible name of) just exec_trace;
the value of stack_trace is implied.
Turn off the specialization of deep profiling for self-tail-recursive
procedures for now. Due to the changes made by this diff in the data
structures involved in debugging, it cannot be debugged until this
change has been installed. Handling the full language is more important
than a specialization that reduces only stack space overheads, not
runtime overheads.
compiler/compile_target_code.m:
Conform to the changes in options.m and runtime/mercury_grade.h.
compiler/hlds_data.m:
Replace the deep_profiling_proc_static cons_id, and its associated tag,
to deep_profiling_proc_layout, since we now generate addresses of proc
layout structures, not of proc_static structures.
compiler/code_util.m:
Simplify some code based on the new info in rtti_proc_labels.
compiler/bytecode_gen.m:
compiler/dependency_graph.m:
compiler/higher_order.m:
compiler/hlds_out.m:
compiler/mercury_to_mercury.m:
compiler/ml_code_util.m:
compiler/ml_unify_gen.m:
compiler/opt_debug.m:
compiler/proc_label.m:
compiler/prog_rep.m:
compiler/rl_exprn.m:
compiler/rtti_out.m:
compiler/rtti_to_mlds.m:
compiler/saved_vars.m:
compiler/switch_util.m:
compiler/unify_gen.m:
Minor changes to conform to the change from deep_profiling_proc_static
to deep_profiling_proc_layout, to the change in the structure of
rtti_proc_labels, to the changes in types of layout.m, and/or to the
new goal feature.
deep_profiler/measurements.m:
Reserve space for exception counts.
deep_profiler/html_format.m:
Add a column for exception counts.
deep_profiler/profile.m:
deep_profiler/read_profile.m:
Rename the data structures referring to compiler generated unify,
compare and index predicates to avoid misleading names: they are
not the only compiler generated predicates.
deep_profiler/read_profile.m:
runtime/mercury_deep_profiling.c:
Update the string that identifies deep profiling data files.
This is necessary because the format has changed: it now includes
information about exception port counts.
library/exception.m:
In deep profiling grades, execute the exception port code for every
procedure invocation between a throw and a catch, using the procedure
layout structures now generated by the compiler for every procedure.
Rename the function involved to reflect its new, more general purpose.
Update the definitions of the hand-written proc_static and proc_layout
structures for the procedures implemented via hand-written C code.
Indent C preprocessor directives and foreign_procs according to our
coding standards.
library/profiling_builtin.m:
Change the parameters of the call port code procedures from proc_static
to proc_layout. Reach the proc_static structure from the proc_layout
structure when needed. Include the proc_layout structure in any
messages from assertion failures.
Add some conditionally compiled debugging code.
Give some variables better names.
runtime/mercury_type_info.h:
runtime/mercury_builtin_types.c:
Move the macros required to create the proc_static structures
of unify and compare predicates from mercury_type_info.h
to mercury_builtin_types.c, since the latter is the only file
that needs them.
Use the same macros for creating the proc_static structures
of hand-written unify, compare and compare_reprentation predicates
as for user defined predicates. This required changing their naming
scheme.
runtime/mercury_unify_compare_body.h:
Conform to the new naming scheme.
runtime/mercury_ho_call.c:
Provide the mechanism for mercury_unify_compare_body.h to conform
to the new naming scheme.
Remove the definitions of the proc_static structures for
hand-written unify, compare and compare_reprentation predicates,
since these now have to be defined together with the corresponding
proc_layout structures in mercury_builtin_types.c.
runtime/mercury_builtin_types.[ch]:
Update the definitions of the hand-written proc_static and proc_layout
structures for the procedures implemented via hand-written C code,
and add the required declarations first.
Handle deep profiling of compare_representation as well as unify
and compare predicates on builtin types.
Handle deep profiling of compare_representation on user-defined types,
since this is done entirely in the runtime, not by compiler generated
predicates.
runtime/mercury_builtin_types_proc_layouts.h:
New header file containing the declarations of the proc layout
structures of the unify, compare and index predicates of builtin types.
Logically, these declarations belong in mercury_builtin_types.h,
but putting them there causes problems for the linker; the details
are explained in the file itself.
runtime/Mmakefile:
Add the new header file.
runtime/mercury_minimal_model.[ch]:
Update the definitions of the hand-written proc_static and proc_layout
structures for the procedures implemented via hand-written C code,
and add the required declarations first.
runtime/mercury_grade.h:
Replace the MR_REQUIRE_TRACING grade option with MR_EXEC_TRACING.
Besides being better named, the MR_EXEC_TRACING option implies
MR_STACK_TRACE.
Besides the overall binary compatibility version number, add subsidiary
version numbers for binary compatibility in deep profiling and
debugging grades. These will make it easier to bootstrap changes
(such as this) that affect binary compatibility only in such grades.
runtime/mercury_trace_base.c:
trace/mercury_trace.c:
Conform to the new names of the configuration parameters.
runtime/mercury_hand_compare_body.h:
runtime/mercury_hand_unify_body.h:
runtime/mercury_hand_unify_compare_body.h:
runtime/mercury_ho_call.c:
tools/make_port_code:
Pass proc_layout structures instead of proc_static structures
to deep profiling port routines.
runtime/mercury_conf_param.h:
Make MR_DEEP_PROFILING as well as MR_EXEC_TRACING imply MR_STACK_TRACE,
since deep profiling now needs stack tracing. (MR_STACK_TRACE needs
to be set in this file, because tests in this file depend on knowing
its value, and this file is among the first files included (in this
case indirectly) in mercury_imp.h.)
Document the macros controlling the debugging of deep profiling.
Enable printing of label names when the relevant deep profiling
debugging macro is set.
runtime/mercury_debug.c:
runtime/mercury_deep_rec_depth_actions.h:
runtime/mercury_deep_rec_depth_body.h:
runtime/mercury_exception_catch_body.h:
Get to proc_statics via proc_layouts.
runtime/mercury_deep_call_port_body.c:
runtime/mercury_deep_leave_port_body.c:
Get to proc_statics via proc_layouts.
Allow the debugger to disable deep profiling in Mercury code that is
part of the debugger, not of the user program being executed.
Add some more assertions.
runtime/mercury_engine.[ch]:
Add a new debugging flag that controls at runtime whether we generate
a human readable Deep.debug equivalent to the binary Deep.data files.
(We already had a mechanism for controlling this at compile time,
but this isn't flexible enough.)
runtime/mercury_wrapper.c:
Allow this new debugging flag to be set from MERCURY_OPTIONS.
runtime/mercury_deep_profiling.[ch]:
Respect this new debugging flag.
Update the hand-written proc_static structures representing the runtime
system.
Print out addresses of proc_layout as well as proc_static structures
when assertions fail.
Add a field to the measurement structure for exception port counts,
and write out this field with the other port counts.
Remove procedure id information from proc_static structures,
deep profiling now uses the procedure id in the proc_layout structure.
Add to proc_static structures fields that specify where, if anywhere,
the variables needed by exception.m to executed the exception port code
are in the procedure's stack frame.
Define a global flag that allows the debugger to disable deep
profiling in Mercury code that is part of the debugger, not of the
user program being executed.
Increase type safety by providing two versions of the function
for registering proc_layouts, one for the proc_layout structures
of user-defined predicates and one for unify, compare and index
predicates.
Fix a bug that occurs only if MR_DEEP_PROFILING_EXPLICIT_CALL_COUNTS is
defined (which it usually isn't): the initial call count was wrong.
runtime/mercury_deep_profiling_hand.h:
Fix a bug: the handwritten code saving deep profiling variables was
saving them in slots that didn't belong to the relevant stack frame.
Update to conform to the modified definitions of proc_static structures
and the fact that we now reach them via proc_layout structures.
runtime/mercury_exception_catch_body.h:
runtime/mercury_stacks.h:
Fix the other side of the bug in mercury_deep_profiling_hand.h
by reserving the right number of stack slots in the stack frames
of the various modes of exception__catch. Make it harder to make
the same bug in the future by getting the needed info from the
place in mercury_stacks.h that defines the structure of the relevant
stack frame.
runtime/mercury_proc_id.h:
Rename the procedure id structure fields referring to compiler
generated unify, compare and index predicates: they are not the only
compiler-generated predicates.
runtime/mercury_stack_layout.h:
Change procedure layout structures to allow them to be used for deep
profiling as well as for debugging, as described in the prologue above.
We don't need the capability to support label layout structures with
links to misnamed proc layout structures, and supporting it is
inconvenient, so delete the capability.
runtime/mercury_debug.c:
runtime/mercury_deep_profiling_hand.h:
runtime/mercury_layout_util.c:
runtime/mercury_ml_expand_body.h:
runtime/mercury_stack_trace.c:
runtime/mercury_types.h:
trace/mercury_trace_external.c:
Conform to the new names of the procedure id structure fields.
runtime/mercury_std.h:
Add some more arities for MR_PASTE for use in some of the modified
modules in the runtime.
trace/mercury_trace_internal.c:
Disable deep profiling actions in Mercury code that is part of the
debugger, not of the program being debugged.
scripts/init_grade_options.sh-subr:
scripts/parse_grade_options.sh-subr:
Make changes parallel to the ones in runtime/mercury_grade.h: delete
--stack-trace as an independent option, and make --debug set its
own option, not --require-tracing.
scripts/canonical_grade.sh-subr:
scripts/final_grade_options.sh-subr:
scripts/c2init.in:
scripts/mgnuc.in:
scripts/ml.in:
Conform to the changes in grade options for debugging and for deep
profiling.
tools/bootcheck:
If Mmake.stage.{browser,deep,library,runtime,trace}.params exist,
copy them to become the file Mmake.$dir.params in stage2/$dir
(where dir is derived from the name of the original file in the obvious
way). This allows more flexibility in the creation of the stage2;
for example, it allows some directories (e.g. runtime or library)
to be compiled with more debugging than other directories (e.g.
compiler). This may be required because compiling all directories
with lots of debugging may cause the linker to thrash.
Add an option, --disable-debug-libs, that clobbers the libraries
that should be linked in only in debugging grades.
To conserve disk space, remove Deep.data files created by the bootcheck
by default. Add an option, --keep-deep-data, to preserve these files.
Use a consistent mechanism (test -f) for testing the existence of
all files whose existence is tested.
When recording modification times, record the modification times
of some more files.
tests/hard_coded/Mmakefile:
In deep profiling grades, disable the test cases that we don't now
pass in such grades, and document the reasons for their failure.
Fix the misclassification of the write_binary test case.
|
||
|
Fergus Henderson
|
b4e761a51c |
Improve the documentation of deep profiling.
Estimated hours taken: 1 Branches: main Improve the documentation of deep profiling. deep_profiler/README: New file, containing pointers to the different sources of documentation on deep profiling. runtime/mercury_deep_profiling.h: runtime/mercury_deep_profiling.c: Add a comment pointing to deep_profiler/README. |
||
|
Zoltan Somogyi
|
36ab80e0ca |
Fix some conditionally-compiled references to MR_comp_arity.
Estimated hours taken: 0.1 Branches: main runtime/mercury_debug.c: runtime/mercury_deep_profiling.c: Fix some conditionally-compiled references to MR_comp_arity. |
||
|
Zoltan Somogyi
|
1315102d7c |
Fix several occurrences of a bug that prevented the compiler from passing
Estimated hours taken: 20 Branches: main Fix several occurrences of a bug that prevented the compiler from passing all the tests in deep profiling grades. We now pass all the tests in deep profiling grades. Some additional changes were useful in tracking this bug down. Using C types to represent deep profiling data structures allowed us to delete a bunch of casts and thus eliminate them as potential bug locations. Another was to fix some bugs in low level debugging support. In light of the fragility of the deep profiling invariants, always check them when writing out deep profiling data files. Since mdprof depends on these invariants, they have to be checked somewhere, and we already have code for the checks in the runtime. Reenable tail recursion support in deep profiling grades, since it now works. (The bug this change fixes used to occur most in tail recursive procedures, which is why previously I turned it off.) compiler/options.m: Reenable tail recursion support in deep profiling grades. runtime/mercury_unify_compare_body.h: Fix the bug that prevented us from passing all the tests with invariants checked and tail recursion support enabled. The bug was that in several cases, the code in mercury_unify_compare_body.h did not invoke the appropriate deep profiling routines and thus did not fill in the deep profiling data structure they were supposed to fill in, breaking an invariant. There were several instances of this bug: unify, compare and compare_representation on tuples, and compare_representation on functions, predicates and user-defined types. These oversights were possible because the actions of returning an answer and filling in the deep profiling data structures (in deep profiling grades) were separate. If one omitted the latter, tests could still work in all grades except deep profiling grades. The fix therefore uses one macro (return_compare_answer or return unify_answer, depending on the the operation) to fill in the deep profiling data structure (in deep profiling grades) and return the answer, making it impossible to forget to do the former. The new approach treats compare_representation the same as compare, and gathers the same information for it. runtime/mercury_ho_call.c: Factor out some commonalities in the definitions of the MR_ProcStatic data structures used by the code in mercury_unify_compare_body.h. Change a macro name to support the changes in mercury_unify_compare_body.h. Change the module name of compare_representation/3 from std_util to builtin, to match unify/2 and compare/3. compiler/deep_profiling.m: Treat compare_representation/3 the same way as we treat compare/3. library/builtin.m library/std_util.m Move the declaration of compare_representation/3 from std_util to builtin, to make it easier to handle it the same way as compare/3. Since it is just a variant of compare/3 and is at least as built into the implementation, it belongs there anyway. library/profiling_builtin.m: Use C types to represent deep profiling data structures. Delete the casts that are redundant after this change. Fix formatting of foreign_procs. runtime/mercury_deep_profiling.c: As mentioned above, always check the invariants of the deep profiling data structures when writing them out. runtime/mercury_deep_profiling.h: Avoid a warning about nested declarations of variable "i". runtime/mercury_deep_rec_depth_body.h: Delete a bunch of casts we don't need anymore. runtime/mercury_wrapper.[ch]: Initialize a couple of predicate names in the low level debug support to "" instead of NULL. The code that accesses these variables passes them to strcmp directly, which lead to core dumps. We could change the accessing code to test for NULL, but since that code is executed on every call when low level debugging is enabled, that would lead to even greater slowdowns, which we can do without. To avoid a warning about casting away const, make the variables involved "const char *" instead of plain "char *"; we don't want to update the pointed-to strings anyway. tools/bootcheck: In profiling grades, write out profiling data files by default, and require the use of an option to turn them off. This is needed to make sure that the deep profiling invariants are checked by default. tests/hard_coded/compare_rep_usereq.m: tests/hard_coded/compare_representation.m: Avoid hard-coding the name of the module defining compare_representation/3. |
||
|
Zoltan Somogyi
|
010a9ecf74 |
Fix several bugs in deep profiling. These allow the compiler to bootstrap
Estimated hours taken: 30
Branches: main
Fix several bugs in deep profiling. These allow the compiler to bootstrap
again, *including* writing out the profiling data, with sanity checks enabled.
Some test cases still fail, but significantly fewer than before.
These fixes required several improvements in the infrastructure for low
level debugging in the LLDS grades.
compiler/deep_profiling.m:
Mark calls that have a prepare_for_{normal,ho,...}_call inserted before
them as impure, to prevent simplify from optimizing them away, e.g. as
duplicate calls. This is needed because a prepare_for_{...}_call that
is not followed immediately by the call port code of the callee leaves
the profiling tree in a state that violates its invariants.
Mark the redo port code of model_non predicates as needing to be
preserved, even if determinism analysis would normally cause it to be
cut by marking the disjunction it is part of (whose two disjuncts are
the det exit port code and the failure redo port code) as det.
Fix the generation of goal paths to match what the rest of the compiler
does. Start number conjuncts, disjuncts and switch arms from 1, not 0,
and do not reverse goal paths before attaching them to goals; they will
be reversed when they are converted to strings.
compiler/det_analysis.m:
If a disjunct has determinism failure but is impure, treat it as being
able to succeed when computing the max number of solutions of the
disjunction as a whole, *provided* that some earlier disjuct could
succeed. The idea is that ( impure failure ; det ) should be treated
as det, since all backtracking is local within it, while disjunctions
of the form ( det ; impure failure ) should be treated as multi, since
we want to be able to backtrack to the second disjunct from *outside*
the disjunction.
At the moment, we do this not for all impure goals, but only for the
impure goals that deep_profiling marks with the preserve_backtrack_into
feature.
compiler/hlds_goal.m:
Add the preserve_backtrack_into feature.
Add utility predicates for handling the features of a goal directly,
without explicitly dealing with its goal_info.
runtime/mercury_debug.[ch]:
Add mechanisms for turning the printing of low level debugging messages
on and off. Without this, enabling low level debugging can generate
literally gigabytes of debugging output.
The mechanisms all depend on numbering calls.
One mechanism allows messages to be printed starting from calls in
given ranges, by including e.g. -di100-200,300-400 in MERCURY_OPTIONS.
Another mechanism allows N messages to be printed starting from calls
to a given procedure or from calls at which next_call_site_dynamic
has a given value. The value of N (the size of the block of calls)
can be specified by include -dB<num> in MERCURY_OPTIONS. The name of
the given procedure (actually the name of its entry label) can be
specified by including -dj<entrylabel> in MERCURY_OPTIONS. The address
of the call_site_dynamic structure to watch for is specified with the
-dW<addr> option in MERCURY_OPTIONS, as before.
runtime/mercury_wrapper.[ch]:
Add the global variables required to implement the new low level
debugging functionality, as well as the option processing code required
to set those global variables.
Separate the flags controlling the printing of the values of stack
control registers (sp, curfr etc) and ordinary registers (r1, r2 etc).
Print ordinary registers only if explicitly requested.
runtime/mercury_engine.h:
Add the required global flags.
runtime/mercury_deep_profiling.[ch]:
Add two extra arguments to MR_deep_assert, and print them if an
assertion fails. This makes it easier to decide what parameters to give
to the new low level debugging mechanisms.
runtime/mercury_deep_*_port_body.h:
runtime/mercury_deep_rec_depth_*.h:
library/profiling_builtin.m:
Pass the extra arguments to MR_deep_assert, and print low level
debugging messages only if the relevant flag is set.
tools/lmc:
Add mechanisms to add to the list of C compiler flags the program is
compiled with by lmc.
|
||
|
Simon Taylor
|
b7c4a317e9 |
Add MR_ prefixes to the remaining non-prefixed symbols.
Estimated hours taken: 4 Branches: main Add MR_ prefixes to the remaining non-prefixed symbols. This change will require all workspaces to be updated The compiler will start generating references to MR_TRUE, MR_bool, etc., which are not defined in the old runtime header files. runtime/mercury_std.h: Add MR_ prefixes to bool, TRUE, FALSE, max, min, streq, strdiff, strtest, strntest, strneq, strndiff, strntest, NO_RETURN. Delete a commented out definition of `reg'. runtime/mercury_tags.h: Add an MR_ prefix to TAGBITS. configure.in: runtime/mercury_goto.h: runtime/machdeps/i386_regs.h/mercury_goto.h: Add an MR_ prefix to PIC. runtime/mercury_conf_param.h: Allow non-prefixed PIC and HIGHTAGS to be defined on the command line. runtime/mercury_bootstrap.h: Add backwards compatibility definitions. RESERVED_MACRO_NAMES: Remove the renamed macros. compiler/export.m: compiler/ml_code_gen.m: Use MR_bool rather than MR_Bool (MR_Bool is meant to be for references to the Mercury type bool__bool). runtime/mercury_types.h: Add a comment the MR_Bool is for references to bool__bool. */*.c: */*.h: */*.m: Add MR_ prefixes. |
||
|
Zoltan Somogyi
|
b456621269 |
New module for utility functions. Contains the checked versions of
Estimated hours taken: 1 Branches: main runtime/mercury_runtime_util.[ch]: New module for utility functions. Contains the checked versions of fopen, fclose, atexit which used to be static in mercury_prof.c, and the backup version of strerror which used to be in mercury_strerror.c. runtime/mercury_strerror.[ch]: Deleted this now obsolete module. runtime/mercury_prof.c: runtime/Mmakefile: Mention the new module and delete references to the deleted module. runtime/mercury_deep_profiling.c: runtime/mercury_trace_base.c: Add #includes of the new module. |
||
|
Zoltan Somogyi
|
9c880d612e |
Use more descriptive variable names.
Estimated hours taken: 0.2 Branches: main runtime/mercury_deep_profiling.c: Use more descriptive variable names. |
||
|
Zoltan Somogyi
|
a9fed43d57 |
Major enhancements of the deep profiler. The most important ones are:
Estimated hours taken: 80 Branches: main Major enhancements of the deep profiler. The most important ones are: - The addition of several new classes of preferences, including: the use of colour, boxing, sorting, summarizing of higher order call sites, and time formats. - Making preferences persistent across different queries, so that once a preference is set, it "sticks" until the user changes it. Previously, preferences stuck during query sequences of the same command type. - Several new command types: - listing all the modules of the program - listing all the procedures in a module - listing all the callers of a procedure, grouped by calling call site, procedure, module, or clique, and filtering out certain classes of ancestors - jumping into the call graph close to the action - restarting the server (useful when the data file changes) - New optional columns showing time per call, allocations per call and words allocated per call. - Can now display memory consumption in bytes as well as words. - More robustness in the face of external events, e.g. machine shutdowns. - Fix a bug reported by Tom in the summaries of procedures that make higher order calls. The new functionality required adding some fields to ProcStatic structures; as a result, compilers and runtime systems that have this change are incompatible with compilers and runtime systems before this change in deep profiling grades. (They of course remain compatible in other grades.) compiler/deep_profiling.m: compiler/layout.m: compiler/layout_out.m: Add two new fields to ProcStatic structures, one giving the line number of procedure's context and one stating whether the procedure is exported from its module. compiler/layout.m: Be consistent about filename vs file_name in field names. compiler/*.m: Minor changes to handle the new fields. deep_profiler/interface.m: Define new command types, modify some of the parameters of existing ones, and delete obsolete ones. Define the types and predicates used by the new system of preferences, Update the predicates for recognizing and generating queries accordingly. Make the order of declarations and definitions more consistent. deep_profiler/split.m: Complete rewrite of the only predicate of this module. The old split predicate deleted any empty substrings resulting from the breakup of the original string. The new one preserves them, because they are needed by the new encoding scheme used in interface.m. deep_profiler/query.m: New module, containing code dealing with the computational issues of queries. Some of its code is old (from server.m), much of it is new. deep_profiler/html_format.m: New module, containing code dealing with HTML formatting. Some of its code is old (from server.m), much of it is new. deep_profiler/top_procs.m: New module, containing code dealing with issues involving sorting by various criteria. Some of its code is old (from server.m), much of it is new. deep_profiler/exclude.m: New module to handle contour exclusion. This means that when listing the callers of a procedure, we display the nearest parent that is *not* excluded by virtue of being in a given module. The set of modules so excluded forms a contour drawn through the program. deep_profiler/mdprof_cgi.m: deep_profiler/mdprof_server.m: deep_profiler/server.m: Minor changes to adapt to the new system of preferences. deep_profiler/array_util.m: Add a mode to foldl2. deep_profiler/io_combinator.m: Add predicates for reading in sequences of ten things. deep_profiler/measurements.m: Add a function needed by new code. deep_profiler/timeout.m: Make the profiler robust in the face of signals. deep_profiler/canonical.m: Some more work towards working canonicalization; not there yet. Move some procedures to profile.m, since other modules also need them now. deep_profiler/profile.m: Add the new fields to ProcStatic structures. Record the word size. Record more information about procedures whose activation counters are ever zeroed, in order to allow query.m to avoid giving misleading information in cases where a procedure calls itself through a higher order call site. Record information about the modules of the program. Add a bunch of lookup predicates, some moved from canonical.m. deep_profiler/call_graph.m: Minor changes to conform to changes in profile.m. deep_profiler/startup.m: Fill in the new parts of the profile data structure. deep_profiler/read_profile.m: Read the new fields in ProcStatic structures. Read in the id of the root node as a fixed part of the header, not as a node. Read in the word size. Make it easier to find all the debugging output sites. Record, for each call site which can call more than one procedure, whether this causes the caller's ProcStatic structure's activation count to be zeroed. runtime/mercury_deep_profiling.h: Add the new fields to ProcStatic structures. runtime/mercury_deep_profiling.c: Write out the new fields to ProcStatic structures. Write out the id of the root node as a fixed part of the header, not as a node. Write out the word size. Remove incorrect L suffixes on constants. Record that the artificial procedure "parent of main" is called once, not zero times, to avoid division by zero when computing per-call statistics. runtime/mercury_deep_profiling_hand.h: Add the new fields to the macros for creating ProcStatic structures. runtime/mercury_ho_call.c: library/array.m: library/builtin.m: library/exception.m: library/std_util.m: Add the new fields to the invocations of those macros. |
||
|
Zoltan Somogyi
|
924316cd2a |
Add missing "volatile" qualifier on a definition (the declaration
Estimated hours taken: 0.1 Branches: main runtime/mercury_deep_profiling.c: Add missing "volatile" qualifier on a definition (the declaration already had it). |
||
|
Zoltan Somogyi
|
230cb689f6 |
Enhance deep profiling by
Estimated hours taken: 20 Branches: main Enhance deep profiling by (1) optimizing self-tail-recursive calls by using a single service function, prepare_for_tail_call, instead of the prepare_for_normal_call and call_code pair. Since there is no callcode in the inner procedure, the inner procedure doesn't pick up the value of MiddleCSD, and thus cannot pass it to other service functions. Those functions therefore now pick up MiddleCSD from the global current_call_site_dynamic. We also avoid calling set_current_csd in the exit and failure continuations after tail calls in the outer procedure, since their effects will be overridden by the exitcode or failcode of the outer procedure. (2) using a prepare_for_method_call that is separate from prepare_for_ho_call, since extracting a key from a typeclass_info as if it were a closure is not robust (3) adding callcode/exitcode/failcode to the unify/compare routines for the builtin types. This is necessary to record their identities and to count their exits and fails. The resulting system should conform to the invariant that every prepare_for_special_call is matched either by the callcode of a builtin unify/compare routine or by the callcode of a compiler-generated __Unify__ or __Compare__ procedure. The third change records information that we need. It leads to slowdown, but the speedup from the first change more than compensates for it. (The second change is performance neutral.) compiler/deep_profiling: Modify the source transformation to achieve the effects described above. library/profiling_builtin.m: Add prepare_for_tail_call and prepare_for_method_call. Remove MiddleCSD from the argument list of service functions that do not need it. Delete two unused predicates. runtime/mercury_deep_profiling_hand.h: Do not pass MiddleCSD to the service functions from which that argument was removed. runtime/mercury_deep_rec_depth_actions.h: runtime/mercury_deep_rec_depth_body.h: Fix a situation in which one local variable shadowed another. runtime/mercury_ho_call.c: runtime/mercury_unify_compare_body.h: Add callcode/exitcode/failcode to the unify and compare routines of builtin types. Since these routines never call any other routine, we combine callcode/exitcode and callcode/failcode into one action. Write out the proc_static structures of the unify/compare procedures for the builtin types. runtime/mercury_deep_profiling.[ch]: Add statistics for the new service functions. Add statistics for callbacks, for the sake of completeness. Delete an unused argument from the histogram statistics functions. |
||
|
Zoltan Somogyi
|
04e614485d |
Implement deep profiling; merge the changes on the deep2 branch back
Estimated hours taken: 500 Branches: main Implement deep profiling; merge the changes on the deep2 branch back onto the trunk. The main documentation on the general architecture of the deep profiler is the deep profiling paper. doc/user_guide.texi: Document how to use the deep profiler. deep_profiler: deep_profiler/Mmakefile: A new directory holding the deep profiler and its mmakefile. Mmakefile: Add targets for the new directory. Add support for removing inappropriate files from directories. deep_profiler/interface.m: The deep profiler consists of two programs: mdprof_cgi.m, which acts as a CGI "script", and mdprof_server.m, which implements the server process that the CGI script talks to. Interface.m defines the interface between them. script/mdprof.in: A shell script template. ../configure uses it to generate mdprof, which is a wrapper around mdprof_cgi that tells it how to find mdprof_server. deep_profiler/mdprof_cgi.m: The CGI "script" program. deep_profiler/mdprof_server.m: The top level predicates of the server. deep_profiler/profile.m: The main data structures of the server and their operations. deep_profiler/read_profile.m: Code for reading in profiling data files. deep_profiler/startup.m: Code for post-processing the information in profiling data files, propagating costs from procedures to their ancestors and performing various kinds of summaries. deep_profiler/server.m: Code for responding to requests from the CGI script. deep_profiler/cliques.m: Code to find cliques in graphs. deep_profiler/array_util.m: deep_profiler/util.m: Utility predicates. deep_profiler/dense_bitset.m: An implementation of (part of) the set ADT with dense bit vectors. deep_profiler/measurements.m: Operations on profiling measurements. deep_profiler/timeout.m: An implementation of a timeout facility. deep_profiler/conf.m: Functions that depend on autoconfigured settings. configure.in: Find out what command to use to find the name of the local host. Install deep profiling versions of the standard library along with the other profiling versions. runtime/mercury_conf.h.in: Add some macros for deep_profiler/conf.m to use. library/profiling_builtin.m: runtime/mercury_deep_call_port_body.h: runtime/mercury_deep_leave_port_body.h: runtime/mercury_deep_redo_port_body.h: A new library module that implements deep profiling primitives. Some of these primitives have many versions, whose common code is factor is factored out in three new include files in the runtime. compiler/deep_profiling.m: New module to perform the program transformations described in the paper. compiler/notes/compiler_design.html: Document the new compiler module. compiler/mercury_compiler.m: Invoke the new module in deep profiling grades. Allow global static data to be generated by deep_profiling.m. compiler/options.m: Add options to turn on deep profiling and (for benchmarking purposes) control its implementation. Add an optiooption disable tailcall optimization in the LLDS backend, to help benchmarking deep profiling. compiler/jumpopt.m: compiler/optimize.m: Obey the option to disable tailcalls. compiler/handle_options.m: Handle the implications of deep profiling. compiler/modules.m: In deep profiling grades, automatically import profiling_builtin.m. compiler/prog_util.m: doc/Makefile: library/library.m: Handle the new builtin module. compiler/export.m: In deep profiling grades, wrap deep profiling code around exported procedures to handle the "unscheduled call" aspects of callbacks to Mercury from the foreign language. compiler/higher_order.m: profiler/demangle.m: util/demangle.c: When creating a name for a higher-order-specialized predicate, include the mode number in the name. compiler/add_trail_ops.m: compiler/type_util.m: Move c_pointer_type from add_trail_ops to type_util, so it can also be used by deep_profiling.m. compiler/hlds_goal.m: Add a new goal feature that marks a tail call, for use by deep_profiling.m. compiler/hlds_pred.m: Add a new field to proc_info structures for use by deep_profiling.m. Add a mechanism for getting proc_ids for procedure clones. Remove next_proc_id, an obsolete and unused predicate. compiler/hlds_data.m: Add a new cons_id to refer to the proc_static structure of a procedure. compiler/bytecode_gen.m: compiler/code_util.m: compiler/dependency_graph.m: compiler/hlds_out.m: compiler/mercury_to_mercury.m: compiler/ml_unify_gen.m: compiler/opt_debug.m: compiler/prog_rep.m: compiler/rl_exprn.m: compiler/switch_util.m: compiler/unify_gen.m: Trivial changes to handle the new cons_id, goal feature and/or proc_info argument. compiler/rtti.m: Add a utility predicate for extracting pred_id and proc_id from an rtti_proc_label, for use by hlds_out.m compiler/layout.m: compiler/layout_out.m: compiler/llds.m: compiler/llds_common.m: Add support for proc_static and call_site_static structures. compiler/layout_out.m: compiler/llds_out.m: Add code for the output of proc_static structures. compiler/code_util.m: Make code_util__make_proc_label_from_rtti a function, and export it. util/mkinit.c: compiler/llds_out.m: compiler/layout.m: compiler/modules.m: Add support for a fourth per-module C function, for writing out proc_static structures (and the call_site_static structures they contains). Since proc_static structures can be referred to from LLDS code (and not just from other static structures and compiler-generated C code), reorganize the declarations of static structures slightly. Change the schema for the name of the first per-module C function slightly, to make it the addition of the fourth function easier. The scheme now is: mercury__<modulename>__init mercury__<modulename>__init_type_tables mercury__<modulename>__init_debugger mercury__<modulename>__write_out_proc_statics Improve formatting of the generated C code. library/*.m: runtime/mercury.c: runtime/mercury_context.c: runtime/mercury_engine.c: runtime/mercury_ho_call.c: runtime/mercury_tabling.c: runtime/mercury_trace_base.c: runtime/mercury_wrapper.c: trace/mercrury_trace.[ch]: trace/mercrury_trace_declarative.c: trace/mercrury_trace_external.c: trace/mercrury_trace_internal.c: Conform to the new scheme for initialization functions for hand-written modules. compiler/mercury_compile.m: library/benchmarking.m: runtime/mercury_conf_param.h: runtime/mercury.h: runtime/mercury_engine.c: runtime/mercury_goto.c: runtime/mercury_grade.h: runtime/mercury_ho_call.c: runtime/mercury_label.[ch]: runtime/mercury_prof.[ch]: Add an MR_MPROF_ prefix in front of the C macros used to control the old profiler. compiler/handle_options.m: runtime/mercury_grade.h: scripts/canonical_grade.sh-subr: scripts/init_grade_options.sh-subr: scripts/parse_grade_options.sh-subr: Make deep profiling completely separate from the old profiling system, by making the deep profiling grade independent of MR_MPROF_PROFILE_TIME and the compiler option --profile-time. library/array.m: library/builtin.m: library/std_util.m: runtime/mercury_hand_unify_body.h: runtime/mercury_hand_compare_body.h: In deep profiling grades, wrap the deep profiling call, exit, fail and redo codes around the bodies of hand-written unification and comparison procedures. Make the reporting of array bounds violations switchable between making them fatal errors, as we currently, and reporting them by throwing an exception. Throwing an exception makes debugging code using arrays easier, but since exceptions aren't (yet) propagated across engine boundaries, we keep the old behaviour as the default; the new behaviour is for implementors. runtime/mercury_deep_profiling_hand.h: New file that defines macros for use in Mercury predicates whose definition is in hand-written C code. library/exception.m: runtime/mercury_exception_catch_body.h: runtime/mercury_stacks.h: In deep profiling grades, wrap the deep profiling call, exit, fail and redo codes around the bodies of the various modes of builtin_catch. Provide a function that C code can use to throw exceptions. library/benchmarking.m: library/exception.m: library/gc.m: library/std_util.m: runtime/mercury_context.[ch]: runtime/mercury_engine.[ch]: runtime/mercury_debug.c: runtime/mercury_deep_copy.c: runtime/mercury_overflow.h: runtime/mercury_regs.h: runtime/mercury_stacks.h: runtime/mercury_thread.c: runtime/mercury_wrapper.c: Add prefixes to the names of the fields in the engine and context structures, to make code using them easier to understand and modify. runtime/mercury_deep_profiling.[ch]: New module containing support functions for deep profiling and functions for writing out a deep profiling data file at the end of execution. runtime/mercury_debug.[ch]: Add support for debugging deep profiling. Add support for watching the value at a given address. Make the buffered/unbuffered nature of debugging output controllable via the -du option. Print register contents only if -dr is specified. runtime/mercury_goto.h: runtime/mercury_std.h: Use the macros in mercury_std.h instead of defining local variants. runtime/mercury_goto.h: runtime/mercury_stack_layout.h: runtime/mercury_stack_trace.c: runtime/mercury_tabling.c: trace/mercury_trace.c: trace/mercury_trace_declarative.c: trace/mercury_trace_external.c: trace/mercury_trace_vars.c: Standardize some of the macro names with those used in the debugger paper. runtime/mercury_heap.h: Add support for memory profiling with the deep profiler. runtime/mercury_prof.[ch]: runtime/mercury_prof_time.[ch]: Move the functionality that both the old profiler and the deep profiler need into the new module mercury_prof_time. Leave mercury_prof containing stuff that is only relevant to the old profiler. runtime/mercury_prof.[ch]: runtime/mercury_strerror.[ch]: Move the definition of strerror from mercury_prof to its own file. runtime/mercury_wrapper.[ch]: Add support for deep profiling. Add suppory for controlling whether debugging output is buffered or not. Add support for watching the value at a given address. runtime/Mmakefile: Mention all the added files. scripts/mgnuc.in: Add an option for turning on deep profiling. Add options for controlling the details of deep profiling. These are not documented because they are intended only for benchmarking the deep profiler itself, for the paper; they are not for general use. tools/bootcheck: Compile the deep_profiler directory as well as the other directories containing Mercury code. Turn off the creation of deep profiling data files during bootcheck, since all but one of these in each directory will be overwritten anyway. Add support for turning on --keep-objs by default in a workspace. tools/speedtest: Preserve any deep profiling data files created by the tests. trace/mercury_trace.c: Trap attempts to perform retries in deep profiling grades, since they would lead to core dumps otherwise. util/Mmakefile: Avoid compile-time warnings when compiling getopt. tests/*/Mmakefile: tests/*/*/Mmakefile: In deep profiling grades, switch off the tests that test features that don't work with deep profiling, either by design or because the combination hasn't been implemented yet. |