deep_profiler/analysis_utils.m:
deep_profiler/autopar_find_best_par.m:
deep_profiler/autopar_reports.m:
deep_profiler/autopar_search_callgraph.m:
deep_profiler/autopar_search_goals.m:
deep_profiler/callgraph.m:
deep_profiler/canonical.m:
deep_profiler/cliques.m:
deep_profiler/coverage.m:
deep_profiler/dump.m:
deep_profiler/mdprof_cgi.m:
deep_profiler/mdprof_create_feedback.m:
deep_profiler/mdprof_dump.m:
deep_profiler/mdprof_procrep.m:
deep_profiler/mdprof_report_feedback.m:
deep_profiler/mdprof_test.m:
deep_profiler/profile.m:
deep_profiler/read_profile.m:
deep_profiler/recursion_patterns.m:
deep_profiler/startup.m:
deep_profiler/var_use_analysis.m:
Replace implicit streams with explicit streams.
In some places, simplify some code, often using constructs such as
string.format that either did not exist or were too expensive to use
when the original code was written.
Consistenly use the spelling StdErr over Stderr.
In mdbprof_dump.m, put filename and reason-for-failing-to-open-that-file
in the right order in an error message.
deep_profiler/DEEP_FLAGS.in:
Turn on --warn-implicit-stream-calls for the entire deep_profiler
directory.
mdbcomp/program_representation.m:
mdbcomp/trace_counts.m:
Replace implicit streams with explicit streams. These are the two mdbcomp
modules that (a) used to use implicit streams, and (2) are used by the
deep profiler.
mdbcomp/Mercury.options:
Turn on --warn-implicit-stream-calls for these two modules.
slice/mcov.m:
slice/mtc_union.m:
Conform to the changes in mdbcomp.
deep_profiler/*.m:
Fix inconsistencies between (a) the order in which functions and predicates
are declared, and (b) the order in which they are defined.
In most modules, either the order of the declarations or the order
of the definitions made sense, and I changed the other to match.
In some modules, neither made sense, so I changed *both* to an order
that *does* make sense (i.e. it has related predicates together).
In query.m, put the various commands in the same sensible order
as the code processing them.
In html_format.m, merge two exported functions together, since
they can't be used separately.
In some places, put dividers between groups of related
functions/predicates, to make the groups themselves more visible.
In some places, fix comments or programming style.
deep_profiler/DEEP_FLAGS.in:
Since all the modules in this directory are now free from any warnings
generated by --warn-inconsistent-pred-order-clauses, specify that option
by default in this directory to keep it that way.
deep_profiler/*.m:
Replace ( C -> T ; E ) if-then-elses with (if C then T else E ).
Replace calls to error/1 with calls to unexpected/3.
Add some module qualifications where this makes the code easier to read.
Fix some other issues as well that I found while fixing the spelling.
mdbcomp/feedback.automatic_parallelism.m:
deep_profiler/autopar_find_best_par.m:
deep_profiler/mdprof_create_feedback.m:
Rename the best_par_algorithm type to alg_for_finding_best_par,
since the old name was misleading. Perform the same rename for
another type based on it, and the option specifying it.
Remove the functor estimate_speedup_by_num_vars, since it hasn't
been used by anything in a long time, and won't in the future.
deep_profiler/autopar_calc_overlap.m:
deep_profiler/autopar_costs.m:
deep_profiler/autopar_reports.m:
deep_profiler/autopar_search_callgraph.m:
deep_profiler/autopar_search_goals.m:
deep_profiler/coverage.m:
deep_profiler/create_report.m:
deep_profiler/dump.m:
deep_profiler/mdprof_report_feedback.m:
deep_profiler/measurement_units.m:
deep_profiler/measurements.m:
deep_profiler/message.m:
deep_profiler/query.m:
deep_profiler/recursion_patterns.m:
deep_profiler/report.m:
deep_profiler/startup.m:
deep_profiler/var_use_analysis.m:
mdbcomp/mdbcomp.goal_path.m:
mdbcomp/program_representation.m:
Conform to the above. Fix spelling errors. In some places, improve
comments and/or variable names.
Branches: main
Mark the predicates in the sv* modules in the standard library (except for the
recently added ones) as obsolete. They will be deleted after the 11.07
release.
library/svarray.m:
library/svbag.m:
library/svbimap.m:
library/sveqvclass.m:
library/svmap.m:
library/svmulti_map.m:
library/svqueue.m:
library/svset.m:
library/svvarset.m:
Mark these modules as obsolete.
deep_profiler/dump.m:
deep_profiler/recursion_patterns.m:
profiler/snapshots.m:
Avoid using predicates from sv* modules.
library where it can be used by the deep profiler.
Also move the goal path code from program_representation.m to the new module,
goal_path.m in mdbcomp/
mdbcomp/goal_path.m:
New module containing goal path code.
mdbcomp/program_representation.m:
Original location of goal path code.
compiler/goal_path.m:
Move some of this goal_path code into mdbcomp/goal_path.m
mdbcomp/feedback.automatic_parallelisation.m:
mdbcomp/rtti_access.m:
mdbcomp/slice_and_dice.m:
mdbcomp/trace_counts.m:
browser/debugger_interface.m:
browser/declarative_execution.m:
browser/declarative_tree.m:
compiler/build_mode_constraints.m:
compiler/call_gen.m:
compiler/code_info.m:
compiler/continuation_info.m:
compiler/coverage_profiling.m:
compiler/deep_profiling.m:
compiler/format_call.m:
compiler/goal_path.m:
compiler/goal_util.m:
compiler/hlds_data.m:
compiler/hlds_goal.m:
compiler/hlds_out_goal.m:
compiler/hlds_out_pred.m:
compiler/hlds_pred.m:
compiler/interval.m:
compiler/introduce_parallelism.m:
compiler/layout_out.m:
compiler/llds.m:
compiler/mode_constraint_robdd.m:
compiler/mode_constraints.m:
compiler/mode_ordering.m:
compiler/ordering_mode_constraints.m:
compiler/polymorphism.m:
compiler/post_typecheck.m:
compiler/prog_rep.m:
compiler/prop_mode_constraints.m:
compiler/push_goals_together.m:
compiler/rbmm.condition_renaming.m:
compiler/smm_common.m:
compiler/stack_layout.m:
compiler/stack_opt.m:
compiler/trace_gen.m:
compiler/tupling.m:
compiler/type_constraints.m:
compiler/typecheck.m:
compiler/unify_gen.m:
compiler/unneeded_code.m:
deep_profiler/Mmakefile:
deep_profiler/analysis_utils.m:
deep_profiler/coverage.m:
deep_profiler/create_report.m:
deep_profiler/display_report.m:
deep_profiler/dump.m:
deep_profiler/mdprof_fb.automatic_parallelism.m:
deep_profiler/message.m:
deep_profiler/old_query.m:
deep_profiler/profile.m:
deep_profiler/program_representation_utils.m:
deep_profiler/read_profile.m:
deep_profiler/recursion_patterns.m:
deep_profiler/report.m:
deep_profiler/var_use_analysis.m:
slice/Mmakefile:
slice/mcov.m:
Conform to the move of the goal path code.
analysis.
When instrumenting a program for deep and coverage profiling the compiler adds
goal path information to the program. The compiler also writes out a bytecode
representation of the program. The step_switch goal path step includes a field
for the number of functors of the type of the variable that is being switched
upon. When this is included in the deep profiling data (as it is now) and
these goals paths are used as the indexes in tables the deep profiler cannot
reconstruct matching goal paths from the program representation bytecode,
causing many important map lookups to fail or throw an exception.
We fix this not by preventing the compiler from writing out this information,
but by stripping it out of goal paths before the analysis tool uses those goal
paths.
mdbcomp/program_representation.m:
Add a new predicate rev_goal_path_remove_type_info/2 that removes
type-dependant information from goals.
deep_profiler/profile.m:
Change the goal_path string in the call_site_static type to contain a
reverse_goal_path rather than a string. This hides the abstraction of the
stripping of type dependant information within read_profile.m
deep_profiler/read_profile.m:
Strip type dependant information from goal path strings as they are read in.
Conform to changes in profile.m
deep_profiler/report.m:
In the call site static dump information use a goal_path field rather than
a string.
deep_profiler/analysis_utils.m:
deep_profiler/create_report.m:
deep_profiler/dump.m:
deep_profiler/mdprof_fb.automatic_parallelism.m:
deep_profiler/old_query.m:
Conform to changes in profile.m.
deep_profiler/display_report.m:
Conform to changes in report.m.
Estimated hours taken: 80
Branches: main
The existing representation of goal_paths is suboptimal for several reasons.
- Sometimes we need forward goal paths (e.g. to look up goals), and sometimes
we need reverse goal paths (e.g. when computing goal paths in the first
place). We had two types for them, but
- their names, goal_path and goal_path_consable, were not expressive, and
- we could store only one of them in goal_infos.
- Testing whether goal A is a subgoal of goal B is quite error-prone using
either form of goal paths.
- Using a goal path as a key in a map, which several compiler passes want to
do, requires lots of expensive comparisons.
This diff replaces most uses of goal paths with goal ids. A goal id is an
integer, so it can be used as a key in faster maps, or even in arrays.
Every goal in the body of a procedure gets its id allocated in a depth first
search. Since we process each goal before we dive into is descendants,
the goal representing the whole body of a procedure always gets goal id 0.
The depth first traversal also builds up a map (the containing goal map)
that tells us the parent goal of ever subgoal, with the obvious exception
of the root goal itself. From the containing goal map, one can compute
both reverse and forward goal paths. It can also serve as the basis of an
efficient test of whether the goal identified by goal id A is an ancestor
of another goal identified by goal id B. We don't yet use this test,
but I expect we will in the future.
mdbcomp/program_representation.m:
Add the goal_id type.
Replace the existing goal_path and goal_path_consable types
with two new types, forward_goal_path and reverse_goal_path.
Since these now have wrappers around the list of goal path steps
that identify each kind of goal path, it is now ok to expose their
representations. This makes several compiler passes easier to code.
Update the set of operations on goal paths to work on the new data
structures.
Add a couple of step types to represent lambdas and try goals.
Their omission prior to this would have been a bug for constraint-based
mode analysis, or any other compiler pass prior to the expansion out
of lambda and try goals that wanted to use goal paths to identify
subgoals.
browser/declarative_tree.m:
mdbcomp/rtti_access.m:
mdbcomp/slice_and_dice.m:
mdbcomp/trace_counts.m:
slice/mcov.m:
deep_profiler/*.m:
Conform to the changes in goal path representation.
compiler/hlds_goal:
Replace the goal_path field with a goal_id field in the goal_info,
indicating that from now on, this should be used to identify goals.
Keep a reverse_goal_path field in the goal_info for use by RBMM and
CTGC. Those analyses were too hard to convert to using goal_ids,
especially since RBMM uses goal_paths to identify goals in multi-pass
algorithms that should be one-pass and should not NEED to identify
any goals for later processing.
compiler/goal_path:
Add predicates to fill in goal_ids, and update the predicates
filling in the now deprecated reverse goal path fields.
Add the operations needed by the rest of the compiler
on goal ids and containing goal maps.
Remove the option to set goal paths using "mode equivalent steps".
Constraint based mode analysis now uses goal ids, and can now
do its own equivalent optimization quite simply.
Move the goal_path module from the check_hlds package to the hlds
package.
compiler/*.m:
Conform to the changes in goal path representation.
Most modules now use goal_ids to identify goals, and use a containing
goal map to convert the goal ids to goal paths when needed.
However, the ctgc and rbmm modules still use (reverse) goal paths.
library/digraph.m:
library/group.m:
library/injection.m:
library/pprint.m:
library/pretty_printer.m:
library/term_to_xml.m:
Minor style improvements.
Estimated hours taken: 2
Branches: main
Add the predicates sorry, unexpected and expect to library/error.m.
compiler/compiler_util.m:
library/error.m:
Move the predicates sorry, unexpected and expect from compiler_util
to error.
Put the predicates in error.m into the same order as their
declarations.
compiler/*.m:
Change imports as needed.
compiler/lp.m:
compiler/lp_rational.m:
Change imports as needed, and some minor cleanups.
deep_profiler/*.m:
Switch to using the new library predicates, instead of calling error
directly. Some other minor cleanups.
NEWS:
Mention the new predicates in the standard library.
Estimated hours taken: 12
Branches: main
Allow more than 2^15 vars in a procedure representation.
mdbcomp/program_representation.m:
Allow a variable number to be represented by four bytes as well as
two and one. This means that we also have to represent the number
of variables in a procedure using a four-byte number, not a two-byte
number.
Use four bytes to represent line numbers. Programs that overflow
16-bit var numbers may also overflow 16 bit line numbers.
These requires a change in the deep profiler data's binary
compatibility version number.
compiler/prog_rep.m:
Encode vars using four bytes if necessary. Be consistent in using
only signed 8-bit as well as signed 16-bit numbers.
compiler/implicit_parallelism.m:
Conform to the change in program_representation.m.
deep_profiler/profile.m:
deep_profiler/read_profile.m:
Add a compression flag to the set of flags read from the data file.
Put the flags into the profile_stats as a group, not one-by-one.
deep_profiler/canonical.m:
deep_profiler/create_report.m:
deep_profiler/dump.m:
deep_profiler/mdprof_feedback.m:
deep_profiler/old_html_format.m:
deep_profiler/old_query.m:
deep_profiler/query.m:
Conform to the change in profile.m.
runtime/mercury_deep_profiling.c:
Prepare for compression of profiling data files being implemented.
runtime/mercury_stack_layout.h:
Fix some documentation rot.
runtime/mercury_conf_param.h:
Add an implication between debug flags to make debugging easier.
The deep profiler associates measurements with their context in the call graph
modulo recursion. This has been true for all measurements except for coverage
profiling data. This patch allows coverage data to be associated with
ProcDynamic structures so that it is keyed by this context not just the static
procedure. This new behaviour is the default the old option of static coverage
profiling is still available for testing, as is no coverage profiling. Note
that, as before, coverage profiling is supported by default however coverage
points are not inserted by default.
This change will be used to measure the depth of recursion, and therefore the
average cost of recursion as well as the likely times when variables are
produced in calls for the automatic parallelisation analysis.
runtime/mercury_conf_param.h:
Create three new preprocessor macros:
MR_DEEP_PROFILING_COVERAGE - defined when coverage profiling is
enabled.
MR_DEEP_PROFILING_COVERAGE_STATIC - defined when static coverage
profiling is being used.
MR_DEEP_PROFILING_COVERAGE_DYNAMIC - defined when dynamic coverage
profiling is being used.
runtime/mercury_deep_profiling.h:
Update data structures to support dynamic coverage profiling.
Use conditional compilation to allow us to test the deep profiler in three
different modes, without coverage profiling, with static coverage profiling
and with dynamic coverage profiling.
Rename MR_PROFILING_MALLOC, since it takes a type rather than a size in
bytes it should be called MR_PROFILING_NEW to conform with existing malloc
and new functions.
runtime/mercury_deep_profiling.c:
Avoid a C compiler warning.
MR_write_out_coverage_point has been removed, it's replaced with:
MR_write_out_coverage_points_static and
MR_write_out_coverage_points_dynamic.
These write out more than one coverage point and write out either static or
dynamic coverage points.
Write a 64bit flags value (a bitfield) to the header of the Deep.data file.
This replaces the canonical byte (with a bit). and the byte that describes
the word size. This value also includes two bits describing the whether no
coverage data, static coverage data or dynamic coverage data is present in
the file. A bit is reserved ti indicate if the data is compressed (which
is not yet supported).
MR_write_fixed_size_int now writes out 8 byte integers, this is only used
for some counts present at the beginning of the data file along with the
new flags value. It now takes a MR_uint_least64_t integer as it's
parameter. The assertion to test for negative numbers has been removed
since this type is unsigned.
Increment the Deep.data file format version number.
compiler/layout_out.m:
Conditionally compile the NULL pointer that represents the coverage points
list in proc statics. This is conditional on the
MR_DEEP_PROFILING_COVERAGE_STATIC macro being defined.
compiler/coverage_profiling.m:
Add support for generating dynamic coverage points.
compiler/options.m:
compiler/handle_options.m:
Implement the new developer options for controlling coverage profiling.
library/profiling_builtin.m:
Specialize increment_coverage_point_count for both static and dynamic
coverage profiling. This creates
increment_{static,dynamic}_coverage_point_count.
deep_profiler/profile.m:
Add an extra field to profile_stats, this tracks whether the file reader
should try to read none, static or dynamic coverage data.
Add an extra field to proc_dynamic, An array of coverage counts wrapped by
a maybe type. It's indexed the same as the array of coverage infos in
proc_static. This array is present if dynamic coverage profiling is being
done (the default).
Modify the coverage_points field in proc static, now there are two fields,
an array of coverage_point_info values which store compile-time data. And
an optional array of coverage points (present if static coverage profiling
was performed).
Updated the formatting of the proc static structure.
Moved the coverage_point type to coverage.m.
Created a new type, coverage_data_type which enumerates the possibilities
for coverage profiling: none, static and dynamic.
deep_profiler/coverage.m:
Move the coverage point type here from profile.m, as the profile data
structure no longer refers to it directly.
Create a predicate coverage_point_arrays_to_list/3 which merges coverage
point information and the counts themselves into coverage points. This can
be used to construct a list of coverage points regardless of whether static
or dynamic coverage points are being used.
deep_profiler/read_profile.m:
Conform to changes in runtime/mercury_deep_profiling.c.
Refactored reading of the file header, a new named predicate is now used
rather than a lambda expression.
Incremented the Deep.data version number.
deep_profiler/report.m:
Updated the proc dynamic dump report structure to include a list of
coverage points.
deep_profiler/create_report.m:
deep_profiler/display_report.m:
Conform to changes in profile.m.
The proc dynamic dump now shows coverage information that was contained in
that proc dynamic.
deep_profiler/canonical.m:
deep_profiler/dump.m:
Conform to changes in profile.m.
deep_profiler/io_combinator.m:
Add a 13-arg version of maybe_error_sequence.
deep_profiler/Mercury.options:
Documented another trace flag.
Estimated hours taken: 6
Branches: main
Several improvements to make the deep profiler more convenient to use.
deep_profiler/mdprof_cgi.m:
If the data file does not end in .data, generate an explicit error
message. Without this, the deep profiler just hangs forever on such
filenames.
deep_profiler/query.m:
Add a couple more categories of user preferences. The first new
category controls whether, in reports for procedures and cliques,
the deep profiler will display call sites through which there were no
calls. The second new category controls whether the deep profiler
includes module qualifications in the names of procedures. Module
qualifying everything, as we did before, can clutter up the output,
so the new default is to module qualify only the names of procedures
that are in different modules from the module that defines the current
procedure.
deep_profiler/profile.m:
deep_profiler/report.m:
Remember module names and unqualified as well as qualified names
of procedures.
deep_profiler/create_report.m:
deep_profiler/read_profile.m:
Remember the newly needed information.
deep_profiler/display_report.m:
Implement the two new categories of preferences.
When generating the menu, make the "follow the action" links
more useful by implicitly preferring to (a) hide inactive call sites,
and (b) sort by overall time.
When sorting by time, if two times (clock ticks) are equal,
try to use call sequence numbers to resolve the order.
When generating the menu, print the total quanta as well as the total
runtime computed from it.
deep_profiler/dump.m:
deep_profiler/old_html_format.m:
deep_profiler/old_query.m:
Conform to the changes above.
Estimated hours taken: 7.5
Branches: main
Deep profiling and program representation file format changes.
Include a table of variable names in the program representation used by the
deep profiler (Deep.procrep). The table shouldn't contain variables
introduced by the compiler.
Include the name of the program from which the deep profile was generated in
the header of the deep profiling data file (Deep.data).
compiler/prog_rep.m:
Optionally create the variable table within the bytecode generated for
each procedure, The actual variable names are stored in the string table
for the module.
compiler/stack_layout.m:
Enable the variable table only for deep profiling builds.
mdbcomp/program_representation.m:
Include variable table in each program representation structure.
Create predicates to read in the variable table if it is present.
Create predicates to lookup a variable name from a table or generate a
name automatically.
Incremented version number for the program representation file format.
runtime/mercury_deep_profiling.c:
Write out program name in header of deep profiling data.
Increment version numbers of deep profiling and program representation
file formats.
browser/declarative_tree.m:
deep_profiler/canonical.m:
tests/debugger/declarative/builtin_call_rep.exp:
tests/debugger/declarative/dependency.exp:
Conform to changes in mdbcomp/program_representation.m
deep_profiler/mdprof_procrep.m:
Lookup and display variable names when displaying program representation.
deep_profiler/profile.m:
Make program name part of the profile_stats structure.
deep_profiler/read_profile.m:
deep_profiler/io_combinator.m:
Read in program name from deep profiling data.
Incremented version number for the deep profiling file format.
deep_profiler/report.m:
deep_profiler/create_report.m:
deep_profiler/display_report.m:
Display program name with the statistics on the menu report.
deep_profiler/dump.m
Conform to changes in deep_profiler/profile.m.
Estimated hours taken: 6
Branches: main
Convert Paul's new code to use cords of strings to represent HTML.
deep_profiler/html_format.m:
Convert to using cords. Restructure the code in a couple of places
to always put start and end tags around HTML fragments together.
Fix a missing "=" in a tag.
deep_profiler/interface.m:
deep_profiler/mdprof_cgi.m:
deep_profiler/read_profile.m:
Provide better diagnostics.
deep_profiler/create_report.m:
deep_profiler/display.m:
deep_profiler/display_report.m:
deep_profiler/mdprof_feedback.m:
deep_profiler/measurement_units.m:
deep_profiler/query.m:
deep_profiler/report.m:
mdbcomp/feedback.m:
Misc cleanups. They can be considered my post-commit review of Paul's
diff.
In mdprof_feedback.m, delete a strange test that prevented the program
from being used from the command line.
deep_profiler/dump.m:
deep_profiler/mdprof_dump.m:
deep_profiler/timeout.m:
deep_profiler/util.m:
Misc cleanups of old code.
Estimated Hours Taken: 100
Branches: main
Introduce coverage profiling. While regular profiling shows which procedures
are used the most, coverage profiling goes further, and also shows the most
common execution paths through each procedure. We intend coverage profiling
data to be used by a future automatic parallelization pass in the compiler.
However, it should also be useful for other purposes.
This diff adds a compiler option, --coverage-profiling, that, when specified,
adds program instrumentation to record execution counts at selected points
in procedure bodies. The implementation currently stores coverage profiling
information in ProcStatic structures. We will later investigate the impact
of storing this information in ProcDynamic structures instead.
For now coverage statistics can be viewed with the mdprof_dump tool.
compiler/deep_profiling.m:
Introduced coverage profiling transformation after deep profiling
transformation, it will be run if at least one coverage point type is
enabled.
compiler/goal_util.m:
Created create_conj_from_list/3, to create a conjunction from a list of
goals.
compiler/hlds_goal.m:
Added dp_goal_info structure that stores information relevant to the deep
profiling pass.
Added egi_maybe_dp field to extra_goal_info to store a dp_goal_info
structure when required by the deep profiler.
compiler/hlds_pred.m:
Added a list of coverage_point_info structures to the hlds_proc_static
structure.
compiler/hlds_out.m:
Added the ability to dump the dp_goal_info structure when the correct
option is given.
compiler/layout.m:
Added extra layout structures to store coverage point static and dynamic
data for each procedure.
compiler/layout_out.m:
Added code to write out new layout structures in the Low Level C
Backend.
Added code to write out references to coverage point data from the proc
static structures.
Conform to changes in layout.m.
compiler/opt_debug.m:
Conform to changes in layout.m,
compiler/options.m:
Added command line parameters to enable coverage profiling and different
coverage points, as well as options that can cause coverage points not
to be inserted in some circumstances.
compiler/handle_options.m:
Added hlds dump options to the 'all' aliases for dumping the new
dp_goal_info structure in the hlds_info.
deep_profiler/dump.m:
Modified to dump coverage profiling data read in from Deep.data file.
deep_profiler/profile.m:
Added coverage points to proc static structure.
deep_profiler/read_profile.m:
Incremented Deep.data format version number.
Read coverage points from Deep.data file.
Confirm to changes in profile.m.
library/profiling_builtin.m:
Added real declaration and dummy implementation of
increment_coverage_point_count/2. This represents the instrumentation
introduced by the coverage profiling transformation.
mdbcomp/program_representation.m:
Added types to support coverage profiling, including a foreign_enum for
cp_type.
Added coverage_point_type_c_value to convert a coverage point type to a
string representing it's C value.
runtime/mercury_deep_profiling.c:
Incremented Deep.data format version.
Implemented writing out of coverage points to the Deep.data file.
runtime/mercury_deep_profiling.h:
Modified runtime structures to support storing coverage point
information in proc static structures.
doc/user_guide.texi:
Documented coverage profiling options, however this is commented out since
it's experimental.
Estimated hours taken: 2
Branch: main
Improved mdprof_dump's command line parsing. It's behaviour is now more
obvious. Major changes include The "-D restrict" option is now "--restrict",
and is no-longer affected by other -D options. Silently ignored -D options
have now been documented as unimplemented.
deep_profiler/dump.m:
use a structured representation of command line parameters used to control
deep_dump/4 and deep_initial_dump/4.
deep_profiler/startup.m:
Conform to changes in dump.m, read_and_startup now takes a deep_options
type rather than a list of strings. A new version of read_and_startup has
been created that doesn't take any deep_options and assumes the defaults.
deep_profiler/mdprof_dump.m:
Modified command line options, The '-D' option is now only used to specify
what arrays to dump. A new option, '--restrict', controls whether to
print unreferenced call_site_statics and proc_statics.
deep_profiler/mdprof_cgi.m:
deep_profiler/mdprof_feedback.m:
deep_profiler/mdprof_test.m:
Conform to changes in startup.m, now makes use of dump.m.
Estimated hours taken: 20
Branches: main
Add a new compiler option. --inform-ite-instead-of-switch. If this is enabled,
the compiler will generate informational messages about if-then-elses that
it thinks should be converted to switches for the sake of program reliability.
Act on the output generated by this option.
compiler/simplify.m:
Implement the new option.
Fix an old bug that could cause us to generate warnings about code
that was OK in one duplicated copy but not in another (where a switch
arm's code is duplicated due to the case being selected for more than
one cons_id).
compiler/options.m:
Add the new option.
Add a way to test for the bug fix in simplify.
doc/user_guide.texi:
Document the new option.
NEWS:
Mention the new option.
library/*.m:
mdbcomp/*.m:
browser/*.m:
compiler/*.m:
deep_profiler/*.m:
Convert if-then-elses to switches at most of the sites suggested by the
new option. At the remaining sites, switching to switches would have
nontrivial downsides. This typically happens with the switched-on type
has many functors, and we treat one or two specially (e.g. cons/2 in
the cons_id type).
Perform misc cleanups in the vicinity of the if-then-else to switch
conversions.
In a few cases, improve the error messages generated.
compiler/accumulator.m:
compiler/hlds_goal.m:
(Rename and) move insts for particular kinds of goal from
accumulator.m to hlds_goal.m, to allow them to be used in other
modules. Using these insts allowed us to eliminate some if-then-elses
entirely.
compiler/exprn_aux.m:
Instead of fixing some if-then-elses, delete the predicates containing
them, since they aren't used, and (as pointed out by the new option)
would need considerable other fixing if they were ever needed again.
compiler/lp_rational.m:
Add prefixes to the names of the function symbols on some types,
since without those prefixes, it was hard to figure out what type
the switch corresponding to an old if-then-else was switching on.
tests/invalid/reserve_tag.err_exp:
Expect a new, improved error message.
Estimated hours taken: 50
Branches: main
Make a representation of the program available to the deep profiler. We do
this by letting the user request, via the option "--deep-procrep-file"
in MERCURY_OPTIONS, that when the Deep.data file is written, a Deep.procrep
file should be written alongside it.
The intended use of this information is the discovery of profitable
parallelism. When a conjunction contains two expensive calls, e.g. p(...) and
q(...) connected by some shared variables, the potential gain from executing
them in parallel is limited by how early p produces those variables and how
late q consumes them, and knowing this requires access to the code of p and q.
Since the debugger and the deep profiler both need access to program
representations, put the relevant data structures and the operations on them
in mdbcomp. The data structures are significantly expanded, since the deep
profiler deals with the whole program, while the debugger was interested only
in one procedure at a time.
The layout structures have to change as well. In a previous change, I changed
proc layout structures to make room for the procedure representation even in
non-debugging grades, but this isn't enough, since the procedure representation
refers to the module's string table. This diff therefore makes some parts of
the module layout structure, including of course the string table, also
available in non-debugging grades.
configure.in:
Check whether the installed compiler can process switches on foreign
enums correctly, since this diff depends on that.
runtime/mercury_stack_layout.[ch]:
runtime/mercury_types.h:
Add a new structure, MR_ModuleCommonLayout, that holds the part of
the module layout that is common to deep profiling and debugging.
runtime/mercury_deep_profiling.[ch]:
The old "deep profiling token" enum type was error prone, since at
each point in the data file, only a subset was applicable. This diff
breaks up the this enum into several enums, each consisting of the
choice applicable at a given point.
This also allows some of the resulting enums to be used in procrep
files.
Rename some enums and functions to avoid ambiguities, and in one case
to conform to our naming scheme.
Make write_out_proc_statics take a second argument. This is a FILE *
that (if not NULL) asks write_out_proc_statics to write the
representation of the current module to specified stream.
These module representations go into the middle part of the program
representation file. Add functions to write out the prologue and
epilogue of this file.
Write out procedure representations if this is requested.
Factor out some code that is now used in more than one place.
runtime/mercury_deep_profiling_hand.h:
Conform to the changes to mercury_deep_profiling.h.
runtime/mercury_builtin_types.c:
Pass the extra argument in the argument lists of invocations of
write_out_proc_statics.
runtime/mercury_trace_base.[ch]:
Conform to the name change from proc_rep to proc_defn_rep in mdbcomp.
runtime/mercury_grade.h:
Due to the change to layout structures, increment the binary
compatibility version numbers for both debug and deep profiling grades.
runtime/mercury_wrapper.[ch]:
Provide two new MERCURY_OPTION options. The first --deep-procrep-file,
allows the user to ask for the program representation to be generated.
The second, --deep-random-write, allows tools/bootcheck to request that
only a fraction of all program invocations should generate any deep
profiling output.
The first option will be documented once it is tested much more fully.
The second option is deliberately not documented.
Update the type of the variable that holds the address of the
(mkinit-generated) write_out_proc_statics function to accept the second
argument.
util/mkinit.c:
Pass the extra argument in the argument list of write_out_proc_statics.
mdbcomp/program_representation.m:
Extend the existing data structures for representing a procedure body
to represent a procedure (complete with name), a module and a program.
The name is implemented as string_proc_label, a form of proc_label that
can be written out to files. This replaces the old proc_id type the
deep profiler.
Extend the representation of switches to record the identity of the
variable being switched on, and the cons_ids of the arms. Without the
former, we cannot be sure when a variable is first used, and the latter
is needed for meaningful prettyprinting of procedure bodies.
Add code for reading in files of bytecodes, and for making sense of the
bytecodes themselves. (It is this code that uses foreign enums.)
mdbcomp/prim_data.m:
Note the relationship of proc_label with string_proc_label.
mdbcomp/rtti_access.m:
Add the access operations needed to find module string tables with the
new organization of layout structures.
Provide operations on bytecodes and string tables generally.
trace/mercury_trace_cmd_browsing.c:
Conform to the change to mdbcomp/program_representation.m.
compiler/layout.m:
Add support for a MR_ModuleCommonLayout.
Rename some function symbols to avoid ambiguities.
compiler/layout_out.m:
Handle the new structure.
compiler/stack_layout.m:
Generate the new structure and the procedure representation bytecode
in deep profiling grades.
compiler/llds_out.m:
Generate the code required to write out the prologue and epilogue
of program representation files.
Pass the extra argument in the argument lists of invocations of
write_out_proc_statics that tells those invocations to write out
the module representations between the prologue and the epilogue.
compiler/prog_rep.m:
When generating bytecodes, include the new information for switches.
compiler/continuation_info.m:
Replace a bool with a more expressive type.
compiler/proc_rep.m:
Conform to the change to continuation_info.m.
compiler/opt_debug.m:
Conform to the change to layout.m.
deep_profiler/mdprof_procrep.m:
A new test program to test the reading of program representations.
deep_profiler/DEEP_FLAGS.in:
deep_profiler/Mmakefile:
Copy the contents of the mdbcomp module to this directory on demand,
instead of linking to it. This is necessary now that the deep profiler
depends directly on mdbcomp even if it is compiled in a non-debugging
grade.
The arrangements for doing this were copied from the slice directory,
which has long done the same.
Avoid a duplicate include of Mmake.deep.params.
Add the new test program to the list of programs in this directory.
Mmakefile:
Go through deep_profiler/Mmakefile when deciding whether to do "mmake
depend" in the deep_profiler directory. The old actions won't work
correctly now that we need to copy some files from mdbcomp before we
can run "mmake depend".
deep_profiler/profile.m:
Remove the code that was moved (in cleaned-up form) to mdbcomp.
deep_profiler/dump.m:
deep_profiler/profile.m:
Conform to the changes above.
browser/declarative_execution.m:
browser/declarative_tree.m:
Conform to the changes in mdbcomp.
doc/user_guide.texi:
Add commented out documentation of the two new options.
slice/Mmakefile:
Fix formatting, and a bug.
library/exception.m:
library/par_builtin.m:
library/thread.m:
library/thread.semaphore.m:
Update all the handwritten modules to pass the extra argument now
required by write_out_proc_statics.
tests/debugger/declarative/dependency.exp:
Conform to the change from proc_rep to proc_defn_rep.
tools/bootcheck:
Write out deep profiling data only from every 25th invocation, since
otherwise the time for a bootcheck takes six times as long in deep
profiling grades than in asm_fast.gc.
However, do test the ability to write out program representations.
Use the mkinit from the workspace, not the installed one.
Don't disable line wrapping.
Estimated hours taken: 80
Branches: main
This change adds two new passes to the compiler. The first one,
implicit_parallelism, uses deep profiling feedback information, generated by
mdprof_feedback, to introduce parallel conjunctions where it could be
worthwhile. It deals with both independent and dependent parallelism.
The second new pass, distance_granularity, applies a transformation that
controls the granularity of parallelism for recursive procedures using the
distance metric.
This change also fixes a bug in mdprof_feedback regarding the construction of
the list of CSSs.
compiler/implicit_parallelism.m:
New module which uses the profiling feedback file generated by
mdprof_feedback to introduce parallel conjunction where it could be
useful.
compiler/distance_granularity.m:
New module. A program transformation that implements granularity control
of parallel execution using the distance metric.
compiler/dep_par_conj.m:
Moved find_shared_variables into the interface (needed for
implicit_parallelism.m).
compiler/goal_util.m:
Add two new predicates: flatten_conj and create_conj.
compiler/hhf.m:
Delete flatten_conj and use the one of goal_util instead.
compiler/hlds_pred.m:
Add a predicate to set the arity of a predicate (needed for
distance_granularity).
compiler/mercury_compile.m:
Add the calls to apply implicit parallelism and to control granularity
using the distance metric.
compiler/options:
Add implicit-parallelism, feedback-file and distance-granularity options.
compiler/pred_table.m:
Add a predicate to get the next pred_id available (needed for
distance_granularity).
compiler/prog_util.m:
Extend the predicate make_pred_name and the type new_pred_id for
creating a predicate name for distance_granularity.
compiler/transform_hlds.m:
Include implicit_parallelism and distance_granularity.
deep_profiler/mdprof_feedback.m:
Rename distribution to measure.
Add handling of dump_stages and dump_options.
Insert elements into the list of CSSs in the correct order.
deep_profiler/dump.m:
Add "all" option to dump everything out of the Deep.data file.
doc/user_guide.texi:
Add the following options: --distance-granularity, --implicit-parallelism and
--feedback-file.
tests/par_conj/Mercury.options:
tests/par_conj/dg_fib.{m,exp}:
tests/par_conj/dg_fib_func.{m,exp}:
Add two test cases for the distance_granularity module:dg_fib and
dg_fib_func. As things are, we do not check whether the granularity
control transformation using the distance metric is applied correctly or
not. We only check the output of these test cases.
Estimated hours taken: 3
Branches: main
Clean up in unused module imports in the Mercury system detected
by --warn-unused-imports.
analysis/*.m:
browser/*.m:
deep_profiler/*.m:
compiler/*.m:
library/*.m:
mdbcomp/*.m:
profiler/*.m:
slice/*.m:
Remove unused module imports.
Fix some minor departures from our coding standards.
analysis/Mercury.options:
browser/Mercury.options:
deep_profiler/Mercury.options:
compiler/Mercury.options:
library/Mercury.options:
mdbcomp/Mercury.options:
profiler/Mercury.options:
slice/Mercury.options:
Set --no-warn-unused-imports for those modules that are used as
packages or otherwise break --warn-unused-imports, e.g. because they
contain predicates with both foreign and Mercury clauses and some of
the imports only depend on the latter.
Estimated hours taken: 16
Branches: main
Include call sequence numbers in the pages generated by the deep profiler
(if the user's preferences ask for this).
NEWS:
Announce the new capability.
deep_profiler/measurements.m:
Standardize on one spelling of call sequence numbers (callseqs).
Add some utility predicates for use in dump.m.
Fix a bug in the calculation of the number of calls: redos enter
procedure invocations, not leave them.
deep_profiler/profile.m:
Fix a bug: the num_callseqs field was in the wrong place.
deep_profiler/html_format.m:
When generating a line of the clique description, include call sequence
number information if requested.
Do likewise when generating other HTML fragments that have to match
the contents of those lines (e.g. the table headers).
Generate the links that allow the users to sort rows by call
sequence numbers.
Generate the links that allow the users to toggle the switches
controlling the display of call sequence number information.
Do not show the toggles relevant to the display of times if times are
not being displayed.
deep_profiler/interface.m:
Extend the preferences to allow requests for call sequence number
information.
Include the display of call sequence numbers in the default set of
preferences. However, include the display of time information in the
default only of the profiling run lasted long enough for this
information to have some hope of being useful.
Rename some function symbols to avoid ambiguities.
deep_profiler/query.m:
Extend the menu to include links for the most expensive
cliques/procedures as measured by call sequence numbers.
Include the links for the most expensive cliques/procedures as
measured by clock ticks in the menu only if the profiling run
lasted long enough for this to be useful.
Print out the total number of call sequence numbers with the other
statistics.
Use call sequence numbers instead of time to decide where the "action"
begins, since this works even for short profiling runs.
deep_profiler/top_procs.m:
Provide the capability of sorting on call sequence numbers.
deep_profiler/dump.m:
Provide the capability to dump out several more fields of the top level
structure (profile.deep), including the cliques, the reverse links and
the propagated measurement information.
deep_profiler/startup.m:
Dump out the cliques, the reverse links and the propagated measurement
information when computed.
deep_profiler/mdprof_cgi.m:
Add an option for setting the hostname in the generated links to
"localhost". This is useful on laptops temporarily disconnected
from the Domain Name System (e.g. my laptop).
Set this option automatically if the name of the file in the initial
query has a ".localhost" suffix added to it.
Add some tracing hooks.
deep_profiler/mdprof_test.m:
Add an option to specify what info to dump during startup.
deep_profiler/mdprof_dump.m:
Switch to using the compiler's method of specifying what to dump,
since this is more maintainable.
deep_profiler/exclude.m:
Rename some function symbols to avoid ambiguities.
deep_profiler/timeout.m:
Add conditionally compiled code to help debug lock management.
deep_profiler/callgraph.m:
deep_profiler/canonical.m:
deep_profiler/cliques.m:
deep_profiler/conf.m:
deep_profiler/read_profile.m:
Minor cleanups.
Estimated hours taken: 1
Branches: main
Include the count of call sequence numbers in the Deep.data file, since the
deep profiler needs to know this to generate percentages.
runtime/mercury_deep_profiling.c:
Write the count of call sequence numbers to the Deep.data file.
deep_profiler/profile.m:
Add a field to the profile for holding the count of call sequence
numbers.
deep_profiler/read_profile.m:
Read in the count of call sequence numbers from the Deep.data file.
deep_profiler/io_combinator.m:
Add some functions now required by read_profile.m.
Convert from DCGs to state variables.
deep_profiler/canonical.m:
deep_profiler/dump.m:
Conform to the change to profile.m.
Estimated hours taken: 0.5
Branches: main
Enhance the deep profiler by including in the output a measure of execution
time that has much finer granularity than clock ticks: a count of procedure
calls. The deep profiler does not yet display this data; that is for a later
diff.
runtime/mercury_deep_profiling.[ch]:
Add a field containing this counter if the appropriate macro is
defined. Initialize the new field when the profiling metrics structure
it is in is initialized.
Use masks that fit into one byte for the commonly needed fields
of the profiling metrics structure.
runtime/mercury_conf_params.h:
Define this macro in deep profiling grades unless its absence is
expressly requested (for performance tests of the deep profiler
itself).
runtime/mercury_grade.h:
Increment the binary compatibility number for the deep profiling
grades.
runtime/mercury_deep_{call,redo,leave}_port_body.h:
Update the call sequence number global at call sites.
Update the new field when control enters a procedure invocation
through the call or redo ports, and when it leaves the invocation
through the exit or fail ports.
library/exception.m:
Update the new field when control leaves a procedure invocation
through the excp port.
deep_profiler/measurements.m:
Include the new field in the deep profiler's version of the
profiling metric structure.
deep_profiler/read_profile.m:
Read in the updated format now generated by the runtime.
deep_profiler/dump.m:
Include the new field in any dumps of the deep profiler's
internal data structures.
Estimated hours taken: 0.5
Branches: main
profiler/*.m:
deep_profiler/*.m:
compiler/*.m:
Replace __ with . as the module qualifier everywhere in all the modules
of the profiler and deep profiler and in some modules of the compiler.
Estimated hours taken: 1.5
Branches: main
Convert the rest of the deep profiler to four-space indentation.
There are no changes to any algorithms (other than introducing state
variables in a few spots).
Remove an old bug workaround.
deep_profiler/*.m:
Convert to four-space indentation where that has not already
been done.
Fix some bad line breaks.
Use state variables in a few places.
Reposition comments according to our current coding standard.
deep_profiler/Mercury.options:
Remove some old bug workarounds.
Estimated hours taken: 1
Branches: main
deep_profiler/dump.m:
Complete the '--restrict-statics' option for the mdprof_dump
tool so that it also includes call site statics.
Estimated hours taken: 8
Branches: main
Significantly enhance the infrastructure for debugging problems with the deep
profiler by providing facilities for dumping out some of the main data
structures. These were needed for my recent fix of mercury_deep_profiling.c
in the runtime.
There are no algorithmic changes in the non-debugging parts of the code.
deep_profiler/dump.m:
New module for containing predicates that dump data structures.
The first draft was by Julien.
deep_profiler/mdprof_dump.m:
New program for dumping out the contents of Deep.data files.
The first draft was by Julien.
deep_profiler/startup.m:
Provide a mechanism for dumping out the main data structures at three
stages.
deep_profiler/mdprof_cgi.m:
deep_profiler/mdprof_test.m:
Pass the options to startup to control these dumps.
deep_profiler/measurements.m:
Provide a mechanism for getting all the counts of a measurement at
once.
deep_profiler/array_util.m:
Provide extra modes of some predicates for use in the other modules.
deep_profiler/read_profile.m:
Fix Julien's fix: handle init predicates as well as the other uci
predicates.
deep_profiler/profile.m:
Reorder the arguments of predicates as required to enable the use
of state variable notation in the callers.
deep_profiler/*.m:
Conform to the change in profile.m, and use state variable notation
where this is advantageous.
Convert some modules to four-space indentation where this eliminates
excessive indentation or bad line breaks.
deep_profiler/Mmakefile:
Handle the new mdprof_dump program.
