configure.ac:
Require the installed compiler to support the new attribute.
compiler/ctgc.selector.m:
deep_profiler/query.m:
Use the attribute to shut up warnings in non-C grades.
library/math.m:
library/exception.m:
Move the domain_error type from math.m to exception.m.
NEWS:
Announce the change as a potentially breaking change.
deep_profiler/query.m:
library/float.m:
library/int.m:
library/int16.m:
library/int32.m:
library/int64.m:
library/int8.m:
library/integer.m:
library/uint.m:
library/uint16.m:
library/uint32.m:
library/uint64.m:
library/uint8.m:
Replace all references to math.domain_error with just domain_error.
Delete imports of math.m where this was the only reason for the import.
browser/*.m:
compiler/*.m:
deep_profiler/*.m:
library/*.m:
mdbcomp/*.m:
ssdb/*.m:
Specify the type constructor for every inst definition that lists
the functors that values of that type may be bound to.
In library/maybe.m, delete the inst maybe_errors/1, because
(a) its name is misleading, since it is for the maybe_error/1 type (no s),
and (b) there is already an inst with the non-misleading name maybe_error
which had an identical definition.
In compiler/dep_par_conj.m, delete two insts that were duplicates
of insts defined in hlds_goal.m, and replace references to them
accordingly.
deep_profiler/*.m:
Fix inconsistencies between (a) the order in which functions and predicates
are declared, and (b) the order in which they are defined.
In most modules, either the order of the declarations or the order
of the definitions made sense, and I changed the other to match.
In some modules, neither made sense, so I changed *both* to an order
that *does* make sense (i.e. it has related predicates together).
In query.m, put the various commands in the same sensible order
as the code processing them.
In html_format.m, merge two exported functions together, since
they can't be used separately.
In some places, put dividers between groups of related
functions/predicates, to make the groups themselves more visible.
In some places, fix comments or programming style.
deep_profiler/DEEP_FLAGS.in:
Since all the modules in this directory are now free from any warnings
generated by --warn-inconsistent-pred-order-clauses, specify that option
by default in this directory to keep it that way.
If a module has two or more import_module or use_module declarations
for the same module, (typically, but not always, one being in its interface
and one in its implementation), generate an informational message about
each redundant declaration if --warn-unused-imports is enabled.
compiler/hlds_module.m:
We used to record the set of imported/used modules, and the set of
modules imported/used in the interface of the current module. However,
these sets
- did not record the distinction between imports and uses;
- did not allow distinction between single and multiple imports/uses;
- did not record the locations of the imports/uses.
The first distinction was needed only by module_qual.m, which *did*
pay attention to it; the other two were not needed at all.
To generate messages for imports/uses shadowing other imports/uses,
we need all three, so change the data structure storing such information
for *direct* imports to one that records all three of the above kinds
of information. (For imports made by read-in interface and optimization
files, the old set of modules approach is fine, and this diff leaves
the set of thus *indirectly* imported module names alone.)
compiler/unused_imports.m:
Use the extra information now available to generate a
severity_informational message about any import or use that is made
redundant by an earlier, more general import or use.
Fix two bugs in the code that generated warnings for just plain unused
modules.
(1) It did not consider that a use of the builtin type char justified
an import of char.m, but without that import, the type is not visible.
(2) It scanned cons_ids in goals in procedure bodies, but did not scan
cons_ids that have been put into the const_struct_db. (I did not update
the code here when I added the const_struct_db.)
Also, add a (hopefully temporary) workaround for a bug in
make_hlds_passes.m, which is noted below.
However, there are at least three problems that prevent us from enabling
--warn-unused-imports by default.
(1) In some places, the import of a module is used only by clauses for
a predicate that also has foreign procs. When compiled in a grade that
selects one of those foreign_procs as the implementation of the predicate,
the clauses are discarded *without* being added to the HLDS at all.
This leads unused_imports.m to generate an uncalled-for warning in such
cases. To fix this, we would need to preserve the Mercury clauses for
*all* predicates, even those with foreign procs, and do all the semantic
checks on them before throwing them away. (I tried to do this once, and
failed, but the task should be easier after the item list change.)
(2) We have two pieces of code to generate import warnings. The one in
unused_imports.m operates on the HLDS after type and mode checking,
while module_qual.m operates on the parse tree before the creation of
the HLDS. The former is more powerful, since it knows e.g. what types and
modes are used in the bodies of predicates, and hence can generate warnings
about an import being unused *anywhere* in a module, as opposed to just
unused in its interface.
If --warn-unused-imports is enabled, we will get two separate set of
reports about an interface import being unused in the interface,
*unless* we get a type or mode error, in which case unused_imports.m
won't be invoked. But in case we do get such errors, we don't want to
throw away the warnings from module_qual.m. We could store them and
throw them away only after we know we won't need them, or just get
the two modules to generate identical error_specs for each warning,
so that the sort_and_remove_dups of the error specs will do the
throwing away for us for free, if we get that far.
(3) The valid/bug100.m test case was added as a regression test for a bug
that was fixed in module_qual.m. However the bug is still present in
unused_imports.m.
compiler/make_hlds_passes.m:
Give hlds_module.m the extra information it now needs for each item_avail.
Add an XXX for a bug that cannot be fixed right now: the setting of
the status of abstract instances to abstract_imported. (The "abstract"
part is correct; the "imported" part may not be.)
compiler/intermod.m:
compiler/try_expand.m:
compiler/xml_documentation.m:
Conform to the change in hlds_module.m.
compiler/module_qual.m:
Update the documentation of the relationship of this module
with unused_imports.m.
compiler/hlds_data.m:
Document a problem with the status of instance definitions.
compiler/hlds_out_module.m:
Update the code that prints out the module_info to conform to the change
to hlds_module.m.
Print status information about instances, which was needed to diagnose
one of the bugs in unused_imports.m. Format the output for instances
nicer.
compiler/prog_item.m:
Add a convenience predicate.
compiler/prog_data.m:
Remove a type synonym that makes things harder to understand, not easier.
compiler/modules.m:
Delete an XXX that asks for the feature this diff implements.
Add another XXX about how that feature could be improved.
compiler/Mercury.options.m:
Add some more modules to the list of modules on which the compiler
should be invoked with --no-warn-unused-imports.
compiler/*.m:
library/*.m:
mdbcomp/*.m:
browser/*.m:
deep_profiler/*.m:
mfilterjavac/*.m:
Delete unneeded imports. Many of these shadow other imports, and some
are just plain unneeded, as shown by --warn-unused-imports. In a few
modules, there were a *lot* of unneeded imports, but most had just
one or two.
In a few cases, removing an import from a module, because it *itself*
does not need it, required adding that same import to those of its
submodules which *do* need it.
In a few cases, conform to other changes above.
tests/invalid/Mercury.options:
Test the generation of messages about import shadowing on the existing
import_in_parent.m test case (although it was also tested very thoroughly
when giving me the information needed for the deletion of all the
unneeded imports above).
tests/*/*.{m,*exp}:
Delete unneeded imports, and update any expected error messages
to expect the now-smaller line numbers.
deep_profiler/*.m:
Replace ( C -> T ; E ) if-then-elses with (if C then T else E ).
Replace calls to error/1 with calls to unexpected/3.
Add some module qualifications where this makes the code easier to read.
runtime/mercury_goto.h:
Cast the "const MR_LabelLayout *" argument of calls
MR_insert_internal_label, since without this cast, we get C compiler
warnings in the bootcheck output when the caller passes a
"const MR_LabelLayoutNoVarInfo *" for this argument.
tests/hard_coded/Mmakefile:
Redirect the output of the weak_ptr test to a file instead of including it
in the output of bootcheck itself. The output is maybe be needed in a rare
cases, but is not needed most of the time.
tests/hard_coded/weak_ptr.m:
Fix white space.
tests/feedback/Mmakefile:
Specify verbosity level 0 when invoking mdprof_create_feedback.
deep_profiling/mdprof_create_feedback.m:
Record the verbosity level when we postprocess the options. We used to
record it *after* creating the feedback file, and before printing the
messages gathered during this process. This *mostly* worked, but some
messages are printed using trace goals *during* the creation of the
feedback file, and these ignored the specified verbosity level.
deep_profiling/query.m:
Remove the IO arguments from exec and try_exec, since they do not use them.
deep_profiling/var_use_analysis.m:
Eliminate some ambiguities by renaming two predicates.
Simplify some overly complex code.
deep_profiling/autopar_costs.m:
deep_profiling/create_report.m:
deep_profiling/mdprof_cgi.m:
deep_profiling/mdprof_test.m:
Conform to the above changes.
mdbcomp/feedback.m:
Fix a typo.
Fix some other issues as well that I found while fixing the spelling.
mdbcomp/feedback.automatic_parallelism.m:
deep_profiler/autopar_find_best_par.m:
deep_profiler/mdprof_create_feedback.m:
Rename the best_par_algorithm type to alg_for_finding_best_par,
since the old name was misleading. Perform the same rename for
another type based on it, and the option specifying it.
Remove the functor estimate_speedup_by_num_vars, since it hasn't
been used by anything in a long time, and won't in the future.
deep_profiler/autopar_calc_overlap.m:
deep_profiler/autopar_costs.m:
deep_profiler/autopar_reports.m:
deep_profiler/autopar_search_callgraph.m:
deep_profiler/autopar_search_goals.m:
deep_profiler/coverage.m:
deep_profiler/create_report.m:
deep_profiler/dump.m:
deep_profiler/mdprof_report_feedback.m:
deep_profiler/measurement_units.m:
deep_profiler/measurements.m:
deep_profiler/message.m:
deep_profiler/query.m:
deep_profiler/recursion_patterns.m:
deep_profiler/report.m:
deep_profiler/startup.m:
deep_profiler/var_use_analysis.m:
mdbcomp/mdbcomp.goal_path.m:
mdbcomp/program_representation.m:
Conform to the above. Fix spelling errors. In some places, improve
comments and/or variable names.
Estimated hours taken: 120
Branches: main
The algorithm that decides whether the order independent state update
transformation is applicable in a given module needs access to the list
of oisu pragmas in that module, and to information about the types
of variables in the procedures named in those pragmas. This diff
puts this information in Deep.procrep files, to make them available
to the autoparallelization feedback program, to which that algorithm
will later be added.
Compilers that have this diff will generate Deep.procrep files in a new,
slightly different format, but the deep profiler will be able to read
Deep.procrep files not just in the new format, but in the old format as well.
runtime/mercury_stack_layout.h:
Add to module layout structures the fields holding the new information
we want to put into Deep.procrep files. This means three things:
- a bytecode array in module layout structures encoding the list
of oisu pragmas in the module;
- additions to the bytecode arrays in procedure layout structures
mapping the procedure's variables to their types; and
- a bytecode array containing the encoded versions of those types
themselves in the module layout structure. This allows us to
represent each type used in the module just once.
Since there is now information in module layout structures that
is needed only for deep profiling, as well as information that is
needed only for debugging, the old arrangement that split a module's
information between two structures, MR_ModuleLayout (debug specific
info) and MR_ModuleCommonLayout (info used by both debugging and
profiling), is no longer approriate. We could add a third structure
containing profiling-specific info, but it is simpler to move
all the info into just one structure, some of whose fields
may not be used. This wastes only a few words of memory per module,
but allows the runtime system to avoid unnecessary indirections.
runtime/mercury_types.h:
Remove the type synonym for the deleted type.
runtime/mercury_grade.h:
The change in mercury_stack_layout.h destroys binary compatibility
with previous versions of Mercury for debug and deep profiling grades,
so bump their grade-component-specific version numbers.
runtime/mercury_deep_profiling.c:
Write out the information in the new fields in module layout
structures, if they are filled in.
Since this changes the format of the Deep.procrep file, bump
its version number.
runtime/mercury_deep_profiling.h:
runtime/mercury_stack_layout.c:
Conform to the change to mercury_stack_layout.h.
mdbcomp/program_representation.m:
Add to module representations information about the oisu pragmas
defined in that module, and the type table of the module.
Optionally add to procedure representations a map mapping
the variables of the procedure to their types.
Rename the old var_table type to be the var_name_table type,
since it contains just names. Make the var to type map separate,
since it will be there only for selected procedures.
Modify the predicates reading in module and procedure representations
to allow them to read in the new representation, while still accepting
the old one. Use the version number in the Deep.procrep file to decide
which format to expect.
mdbcomp/rtti_access.m:
Add functions to encode the data representations that this module
also decodes.
Conform to the changes above.
mdbcomp/feedback.automatic_parallelism.m:
Conform the changes above.
mdbcomp/prim_data.m:
Fix layout.
compiler/layout.m:
Update the compiler's representation of layout structures
to conform to the change to runtime/mercury_stack_layout.h.
compiler/layout_out.m:
Output the new parts of module layout structures.
compiler/opt_debug.m:
Allow the debugging of code referring to the new parts of
module layout structures.
compiler/llds_out_file.m:
Conform to the move to a single module layout structure.
compiler/prog_rep_tables.m:
This new module provided mechanisms for building the string table
and the type table components of module layouts. The string table
part is old (it is moved here from stack_layout.m); the type table
part is new.
Putting this code in a module of its own allows us to remove
a circular dependency between prog_rep.m and stack_layout.m;
instead, both now just depend on prog_rep_tables.m.
compiler/ll_backend.m:
Add the new module.
compiler/notes/compiler_design.html:
Describe the new module.
compiler/prog_rep.m:
When generating the representation of a module for deep profiling,
include the information needed by the order independent state update
analysis: the list of oisu pragmas in the module, if any, and
information about the types of variables in selected procedures.
To avoid having these additions increasing the size of the bytecode
representation too much, convert some fixed 32 bit numbers in the
bytecode to use variable sized numbers, which will usually be 8 or 16
bits.
Do not use predicates from bytecode_gen.m to encode numbers,
since there is nothing keeping these in sync with the code that
reads them in mdbcomp/program_representation.m. Instead, use
new predicates in program_representation.m itself.
compiler/stack_layout.m:
Generate the new parts of module layouts.
Remove the code moved to prog_rep_tables.m.
compiler/continuation_info.m:
compiler/proc_gen.m:
Make some more information available to stack_layout.m.
compiler/prog_data.m:
Fix some formatting.
compiler/introduce_parallelism.m:
Conform to the renaming of the var_table type.
compiler/follow_code.m:
Fix the bug that used to cause the failure of the
hard_coded/mode_check_clauses test case in deep profiling grades.
deep_profiler/program_representation_utils.m:
Output the new parts of module and procedure representations,
to allow the correctness of this change to be tested.
deep_profiler/mdprof_create_feedback.m:
If we cannot read the Deep.procrep file, print a single error message
and exit, instead of continuing with an analysis that will generate
a whole bunch of error messages, one for each attempt to access
a procedure's representation.
deep_profiler/mdprof_procrep.m:
Give this program an option that specifies what file it is to
look at; do not hardwire in "Deep.procrep" in the current directory.
deep_profiler/report.m:
Add a report type that just prints the representation of a module.
It returns the same information as mdprof_procrep, but from within
the deep profiler, which can be more convenient.
deep_profiler/create_report.m:
deep_profiler/display_report.m:
Respectively create and display the new report type.
deep_profiler/query.m:
Recognize a query asking for the new report type.
deep_profiler/autopar_calc_overlap.m:
deep_profiler/autopar_find_best_par.m:
deep_profiler/autopar_reports.m:
deep_profiler/autopar_search_callgraph.m:
deep_profiler/autopar_search_goals.m:
deep_profiler/autopar_types.m:
deep_profiler/branch_and_bound.m:
deep_profiler/coverage.m:
deep_profiler/display.m:
deep_profiler/html_format.m:
deep_profiler/mdprof_test.m:
deep_profiler/measurements.m:
deep_profiler/query.m:
deep_profiler/read_profile.m:
deep_profiler/recursion_patterns.m:
deep_profiler/top_procs.m:
deep_profiler/top_procs.m:
Conform to the changes above.
Fix layout.
tests/debugger/declarative/dependency.exp2:
Add this file as a possible expected output. It contains the new
field added to module representations.
Estimated hours taken: 2
deep_profiler/autopar_calc_overlap.m:
New module for calculating the overlap between the conjuncts of a
parallelised conjunction. Its contents are taken from the old
autopar_search_callgraph.m.
deep_profiler/autopar_costs.m:
New module for calculating the costs of goals. Its contents
are taken from the old autopar_search_callgraph.m.
deep_profiler/autopar_reports.m:
New module for creating reports. Its contents are taken from
the old autopar_search_callgraph.m.
deep_profiler/autopar_search_goals.m:
New module for searching goals for parallelizable conjunctions.
Its contents are taken from the old autopar_search_callgraph.m.
deep_profiler/autopar_search_callgraph.m:
Remove the code moved to other modules.
deep_profiler/mdprof_fb.automatic_parallelism.m:
Add the new modules.
deep_profiler/*.m:
Remove unnecessary imports.
Fix copyright years on the new modules.
browser/*.m:
compiler/*.m:
mdbcomp/*.m:
Remove unnecessary imports.
library/Mercury.options:
Make it possible to compile a whole workspace with
--warn-unused-imports by turning that option off for type_desc.m
(which has a necessary import that --warn-unused-imports thinks
is unused).
Estimated hours taken: 0.1
Branches: main
deep_profiler/old_query.m:
deep_profiler/old_html_format.m:
Remove these modules. We have used their replacements without trouble
for a long while now.
deep_profiler/query.m:
Remove the code that used to invoke those modules.
Estimated hours taken: 2
Branches: main
Add the predicates sorry, unexpected and expect to library/error.m.
compiler/compiler_util.m:
library/error.m:
Move the predicates sorry, unexpected and expect from compiler_util
to error.
Put the predicates in error.m into the same order as their
declarations.
compiler/*.m:
Change imports as needed.
compiler/lp.m:
compiler/lp_rational.m:
Change imports as needed, and some minor cleanups.
deep_profiler/*.m:
Switch to using the new library predicates, instead of calling error
directly. Some other minor cleanups.
NEWS:
Mention the new predicates in the standard library.
Estimated hours taken: 12
Branches: main
Allow more than 2^15 vars in a procedure representation.
mdbcomp/program_representation.m:
Allow a variable number to be represented by four bytes as well as
two and one. This means that we also have to represent the number
of variables in a procedure using a four-byte number, not a two-byte
number.
Use four bytes to represent line numbers. Programs that overflow
16-bit var numbers may also overflow 16 bit line numbers.
These requires a change in the deep profiler data's binary
compatibility version number.
compiler/prog_rep.m:
Encode vars using four bytes if necessary. Be consistent in using
only signed 8-bit as well as signed 16-bit numbers.
compiler/implicit_parallelism.m:
Conform to the change in program_representation.m.
deep_profiler/profile.m:
deep_profiler/read_profile.m:
Add a compression flag to the set of flags read from the data file.
Put the flags into the profile_stats as a group, not one-by-one.
deep_profiler/canonical.m:
deep_profiler/create_report.m:
deep_profiler/dump.m:
deep_profiler/mdprof_feedback.m:
deep_profiler/old_html_format.m:
deep_profiler/old_query.m:
deep_profiler/query.m:
Conform to the change in profile.m.
runtime/mercury_deep_profiling.c:
Prepare for compression of profiling data files being implemented.
runtime/mercury_stack_layout.h:
Fix some documentation rot.
runtime/mercury_conf_param.h:
Add an implication between debug flags to make debugging easier.
This patch fixes various problems, the most significant is the calculation of
variable use information. The parallelisation analysis uses deep profiling
data. In other words, profiling data that is attached to context information
referring to not just the procedure but the chain of calls leading to that
invocation of that procedure (modulo recursion). The variable use analysis did
not use deep profiling data, therefore comparing the time that a variable is
produced with a call to the time in total of that call was not sound, and
sometimes resulted in information that is not possible, such as a variable
being produced or consumed after the call that produces or consumes it has
exited.
This change-set updates the variable use analysis to use deep profiling data to
avoid these problems. At the same time it provides more accurate information
to the automatic parallelisation pass. This is possible because of an earlier
change that allowed the coverage data to use deep profiling data.
In its current state, the parallelisation analysis now finishes without errors
and computes meaningful results when analysing a profile of the mercury
compiler's execution.
deep_profiler/report.m:
The proc var use report is now a call site dynamic var use report.
1) It now uses deep profiling data.
2) It makes more sense from the callers perspective so it's now based
around a call site rather than a proc.
Add inst subtypes to the recursion_type type.
deep_profiler/query.m:
The proc var use query is now a call site dynamic var use query, see
report.m.
deep_profiler/var_use_analysis.m:
Fix a bug here and in mdprof_fb.automatic_parallelism.m: If a
variable is consumed by a call and appears in it's argument list more than
once, take the earliest consumption time rather than the one for the
earliest argument.
Variable use analysis now uses recursion_patterns.m to correctly compute
the cost of recursive calls. It also uses 'deep' profiler data.
Only measure variable use relative to the entry into a procedure, rather
than either relative to the entry or exit. This allows us to simplify a
lot of code.
deep_profiler/create_report.m:
The proc var use info report is now a call site dynamic var use info
report.
Move some utility code from here to the new analysis_utils.m module.
deep_profiler/display_report.m:
Conform to changes in report.m.
Improve the information displayed for variable first-use time
reports.
deep_profiler/mdprof_fb.automatic_parallelism.m:
Conform to changes in report.m
Refactored the walk down the clique tree. This no-longer uses the
clique reports from the deep profiling tool.
We now explore the same static procedure more than once. It may be best to
parallelise it in some contexts rather than others but for now we assume
that the benefits in some context are worth the costs without benefit in
the other contexts. This is better than reaching a context where it is
undesirable first and never visiting a case where parallelisation is
desirable.
Fix a bug in the calculation of how much parallelisation is used by
parallelisations in a clique's parents. This used to trigger an
assertion.
Don't try to parallelize anything in the "exception" module.
There's probably other builtin code we should skip over here.
Removed an overzealous assertion that was too easily triggered by the
inaccuracies of IEEE-754 arithmetic.
Compute variable use information lazily for each variable in each call. I
believe that this has made our implementation much faster as it no-longer
computes information that is never used.
Refactor and move build_recursive_call_site_cost_map to the new
module analysis_utils.m where it can be used by other analyses.
Call site cost maps now use the cs_cost_csq type to store costs,
code in this module now conforms to this change.
Conform to changes in messages.m
deep_profiler/recursion_patterns.m:
Export a new predicate, recursion_type_get_maybe_avg_max_depth/2. This
retrieves the average maximum recursion depth from recursion types that know
this information.
Move code that builds a call site cost map for a procedure to
analysis_utils.m where it can be used by other analyses.
deep_profiler/analysis_utils.m:
Added a new module containing various utility predicates for profile
analysis.
deep_profiler/coverage.m:
Added an extra utility predicate get_coverage_after/2.
deep_profiler/message.m:
Each message has a location that it refers to, a new location type has
been added: call_site_dynamic.
Added a new warning that can be used to describe when a call site's
argument's use time cannot be computed.
Added new predicates for printing out messages whose level is below a
certain threshold. These predicates can be called from io trace goals.
Message levels start at 0 and currently go to 4, more critical messages
have lower levels. The desired verbosity level is stored in a module local
mutable.
deep_profiler/mdprof_feedback.m:
Move the message printing code from here to message.m.
deep_profiler/old_html_format.m:
deep_profiler/old_query.m:
Conform to changes in query.m.
mdbcomp/feedback.automatic_parallelism.m:
Added a new function for computing the 'cpu time' of a parallel
computation.
library/lazy.m:
Moved lazy.m from extras to the standard library.
library/list.m:
Add a new predicate member_index0/3. Like member/2 except it also gives
the zero-based index of the current element within the list.
library/maybe.m:
Add two new insts.
maybe_yes(I) for the maybe type's yes/1 constructor.
maybe_error_ok(I) for the maybe_error type's ok/1 constructor.
library/Mercury.options:
Add a work around for compiling lazy.m with intermodule optimisations.
NEWS:
Update news file for the addition of lazy.m and the member_index0 predicate
in list.m
deep_profiler/.cvsignore:
Ignore feedback.automatic_parallelism.m which is copied by Mmakefile from
the mdbcomp/ directory.
This patch separates the coverage annotated procedure report into two
reports, one with dynamic coverage data and one with static coverage data.
This restores the functionality of the static coverage report since my last
change, and provides access to the dynamic report via new controls only
visible to developers.
deep_profiler/query.m:
In the cmd data-type:
Rename deep_cmd_procrep_coverage constructor to
deep_cmd_static_procrep_coverage.
Add deep_cmd_procrep_dynamic_coverage.
In the preferences data-type:
Add a new field pref_developer_mode which indicates if developer-only
options are visible or not.
Add code to parse and print the new command and preference option.
deep_profiler/create_report.m:
Specialise create_procrep_coverage_report/3 into
create_{static,dynamic}_coverage_report/4.
Created a new exported function deep_get_maybe_procrep. This is useful for
getting a procedure representation from the deep data-structure in one
step.
deep_profiler/display.m:
Add a new display item, display_developer. This wraps another display
item but is only displayed when developer mode is active.
deep_profiler/display_report.m:
Add a control to the main screen that enables or disabled developer mode.
This control has been placed at the bottom of the screen so that it's out
of the way.
Put the developer controls on the main screen into their own list (there's
only one at the moment).
For now the coverage-annotated procedure representation link on a (static)
procedure's page is not a developer option. Should this be a developer
option?
Added a link to the dynamic coverage annotated procedure representation
report from the dump proc dynamic report.
Added a link to the clique dump report from the clique report, the dynamic
coverage annotated procedure representation report can be accessed
transitively through this link.
Added a link the variable use analysis report and proc static report to the
procedure report and static coverage annotated procedure representation
report.
deep_profiler/html_format.m:
Support the new display_developer item.
Refactor the item_to_html code for lists.
deep_profiler/profile.m:
Include a new field in the deep data-structure for storing coverage data
that is indexed by a proc_static_ptr. When dynamic coverage information is
used this field is populated by adding per ProcDynamic data for each static
procedure.
deep_profiler/startup.m:
Fill in the per ProcStatic coverage data when the deep profiler starts up.
deep_profiler/measurements.m:
Create a new data type static_coverage_info which represents per ProcStatic
coverage information.
Include functions that are used when calculating per ProcStatic coverage
information from per ProcDynamic coverage information.
deep_profiler/mdprof_cgi.m:
Remove rarely used command line option rather making it conform to changes
in query.m.
deep_profiler/old_html_format.m:
deep_profiler/old_query.m:
Conform to changes in query.m.
deep_profiler/mdprof_test.m:
deep_profiler/mdprof_fb.automatic_parallelism.m:
deep_profiler/recursion_patterns.m:
deep_profiler/var_use_analysis.m:
Conform to changes in create_report.
deep_profiler/array_util.m:
Add a new predicate, array_foldl3_from_1 to support propagation of coverage
information from proc dynamics to proc statics.
Create two new deep profiler reports: the first calculates the costs of
recursive calls at any depth of recursion for a clique. The second summarises
the types of recursion seen in a program.
deep_profiler/query.m:
Introduce new cmd types.
Conform to changes of the cmd type.
Add a new preference: the maximum number of proc statics to display for
each recursion type.
Memoize the creation of the recursion types histogram report as it can take
a while to generate, 39 minutes on a Core 4 when generating a report for
the compiler it's self.
deep_profiler/report.m:
Define the new report structures.
deep_profiler/create_report.m:
Handle creation of the new reports.
Export describe_proc and own_and_inherit_to_perf_row_data so that the
recursion_patterns module can use them.
Write a find_clique_first_and_other_procs predicate to find the first
procedure in a clique, it also returns a list of the remaining procedures.
This is exported so that recursion_patterns can use it but it belongs here
as it is generally used for creating reports.
Refactor the retrieval of the progrep data from the deep data structure.
Make a note about a change that could be made to speed up large analyses.
deep_profiler/profile.m:
Refactor the retrieval of the progrep data from the deep data structure.
deep_profiler/display_report.m:
Handle translation of the new reports to the display data type.
Link to this report from clique reports, and to clique reports from the new
clique recursion costs report.
Refactor the code that constructs the lists of related reports.
deep_profiler/recursion_patterns.m:
Create a new module to contain the recursion pattern analysis code.
deep_profiler/old_html_format.m:
deep_profiler/old_query.m:
Conform to changes in query.m
deep_profiler/mdprof_fb.automatic_parallelism.m:
Remove add_call_site_to_map it is a duplicate of add_call_site_report_to_map.
Move add_call_site_report_to_map and proc_label_from_proc_desc to report.m
deep_profiler/measurement_units.m:
Include functions to manipulate probabilities of conjoint and disjoint events.
Allow percentages to be greater than 100% and less than 0% this can occur
legitimately in some cases.
deep_profiler/measurements.m:
deep_profiler/var_use_analysis.m:
Move weighted_average/3 from var_use_analysis.m to measurements.m where it
can be used globally.
deep_profiler/mdprof_test.m:
Modify the mdprof_test program so that it can generate the recursion types
histogram report, this is useful for profiling and debugging.
Estimated hours taken: 8
Branches: main
Fix some usability problems in the deep profiler.
deep_profiler/exclude.m:
Provide a better, more explicit representation of the contour exclusion
file. This representation allows the application of contour exclusion
even if the contour exclusion file contains some unrecognized module
names. This is useful, because some modules of the Mercury standard
library, such as profiling_builtin and mer_std, are unrecognized.
They are unrecognized because either they export no predicates,
or because deep profiling is turned off for them.
deep_profiler/query.m:
Make a provision for controlling the number of a procedure's callers
displayed at a time.
deep_profiler/create_report.m:
Take advantage of the new capabilities listed above.
deep_profiler/display_report.m:
When listing a procedure's callers, print any warning messages
generated at startup about recognized modules.
When listing a procedure's callers, list the application of contour
exclusion among the links only if contour exclusion is in fact
applicable.
When listing a procedure's callers, allow the user to control how
many callers are shown at a time. Make the default bunch size 20,
instead of previous hardwired (and too small) 5.
Fix some counterintuitive behavior. When clicking on one of the fields
in the header of a table, in all but one of the displayed reports, this
sorts the data being displayed on the selected column. The exception
was top_procs, in which it changed the criterion on which the top_procs
were selected, and thus changed the procedures being displayed as well.
This diff eliminates the exception. The links in the header cells now
do the expected thing, obeying the law of least astonishment. The
criterion for selected what procs are top can still be changed using
the already existing (and previously mostly redundant) links below
the table.
deep_profiler/startup.m:
Look for contour exclusion files not in Deep.data.contour, but in
Deep.contour, since this is more consistent with what we do for other
auxiliary files, like Data.procrep.
Make the reading of the exclusion file and the program representation
file part of the normal startup process, instead of setting their
fields to dummies and filling them in with real values later.
If there is a program representation file, make sure that the names
of the modules in it are not unrecognized by contour exclusion.
Fix a minor old bug: don't count the time spent reading Deep.contour
as part of the propagation pass.
Print meaningful messages (to the log file in /var/tmp) about
any problems in reading Deep.contour files.
deep_profiler/apply_exclusion.m:
deep_profiler/old_html_format.m:
deep_profiler/old_query.m:
deep_profiler/profile.m:
deep_profiler/report.m:
Conform to the changes above.
doc/user_guide.texi:
Document contour exclusion, and to make this possible, expand
considerably the documentation of the deep profiler as a whole.
doc/Mmakefile:
Ensure that invocations of latex cannot hang waiting for user input.
Estimated hours taken: 6
Branches: main
Several improvements to make the deep profiler more convenient to use.
deep_profiler/mdprof_cgi.m:
If the data file does not end in .data, generate an explicit error
message. Without this, the deep profiler just hangs forever on such
filenames.
deep_profiler/query.m:
Add a couple more categories of user preferences. The first new
category controls whether, in reports for procedures and cliques,
the deep profiler will display call sites through which there were no
calls. The second new category controls whether the deep profiler
includes module qualifications in the names of procedures. Module
qualifying everything, as we did before, can clutter up the output,
so the new default is to module qualify only the names of procedures
that are in different modules from the module that defines the current
procedure.
deep_profiler/profile.m:
deep_profiler/report.m:
Remember module names and unqualified as well as qualified names
of procedures.
deep_profiler/create_report.m:
deep_profiler/read_profile.m:
Remember the newly needed information.
deep_profiler/display_report.m:
Implement the two new categories of preferences.
When generating the menu, make the "follow the action" links
more useful by implicitly preferring to (a) hide inactive call sites,
and (b) sort by overall time.
When sorting by time, if two times (clock ticks) are equal,
try to use call sequence numbers to resolve the order.
When generating the menu, print the total quanta as well as the total
runtime computed from it.
deep_profiler/dump.m:
deep_profiler/old_html_format.m:
deep_profiler/old_query.m:
Conform to the changes above.
Estimated hours taken: 12
Branches: main
Add a query to the deep profiler for reporting on the getter and setter
predicates in a module. This can be used to decide whether a data structure
should be split in two, and if so, which fields should go into the
substructure.
At the same time, improve the displays we generate for some other queries
by making more table column headers capable of sorting the table.
deep_profiler/query.m:
Add the new query type.
deep_profiler/report.m:
Add the definition of the new report.
Include the identity of the callee, if known, in call site
descriptions, to allow sorting on this.
deep_profiler/create_report.m:
Add code to execute the new kind of query.
Include the new field in call site descriptions.
Remove "dump" from the name of the predicate that builds
procrep_coverage reports, since the result isn't a raw dump.
deep_profiler/display_report.m:
Add code to display the new kind of report.
Add a link to the new report to display we generate for the module
query.
Allow the table in the procedure report to be sorted by context
or by callee.
deep_profiler/display.m:
Add a new table column class for displaying field names.
deep_profiler/html_format.m:
Handle the new table column class.
deep_profiler/old_html_format.m:
deep_profiler/old_query.m:
"Handle" the new query.
Estimated Hours Taken: 15
Branches: main
Add an analysis report to the deep profiler that can determine when an
argument to a procedure is likely to be consumed or produced. This is useful
for determining the parallelism within dependant conjunctions.
Due to limitations in the deep profiler recursive code cannot be handled well,
the following two problems (and workarounds) exist.
If the variable we're searching for uses of is consumed or produced within
a recursive (or mutually recursive) call, we do not follow this recursion.
Instead assume a pessimistic default, that if the call produces the
variable assume it's produced at the end of the call, if the call consumes
the variable assume it's consumed at the beginning of the call.
When calculating the cost of a goal that occurs before a consumption or
after a production we cannot relative an accurate cost of a recursive call
from the deep profiler (A cost of 1.0 is returned).
deep_profiler/report.m:
Add the new report structure.
deep_profiler/program_representation_utils.m:
The code for the new analysis depends on the program representation
bytecode, it has been added to this module.
deep_profiler/create_report.m:
Code to create the new report.
deep_profiler/display_report.m:
Basic code to display the report.
deep_profiler/query.m:
Include support for parsing and running the commands that refer to this
new report.
Conform to changes in report.m
deep_profiler/old_html_format.m:
deep_profiler/old_query.m:
Conform to changes in report.m
Estimated hours taken: 1
Branches: main
Put the program representation inside the deep profiler's main data structure,
since this simplifies things. (We already handled the contents of the
exclusion threshold file this way.)
deep_profiler/profile.m:
Change the data structure's type definition as above.
deep_profiler/create_report.m:
deep_profiler/mdprof_cgi.m:
deep_profiler/mdprof_feedback.m:
deep_profiler/mdprof_test.m:
deep_profiler/query.m:
Conform to the change above.
deep_profiler/startup.m:
Conform to the change above.
Rename a predicate to avoid ambiguity.
Estimated hours taken: 1
Branches: main
deep_profiler/html_format.m:
deep_profiler/old_html_format.m:
deep_profiler/query.m:
deep_profiler/old_query.m:
Move old, now deprecated code from query.m and html_format.m to two new
modules: old_query.m and old_html_format.m. In each case, the leftover
module, whose cord works via reports, is significantly smaller.
deep_profiler/measurement_units.m:
Move some functions here from html_format.m, since they are needed
with both the old and the new approaches, and the functions belong
here.
Estimated hours taken: 20
Branches: main
Move the remaining deep profiler commands, dump_clique, clique and root,
to the report mechanism.
deep_profiler/report.m:
Define the representation of the new reports.
Change some names to be more consistent.
deep_profiler/create_report.m:
Add code to create the new reports.
deep_profiler/measurement units.m:
Add a predicate to needed by the root command.
deep_profiler/display_report.m:
Add code for converting the new reports to displays.
Factor out some previously duplicated code for describing procedures
and call sites and for comparing them by name and by context.
Conform to the changes in display.m.
deep_profiler/display.m:
Expand the display representation to allow the specification of
attributed text, with the available attributes being bold, italic
and underlined. We now use bold in clique reports.
deep_profiler/html_format.m:
Handle the changes in display.m.
Generate HTML with more structured line breaks.
deep_profiler/query.m:
Switch the default handling of the three commands over to the new
mechanism.
Remove a long unused function.
deep_profiler/profile.m:
Parameterize the call_site_and_callee type, to allow report.m
to associate a procedure description, not a proc_static_ptr,
with a normal call site.
Estimated Hours Taken: 10
Branches: main
Modify the deep profiling tools, in particular mdprof_cgi, to make use of
program representation files, and begin implementing the procrep coverage
report.
The mdprof coverage report is incomplete, however it is convenient to post
this for review now. Currently it can display a pretty-printed procrep
structure without coverage annotations.
mdbcomp/program_representation.m:
Modified program representation data structures to allow goals to be
annotated with another type.
deep_profiler/query.m:
Create new deep profiling command, to display a coverage-annotated
procedure representation.
Support use of program representation information by new-style report
generating code.
Centralised command parsing to code within this module.
Created constants for strings shared between cmd_to_string and
string_to_maybe_cmd.
deep_profiler/report.m:
Add new report, report_procrep_coverage_dump.
Modified the proc_static dump report to include the number of coverage
points in the proc_static.
deep_profiler/create_report.m:
create_report now accepts an optional program representation structure,
making it possible for report generation code to use the program
representation structure.
Support the new procrep coverage report, which is incomplete.
Modify the proc static dump report, to display the number of coverage
points within a proc_static structure.
deep_profiler/display.m:
Introduce display_verbatim, a new display element that is able to display
verbatim text, such as source code.
deep_profiler/display_report.m:
Add support for displaying the new mdbcomp_coverage report.
Modify the proc static dump display to show the number of coverage points,
conforming to a change in report.m
deep_profiler/html_format.m:
Support the display_verbatim element.
Use query_to_string from query.m to create the query part of a URL.
Conform to changes in deep_profiler/query.m
deep_profiler/startup.m:
Modify read_and_startup to also read the program representation data if
available.
deep_profiler/mdprof_cgi.m:
Use string_to_maybe_query from query.m to parse a query string.
Renamed some variables.
Modified command-loop to pass program representation between commands.
deep_profiler/program_representation_utils.m:
A new module to hold utilities from working with program representation
data.
Pretty printing code from mdprof_procrep has been moved here and modified
to output a cord of strings rather than write to standard output.
deep_profiler/mdprof_feedback.m:
deep_profiler/mdprof_test.m:
Conform to changes in query.m
deep_profiler/mdprof_procrep.m:
Move proc_rep pretty-printing code to program_representation_utils.m
browser/declarative_tree.m:
Conform to changes in mdbcomp/program_representation.m
Estimated hours taken: 16
Branches: main
Move three deep profiler commands to the new framework: the one that reports
all of a procedure's callers, the one that lists all the modules, and the one
that lists all the procedures in a module.
deep_profiler/report.m:
Add report structures for the three newly handled commands.
Modify the perf_row_data type to allow the representation of rows
in which the self measures are meaningful, but the self_and_desc
measures aren't, since this is true for module summaries.
Remove a redundant field from an existing report structure.
deep_profiler/query.m:
Change the cmd type slightly to (a) make command names more distinct,
(b) make it less error prone by replacing ints with purpose-specific
wrapper types, and (c) to specify whether contour exclusion should be
applied in the proc_callers command. Change (c) is needed because
we contour exclusion alters the report we want to generate for the
proc_callers command, but we don't want to pass the preferences to the
code that computes reports.
deep_profiler/create_report.m:
Add code for computing the new report structures.
Conform to the change to perf_row_data.
deep_profiler/display_report.m:
Add code for displaying the new report structures.
Conform to the change to perf_row_data.
deep_profiler/display.m:
Avoid the need for a number-of-columns field in most table cells
by having separate function symbols for single- and multi-column cells.
Add a mechanism that allows us to avoid artifically nested lists
(horizontal list inside a vertical list).
Add the column classes needed by the newly implemented commands.
deep_profiler/html_format.m:
Conform to the changes in display.m and query.m.
deep_profiler/profile.m:
We used to read in the contour exclusion file just before computing
and displaying a HTML page whose contents required it. We cannot do
this with the report framework (since the predicate that computes
report structures does not take I/O states as arguments), and it is
better to read the contour exclusion file just once anyway. This diff
therefore adds a field to the deep structure for holding the result
of reading in the contour exclusion file (if any).
deep_profiler/startup.m:
Add code to fill in this new field.
Switch to using state variables.
deep_profiler/apply_exclusion.m:
New module. It contains some code that used to be in query.m, but
is now also needed by create_report.m, slightly modified. The
modifications are to add some code that used to be in the callers
of the moved predicates. This code used to be needed only once,
but is now needed twice (since each predicate now has two callers).
The reason why I didn't put all this into the existing exclude.m
is that that would have required profile.m and exclude.m to import
each other.
deep_profiler/exclude.m:
Change the return value to separate the case of a non-existent contour
exclusion file from a file that exists but whose reading yields an
error.
Give variables better names.
deep_profiler/mdprof_cgi.m:
deep_profiler/mdprof_test.m:
Conform to the changes in query.m.
deep_profiler/apply_exclusion.m:
New module. It contains some code that used to be in query.m, but
is now also needed by create_report.m, slightly modified. The
modifications are to add some code that used to be in the callers
of the moved predicates. This code used to be needed only once,
but is now needed twice (since each predicate now has two callers).
The reason why I didn't put all this into the existing exclude.m
is that that would have required profile.m and exclude.m to import
each other.
deep_profiler/exclude.m:
Give variables better names.
deep_profiler/measurements.m:
Turn a semidet predicate into a function.
Estimated hours taken: 16
Branches: main
Implement the proc command of the deep profiler using the report mechanism.
Some of the detailed changes also address Paul's post-commit review of my
previous diff.
deep_profiler/report.m:
Define the representation of the proc report.
Change some names to be more consistent.
deep_profiler/profile.m:
Add a type for use in report.m.
Document the fields of an existing type.
deep_profiler/create_report.m:
Add code to create proc reports.
Change some names to be more consistent.
Fix a bug: avoid divide by zero exceptions in some rare cases.
deep_profiler/measurement units.m:
Fix a bug: avoid divide by zero exceptions in some rare cases.
deep_profiler/display_report.m:
Add the code for converting proc reports to displays. This required
rewriting almost everthing in this module to remove the assumption,
which was previously embedded in many places, that the code is part
of the implementation of the top_procs command. The new code is
separated into two parts; the command-specific parts, and the parts
that should be usable for all commands. (The reusable part is much
bigger.)
deep_profiler/display.m:
Expand the display representation to allow the specification of
- plain text as distinguished from headings
- paragraph breaks
- pseudo-links, i.e. text fragments that are formatted like links
- table separator rows
- table cells that span more than one column
which are used in the new version of display_report.m.
Simplify the mechanism for setting up table header groups.
deep_profiler/html_format.m:
Handle the changes in display.m.
Fix some anomalies in the formatting of lists and tables.
Group related code together.
deep_profiler/query.m:
Provide a switch for selecting which of the two mechanisms,
the old direct generation of HTML or the new indirect generation
through the report and display structures, should generate the HTML
page for a command that both mechanisms can implement. The default
is the new mechanism, but it can be overridden by implementors
by creating a file called "/tmp/old_deep_profiler".
Switch the default handling of the proc command over to the new
mechanism.
Rename some types and function symbols to make them more consistent
and expressive.
Generate more informative error messages for domain error exceptions.
deep_profiler/mdprof_cgi.m:
Add a mechanism for decoding the links in the generated pages.
The mechanism takes the form of three new options: --decode,
--decode-cmd, and --decode-prefs. When one or more of these are given,
mdprof_cgi will not do its usual stuff, instead of just loops waiting
for lines whose contents it will then try to decode as queries,
as commands and/or as preferences.
deep_profiler/Mercury.options:
Try to postpone thrashing in the deep compiler when compiled in
debug grades, by compiling their contents with shallow tracing.
deep_profiler/array_util.m:
Make shallow tracing effective by moving the deep recursions
into non-exported predicates.
deep_profiler/callgraph.m:
deep_profiler/canonical.m:
deep_profiler/startup.m:
Convert the debugging code to use trace goals.
deep_profiler/cliques.m:
deep_profiler/read_profile.m:
Convert the debugging code to use trace goals.
Use more expressive names for lots of variables.
In read_profile.m, avoid an unnecessary repackaging of data.
deep_profiler/util.m:
Fix some grammar errors in a comment, and comment some code to use
state variables.
deep_profiler/interface.m:
Fix some grammar errors in a comment.
deep_profiler/mdprof_feedback.m:
Change some variable names to be consistent with the other modules.
library/float.m:
library/int.m:
Make the messages in the domain errors we throw more expressive.
library/io.m:
Minor style changes.
Estimated hours taken: 5
Branches: main
Implement the colouring of column groups.
deep_profiler/display.m:
Modify the types representing tables to allow some column groups
to specify that the column classes they represent should affect the
style in which that column is rendered. Refactor the table_header_group
type to avoid representing this information, and the column class,
in a redundant manner.
Avoid abbrevations in some type names and function symbols.
deep_profiler/display_report.m:
Conform to the change in the representation of tables.
Use more consistent variable names.
deep_profiler/html_format.m:
When a table column group's header asks for it, set up the background
colour of the columns in that group so that alternating column groups
have different backgrounds.
Instead of representing the CSS as a single constant string, we
now represent it as a map that we update from a default when we
construct the body of the page.
Implement Paul's review suggestions after my last change.
library/cord.m:
NEWS:
Add a utility predicate for use by the new code.
Estimated hours taken: 3
Branches: main
Use reports to handle the call_site_static and call_site_dynamic queries.
deep_profiler/report.m:
Add the two new report types.
Give a better name to a type.
deep_profiler/create_report.m:
Add the code for the two new report types.
Make the predicate for creating a perf_row_data more general,
and avoid defining the perf_row_data field-by-field.
deep_profiler/display.m:
Rename a table_class that is now used not just by the top_procs
command.
Represent columns as cords, to avoid the need for keeping track
of whether the list of columns is reversed.
deep_profiler/display_report.m:
Add the code to handle the two new report types.
Refactor some existing code to allow it to called by the new code.
Conform to the other changes in report.m.
deep_profiler/html_format.m:
Add newlines to the output, in order to make it more readable
and to make using diff between two versions of the output feasible.
Print the HTML title as a heading.
Add a XXX about a disrespected control.
deep_profiler/query.m:
Use the new method to handle the two query types.
Estimated hours taken: 4
Branches: main
Use reports to handle the proc_static and proc_dynamic queries.
deep_profiler/report.m:
Add the two new report types.
Change the report type to give each query its own answer, without
needing to use the report_message for reporting errors. Also, group
each kind of report into a single data structure that can be
manipulated independently of the other kinds of answers (like I did
some time ago for items in the compiler.)
deep_profiler/create_report.m:
Add the code for the two new report types.
Conform to the other changes in report.m.
Inline some unnecessary intermediate predicates.
deep_profiler/display.m:
deep_profiler/html_format.m:
Rename one table_class that is now used not just by the menu command.
deep_profiler/display_report.m:
Add the code to handle the two new report types.
Refactor some existing code to allow it to called by the new code.
Conform to the other changes in report.m.
Fix several predicate names that would have become misleading as soon
as we added new kinds of reports. The fix is to give them the prefix
"top_procs_", since they all generate parts of top_procs pages.
deep_profiler/query.m:
Use the new method to handle the two query types.
Estimated hours taken: 6
Branches: main
Convert Paul's new code to use cords of strings to represent HTML.
deep_profiler/html_format.m:
Convert to using cords. Restructure the code in a couple of places
to always put start and end tags around HTML fragments together.
Fix a missing "=" in a tag.
deep_profiler/interface.m:
deep_profiler/mdprof_cgi.m:
deep_profiler/read_profile.m:
Provide better diagnostics.
deep_profiler/create_report.m:
deep_profiler/display.m:
deep_profiler/display_report.m:
deep_profiler/mdprof_feedback.m:
deep_profiler/measurement_units.m:
deep_profiler/query.m:
deep_profiler/report.m:
mdbcomp/feedback.m:
Misc cleanups. They can be considered my post-commit review of Paul's
diff.
In mdprof_feedback.m, delete a strange test that prevented the program
from being used from the command line.
deep_profiler/dump.m:
deep_profiler/mdprof_dump.m:
deep_profiler/timeout.m:
deep_profiler/util.m:
Misc cleanups of old code.
Estimated Hours Taken: 60
Branches: main
Introduced intermediate data structures to mdprof_cgi. This will make it
easier to extract deep profiling information for any new tools. It also
enables other viewers for deep profiling data to more easily be developed.
New code making use of these data structures follows a pipeline pattern
as it is converted between the following data structures in this order:
Deep -> Report -> Display -> HTML
This work is in progress and only a handful of deep profiler commands have
been converted to use the new data structures. The old code has been kept for
reference and should be removed in the future.
deep_profiler/report.m:
Created new module that defines a data structure for a deep profiler report.
The new report data structure can be used to describe the content of a
particular report.
deep_profiler/display.m:
Created new module that defines a data structure for displaying deep
profiler information.
The new display data structure can be used to describe the structure and
content shown when a user views a deep profiler report.
deep_profiler/measurement_units.m:
Created new module defining data types to represent percentages, amounts
of computer memory and time.
deep_profiler/query.m:
Move memory_units type to data_types.m.
Handling of the following deep profiler commands has been re-written to
use the new data structures:
deep_cmd_quit,
deep_cmd_timeout,
deep_cmd_restart,
deep_cmd_menu,
deep_cmd_top_procs
deep_profiler/create_report.m:
Created new module for creating reports from the deep data structure.
deep_profiler/display_report.m:
Created new module for converting a report structure to a display
structure.
deep_profiler/html_format.m:
Introduce new code to convert a display data structure into a string of
HTML formatted text.
Moved number formatting code to measurement_units.m.
Estimated hours taken: 0.10
Branches: main
Rename some fields within the profile_stats data-structure, The new names
now correctly convey information the fields represent.
deep_profiler/profile.m:
Renamed fields beginning with max_ to begin with num_, as they contain
the number of items in the respective array.
deep_profiler/query.m:
Conform to changes in profile.m
Estimated Hours Taken: 1
Branches: main
Tidying up some of the deep profiler tools code. Moved data structures,
predicates and functions that are not specific to the mdprof_cgi program
from interface.m to other modules.
deep_profilier/query.m:
Moved data structures from interface.m to query.m including the cmd and
preferences data structures.
Renamed functions to replace url_compoent with string.
deep_profilier/interface.m:
Moved code from this module that doesn't relate to the mdprof_cgi
program's client-server nature to query.m and html_format.m
Conformed to predicate-renaming changes in query.m
deep_profilier/html_format.m:
Moved machine_datafile_cmd_pref_to_url function to this module.
Conform to changes in interface.m and query.m.
deep_profiler/top_procs.m:
Conform to changes in interface.m and query.m.
Estimated hours taken: 20
Branches: main
Add a new compiler option. --inform-ite-instead-of-switch. If this is enabled,
the compiler will generate informational messages about if-then-elses that
it thinks should be converted to switches for the sake of program reliability.
Act on the output generated by this option.
compiler/simplify.m:
Implement the new option.
Fix an old bug that could cause us to generate warnings about code
that was OK in one duplicated copy but not in another (where a switch
arm's code is duplicated due to the case being selected for more than
one cons_id).
compiler/options.m:
Add the new option.
Add a way to test for the bug fix in simplify.
doc/user_guide.texi:
Document the new option.
NEWS:
Mention the new option.
library/*.m:
mdbcomp/*.m:
browser/*.m:
compiler/*.m:
deep_profiler/*.m:
Convert if-then-elses to switches at most of the sites suggested by the
new option. At the remaining sites, switching to switches would have
nontrivial downsides. This typically happens with the switched-on type
has many functors, and we treat one or two specially (e.g. cons/2 in
the cons_id type).
Perform misc cleanups in the vicinity of the if-then-else to switch
conversions.
In a few cases, improve the error messages generated.
compiler/accumulator.m:
compiler/hlds_goal.m:
(Rename and) move insts for particular kinds of goal from
accumulator.m to hlds_goal.m, to allow them to be used in other
modules. Using these insts allowed us to eliminate some if-then-elses
entirely.
compiler/exprn_aux.m:
Instead of fixing some if-then-elses, delete the predicates containing
them, since they aren't used, and (as pointed out by the new option)
would need considerable other fixing if they were ever needed again.
compiler/lp_rational.m:
Add prefixes to the names of the function symbols on some types,
since without those prefixes, it was hard to figure out what type
the switch corresponding to an old if-then-else was switching on.
tests/invalid/reserve_tag.err_exp:
Expect a new, improved error message.
Estimated hours taken: 1.5
Branches: main
Add the ability to select top procedures in the deep profiling tool based on a
raw threshold rather than a percentage. This will be useful when selecting
procedures to be implicitly parallelized. Add links to the top-level screen of
the deep profiler for accessing this feature.
deep_profiler/interface.m:
Added extra value for display_limit type, threshold_value to represent this
new value-based threshold of procedures.
Alter string conversion predicates for limits to include this new value and
ensure it's distinct from existing values.
deep_profiler/query.m:
Added links to main screen to access reports using this feature. Default
thresholds have been chosen.
deep_profiler/top_procs.m:
Altered find_top_procs to accept this new display_limit and properly
exclude procedures below the value threshold.
Added new predicate find_threshold_value_predicate which reference new
predicates to select rows above the threshold value.
Renamed existing find_threshold_predicate to
find_threshold_percent_predicate, as well as the predicates it looks up.
Branches: main
Some interface improvements for mdprof_cgi.
deep_profiler/html_format.m:
deep_profiler/query.m:
Insert "<WBR>" tags after some characters to suggest where the browser
may break long module, procedure and file names over multiple lines.
This improves the chances that tables will not become too wide to fit
in the browser window.
Add square brackets around the links at the bottom of generated pages,
e.g. "[Sort by name]". Give such links the CSS class "button" and add
margins to increase visual separation between the links.
Write the "Toggle fields:" etc. labels in bold to make them stand out,
and add line breaks so that the first button after the label is easier
to find (on the next line, at the the left margin).
Spell out "self+descendants." rather than "self+desc."
Estimated hours taken: 3
Branches: main
Clean up in unused module imports in the Mercury system detected
by --warn-unused-imports.
analysis/*.m:
browser/*.m:
deep_profiler/*.m:
compiler/*.m:
library/*.m:
mdbcomp/*.m:
profiler/*.m:
slice/*.m:
Remove unused module imports.
Fix some minor departures from our coding standards.
analysis/Mercury.options:
browser/Mercury.options:
deep_profiler/Mercury.options:
compiler/Mercury.options:
library/Mercury.options:
mdbcomp/Mercury.options:
profiler/Mercury.options:
slice/Mercury.options:
Set --no-warn-unused-imports for those modules that are used as
packages or otherwise break --warn-unused-imports, e.g. because they
contain predicates with both foreign and Mercury clauses and some of
the imports only depend on the latter.
Estimated hours taken: 16
Branches: main
Include call sequence numbers in the pages generated by the deep profiler
(if the user's preferences ask for this).
NEWS:
Announce the new capability.
deep_profiler/measurements.m:
Standardize on one spelling of call sequence numbers (callseqs).
Add some utility predicates for use in dump.m.
Fix a bug in the calculation of the number of calls: redos enter
procedure invocations, not leave them.
deep_profiler/profile.m:
Fix a bug: the num_callseqs field was in the wrong place.
deep_profiler/html_format.m:
When generating a line of the clique description, include call sequence
number information if requested.
Do likewise when generating other HTML fragments that have to match
the contents of those lines (e.g. the table headers).
Generate the links that allow the users to sort rows by call
sequence numbers.
Generate the links that allow the users to toggle the switches
controlling the display of call sequence number information.
Do not show the toggles relevant to the display of times if times are
not being displayed.
deep_profiler/interface.m:
Extend the preferences to allow requests for call sequence number
information.
Include the display of call sequence numbers in the default set of
preferences. However, include the display of time information in the
default only of the profiling run lasted long enough for this
information to have some hope of being useful.
Rename some function symbols to avoid ambiguities.
deep_profiler/query.m:
Extend the menu to include links for the most expensive
cliques/procedures as measured by call sequence numbers.
Include the links for the most expensive cliques/procedures as
measured by clock ticks in the menu only if the profiling run
lasted long enough for this to be useful.
Print out the total number of call sequence numbers with the other
statistics.
Use call sequence numbers instead of time to decide where the "action"
begins, since this works even for short profiling runs.
deep_profiler/top_procs.m:
Provide the capability of sorting on call sequence numbers.
deep_profiler/dump.m:
Provide the capability to dump out several more fields of the top level
structure (profile.deep), including the cliques, the reverse links and
the propagated measurement information.
deep_profiler/startup.m:
Dump out the cliques, the reverse links and the propagated measurement
information when computed.
deep_profiler/mdprof_cgi.m:
Add an option for setting the hostname in the generated links to
"localhost". This is useful on laptops temporarily disconnected
from the Domain Name System (e.g. my laptop).
Set this option automatically if the name of the file in the initial
query has a ".localhost" suffix added to it.
Add some tracing hooks.
deep_profiler/mdprof_test.m:
Add an option to specify what info to dump during startup.
deep_profiler/mdprof_dump.m:
Switch to using the compiler's method of specifying what to dump,
since this is more maintainable.
deep_profiler/exclude.m:
Rename some function symbols to avoid ambiguities.
deep_profiler/timeout.m:
Add conditionally compiled code to help debug lock management.
deep_profiler/callgraph.m:
deep_profiler/canonical.m:
deep_profiler/cliques.m:
deep_profiler/conf.m:
deep_profiler/read_profile.m:
Minor cleanups.
Estimated hours taken: 18
Branches: main
Move the univ, maybe, pair and unit types from std_util into their own
modules. std_util still contains the general purpose higher-order programming
constructs.
library/std_util.m:
Move univ, maybe, pair and unit (plus any other related types
and procedures) into their own modules.
library/maybe.m:
New module. This contains the maybe and maybe_error types and
the associated procedures.
library/pair.m:
New module. This contains the pair type and associated procedures.
library/unit.m:
New module. This contains the types unit/0 and unit/1.
library/univ.m:
New module. This contains the univ type and associated procedures.
library/library.m:
Add the new modules.
library/private_builtin.m:
Update the declaration of the type_ctor_info struct for univ.
runtime/mercury.h:
Update the declaration for the type_ctor_info struct for univ.
runtime/mercury_mcpp.h:
runtime/mercury_hlc_types.h:
Update the definition of MR_Univ.
runtime/mercury_init.h:
Fix a comment: ML_type_name is now exported from type_desc.m.
compiler/mlds_to_il.m:
Update the the name of the module that defines univs (which are
handled specially by the il code generator.)
library/*.m:
compiler/*.m:
browser/*.m:
mdbcomp/*.m:
profiler/*.m:
deep_profiler/*.m:
Conform to the above changes. Import the new modules where they
are needed; don't import std_util where it isn't needed.
Fix formatting in lots of modules. Delete duplicate module
imports.
tests/*:
Update the test suite to confrom to the above changes.
Estimated hours taken: 0.5
Branches: main
profiler/*.m:
deep_profiler/*.m:
compiler/*.m:
Replace __ with . as the module qualifier everywhere in all the modules
of the profiler and deep profiler and in some modules of the compiler.
Estimated hours taken: 1.5
Branches: main
Convert the rest of the deep profiler to four-space indentation.
There are no changes to any algorithms (other than introducing state
variables in a few spots).
Remove an old bug workaround.
deep_profiler/*.m:
Convert to four-space indentation where that has not already
been done.
Fix some bad line breaks.
Use state variables in a few places.
Reposition comments according to our current coding standard.
deep_profiler/Mercury.options:
Remove some old bug workarounds.
Estimated hours taken: 2
Branches: main, release
Improvements to the deep profiling tool.
Provide the facility to hide modules/procedures to which there have been no
calls. This reduces the amount of clutter when exploring the program module
by module and when listing the procedures in a module (the clutter is
primarily a result of unused standard library modules). By default, we
currently do not hide any modules/procedures.
Add a toggle to sort lists of procedures by the number of redos they perform.
We have had control at the head of tables to do this for a while now.
Delete the function int_list_from_to/2 since that functionality is now
available in the standard library.
deep_profiler/html_format.m:
Provide toggles that allow the user to show/hide inactive modules and
procedures.
Add a toggle to sort lists of procedures by the number of redos.
Rename a numbered sequence of variables so that it is easier to insert
new ones in the middle of the sequence.
deep_profiler/interface.m:
Add support for hiding/showing inactive modules.
Clean up some of the existing code.
deep_profiler/measurements.m:
Add a predicate, is_inactive/1, that test if an own_prof_info
for a given entity is inactive.
deep_profiler/query.m:
When generating the module and procedure summaries, respect the new
preference settings for hiding/showing inactive modules/procedures.
deep_profiler/top_procs.m:
Delete int_list_from_to/2 and replace calls to it with
'..' from the standard library.
Estimated hours taken: 1
Branches: main
Cleanup the deep profiler.
There are no changes to any algorithms.
deep_profiler/*.m:
Switch to state variable notation throughout. Rearrange
the order of some procedure arguments to facilitate this.
Remove some unnecessary module imports.
Reformat comments according to our current coding standard.
Update comments that refer to modules that no longer exist.
Estimated hours taken: 3
Branches: main, release
Workaround a problem where the deep profiling tool wasn't
escaping HTML special characters in procedure and file names.
deep_profiler/html_format.m:
deep_profiler/query.m:
When generating HTML make sure to escape special
characters, like '<', correctly.
Estimated hours taken: 1
Branches: main
profiler/*.m:
deep_profiler/*.m:
Import only one module per line. Misc other updates to bring these
modules closer to being up to date with our current style guidelines.
Estimated hours taken: 2
Branches: main
Apply Zoltan's deep profiler changes to the main branch. Zoltan committed
these changes to the release branch, but forgot to commit them to the main
branch.
Mmakefile:
NEWS:
configure.in:
bindist/Mmakefile:
bindist/bindist.Makefile.in:
doc/user_guide.texi:
library/exception.m:
library/io.m:
runtime/mercury_conf.h.in:
runtime/mercury_conf_param.h:
runtime/mercury_misc.c:
runtime/mercury_misc.h:
scripts/Mmakefile:
deep_profiler/.nocopyright:
deep_profiler/Mercury.options:
deep_profiler/Mmakefile:
deep_profiler/callgraph.m:
deep_profiler/cliques.m:
deep_profiler/conf.m:
deep_profiler/html_format.m:
deep_profiler/interface.m:
deep_profiler/mdprof_cgi.m:
deep_profiler/query.m:
deep_profiler/startup.m:
deep_profiler/timeout.m:
deep_profiler/mdprof_server.m:
deep_profiler/server.m:
Merge in changes from the release branch.
Enclosed below is Zoltan's original log message.
----------
Estimated hours taken: 0.3 (on release branch)
Branches: release
Make the deep profiler work again.
The existing process structure of the deep profiler made it very hard to debug,
mainly because the Mercury debugger is confused by forks and stymied by execs.
This change completely replaces the process structure. The old structure had
two separate programs, mdprof_cgi and mdprof_server, the first always acting
as client and the second always acting as server. The new structure has only
one program, mdprof_cgi, which acts as a server if there is no existing server
for the relevant profiling data file, and as a client otherwise.
Although mdprof_cgi normally forks when it becomes a server to let the parent
exit and let the web server know that the web page it has generated is
complete, the fork can be disabled for debugging via an option. This allows
the communication between client and server to be debugged by running two
instances of mdprof_cgi in different windows, one or both under mdb.
deep_profiler/DESIGN:
New file describing the new process structure, its race conditions,
and their solutions.
deep_profiler/.nocopyright:
Add DESIGN.
deep_profiler/mdprof_cgi.m:
A complete rewrite of this module to enable it act as both client and
server.
deep_profiler/mdprof_test.m:
A new module to hold the testing functionality of mdprof_server.m.
deep_profiler/mdprof_server.m:
deep_profiler/server.m:
Delete these now unnecessary modules.
Mmakefile:
Replace references to mdprof_server with references to mdprof_test.
deep_profiler/Mmakefile:
Replace references to mdprof_server with references to mdprof_test.
Move the include of Mmake.deep.params, to allow it to override
top level parameter settings.
deep_profiler/Mercury.options:
Work around a compiler bug by turning off the offending optimization.
deep_profiler/timeout.m:
A rewrite of major parts of this module to support the new design
of mdprof_cgi.m, and to make unexpected signals easier to debug.
deep_profiler/interface.m:
Implement the mechanisms needed by the new process structure.
Change the characters we use to separate components of the URL.
The old ones were special to the shell, and screwed up command lines.
(As double insurance, we ignore the command line anyway when invoked
by the web server.)
Change some names to be more expressive.
deep_profiler/conf.m:
Add a new function, getpid, for use by interface.m.
Rewrite some code to use streams explicitly, not implicitly.
deep_profiler/callgraph.m:
deep_profiler/cliques.m:
Add (now commented out) code to help debug these modules, for use
in cases where mdb doesn't help, because the program works perfectly
with debugging enabled :-(
deep_profiler/query.m:
Move the predicate try_exec here from the deleted file server.m.
deep_profiler/html_format.m:
Trivial change to conform to name change in interface.m.
deep_profiler/startup.m:
Generate debugging output to a caller specified stream, not to
stdout and stderr.
Disable the generation of statistics temporarily, since the diff
to make statistics reporting routines write to a specified stream
instead of stdout and stderr won't be committed on the release branch.
Currently, they always write to stdout, which in the new design
goes to the web page, not to the startup file.
configure.in:
Detect the presence of opendir, readdir and closedir, and the header
file they need, dirent.h. Enable the deep profiler only if all exist,
since the deep profiler now needs them.
runtime/mercury_conf.h.in:
Support the changes to configure.in.
runtime/mercury_misc.[ch]:
Add a mechanism for registering cleanup functions to be executed when
we terminate the program due to an uncaught exception.
library/exception.m:
Invoke the registered cleanup functions just before terminating
the program due to an uncaught exception.
Estimated hours taken: 8
Branches: main
Misc cleanups and documentation of various aspects of the deep profiler.
More will come later.
deep_profiler/startup.m:
Fix a bug in the algorithm that we used to compensate for zeroing.
The new algorithm is documented in the paper.
deep_profiler/profile.m:
deep_profiler/startup.m:
deep_profiler/query.m:
Switch the names we use for the data structures used in that algorithm
to match the names in the paper.
deep_profiler/query.m:
Fix an off-by-one bug.
deep_profiler/interface.m:
Document this module.
deep_profiler/interface.m:
deep_profiler/mdprof_cgi.m:
Move some code in interface.m that was used only in mdprof_cgi.m to
mdprof_cgi.m.
