mirror of
https://github.com/Mercury-Language/mercury.git
synced 2026-04-20 03:43:51 +00:00
083d376e6598628362ee91c2da170febd83590f4
39 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Zoltan Somogyi
|
3c07fc2121 |
Use explicit streams in deep_profiler/*.m.
deep_profiler/analysis_utils.m:
deep_profiler/autopar_find_best_par.m:
deep_profiler/autopar_reports.m:
deep_profiler/autopar_search_callgraph.m:
deep_profiler/autopar_search_goals.m:
deep_profiler/callgraph.m:
deep_profiler/canonical.m:
deep_profiler/cliques.m:
deep_profiler/coverage.m:
deep_profiler/dump.m:
deep_profiler/mdprof_cgi.m:
deep_profiler/mdprof_create_feedback.m:
deep_profiler/mdprof_dump.m:
deep_profiler/mdprof_procrep.m:
deep_profiler/mdprof_report_feedback.m:
deep_profiler/mdprof_test.m:
deep_profiler/profile.m:
deep_profiler/read_profile.m:
deep_profiler/recursion_patterns.m:
deep_profiler/startup.m:
deep_profiler/var_use_analysis.m:
Replace implicit streams with explicit streams.
In some places, simplify some code, often using constructs such as
string.format that either did not exist or were too expensive to use
when the original code was written.
Consistenly use the spelling StdErr over Stderr.
In mdbprof_dump.m, put filename and reason-for-failing-to-open-that-file
in the right order in an error message.
deep_profiler/DEEP_FLAGS.in:
Turn on --warn-implicit-stream-calls for the entire deep_profiler
directory.
mdbcomp/program_representation.m:
mdbcomp/trace_counts.m:
Replace implicit streams with explicit streams. These are the two mdbcomp
modules that (a) used to use implicit streams, and (2) are used by the
deep profiler.
mdbcomp/Mercury.options:
Turn on --warn-implicit-stream-calls for these two modules.
slice/mcov.m:
slice/mtc_union.m:
Conform to the changes in mdbcomp.
|
||
|
Zoltan Somogyi
|
9095985aa8 |
Fix more warnings from --warn-inconsistent-pred-order-clauses.
deep_profiler/*.m:
Fix inconsistencies between (a) the order in which functions and predicates
are declared, and (b) the order in which they are defined.
In most modules, either the order of the declarations or the order
of the definitions made sense, and I changed the other to match.
In some modules, neither made sense, so I changed *both* to an order
that *does* make sense (i.e. it has related predicates together).
In query.m, put the various commands in the same sensible order
as the code processing them.
In html_format.m, merge two exported functions together, since
they can't be used separately.
In some places, put dividers between groups of related
functions/predicates, to make the groups themselves more visible.
In some places, fix comments or programming style.
deep_profiler/DEEP_FLAGS.in:
Since all the modules in this directory are now free from any warnings
generated by --warn-inconsistent-pred-order-clauses, specify that option
by default in this directory to keep it that way.
|
||
|
Zoltan Somogyi
|
270170416d |
Bring the style of deep_profiler/* up-to-date.
deep_profiler/*.m:
Replace ( C -> T ; E ) if-then-elses with (if C then T else E ).
Replace calls to error/1 with calls to unexpected/3.
Add some module qualifications where this makes the code easier to read.
|
||
|
Julien Fischer
|
b9eca3ce6b |
Change the order of predicate arguments in the array module to make them
Branches: main Change the order of predicate arguments in the array module to make them more conducive to the use of state variable notation. library/array.m: As above. Group clauses for functions with those of the corresponding predicate. library/svarray.m: library/hash_table.m: library/io.m: library/random.m: compiler/lambda.m: deep_profiler/array_util.m: deep_profiler/callgraph.m: deep_profiler/canonical.m: deep_profiler/cliques.m: deep_profiler/dense_bitset.m: deep_profiler/measurements.m: deep_profiler/profile.m: deep_profiler/read_profile.m: deep_profiler/startup.m: tests/general/array_test.m: tests/general/mode_inf.m: tests/hard_coded/array_test2.m: tests/hard_coded/lp.m: tests/hard_coded/reuse_array.m: Conform to the above change and remove dependencies on the svarray module. tests/general/set_test.m: Replace calls to set_bbbtree.size/2 with calls to set_bbbtree.count/2. NEWS: Announce the above change. |
||
|
Paul Bone
|
d43239d6a7 |
Move some of the goal path code from compiler/goal_path.m to the mdbcomp
library where it can be used by the deep profiler.
Also move the goal path code from program_representation.m to the new module,
goal_path.m in mdbcomp/
mdbcomp/goal_path.m:
New module containing goal path code.
mdbcomp/program_representation.m:
Original location of goal path code.
compiler/goal_path.m:
Move some of this goal_path code into mdbcomp/goal_path.m
mdbcomp/feedback.automatic_parallelisation.m:
mdbcomp/rtti_access.m:
mdbcomp/slice_and_dice.m:
mdbcomp/trace_counts.m:
browser/debugger_interface.m:
browser/declarative_execution.m:
browser/declarative_tree.m:
compiler/build_mode_constraints.m:
compiler/call_gen.m:
compiler/code_info.m:
compiler/continuation_info.m:
compiler/coverage_profiling.m:
compiler/deep_profiling.m:
compiler/format_call.m:
compiler/goal_path.m:
compiler/goal_util.m:
compiler/hlds_data.m:
compiler/hlds_goal.m:
compiler/hlds_out_goal.m:
compiler/hlds_out_pred.m:
compiler/hlds_pred.m:
compiler/interval.m:
compiler/introduce_parallelism.m:
compiler/layout_out.m:
compiler/llds.m:
compiler/mode_constraint_robdd.m:
compiler/mode_constraints.m:
compiler/mode_ordering.m:
compiler/ordering_mode_constraints.m:
compiler/polymorphism.m:
compiler/post_typecheck.m:
compiler/prog_rep.m:
compiler/prop_mode_constraints.m:
compiler/push_goals_together.m:
compiler/rbmm.condition_renaming.m:
compiler/smm_common.m:
compiler/stack_layout.m:
compiler/stack_opt.m:
compiler/trace_gen.m:
compiler/tupling.m:
compiler/type_constraints.m:
compiler/typecheck.m:
compiler/unify_gen.m:
compiler/unneeded_code.m:
deep_profiler/Mmakefile:
deep_profiler/analysis_utils.m:
deep_profiler/coverage.m:
deep_profiler/create_report.m:
deep_profiler/display_report.m:
deep_profiler/dump.m:
deep_profiler/mdprof_fb.automatic_parallelism.m:
deep_profiler/message.m:
deep_profiler/old_query.m:
deep_profiler/profile.m:
deep_profiler/program_representation_utils.m:
deep_profiler/read_profile.m:
deep_profiler/recursion_patterns.m:
deep_profiler/report.m:
deep_profiler/var_use_analysis.m:
slice/Mmakefile:
slice/mcov.m:
Conform to the move of the goal path code.
|
||
|
Paul Bone
|
436d37cc8d |
Fix a regression that affected coverage profiling and automatic parallelism
analysis.
When instrumenting a program for deep and coverage profiling the compiler adds
goal path information to the program. The compiler also writes out a bytecode
representation of the program. The step_switch goal path step includes a field
for the number of functors of the type of the variable that is being switched
upon. When this is included in the deep profiling data (as it is now) and
these goals paths are used as the indexes in tables the deep profiler cannot
reconstruct matching goal paths from the program representation bytecode,
causing many important map lookups to fail or throw an exception.
We fix this not by preventing the compiler from writing out this information,
but by stripping it out of goal paths before the analysis tool uses those goal
paths.
mdbcomp/program_representation.m:
Add a new predicate rev_goal_path_remove_type_info/2 that removes
type-dependant information from goals.
deep_profiler/profile.m:
Change the goal_path string in the call_site_static type to contain a
reverse_goal_path rather than a string. This hides the abstraction of the
stripping of type dependant information within read_profile.m
deep_profiler/read_profile.m:
Strip type dependant information from goal path strings as they are read in.
Conform to changes in profile.m
deep_profiler/report.m:
In the call site static dump information use a goal_path field rather than
a string.
deep_profiler/analysis_utils.m:
deep_profiler/create_report.m:
deep_profiler/dump.m:
deep_profiler/mdprof_fb.automatic_parallelism.m:
deep_profiler/old_query.m:
Conform to changes in profile.m.
deep_profiler/display_report.m:
Conform to changes in report.m.
|
||
|
Zoltan Somogyi
|
8a28e40c9b |
Add the predicates sorry, unexpected and expect to library/error.m.
Estimated hours taken: 2 Branches: main Add the predicates sorry, unexpected and expect to library/error.m. compiler/compiler_util.m: library/error.m: Move the predicates sorry, unexpected and expect from compiler_util to error. Put the predicates in error.m into the same order as their declarations. compiler/*.m: Change imports as needed. compiler/lp.m: compiler/lp_rational.m: Change imports as needed, and some minor cleanups. deep_profiler/*.m: Switch to using the new library predicates, instead of calling error directly. Some other minor cleanups. NEWS: Mention the new predicates in the standard library. |
||
|
Zoltan Somogyi
|
58211e2f2e |
Allow more than 2^15 vars in a procedure representation.
Estimated hours taken: 12 Branches: main Allow more than 2^15 vars in a procedure representation. mdbcomp/program_representation.m: Allow a variable number to be represented by four bytes as well as two and one. This means that we also have to represent the number of variables in a procedure using a four-byte number, not a two-byte number. Use four bytes to represent line numbers. Programs that overflow 16-bit var numbers may also overflow 16 bit line numbers. These requires a change in the deep profiler data's binary compatibility version number. compiler/prog_rep.m: Encode vars using four bytes if necessary. Be consistent in using only signed 8-bit as well as signed 16-bit numbers. compiler/implicit_parallelism.m: Conform to the change in program_representation.m. deep_profiler/profile.m: deep_profiler/read_profile.m: Add a compression flag to the set of flags read from the data file. Put the flags into the profile_stats as a group, not one-by-one. deep_profiler/canonical.m: deep_profiler/create_report.m: deep_profiler/dump.m: deep_profiler/mdprof_feedback.m: deep_profiler/old_html_format.m: deep_profiler/old_query.m: deep_profiler/query.m: Conform to the change in profile.m. runtime/mercury_deep_profiling.c: Prepare for compression of profiling data files being implemented. runtime/mercury_stack_layout.h: Fix some documentation rot. runtime/mercury_conf_param.h: Add an implication between debug flags to make debugging easier. |
||
|
Paul Bone
|
067934f008 |
Start using the new dynamic coverage information in the deep profiler.
This patch separates the coverage annotated procedure report into two
reports, one with dynamic coverage data and one with static coverage data.
This restores the functionality of the static coverage report since my last
change, and provides access to the dynamic report via new controls only
visible to developers.
deep_profiler/query.m:
In the cmd data-type:
Rename deep_cmd_procrep_coverage constructor to
deep_cmd_static_procrep_coverage.
Add deep_cmd_procrep_dynamic_coverage.
In the preferences data-type:
Add a new field pref_developer_mode which indicates if developer-only
options are visible or not.
Add code to parse and print the new command and preference option.
deep_profiler/create_report.m:
Specialise create_procrep_coverage_report/3 into
create_{static,dynamic}_coverage_report/4.
Created a new exported function deep_get_maybe_procrep. This is useful for
getting a procedure representation from the deep data-structure in one
step.
deep_profiler/display.m:
Add a new display item, display_developer. This wraps another display
item but is only displayed when developer mode is active.
deep_profiler/display_report.m:
Add a control to the main screen that enables or disabled developer mode.
This control has been placed at the bottom of the screen so that it's out
of the way.
Put the developer controls on the main screen into their own list (there's
only one at the moment).
For now the coverage-annotated procedure representation link on a (static)
procedure's page is not a developer option. Should this be a developer
option?
Added a link to the dynamic coverage annotated procedure representation
report from the dump proc dynamic report.
Added a link to the clique dump report from the clique report, the dynamic
coverage annotated procedure representation report can be accessed
transitively through this link.
Added a link the variable use analysis report and proc static report to the
procedure report and static coverage annotated procedure representation
report.
deep_profiler/html_format.m:
Support the new display_developer item.
Refactor the item_to_html code for lists.
deep_profiler/profile.m:
Include a new field in the deep data-structure for storing coverage data
that is indexed by a proc_static_ptr. When dynamic coverage information is
used this field is populated by adding per ProcDynamic data for each static
procedure.
deep_profiler/startup.m:
Fill in the per ProcStatic coverage data when the deep profiler starts up.
deep_profiler/measurements.m:
Create a new data type static_coverage_info which represents per ProcStatic
coverage information.
Include functions that are used when calculating per ProcStatic coverage
information from per ProcDynamic coverage information.
deep_profiler/mdprof_cgi.m:
Remove rarely used command line option rather making it conform to changes
in query.m.
deep_profiler/old_html_format.m:
deep_profiler/old_query.m:
Conform to changes in query.m.
deep_profiler/mdprof_test.m:
deep_profiler/mdprof_fb.automatic_parallelism.m:
deep_profiler/recursion_patterns.m:
deep_profiler/var_use_analysis.m:
Conform to changes in create_report.
deep_profiler/array_util.m:
Add a new predicate, array_foldl3_from_1 to support propagation of coverage
information from proc dynamics to proc statics.
|
||
|
Paul Bone
|
7e7d77e23f |
Make coverage profiling data 'deep'.
The deep profiler associates measurements with their context in the call graph
modulo recursion. This has been true for all measurements except for coverage
profiling data. This patch allows coverage data to be associated with
ProcDynamic structures so that it is keyed by this context not just the static
procedure. This new behaviour is the default the old option of static coverage
profiling is still available for testing, as is no coverage profiling. Note
that, as before, coverage profiling is supported by default however coverage
points are not inserted by default.
This change will be used to measure the depth of recursion, and therefore the
average cost of recursion as well as the likely times when variables are
produced in calls for the automatic parallelisation analysis.
runtime/mercury_conf_param.h:
Create three new preprocessor macros:
MR_DEEP_PROFILING_COVERAGE - defined when coverage profiling is
enabled.
MR_DEEP_PROFILING_COVERAGE_STATIC - defined when static coverage
profiling is being used.
MR_DEEP_PROFILING_COVERAGE_DYNAMIC - defined when dynamic coverage
profiling is being used.
runtime/mercury_deep_profiling.h:
Update data structures to support dynamic coverage profiling.
Use conditional compilation to allow us to test the deep profiler in three
different modes, without coverage profiling, with static coverage profiling
and with dynamic coverage profiling.
Rename MR_PROFILING_MALLOC, since it takes a type rather than a size in
bytes it should be called MR_PROFILING_NEW to conform with existing malloc
and new functions.
runtime/mercury_deep_profiling.c:
Avoid a C compiler warning.
MR_write_out_coverage_point has been removed, it's replaced with:
MR_write_out_coverage_points_static and
MR_write_out_coverage_points_dynamic.
These write out more than one coverage point and write out either static or
dynamic coverage points.
Write a 64bit flags value (a bitfield) to the header of the Deep.data file.
This replaces the canonical byte (with a bit). and the byte that describes
the word size. This value also includes two bits describing the whether no
coverage data, static coverage data or dynamic coverage data is present in
the file. A bit is reserved ti indicate if the data is compressed (which
is not yet supported).
MR_write_fixed_size_int now writes out 8 byte integers, this is only used
for some counts present at the beginning of the data file along with the
new flags value. It now takes a MR_uint_least64_t integer as it's
parameter. The assertion to test for negative numbers has been removed
since this type is unsigned.
Increment the Deep.data file format version number.
compiler/layout_out.m:
Conditionally compile the NULL pointer that represents the coverage points
list in proc statics. This is conditional on the
MR_DEEP_PROFILING_COVERAGE_STATIC macro being defined.
compiler/coverage_profiling.m:
Add support for generating dynamic coverage points.
compiler/options.m:
compiler/handle_options.m:
Implement the new developer options for controlling coverage profiling.
library/profiling_builtin.m:
Specialize increment_coverage_point_count for both static and dynamic
coverage profiling. This creates
increment_{static,dynamic}_coverage_point_count.
deep_profiler/profile.m:
Add an extra field to profile_stats, this tracks whether the file reader
should try to read none, static or dynamic coverage data.
Add an extra field to proc_dynamic, An array of coverage counts wrapped by
a maybe type. It's indexed the same as the array of coverage infos in
proc_static. This array is present if dynamic coverage profiling is being
done (the default).
Modify the coverage_points field in proc static, now there are two fields,
an array of coverage_point_info values which store compile-time data. And
an optional array of coverage points (present if static coverage profiling
was performed).
Updated the formatting of the proc static structure.
Moved the coverage_point type to coverage.m.
Created a new type, coverage_data_type which enumerates the possibilities
for coverage profiling: none, static and dynamic.
deep_profiler/coverage.m:
Move the coverage point type here from profile.m, as the profile data
structure no longer refers to it directly.
Create a predicate coverage_point_arrays_to_list/3 which merges coverage
point information and the counts themselves into coverage points. This can
be used to construct a list of coverage points regardless of whether static
or dynamic coverage points are being used.
deep_profiler/read_profile.m:
Conform to changes in runtime/mercury_deep_profiling.c.
Refactored reading of the file header, a new named predicate is now used
rather than a lambda expression.
Incremented the Deep.data version number.
deep_profiler/report.m:
Updated the proc dynamic dump report structure to include a list of
coverage points.
deep_profiler/create_report.m:
deep_profiler/display_report.m:
Conform to changes in profile.m.
The proc dynamic dump now shows coverage information that was contained in
that proc dynamic.
deep_profiler/canonical.m:
deep_profiler/dump.m:
Conform to changes in profile.m.
deep_profiler/io_combinator.m:
Add a 13-arg version of maybe_error_sequence.
deep_profiler/Mercury.options:
Documented another trace flag.
|
||
|
Paul Bone
|
699663e5e2 |
Deep profiler enhancements.
Create two new deep profiler reports: the first calculates the costs of
recursive calls at any depth of recursion for a clique. The second summarises
the types of recursion seen in a program.
deep_profiler/query.m:
Introduce new cmd types.
Conform to changes of the cmd type.
Add a new preference: the maximum number of proc statics to display for
each recursion type.
Memoize the creation of the recursion types histogram report as it can take
a while to generate, 39 minutes on a Core 4 when generating a report for
the compiler it's self.
deep_profiler/report.m:
Define the new report structures.
deep_profiler/create_report.m:
Handle creation of the new reports.
Export describe_proc and own_and_inherit_to_perf_row_data so that the
recursion_patterns module can use them.
Write a find_clique_first_and_other_procs predicate to find the first
procedure in a clique, it also returns a list of the remaining procedures.
This is exported so that recursion_patterns can use it but it belongs here
as it is generally used for creating reports.
Refactor the retrieval of the progrep data from the deep data structure.
Make a note about a change that could be made to speed up large analyses.
deep_profiler/profile.m:
Refactor the retrieval of the progrep data from the deep data structure.
deep_profiler/display_report.m:
Handle translation of the new reports to the display data type.
Link to this report from clique reports, and to clique reports from the new
clique recursion costs report.
Refactor the code that constructs the lists of related reports.
deep_profiler/recursion_patterns.m:
Create a new module to contain the recursion pattern analysis code.
deep_profiler/old_html_format.m:
deep_profiler/old_query.m:
Conform to changes in query.m
deep_profiler/mdprof_fb.automatic_parallelism.m:
Remove add_call_site_to_map it is a duplicate of add_call_site_report_to_map.
Move add_call_site_report_to_map and proc_label_from_proc_desc to report.m
deep_profiler/measurement_units.m:
Include functions to manipulate probabilities of conjoint and disjoint events.
Allow percentages to be greater than 100% and less than 0% this can occur
legitimately in some cases.
deep_profiler/measurements.m:
deep_profiler/var_use_analysis.m:
Move weighted_average/3 from var_use_analysis.m to measurements.m where it
can be used globally.
deep_profiler/mdprof_test.m:
Modify the mdprof_test program so that it can generate the recursion types
histogram report, this is useful for profiling and debugging.
|
||
|
Zoltan Somogyi
|
190764a083 |
Fix some usability problems in the deep profiler.
Estimated hours taken: 8 Branches: main Fix some usability problems in the deep profiler. deep_profiler/exclude.m: Provide a better, more explicit representation of the contour exclusion file. This representation allows the application of contour exclusion even if the contour exclusion file contains some unrecognized module names. This is useful, because some modules of the Mercury standard library, such as profiling_builtin and mer_std, are unrecognized. They are unrecognized because either they export no predicates, or because deep profiling is turned off for them. deep_profiler/query.m: Make a provision for controlling the number of a procedure's callers displayed at a time. deep_profiler/create_report.m: Take advantage of the new capabilities listed above. deep_profiler/display_report.m: When listing a procedure's callers, print any warning messages generated at startup about recognized modules. When listing a procedure's callers, list the application of contour exclusion among the links only if contour exclusion is in fact applicable. When listing a procedure's callers, allow the user to control how many callers are shown at a time. Make the default bunch size 20, instead of previous hardwired (and too small) 5. Fix some counterintuitive behavior. When clicking on one of the fields in the header of a table, in all but one of the displayed reports, this sorts the data being displayed on the selected column. The exception was top_procs, in which it changed the criterion on which the top_procs were selected, and thus changed the procedures being displayed as well. This diff eliminates the exception. The links in the header cells now do the expected thing, obeying the law of least astonishment. The criterion for selected what procs are top can still be changed using the already existing (and previously mostly redundant) links below the table. deep_profiler/startup.m: Look for contour exclusion files not in Deep.data.contour, but in Deep.contour, since this is more consistent with what we do for other auxiliary files, like Data.procrep. Make the reading of the exclusion file and the program representation file part of the normal startup process, instead of setting their fields to dummies and filling them in with real values later. If there is a program representation file, make sure that the names of the modules in it are not unrecognized by contour exclusion. Fix a minor old bug: don't count the time spent reading Deep.contour as part of the propagation pass. Print meaningful messages (to the log file in /var/tmp) about any problems in reading Deep.contour files. deep_profiler/apply_exclusion.m: deep_profiler/old_html_format.m: deep_profiler/old_query.m: deep_profiler/profile.m: deep_profiler/report.m: Conform to the changes above. doc/user_guide.texi: Document contour exclusion, and to make this possible, expand considerably the documentation of the deep profiler as a whole. doc/Mmakefile: Ensure that invocations of latex cannot hang waiting for user input. |
||
|
Zoltan Somogyi
|
a34eb73d2d |
Several improvements to make the deep profiler more convenient to use.
Estimated hours taken: 6 Branches: main Several improvements to make the deep profiler more convenient to use. deep_profiler/mdprof_cgi.m: If the data file does not end in .data, generate an explicit error message. Without this, the deep profiler just hangs forever on such filenames. deep_profiler/query.m: Add a couple more categories of user preferences. The first new category controls whether, in reports for procedures and cliques, the deep profiler will display call sites through which there were no calls. The second new category controls whether the deep profiler includes module qualifications in the names of procedures. Module qualifying everything, as we did before, can clutter up the output, so the new default is to module qualify only the names of procedures that are in different modules from the module that defines the current procedure. deep_profiler/profile.m: deep_profiler/report.m: Remember module names and unqualified as well as qualified names of procedures. deep_profiler/create_report.m: deep_profiler/read_profile.m: Remember the newly needed information. deep_profiler/display_report.m: Implement the two new categories of preferences. When generating the menu, make the "follow the action" links more useful by implicitly preferring to (a) hide inactive call sites, and (b) sort by overall time. When sorting by time, if two times (clock ticks) are equal, try to use call sequence numbers to resolve the order. When generating the menu, print the total quanta as well as the total runtime computed from it. deep_profiler/dump.m: deep_profiler/old_html_format.m: deep_profiler/old_query.m: Conform to the changes above. |
||
|
Paul Bone
|
e70295415d |
Various changes for automatic parallelism, the two major changes are:
Estimated hours taken: 20.
Branches: main
Various changes for automatic parallelism, the two major changes are:
Refactored some of the search for parallel conjunctions to use types that
describe the cost of a call site and the cost of a clique-procedure. These
new types make it harder for programmers to mistakingly compare values of
either type accidentally.
Where possible, use the body of a clique to determine the cost of recursive
calls at the top level of recursion. This improves the accuracy of this
calculation significantly.
deep_profiler/mdprof_fb.automatic_parallelism.m:
As above.
deep_profiler/measurements.m:
New cost data types as above.
deep_profiler/coverage.m:
When coverage information completeness tests fail print out the procedure
where the coverage information is incomplete.
deep_profiler/message.m:
Introduce a new warning used in the automatic parallelism analysis.
deep_profiler/profile.m:
Introduce a semidet version of deep_get_progrep_det.
mdbcomp/program_representation.m:
Introduce a predicate to return the goal_rep from inside a case_rep
structure. This can be used as higher order code to turn a case list into
a goal list for example.
deep_profiler/Mercury.options:
Keep a commented out MCFLAGS definition that can be used to enable
debugging output for the automatic parallelism analysis.
|
||
|
Paul Bone
|
b1257cbf48 |
Search for parallelism opportunities in a clique-wise manner.
Estimated hours taken: 25
Branches: main
Search for parallelism opportunities in a clique-wise manner. Starting at the
root clique traverse expensive calls to find cliques with procedures that
should be parallelised. This has several advantages, namely it gives a simple
method for calculating an approximation of the cost of a recursive call site.
In the future it will make it easier to perform specialise parallel versions
of procedures.
deep_profiler/mdprof_feedback.m:
As above.
Create a logging facility that provides some information about the
analysis to the user, I'm using this to determine the order of features I
should implement based on the usefulness of such a feature in analysed
programs. This has necessitated changes to the command line interface.
deep_profiler/profile.m:
Introduce deep_get_progrep_det/2 to retrieve the program representation
structure from a deep structure or throw an exception.
deep_profiler/report.m:
Add more information to a comment.
mdbcomp/feedback.m:
Add more information about a structure field's semantics in the field name
and comment.
Incremented the feedback file version number.
deep_profiler/Mercury.options
Remove some old --no-warn-unused-imports flags.
|
||
|
Zoltan Somogyi
|
c91f385afc |
Put the program representation inside the deep profiler's main data structure,
Estimated hours taken: 1 Branches: main Put the program representation inside the deep profiler's main data structure, since this simplifies things. (We already handled the contents of the exclusion threshold file this way.) deep_profiler/profile.m: Change the data structure's type definition as above. deep_profiler/create_report.m: deep_profiler/mdprof_cgi.m: deep_profiler/mdprof_feedback.m: deep_profiler/mdprof_test.m: deep_profiler/query.m: Conform to the change above. deep_profiler/startup.m: Conform to the change above. Rename a predicate to avoid ambiguity. |
||
|
Zoltan Somogyi
|
be95a5c643 |
Move the remaining deep profiler commands, dump_clique, clique and root,
Estimated hours taken: 20 Branches: main Move the remaining deep profiler commands, dump_clique, clique and root, to the report mechanism. deep_profiler/report.m: Define the representation of the new reports. Change some names to be more consistent. deep_profiler/create_report.m: Add code to create the new reports. deep_profiler/measurement units.m: Add a predicate to needed by the root command. deep_profiler/display_report.m: Add code for converting the new reports to displays. Factor out some previously duplicated code for describing procedures and call sites and for comparing them by name and by context. Conform to the changes in display.m. deep_profiler/display.m: Expand the display representation to allow the specification of attributed text, with the available attributes being bold, italic and underlined. We now use bold in clique reports. deep_profiler/html_format.m: Handle the changes in display.m. Generate HTML with more structured line breaks. deep_profiler/query.m: Switch the default handling of the three commands over to the new mechanism. Remove a long unused function. deep_profiler/profile.m: Parameterize the call_site_and_callee type, to allow report.m to associate a procedure description, not a proc_static_ptr, with a normal call site. |
||
|
Zoltan Somogyi
|
f637aa2458 |
Move three deep profiler commands to the new framework: the one that reports
Estimated hours taken: 16 Branches: main Move three deep profiler commands to the new framework: the one that reports all of a procedure's callers, the one that lists all the modules, and the one that lists all the procedures in a module. deep_profiler/report.m: Add report structures for the three newly handled commands. Modify the perf_row_data type to allow the representation of rows in which the self measures are meaningful, but the self_and_desc measures aren't, since this is true for module summaries. Remove a redundant field from an existing report structure. deep_profiler/query.m: Change the cmd type slightly to (a) make command names more distinct, (b) make it less error prone by replacing ints with purpose-specific wrapper types, and (c) to specify whether contour exclusion should be applied in the proc_callers command. Change (c) is needed because we contour exclusion alters the report we want to generate for the proc_callers command, but we don't want to pass the preferences to the code that computes reports. deep_profiler/create_report.m: Add code for computing the new report structures. Conform to the change to perf_row_data. deep_profiler/display_report.m: Add code for displaying the new report structures. Conform to the change to perf_row_data. deep_profiler/display.m: Avoid the need for a number-of-columns field in most table cells by having separate function symbols for single- and multi-column cells. Add a mechanism that allows us to avoid artifically nested lists (horizontal list inside a vertical list). Add the column classes needed by the newly implemented commands. deep_profiler/html_format.m: Conform to the changes in display.m and query.m. deep_profiler/profile.m: We used to read in the contour exclusion file just before computing and displaying a HTML page whose contents required it. We cannot do this with the report framework (since the predicate that computes report structures does not take I/O states as arguments), and it is better to read the contour exclusion file just once anyway. This diff therefore adds a field to the deep structure for holding the result of reading in the contour exclusion file (if any). deep_profiler/startup.m: Add code to fill in this new field. Switch to using state variables. deep_profiler/apply_exclusion.m: New module. It contains some code that used to be in query.m, but is now also needed by create_report.m, slightly modified. The modifications are to add some code that used to be in the callers of the moved predicates. This code used to be needed only once, but is now needed twice (since each predicate now has two callers). The reason why I didn't put all this into the existing exclude.m is that that would have required profile.m and exclude.m to import each other. deep_profiler/exclude.m: Change the return value to separate the case of a non-existent contour exclusion file from a file that exists but whose reading yields an error. Give variables better names. deep_profiler/mdprof_cgi.m: deep_profiler/mdprof_test.m: Conform to the changes in query.m. deep_profiler/apply_exclusion.m: New module. It contains some code that used to be in query.m, but is now also needed by create_report.m, slightly modified. The modifications are to add some code that used to be in the callers of the moved predicates. This code used to be needed only once, but is now needed twice (since each predicate now has two callers). The reason why I didn't put all this into the existing exclude.m is that that would have required profile.m and exclude.m to import each other. deep_profiler/exclude.m: Give variables better names. deep_profiler/measurements.m: Turn a semidet predicate into a function. |
||
|
Zoltan Somogyi
|
599e1c2fdb |
Implement the proc command of the deep profiler using the report mechanism.
Estimated hours taken: 16 Branches: main Implement the proc command of the deep profiler using the report mechanism. Some of the detailed changes also address Paul's post-commit review of my previous diff. deep_profiler/report.m: Define the representation of the proc report. Change some names to be more consistent. deep_profiler/profile.m: Add a type for use in report.m. Document the fields of an existing type. deep_profiler/create_report.m: Add code to create proc reports. Change some names to be more consistent. Fix a bug: avoid divide by zero exceptions in some rare cases. deep_profiler/measurement units.m: Fix a bug: avoid divide by zero exceptions in some rare cases. deep_profiler/display_report.m: Add the code for converting proc reports to displays. This required rewriting almost everthing in this module to remove the assumption, which was previously embedded in many places, that the code is part of the implementation of the top_procs command. The new code is separated into two parts; the command-specific parts, and the parts that should be usable for all commands. (The reusable part is much bigger.) deep_profiler/display.m: Expand the display representation to allow the specification of - plain text as distinguished from headings - paragraph breaks - pseudo-links, i.e. text fragments that are formatted like links - table separator rows - table cells that span more than one column which are used in the new version of display_report.m. Simplify the mechanism for setting up table header groups. deep_profiler/html_format.m: Handle the changes in display.m. Fix some anomalies in the formatting of lists and tables. Group related code together. deep_profiler/query.m: Provide a switch for selecting which of the two mechanisms, the old direct generation of HTML or the new indirect generation through the report and display structures, should generate the HTML page for a command that both mechanisms can implement. The default is the new mechanism, but it can be overridden by implementors by creating a file called "/tmp/old_deep_profiler". Switch the default handling of the proc command over to the new mechanism. Rename some types and function symbols to make them more consistent and expressive. Generate more informative error messages for domain error exceptions. deep_profiler/mdprof_cgi.m: Add a mechanism for decoding the links in the generated pages. The mechanism takes the form of three new options: --decode, --decode-cmd, and --decode-prefs. When one or more of these are given, mdprof_cgi will not do its usual stuff, instead of just loops waiting for lines whose contents it will then try to decode as queries, as commands and/or as preferences. deep_profiler/Mercury.options: Try to postpone thrashing in the deep compiler when compiled in debug grades, by compiling their contents with shallow tracing. deep_profiler/array_util.m: Make shallow tracing effective by moving the deep recursions into non-exported predicates. deep_profiler/callgraph.m: deep_profiler/canonical.m: deep_profiler/startup.m: Convert the debugging code to use trace goals. deep_profiler/cliques.m: deep_profiler/read_profile.m: Convert the debugging code to use trace goals. Use more expressive names for lots of variables. In read_profile.m, avoid an unnecessary repackaging of data. deep_profiler/util.m: Fix some grammar errors in a comment, and comment some code to use state variables. deep_profiler/interface.m: Fix some grammar errors in a comment. deep_profiler/mdprof_feedback.m: Change some variable names to be consistent with the other modules. library/float.m: library/int.m: Make the messages in the domain errors we throw more expressive. library/io.m: Minor style changes. |
||
|
Paul Bone
|
1de09830dc |
Deep profiling and program representation file format changes.
Estimated hours taken: 7.5 Branches: main Deep profiling and program representation file format changes. Include a table of variable names in the program representation used by the deep profiler (Deep.procrep). The table shouldn't contain variables introduced by the compiler. Include the name of the program from which the deep profile was generated in the header of the deep profiling data file (Deep.data). compiler/prog_rep.m: Optionally create the variable table within the bytecode generated for each procedure, The actual variable names are stored in the string table for the module. compiler/stack_layout.m: Enable the variable table only for deep profiling builds. mdbcomp/program_representation.m: Include variable table in each program representation structure. Create predicates to read in the variable table if it is present. Create predicates to lookup a variable name from a table or generate a name automatically. Incremented version number for the program representation file format. runtime/mercury_deep_profiling.c: Write out program name in header of deep profiling data. Increment version numbers of deep profiling and program representation file formats. browser/declarative_tree.m: deep_profiler/canonical.m: tests/debugger/declarative/builtin_call_rep.exp: tests/debugger/declarative/dependency.exp: Conform to changes in mdbcomp/program_representation.m deep_profiler/mdprof_procrep.m: Lookup and display variable names when displaying program representation. deep_profiler/profile.m: Make program name part of the profile_stats structure. deep_profiler/read_profile.m: deep_profiler/io_combinator.m: Read in program name from deep profiling data. Incremented version number for the deep profiling file format. deep_profiler/report.m: deep_profiler/create_report.m: deep_profiler/display_report.m: Display program name with the statistics on the menu report. deep_profiler/dump.m Conform to changes in deep_profiler/profile.m. |
||
|
Paul Bone
|
7593b61b70 |
Introduce coverage profiling. While regular profiling shows which procedures
Estimated Hours Taken: 100
Branches: main
Introduce coverage profiling. While regular profiling shows which procedures
are used the most, coverage profiling goes further, and also shows the most
common execution paths through each procedure. We intend coverage profiling
data to be used by a future automatic parallelization pass in the compiler.
However, it should also be useful for other purposes.
This diff adds a compiler option, --coverage-profiling, that, when specified,
adds program instrumentation to record execution counts at selected points
in procedure bodies. The implementation currently stores coverage profiling
information in ProcStatic structures. We will later investigate the impact
of storing this information in ProcDynamic structures instead.
For now coverage statistics can be viewed with the mdprof_dump tool.
compiler/deep_profiling.m:
Introduced coverage profiling transformation after deep profiling
transformation, it will be run if at least one coverage point type is
enabled.
compiler/goal_util.m:
Created create_conj_from_list/3, to create a conjunction from a list of
goals.
compiler/hlds_goal.m:
Added dp_goal_info structure that stores information relevant to the deep
profiling pass.
Added egi_maybe_dp field to extra_goal_info to store a dp_goal_info
structure when required by the deep profiler.
compiler/hlds_pred.m:
Added a list of coverage_point_info structures to the hlds_proc_static
structure.
compiler/hlds_out.m:
Added the ability to dump the dp_goal_info structure when the correct
option is given.
compiler/layout.m:
Added extra layout structures to store coverage point static and dynamic
data for each procedure.
compiler/layout_out.m:
Added code to write out new layout structures in the Low Level C
Backend.
Added code to write out references to coverage point data from the proc
static structures.
Conform to changes in layout.m.
compiler/opt_debug.m:
Conform to changes in layout.m,
compiler/options.m:
Added command line parameters to enable coverage profiling and different
coverage points, as well as options that can cause coverage points not
to be inserted in some circumstances.
compiler/handle_options.m:
Added hlds dump options to the 'all' aliases for dumping the new
dp_goal_info structure in the hlds_info.
deep_profiler/dump.m:
Modified to dump coverage profiling data read in from Deep.data file.
deep_profiler/profile.m:
Added coverage points to proc static structure.
deep_profiler/read_profile.m:
Incremented Deep.data format version number.
Read coverage points from Deep.data file.
Confirm to changes in profile.m.
library/profiling_builtin.m:
Added real declaration and dummy implementation of
increment_coverage_point_count/2. This represents the instrumentation
introduced by the coverage profiling transformation.
mdbcomp/program_representation.m:
Added types to support coverage profiling, including a foreign_enum for
cp_type.
Added coverage_point_type_c_value to convert a coverage point type to a
string representing it's C value.
runtime/mercury_deep_profiling.c:
Incremented Deep.data format version.
Implemented writing out of coverage points to the Deep.data file.
runtime/mercury_deep_profiling.h:
Modified runtime structures to support storing coverage point
information in proc static structures.
doc/user_guide.texi:
Documented coverage profiling options, however this is commented out since
it's experimental.
|
||
|
Paul Bone
|
622b7171de |
Rename some fields within the profile_stats data-structure, The new names
Estimated hours taken: 0.10 Branches: main Rename some fields within the profile_stats data-structure, The new names now correctly convey information the fields represent. deep_profiler/profile.m: Renamed fields beginning with max_ to begin with num_, as they contain the number of items in the respective array. deep_profiler/query.m: Conform to changes in profile.m |
||
|
Zoltan Somogyi
|
2dc982cfe4 |
Make a representation of the program available to the deep profiler.
Estimated hours taken: 50 Branches: main Make a representation of the program available to the deep profiler. We do this by letting the user request, via the option "--deep-procrep-file" in MERCURY_OPTIONS, that when the Deep.data file is written, a Deep.procrep file should be written alongside it. The intended use of this information is the discovery of profitable parallelism. When a conjunction contains two expensive calls, e.g. p(...) and q(...) connected by some shared variables, the potential gain from executing them in parallel is limited by how early p produces those variables and how late q consumes them, and knowing this requires access to the code of p and q. Since the debugger and the deep profiler both need access to program representations, put the relevant data structures and the operations on them in mdbcomp. The data structures are significantly expanded, since the deep profiler deals with the whole program, while the debugger was interested only in one procedure at a time. The layout structures have to change as well. In a previous change, I changed proc layout structures to make room for the procedure representation even in non-debugging grades, but this isn't enough, since the procedure representation refers to the module's string table. This diff therefore makes some parts of the module layout structure, including of course the string table, also available in non-debugging grades. configure.in: Check whether the installed compiler can process switches on foreign enums correctly, since this diff depends on that. runtime/mercury_stack_layout.[ch]: runtime/mercury_types.h: Add a new structure, MR_ModuleCommonLayout, that holds the part of the module layout that is common to deep profiling and debugging. runtime/mercury_deep_profiling.[ch]: The old "deep profiling token" enum type was error prone, since at each point in the data file, only a subset was applicable. This diff breaks up the this enum into several enums, each consisting of the choice applicable at a given point. This also allows some of the resulting enums to be used in procrep files. Rename some enums and functions to avoid ambiguities, and in one case to conform to our naming scheme. Make write_out_proc_statics take a second argument. This is a FILE * that (if not NULL) asks write_out_proc_statics to write the representation of the current module to specified stream. These module representations go into the middle part of the program representation file. Add functions to write out the prologue and epilogue of this file. Write out procedure representations if this is requested. Factor out some code that is now used in more than one place. runtime/mercury_deep_profiling_hand.h: Conform to the changes to mercury_deep_profiling.h. runtime/mercury_builtin_types.c: Pass the extra argument in the argument lists of invocations of write_out_proc_statics. runtime/mercury_trace_base.[ch]: Conform to the name change from proc_rep to proc_defn_rep in mdbcomp. runtime/mercury_grade.h: Due to the change to layout structures, increment the binary compatibility version numbers for both debug and deep profiling grades. runtime/mercury_wrapper.[ch]: Provide two new MERCURY_OPTION options. The first --deep-procrep-file, allows the user to ask for the program representation to be generated. The second, --deep-random-write, allows tools/bootcheck to request that only a fraction of all program invocations should generate any deep profiling output. The first option will be documented once it is tested much more fully. The second option is deliberately not documented. Update the type of the variable that holds the address of the (mkinit-generated) write_out_proc_statics function to accept the second argument. util/mkinit.c: Pass the extra argument in the argument list of write_out_proc_statics. mdbcomp/program_representation.m: Extend the existing data structures for representing a procedure body to represent a procedure (complete with name), a module and a program. The name is implemented as string_proc_label, a form of proc_label that can be written out to files. This replaces the old proc_id type the deep profiler. Extend the representation of switches to record the identity of the variable being switched on, and the cons_ids of the arms. Without the former, we cannot be sure when a variable is first used, and the latter is needed for meaningful prettyprinting of procedure bodies. Add code for reading in files of bytecodes, and for making sense of the bytecodes themselves. (It is this code that uses foreign enums.) mdbcomp/prim_data.m: Note the relationship of proc_label with string_proc_label. mdbcomp/rtti_access.m: Add the access operations needed to find module string tables with the new organization of layout structures. Provide operations on bytecodes and string tables generally. trace/mercury_trace_cmd_browsing.c: Conform to the change to mdbcomp/program_representation.m. compiler/layout.m: Add support for a MR_ModuleCommonLayout. Rename some function symbols to avoid ambiguities. compiler/layout_out.m: Handle the new structure. compiler/stack_layout.m: Generate the new structure and the procedure representation bytecode in deep profiling grades. compiler/llds_out.m: Generate the code required to write out the prologue and epilogue of program representation files. Pass the extra argument in the argument lists of invocations of write_out_proc_statics that tells those invocations to write out the module representations between the prologue and the epilogue. compiler/prog_rep.m: When generating bytecodes, include the new information for switches. compiler/continuation_info.m: Replace a bool with a more expressive type. compiler/proc_rep.m: Conform to the change to continuation_info.m. compiler/opt_debug.m: Conform to the change to layout.m. deep_profiler/mdprof_procrep.m: A new test program to test the reading of program representations. deep_profiler/DEEP_FLAGS.in: deep_profiler/Mmakefile: Copy the contents of the mdbcomp module to this directory on demand, instead of linking to it. This is necessary now that the deep profiler depends directly on mdbcomp even if it is compiled in a non-debugging grade. The arrangements for doing this were copied from the slice directory, which has long done the same. Avoid a duplicate include of Mmake.deep.params. Add the new test program to the list of programs in this directory. Mmakefile: Go through deep_profiler/Mmakefile when deciding whether to do "mmake depend" in the deep_profiler directory. The old actions won't work correctly now that we need to copy some files from mdbcomp before we can run "mmake depend". deep_profiler/profile.m: Remove the code that was moved (in cleaned-up form) to mdbcomp. deep_profiler/dump.m: deep_profiler/profile.m: Conform to the changes above. browser/declarative_execution.m: browser/declarative_tree.m: Conform to the changes in mdbcomp. doc/user_guide.texi: Add commented out documentation of the two new options. slice/Mmakefile: Fix formatting, and a bug. library/exception.m: library/par_builtin.m: library/thread.m: library/thread.semaphore.m: Update all the handwritten modules to pass the extra argument now required by write_out_proc_statics. tests/debugger/declarative/dependency.exp: Conform to the change from proc_rep to proc_defn_rep. tools/bootcheck: Write out deep profiling data only from every 25th invocation, since otherwise the time for a bootcheck takes six times as long in deep profiling grades than in asm_fast.gc. However, do test the ability to write out program representations. Use the mkinit from the workspace, not the installed one. Don't disable line wrapping. |
||
|
Peter Wang
|
5ed14375b1 |
Respect the CGI environment variables SERVER_NAME, SERVER_PORT and
Branches: main deep_profiler/conf.m: deep_profiler/html_format.m: deep_profiler/interface.m: deep_profiler/mdprof_cgi.m: deep_profiler/mdprof_feedback.m: deep_profiler/mdprof_test.m: deep_profiler/profile.m: deep_profiler/startup.m: Respect the CGI environment variables SERVER_NAME, SERVER_PORT and SCRIPT_NAME. This allows the mdprof_cgi to be used with any web server, on an arbitrary port, at any URL. |
||
|
Zoltan Somogyi
|
c262737998 |
Include call sequence numbers in the pages generated by the deep profiler
Estimated hours taken: 16 Branches: main Include call sequence numbers in the pages generated by the deep profiler (if the user's preferences ask for this). NEWS: Announce the new capability. deep_profiler/measurements.m: Standardize on one spelling of call sequence numbers (callseqs). Add some utility predicates for use in dump.m. Fix a bug in the calculation of the number of calls: redos enter procedure invocations, not leave them. deep_profiler/profile.m: Fix a bug: the num_callseqs field was in the wrong place. deep_profiler/html_format.m: When generating a line of the clique description, include call sequence number information if requested. Do likewise when generating other HTML fragments that have to match the contents of those lines (e.g. the table headers). Generate the links that allow the users to sort rows by call sequence numbers. Generate the links that allow the users to toggle the switches controlling the display of call sequence number information. Do not show the toggles relevant to the display of times if times are not being displayed. deep_profiler/interface.m: Extend the preferences to allow requests for call sequence number information. Include the display of call sequence numbers in the default set of preferences. However, include the display of time information in the default only of the profiling run lasted long enough for this information to have some hope of being useful. Rename some function symbols to avoid ambiguities. deep_profiler/query.m: Extend the menu to include links for the most expensive cliques/procedures as measured by call sequence numbers. Include the links for the most expensive cliques/procedures as measured by clock ticks in the menu only if the profiling run lasted long enough for this to be useful. Print out the total number of call sequence numbers with the other statistics. Use call sequence numbers instead of time to decide where the "action" begins, since this works even for short profiling runs. deep_profiler/top_procs.m: Provide the capability of sorting on call sequence numbers. deep_profiler/dump.m: Provide the capability to dump out several more fields of the top level structure (profile.deep), including the cliques, the reverse links and the propagated measurement information. deep_profiler/startup.m: Dump out the cliques, the reverse links and the propagated measurement information when computed. deep_profiler/mdprof_cgi.m: Add an option for setting the hostname in the generated links to "localhost". This is useful on laptops temporarily disconnected from the Domain Name System (e.g. my laptop). Set this option automatically if the name of the file in the initial query has a ".localhost" suffix added to it. Add some tracing hooks. deep_profiler/mdprof_test.m: Add an option to specify what info to dump during startup. deep_profiler/mdprof_dump.m: Switch to using the compiler's method of specifying what to dump, since this is more maintainable. deep_profiler/exclude.m: Rename some function symbols to avoid ambiguities. deep_profiler/timeout.m: Add conditionally compiled code to help debug lock management. deep_profiler/callgraph.m: deep_profiler/canonical.m: deep_profiler/cliques.m: deep_profiler/conf.m: deep_profiler/read_profile.m: Minor cleanups. |
||
|
Zoltan Somogyi
|
35df0c4944 |
Include the count of call sequence numbers in the Deep.data file, since the
Estimated hours taken: 1 Branches: main Include the count of call sequence numbers in the Deep.data file, since the deep profiler needs to know this to generate percentages. runtime/mercury_deep_profiling.c: Write the count of call sequence numbers to the Deep.data file. deep_profiler/profile.m: Add a field to the profile for holding the count of call sequence numbers. deep_profiler/read_profile.m: Read in the count of call sequence numbers from the Deep.data file. deep_profiler/io_combinator.m: Add some functions now required by read_profile.m. Convert from DCGs to state variables. deep_profiler/canonical.m: deep_profiler/dump.m: Conform to the change to profile.m. |
||
|
Zoltan Somogyi
|
119ad537ae |
Rename some function symbols to avoid ambiguities.
Estimated hours taken: 0.1 Branches: main deep_profiler/*.m: Rename some function symbols to avoid ambiguities. |
||
|
Zoltan Somogyi
|
e33d2f67e7 |
Fix indentation.
Estimated hours taken: 0.1 Branches: main deep_profiler/profile.m: Fix indentation. deep_profiler/measurements.m: Avoid ambiguities in the names of some function symbols. |
||
|
Julien Fischer
|
459847a064 |
Move the univ, maybe, pair and unit types from std_util into their own
Estimated hours taken: 18 Branches: main Move the univ, maybe, pair and unit types from std_util into their own modules. std_util still contains the general purpose higher-order programming constructs. library/std_util.m: Move univ, maybe, pair and unit (plus any other related types and procedures) into their own modules. library/maybe.m: New module. This contains the maybe and maybe_error types and the associated procedures. library/pair.m: New module. This contains the pair type and associated procedures. library/unit.m: New module. This contains the types unit/0 and unit/1. library/univ.m: New module. This contains the univ type and associated procedures. library/library.m: Add the new modules. library/private_builtin.m: Update the declaration of the type_ctor_info struct for univ. runtime/mercury.h: Update the declaration for the type_ctor_info struct for univ. runtime/mercury_mcpp.h: runtime/mercury_hlc_types.h: Update the definition of MR_Univ. runtime/mercury_init.h: Fix a comment: ML_type_name is now exported from type_desc.m. compiler/mlds_to_il.m: Update the the name of the module that defines univs (which are handled specially by the il code generator.) library/*.m: compiler/*.m: browser/*.m: mdbcomp/*.m: profiler/*.m: deep_profiler/*.m: Conform to the above changes. Import the new modules where they are needed; don't import std_util where it isn't needed. Fix formatting in lots of modules. Delete duplicate module imports. tests/*: Update the test suite to confrom to the above changes. |
||
|
Zoltan Somogyi
|
a9a2825ace |
Replace __ with . as the module qualifier everywhere in all the modules
Estimated hours taken: 0.5 Branches: main profiler/*.m: deep_profiler/*.m: compiler/*.m: Replace __ with . as the module qualifier everywhere in all the modules of the profiler and deep profiler and in some modules of the compiler. |
||
|
Julien Fischer
|
2c686ea193 |
Convert the rest of the deep profiler to four-space indentation.
Estimated hours taken: 1.5 Branches: main Convert the rest of the deep profiler to four-space indentation. There are no changes to any algorithms (other than introducing state variables in a few spots). Remove an old bug workaround. deep_profiler/*.m: Convert to four-space indentation where that has not already been done. Fix some bad line breaks. Use state variables in a few places. Reposition comments according to our current coding standard. deep_profiler/Mercury.options: Remove some old bug workarounds. |
||
|
Julien Fischer
|
4083fe5bf2 |
Cleanup the deep profiler.
Estimated hours taken: 1 Branches: main Cleanup the deep profiler. There are no changes to any algorithms. deep_profiler/*.m: Switch to state variable notation throughout. Rearrange the order of some procedure arguments to facilitate this. Remove some unnecessary module imports. Reformat comments according to our current coding standard. Update comments that refer to modules that no longer exist. |
||
|
Zoltan Somogyi
|
d8694dc1aa |
Significantly enhance the infrastructure for debugging problems with the deep
Estimated hours taken: 8 Branches: main Significantly enhance the infrastructure for debugging problems with the deep profiler by providing facilities for dumping out some of the main data structures. These were needed for my recent fix of mercury_deep_profiling.c in the runtime. There are no algorithmic changes in the non-debugging parts of the code. deep_profiler/dump.m: New module for containing predicates that dump data structures. The first draft was by Julien. deep_profiler/mdprof_dump.m: New program for dumping out the contents of Deep.data files. The first draft was by Julien. deep_profiler/startup.m: Provide a mechanism for dumping out the main data structures at three stages. deep_profiler/mdprof_cgi.m: deep_profiler/mdprof_test.m: Pass the options to startup to control these dumps. deep_profiler/measurements.m: Provide a mechanism for getting all the counts of a measurement at once. deep_profiler/array_util.m: Provide extra modes of some predicates for use in the other modules. deep_profiler/read_profile.m: Fix Julien's fix: handle init predicates as well as the other uci predicates. deep_profiler/profile.m: Reorder the arguments of predicates as required to enable the use of state variable notation in the callers. deep_profiler/*.m: Conform to the change in profile.m, and use state variable notation where this is advantageous. Convert some modules to four-space indentation where this eliminates excessive indentation or bad line breaks. deep_profiler/Mmakefile: Handle the new mdprof_dump program. |
||
|
Zoltan Somogyi
|
69d84ff1bb |
Import only one module per line. Misc other updates to bring these
Estimated hours taken: 1 Branches: main profiler/*.m: deep_profiler/*.m: Import only one module per line. Misc other updates to bring these modules closer to being up to date with our current style guidelines. |
||
|
Zoltan Somogyi
|
c80d143cc8 |
The following change is only 98% complete, not 100%.
Estimated hours taken: 120
Branches: main
The following change is only 98% complete, not 100%. I am committing it in
this state because (1) we pass many more test cases in deep profiling grade
with it than without it, and (2) the double maintanance involved in fixing
CVS conflicts is preventing me from doing the last 2%.
Get the deep profiler to work for code that sets up to catch exceptions,
which for the last year or more has included the compiler, and to get it
almost working for code that actually catching exceptions.
The basic problem is that code that throws exceptions will in general cause
several calls to "return" without executing the exit or fail port codes that
the deep profiling transformation inserted into their bodies, and this leaves
the data structure being built by the deep profiling inconsistent. The solution
uses the same approach as we have adopted for the debugger: have the code that
handles the throwing of exceptions simulate a return from each call between
the throw and the catch, updating the deep profiling data structures as needed.
This requires us to be able to walk the stack at runtime not just in debugging
grades but also in deep profiling grades. Since the debugger already has the
mechanisms required for this, we reuse them. The procedure layouts used by the
debugger were designed to have three segments: the procedure stack walk
information, the procedure id, and the execution tracing information. We now
modify this design to make the third segment contain two pointers: to the
execution tracing information (for use by the debugger), and to the procedure's
proc_static structure (for use by deep profiling). Each pointer will be null
unless the pointed-to structure is required by compile-time options.
This common use by the debugger and by deep profiling of the stack-walk
structure and the procedure id structure (which deep profiling used to
generate independently and possibly redundantly) required some rearrangement
of the compiler's version of these data structures.
To make this rearrangement simpler, this diff removes a capability that
we theoretically supported but never used: turning on stack traces without
turning on execution tracing and vice versa. After this diff, stack tracing
is enabled if and only if either execution tracing or deep profiling is
enabled.
The diff also includes improvements in the debugging infrastructure for
debugging deep profiling, which were necessary for the implementation of the
rest of the changes.
compiler/deep_profiling.m:
The code in exception.m needs to know the locations of the variables
that we would pass to the exit or fail port code, so it can simulate
leaving the procedure invocation through the exception port. Without
this information, throwing an exception leaves the deep profiling
data structures of the procedure invocations between throw and catch
in an inconsistent state.
Deep_profiling.m creates these variables, but it doesn't know where
they will be at runtime, so it records their identities; the code
generator will allocate them stack slots and record the numbers of
these stack slots for placement in the now expanded proc layout
structures. Deep profiling used to generate static data structures
separately from the HLDS, but since the code generator now needs
access to them, we store their information in proc_infos in the HLDS.
Instead of passing the addresses of proc_static structures to the deep
profiling port procedures, pass the address of proc_layout structures,
since the information about the identities of procedures are now stored
not in the proc_static structure, but in the proc_layout structure
that points to the proc_static structure.
compiler/hlds_pred.m:
compiler/layout.m:
Move the definitions of the static data structures generated by deep
profiling from layout.m to hlds_pred.m, to allow deep_profiling.m
to store them in proc_infos.
compiler/hlds_pred.m:
compiler/rtti.m:
Move the definition of rtti_proc_label from rtti.m to hlds_pred.m,
since some of the new data structures in hlds_pred.m need it. Despite
its name, the rtti_proc_label type doesn't contain any info that
doesn't belong in the HLDS.
Add some information to the rtti_proc_label type that is now needed
by deep profiling, e.g. record determinisms instead of just code
models. Record explicitly the outcome of some tests that used to be
duplicated in more than one place in the compiler, e.g. for whether
the procedure (as opposed to the predicate) is imported. Change some
of the field names to be more precise about the field's meaning.
compiler/code_gen.m:
Transmit the contents of the deep profiling data structures stored in
the proc_info by deep_profiling.m to continuation_info.m, together
with the layout structures created for execution tracing and the
identities of the variables needed for handling exceptions,
when code generation for a procedure is complete.
After the goal that generates these variables, save them to stack
for use by the exception handler.
compiler/hlds_goal.m:
Add a feature to mark the goal that generates the deep profiling
variables needed by the exception handler.
compiler/hlds_llds.m:
Add a utility predicate for new code in code_gen.m
compiler/continuation_info.m:
Hold the deep profiling information computed by code_gen.m for use by
stack_layout.m.
compiler/layout.m:
compiler/layout_out.m:
Update the definitions of the data structures describing procedure
layouts, and the code writing them out, to reflect the use of some
parts of procedure layouts by deep profiling as well as debugging.
Change the layout structures generated by deep profiling to use
rtti_proc_labels, which are backend independent, instead of
proc_labels, which are specific to the LLDS backend.
Conform to the changes in runtime/mercury_stack_layout.h.
compiler/stack_layout.m:
Generate the updated version of proc_layout structures.
compiler/mercury_compile.m:
compiler/global_data.m:
Conform to the fact that deep profiling no longer generates layout
structures separate from proc_infos.
compiler/llds_out.m:
Register proc_layout structures instead of proc_static structures
for use by runtime/mercury_deep_profiling.c.
compiler/options.m:
compiler/handle_options.m:
Rename the require_tracing option as exec_trace, since this more
directly reflects its meaning.
Instead of having --debug set both require_tracing and stack_trace,
make it set (be the user-visible name of) just exec_trace;
the value of stack_trace is implied.
Turn off the specialization of deep profiling for self-tail-recursive
procedures for now. Due to the changes made by this diff in the data
structures involved in debugging, it cannot be debugged until this
change has been installed. Handling the full language is more important
than a specialization that reduces only stack space overheads, not
runtime overheads.
compiler/compile_target_code.m:
Conform to the changes in options.m and runtime/mercury_grade.h.
compiler/hlds_data.m:
Replace the deep_profiling_proc_static cons_id, and its associated tag,
to deep_profiling_proc_layout, since we now generate addresses of proc
layout structures, not of proc_static structures.
compiler/code_util.m:
Simplify some code based on the new info in rtti_proc_labels.
compiler/bytecode_gen.m:
compiler/dependency_graph.m:
compiler/higher_order.m:
compiler/hlds_out.m:
compiler/mercury_to_mercury.m:
compiler/ml_code_util.m:
compiler/ml_unify_gen.m:
compiler/opt_debug.m:
compiler/proc_label.m:
compiler/prog_rep.m:
compiler/rl_exprn.m:
compiler/rtti_out.m:
compiler/rtti_to_mlds.m:
compiler/saved_vars.m:
compiler/switch_util.m:
compiler/unify_gen.m:
Minor changes to conform to the change from deep_profiling_proc_static
to deep_profiling_proc_layout, to the change in the structure of
rtti_proc_labels, to the changes in types of layout.m, and/or to the
new goal feature.
deep_profiler/measurements.m:
Reserve space for exception counts.
deep_profiler/html_format.m:
Add a column for exception counts.
deep_profiler/profile.m:
deep_profiler/read_profile.m:
Rename the data structures referring to compiler generated unify,
compare and index predicates to avoid misleading names: they are
not the only compiler generated predicates.
deep_profiler/read_profile.m:
runtime/mercury_deep_profiling.c:
Update the string that identifies deep profiling data files.
This is necessary because the format has changed: it now includes
information about exception port counts.
library/exception.m:
In deep profiling grades, execute the exception port code for every
procedure invocation between a throw and a catch, using the procedure
layout structures now generated by the compiler for every procedure.
Rename the function involved to reflect its new, more general purpose.
Update the definitions of the hand-written proc_static and proc_layout
structures for the procedures implemented via hand-written C code.
Indent C preprocessor directives and foreign_procs according to our
coding standards.
library/profiling_builtin.m:
Change the parameters of the call port code procedures from proc_static
to proc_layout. Reach the proc_static structure from the proc_layout
structure when needed. Include the proc_layout structure in any
messages from assertion failures.
Add some conditionally compiled debugging code.
Give some variables better names.
runtime/mercury_type_info.h:
runtime/mercury_builtin_types.c:
Move the macros required to create the proc_static structures
of unify and compare predicates from mercury_type_info.h
to mercury_builtin_types.c, since the latter is the only file
that needs them.
Use the same macros for creating the proc_static structures
of hand-written unify, compare and compare_reprentation predicates
as for user defined predicates. This required changing their naming
scheme.
runtime/mercury_unify_compare_body.h:
Conform to the new naming scheme.
runtime/mercury_ho_call.c:
Provide the mechanism for mercury_unify_compare_body.h to conform
to the new naming scheme.
Remove the definitions of the proc_static structures for
hand-written unify, compare and compare_reprentation predicates,
since these now have to be defined together with the corresponding
proc_layout structures in mercury_builtin_types.c.
runtime/mercury_builtin_types.[ch]:
Update the definitions of the hand-written proc_static and proc_layout
structures for the procedures implemented via hand-written C code,
and add the required declarations first.
Handle deep profiling of compare_representation as well as unify
and compare predicates on builtin types.
Handle deep profiling of compare_representation on user-defined types,
since this is done entirely in the runtime, not by compiler generated
predicates.
runtime/mercury_builtin_types_proc_layouts.h:
New header file containing the declarations of the proc layout
structures of the unify, compare and index predicates of builtin types.
Logically, these declarations belong in mercury_builtin_types.h,
but putting them there causes problems for the linker; the details
are explained in the file itself.
runtime/Mmakefile:
Add the new header file.
runtime/mercury_minimal_model.[ch]:
Update the definitions of the hand-written proc_static and proc_layout
structures for the procedures implemented via hand-written C code,
and add the required declarations first.
runtime/mercury_grade.h:
Replace the MR_REQUIRE_TRACING grade option with MR_EXEC_TRACING.
Besides being better named, the MR_EXEC_TRACING option implies
MR_STACK_TRACE.
Besides the overall binary compatibility version number, add subsidiary
version numbers for binary compatibility in deep profiling and
debugging grades. These will make it easier to bootstrap changes
(such as this) that affect binary compatibility only in such grades.
runtime/mercury_trace_base.c:
trace/mercury_trace.c:
Conform to the new names of the configuration parameters.
runtime/mercury_hand_compare_body.h:
runtime/mercury_hand_unify_body.h:
runtime/mercury_hand_unify_compare_body.h:
runtime/mercury_ho_call.c:
tools/make_port_code:
Pass proc_layout structures instead of proc_static structures
to deep profiling port routines.
runtime/mercury_conf_param.h:
Make MR_DEEP_PROFILING as well as MR_EXEC_TRACING imply MR_STACK_TRACE,
since deep profiling now needs stack tracing. (MR_STACK_TRACE needs
to be set in this file, because tests in this file depend on knowing
its value, and this file is among the first files included (in this
case indirectly) in mercury_imp.h.)
Document the macros controlling the debugging of deep profiling.
Enable printing of label names when the relevant deep profiling
debugging macro is set.
runtime/mercury_debug.c:
runtime/mercury_deep_rec_depth_actions.h:
runtime/mercury_deep_rec_depth_body.h:
runtime/mercury_exception_catch_body.h:
Get to proc_statics via proc_layouts.
runtime/mercury_deep_call_port_body.c:
runtime/mercury_deep_leave_port_body.c:
Get to proc_statics via proc_layouts.
Allow the debugger to disable deep profiling in Mercury code that is
part of the debugger, not of the user program being executed.
Add some more assertions.
runtime/mercury_engine.[ch]:
Add a new debugging flag that controls at runtime whether we generate
a human readable Deep.debug equivalent to the binary Deep.data files.
(We already had a mechanism for controlling this at compile time,
but this isn't flexible enough.)
runtime/mercury_wrapper.c:
Allow this new debugging flag to be set from MERCURY_OPTIONS.
runtime/mercury_deep_profiling.[ch]:
Respect this new debugging flag.
Update the hand-written proc_static structures representing the runtime
system.
Print out addresses of proc_layout as well as proc_static structures
when assertions fail.
Add a field to the measurement structure for exception port counts,
and write out this field with the other port counts.
Remove procedure id information from proc_static structures,
deep profiling now uses the procedure id in the proc_layout structure.
Add to proc_static structures fields that specify where, if anywhere,
the variables needed by exception.m to executed the exception port code
are in the procedure's stack frame.
Define a global flag that allows the debugger to disable deep
profiling in Mercury code that is part of the debugger, not of the
user program being executed.
Increase type safety by providing two versions of the function
for registering proc_layouts, one for the proc_layout structures
of user-defined predicates and one for unify, compare and index
predicates.
Fix a bug that occurs only if MR_DEEP_PROFILING_EXPLICIT_CALL_COUNTS is
defined (which it usually isn't): the initial call count was wrong.
runtime/mercury_deep_profiling_hand.h:
Fix a bug: the handwritten code saving deep profiling variables was
saving them in slots that didn't belong to the relevant stack frame.
Update to conform to the modified definitions of proc_static structures
and the fact that we now reach them via proc_layout structures.
runtime/mercury_exception_catch_body.h:
runtime/mercury_stacks.h:
Fix the other side of the bug in mercury_deep_profiling_hand.h
by reserving the right number of stack slots in the stack frames
of the various modes of exception__catch. Make it harder to make
the same bug in the future by getting the needed info from the
place in mercury_stacks.h that defines the structure of the relevant
stack frame.
runtime/mercury_proc_id.h:
Rename the procedure id structure fields referring to compiler
generated unify, compare and index predicates: they are not the only
compiler-generated predicates.
runtime/mercury_stack_layout.h:
Change procedure layout structures to allow them to be used for deep
profiling as well as for debugging, as described in the prologue above.
We don't need the capability to support label layout structures with
links to misnamed proc layout structures, and supporting it is
inconvenient, so delete the capability.
runtime/mercury_debug.c:
runtime/mercury_deep_profiling_hand.h:
runtime/mercury_layout_util.c:
runtime/mercury_ml_expand_body.h:
runtime/mercury_stack_trace.c:
runtime/mercury_types.h:
trace/mercury_trace_external.c:
Conform to the new names of the procedure id structure fields.
runtime/mercury_std.h:
Add some more arities for MR_PASTE for use in some of the modified
modules in the runtime.
trace/mercury_trace_internal.c:
Disable deep profiling actions in Mercury code that is part of the
debugger, not of the program being debugged.
scripts/init_grade_options.sh-subr:
scripts/parse_grade_options.sh-subr:
Make changes parallel to the ones in runtime/mercury_grade.h: delete
--stack-trace as an independent option, and make --debug set its
own option, not --require-tracing.
scripts/canonical_grade.sh-subr:
scripts/final_grade_options.sh-subr:
scripts/c2init.in:
scripts/mgnuc.in:
scripts/ml.in:
Conform to the changes in grade options for debugging and for deep
profiling.
tools/bootcheck:
If Mmake.stage.{browser,deep,library,runtime,trace}.params exist,
copy them to become the file Mmake.$dir.params in stage2/$dir
(where dir is derived from the name of the original file in the obvious
way). This allows more flexibility in the creation of the stage2;
for example, it allows some directories (e.g. runtime or library)
to be compiled with more debugging than other directories (e.g.
compiler). This may be required because compiling all directories
with lots of debugging may cause the linker to thrash.
Add an option, --disable-debug-libs, that clobbers the libraries
that should be linked in only in debugging grades.
To conserve disk space, remove Deep.data files created by the bootcheck
by default. Add an option, --keep-deep-data, to preserve these files.
Use a consistent mechanism (test -f) for testing the existence of
all files whose existence is tested.
When recording modification times, record the modification times
of some more files.
tests/hard_coded/Mmakefile:
In deep profiling grades, disable the test cases that we don't now
pass in such grades, and document the reasons for their failure.
Fix the misclassification of the write_binary test case.
|
||
|
Zoltan Somogyi
|
d2b4120dfa |
Misc cleanups and documentation of various aspects of the deep profiler.
Estimated hours taken: 8 Branches: main Misc cleanups and documentation of various aspects of the deep profiler. More will come later. deep_profiler/startup.m: Fix a bug in the algorithm that we used to compensate for zeroing. The new algorithm is documented in the paper. deep_profiler/profile.m: deep_profiler/startup.m: deep_profiler/query.m: Switch the names we use for the data structures used in that algorithm to match the names in the paper. deep_profiler/query.m: Fix an off-by-one bug. deep_profiler/interface.m: Document this module. deep_profiler/interface.m: deep_profiler/mdprof_cgi.m: Move some code in interface.m that was used only in mdprof_cgi.m to mdprof_cgi.m. |
||
|
Zoltan Somogyi
|
a9fed43d57 |
Major enhancements of the deep profiler. The most important ones are:
Estimated hours taken: 80 Branches: main Major enhancements of the deep profiler. The most important ones are: - The addition of several new classes of preferences, including: the use of colour, boxing, sorting, summarizing of higher order call sites, and time formats. - Making preferences persistent across different queries, so that once a preference is set, it "sticks" until the user changes it. Previously, preferences stuck during query sequences of the same command type. - Several new command types: - listing all the modules of the program - listing all the procedures in a module - listing all the callers of a procedure, grouped by calling call site, procedure, module, or clique, and filtering out certain classes of ancestors - jumping into the call graph close to the action - restarting the server (useful when the data file changes) - New optional columns showing time per call, allocations per call and words allocated per call. - Can now display memory consumption in bytes as well as words. - More robustness in the face of external events, e.g. machine shutdowns. - Fix a bug reported by Tom in the summaries of procedures that make higher order calls. The new functionality required adding some fields to ProcStatic structures; as a result, compilers and runtime systems that have this change are incompatible with compilers and runtime systems before this change in deep profiling grades. (They of course remain compatible in other grades.) compiler/deep_profiling.m: compiler/layout.m: compiler/layout_out.m: Add two new fields to ProcStatic structures, one giving the line number of procedure's context and one stating whether the procedure is exported from its module. compiler/layout.m: Be consistent about filename vs file_name in field names. compiler/*.m: Minor changes to handle the new fields. deep_profiler/interface.m: Define new command types, modify some of the parameters of existing ones, and delete obsolete ones. Define the types and predicates used by the new system of preferences, Update the predicates for recognizing and generating queries accordingly. Make the order of declarations and definitions more consistent. deep_profiler/split.m: Complete rewrite of the only predicate of this module. The old split predicate deleted any empty substrings resulting from the breakup of the original string. The new one preserves them, because they are needed by the new encoding scheme used in interface.m. deep_profiler/query.m: New module, containing code dealing with the computational issues of queries. Some of its code is old (from server.m), much of it is new. deep_profiler/html_format.m: New module, containing code dealing with HTML formatting. Some of its code is old (from server.m), much of it is new. deep_profiler/top_procs.m: New module, containing code dealing with issues involving sorting by various criteria. Some of its code is old (from server.m), much of it is new. deep_profiler/exclude.m: New module to handle contour exclusion. This means that when listing the callers of a procedure, we display the nearest parent that is *not* excluded by virtue of being in a given module. The set of modules so excluded forms a contour drawn through the program. deep_profiler/mdprof_cgi.m: deep_profiler/mdprof_server.m: deep_profiler/server.m: Minor changes to adapt to the new system of preferences. deep_profiler/array_util.m: Add a mode to foldl2. deep_profiler/io_combinator.m: Add predicates for reading in sequences of ten things. deep_profiler/measurements.m: Add a function needed by new code. deep_profiler/timeout.m: Make the profiler robust in the face of signals. deep_profiler/canonical.m: Some more work towards working canonicalization; not there yet. Move some procedures to profile.m, since other modules also need them now. deep_profiler/profile.m: Add the new fields to ProcStatic structures. Record the word size. Record more information about procedures whose activation counters are ever zeroed, in order to allow query.m to avoid giving misleading information in cases where a procedure calls itself through a higher order call site. Record information about the modules of the program. Add a bunch of lookup predicates, some moved from canonical.m. deep_profiler/call_graph.m: Minor changes to conform to changes in profile.m. deep_profiler/startup.m: Fill in the new parts of the profile data structure. deep_profiler/read_profile.m: Read the new fields in ProcStatic structures. Read in the id of the root node as a fixed part of the header, not as a node. Read in the word size. Make it easier to find all the debugging output sites. Record, for each call site which can call more than one procedure, whether this causes the caller's ProcStatic structure's activation count to be zeroed. runtime/mercury_deep_profiling.h: Add the new fields to ProcStatic structures. runtime/mercury_deep_profiling.c: Write out the new fields to ProcStatic structures. Write out the id of the root node as a fixed part of the header, not as a node. Write out the word size. Remove incorrect L suffixes on constants. Record that the artificial procedure "parent of main" is called once, not zero times, to avoid division by zero when computing per-call statistics. runtime/mercury_deep_profiling_hand.h: Add the new fields to the macros for creating ProcStatic structures. runtime/mercury_ho_call.c: library/array.m: library/builtin.m: library/exception.m: library/std_util.m: Add the new fields to the invocations of those macros. |
||
|
Zoltan Somogyi
|
c2adfa6a9d |
Delete the ptr_info structure; it is not needed anymore.
Estimated hours taken: 12 Branches: main deep_profiler/read_profile.m: Delete the ptr_info structure; it is not needed anymore. deep_profiler/read_profile.m: deep_profiler/io_combinator.m: Separate out the I/O combinator part of read_profile.m into a new file. deep_profiler/startup.m: deep_profiler/canonical.m: Separate out the part of startup.m that attempts to canonicalize call graphs, and add code for merging call graphs from separate profiling runs. This code is not yet complete. deep_profiler/startup.m: deep_profiler/callgraph.m: Separate out the part of startup.m that generates the call graph (whose cliques we compute), for use by the new file canonical.m, and clean it up. deep_profiler/array_util.m: Make the names of the predicates more expressive. Add a new utility predicate. deep_profiler/interface.m: deep_profiler/server.m: Make the number of ancestor call sites displayed configurable. deep_profiler/measurements.m: Use a more compact representation for profiling information. This change reduced the resident set size of the server in one case from 130 Mb to 110 Mb. deep_profiler/profile.m: Remove the redirection field, since that info is now stored in separate, temporary arrays by canonical.m. Clarify the documentation of the fields giving the sizes of arrays. deep_profiler/server.m: Output more structured HTML. Make the format for times prettier. deep_profiler/notes/deep_profiling.html: Document the new modules. |
||
|
Zoltan Somogyi
|
04e614485d |
Implement deep profiling; merge the changes on the deep2 branch back
Estimated hours taken: 500 Branches: main Implement deep profiling; merge the changes on the deep2 branch back onto the trunk. The main documentation on the general architecture of the deep profiler is the deep profiling paper. doc/user_guide.texi: Document how to use the deep profiler. deep_profiler: deep_profiler/Mmakefile: A new directory holding the deep profiler and its mmakefile. Mmakefile: Add targets for the new directory. Add support for removing inappropriate files from directories. deep_profiler/interface.m: The deep profiler consists of two programs: mdprof_cgi.m, which acts as a CGI "script", and mdprof_server.m, which implements the server process that the CGI script talks to. Interface.m defines the interface between them. script/mdprof.in: A shell script template. ../configure uses it to generate mdprof, which is a wrapper around mdprof_cgi that tells it how to find mdprof_server. deep_profiler/mdprof_cgi.m: The CGI "script" program. deep_profiler/mdprof_server.m: The top level predicates of the server. deep_profiler/profile.m: The main data structures of the server and their operations. deep_profiler/read_profile.m: Code for reading in profiling data files. deep_profiler/startup.m: Code for post-processing the information in profiling data files, propagating costs from procedures to their ancestors and performing various kinds of summaries. deep_profiler/server.m: Code for responding to requests from the CGI script. deep_profiler/cliques.m: Code to find cliques in graphs. deep_profiler/array_util.m: deep_profiler/util.m: Utility predicates. deep_profiler/dense_bitset.m: An implementation of (part of) the set ADT with dense bit vectors. deep_profiler/measurements.m: Operations on profiling measurements. deep_profiler/timeout.m: An implementation of a timeout facility. deep_profiler/conf.m: Functions that depend on autoconfigured settings. configure.in: Find out what command to use to find the name of the local host. Install deep profiling versions of the standard library along with the other profiling versions. runtime/mercury_conf.h.in: Add some macros for deep_profiler/conf.m to use. library/profiling_builtin.m: runtime/mercury_deep_call_port_body.h: runtime/mercury_deep_leave_port_body.h: runtime/mercury_deep_redo_port_body.h: A new library module that implements deep profiling primitives. Some of these primitives have many versions, whose common code is factor is factored out in three new include files in the runtime. compiler/deep_profiling.m: New module to perform the program transformations described in the paper. compiler/notes/compiler_design.html: Document the new compiler module. compiler/mercury_compiler.m: Invoke the new module in deep profiling grades. Allow global static data to be generated by deep_profiling.m. compiler/options.m: Add options to turn on deep profiling and (for benchmarking purposes) control its implementation. Add an optiooption disable tailcall optimization in the LLDS backend, to help benchmarking deep profiling. compiler/jumpopt.m: compiler/optimize.m: Obey the option to disable tailcalls. compiler/handle_options.m: Handle the implications of deep profiling. compiler/modules.m: In deep profiling grades, automatically import profiling_builtin.m. compiler/prog_util.m: doc/Makefile: library/library.m: Handle the new builtin module. compiler/export.m: In deep profiling grades, wrap deep profiling code around exported procedures to handle the "unscheduled call" aspects of callbacks to Mercury from the foreign language. compiler/higher_order.m: profiler/demangle.m: util/demangle.c: When creating a name for a higher-order-specialized predicate, include the mode number in the name. compiler/add_trail_ops.m: compiler/type_util.m: Move c_pointer_type from add_trail_ops to type_util, so it can also be used by deep_profiling.m. compiler/hlds_goal.m: Add a new goal feature that marks a tail call, for use by deep_profiling.m. compiler/hlds_pred.m: Add a new field to proc_info structures for use by deep_profiling.m. Add a mechanism for getting proc_ids for procedure clones. Remove next_proc_id, an obsolete and unused predicate. compiler/hlds_data.m: Add a new cons_id to refer to the proc_static structure of a procedure. compiler/bytecode_gen.m: compiler/code_util.m: compiler/dependency_graph.m: compiler/hlds_out.m: compiler/mercury_to_mercury.m: compiler/ml_unify_gen.m: compiler/opt_debug.m: compiler/prog_rep.m: compiler/rl_exprn.m: compiler/switch_util.m: compiler/unify_gen.m: Trivial changes to handle the new cons_id, goal feature and/or proc_info argument. compiler/rtti.m: Add a utility predicate for extracting pred_id and proc_id from an rtti_proc_label, for use by hlds_out.m compiler/layout.m: compiler/layout_out.m: compiler/llds.m: compiler/llds_common.m: Add support for proc_static and call_site_static structures. compiler/layout_out.m: compiler/llds_out.m: Add code for the output of proc_static structures. compiler/code_util.m: Make code_util__make_proc_label_from_rtti a function, and export it. util/mkinit.c: compiler/llds_out.m: compiler/layout.m: compiler/modules.m: Add support for a fourth per-module C function, for writing out proc_static structures (and the call_site_static structures they contains). Since proc_static structures can be referred to from LLDS code (and not just from other static structures and compiler-generated C code), reorganize the declarations of static structures slightly. Change the schema for the name of the first per-module C function slightly, to make it the addition of the fourth function easier. The scheme now is: mercury__<modulename>__init mercury__<modulename>__init_type_tables mercury__<modulename>__init_debugger mercury__<modulename>__write_out_proc_statics Improve formatting of the generated C code. library/*.m: runtime/mercury.c: runtime/mercury_context.c: runtime/mercury_engine.c: runtime/mercury_ho_call.c: runtime/mercury_tabling.c: runtime/mercury_trace_base.c: runtime/mercury_wrapper.c: trace/mercrury_trace.[ch]: trace/mercrury_trace_declarative.c: trace/mercrury_trace_external.c: trace/mercrury_trace_internal.c: Conform to the new scheme for initialization functions for hand-written modules. compiler/mercury_compile.m: library/benchmarking.m: runtime/mercury_conf_param.h: runtime/mercury.h: runtime/mercury_engine.c: runtime/mercury_goto.c: runtime/mercury_grade.h: runtime/mercury_ho_call.c: runtime/mercury_label.[ch]: runtime/mercury_prof.[ch]: Add an MR_MPROF_ prefix in front of the C macros used to control the old profiler. compiler/handle_options.m: runtime/mercury_grade.h: scripts/canonical_grade.sh-subr: scripts/init_grade_options.sh-subr: scripts/parse_grade_options.sh-subr: Make deep profiling completely separate from the old profiling system, by making the deep profiling grade independent of MR_MPROF_PROFILE_TIME and the compiler option --profile-time. library/array.m: library/builtin.m: library/std_util.m: runtime/mercury_hand_unify_body.h: runtime/mercury_hand_compare_body.h: In deep profiling grades, wrap the deep profiling call, exit, fail and redo codes around the bodies of hand-written unification and comparison procedures. Make the reporting of array bounds violations switchable between making them fatal errors, as we currently, and reporting them by throwing an exception. Throwing an exception makes debugging code using arrays easier, but since exceptions aren't (yet) propagated across engine boundaries, we keep the old behaviour as the default; the new behaviour is for implementors. runtime/mercury_deep_profiling_hand.h: New file that defines macros for use in Mercury predicates whose definition is in hand-written C code. library/exception.m: runtime/mercury_exception_catch_body.h: runtime/mercury_stacks.h: In deep profiling grades, wrap the deep profiling call, exit, fail and redo codes around the bodies of the various modes of builtin_catch. Provide a function that C code can use to throw exceptions. library/benchmarking.m: library/exception.m: library/gc.m: library/std_util.m: runtime/mercury_context.[ch]: runtime/mercury_engine.[ch]: runtime/mercury_debug.c: runtime/mercury_deep_copy.c: runtime/mercury_overflow.h: runtime/mercury_regs.h: runtime/mercury_stacks.h: runtime/mercury_thread.c: runtime/mercury_wrapper.c: Add prefixes to the names of the fields in the engine and context structures, to make code using them easier to understand and modify. runtime/mercury_deep_profiling.[ch]: New module containing support functions for deep profiling and functions for writing out a deep profiling data file at the end of execution. runtime/mercury_debug.[ch]: Add support for debugging deep profiling. Add support for watching the value at a given address. Make the buffered/unbuffered nature of debugging output controllable via the -du option. Print register contents only if -dr is specified. runtime/mercury_goto.h: runtime/mercury_std.h: Use the macros in mercury_std.h instead of defining local variants. runtime/mercury_goto.h: runtime/mercury_stack_layout.h: runtime/mercury_stack_trace.c: runtime/mercury_tabling.c: trace/mercury_trace.c: trace/mercury_trace_declarative.c: trace/mercury_trace_external.c: trace/mercury_trace_vars.c: Standardize some of the macro names with those used in the debugger paper. runtime/mercury_heap.h: Add support for memory profiling with the deep profiler. runtime/mercury_prof.[ch]: runtime/mercury_prof_time.[ch]: Move the functionality that both the old profiler and the deep profiler need into the new module mercury_prof_time. Leave mercury_prof containing stuff that is only relevant to the old profiler. runtime/mercury_prof.[ch]: runtime/mercury_strerror.[ch]: Move the definition of strerror from mercury_prof to its own file. runtime/mercury_wrapper.[ch]: Add support for deep profiling. Add suppory for controlling whether debugging output is buffered or not. Add support for watching the value at a given address. runtime/Mmakefile: Mention all the added files. scripts/mgnuc.in: Add an option for turning on deep profiling. Add options for controlling the details of deep profiling. These are not documented because they are intended only for benchmarking the deep profiler itself, for the paper; they are not for general use. tools/bootcheck: Compile the deep_profiler directory as well as the other directories containing Mercury code. Turn off the creation of deep profiling data files during bootcheck, since all but one of these in each directory will be overwritten anyway. Add support for turning on --keep-objs by default in a workspace. tools/speedtest: Preserve any deep profiling data files created by the tests. trace/mercury_trace.c: Trap attempts to perform retries in deep profiling grades, since they would lead to core dumps otherwise. util/Mmakefile: Avoid compile-time warnings when compiling getopt. tests/*/Mmakefile: tests/*/*/Mmakefile: In deep profiling grades, switch off the tests that test features that don't work with deep profiling, either by design or because the combination hasn't been implemented yet. |