mirror of
https://github.com/Mercury-Language/mercury.git
synced 2026-04-17 10:23:46 +00:00
083d376e6598628362ee91c2da170febd83590f4
26 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Zoltan Somogyi
|
95f8f56716 | Delete unneeded $module args from calls to expect/unexpected. | ||
|
Zoltan Somogyi
|
9095985aa8 |
Fix more warnings from --warn-inconsistent-pred-order-clauses.
deep_profiler/*.m:
Fix inconsistencies between (a) the order in which functions and predicates
are declared, and (b) the order in which they are defined.
In most modules, either the order of the declarations or the order
of the definitions made sense, and I changed the other to match.
In some modules, neither made sense, so I changed *both* to an order
that *does* make sense (i.e. it has related predicates together).
In query.m, put the various commands in the same sensible order
as the code processing them.
In html_format.m, merge two exported functions together, since
they can't be used separately.
In some places, put dividers between groups of related
functions/predicates, to make the groups themselves more visible.
In some places, fix comments or programming style.
deep_profiler/DEEP_FLAGS.in:
Since all the modules in this directory are now free from any warnings
generated by --warn-inconsistent-pred-order-clauses, specify that option
by default in this directory to keep it that way.
|
||
|
Zoltan Somogyi
|
270170416d |
Bring the style of deep_profiler/* up-to-date.
deep_profiler/*.m:
Replace ( C -> T ; E ) if-then-elses with (if C then T else E ).
Replace calls to error/1 with calls to unexpected/3.
Add some module qualifications where this makes the code easier to read.
|
||
|
Zoltan Somogyi
|
59b0edacbe |
New module for calculating the overlap between the conjuncts of a
Estimated hours taken: 2 deep_profiler/autopar_calc_overlap.m: New module for calculating the overlap between the conjuncts of a parallelised conjunction. Its contents are taken from the old autopar_search_callgraph.m. deep_profiler/autopar_costs.m: New module for calculating the costs of goals. Its contents are taken from the old autopar_search_callgraph.m. deep_profiler/autopar_reports.m: New module for creating reports. Its contents are taken from the old autopar_search_callgraph.m. deep_profiler/autopar_search_goals.m: New module for searching goals for parallelizable conjunctions. Its contents are taken from the old autopar_search_callgraph.m. deep_profiler/autopar_search_callgraph.m: Remove the code moved to other modules. deep_profiler/mdprof_fb.automatic_parallelism.m: Add the new modules. deep_profiler/*.m: Remove unnecessary imports. Fix copyright years on the new modules. browser/*.m: compiler/*.m: mdbcomp/*.m: Remove unnecessary imports. library/Mercury.options: Make it possible to compile a whole workspace with --warn-unused-imports by turning that option off for type_desc.m (which has a necessary import that --warn-unused-imports thinks is unused). |
||
|
Zoltan Somogyi
|
599e1c2fdb |
Implement the proc command of the deep profiler using the report mechanism.
Estimated hours taken: 16 Branches: main Implement the proc command of the deep profiler using the report mechanism. Some of the detailed changes also address Paul's post-commit review of my previous diff. deep_profiler/report.m: Define the representation of the proc report. Change some names to be more consistent. deep_profiler/profile.m: Add a type for use in report.m. Document the fields of an existing type. deep_profiler/create_report.m: Add code to create proc reports. Change some names to be more consistent. Fix a bug: avoid divide by zero exceptions in some rare cases. deep_profiler/measurement units.m: Fix a bug: avoid divide by zero exceptions in some rare cases. deep_profiler/display_report.m: Add the code for converting proc reports to displays. This required rewriting almost everthing in this module to remove the assumption, which was previously embedded in many places, that the code is part of the implementation of the top_procs command. The new code is separated into two parts; the command-specific parts, and the parts that should be usable for all commands. (The reusable part is much bigger.) deep_profiler/display.m: Expand the display representation to allow the specification of - plain text as distinguished from headings - paragraph breaks - pseudo-links, i.e. text fragments that are formatted like links - table separator rows - table cells that span more than one column which are used in the new version of display_report.m. Simplify the mechanism for setting up table header groups. deep_profiler/html_format.m: Handle the changes in display.m. Fix some anomalies in the formatting of lists and tables. Group related code together. deep_profiler/query.m: Provide a switch for selecting which of the two mechanisms, the old direct generation of HTML or the new indirect generation through the report and display structures, should generate the HTML page for a command that both mechanisms can implement. The default is the new mechanism, but it can be overridden by implementors by creating a file called "/tmp/old_deep_profiler". Switch the default handling of the proc command over to the new mechanism. Rename some types and function symbols to make them more consistent and expressive. Generate more informative error messages for domain error exceptions. deep_profiler/mdprof_cgi.m: Add a mechanism for decoding the links in the generated pages. The mechanism takes the form of three new options: --decode, --decode-cmd, and --decode-prefs. When one or more of these are given, mdprof_cgi will not do its usual stuff, instead of just loops waiting for lines whose contents it will then try to decode as queries, as commands and/or as preferences. deep_profiler/Mercury.options: Try to postpone thrashing in the deep compiler when compiled in debug grades, by compiling their contents with shallow tracing. deep_profiler/array_util.m: Make shallow tracing effective by moving the deep recursions into non-exported predicates. deep_profiler/callgraph.m: deep_profiler/canonical.m: deep_profiler/startup.m: Convert the debugging code to use trace goals. deep_profiler/cliques.m: deep_profiler/read_profile.m: Convert the debugging code to use trace goals. Use more expressive names for lots of variables. In read_profile.m, avoid an unnecessary repackaging of data. deep_profiler/util.m: Fix some grammar errors in a comment, and comment some code to use state variables. deep_profiler/interface.m: Fix some grammar errors in a comment. deep_profiler/mdprof_feedback.m: Change some variable names to be consistent with the other modules. library/float.m: library/int.m: Make the messages in the domain errors we throw more expressive. library/io.m: Minor style changes. |
||
|
Zoltan Somogyi
|
03035ad2e6 |
Convert Paul's new code to use cords of strings to represent HTML.
Estimated hours taken: 6 Branches: main Convert Paul's new code to use cords of strings to represent HTML. deep_profiler/html_format.m: Convert to using cords. Restructure the code in a couple of places to always put start and end tags around HTML fragments together. Fix a missing "=" in a tag. deep_profiler/interface.m: deep_profiler/mdprof_cgi.m: deep_profiler/read_profile.m: Provide better diagnostics. deep_profiler/create_report.m: deep_profiler/display.m: deep_profiler/display_report.m: deep_profiler/mdprof_feedback.m: deep_profiler/measurement_units.m: deep_profiler/query.m: deep_profiler/report.m: mdbcomp/feedback.m: Misc cleanups. They can be considered my post-commit review of Paul's diff. In mdprof_feedback.m, delete a strange test that prevented the program from being used from the command line. deep_profiler/dump.m: deep_profiler/mdprof_dump.m: deep_profiler/timeout.m: deep_profiler/util.m: Misc cleanups of old code. |
||
|
Paul Bone
|
219724d9cb |
Tidying up some of the deep profiler tools code.
Estimated Hours Taken: 1
Branches: main
Tidying up some of the deep profiler tools code. Moved data structures,
predicates and functions that are not specific to the mdprof_cgi program
from interface.m to other modules.
deep_profilier/query.m:
Moved data structures from interface.m to query.m including the cmd and
preferences data structures.
Renamed functions to replace url_compoent with string.
deep_profilier/interface.m:
Moved code from this module that doesn't relate to the mdprof_cgi
program's client-server nature to query.m and html_format.m
Conformed to predicate-renaming changes in query.m
deep_profilier/html_format.m:
Moved machine_datafile_cmd_pref_to_url function to this module.
Conform to changes in interface.m and query.m.
deep_profiler/top_procs.m:
Conform to changes in interface.m and query.m.
|
||
|
Zoltan Somogyi
|
672f77c4ec |
Add a new compiler option. --inform-ite-instead-of-switch.
Estimated hours taken: 20 Branches: main Add a new compiler option. --inform-ite-instead-of-switch. If this is enabled, the compiler will generate informational messages about if-then-elses that it thinks should be converted to switches for the sake of program reliability. Act on the output generated by this option. compiler/simplify.m: Implement the new option. Fix an old bug that could cause us to generate warnings about code that was OK in one duplicated copy but not in another (where a switch arm's code is duplicated due to the case being selected for more than one cons_id). compiler/options.m: Add the new option. Add a way to test for the bug fix in simplify. doc/user_guide.texi: Document the new option. NEWS: Mention the new option. library/*.m: mdbcomp/*.m: browser/*.m: compiler/*.m: deep_profiler/*.m: Convert if-then-elses to switches at most of the sites suggested by the new option. At the remaining sites, switching to switches would have nontrivial downsides. This typically happens with the switched-on type has many functors, and we treat one or two specially (e.g. cons/2 in the cons_id type). Perform misc cleanups in the vicinity of the if-then-else to switch conversions. In a few cases, improve the error messages generated. compiler/accumulator.m: compiler/hlds_goal.m: (Rename and) move insts for particular kinds of goal from accumulator.m to hlds_goal.m, to allow them to be used in other modules. Using these insts allowed us to eliminate some if-then-elses entirely. compiler/exprn_aux.m: Instead of fixing some if-then-elses, delete the predicates containing them, since they aren't used, and (as pointed out by the new option) would need considerable other fixing if they were ever needed again. compiler/lp_rational.m: Add prefixes to the names of the function symbols on some types, since without those prefixes, it was hard to figure out what type the switch corresponding to an old if-then-else was switching on. tests/invalid/reserve_tag.err_exp: Expect a new, improved error message. |
||
|
Paul Bone
|
907e6aab3c |
Add the ability to select top procedures in the deep profiling tool based on a
Estimated hours taken: 1.5
Branches: main
Add the ability to select top procedures in the deep profiling tool based on a
raw threshold rather than a percentage. This will be useful when selecting
procedures to be implicitly parallelized. Add links to the top-level screen of
the deep profiler for accessing this feature.
deep_profiler/interface.m:
Added extra value for display_limit type, threshold_value to represent this
new value-based threshold of procedures.
Alter string conversion predicates for limits to include this new value and
ensure it's distinct from existing values.
deep_profiler/query.m:
Added links to main screen to access reports using this feature. Default
thresholds have been chosen.
deep_profiler/top_procs.m:
Altered find_top_procs to accept this new display_limit and properly
exclude procedures below the value threshold.
Added new predicate find_threshold_value_predicate which reference new
predicates to select rows above the threshold value.
Renamed existing find_threshold_predicate to
find_threshold_percent_predicate, as well as the predicates it looks up.
|
||
|
Peter Wang
|
5ed14375b1 |
Respect the CGI environment variables SERVER_NAME, SERVER_PORT and
Branches: main deep_profiler/conf.m: deep_profiler/html_format.m: deep_profiler/interface.m: deep_profiler/mdprof_cgi.m: deep_profiler/mdprof_feedback.m: deep_profiler/mdprof_test.m: deep_profiler/profile.m: deep_profiler/startup.m: Respect the CGI environment variables SERVER_NAME, SERVER_PORT and SCRIPT_NAME. This allows the mdprof_cgi to be used with any web server, on an arbitrary port, at any URL. |
||
|
Julien Fischer
|
b4c3bb1387 |
Clean up in unused module imports in the Mercury system detected
Estimated hours taken: 3 Branches: main Clean up in unused module imports in the Mercury system detected by --warn-unused-imports. analysis/*.m: browser/*.m: deep_profiler/*.m: compiler/*.m: library/*.m: mdbcomp/*.m: profiler/*.m: slice/*.m: Remove unused module imports. Fix some minor departures from our coding standards. analysis/Mercury.options: browser/Mercury.options: deep_profiler/Mercury.options: compiler/Mercury.options: library/Mercury.options: mdbcomp/Mercury.options: profiler/Mercury.options: slice/Mercury.options: Set --no-warn-unused-imports for those modules that are used as packages or otherwise break --warn-unused-imports, e.g. because they contain predicates with both foreign and Mercury clauses and some of the imports only depend on the latter. |
||
|
Zoltan Somogyi
|
c262737998 |
Include call sequence numbers in the pages generated by the deep profiler
Estimated hours taken: 16 Branches: main Include call sequence numbers in the pages generated by the deep profiler (if the user's preferences ask for this). NEWS: Announce the new capability. deep_profiler/measurements.m: Standardize on one spelling of call sequence numbers (callseqs). Add some utility predicates for use in dump.m. Fix a bug in the calculation of the number of calls: redos enter procedure invocations, not leave them. deep_profiler/profile.m: Fix a bug: the num_callseqs field was in the wrong place. deep_profiler/html_format.m: When generating a line of the clique description, include call sequence number information if requested. Do likewise when generating other HTML fragments that have to match the contents of those lines (e.g. the table headers). Generate the links that allow the users to sort rows by call sequence numbers. Generate the links that allow the users to toggle the switches controlling the display of call sequence number information. Do not show the toggles relevant to the display of times if times are not being displayed. deep_profiler/interface.m: Extend the preferences to allow requests for call sequence number information. Include the display of call sequence numbers in the default set of preferences. However, include the display of time information in the default only of the profiling run lasted long enough for this information to have some hope of being useful. Rename some function symbols to avoid ambiguities. deep_profiler/query.m: Extend the menu to include links for the most expensive cliques/procedures as measured by call sequence numbers. Include the links for the most expensive cliques/procedures as measured by clock ticks in the menu only if the profiling run lasted long enough for this to be useful. Print out the total number of call sequence numbers with the other statistics. Use call sequence numbers instead of time to decide where the "action" begins, since this works even for short profiling runs. deep_profiler/top_procs.m: Provide the capability of sorting on call sequence numbers. deep_profiler/dump.m: Provide the capability to dump out several more fields of the top level structure (profile.deep), including the cliques, the reverse links and the propagated measurement information. deep_profiler/startup.m: Dump out the cliques, the reverse links and the propagated measurement information when computed. deep_profiler/mdprof_cgi.m: Add an option for setting the hostname in the generated links to "localhost". This is useful on laptops temporarily disconnected from the Domain Name System (e.g. my laptop). Set this option automatically if the name of the file in the initial query has a ".localhost" suffix added to it. Add some tracing hooks. deep_profiler/mdprof_test.m: Add an option to specify what info to dump during startup. deep_profiler/mdprof_dump.m: Switch to using the compiler's method of specifying what to dump, since this is more maintainable. deep_profiler/exclude.m: Rename some function symbols to avoid ambiguities. deep_profiler/timeout.m: Add conditionally compiled code to help debug lock management. deep_profiler/callgraph.m: deep_profiler/canonical.m: deep_profiler/cliques.m: deep_profiler/conf.m: deep_profiler/read_profile.m: Minor cleanups. |
||
|
Julien Fischer
|
fa9d39c7c3 |
Change the behaviour of the deep profiling tool so that it
Estimated hours taken: 0.1 Branches: main, release Change the behaviour of the deep profiling tool so that it hides inactive modules/procedures by default. deep_profiler/interface.m: Hide inactive modules/procedures by default. |
||
|
Julien Fischer
|
459847a064 |
Move the univ, maybe, pair and unit types from std_util into their own
Estimated hours taken: 18 Branches: main Move the univ, maybe, pair and unit types from std_util into their own modules. std_util still contains the general purpose higher-order programming constructs. library/std_util.m: Move univ, maybe, pair and unit (plus any other related types and procedures) into their own modules. library/maybe.m: New module. This contains the maybe and maybe_error types and the associated procedures. library/pair.m: New module. This contains the pair type and associated procedures. library/unit.m: New module. This contains the types unit/0 and unit/1. library/univ.m: New module. This contains the univ type and associated procedures. library/library.m: Add the new modules. library/private_builtin.m: Update the declaration of the type_ctor_info struct for univ. runtime/mercury.h: Update the declaration for the type_ctor_info struct for univ. runtime/mercury_mcpp.h: runtime/mercury_hlc_types.h: Update the definition of MR_Univ. runtime/mercury_init.h: Fix a comment: ML_type_name is now exported from type_desc.m. compiler/mlds_to_il.m: Update the the name of the module that defines univs (which are handled specially by the il code generator.) library/*.m: compiler/*.m: browser/*.m: mdbcomp/*.m: profiler/*.m: deep_profiler/*.m: Conform to the above changes. Import the new modules where they are needed; don't import std_util where it isn't needed. Fix formatting in lots of modules. Delete duplicate module imports. tests/*: Update the test suite to confrom to the above changes. |
||
|
Zoltan Somogyi
|
a9a2825ace |
Replace __ with . as the module qualifier everywhere in all the modules
Estimated hours taken: 0.5 Branches: main profiler/*.m: deep_profiler/*.m: compiler/*.m: Replace __ with . as the module qualifier everywhere in all the modules of the profiler and deep profiler and in some modules of the compiler. |
||
|
Julien Fischer
|
2c686ea193 |
Convert the rest of the deep profiler to four-space indentation.
Estimated hours taken: 1.5 Branches: main Convert the rest of the deep profiler to four-space indentation. There are no changes to any algorithms (other than introducing state variables in a few spots). Remove an old bug workaround. deep_profiler/*.m: Convert to four-space indentation where that has not already been done. Fix some bad line breaks. Use state variables in a few places. Reposition comments according to our current coding standard. deep_profiler/Mercury.options: Remove some old bug workarounds. |
||
|
Julien Fischer
|
b59dc73825 |
Improvements to the deep profiling tool.
Estimated hours taken: 2 Branches: main, release Improvements to the deep profiling tool. Provide the facility to hide modules/procedures to which there have been no calls. This reduces the amount of clutter when exploring the program module by module and when listing the procedures in a module (the clutter is primarily a result of unused standard library modules). By default, we currently do not hide any modules/procedures. Add a toggle to sort lists of procedures by the number of redos they perform. We have had control at the head of tables to do this for a while now. Delete the function int_list_from_to/2 since that functionality is now available in the standard library. deep_profiler/html_format.m: Provide toggles that allow the user to show/hide inactive modules and procedures. Add a toggle to sort lists of procedures by the number of redos. Rename a numbered sequence of variables so that it is easier to insert new ones in the middle of the sequence. deep_profiler/interface.m: Add support for hiding/showing inactive modules. Clean up some of the existing code. deep_profiler/measurements.m: Add a predicate, is_inactive/1, that test if an own_prof_info for a given entity is inactive. deep_profiler/query.m: When generating the module and procedure summaries, respect the new preference settings for hiding/showing inactive modules/procedures. deep_profiler/top_procs.m: Delete int_list_from_to/2 and replace calls to it with '..' from the standard library. |
||
|
Julien Fischer
|
5bde1ac946 |
Add a control that allows procedures to be ordered by the
Estimated hours taken: 0.5 Branches: main, release deep_profiler/html_format.m: deep_profiler/interface.m: deep_profiler/top_procs.m: Add a control that allows procedures to be ordered by the number of redos they execute. |
||
|
Julien Fischer
|
bb67a2d90b |
Fix some problems in the deep profiling tool.
Estimated hours taken: 3.5 Branches: main, release Fix some problems in the deep profiling tool. deep_profiler/html_format.m: The control to view times was mislabeled as 'Ticks and times'. Update the number of columns used for displaying the port counts. Since (some) support for exceptions was added there are now five. This fixes a problem where the width of separator rows was incorrect. Avoid the division by zero that occurs when calculating the time-per-call and there have been zero calls. deep_profiler/interface.m: Change the query separator character from '%' to '&'. The former is used for encoding special characters in URLs and our usage of it causes problems for some web browsers. In particular, this is necessary to get the deep profiling tool to work with Safari, which in turn is necessary for getting the deep profiler to work "out-of-the-box" on OS X. |
||
|
Zoltan Somogyi
|
69d84ff1bb |
Import only one module per line. Misc other updates to bring these
Estimated hours taken: 1 Branches: main profiler/*.m: deep_profiler/*.m: Import only one module per line. Misc other updates to bring these modules closer to being up to date with our current style guidelines. |
||
|
Zoltan Somogyi
|
3c82dc3867 |
Switch to using state variable notation where this is beneficial.
Estimated hours taken: 0.5 Branches: main deep_profiler/*.m: Switch to using state variable notation where this is beneficial. There are no changes in algorithms. |
||
|
Fergus Henderson
|
e86057942c |
Apply Zoltan's deep profiler changes to the main branch.
Estimated hours taken: 2 Branches: main Apply Zoltan's deep profiler changes to the main branch. Zoltan committed these changes to the release branch, but forgot to commit them to the main branch. Mmakefile: NEWS: configure.in: bindist/Mmakefile: bindist/bindist.Makefile.in: doc/user_guide.texi: library/exception.m: library/io.m: runtime/mercury_conf.h.in: runtime/mercury_conf_param.h: runtime/mercury_misc.c: runtime/mercury_misc.h: scripts/Mmakefile: deep_profiler/.nocopyright: deep_profiler/Mercury.options: deep_profiler/Mmakefile: deep_profiler/callgraph.m: deep_profiler/cliques.m: deep_profiler/conf.m: deep_profiler/html_format.m: deep_profiler/interface.m: deep_profiler/mdprof_cgi.m: deep_profiler/query.m: deep_profiler/startup.m: deep_profiler/timeout.m: deep_profiler/mdprof_server.m: deep_profiler/server.m: Merge in changes from the release branch. Enclosed below is Zoltan's original log message. ---------- Estimated hours taken: 0.3 (on release branch) Branches: release Make the deep profiler work again. The existing process structure of the deep profiler made it very hard to debug, mainly because the Mercury debugger is confused by forks and stymied by execs. This change completely replaces the process structure. The old structure had two separate programs, mdprof_cgi and mdprof_server, the first always acting as client and the second always acting as server. The new structure has only one program, mdprof_cgi, which acts as a server if there is no existing server for the relevant profiling data file, and as a client otherwise. Although mdprof_cgi normally forks when it becomes a server to let the parent exit and let the web server know that the web page it has generated is complete, the fork can be disabled for debugging via an option. This allows the communication between client and server to be debugged by running two instances of mdprof_cgi in different windows, one or both under mdb. deep_profiler/DESIGN: New file describing the new process structure, its race conditions, and their solutions. deep_profiler/.nocopyright: Add DESIGN. deep_profiler/mdprof_cgi.m: A complete rewrite of this module to enable it act as both client and server. deep_profiler/mdprof_test.m: A new module to hold the testing functionality of mdprof_server.m. deep_profiler/mdprof_server.m: deep_profiler/server.m: Delete these now unnecessary modules. Mmakefile: Replace references to mdprof_server with references to mdprof_test. deep_profiler/Mmakefile: Replace references to mdprof_server with references to mdprof_test. Move the include of Mmake.deep.params, to allow it to override top level parameter settings. deep_profiler/Mercury.options: Work around a compiler bug by turning off the offending optimization. deep_profiler/timeout.m: A rewrite of major parts of this module to support the new design of mdprof_cgi.m, and to make unexpected signals easier to debug. deep_profiler/interface.m: Implement the mechanisms needed by the new process structure. Change the characters we use to separate components of the URL. The old ones were special to the shell, and screwed up command lines. (As double insurance, we ignore the command line anyway when invoked by the web server.) Change some names to be more expressive. deep_profiler/conf.m: Add a new function, getpid, for use by interface.m. Rewrite some code to use streams explicitly, not implicitly. deep_profiler/callgraph.m: deep_profiler/cliques.m: Add (now commented out) code to help debug these modules, for use in cases where mdb doesn't help, because the program works perfectly with debugging enabled :-( deep_profiler/query.m: Move the predicate try_exec here from the deleted file server.m. deep_profiler/html_format.m: Trivial change to conform to name change in interface.m. deep_profiler/startup.m: Generate debugging output to a caller specified stream, not to stdout and stderr. Disable the generation of statistics temporarily, since the diff to make statistics reporting routines write to a specified stream instead of stdout and stderr won't be committed on the release branch. Currently, they always write to stdout, which in the new design goes to the web page, not to the startup file. configure.in: Detect the presence of opendir, readdir and closedir, and the header file they need, dirent.h. Enable the deep profiler only if all exist, since the deep profiler now needs them. runtime/mercury_conf.h.in: Support the changes to configure.in. runtime/mercury_misc.[ch]: Add a mechanism for registering cleanup functions to be executed when we terminate the program due to an uncaught exception. library/exception.m: Invoke the registered cleanup functions just before terminating the program due to an uncaught exception. |
||
|
Zoltan Somogyi
|
d2b4120dfa |
Misc cleanups and documentation of various aspects of the deep profiler.
Estimated hours taken: 8 Branches: main Misc cleanups and documentation of various aspects of the deep profiler. More will come later. deep_profiler/startup.m: Fix a bug in the algorithm that we used to compensate for zeroing. The new algorithm is documented in the paper. deep_profiler/profile.m: deep_profiler/startup.m: deep_profiler/query.m: Switch the names we use for the data structures used in that algorithm to match the names in the paper. deep_profiler/query.m: Fix an off-by-one bug. deep_profiler/interface.m: Document this module. deep_profiler/interface.m: deep_profiler/mdprof_cgi.m: Move some code in interface.m that was used only in mdprof_cgi.m to mdprof_cgi.m. |
||
|
Zoltan Somogyi
|
a9fed43d57 |
Major enhancements of the deep profiler. The most important ones are:
Estimated hours taken: 80 Branches: main Major enhancements of the deep profiler. The most important ones are: - The addition of several new classes of preferences, including: the use of colour, boxing, sorting, summarizing of higher order call sites, and time formats. - Making preferences persistent across different queries, so that once a preference is set, it "sticks" until the user changes it. Previously, preferences stuck during query sequences of the same command type. - Several new command types: - listing all the modules of the program - listing all the procedures in a module - listing all the callers of a procedure, grouped by calling call site, procedure, module, or clique, and filtering out certain classes of ancestors - jumping into the call graph close to the action - restarting the server (useful when the data file changes) - New optional columns showing time per call, allocations per call and words allocated per call. - Can now display memory consumption in bytes as well as words. - More robustness in the face of external events, e.g. machine shutdowns. - Fix a bug reported by Tom in the summaries of procedures that make higher order calls. The new functionality required adding some fields to ProcStatic structures; as a result, compilers and runtime systems that have this change are incompatible with compilers and runtime systems before this change in deep profiling grades. (They of course remain compatible in other grades.) compiler/deep_profiling.m: compiler/layout.m: compiler/layout_out.m: Add two new fields to ProcStatic structures, one giving the line number of procedure's context and one stating whether the procedure is exported from its module. compiler/layout.m: Be consistent about filename vs file_name in field names. compiler/*.m: Minor changes to handle the new fields. deep_profiler/interface.m: Define new command types, modify some of the parameters of existing ones, and delete obsolete ones. Define the types and predicates used by the new system of preferences, Update the predicates for recognizing and generating queries accordingly. Make the order of declarations and definitions more consistent. deep_profiler/split.m: Complete rewrite of the only predicate of this module. The old split predicate deleted any empty substrings resulting from the breakup of the original string. The new one preserves them, because they are needed by the new encoding scheme used in interface.m. deep_profiler/query.m: New module, containing code dealing with the computational issues of queries. Some of its code is old (from server.m), much of it is new. deep_profiler/html_format.m: New module, containing code dealing with HTML formatting. Some of its code is old (from server.m), much of it is new. deep_profiler/top_procs.m: New module, containing code dealing with issues involving sorting by various criteria. Some of its code is old (from server.m), much of it is new. deep_profiler/exclude.m: New module to handle contour exclusion. This means that when listing the callers of a procedure, we display the nearest parent that is *not* excluded by virtue of being in a given module. The set of modules so excluded forms a contour drawn through the program. deep_profiler/mdprof_cgi.m: deep_profiler/mdprof_server.m: deep_profiler/server.m: Minor changes to adapt to the new system of preferences. deep_profiler/array_util.m: Add a mode to foldl2. deep_profiler/io_combinator.m: Add predicates for reading in sequences of ten things. deep_profiler/measurements.m: Add a function needed by new code. deep_profiler/timeout.m: Make the profiler robust in the face of signals. deep_profiler/canonical.m: Some more work towards working canonicalization; not there yet. Move some procedures to profile.m, since other modules also need them now. deep_profiler/profile.m: Add the new fields to ProcStatic structures. Record the word size. Record more information about procedures whose activation counters are ever zeroed, in order to allow query.m to avoid giving misleading information in cases where a procedure calls itself through a higher order call site. Record information about the modules of the program. Add a bunch of lookup predicates, some moved from canonical.m. deep_profiler/call_graph.m: Minor changes to conform to changes in profile.m. deep_profiler/startup.m: Fill in the new parts of the profile data structure. deep_profiler/read_profile.m: Read the new fields in ProcStatic structures. Read in the id of the root node as a fixed part of the header, not as a node. Read in the word size. Make it easier to find all the debugging output sites. Record, for each call site which can call more than one procedure, whether this causes the caller's ProcStatic structure's activation count to be zeroed. runtime/mercury_deep_profiling.h: Add the new fields to ProcStatic structures. runtime/mercury_deep_profiling.c: Write out the new fields to ProcStatic structures. Write out the id of the root node as a fixed part of the header, not as a node. Write out the word size. Remove incorrect L suffixes on constants. Record that the artificial procedure "parent of main" is called once, not zero times, to avoid division by zero when computing per-call statistics. runtime/mercury_deep_profiling_hand.h: Add the new fields to the macros for creating ProcStatic structures. runtime/mercury_ho_call.c: library/array.m: library/builtin.m: library/exception.m: library/std_util.m: Add the new fields to the invocations of those macros. |
||
|
Zoltan Somogyi
|
c2adfa6a9d |
Delete the ptr_info structure; it is not needed anymore.
Estimated hours taken: 12 Branches: main deep_profiler/read_profile.m: Delete the ptr_info structure; it is not needed anymore. deep_profiler/read_profile.m: deep_profiler/io_combinator.m: Separate out the I/O combinator part of read_profile.m into a new file. deep_profiler/startup.m: deep_profiler/canonical.m: Separate out the part of startup.m that attempts to canonicalize call graphs, and add code for merging call graphs from separate profiling runs. This code is not yet complete. deep_profiler/startup.m: deep_profiler/callgraph.m: Separate out the part of startup.m that generates the call graph (whose cliques we compute), for use by the new file canonical.m, and clean it up. deep_profiler/array_util.m: Make the names of the predicates more expressive. Add a new utility predicate. deep_profiler/interface.m: deep_profiler/server.m: Make the number of ancestor call sites displayed configurable. deep_profiler/measurements.m: Use a more compact representation for profiling information. This change reduced the resident set size of the server in one case from 130 Mb to 110 Mb. deep_profiler/profile.m: Remove the redirection field, since that info is now stored in separate, temporary arrays by canonical.m. Clarify the documentation of the fields giving the sizes of arrays. deep_profiler/server.m: Output more structured HTML. Make the format for times prettier. deep_profiler/notes/deep_profiling.html: Document the new modules. |
||
|
Zoltan Somogyi
|
04e614485d |
Implement deep profiling; merge the changes on the deep2 branch back
Estimated hours taken: 500 Branches: main Implement deep profiling; merge the changes on the deep2 branch back onto the trunk. The main documentation on the general architecture of the deep profiler is the deep profiling paper. doc/user_guide.texi: Document how to use the deep profiler. deep_profiler: deep_profiler/Mmakefile: A new directory holding the deep profiler and its mmakefile. Mmakefile: Add targets for the new directory. Add support for removing inappropriate files from directories. deep_profiler/interface.m: The deep profiler consists of two programs: mdprof_cgi.m, which acts as a CGI "script", and mdprof_server.m, which implements the server process that the CGI script talks to. Interface.m defines the interface between them. script/mdprof.in: A shell script template. ../configure uses it to generate mdprof, which is a wrapper around mdprof_cgi that tells it how to find mdprof_server. deep_profiler/mdprof_cgi.m: The CGI "script" program. deep_profiler/mdprof_server.m: The top level predicates of the server. deep_profiler/profile.m: The main data structures of the server and their operations. deep_profiler/read_profile.m: Code for reading in profiling data files. deep_profiler/startup.m: Code for post-processing the information in profiling data files, propagating costs from procedures to their ancestors and performing various kinds of summaries. deep_profiler/server.m: Code for responding to requests from the CGI script. deep_profiler/cliques.m: Code to find cliques in graphs. deep_profiler/array_util.m: deep_profiler/util.m: Utility predicates. deep_profiler/dense_bitset.m: An implementation of (part of) the set ADT with dense bit vectors. deep_profiler/measurements.m: Operations on profiling measurements. deep_profiler/timeout.m: An implementation of a timeout facility. deep_profiler/conf.m: Functions that depend on autoconfigured settings. configure.in: Find out what command to use to find the name of the local host. Install deep profiling versions of the standard library along with the other profiling versions. runtime/mercury_conf.h.in: Add some macros for deep_profiler/conf.m to use. library/profiling_builtin.m: runtime/mercury_deep_call_port_body.h: runtime/mercury_deep_leave_port_body.h: runtime/mercury_deep_redo_port_body.h: A new library module that implements deep profiling primitives. Some of these primitives have many versions, whose common code is factor is factored out in three new include files in the runtime. compiler/deep_profiling.m: New module to perform the program transformations described in the paper. compiler/notes/compiler_design.html: Document the new compiler module. compiler/mercury_compiler.m: Invoke the new module in deep profiling grades. Allow global static data to be generated by deep_profiling.m. compiler/options.m: Add options to turn on deep profiling and (for benchmarking purposes) control its implementation. Add an optiooption disable tailcall optimization in the LLDS backend, to help benchmarking deep profiling. compiler/jumpopt.m: compiler/optimize.m: Obey the option to disable tailcalls. compiler/handle_options.m: Handle the implications of deep profiling. compiler/modules.m: In deep profiling grades, automatically import profiling_builtin.m. compiler/prog_util.m: doc/Makefile: library/library.m: Handle the new builtin module. compiler/export.m: In deep profiling grades, wrap deep profiling code around exported procedures to handle the "unscheduled call" aspects of callbacks to Mercury from the foreign language. compiler/higher_order.m: profiler/demangle.m: util/demangle.c: When creating a name for a higher-order-specialized predicate, include the mode number in the name. compiler/add_trail_ops.m: compiler/type_util.m: Move c_pointer_type from add_trail_ops to type_util, so it can also be used by deep_profiling.m. compiler/hlds_goal.m: Add a new goal feature that marks a tail call, for use by deep_profiling.m. compiler/hlds_pred.m: Add a new field to proc_info structures for use by deep_profiling.m. Add a mechanism for getting proc_ids for procedure clones. Remove next_proc_id, an obsolete and unused predicate. compiler/hlds_data.m: Add a new cons_id to refer to the proc_static structure of a procedure. compiler/bytecode_gen.m: compiler/code_util.m: compiler/dependency_graph.m: compiler/hlds_out.m: compiler/mercury_to_mercury.m: compiler/ml_unify_gen.m: compiler/opt_debug.m: compiler/prog_rep.m: compiler/rl_exprn.m: compiler/switch_util.m: compiler/unify_gen.m: Trivial changes to handle the new cons_id, goal feature and/or proc_info argument. compiler/rtti.m: Add a utility predicate for extracting pred_id and proc_id from an rtti_proc_label, for use by hlds_out.m compiler/layout.m: compiler/layout_out.m: compiler/llds.m: compiler/llds_common.m: Add support for proc_static and call_site_static structures. compiler/layout_out.m: compiler/llds_out.m: Add code for the output of proc_static structures. compiler/code_util.m: Make code_util__make_proc_label_from_rtti a function, and export it. util/mkinit.c: compiler/llds_out.m: compiler/layout.m: compiler/modules.m: Add support for a fourth per-module C function, for writing out proc_static structures (and the call_site_static structures they contains). Since proc_static structures can be referred to from LLDS code (and not just from other static structures and compiler-generated C code), reorganize the declarations of static structures slightly. Change the schema for the name of the first per-module C function slightly, to make it the addition of the fourth function easier. The scheme now is: mercury__<modulename>__init mercury__<modulename>__init_type_tables mercury__<modulename>__init_debugger mercury__<modulename>__write_out_proc_statics Improve formatting of the generated C code. library/*.m: runtime/mercury.c: runtime/mercury_context.c: runtime/mercury_engine.c: runtime/mercury_ho_call.c: runtime/mercury_tabling.c: runtime/mercury_trace_base.c: runtime/mercury_wrapper.c: trace/mercrury_trace.[ch]: trace/mercrury_trace_declarative.c: trace/mercrury_trace_external.c: trace/mercrury_trace_internal.c: Conform to the new scheme for initialization functions for hand-written modules. compiler/mercury_compile.m: library/benchmarking.m: runtime/mercury_conf_param.h: runtime/mercury.h: runtime/mercury_engine.c: runtime/mercury_goto.c: runtime/mercury_grade.h: runtime/mercury_ho_call.c: runtime/mercury_label.[ch]: runtime/mercury_prof.[ch]: Add an MR_MPROF_ prefix in front of the C macros used to control the old profiler. compiler/handle_options.m: runtime/mercury_grade.h: scripts/canonical_grade.sh-subr: scripts/init_grade_options.sh-subr: scripts/parse_grade_options.sh-subr: Make deep profiling completely separate from the old profiling system, by making the deep profiling grade independent of MR_MPROF_PROFILE_TIME and the compiler option --profile-time. library/array.m: library/builtin.m: library/std_util.m: runtime/mercury_hand_unify_body.h: runtime/mercury_hand_compare_body.h: In deep profiling grades, wrap the deep profiling call, exit, fail and redo codes around the bodies of hand-written unification and comparison procedures. Make the reporting of array bounds violations switchable between making them fatal errors, as we currently, and reporting them by throwing an exception. Throwing an exception makes debugging code using arrays easier, but since exceptions aren't (yet) propagated across engine boundaries, we keep the old behaviour as the default; the new behaviour is for implementors. runtime/mercury_deep_profiling_hand.h: New file that defines macros for use in Mercury predicates whose definition is in hand-written C code. library/exception.m: runtime/mercury_exception_catch_body.h: runtime/mercury_stacks.h: In deep profiling grades, wrap the deep profiling call, exit, fail and redo codes around the bodies of the various modes of builtin_catch. Provide a function that C code can use to throw exceptions. library/benchmarking.m: library/exception.m: library/gc.m: library/std_util.m: runtime/mercury_context.[ch]: runtime/mercury_engine.[ch]: runtime/mercury_debug.c: runtime/mercury_deep_copy.c: runtime/mercury_overflow.h: runtime/mercury_regs.h: runtime/mercury_stacks.h: runtime/mercury_thread.c: runtime/mercury_wrapper.c: Add prefixes to the names of the fields in the engine and context structures, to make code using them easier to understand and modify. runtime/mercury_deep_profiling.[ch]: New module containing support functions for deep profiling and functions for writing out a deep profiling data file at the end of execution. runtime/mercury_debug.[ch]: Add support for debugging deep profiling. Add support for watching the value at a given address. Make the buffered/unbuffered nature of debugging output controllable via the -du option. Print register contents only if -dr is specified. runtime/mercury_goto.h: runtime/mercury_std.h: Use the macros in mercury_std.h instead of defining local variants. runtime/mercury_goto.h: runtime/mercury_stack_layout.h: runtime/mercury_stack_trace.c: runtime/mercury_tabling.c: trace/mercury_trace.c: trace/mercury_trace_declarative.c: trace/mercury_trace_external.c: trace/mercury_trace_vars.c: Standardize some of the macro names with those used in the debugger paper. runtime/mercury_heap.h: Add support for memory profiling with the deep profiler. runtime/mercury_prof.[ch]: runtime/mercury_prof_time.[ch]: Move the functionality that both the old profiler and the deep profiler need into the new module mercury_prof_time. Leave mercury_prof containing stuff that is only relevant to the old profiler. runtime/mercury_prof.[ch]: runtime/mercury_strerror.[ch]: Move the definition of strerror from mercury_prof to its own file. runtime/mercury_wrapper.[ch]: Add support for deep profiling. Add suppory for controlling whether debugging output is buffered or not. Add support for watching the value at a given address. runtime/Mmakefile: Mention all the added files. scripts/mgnuc.in: Add an option for turning on deep profiling. Add options for controlling the details of deep profiling. These are not documented because they are intended only for benchmarking the deep profiler itself, for the paper; they are not for general use. tools/bootcheck: Compile the deep_profiler directory as well as the other directories containing Mercury code. Turn off the creation of deep profiling data files during bootcheck, since all but one of these in each directory will be overwritten anyway. Add support for turning on --keep-objs by default in a workspace. tools/speedtest: Preserve any deep profiling data files created by the tests. trace/mercury_trace.c: Trap attempts to perform retries in deep profiling grades, since they would lead to core dumps otherwise. util/Mmakefile: Avoid compile-time warnings when compiling getopt. tests/*/Mmakefile: tests/*/*/Mmakefile: In deep profiling grades, switch off the tests that test features that don't work with deep profiling, either by design or because the combination hasn't been implemented yet. |