mirror of
https://github.com/Mercury-Language/mercury.git
synced 2025-12-17 14:57:03 +00:00
faa18a15bd332b51ec6d6fdae4d999ec2bf3fa82
18 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Julien Fischer
|
0898dab6d9 |
Avoid a C compiler warning on x86_64.
Estimated hours taken: 0.1 Branches: main Avoid a C compiler warning on x86_64. runtime/mercury_deep_profiling.c: In the MR_hash_ptr macro, cast `void * ' to `MR_Unsigned' rather than `unsigned int'. |
||
|
Julien Fischer
|
28125db92a |
Fix a bug that causes general/read_line_as_string to seg fault
Estimated hours taken: 9 Branches: main, release runtime/mercury_deep_profiling.c: Fix a bug that causes general/read_line_as_string to seg fault in the .profdeep grades. The problem was that the slots in the hash tables were not being initialised to NULL. The above test case triggered this problem because the start of the MR_call_site_dynamic_table was reusing memory that io.read_line_as_string_2/6 had previously used as a temporary buffer. Consequently, the initial values of many of the slots in the hash table contained rubbish. The code that writes out the Deep.data treated these junk values as pointers. It seg faulted because these "pointers" were to memory that was inaccessible. library/io.m: library/string.m: A little bit of preventative maintenance: s/memcpy/MR_memcpy/ |
||
|
Julien Fischer
|
b1d348845c |
Fix the typo that has been causing the runtime to abort
Estimated hours taken: 6 Branches: main, release runtime/mercury_deep_profiling.c: Fix the typo that has been causing the runtime to abort when writing out the Deep.data files. |
||
|
Zoltan Somogyi
|
9d909aa8df |
Fix two bugs in the code that writes out Deep.data.
Estimated hours taken: at least 16, over some months Branches: main, release runtime/mercury_deep_profiling.c: Fix two bugs in the code that writes out Deep.data. This required improving the infrastructure for debugging this module. The first bug involved the management of the table that mapped the addresses of proc_layout structures (which point to proc statics) to numerical ids. Of the three places that inserted into this table, one inserted the address of the proc static structure directly, leaving dangling proc static ids (ids that don't refer to anything) in the generated Deep.data file. This caused mdprof to abort when processing the Deep.data file. To make such bugs less likely in the future, provide type-safe interface functions to the main functions for manipulating hash tables. The second bug was effectively invisible: the "written" flag wasn't set on call site dynamic structures as they were written out. Convert the module to four-space indentation to reduce the number of bad line breaks. |
||
|
Zoltan Somogyi
|
642c902c90 |
Reduce the sizes of .c and .o files in debug grades by 8%.
Estimated hours taken: 4
Branches: main
Reduce the sizes of .c and .o files in debug grades by 8%.
Add to mdb the capability of putting breakpoints on unify, compare and index
predicates.
compiler/code_gen.m:
Since compiler-generated unify, index and compare pred can be presumed
to be correct, compile them with shallow tracing instead of deep
tracing even if the user asks for deep tracing of the rest of the
module. This saves disk space, compile time and run time.
Move the code for setting the trace level to generate_proc_code,
where it is executed whether we generate code by predicates or by
phases.
compiler/trace_params.m:
Provide a function for use by code_gen.m.
Consider unify and compare predicates to behave as if their address
were taken, since they may be called from builtin.unify or
builtin.compare.
compiler/jumpopt.m:
Do not replace jumps with the block being jumped to if the block
contains an instruction that represents a call to the debugger.
In such cases, the jump is a minor cost in runtime compared to the
call, and the locality benefit from not increasing the code size is
quite likely to exceed that cost.
compiler/llds.m:
Add field names to pragma_c instructions to make the change to jumpopt
easier.
tests/debugger/uci.{inp,exp}:
Update this test case. An update is required since we now generate
fewer events from unify and compare preds due to their now being
shallow traced.
To make this the update easier, use the new capability of putting
breakpoints on unify and compare predicates.
trace/mercury_trace_tables.[ch]:
Expand procedure specifications to allow the specification of unify,
compare and index predicates, and expand the function that checks for
matches between a procedure layout and a procedure specification
accordingly.
Fix two instances of a bug: if a procedure specification has no name,
represent it as NULL, not as a non-NULL pointer to a null character,
since the match function expects only the former.
doc/user_guide.texi:
Document the new definition of procedure specifications.
NEWS:
Mention the new definition of procedure specifications, and expand
on some previous additions to the debugger.
runtime/mercury_stack_trace.c:
Allow mdb breakpoint specifications on uci predicates to be saved.
Factor out some common data structure accesses.
runtime/mercury_proc_id.h:
Change the name of a macro from MR_PROC_ID_COMPILER_GENERATED to
MR_PROC_ID_IS_UCI, for consistency with other names dealing with
unify, compare and index predicates.
runtime/*.[ch]:
trace/*.[ch]:
Conform to the change in mercury_proc_id.h.
|
||
|
Zoltan Somogyi
|
e0c212359f |
Fix problems in white space in the commit of the fix of the deep profiler.
Estimated hours taken: 0.2 Branches: main Fix problems in white space in the commit of the fix of the deep profiler. These were caused by the fact that the workspace in which the original commit took place was created by applying a patch to a fresh workspace, and the patch was created with diff -b. |
||
|
Zoltan Somogyi
|
c80d143cc8 |
The following change is only 98% complete, not 100%.
Estimated hours taken: 120
Branches: main
The following change is only 98% complete, not 100%. I am committing it in
this state because (1) we pass many more test cases in deep profiling grade
with it than without it, and (2) the double maintanance involved in fixing
CVS conflicts is preventing me from doing the last 2%.
Get the deep profiler to work for code that sets up to catch exceptions,
which for the last year or more has included the compiler, and to get it
almost working for code that actually catching exceptions.
The basic problem is that code that throws exceptions will in general cause
several calls to "return" without executing the exit or fail port codes that
the deep profiling transformation inserted into their bodies, and this leaves
the data structure being built by the deep profiling inconsistent. The solution
uses the same approach as we have adopted for the debugger: have the code that
handles the throwing of exceptions simulate a return from each call between
the throw and the catch, updating the deep profiling data structures as needed.
This requires us to be able to walk the stack at runtime not just in debugging
grades but also in deep profiling grades. Since the debugger already has the
mechanisms required for this, we reuse them. The procedure layouts used by the
debugger were designed to have three segments: the procedure stack walk
information, the procedure id, and the execution tracing information. We now
modify this design to make the third segment contain two pointers: to the
execution tracing information (for use by the debugger), and to the procedure's
proc_static structure (for use by deep profiling). Each pointer will be null
unless the pointed-to structure is required by compile-time options.
This common use by the debugger and by deep profiling of the stack-walk
structure and the procedure id structure (which deep profiling used to
generate independently and possibly redundantly) required some rearrangement
of the compiler's version of these data structures.
To make this rearrangement simpler, this diff removes a capability that
we theoretically supported but never used: turning on stack traces without
turning on execution tracing and vice versa. After this diff, stack tracing
is enabled if and only if either execution tracing or deep profiling is
enabled.
The diff also includes improvements in the debugging infrastructure for
debugging deep profiling, which were necessary for the implementation of the
rest of the changes.
compiler/deep_profiling.m:
The code in exception.m needs to know the locations of the variables
that we would pass to the exit or fail port code, so it can simulate
leaving the procedure invocation through the exception port. Without
this information, throwing an exception leaves the deep profiling
data structures of the procedure invocations between throw and catch
in an inconsistent state.
Deep_profiling.m creates these variables, but it doesn't know where
they will be at runtime, so it records their identities; the code
generator will allocate them stack slots and record the numbers of
these stack slots for placement in the now expanded proc layout
structures. Deep profiling used to generate static data structures
separately from the HLDS, but since the code generator now needs
access to them, we store their information in proc_infos in the HLDS.
Instead of passing the addresses of proc_static structures to the deep
profiling port procedures, pass the address of proc_layout structures,
since the information about the identities of procedures are now stored
not in the proc_static structure, but in the proc_layout structure
that points to the proc_static structure.
compiler/hlds_pred.m:
compiler/layout.m:
Move the definitions of the static data structures generated by deep
profiling from layout.m to hlds_pred.m, to allow deep_profiling.m
to store them in proc_infos.
compiler/hlds_pred.m:
compiler/rtti.m:
Move the definition of rtti_proc_label from rtti.m to hlds_pred.m,
since some of the new data structures in hlds_pred.m need it. Despite
its name, the rtti_proc_label type doesn't contain any info that
doesn't belong in the HLDS.
Add some information to the rtti_proc_label type that is now needed
by deep profiling, e.g. record determinisms instead of just code
models. Record explicitly the outcome of some tests that used to be
duplicated in more than one place in the compiler, e.g. for whether
the procedure (as opposed to the predicate) is imported. Change some
of the field names to be more precise about the field's meaning.
compiler/code_gen.m:
Transmit the contents of the deep profiling data structures stored in
the proc_info by deep_profiling.m to continuation_info.m, together
with the layout structures created for execution tracing and the
identities of the variables needed for handling exceptions,
when code generation for a procedure is complete.
After the goal that generates these variables, save them to stack
for use by the exception handler.
compiler/hlds_goal.m:
Add a feature to mark the goal that generates the deep profiling
variables needed by the exception handler.
compiler/hlds_llds.m:
Add a utility predicate for new code in code_gen.m
compiler/continuation_info.m:
Hold the deep profiling information computed by code_gen.m for use by
stack_layout.m.
compiler/layout.m:
compiler/layout_out.m:
Update the definitions of the data structures describing procedure
layouts, and the code writing them out, to reflect the use of some
parts of procedure layouts by deep profiling as well as debugging.
Change the layout structures generated by deep profiling to use
rtti_proc_labels, which are backend independent, instead of
proc_labels, which are specific to the LLDS backend.
Conform to the changes in runtime/mercury_stack_layout.h.
compiler/stack_layout.m:
Generate the updated version of proc_layout structures.
compiler/mercury_compile.m:
compiler/global_data.m:
Conform to the fact that deep profiling no longer generates layout
structures separate from proc_infos.
compiler/llds_out.m:
Register proc_layout structures instead of proc_static structures
for use by runtime/mercury_deep_profiling.c.
compiler/options.m:
compiler/handle_options.m:
Rename the require_tracing option as exec_trace, since this more
directly reflects its meaning.
Instead of having --debug set both require_tracing and stack_trace,
make it set (be the user-visible name of) just exec_trace;
the value of stack_trace is implied.
Turn off the specialization of deep profiling for self-tail-recursive
procedures for now. Due to the changes made by this diff in the data
structures involved in debugging, it cannot be debugged until this
change has been installed. Handling the full language is more important
than a specialization that reduces only stack space overheads, not
runtime overheads.
compiler/compile_target_code.m:
Conform to the changes in options.m and runtime/mercury_grade.h.
compiler/hlds_data.m:
Replace the deep_profiling_proc_static cons_id, and its associated tag,
to deep_profiling_proc_layout, since we now generate addresses of proc
layout structures, not of proc_static structures.
compiler/code_util.m:
Simplify some code based on the new info in rtti_proc_labels.
compiler/bytecode_gen.m:
compiler/dependency_graph.m:
compiler/higher_order.m:
compiler/hlds_out.m:
compiler/mercury_to_mercury.m:
compiler/ml_code_util.m:
compiler/ml_unify_gen.m:
compiler/opt_debug.m:
compiler/proc_label.m:
compiler/prog_rep.m:
compiler/rl_exprn.m:
compiler/rtti_out.m:
compiler/rtti_to_mlds.m:
compiler/saved_vars.m:
compiler/switch_util.m:
compiler/unify_gen.m:
Minor changes to conform to the change from deep_profiling_proc_static
to deep_profiling_proc_layout, to the change in the structure of
rtti_proc_labels, to the changes in types of layout.m, and/or to the
new goal feature.
deep_profiler/measurements.m:
Reserve space for exception counts.
deep_profiler/html_format.m:
Add a column for exception counts.
deep_profiler/profile.m:
deep_profiler/read_profile.m:
Rename the data structures referring to compiler generated unify,
compare and index predicates to avoid misleading names: they are
not the only compiler generated predicates.
deep_profiler/read_profile.m:
runtime/mercury_deep_profiling.c:
Update the string that identifies deep profiling data files.
This is necessary because the format has changed: it now includes
information about exception port counts.
library/exception.m:
In deep profiling grades, execute the exception port code for every
procedure invocation between a throw and a catch, using the procedure
layout structures now generated by the compiler for every procedure.
Rename the function involved to reflect its new, more general purpose.
Update the definitions of the hand-written proc_static and proc_layout
structures for the procedures implemented via hand-written C code.
Indent C preprocessor directives and foreign_procs according to our
coding standards.
library/profiling_builtin.m:
Change the parameters of the call port code procedures from proc_static
to proc_layout. Reach the proc_static structure from the proc_layout
structure when needed. Include the proc_layout structure in any
messages from assertion failures.
Add some conditionally compiled debugging code.
Give some variables better names.
runtime/mercury_type_info.h:
runtime/mercury_builtin_types.c:
Move the macros required to create the proc_static structures
of unify and compare predicates from mercury_type_info.h
to mercury_builtin_types.c, since the latter is the only file
that needs them.
Use the same macros for creating the proc_static structures
of hand-written unify, compare and compare_reprentation predicates
as for user defined predicates. This required changing their naming
scheme.
runtime/mercury_unify_compare_body.h:
Conform to the new naming scheme.
runtime/mercury_ho_call.c:
Provide the mechanism for mercury_unify_compare_body.h to conform
to the new naming scheme.
Remove the definitions of the proc_static structures for
hand-written unify, compare and compare_reprentation predicates,
since these now have to be defined together with the corresponding
proc_layout structures in mercury_builtin_types.c.
runtime/mercury_builtin_types.[ch]:
Update the definitions of the hand-written proc_static and proc_layout
structures for the procedures implemented via hand-written C code,
and add the required declarations first.
Handle deep profiling of compare_representation as well as unify
and compare predicates on builtin types.
Handle deep profiling of compare_representation on user-defined types,
since this is done entirely in the runtime, not by compiler generated
predicates.
runtime/mercury_builtin_types_proc_layouts.h:
New header file containing the declarations of the proc layout
structures of the unify, compare and index predicates of builtin types.
Logically, these declarations belong in mercury_builtin_types.h,
but putting them there causes problems for the linker; the details
are explained in the file itself.
runtime/Mmakefile:
Add the new header file.
runtime/mercury_minimal_model.[ch]:
Update the definitions of the hand-written proc_static and proc_layout
structures for the procedures implemented via hand-written C code,
and add the required declarations first.
runtime/mercury_grade.h:
Replace the MR_REQUIRE_TRACING grade option with MR_EXEC_TRACING.
Besides being better named, the MR_EXEC_TRACING option implies
MR_STACK_TRACE.
Besides the overall binary compatibility version number, add subsidiary
version numbers for binary compatibility in deep profiling and
debugging grades. These will make it easier to bootstrap changes
(such as this) that affect binary compatibility only in such grades.
runtime/mercury_trace_base.c:
trace/mercury_trace.c:
Conform to the new names of the configuration parameters.
runtime/mercury_hand_compare_body.h:
runtime/mercury_hand_unify_body.h:
runtime/mercury_hand_unify_compare_body.h:
runtime/mercury_ho_call.c:
tools/make_port_code:
Pass proc_layout structures instead of proc_static structures
to deep profiling port routines.
runtime/mercury_conf_param.h:
Make MR_DEEP_PROFILING as well as MR_EXEC_TRACING imply MR_STACK_TRACE,
since deep profiling now needs stack tracing. (MR_STACK_TRACE needs
to be set in this file, because tests in this file depend on knowing
its value, and this file is among the first files included (in this
case indirectly) in mercury_imp.h.)
Document the macros controlling the debugging of deep profiling.
Enable printing of label names when the relevant deep profiling
debugging macro is set.
runtime/mercury_debug.c:
runtime/mercury_deep_rec_depth_actions.h:
runtime/mercury_deep_rec_depth_body.h:
runtime/mercury_exception_catch_body.h:
Get to proc_statics via proc_layouts.
runtime/mercury_deep_call_port_body.c:
runtime/mercury_deep_leave_port_body.c:
Get to proc_statics via proc_layouts.
Allow the debugger to disable deep profiling in Mercury code that is
part of the debugger, not of the user program being executed.
Add some more assertions.
runtime/mercury_engine.[ch]:
Add a new debugging flag that controls at runtime whether we generate
a human readable Deep.debug equivalent to the binary Deep.data files.
(We already had a mechanism for controlling this at compile time,
but this isn't flexible enough.)
runtime/mercury_wrapper.c:
Allow this new debugging flag to be set from MERCURY_OPTIONS.
runtime/mercury_deep_profiling.[ch]:
Respect this new debugging flag.
Update the hand-written proc_static structures representing the runtime
system.
Print out addresses of proc_layout as well as proc_static structures
when assertions fail.
Add a field to the measurement structure for exception port counts,
and write out this field with the other port counts.
Remove procedure id information from proc_static structures,
deep profiling now uses the procedure id in the proc_layout structure.
Add to proc_static structures fields that specify where, if anywhere,
the variables needed by exception.m to executed the exception port code
are in the procedure's stack frame.
Define a global flag that allows the debugger to disable deep
profiling in Mercury code that is part of the debugger, not of the
user program being executed.
Increase type safety by providing two versions of the function
for registering proc_layouts, one for the proc_layout structures
of user-defined predicates and one for unify, compare and index
predicates.
Fix a bug that occurs only if MR_DEEP_PROFILING_EXPLICIT_CALL_COUNTS is
defined (which it usually isn't): the initial call count was wrong.
runtime/mercury_deep_profiling_hand.h:
Fix a bug: the handwritten code saving deep profiling variables was
saving them in slots that didn't belong to the relevant stack frame.
Update to conform to the modified definitions of proc_static structures
and the fact that we now reach them via proc_layout structures.
runtime/mercury_exception_catch_body.h:
runtime/mercury_stacks.h:
Fix the other side of the bug in mercury_deep_profiling_hand.h
by reserving the right number of stack slots in the stack frames
of the various modes of exception__catch. Make it harder to make
the same bug in the future by getting the needed info from the
place in mercury_stacks.h that defines the structure of the relevant
stack frame.
runtime/mercury_proc_id.h:
Rename the procedure id structure fields referring to compiler
generated unify, compare and index predicates: they are not the only
compiler-generated predicates.
runtime/mercury_stack_layout.h:
Change procedure layout structures to allow them to be used for deep
profiling as well as for debugging, as described in the prologue above.
We don't need the capability to support label layout structures with
links to misnamed proc layout structures, and supporting it is
inconvenient, so delete the capability.
runtime/mercury_debug.c:
runtime/mercury_deep_profiling_hand.h:
runtime/mercury_layout_util.c:
runtime/mercury_ml_expand_body.h:
runtime/mercury_stack_trace.c:
runtime/mercury_types.h:
trace/mercury_trace_external.c:
Conform to the new names of the procedure id structure fields.
runtime/mercury_std.h:
Add some more arities for MR_PASTE for use in some of the modified
modules in the runtime.
trace/mercury_trace_internal.c:
Disable deep profiling actions in Mercury code that is part of the
debugger, not of the program being debugged.
scripts/init_grade_options.sh-subr:
scripts/parse_grade_options.sh-subr:
Make changes parallel to the ones in runtime/mercury_grade.h: delete
--stack-trace as an independent option, and make --debug set its
own option, not --require-tracing.
scripts/canonical_grade.sh-subr:
scripts/final_grade_options.sh-subr:
scripts/c2init.in:
scripts/mgnuc.in:
scripts/ml.in:
Conform to the changes in grade options for debugging and for deep
profiling.
tools/bootcheck:
If Mmake.stage.{browser,deep,library,runtime,trace}.params exist,
copy them to become the file Mmake.$dir.params in stage2/$dir
(where dir is derived from the name of the original file in the obvious
way). This allows more flexibility in the creation of the stage2;
for example, it allows some directories (e.g. runtime or library)
to be compiled with more debugging than other directories (e.g.
compiler). This may be required because compiling all directories
with lots of debugging may cause the linker to thrash.
Add an option, --disable-debug-libs, that clobbers the libraries
that should be linked in only in debugging grades.
To conserve disk space, remove Deep.data files created by the bootcheck
by default. Add an option, --keep-deep-data, to preserve these files.
Use a consistent mechanism (test -f) for testing the existence of
all files whose existence is tested.
When recording modification times, record the modification times
of some more files.
tests/hard_coded/Mmakefile:
In deep profiling grades, disable the test cases that we don't now
pass in such grades, and document the reasons for their failure.
Fix the misclassification of the write_binary test case.
|
||
|
Fergus Henderson
|
b4e761a51c |
Improve the documentation of deep profiling.
Estimated hours taken: 1 Branches: main Improve the documentation of deep profiling. deep_profiler/README: New file, containing pointers to the different sources of documentation on deep profiling. runtime/mercury_deep_profiling.h: runtime/mercury_deep_profiling.c: Add a comment pointing to deep_profiler/README. |
||
|
Zoltan Somogyi
|
36ab80e0ca |
Fix some conditionally-compiled references to MR_comp_arity.
Estimated hours taken: 0.1 Branches: main runtime/mercury_debug.c: runtime/mercury_deep_profiling.c: Fix some conditionally-compiled references to MR_comp_arity. |
||
|
Zoltan Somogyi
|
1315102d7c |
Fix several occurrences of a bug that prevented the compiler from passing
Estimated hours taken: 20 Branches: main Fix several occurrences of a bug that prevented the compiler from passing all the tests in deep profiling grades. We now pass all the tests in deep profiling grades. Some additional changes were useful in tracking this bug down. Using C types to represent deep profiling data structures allowed us to delete a bunch of casts and thus eliminate them as potential bug locations. Another was to fix some bugs in low level debugging support. In light of the fragility of the deep profiling invariants, always check them when writing out deep profiling data files. Since mdprof depends on these invariants, they have to be checked somewhere, and we already have code for the checks in the runtime. Reenable tail recursion support in deep profiling grades, since it now works. (The bug this change fixes used to occur most in tail recursive procedures, which is why previously I turned it off.) compiler/options.m: Reenable tail recursion support in deep profiling grades. runtime/mercury_unify_compare_body.h: Fix the bug that prevented us from passing all the tests with invariants checked and tail recursion support enabled. The bug was that in several cases, the code in mercury_unify_compare_body.h did not invoke the appropriate deep profiling routines and thus did not fill in the deep profiling data structure they were supposed to fill in, breaking an invariant. There were several instances of this bug: unify, compare and compare_representation on tuples, and compare_representation on functions, predicates and user-defined types. These oversights were possible because the actions of returning an answer and filling in the deep profiling data structures (in deep profiling grades) were separate. If one omitted the latter, tests could still work in all grades except deep profiling grades. The fix therefore uses one macro (return_compare_answer or return unify_answer, depending on the the operation) to fill in the deep profiling data structure (in deep profiling grades) and return the answer, making it impossible to forget to do the former. The new approach treats compare_representation the same as compare, and gathers the same information for it. runtime/mercury_ho_call.c: Factor out some commonalities in the definitions of the MR_ProcStatic data structures used by the code in mercury_unify_compare_body.h. Change a macro name to support the changes in mercury_unify_compare_body.h. Change the module name of compare_representation/3 from std_util to builtin, to match unify/2 and compare/3. compiler/deep_profiling.m: Treat compare_representation/3 the same way as we treat compare/3. library/builtin.m library/std_util.m Move the declaration of compare_representation/3 from std_util to builtin, to make it easier to handle it the same way as compare/3. Since it is just a variant of compare/3 and is at least as built into the implementation, it belongs there anyway. library/profiling_builtin.m: Use C types to represent deep profiling data structures. Delete the casts that are redundant after this change. Fix formatting of foreign_procs. runtime/mercury_deep_profiling.c: As mentioned above, always check the invariants of the deep profiling data structures when writing them out. runtime/mercury_deep_profiling.h: Avoid a warning about nested declarations of variable "i". runtime/mercury_deep_rec_depth_body.h: Delete a bunch of casts we don't need anymore. runtime/mercury_wrapper.[ch]: Initialize a couple of predicate names in the low level debug support to "" instead of NULL. The code that accesses these variables passes them to strcmp directly, which lead to core dumps. We could change the accessing code to test for NULL, but since that code is executed on every call when low level debugging is enabled, that would lead to even greater slowdowns, which we can do without. To avoid a warning about casting away const, make the variables involved "const char *" instead of plain "char *"; we don't want to update the pointed-to strings anyway. tools/bootcheck: In profiling grades, write out profiling data files by default, and require the use of an option to turn them off. This is needed to make sure that the deep profiling invariants are checked by default. tests/hard_coded/compare_rep_usereq.m: tests/hard_coded/compare_representation.m: Avoid hard-coding the name of the module defining compare_representation/3. |
||
|
Zoltan Somogyi
|
010a9ecf74 |
Fix several bugs in deep profiling. These allow the compiler to bootstrap
Estimated hours taken: 30
Branches: main
Fix several bugs in deep profiling. These allow the compiler to bootstrap
again, *including* writing out the profiling data, with sanity checks enabled.
Some test cases still fail, but significantly fewer than before.
These fixes required several improvements in the infrastructure for low
level debugging in the LLDS grades.
compiler/deep_profiling.m:
Mark calls that have a prepare_for_{normal,ho,...}_call inserted before
them as impure, to prevent simplify from optimizing them away, e.g. as
duplicate calls. This is needed because a prepare_for_{...}_call that
is not followed immediately by the call port code of the callee leaves
the profiling tree in a state that violates its invariants.
Mark the redo port code of model_non predicates as needing to be
preserved, even if determinism analysis would normally cause it to be
cut by marking the disjunction it is part of (whose two disjuncts are
the det exit port code and the failure redo port code) as det.
Fix the generation of goal paths to match what the rest of the compiler
does. Start number conjuncts, disjuncts and switch arms from 1, not 0,
and do not reverse goal paths before attaching them to goals; they will
be reversed when they are converted to strings.
compiler/det_analysis.m:
If a disjunct has determinism failure but is impure, treat it as being
able to succeed when computing the max number of solutions of the
disjunction as a whole, *provided* that some earlier disjuct could
succeed. The idea is that ( impure failure ; det ) should be treated
as det, since all backtracking is local within it, while disjunctions
of the form ( det ; impure failure ) should be treated as multi, since
we want to be able to backtrack to the second disjunct from *outside*
the disjunction.
At the moment, we do this not for all impure goals, but only for the
impure goals that deep_profiling marks with the preserve_backtrack_into
feature.
compiler/hlds_goal.m:
Add the preserve_backtrack_into feature.
Add utility predicates for handling the features of a goal directly,
without explicitly dealing with its goal_info.
runtime/mercury_debug.[ch]:
Add mechanisms for turning the printing of low level debugging messages
on and off. Without this, enabling low level debugging can generate
literally gigabytes of debugging output.
The mechanisms all depend on numbering calls.
One mechanism allows messages to be printed starting from calls in
given ranges, by including e.g. -di100-200,300-400 in MERCURY_OPTIONS.
Another mechanism allows N messages to be printed starting from calls
to a given procedure or from calls at which next_call_site_dynamic
has a given value. The value of N (the size of the block of calls)
can be specified by include -dB<num> in MERCURY_OPTIONS. The name of
the given procedure (actually the name of its entry label) can be
specified by including -dj<entrylabel> in MERCURY_OPTIONS. The address
of the call_site_dynamic structure to watch for is specified with the
-dW<addr> option in MERCURY_OPTIONS, as before.
runtime/mercury_wrapper.[ch]:
Add the global variables required to implement the new low level
debugging functionality, as well as the option processing code required
to set those global variables.
Separate the flags controlling the printing of the values of stack
control registers (sp, curfr etc) and ordinary registers (r1, r2 etc).
Print ordinary registers only if explicitly requested.
runtime/mercury_engine.h:
Add the required global flags.
runtime/mercury_deep_profiling.[ch]:
Add two extra arguments to MR_deep_assert, and print them if an
assertion fails. This makes it easier to decide what parameters to give
to the new low level debugging mechanisms.
runtime/mercury_deep_*_port_body.h:
runtime/mercury_deep_rec_depth_*.h:
library/profiling_builtin.m:
Pass the extra arguments to MR_deep_assert, and print low level
debugging messages only if the relevant flag is set.
tools/lmc:
Add mechanisms to add to the list of C compiler flags the program is
compiled with by lmc.
|
||
|
Simon Taylor
|
b7c4a317e9 |
Add MR_ prefixes to the remaining non-prefixed symbols.
Estimated hours taken: 4 Branches: main Add MR_ prefixes to the remaining non-prefixed symbols. This change will require all workspaces to be updated The compiler will start generating references to MR_TRUE, MR_bool, etc., which are not defined in the old runtime header files. runtime/mercury_std.h: Add MR_ prefixes to bool, TRUE, FALSE, max, min, streq, strdiff, strtest, strntest, strneq, strndiff, strntest, NO_RETURN. Delete a commented out definition of `reg'. runtime/mercury_tags.h: Add an MR_ prefix to TAGBITS. configure.in: runtime/mercury_goto.h: runtime/machdeps/i386_regs.h/mercury_goto.h: Add an MR_ prefix to PIC. runtime/mercury_conf_param.h: Allow non-prefixed PIC and HIGHTAGS to be defined on the command line. runtime/mercury_bootstrap.h: Add backwards compatibility definitions. RESERVED_MACRO_NAMES: Remove the renamed macros. compiler/export.m: compiler/ml_code_gen.m: Use MR_bool rather than MR_Bool (MR_Bool is meant to be for references to the Mercury type bool__bool). runtime/mercury_types.h: Add a comment the MR_Bool is for references to bool__bool. */*.c: */*.h: */*.m: Add MR_ prefixes. |
||
|
Zoltan Somogyi
|
b456621269 |
New module for utility functions. Contains the checked versions of
Estimated hours taken: 1 Branches: main runtime/mercury_runtime_util.[ch]: New module for utility functions. Contains the checked versions of fopen, fclose, atexit which used to be static in mercury_prof.c, and the backup version of strerror which used to be in mercury_strerror.c. runtime/mercury_strerror.[ch]: Deleted this now obsolete module. runtime/mercury_prof.c: runtime/Mmakefile: Mention the new module and delete references to the deleted module. runtime/mercury_deep_profiling.c: runtime/mercury_trace_base.c: Add #includes of the new module. |
||
|
Zoltan Somogyi
|
9c880d612e |
Use more descriptive variable names.
Estimated hours taken: 0.2 Branches: main runtime/mercury_deep_profiling.c: Use more descriptive variable names. |
||
|
Zoltan Somogyi
|
a9fed43d57 |
Major enhancements of the deep profiler. The most important ones are:
Estimated hours taken: 80 Branches: main Major enhancements of the deep profiler. The most important ones are: - The addition of several new classes of preferences, including: the use of colour, boxing, sorting, summarizing of higher order call sites, and time formats. - Making preferences persistent across different queries, so that once a preference is set, it "sticks" until the user changes it. Previously, preferences stuck during query sequences of the same command type. - Several new command types: - listing all the modules of the program - listing all the procedures in a module - listing all the callers of a procedure, grouped by calling call site, procedure, module, or clique, and filtering out certain classes of ancestors - jumping into the call graph close to the action - restarting the server (useful when the data file changes) - New optional columns showing time per call, allocations per call and words allocated per call. - Can now display memory consumption in bytes as well as words. - More robustness in the face of external events, e.g. machine shutdowns. - Fix a bug reported by Tom in the summaries of procedures that make higher order calls. The new functionality required adding some fields to ProcStatic structures; as a result, compilers and runtime systems that have this change are incompatible with compilers and runtime systems before this change in deep profiling grades. (They of course remain compatible in other grades.) compiler/deep_profiling.m: compiler/layout.m: compiler/layout_out.m: Add two new fields to ProcStatic structures, one giving the line number of procedure's context and one stating whether the procedure is exported from its module. compiler/layout.m: Be consistent about filename vs file_name in field names. compiler/*.m: Minor changes to handle the new fields. deep_profiler/interface.m: Define new command types, modify some of the parameters of existing ones, and delete obsolete ones. Define the types and predicates used by the new system of preferences, Update the predicates for recognizing and generating queries accordingly. Make the order of declarations and definitions more consistent. deep_profiler/split.m: Complete rewrite of the only predicate of this module. The old split predicate deleted any empty substrings resulting from the breakup of the original string. The new one preserves them, because they are needed by the new encoding scheme used in interface.m. deep_profiler/query.m: New module, containing code dealing with the computational issues of queries. Some of its code is old (from server.m), much of it is new. deep_profiler/html_format.m: New module, containing code dealing with HTML formatting. Some of its code is old (from server.m), much of it is new. deep_profiler/top_procs.m: New module, containing code dealing with issues involving sorting by various criteria. Some of its code is old (from server.m), much of it is new. deep_profiler/exclude.m: New module to handle contour exclusion. This means that when listing the callers of a procedure, we display the nearest parent that is *not* excluded by virtue of being in a given module. The set of modules so excluded forms a contour drawn through the program. deep_profiler/mdprof_cgi.m: deep_profiler/mdprof_server.m: deep_profiler/server.m: Minor changes to adapt to the new system of preferences. deep_profiler/array_util.m: Add a mode to foldl2. deep_profiler/io_combinator.m: Add predicates for reading in sequences of ten things. deep_profiler/measurements.m: Add a function needed by new code. deep_profiler/timeout.m: Make the profiler robust in the face of signals. deep_profiler/canonical.m: Some more work towards working canonicalization; not there yet. Move some procedures to profile.m, since other modules also need them now. deep_profiler/profile.m: Add the new fields to ProcStatic structures. Record the word size. Record more information about procedures whose activation counters are ever zeroed, in order to allow query.m to avoid giving misleading information in cases where a procedure calls itself through a higher order call site. Record information about the modules of the program. Add a bunch of lookup predicates, some moved from canonical.m. deep_profiler/call_graph.m: Minor changes to conform to changes in profile.m. deep_profiler/startup.m: Fill in the new parts of the profile data structure. deep_profiler/read_profile.m: Read the new fields in ProcStatic structures. Read in the id of the root node as a fixed part of the header, not as a node. Read in the word size. Make it easier to find all the debugging output sites. Record, for each call site which can call more than one procedure, whether this causes the caller's ProcStatic structure's activation count to be zeroed. runtime/mercury_deep_profiling.h: Add the new fields to ProcStatic structures. runtime/mercury_deep_profiling.c: Write out the new fields to ProcStatic structures. Write out the id of the root node as a fixed part of the header, not as a node. Write out the word size. Remove incorrect L suffixes on constants. Record that the artificial procedure "parent of main" is called once, not zero times, to avoid division by zero when computing per-call statistics. runtime/mercury_deep_profiling_hand.h: Add the new fields to the macros for creating ProcStatic structures. runtime/mercury_ho_call.c: library/array.m: library/builtin.m: library/exception.m: library/std_util.m: Add the new fields to the invocations of those macros. |
||
|
Zoltan Somogyi
|
924316cd2a |
Add missing "volatile" qualifier on a definition (the declaration
Estimated hours taken: 0.1 Branches: main runtime/mercury_deep_profiling.c: Add missing "volatile" qualifier on a definition (the declaration already had it). |
||
|
Zoltan Somogyi
|
230cb689f6 |
Enhance deep profiling by
Estimated hours taken: 20 Branches: main Enhance deep profiling by (1) optimizing self-tail-recursive calls by using a single service function, prepare_for_tail_call, instead of the prepare_for_normal_call and call_code pair. Since there is no callcode in the inner procedure, the inner procedure doesn't pick up the value of MiddleCSD, and thus cannot pass it to other service functions. Those functions therefore now pick up MiddleCSD from the global current_call_site_dynamic. We also avoid calling set_current_csd in the exit and failure continuations after tail calls in the outer procedure, since their effects will be overridden by the exitcode or failcode of the outer procedure. (2) using a prepare_for_method_call that is separate from prepare_for_ho_call, since extracting a key from a typeclass_info as if it were a closure is not robust (3) adding callcode/exitcode/failcode to the unify/compare routines for the builtin types. This is necessary to record their identities and to count their exits and fails. The resulting system should conform to the invariant that every prepare_for_special_call is matched either by the callcode of a builtin unify/compare routine or by the callcode of a compiler-generated __Unify__ or __Compare__ procedure. The third change records information that we need. It leads to slowdown, but the speedup from the first change more than compensates for it. (The second change is performance neutral.) compiler/deep_profiling: Modify the source transformation to achieve the effects described above. library/profiling_builtin.m: Add prepare_for_tail_call and prepare_for_method_call. Remove MiddleCSD from the argument list of service functions that do not need it. Delete two unused predicates. runtime/mercury_deep_profiling_hand.h: Do not pass MiddleCSD to the service functions from which that argument was removed. runtime/mercury_deep_rec_depth_actions.h: runtime/mercury_deep_rec_depth_body.h: Fix a situation in which one local variable shadowed another. runtime/mercury_ho_call.c: runtime/mercury_unify_compare_body.h: Add callcode/exitcode/failcode to the unify and compare routines of builtin types. Since these routines never call any other routine, we combine callcode/exitcode and callcode/failcode into one action. Write out the proc_static structures of the unify/compare procedures for the builtin types. runtime/mercury_deep_profiling.[ch]: Add statistics for the new service functions. Add statistics for callbacks, for the sake of completeness. Delete an unused argument from the histogram statistics functions. |
||
|
Zoltan Somogyi
|
04e614485d |
Implement deep profiling; merge the changes on the deep2 branch back
Estimated hours taken: 500 Branches: main Implement deep profiling; merge the changes on the deep2 branch back onto the trunk. The main documentation on the general architecture of the deep profiler is the deep profiling paper. doc/user_guide.texi: Document how to use the deep profiler. deep_profiler: deep_profiler/Mmakefile: A new directory holding the deep profiler and its mmakefile. Mmakefile: Add targets for the new directory. Add support for removing inappropriate files from directories. deep_profiler/interface.m: The deep profiler consists of two programs: mdprof_cgi.m, which acts as a CGI "script", and mdprof_server.m, which implements the server process that the CGI script talks to. Interface.m defines the interface between them. script/mdprof.in: A shell script template. ../configure uses it to generate mdprof, which is a wrapper around mdprof_cgi that tells it how to find mdprof_server. deep_profiler/mdprof_cgi.m: The CGI "script" program. deep_profiler/mdprof_server.m: The top level predicates of the server. deep_profiler/profile.m: The main data structures of the server and their operations. deep_profiler/read_profile.m: Code for reading in profiling data files. deep_profiler/startup.m: Code for post-processing the information in profiling data files, propagating costs from procedures to their ancestors and performing various kinds of summaries. deep_profiler/server.m: Code for responding to requests from the CGI script. deep_profiler/cliques.m: Code to find cliques in graphs. deep_profiler/array_util.m: deep_profiler/util.m: Utility predicates. deep_profiler/dense_bitset.m: An implementation of (part of) the set ADT with dense bit vectors. deep_profiler/measurements.m: Operations on profiling measurements. deep_profiler/timeout.m: An implementation of a timeout facility. deep_profiler/conf.m: Functions that depend on autoconfigured settings. configure.in: Find out what command to use to find the name of the local host. Install deep profiling versions of the standard library along with the other profiling versions. runtime/mercury_conf.h.in: Add some macros for deep_profiler/conf.m to use. library/profiling_builtin.m: runtime/mercury_deep_call_port_body.h: runtime/mercury_deep_leave_port_body.h: runtime/mercury_deep_redo_port_body.h: A new library module that implements deep profiling primitives. Some of these primitives have many versions, whose common code is factor is factored out in three new include files in the runtime. compiler/deep_profiling.m: New module to perform the program transformations described in the paper. compiler/notes/compiler_design.html: Document the new compiler module. compiler/mercury_compiler.m: Invoke the new module in deep profiling grades. Allow global static data to be generated by deep_profiling.m. compiler/options.m: Add options to turn on deep profiling and (for benchmarking purposes) control its implementation. Add an optiooption disable tailcall optimization in the LLDS backend, to help benchmarking deep profiling. compiler/jumpopt.m: compiler/optimize.m: Obey the option to disable tailcalls. compiler/handle_options.m: Handle the implications of deep profiling. compiler/modules.m: In deep profiling grades, automatically import profiling_builtin.m. compiler/prog_util.m: doc/Makefile: library/library.m: Handle the new builtin module. compiler/export.m: In deep profiling grades, wrap deep profiling code around exported procedures to handle the "unscheduled call" aspects of callbacks to Mercury from the foreign language. compiler/higher_order.m: profiler/demangle.m: util/demangle.c: When creating a name for a higher-order-specialized predicate, include the mode number in the name. compiler/add_trail_ops.m: compiler/type_util.m: Move c_pointer_type from add_trail_ops to type_util, so it can also be used by deep_profiling.m. compiler/hlds_goal.m: Add a new goal feature that marks a tail call, for use by deep_profiling.m. compiler/hlds_pred.m: Add a new field to proc_info structures for use by deep_profiling.m. Add a mechanism for getting proc_ids for procedure clones. Remove next_proc_id, an obsolete and unused predicate. compiler/hlds_data.m: Add a new cons_id to refer to the proc_static structure of a procedure. compiler/bytecode_gen.m: compiler/code_util.m: compiler/dependency_graph.m: compiler/hlds_out.m: compiler/mercury_to_mercury.m: compiler/ml_unify_gen.m: compiler/opt_debug.m: compiler/prog_rep.m: compiler/rl_exprn.m: compiler/switch_util.m: compiler/unify_gen.m: Trivial changes to handle the new cons_id, goal feature and/or proc_info argument. compiler/rtti.m: Add a utility predicate for extracting pred_id and proc_id from an rtti_proc_label, for use by hlds_out.m compiler/layout.m: compiler/layout_out.m: compiler/llds.m: compiler/llds_common.m: Add support for proc_static and call_site_static structures. compiler/layout_out.m: compiler/llds_out.m: Add code for the output of proc_static structures. compiler/code_util.m: Make code_util__make_proc_label_from_rtti a function, and export it. util/mkinit.c: compiler/llds_out.m: compiler/layout.m: compiler/modules.m: Add support for a fourth per-module C function, for writing out proc_static structures (and the call_site_static structures they contains). Since proc_static structures can be referred to from LLDS code (and not just from other static structures and compiler-generated C code), reorganize the declarations of static structures slightly. Change the schema for the name of the first per-module C function slightly, to make it the addition of the fourth function easier. The scheme now is: mercury__<modulename>__init mercury__<modulename>__init_type_tables mercury__<modulename>__init_debugger mercury__<modulename>__write_out_proc_statics Improve formatting of the generated C code. library/*.m: runtime/mercury.c: runtime/mercury_context.c: runtime/mercury_engine.c: runtime/mercury_ho_call.c: runtime/mercury_tabling.c: runtime/mercury_trace_base.c: runtime/mercury_wrapper.c: trace/mercrury_trace.[ch]: trace/mercrury_trace_declarative.c: trace/mercrury_trace_external.c: trace/mercrury_trace_internal.c: Conform to the new scheme for initialization functions for hand-written modules. compiler/mercury_compile.m: library/benchmarking.m: runtime/mercury_conf_param.h: runtime/mercury.h: runtime/mercury_engine.c: runtime/mercury_goto.c: runtime/mercury_grade.h: runtime/mercury_ho_call.c: runtime/mercury_label.[ch]: runtime/mercury_prof.[ch]: Add an MR_MPROF_ prefix in front of the C macros used to control the old profiler. compiler/handle_options.m: runtime/mercury_grade.h: scripts/canonical_grade.sh-subr: scripts/init_grade_options.sh-subr: scripts/parse_grade_options.sh-subr: Make deep profiling completely separate from the old profiling system, by making the deep profiling grade independent of MR_MPROF_PROFILE_TIME and the compiler option --profile-time. library/array.m: library/builtin.m: library/std_util.m: runtime/mercury_hand_unify_body.h: runtime/mercury_hand_compare_body.h: In deep profiling grades, wrap the deep profiling call, exit, fail and redo codes around the bodies of hand-written unification and comparison procedures. Make the reporting of array bounds violations switchable between making them fatal errors, as we currently, and reporting them by throwing an exception. Throwing an exception makes debugging code using arrays easier, but since exceptions aren't (yet) propagated across engine boundaries, we keep the old behaviour as the default; the new behaviour is for implementors. runtime/mercury_deep_profiling_hand.h: New file that defines macros for use in Mercury predicates whose definition is in hand-written C code. library/exception.m: runtime/mercury_exception_catch_body.h: runtime/mercury_stacks.h: In deep profiling grades, wrap the deep profiling call, exit, fail and redo codes around the bodies of the various modes of builtin_catch. Provide a function that C code can use to throw exceptions. library/benchmarking.m: library/exception.m: library/gc.m: library/std_util.m: runtime/mercury_context.[ch]: runtime/mercury_engine.[ch]: runtime/mercury_debug.c: runtime/mercury_deep_copy.c: runtime/mercury_overflow.h: runtime/mercury_regs.h: runtime/mercury_stacks.h: runtime/mercury_thread.c: runtime/mercury_wrapper.c: Add prefixes to the names of the fields in the engine and context structures, to make code using them easier to understand and modify. runtime/mercury_deep_profiling.[ch]: New module containing support functions for deep profiling and functions for writing out a deep profiling data file at the end of execution. runtime/mercury_debug.[ch]: Add support for debugging deep profiling. Add support for watching the value at a given address. Make the buffered/unbuffered nature of debugging output controllable via the -du option. Print register contents only if -dr is specified. runtime/mercury_goto.h: runtime/mercury_std.h: Use the macros in mercury_std.h instead of defining local variants. runtime/mercury_goto.h: runtime/mercury_stack_layout.h: runtime/mercury_stack_trace.c: runtime/mercury_tabling.c: trace/mercury_trace.c: trace/mercury_trace_declarative.c: trace/mercury_trace_external.c: trace/mercury_trace_vars.c: Standardize some of the macro names with those used in the debugger paper. runtime/mercury_heap.h: Add support for memory profiling with the deep profiler. runtime/mercury_prof.[ch]: runtime/mercury_prof_time.[ch]: Move the functionality that both the old profiler and the deep profiler need into the new module mercury_prof_time. Leave mercury_prof containing stuff that is only relevant to the old profiler. runtime/mercury_prof.[ch]: runtime/mercury_strerror.[ch]: Move the definition of strerror from mercury_prof to its own file. runtime/mercury_wrapper.[ch]: Add support for deep profiling. Add suppory for controlling whether debugging output is buffered or not. Add support for watching the value at a given address. runtime/Mmakefile: Mention all the added files. scripts/mgnuc.in: Add an option for turning on deep profiling. Add options for controlling the details of deep profiling. These are not documented because they are intended only for benchmarking the deep profiler itself, for the paper; they are not for general use. tools/bootcheck: Compile the deep_profiler directory as well as the other directories containing Mercury code. Turn off the creation of deep profiling data files during bootcheck, since all but one of these in each directory will be overwritten anyway. Add support for turning on --keep-objs by default in a workspace. tools/speedtest: Preserve any deep profiling data files created by the tests. trace/mercury_trace.c: Trap attempts to perform retries in deep profiling grades, since they would lead to core dumps otherwise. util/Mmakefile: Avoid compile-time warnings when compiling getopt. tests/*/Mmakefile: tests/*/*/Mmakefile: In deep profiling grades, switch off the tests that test features that don't work with deep profiling, either by design or because the combination hasn't been implemented yet. |