Commit Graph

109 Commits

Author SHA1 Message Date
Zoltan Somogyi
6f82724091 Pass streams explicitly at the top levels.
compiler/mercury_compile_main.m:
compiler/mercury_compile_front_end.m:
compiler/mercury_compile_llds_back_end.m:
compiler/mercury_compile_make_hlds.m:
compiler/mercury_compile_middle_passes.m:
compiler/mercury_compile_mlds_back_end.m:
    Pass progress and error streams explicitly in these top modules
    of the compiler. Use "XXX STREAM" to mark places where we could switch
    from using stderr for both the progress and error streams to using
    module-specific files as the progress and/or error streams.

compiler/passes_aux.m:
    Add a "maybe_" prefix to the names of the predicates that print progress
    messages at the appropriate verbosity levels, as their printing of those
    messages is conditional.

    Provide versions of those predicates that take explicitly specified
    streams to write to, and mark the versions that write to the current
    output stream as obsolete.

    The predicate that wrote progress messages for procedures
    used to have two versions, one taking a pred_proc_id, and one taking
    a pred_id/proc_id pair. Delete the latter, because the arity difference
    that differentiated the two versions is now needed for the difference
    between supplying and not supplying an explicit stream.

compiler/file_util.m:
compiler/hlds_error_util.m:
compiler/write_error_spec.m:
    Delete several predicates that wrote to the current output stream,
    since all their callers now use the versions that specify an explicit
    output stream.

compiler/check_promise.m:
compiler/check_typeclass.m:
compiler/closure_analysis.m:
compiler/complexity.m:
compiler/cse_detection.m:
compiler/deforest.m:
compiler/delay_construct.m:
compiler/delay_partial_inst.m:
compiler/deps_map.m:
compiler/direct_arg_in_out.m:
compiler/grab_modules.m:
compiler/handle_options.m:
compiler/hhf.m:
compiler/inlining.m:
compiler/make.module_dep_file.m:
compiler/ml_proc_gen.m:
compiler/ml_top_gen.m:
compiler/mode_constraints.m:
compiler/modes.m:
compiler/polymorphism.m:
compiler/purity.m:
compiler/read_modules.m:
compiler/recompilation.check.m:
compiler/saved_vars.m:
compiler/simplify_proc.m:
compiler/size_prof.m:
compiler/stack_opt.m:
compiler/switch_detection.m:
compiler/typecheck.m:
compiler/unique_modes.m:
compiler/unneeded_code.m:
compiler/write_module_interface_files.m:
    Get these modules to take an explicitly specified stream to which
    to write progress messages when they are invoked from mercury_compile_*.m.

    For predicates in these modules that can be invoked both directly
    by mercury_compile_*.m *and* by other modules, the latter effectively
    as a subcontractor, make them take a maybe(stream), with the intention
    being that all the other modules that use the predicate as a subcontractor
    would pass a "no". This avoids the need to pass progress streams
    down to the internals of other passes, and also avoids overwhelming
    the user invoking the compiler with unnecessary details.

    As above, and also delete a progress message that shouldn't be needed
    anymore.

    Move a test of option value compatibility from
    mercury_compile_middle_passes.m to handle_options.m, where it belongs.

compiler/float_regs.m:
    Write a debug message to the debug stream.

compiler/pd_info.m:
    Include the progress stream in the pd_info structure, because this is
    the simplest way to ensure that all parts of the partial deduction pass
    have access to it.

compiler/make.build.m:
compiler/make.program_target.m:
compiler/make.track_flags.m:
    Make the minimal changes needed to conform to the changes above.
    The rest can be done when the make package is converted to consistently
    use explicit streams.

compiler/bytecode_gen.m:
compiler/structure_reuse.direct.m:
compiler/structure_reuse.versions.m:
compiler/structure_sharing.analysis.m:
    Make the minimal changes needed to conform to the changes above.
    The rest can be done when these modules start being maintained again.

compiler/Mercury.options:
    Stop specifying --no-warn-implicit-stream-calls for mercury_compile_*.m,
    since this diff makes that unnecessary.

    Start specifying --no-warn-implicit-stream-calls for some modules that
    are not currently being actively maintained, because the addition of
    progress-reporting predicates that take explicitly specified streams
    would otherwise cause the generation of such warnings for them.
2022-11-01 11:33:41 +11:00
Zoltan Somogyi
307b1dc148 Split up error_util.m into five modules.
compiler/error_spec.m:
    This new module contains the part of the old error_util.m that defines
    the error_spec type, and some functions that can help construct pieces
    of error_specs. Most modules of the compiler that deal with errors
    will need to import only this part of the old error_util.m.

    This change also renames the format_component type to format_piece,
    which matches our long-standing naming convention for variables containing
    (lists of) values of this type.

compiler/write_error_spec.m:
    This new module contains the part of the old error_util.m that
    writes out error specs, and converts them to strings.

    This diff marks as obsolete the versions of predicates that
    write out error specs to the current output stream, without
    *explicitly* specifying the intended stream.

compiler/error_sort.m:
    This new module contains the part of the old error_util.m that
    sorts lists of error specs and error msgs.

compiler/error_type_util.m:
    This new module contains the part of the old error_util.m that
    convert types to format_pieces that generate readable output.

compiler/parse_tree.m:
compiler/notes/compiler_design.html:
    Include and document the new modules.

compiler/error_util.m:
    The code remaining in the original error_util.m consists of
    general utility predicates and functions that don't fit into
    any of the modules above.

    Delete an unneeded pair of I/O states from the argument list
    of a predicate.

compiler/file_util.m:
    Move the unable_to_open_file predicate here from error_util.m,
    since it belongs here. Mark another predicate that writes
    to the current output stream as obsolete.

compiler/hlds_error_util.m:
    Mark two predicates that wrote out error_spec to the current output
    stream as obsolete, and add versions that take an explicit output stream.

compiler/Mercury.options:
    Compile the modules that call the newly obsoleted predicates
    with --no-warn-obsolete, for the time being.

compiler/*.m:
    Conform to the changes above, mostly by updating import_module
    declarations, and renaming format_component to format_piece.
2022-10-12 20:50:16 +11:00
Zoltan Somogyi
07f877bc3f Carve term_context.m out of term.m.
library/term.m:
library/term_context.m:
    As above.

    Rename the term.context type as term_context.term_context, with
    term.context now being defined as an equivalence type.

    Replace the context_init function and predicate and the dummy_context_init
    function with just one function: dummy_context. This name includes
    the important part (the fact that it return a *dummy* context) and deletes
    the nonimportant part (dummy contexts are just about never updated,
    so the function does not really "initialize" them).

    Reduce function/predicate pairs that do the same thing to just a function.

library/MODULES_DOC:
library/library.m:
    Add the new module to the list of standard library modules.

NEWS:
    Mention the new module, and the obsoleting of the moved predicates
    and functions in term.m.

compiler/*.m:
library/*.m:
    Conform to the changes above.
2022-08-23 12:56:37 +10:00
Zoltan Somogyi
7b6bace9ec Don't return dummy parse_trees for missing files.
The predicates that read a Mercury source, interface or optimization file
used to return four things:

- the name of the file (for callers who specified only a module name),
- the timestamp, if requested and available,
- a parse tree, and
- a representation of any errors (sets of error categories, and error_specs).

However, these four things were not independent. Some combinations of their
values were not allowed, but (a) there was no documentation of what these
combinations were, and (b) code that processed these four things had to be
prepared to handle illegal as well as legal combinations.

This diff makes these predicates return only one result, which contains

- all four of the above things, when the file could be opened, but
- only the file name and a representation of the error if the file
  could not be opened,
- only the file name and a representation of *no* errors, if the caller
  asked the predicate to read the file only if its timestamp did not match
  a specified value, and it does match that value.

We use a somewhat modified version of an existing type, have_read_module,
for this. It is modified both by including information that its users
now need that they did not need before, and shortening the names of its
function symbols, which now occur in many more places than before.

compiler/read_modules.m:
    Make the change to the output arguments described above.

    Making this change requires having the affected predicates deal with
    the case where we either cannot find or cannot open the relevant file.
    (The two are effectively the same thing in this module, since we search
    by attempting to open.) Passing that task off to parse_modules.m
    was always inelegant.

    Simplify the have_read_module_map type by making the key always
    be a module_name. We deleted the only use of have_read_module_maps
    with another kind of key a while ago.

    Delete some no-longer-needed predicates.

compiler/parse_module.m:
    Delete the code dealing with the absence of a file stream to parse.

    Replace code that used to construct dummy parse trees with code
    that simply returns *no* parse tree if the file could be opened
    but could not be read, or if its timestamp indicated that reading it
    would have been redundant.

    Delete some utility predicates moved to parse_error.m, so that
    read_modules.m could also use them.

    Fix comment rot.

compiler/parse_error.m:
    Add some utility predicates for use by read_modules.m and parse_module.m.

compiler/deps_map.m:
    Conform to the changes above.

    Document the dubious effects of an old design decision.

    Fix a misleading predicate name.

    Fix comment rot.

compiler/grab_modules.m:
    Conform to the changes above.

    Some predicates used to return parse trees that could be dummies.
    Change them to return just the parts of the parse tree that the
    caller was interested in, which was usually a tiny part, and which
    can be constructed trivially even when we don't have a parse tree.

    Delete an unneeded type.

compiler/recompilation.check.m:
    Conform to the changes above.

    Represent the operations we need to test version numbers in interface files
    as a typeclass, and rewrite the checking operation in terms of that
    typeclass, with one instance for each kind of interface file.

    Move some repeated code into a predicate.

    Shorten some long names.

compiler/mercury_compile_main.m:
    Conform to the changes above.

    Break up a large predicate.

compiler/generate_dep_d_files.m:
compiler/make.module_dep_file.m:
compiler/write_module_interface_files.m:
    Conform to the changes above.

compiler/prog_item.m:
    Delete the parse_tree_some_int type. The change in recompilation.check.m,
    deleted its last few uses there, and allowed the deletion of the last
    uses in read_modules.m.

tests/recompilation/two_module_debug:
    Extend this script to help deal with problems in all stages of the
    execution of a test case, since that was required while debugging
    this diff.

    Document which parts of this script correspond to which parts of
    two_module_test.

tests/recompilation/test_functions:
tests/recompilation/two_module_test:
    Simplify the logic of the main test function, by testing a condition
    once instead of three times.

    Specify whether a recompilation test is expected to succeed or fail
    directly, using alternatives that model a bespoke type, instead
    of alternatives that mimic a boolean answer to a question, which
    does not help readers who don't remember the question.

    Always put shell variable names in braces.

    Note a problem that makes one of the shell functions ineffective.

tests/recompilation/Mmakefile:
    Tell two_module_test what to do using its updated interface.
2022-04-30 14:37:22 +10:00
Zoltan Somogyi
94affaec76 Separate fatal and nonfatal module reading errors, ...
... and bundle sets of error kinds and their associated lists of error_specs
into a single structure.

compiler/parse_error.m:
    Use separate types to represent fatal errors and nonfatal errors,
    because those two kinds of errors are often treated differently.

    Change the read_module_errors type, which used to be just a set of errors,
    to being a set of fatal errors, a set of nonfatal errors, the error
    message for each, and error messages for warnings. This preserves
    distinctions that we should be able to use in the future to improve
    error handling.

    Provide utility routines for operating on this now-structured type.

compiler/parse_types.m:
    Change the definition of the item_or_marker type slighly to preserve
    distinctions that the read_module_errors type now cares about.

compiler/parse_module.m:
    Thread the new read_module_errors structure through the predicates
    that parse source, interface and optimization files.

    Treat the failure of reading the contents of a file as a fatal error.
    Previously, we treated it as a nonfatal error.

    Delete some fields from types when those field were always set to the
    same value, and therefore contained no useful information.

compiler/module_baggage.m:
    Switch to using the new read_module_errors structure in module baggage.

compiler/grab_modules.m:
    Conform to the changes above.

    Use a bespoke type to represent the absence/presence of errors
    when reading in optimization files.

compiler/mercury_compile_make_hlds.m:
    Conform to the changes above.

    Delete a call to pre_hlds_write_out_errors that duplicates
    another call a few lines above.

compiler/deps_map.m:
compiler/generate_dep_d_files.m:
compiler/make.module_dep_file.m:
compiler/mercury_compile_main.m:
compiler/parse_item.m:
compiler/parse_pragma.m:
compiler/parse_type_defn.m:
compiler/read_modules.m:
compiler/recompilation.check.m:
compiler/write_deps_file.m:
compiler/write_module_interface_files.m:
    Conform to the changes above.
2022-04-23 20:56:12 +10:00
Zoltan Somogyi
ea4f95a7ed Use var_tables in lco.m, and when dumping goals.
Since this is the first converted module that dumps out goals when
debugging trace flags are enabled, this required generalizing the code
that does that, to take either varsets or var_tables as a means of
specifying the names of variables. We do this via a new type,
var_name_source, which contains either a varset or a var_table.

Almost all of this diff is there to implement this generalization.
A large part of it affects code in the parse_tree package that we use
to write out the parts of HLDS goals that are defined by types defined
in that package. Since we want to avoid making any part of the parse_tree
package dependent on the hlds package, this required defining the
var_name_source type in the parse_tree package, which in turn requires
var_table.m to be in that same package.

compiler/lco.m:
    Convert this module to use var_tables instead of varsets and vartypes.

compiler/var_table.m:
    Move this module from the hlds package to the parse_tree package.

    To make this, possible, move the parts that required access to the HLDS
    to hlds_pred.m, from where it was usually invoked.

    Export some utility predicates to allow the moved code to work
    in hlds_pred.m without access to the actual definition of the
    var_table type.

    Define the var_name_source type.

    Add some utility functions for use by code writing out variable names.

compiler/hlds_pred.m:
    Add the code moved from var_table.m.

compiler/vartypes.m:
    Move this module from the hlds package to the parse_tree package,
    for symmetry with var_table.m. It did not depend on being in hlds
    in any way.

compiler/hlds.m:
compiler/parse_tree.m:
    Move vartypes.m and var_table.m from the hlds package
    to the parse_tree package.

compiler/hlds_out_goal.m:
    Change all the predicates in this module to take a var_name_source
    instead of a prog_varset.

    Fix some comments.

compiler/hlds_out_util.m:
    Change some of the predicates in this module (those called from
    hlds_out_goal.m) to take a var_name_source instead of a prog_varset.

compiler/parse_tree_out_term.m:
    Provide variants of some existing predicates and functions that take
    var_name_sources instead of varsets. The code of the copies
    duplicates the logic of the originals, though I hope that this
    duplication can be done away with at the end of the transition.
    (The best solution would be to use a typeclass with methods
    that convert vars to their names, but we would want to ensure
    that the compiler can specialize all the affected predicates
    and functions to the two instances of this typeclass, which is
    something that we cannot do yet. In the meantime, the lack of
    any generalization in the old versions preserves their performance.)

tools/sort_imports:
tools/filter_sort_imports:
    A new tool that automatically sorts any occurrences of consecutive
    ":- import_module" declarations in the named files. The sorting is done
    in filter_sort_imports; sort_imports loops over the named files.

    After automatically replacing all occurrences of hlds.{vartypes,var_table}
    in import_module declarations with their parse_tree versions, the updated
    import_module declarations were usually out of order with respect to
    their neighbours. I used this script to fix that, and some earlier
    out-of-order imports.

compiler/accumulator.m:
compiler/add_class.m:
compiler/add_clause.m:
compiler/add_foreign_proc.m:
compiler/add_heap_ops.m:
compiler/add_pragma_type_spec.m:
compiler/add_pred.m:
compiler/add_trail_ops.m:
compiler/analysis.m:
compiler/arg_info.m:
compiler/build_mode_constraints.m:
compiler/bytecode_gen.m:
compiler/call_gen.m:
compiler/check_promise.m:
compiler/closure_analysis.m:
compiler/closure_gen.m:
compiler/code_info.m:
compiler/code_loc_dep.m:
compiler/common.m:
compiler/compile_target_code.m:
compiler/complexity.m:
compiler/const_prop.m:
compiler/constraint.m:
compiler/continuation_info.m:
compiler/convert_parse_tree.m:
compiler/coverage_profiling.m:
compiler/cse_detection.m:
compiler/ctgc.datastruct.m:
compiler/ctgc.util.m:
compiler/dead_proc_elim.m:
compiler/deep_profiling.m:
compiler/deforest.m:
compiler/delay_construct.m:
compiler/delay_partial_inst.m:
compiler/dep_par_conj.m:
compiler/det_analysis.m:
compiler/det_report.m:
compiler/det_util.m:
compiler/direct_arg_in_out.m:
compiler/disj_gen.m:
compiler/distance_granularity.m:
compiler/equiv_type_hlds.m:
compiler/exception_analysis.m:
compiler/file_names.m:
compiler/float_regs.m:
compiler/follow_vars.m:
compiler/format_call.m:
compiler/generate_dep_d_files.m:
compiler/get_dependencies.m:
compiler/goal_expr_to_goal.m:
compiler/goal_mode.m:
compiler/goal_path.m:
compiler/goal_store.m:
compiler/goal_util.m:
compiler/granularity.m:
compiler/hhf.m:
compiler/higher_order.m:
compiler/hlds_clauses.m:
compiler/hlds_code_util.m:
compiler/hlds_error_util.m:
compiler/hlds_goal.m:
compiler/hlds_llds.m:
compiler/hlds_out_pred.m:
compiler/hlds_rtti.m:
compiler/hlds_statistics.m:
compiler/inlining.m:
compiler/inst_check.m:
compiler/inst_test.m:
compiler/inst_user.m:
compiler/instance_method_clauses.m:
compiler/instmap.m:
compiler/intermod.m:
compiler/intermod_analysis.m:
compiler/interval.m:
compiler/introduce_exists_casts.m:
compiler/introduce_parallelism.m:
compiler/item_util.m:
compiler/lambda.m:
compiler/live_vars.m:
compiler/liveness.m:
compiler/llds.m:
compiler/llds_out_data.m:
compiler/llds_out_file.m:
compiler/llds_out_util.m:
compiler/lookup_switch.m:
compiler/loop_inv.m:
compiler/make.module_target.m:
compiler/make.util.m:
compiler/make_goal.m:
compiler/make_hlds_separate_items.m:
compiler/make_hlds_types.m:
compiler/mark_tail_calls.m:
compiler/mercury_compile_mlds_back_end.m:
compiler/middle_rec.m:
compiler/ml_accurate_gc.m:
compiler/ml_args_util.m:
compiler/ml_call_gen.m:
compiler/ml_closure_gen.m:
compiler/ml_code_gen.m:
compiler/ml_code_util.m:
compiler/ml_commit_gen.m:
compiler/ml_disj_gen.m:
compiler/ml_foreign_proc_gen.m:
compiler/ml_gen_info.m:
compiler/ml_lookup_switch.m:
compiler/ml_proc_gen.m:
compiler/ml_simplify_switch.m:
compiler/ml_switch_gen.m:
compiler/ml_tag_switch.m:
compiler/ml_unify_gen.m:
compiler/ml_unify_gen_construct.m:
compiler/ml_unify_gen_deconstruct.m:
compiler/ml_unify_gen_test.m:
compiler/ml_unify_gen_util.m:
compiler/mlds_to_c_data.m:
compiler/mlds_to_c_func.m:
compiler/mlds_to_c_global.m:
compiler/mlds_to_cs_class.m:
compiler/mlds_to_cs_file.m:
compiler/mlds_to_java_data.m:
compiler/mlds_to_java_file.m:
compiler/mlds_to_java_stmt.m:
compiler/mlds_to_java_type.m:
compiler/mmc_analysis.m:
compiler/mode_comparison.m:
compiler/mode_constraints.m:
compiler/mode_debug.m:
compiler/mode_errors.m:
compiler/mode_info.m:
compiler/mode_ordering.m:
compiler/modecheck_call.m:
compiler/modecheck_coerce.m:
compiler/modecheck_goal.m:
compiler/modecheck_unify.m:
compiler/modecheck_util.m:
compiler/modes.m:
compiler/module_cmds.m:
compiler/old_type_constraints.m:
compiler/opt_debug.m:
compiler/optimize.m:
compiler/options_file.m:
compiler/ordering_mode_constraints.m:
compiler/par_loop_control.m:
compiler/parse_item.m:
compiler/parse_string_format.m:
compiler/parse_tree_out_inst.m:
compiler/parse_tree_to_term.m:
compiler/parse_util.m:
compiler/pd_debug.m:
compiler/pd_info.m:
compiler/pd_util.m:
compiler/peephole.m:
compiler/polymorphism.m:
compiler/polymorphism_info.m:
compiler/polymorphism_lambda.m:
compiler/polymorphism_type_class_info.m:
compiler/polymorphism_type_info.m:
compiler/post_typecheck.m:
compiler/pragma_c_gen.m:
compiler/pred_name.m:
compiler/pred_table.m:
compiler/prog_item.m:
compiler/prog_rep.m:
compiler/prop_mode_constraints.m:
compiler/purity.m:
compiler/push_goals_together.m:
compiler/qual_info.m:
compiler/quantification.m:
compiler/rbmm.execution_path.m:
compiler/rbmm.m:
compiler/rbmm.points_to_analysis.m:
compiler/rbmm.points_to_graph.m:
compiler/rbmm.points_to_info.m:
compiler/rbmm.region_resurrection_renaming.m:
compiler/rbmm.region_transformation.m:
compiler/recompilation.used_file.m:
compiler/recompilation.version.m:
compiler/recompute_instmap_deltas.m:
compiler/resolve_unify_functor.m:
compiler/rtti.m:
compiler/rtti_out.m:
compiler/rtti_to_mlds.m:
compiler/saved_vars.m:
compiler/set_of_var.m:
compiler/simplify_goal_call.m:
compiler/simplify_goal_conj.m:
compiler/simplify_goal_disj.m:
compiler/simplify_goal_ite.m:
compiler/simplify_goal_scope.m:
compiler/simplify_goal_switch.m:
compiler/simplify_goal_unify.m:
compiler/simplify_info.m:
compiler/simplify_proc.m:
compiler/size_prof.m:
compiler/smm_common.m:
compiler/ssdebug.m:
compiler/stack_alloc.m:
compiler/stack_layout.m:
compiler/stack_opt.m:
compiler/stm_expand.m:
compiler/store_alloc.m:
compiler/structure_reuse.analysis.m:
compiler/structure_reuse.direct.choose_reuse.m:
compiler/structure_reuse.direct.detect_garbage.m:
compiler/structure_reuse.domain.m:
compiler/structure_reuse.indirect.m:
compiler/structure_reuse.lbu.m:
compiler/structure_reuse.lfu.m:
compiler/structure_sharing.analysis.m:
compiler/structure_sharing.domain.m:
compiler/superhomogeneous.m:
compiler/switch_detection.m:
compiler/switch_gen.m:
compiler/switch_util.m:
compiler/table_gen.m:
compiler/tabling_analysis.m:
compiler/term_constr_build.m:
compiler/term_constr_data.m:
compiler/term_constr_initial.m:
compiler/term_constr_main.m:
compiler/term_constr_main_types.m:
compiler/term_constr_util.m:
compiler/term_pass1.m:
compiler/term_traversal.m:
compiler/term_util.m:
compiler/trace_gen.m:
compiler/trailing_analysis.m:
compiler/transform_llds.m:
compiler/try_expand.m:
compiler/tupling.m:
compiler/type_assign.m:
compiler/type_ctor_info.m:
compiler/type_util.m:
compiler/typecheck.m:
compiler/typecheck_debug.m:
compiler/typecheck_errors.m:
compiler/typecheck_info.m:
compiler/unify_gen_construct.m:
compiler/unify_gen_deconstruct.m:
compiler/unify_proc.m:
compiler/unique_modes.m:
compiler/unneeded_code.m:
compiler/untupling.m:
compiler/unused_args.m:
compiler/unused_imports.m:
compiler/var_locn.m:
compiler/write_deps_file.m:
compiler/write_module_interface_files.m:
    Conform to the changes above.
2022-04-18 02:00:38 +10:00
Zoltan Somogyi
3503128d9b Rename module_imports.m to module_baggage.m.
compiler/module_baggage.m:
    This reflects the changes to this module over the last year.

compiler/parse_tree.m:
    Change the module name in the include_module declaration.

compiler/deps_map.m:
compiler/generate_dep_d_files.m:
compiler/grab_modules.m:
compiler/make.dependencies.m:
compiler/make.module_dep_file.m:
compiler/make.module_target.m:
compiler/mercury_compile_main.m:
compiler/module_dep_info.m:
compiler/recompilation.check.m:
compiler/recompilation.usage.m:
compiler/recompilation.used_file.m:
compiler/write_deps_file.m:
compiler/write_module_interface_files.m:
    Change the module name in import_module declarations.

compiler/notes/compiler_design.html:
    Change the module name in the module description.
2022-04-15 12:41:32 +10:00
Zoltan Somogyi
25b4b67403 Carve io.file.m out of io.m.
library/io.file.m:
library/io.m:
    Move two sections of io.m, the "file handling predicates" section
    and the "handling temporary files" section to the new submodule io.file.m.

    Leave behind in io.m "forwarding predicates", predicates that do nothing
    except call the moved predicates in io.file.m, to provide backward
    compatibility. But do mark the forwarding predicates as obsolete,
    to tell people to update their (at their leisure, since the obsoleteness
    warning can be turned off).

    Also leave behind in io.m the definitions of the two types used
    by some parameters of some of the moved predicates. Document the reason
    why this is done.

library/MODULES_DOC:
    List the new module among the documented modules.

NEWS:
    Announce the changes.

browser/browse.m:
browser/interactive_query.m:
browser/listing.m:
compiler/analysis.file.m:
compiler/compile_target_code.m:
compiler/export.m:
compiler/fact_table.m:
compiler/file_util.m:
compiler/handle_options.m:
compiler/make.build.m:
compiler/make.module_dep_file.m:
compiler/make.module_target.m:
compiler/make.program_target.m:
compiler/make.util.m:
compiler/mercury_compile_main.m:
compiler/module_cmds.m:
compiler/parse_module.m:
compiler/passes_aux.m:
compiler/prog_event.m:
compiler/recompilation.check.m:
compiler/write_deps_file.m:
compiler/write_module_interface_files.m:
deep_profiler/conf.m:
deep_profiler/mdprof_cgi.m:
library/dir.m:
mdbcomp/program_representation.m:
ssdb/ssdb.m:
    Call the file operation predicates directly in io.file.m, not indirectly
    through io.m.

    In two modules, add a #include of fcntl.h in C code. These modules contain
    C code that needs this #include, but until now, they got it via a copy
    in an automatically generated C header file of a foreign_decl pragma
    in io.m that contained that #include. This diff moves that foreign_decl
    to io.file.m, removing that crutch.

tests/debugger/browser_test.m:
tests/hard_coded/bit_buffer_test.m:
tests/hard_coded/bitmap_test.m:
tests/hard_coded/construct_bug.m:
tests/hard_coded/dir_fold.m:
tests/hard_coded/dir_test.m:
tests/hard_coded/read_binary_int16.m:
tests/hard_coded/read_binary_int32.m:
tests/hard_coded/read_binary_int64.m:
tests/hard_coded/read_binary_uint16.m:
tests/hard_coded/read_binary_uint32.m:
tests/hard_coded/read_binary_uint64.m:
tests/hard_coded/read_bitmap_size.m:
tests/hard_coded/remove_file.m:
tests/hard_coded/write_binary.m:
tests/hard_coded/write_binary_int8.m:
tests/hard_coded/write_binary_multibyte_int.m:
tests/hard_coded/write_binary_uint8.m:
    Call the file operation predicates directly in io.file.m, not indirectly
    through io.m.
2022-03-08 06:01:21 +11:00
Zoltan Somogyi
b9908ddaf6 Standardize messages about reading files.
compiler/read_modules.m:
    When reading in Mercury source files, interface files or optimization
    files, we optionally generate progress messages of the form

        % Reading module xyz... done.

    But the form of these messages has not been consistent, so that
    when reading .opt files, we got messages of the form

        % Reading xyz.opt...
        % Done.

    And we sometimes got messages such as

        % Reading .int file for module xyz.int... done.

    This is mostly because responsibility for constructing these messages
    has been dispersed; different parts of it were generated in different
    places, and sometimes when one place switches to a new scheme, not all
    the corresponding places where updated accordingly.

    This diff changes that by constructing these messages in two predicates
    (output_read_msg and output_read_done_msg) in this module, leaving to
    callers only a choice between a set of four message forms.

    To make this possible, move the read_module_{plain,trans}_opt predicates
    here from grab_modules.m. Their equivalents for reading Mercury source
    files and interface files were here already, so here is where they belong.

    Add several "XXX CLEANUP" markers for future changes.

compiler/parse_module.m:
    Do not expect callers to compute the filenames containing .opt
    and .trans_opt files, since the code here already does that.

compiler/grab_modules.m:
    Delete the predicates moved to read_modules.m (under new names, to fit in
    with the predicates already there).

    Delete some obsolete comments.

    Fix some old variable names.

compiler/deps_map.m:
compiler/generate_dep_d_files.m:
compiler/make.module_dep_file.m:
compiler/mercury_compile_main.m:
compiler/recompilation.check.m:
compiler/write_module_interface_files.m:
    Conform to the changes above.
2022-01-21 18:00:22 +11:00
Zoltan Somogyi
243491523d Stop counting errors in module_infos.
Over time, we have almost completely switched over to using error_specs,
using their severity (if not yet printed out) and their effect on the
exit status (if already printed out) to represent the presence of errors,
leaving only a few places that either updated or paid attention to the
num_error field in the module_info. This diff completes that process.

compiler/hlds_module.m:
    Delete the num_errors field in the module_info, and the predicates
    that operate on it.

compiler/hlds_error_util.m:
    Delete module_info arguments whose only purpose was to update the
    now-delete field.

compiler/error_util.m:
    Delete the old write_error_spec predicates that updated a count of warnings
    and a count of errors.

    Rename the write_error_spec_ignore predicates, which ignored those counts,
    by deleting the "_ignore" from their names. This makes them take the
    place of the deleted predicates.

compiler/add_pragma_type_spec.m:
compiler/add_pred.m:
compiler/add_type.m:
compiler/compile_target_code.m:
compiler/cse_detection.m:
compiler/find_module.m:
compiler/generate_dep_d_files.m:
compiler/handle_options.m:
compiler/make.module_dep_file.m:
compiler/make.module_target.m:
compiler/make.program_target.m:
compiler/make.top_level.m:
compiler/make.track_flags.m:
compiler/mercury_compile_front_end.m:
compiler/mercury_compile_llds_back_end.m:
compiler/mercury_compile_main.m:
compiler/mercury_compile_middle_passes.m:
compiler/mercury_compile_mlds_back_end.m:
compiler/mode_info.m:
compiler/modes.m:
compiler/pd_util.m:
compiler/pred_table.m:
compiler/recompilation.check.m:
compiler/typecheck.m:
compiler/write_module_interface_files.m:
    Conform to the changes above.
2022-01-15 16:33:38 +11:00
Zoltan Somogyi
1927c8ee21 Separate always_treat_as_first from treat_as_first.
compiler/error_util.m:
    The internal operation of formatting error messages requires knowing
    whether an error_msg should be treated as the first message, or not.
    (First messages get indented 1 space; other messages get indented by 3.)
    We use the treat_at_first type for this.

    In the past, we also used this type in error_msgs themselves, but there,
    one of the alternatives of the type, do_not_treat_as_first, is misleading.
    Such messages will in fact be treated as first if they are in fact
    the first error_msg in an error_spec, which happens quite frequently,
    since *most* error_specs contain only one error_msg.

    This diff therefore defines a new type, always_treat_as_first,
    whose two values have names that *accurately* reflect their meaning:
    always_treat_as_first, and treat_based_on_posn. Make all code outside
    error_util.m itself use this type; restrict the use of the treat_as_first
    type to just the code that does error message formatting.

compiler/add_class.m:
compiler/add_foreign_proc.m:
compiler/check_typeclass.m:
compiler/common.m:
compiler/compile_target_code.m:
compiler/compiler_util.m:
compiler/compute_grade.m:
compiler/fact_table.m:
compiler/find_module.m:
compiler/grab_modules.m:
compiler/handle_options.m:
compiler/make.top_level.m:
compiler/mercury_compile_main.m:
compiler/mercury_compile_mlds_back_end.m:
compiler/mode_errors.m:
compiler/modes.m:
compiler/options_file.m:
compiler/parse_module.m:
compiler/post_typecheck.m:
compiler/prog_event.m:
compiler/recompilation.check.m:
compiler/term_constr_errors.m:
compiler/term_errors.m:
compiler/typecheck_msgs.m:
compiler/write_module_interface_files.m:
    Conform to the change in error_util.m. In many cases where this is
    appropriate, use the recently introduced simplest_no_context_specs.
2022-01-12 18:03:17 +11:00
Zoltan Somogyi
8f50e16eb2 Fix a possible silent failure when making .int files.
When I split two files in a recent change, I ran into an annoying problem.
The problem was caused by unneeded imports in the interface of the new modules,
whose initial part I originally simply copied from the source module.
The problem was that when I attempted to compile new module A which imported
new module B, the compilation of module A would fail with a message about
not finding module B's .int file. It couldn't find B.int because the
compiler invocation that was supposed to create it failed, but it did not
print any error message about *why* it failed, and as a consequence,
it also did not set the exit status to nonzero to tell mmake that
the file was not actually built, so later build actions that need that file
should not be executed.

The cause of this problem was the following.

- The default value of the --warn-unused-imports is off, but COMP_FLAGS
  turns it on for modules in the compiler directory. This enables warnings
  generated by unused_imports.m.

- There is code that does a similar job in module_qual.qual_errors.m, but
  that one is limited to imports in interface sections. Due to the overlap
  between tasks, when this code finds an unused import_module declaration
  in an interface, it generates an error message that was conditional
  on --warn-unused-imports being off. When it was on, as it is with
  COMP_FLAGS, it generates an error_spec that, when given to write_error_specs,
  generates no output.

- Code in write_module_interface_files.m that decided whether the building
  of the .int file has failed, tested only whether the process of generating
  its contents has returned any error_specs, not whether it returned
  any error_specs that would actually be printed, and, by being printed
  with a sufficiently high severity, would set the exit status to signal
  failure.

compiler/error_util.m:
    The two changes to this file together fix the root cause of this problem.

    First, a new predicate checks whether an error_spec has any part
    whose printing is NOT disabled by being attached to an unmet condition.

    Second, the predicate through which we pass all error_specs created
    during the generation of the contents of the .int file filters out
    any error_specs that yield no output.

The later changes are not strictly part of the bugfix, they are there
simply to make the code simpler to understand, in the hope that this fact
will reduce the probability of similar problems in the future.

compiler/module_qual.qual_errors.m:
    Instead of generating error_specs that are conditional on
    --warn-unused-imports being OFF, generate them conditional on
    the new option --warn-unused-interface-imports being ON.
    Using --warn-unused-imports here was strange because that option controls
    whether the compiler invokes unused_imports.m. It was NOT specific
    to this piece of code, while the new option is.

compiler/options.m:
doc/user_guide.texi:
    Add the new option. Comment out its documentation, since I don't think
    I can describe the reason for its existence simply enough for users
    to understand.

compiler/handle_options.m:
    Turn off the new option --warn-unused-interface-imports if
    --warn-unused-imports is set. This duplicates the old behavior
    of module_qual.qual_errors.m.

    Turn off the new option --warn-unused-interface-imports if
    we are generating interface files. This is because the presence
    of unneeded imports in .m files does not prevent the creation
    of valid .int files, so having to fix such warnings at .int file
    creation time interferes with the programmer's ability to choose
    the order in which he/she works on getting modules to a compilable shape.

compiler/write_module_interface_files.m:
    Generate all output on error as part of an error_spec.

compiler/unused_imports.m:
    Fix indentation.

tests/valid_make_int/extra_interface_import.m:
    A new test case for this bug.

tests/valid_make_int/Mmakefile:
    Enable the new test case.
2021-12-17 12:55:29 +11:00
Zoltan Somogyi
1f9948215f Remove obsoleted interface files.
compiler/write_module_interface_files.m:
    When we get errors that prevent us from constructing a new valid version
    of an interface file, delete both

    - any existing copy of that interface file to prevent its now-outdated info
      from causing misleading error messages when compiling other modules, and

    - any existing copy of its timestamp file, to prevent the build system
      from executing any commands that need the now-deleted interface file.
2021-12-02 00:11:04 +11:00
Zoltan Somogyi
8827b6a466 Make --halt-at-invalid-interface the default.
NEWS:
    Mention this fact.

    Group related changes together.

    Fix some typos.

compiler/error_util.m:
    Let the halt_at_invalid_interface option govern whether we print error
    messages when we generate .int/.int2 files, rather than
    the print_errors_warnings_when_generating_interface option.
    This simplifies the use of that option.

compiler/options.m:
doc/user_guide.texi:
    Document the halt_at_invalid_interface option.

    Delete the print_errors_warnings_when_generating_interface option,
    since its functionality has been subsumed into halt_at_invalid_interface.

compiler/grab_modules.m:
    Check the accessibility of imported modules not just when generating target
    language code, but also when generating .int/.int2 files. Without this,
    the test_nested test case, which this diff moves from invalid to
    invalid_make_int, would miss out on some errors being reported
    in its new home.

    To make the above possible, refactor the code that does the check
    to let it work on aug_make_int_units as well as from aug_compilation_units.
    Make the wording of any error message depend on where the info came from,
    because unlike aug_comp_units, aug_make_int_units work with an
    known-incomplete picture of the relevant info.

    Update some obsolete comments.

compiler/write_module_interface_files.m:
    Add a comment about a design decision that affects this diff.

tests/invalid_make_int/Mercury.options:
tests/invalid_make_int/Mmakefile:
    Add to this directory's list the test cases that this diff moves
    to this directory from other test directories, because the errors
    that they test for are now reported at interface generation time.

    Fix the rule handling multimodule tests, now that we have some :-(

    Specify -j1 for this directory, since the two tests moved here
    from invalid_submodules include nested submodules.

tests/invalid_make_int/bad_type_class_constraint_intermodule.{m,int_err_exp}:
    Move this test case here from invalid, renaming files slightly,
    and update the expected output.

tests/invalid_make_int/bug499.{m,int_err_exp}:
    Move this test case here from invalid, and update the expected output.

tests/invalid_make_int/int_impl_imports.{m,int_err_exp}:
tests/invalid_make_int/int_impl_imports_2.m:
    Move this test case here from invalid, and update the expected output.

tests/invalid_make_int/missing_interface_import2.{m,int_err_exp}:
tests/invalid_make_int/missing_interface_import3.m:
    Move this test case here from invalid, and update the expected output.

tests/invalid_make_int/missing_parent_import.{m,int_err_exp}:
tests/invalid_make_int/children.m:
tests/invalid_make_int/children2.m:
    Move this test case here from invalid_submodules,
    and update the expected output.

tests/invalid_make_int/sub_c.{m,int_err_exp}:
tests/invalid_make_int/sub_a.m:
    Move this test case here from invalid_submodules,
    and update the expected output.

tests/invalid_make_int/test_nested.{m,int_err_exp}:
tests/invalid_make_int/parent.m:
tests/invalid_make_int/parent.private_child.m:
tests/invalid_make_int/parent.public_child.m:
tests/invalid_make_int/parent.undeclared_child.m:
tests/invalid_make_int/parent2.child.m:
tests/invalid_make_int/parent2.m:
    Move this test case here from invalid, and update the expected output.

tests/invalid_make_int/transitive_import.{m,int_err_exp}:
    Move this test case here from invalid, update it,
    and update the expected output.

tests/invalid/Mmakefile:
    Delete from the list of tests in this directory the tests
    moved to invalid_make_int, and test bug521.

tests/invalid/bug521.{m,err_exp}:
tests/invalid/bug521_sub.m:
    Delete this test. invalid_make_int already had a copy of the relevant
    part of this test.

tests/invalid_submodules/Mmakefile:
    Delete from the list of tests in this directory the two tests
    moved to invalid_make_int.

    Delete from the list of tests in this directory two other tests
    for which we now detect some or all of the errors tested for
    at interface generation time, but which are already covered
    by other tests.

tests/valid_make_int/Mmakefile:
    Delete from the list of tests in this directory the test
    moved to invalid_make_int. (We could successfully generate
    a .int file for that test case *only if* we did not check it
    for errors :-)

tests/valid_seq/intermod_nested_module_bug2.m:
tests/valid_seq/intermod_nested_module_bug2.sub.m:
    Move two module imports from the parent to the child module,
    because only the child needs them, and any compiler with this diff
    will now complain about them being unused in the parent
    at interface generation time. Delete a third import, which was
    not used anywhere.
2021-11-20 23:49:19 +11:00
Zoltan Somogyi
6622e5fbb9 Process parse_tree_int[012], not parse_tree_int.
compiler/recompilation.version.m:
    Compute version numbers for parse_tree_int[012], not parse_tree_int.

    Group gathered items and gathered instances into a single type.

    Change argument lists to consistently put old things (gathered items,
    gathered instances, version numbers) before new.

compiler/write_module_interface_files.m:
    Compare new version numbers vs old for parse_tree_int[012],
    not parse_tree_int.

compiler/prog_item.m:
    Move the parse_tree_int type to convert_parse_tree.m, since now
    it is used only by the parser, which them immediately converts
    the generic parse_tree_int to parse_tree_intN for the required N.

compiler/convert_parse_tree.m:
    Move the parse_tree_int type here.

    Delete the predicates that convert parse_tree_int[0123] to parse_tree_int,
    since they are not needed anymore.

compiler/parse_module.m:
compiler/read_modules.m:
    Delete the predicates that read in parse_tree_int,
    since they are not needed anymore.
2021-08-25 15:44:07 +10:00
Zoltan Somogyi
703f52fd7a Stop using exit status to report success/failure.
compiler/make.module_dep_file.m:
    Judge whether calls to write_short_interface_file_int3 succeeded
    by an explicit maybe_succeeded value they now return, instead of
    checking the exit status in the I/O state.

compiler/module_cmds.m:
compiler/parse_tree_out.m:
compiler/write_module_interface_files.m:
    Make operations on files return a maybe_succeeded argument to indicate
    success vs failure.

compiler/make.module_target.m:
compiler/mlds_to_c_file.m:
    Take the new maybe_succeeded arguments in calls into account
    when computing whether an action succeeded.

compiler/analysis.m:
compiler/export.m:
compiler/mercury_compile_front_end.m:
compiler/mercury_compile_main.m:
compiler/mercury_compile_middle_passes.m:
compiler/recompilation.check.m:
    Ignore the maybe_succeeded results returned by calls, for now.

    In export.m, switch from /* */ to // syntax for the C comments we write.
2021-08-23 22:07:14 +10:00
Zoltan Somogyi
567b7c421b Use explicit streams when writing .int* files.
compiler/parse_tree_out.m:
compiler/write_module_interface_files.m:
    Write any progress messages and error messages to explicitly
    provided streams.

compiler/make.module_dep_file.m:
compiler/mercury_compile_main.m:
    Explicitly pass progress and error streams.
2021-08-19 19:10:35 +10:00
Zoltan Somogyi
428aa400eb Introduce aug_make_int_unit.
compiler/prog_item.m:
    Split aug_compilation_unit into two types. One, aug_make_int_unit,
    is a new type, and it is a version of the old aug_compilation_unit
    that contains only what is needed when making .int{,0,2} files.
    The other, the new aug_compilation_unit, contains what is needed
    when generating target language code. Both encode in their argument types
    stronger invariants that the old aug_compilation_unit type did.

compiler/comp_unit_interface.m:
compiler/decide_type_repn.m:
compiler/equiv_type.m:
compiler/get_dependencies.m:
compiler/grab_modules.m:
compiler/make_hlds_separate_items.m:
compiler/module_qual.collect_mq_info.m:
compiler/module_qual.m:
compiler/parse_tree_out.m:
compiler/write_deps_file.m:
compiler/write_module_interface_files.m:
    Conform to the change above.

    In some cases, this means passing around aug_make_int_units instead of
    aug_compilation_units.

    In a few cases, it means creating two copies of a predicate,
    one for aug_compilation_units and one for aug_make_int_units.
    In such cases, factor out any commonalities between the copies.

    In many cases, it means updates to account for the specialized types
    of the fields of aug_compilation_unit or aug_make_int_unit.

    In some cases, it means deleting code that was always unreachable,
    but the not-tight-enough types did not make this fact visible.
    (Due to this effect, this diff removes more lines than it adds.)
2021-08-17 19:23:55 +10:00
Zoltan Somogyi
5c3d4fe4f0 Replace module_and_imports with aug_compilation_unit.
compiler/module_imports.m:
    Delete the module_and_imports type, and replace its uses with
    aug_compilation_units.

    The structure of module_and_imports used to be private, but the
    structure of aug_compilation_units is public. Delete the getter/setter
    predicates on module_and_imports structures that can now be done
    using field names on aug_compilation_units. Delete, along with this,
    the infrastructure for keeping track of which module_and_imports fields
    are ever accessed. It has served its purpose, but it won't be needed
    anymore.

    Change types that used to include module_and_imports to include
    aug_compilation_units instead.

    Rename any types, predicates and functions that used to contain
    the "module_and_imports" string in their name.

compiler/prog_item.m:
    Move the version number map to be the last field of aug_compilation_units,
    as it used to be in module_and_imports structures, since this fits better.

compiler/get_dependencies.m:
    Make the name of a predicate more reflective of the type it returns.

compiler/comp_unit_interface.m:
compiler/deps_map.m:
compiler/equiv_type.m:
compiler/generate_dep_d_files.m:
compiler/grab_modules.m:
compiler/make.m:
compiler/make.module_dep_file.m:
compiler/make_hlds_separate_items.m:
compiler/mercury_compile_main.m:
compiler/module_qual.m:
compiler/write_deps_file.m:
compiler/write_module_interface_files.m:
    Conform to the changes above.
2021-08-17 09:11:53 +10:00
Zoltan Somogyi
b7fb46c073 Introduce and use module_baggage.
compiler/module_imports.m:
    Keep in the module_and_imports structure only the fields that used to
    construct aug_compilation_units, to prepare for the replacement
    of module_and_imports by aug_compilation_unit. Move the remaining fields
    into a new structure, the module_baggage, and introduce a new type,
    module_imports_and_baggage, that contains module_and_imports (for now)
    and module_baggage.

    The point of this is to allow the splitting of aug_compilation_unit
    into types:

    - one (aug_compilation_unit itself) for use when generating
      target language code, and
    - one (maybe named aug_make_int_unit) for use when generating
      .int/.int2 files.

    These two use cases fill some of the fields, such as direct and indirect
    imports, with different kinds of parse trees. Until now, we handled this
    using types such as direct_int_spec, which could contain either kind
    of parse tree, but having separate aug_compilation_unit/aug_make_int_unit
    types would allow the compiler to enforce the different invariants
    of the two use cases.

    This split requires factoring out the common parts of module_and_imports
    structure. The module_baggage type contains these common parts.

    Simplify some operations as permitted by the above change.

    Rename others to reflect the new names of the types they operate on.

    Don't return an output argument that is contained in another output
    argument.

    Delete operations on deleted fields.

    Delete a predicate that has become identical to another.

compiler/grab_modules.m:
    Add operations on module_baggage (that previously were operations on
    module_and_imports) that are needed only in this module.

    Conform to the changes above.

compiler/deps_map.m:
compiler/generate_dep_d_files.m:
compiler/make.m:
compiler/make.module_dep_file.m:
compiler/mercury_compile_main.m:
compiler/write_deps_file.m:
compiler/write_module_interface_files.m:
    Conform to the changes above.
2021-08-14 08:15:09 +10:00
Zoltan Somogyi
f1927afe0b Break a source file info parse_tree_module_srcs ...
... instead of into raw_compilation_units. Besides being more convenient to
work on, a parse_tree_module_src encodes in its type a significant number
of invariants that a raw_compilation unit does not.

compiler/split_parse_tree_src.m:
    Immediately after creating a raw_compilation_unit for one of the modules
    in a potentially multi-module source file, convert it to a
    parse_tree_module_src.

    Fix an old problem that can come up in contrived erroneous code.
    Specifically, when a submodule is both nested inside its parent module,
    *and* it also has an explicit include_module declaration, we used to
    record the contexts of both "inclusions", after generating an error
    message. The more rigorous checking that later code now does on the
    resulting inclusion map would look at this "duplicate inclusion"
    and generate *another* error message. To avoid this redundant message,
    do not record the contexts of erroneous inclusions for which a message
    has already been generated.

compiler/grab_modules.m:
    Operate on parse_tree_module_srcs instead of raw_compilation_units.
    This allows us to avoid having code for doing what converting the
    raw_compilation_unit to a parse_tree_module_src has already done.
    In fact, that conversion code does a better job. The old code assumed
    that all implicitly available modules are used in the interface,
    whereas in fact only some should be used in the interface, with
    the rest being used in the implementation section.

compiler/module_imports.m:
    Make the predicates that create module_and_imports structures
    take a parse_tree_module_src instead of a raw_compilation_unit
    as input. For now, we convert the parse_tree_module_src back to
    a raw_compilation_unit for further processing, but I intend
    a later diff to change this. Nevertheless, one immediate change
    is that init_module_and_imports now stores the *actual*
    parse_tree_module_src in the module_and_imports structure,
    not a dummy.

compiler/prog_item.m:
    Do not include a list of foreign_enum items in the interface section
    of a parse_tree_module_src, since such items are not allowed to occur
    in interface sections.

    For the same reason, delete the field for foreign_enums in the interface
    sections of .int0 and .int files.

compiler/check_raw_comp_unit.m:
    Operate on parse_tree_module_srcs instead of raw_compilation_units.

compiler/comp_unit_interface.m:
    Operate on parse_tree_module_srcs instead of raw_compilation_units.

    Do not expect any foreign_enum items in interface sections, since
    they are not allowed there.

compiler/read_modules.m:
    Provide a mechanism to remember having read a parse_tree_module_src.

compiler/write_module_interface_files.m:
    Operate on parse_tree_module_srcs instead of raw_compilation_units.

compiler/get_dependencies.m:
    Add a new version of an existing utility predicate. The old one operated
    on raw item lists, the new one operates on parse_tree_module_srcs.
    To make this possible, factor out the code pieces that operate on
    each non-ignored kind of item.

compiler/item_util.m:
    Add some new utility predicates/functions.

compiler/convert_parse_tree.m:
    Conform to the change in prog_item.m. (We already generated an error
    message for a foreign_enum item in the interface, but still passed around
    a list of foreign_enum items that was guaranteed to be stay empty.)

compiler/make.module_dep_file.m:
    Conform to the changes above.

    Use explicit streams in one place.

    Do not pass an unneeded argument.

compiler/check_parse_tree_type_defns.m:
compiler/equiv_type.m:
compiler/make_hlds_separate_items.m:
    Conform to the change in prog_item.m.

compiler/mercury_compile_main.m:
compiler/module_qual.collect_mq_info.m:
compiler/module_qual.qualify_items.m:
compiler/parse_tree_out.m:
    Conform to the changes above.

library/map.m:
library/tree234.m:
    Add foldl6, foldl6_values and foldr6 predicates. An earlier version
    of this diff needed foldl6, and I added the others for symmetry.

NEWS:
    Announce the new library predicates.

tests/invalid/bad_mutable.m:
    Export something, to avoid a warning about not exporting anything.

tests/invalid_submodules/duplicate_module.m:
tests/invalid_submodules/duplicate_module_test.m:
    Update programming style.
2021-07-19 13:23:28 +10:00
Zoltan Somogyi
93c306916a Specify streams explicitly in more modules.
compiler/module_cmds.m:
    Require all callers to specify output streams explicitly.

compiler/compile_target_code.m:
    Require callers to specify output streams explicitly in most cases.

    In the remaining cases, add "XXX STREAM" comments to request that
    the explicit streams be passed later.

    Add XXXs where preserving old behavior results in wrong-looking code.

    Modify the signature of compile_java_files to encode the invariant that
    the list of Java files given to it may not be empty.

compiler/error_util.m:
    Provide a version of an error predicate that takes explicit streams.

compiler/make.build.m:
    Mark predicates that help redirect implicit streams as predicates
    that should not be used.

compiler/make.module_target.m:
compiler/make.program_target.m:
    Add "XXX STREAM" comments to request that the explicit streams
    be passed later.

compiler/module_qual.m:
    Provide a means to construct the debug output stream.

compiler/module_qual.collect_mq_info.m:
    Use the debug output stream where relevant.

compiler/analysis.m:
compiler/export.m:
compiler/recompilation.check.m:
compiler/source_file_map.m:
compiler/write_module_interface_files.m:
compiler/xml_documentation.m:
    Conform to the changes above, mostly by passing explicit streams.

compiler/Mercury.options:
    Stop specifying --no-warn-implicit-stream-calls for the above modules
    (those that had it specified in the first place).

    Delete a long-unneeded workaround.

compiler/mercury_compile_front_end.m:
compiler/mercury_compile_llds_back_end.m:
compiler/mercury_compile_main.m:
compiler/mercury_compile_middle_passes.m:
    Specify progress and output streams explicitly when calling the modules
    above (not in other places, just yet).
2021-05-17 22:55:06 +10:00
Zoltan Somogyi
4455f0450e Specify output streams in some places.
Besides this main purpose, this diff also replaces code that calls
io.write_string several times in a row with code that prints the
thing to be printed in one go with io.format. In a couple of places,
this has caught (and fixed) bugs where we wanted to put `' quotes
around a filename, but printed only one of the two quotes.

compiler/file_util.m:
    Provide alternatives to the existing maybe_report_stats,
    maybe_write_string and maybe_flush_output predicates that explicitly
    specify the output stream.

    Rename report_error_to_stream as report_error, to allow
    --warn-implicit-stream-calls to report calls to the existing report_error
    predicate, which does not take an explicit output stream.

    Add a module_name argument to the output_to_file_stream predicate,
    to allow its code to figure out where to print both progress and
    error messages.

compiler/module_cmds.m:
    Add a module_name argument to the predicates that update interface,
    to allow their code to figure out where to print both progress and
    error messages.

    For now, leave the predicates that issue commands that are not
    clearly linked to a single module using implicit streams.

compiler/pd_debug.m:
compiler/analysis.file.m:
    Specify output streams in some places.

    In other places, doing so would require redoing the whole debug
    infrastructure, since the current one is based on higher order predicates
    that always write to the non-explicitly-specified *current* output stream.

compiler/passes_aux.m:
    Provide predicates that get progress, debug and error streams
    given a module_info, by extracting the globals and the module name
    from the module_info, and then calling the predicates in globals.m
    to get those streams. Doing this sequence of actions here factors out
    what would otherwise be repeated code in many other parts of the compiler.

    Delete two predicates that were not used anywhere in the compiler.

compiler/deforest.m:
compiler/export.m:
compiler/intermod.m:
compiler/llds_out_file.m:
compiler/mlds_to_c_file.m:
compiler/mlds_to_cs_file.m:
compiler/mlds_to_java_file.m:
compiler/recompilation.usage.m:
compiler/simplify_goal_conj.m:
compiler/type_assign.m:
compiler/typecheck.m:
compiler/write_deps_file.m:
compiler/write_module_interface_files.m:
    Use explicit streams everywhere where --warn-implicit-stream-calls
    says this is possible.

compiler/Mercury.options:
    Specify --warn-implicit-stream-calls for the modules above
    with the listed exceptions, and with the exception of the modules
    for which it was already specified.

compiler/compile_target_code.m:
compiler/mercury_compile_front_end.m:
compiler/mercury_compile_middle_passes.m:
    Conform to the changes above.
2021-03-21 23:07:59 +11:00
Zoltan Somogyi
409cbcb6a3 Unify getopt.m and getopt_io.m ...
... using an approach proposed by Peter, with an extra twist from Julien.

Instead of having two modules, getopt.m and getopt_io.m, with the former
defining predicates that do not take an I/O state pair, and the latter
defining predicates that do take an I/O state pair, put both kinds of
predicates into a single module. The versions with an I/O state pair
have an "_io" suffix added to their names for disambiguation.
Both versions are a veneer on top of a common infrastructure,
which relies on a simple type class to implement the operation
"give the contents of the file with this name". The predicate versions
with I/O state pairs have a normal implementation of this typeclass,
while the predicate versions that do not have I/O state pairs
have an implementation that always returns an error indication.

The above change just about doubles the number of exported predicates.
We already had two versions of most exported predicates that differed
in whether we returned errors in the form of a string, or in the form
of a structured representation, with names of the latter having
an "_se" suffix. Since we agreed that the structured representation
is the form we want to encourage, this diff deletes the string versions,
and deletes the "_se" suffix from the predicate names that used to have them.
(It still remains at the end of the name of a type.) This "undoubling"
should offset the effect of the doubling in the previous paragraph.

Eventually, we want to have just one module, getopt.m, containing
the updated code described above, but for now, we put the same code
into both getopt_io.m and getopt.m to prevent too big a shock to
people with existing code that uses getopt_io.m.

library/getopt.m:
library/getopt_io.m:
    Make the changes described above.

library/Mmakefile:
    Instead of building both getopt_io.m and getopt.m from getopt_template,
    build getopt.m from getopt_io.m.

tools/bootcheck:
    Delete references to getopt_template.

compiler/typecheck_errors.m:
    When a type error involves one of the getopt/getopt_io predicates
    whose interfaces are changed by this diff, tell the user about
    how these changes could have caused the error, and thus what the
    probable fix is.

compiler/handle_options.m:
browser/parse.m:
deep_profiler/mdprof_cgi.m:
deep_profiler/mdprof_create_feedback.m:
deep_profiler/mdprof_dump.m:
deep_profiler/mdprof_procrep.m:
deep_profiler/mdprof_report_feedback.m:
deep_profiler/mdprof_test.m:
profiler/mercury_profile.m:
slice/mcov.m:
slice/mdice.m:
slice/mslice.m:
slice/mtc_diff.m:
slice/mtc_union.m:
tests/hard_coded/space.m:
    Use the updated getopt interface.

compiler/compile_target_code.m:
compiler/compute_grade.m:
compiler/deforest.m:
compiler/det_report.m:
compiler/format_call.m:
compiler/globals.m:
compiler/goal_expr_to_goal.m:
compiler/make.build.m:
compiler/make.m:
compiler/make.module_dep_file.m:
compiler/make.program_target.m:
compiler/make.util.m:
compiler/mercury_compile_main.m:
compiler/ml_top_gen.m:
compiler/module_cmds.m:
compiler/op_mode.m:
compiler/optimization_options.m:
compiler/options.m:
compiler/write_module_interface_files.m:
tools/make_optimization_options_middle:
tools/make_optimization_options_start:
    Replace references to getopt_io.m with references to getopt.m.

tests/invalid/getopt_io_old.{m,err_exp}:
tests/invalid/getopt_old.{m,err_exp}:
tests/invalid/getopt_old_se.{m, err_exp}:
    New test cases for the extra help

tests/invalid/Mmakefile:
    Enable the new test cases.
2020-10-09 19:30:46 +11:00
Zoltan Somogyi
2e951421dd Replace some bools with bespoke types.
compiler/globals.m:
compiler/write_module_interface_files.m:
    As above.

compiler/error_util.m:
compiler/handle_options.m:
compiler/mercury_compile_main.m:
compiler/read_modules.m:
    Conform to the changes in globals.m.

    In mercury_compile_main.m, add an XXX.
2020-08-19 11:07:25 +10:00
Zoltan Somogyi
c2f92d5454 Partition extensions into ".m" and "all others".
This is a first step towards a much finer grained partition.

compiler/file_names.m:
    Split the ext type into ext_src and ext_other, as mentioned above.

    Add the first predicate for checking whether a string falls into
    a given category of extensions.

    Add an XXX proposing a better solution for an old problem that does not
    actually arise in practice.

compiler/compile_target_code.m:
    Split the two-moded predicate maybe_pic_object_file_extension into
    two separate one-mode predicates, one for each old mode. The
    implementations of the two modes were already separate, because
    the two modes already did different jobs: while one went from PIC
    to an "extension", the other went from an "extension string" to PIC.
    Until now, "extension" and "extension string" were equivalent;
    after this diff, they aren't anymore.

    Delete an unused argument.

compiler/make.util.m:
    Split the two-moded predicate target_extension into
    two separate one-mode predicates, one for each old mode,
    for the same reason as maybe_pic_object_file_extension above:
    the fact that "extension" and "extension string" are now distinct.

compiler/options_file.m:
    Move debug infrastructure here from mercury_compile_main.m, to help
    debug possible problems with options files. (I had such a problem
    while writing this diff.)

    Improve how progress messages are printed.

compiler/options.m:
    Make an error message more useful.

compiler/mercury_compile_main.m:
    Add infrastructure for debugging possible problems with command lines.
    (I had such a problem while writing this diff.)

compiler/analysis.m:
    Conform to the changes above. Put the arguments of some methods
    into the same order as similar predicates in file_names.m.

compiler/find_module.m:
    Conform to the changes above. Delete an unused argument,

compiler/analysis.file.m:
compiler/du_type_layout.m:
compiler/elds_to_erlang.m:
compiler/export.m:
compiler/fact_table.m:
compiler/file_kind.m:
compiler/generate_dep_d_files.m:
compiler/grab_modules.m:
compiler/llds_out_file.m:
compiler/make.build.m:
compiler/make.deps_set.m:
compiler/make.m:
compiler/make.module_dep_file.m:
compiler/make.module_target.m:
compiler/make.program_target.m:
compiler/mercury_compile_front_end.m:
compiler/mercury_compile_llds_back_end.m:
compiler/mercury_compile_middle_passes.m:
compiler/mercury_compile_mlds_back_end.m:
compiler/mlds_to_c_file.m:
compiler/mlds_to_cs_file.m:
compiler/mlds_to_java_file.m:
compiler/mmc_analysis.m:
compiler/mode_constraints.m:
compiler/module_cmds.m:
compiler/prog_foreign.m:
compiler/read_modules.m:
compiler/recompilation.check.m:
compiler/recompilation.usage.m:
compiler/source_file_map.m:
compiler/write_deps_file.m:
compiler/write_module_interface_files.m:
compiler/xml_documentation.m:
2020-08-17 23:43:15 +10:00
Zoltan Somogyi
52c0919975 Make filename extensions a separate type, ...
... to allow later changes to its definition.

compiler/file_names.m:
    We used to represent filename extensions simply as strings. This meant
    all calls to the predicates in file_names.m that convert module names
    to file names with various suffixes had to go through a complicated
    sequence of tests that effectively partition the extensions into
    several classes, with all extensions in a class being treated the same
    but different classes being treated differently. And since this general
    translation process is quite convoluted (which is not helped by it
    being spread across several predicates), it is very hard to construct
    a correctness argument for it.

    It would be better to represent the different classes of extensions
    explicitly, in a du type, with each function symbol of that type
    representing all the extensions in the corresponding class (in the sense
    of the paragraph above). However, getting there in one diff would make
    that diff far too hard to test and to review. So this first diff
    starts by simply making extension a notag type.

    The above is the first step in implementing one old XXX. This diff
    fully implements another old XXX, which is to make the argument order
    of several predicates friendly to higher order code.

    Add infrastructure for profiling how often this code makes directories.

    Delete an unused type.

    Add comments outlining proposed future improvements.

compiler/analysis.file.m:
compiler/analysis.m:
compiler/compile_target_code.m:
compiler/du_type_layout.m:
compiler/elds_to_erlang.m:
compiler/export.m:
compiler/fact_table.m:
compiler/file_kind.m:
compiler/find_module.m:
compiler/generate_dep_d_files.m:
compiler/grab_modules.m:
compiler/llds_out_file.m:
compiler/make.build.m:
compiler/make.m:
compiler/make.module_dep_file.m:
compiler/make.module_target.m:
compiler/make.program_target.m:
compiler/make.util.m:
compiler/mercury_compile_front_end.m:
compiler/mercury_compile_llds_back_end.m:
compiler/mercury_compile_main.m:
compiler/mercury_compile_middle_passes.m:
compiler/mercury_compile_mlds_back_end.m:
compiler/mlds_to_c_file.m:
compiler/mlds_to_cs_file.m:
compiler/mlds_to_java_file.m:
compiler/mmc_analysis.m:
compiler/mode_constraints.m:
compiler/module_cmds.m:
compiler/module_imports.m:
compiler/parse_tree_out.m:
compiler/prog_foreign.m:
compiler/read_modules.m:
compiler/recompilation.check.m:
compiler/recompilation.usage.m:
compiler/write_deps_file.m:
compiler/write_module_interface_files.m:
compiler/xml_documentation.m:
    Conform to the change to file_names.m.

    Consistently use "Ext" for the abstract representation of extensions
    and "ExtStr" for their string representation.

    In a few places, add "XXX EXT" where the code manipulates extensions
    as strings in a way that potentially inferferes with the partition
    of extensions into classes.

    In a few places, rename predicates to avoid ambiguities. factor out
    common code, delete unneeded arguments, replace bools with bespoke types,
    and make similar minor improvements.

    In a few places, remove rafe-isms, such as the use ^elem.
2020-08-14 20:30:36 +10:00
Zoltan Somogyi
86f8696f32 Put globals before error_specs. 2020-06-04 04:47:00 +10:00
Zoltan Somogyi
9cbe5d2caf Put type_repn items for complex types into .int files.
compiler/decide_type_repn.m:
    Previously, this module computed type_repn items to put into .int3 files
    for a subset of the type constructors defined in the current module:
    the direct_dummy, enum and notag types (the *simple* types),
    and the du types whose representation is guaranteed to be
    a word-aligned pointer when targeting C. (We care about pointers
    being word-aligned only when applying the direct arg optimization.
    This optimization is applicable only with the low level data
    representation, which we use only when targeting C.)

    This diff adds code to decide the representations of *all* the
    type constructors defined in the current module.

    This code is based on the existing code in du_type_layout.m,
    which it is intended to eventually replace, but its job is more general,
    because it decides the representation of each type not just for
    one platform (the one we want to generate code), but for all possible
    platforms. This is because we want to put the descriptions of type
    representations into the module's .int file to serve as a single source
    of truth for all modules that use the types defined in this module,
    and the contents of .int files should be platform-independent.
    For our purposes, there are six kinds of platforms, which are
    distinguished along three axes: 64 vs 32 bit machines, spf vs non-spf
    grades, and direct arg optimization enabled vs disabled. That is eight
    combinations, but on 64 bit machines, a float takes up one word whether
    that float is single or double precision, so two combinations aren't valid.

    Some of the change to this module consists of generalizing the existing
    code so that it can decide simple types not just when targeting .int3 files
    but .int files as well. However, the bulk of it is code for deciding
    the representations of non-simple types. The code is not lifted straight
    from du_type_layout.m. There are two main kinds of changes.

    First, I took the opportunity to simplify the algorithms used.
    For example, while du_type_layout.m passes over each function symbol
    in the most general kind of type twice: once to assign it a cons_tag,
    and once to decide how to pack its arguments, the code here does both jobs
    in one pass. Another example is that for historical reasons,
    du_type_layout.m computed the amount of space needed for an argument
    in one place for sub-word-sized arguments, and in another place
    for more-than-word-sized arguments; decide_type_repn.m does it all
    in one place.

    Second, since we compute a representation for each type six times,
    I tried to avoid obvious inefficiencies, but only if the code
    remained simple. In the future, we may want to use an approach
    based on the idea that in the process of computing the first
    representation, we look out for any indication that the representation
    may be different on any of the other five platforms, and if not,
    we just reuse the first representation on the other five platforms as well.
    However, that would be appropriate only *after* we have a simpler
    system that has proven to work in practice.

    There is a third, smaller change: when deciding whether an argument
    is packable, we take into account not just equivalence type
    definitions, but the definitions of notag types as well.
    This takes advantage of the fact that if a notag type is abstract
    exported, its representation is put into the relevant .int3 file
    even though its definition isn't. (This is why du_type_layout.m
    couldn't "see through" notag types: it couldn't depend on knowing
    which types were notags.)

compiler/prog_item.m:
    Change the types we use for type representation information.
    Their previous definitions baked in the assumption that the only
    distinction between platforms that mattered was the 64 vs 32 bit
    distinction, which is not the case.

    Use a more consistent naming scheme for the types we use
    to represent type representation information.

    Include the "dereferenced" types of the arguments in functors'
    representations. (I use "dereferencing" here to mean expanding
    equivalence types and throwing away any notag wrappers.).
    We don't need it when generating C code using the low level
    data representation, but we do need it to create constructors
    when generating e.g. Java code that uses the high level data
    representation.

compiler/parse_type_repn.m:
    Rewrite most of this module due to the changes in prog_item.m.

compiler/parse_tree_out_type_repn.m:
    A new module containing the code for writing out type representations.
    The original code used to be in parse_tree_out.m, but it has been
    mostly rewritten. Partly this is due the changes in prog_item.m,
    but partly it is to provide much more structured output for humans,
    since this makes debugging so much easier.

compiler/parse_tree.m:
    Add the new module to the parse_tree package.

compiler/parse_tree_out.m:
    Delete the code moved to parse_tree_out_type_repn.m.

compiler/parse_tree_out_info.m:
    Provide a mechanism for selecting between output for machines
    (the default) and output for humans.

compiler/hlds_data.m:
compiler/prog_data.m:
    Move the ptag type from hlds_data.m to prog_data.m, to make it
    accessible in prog_item.m.

    Add some documentation in prog_data.m.

compiler/comp_unit_interface.m:
    If the experiment1 option is enabled, invoke decide_type_repn.m
    to decide what type_repn items to put into the .int file we are
    generating. Otherwise, maintain the status quo.

compiler/write_module_interface_files.m:
    Pass the globals to comp_unit_interface.m so it can look up experiment1.

compiler/equiv_type.m:
    Add a predicate for expanding equivalence types for use by
    decide_type_repn.m. This predicate expands just one type,
    but reports any use of circular equivalence types in that type.

    Improve the error message for circular equivalence types by *naming*
    the type constructors involved. To make this possible, pass around
    sets of such type constructors instead of just a boolean saying
    *whether* we have found *some* circular equivalence type.

    Replace bools used as changed/unchanged flag with a bespoke type.

    Standardize some variable names.

compiler/options.m:
    Add the developer-only option --pack-everything, which, if set,
    tells decide_type_repn.m to turn on all the packing options
    that currently work. This is to allow the testing of decide_type_repn.m
    in the eventual intended mode of operation, even if the various
    allow-packing-... options used by du_type_layout.m are set to "no".

compiler/disj_gen.m:
compiler/equiv_type_hlds.m:
compiler/llds_out_data.m:
compiler/lookup_util.m:
compiler/ml_call_gen.m:
compiler/ml_closure_gen.m:
compiler/mlds_to_c_data.m:
compiler/rtti.m:
compiler/rtti_out.m:
compiler/rtti_to_mlds.m:
compiler/tag_switch.m:
    Conform to the changes above (mostly the move of ptag to prog_data.m.)

compiler/parse_pragma.m:
    Improve indentation.

tests/valid_make_int/test_repn.m:
tests/valid_make_int/test_repn_sub.m:
    A fairly comprehensive test case of the new functionality.
    test_repn_sub.m defines one ore more simple type constructors
    of each possible kind, and test_repn.m uses them to define types
    that use each possible kind of complex type representation.

tests/valid_make_int/Mmakefile:
tests/valid_make_int/Mercury.options:
    Enable the new test case.
2020-05-28 07:41:44 +10:00
Zoltan Somogyi
4277af0fb2 Fix a silent failure for missing/malformed files.
compiler/mercury_compile_main.m:
    If attempting to read in a module results in one or more fatal errors,
    print the error messages we got from the attempt. Without this, we just
    got a silent failure, which is very frustrating.

    Delete the code that generated a warning for a mismatch between the
    expected module name based on the file name, and the actual module name.
    We made such mismatches an error a while ago, and there is a test case
    for it: invalid/bad_module_name.m.

compiler/write_module_interface_files.m:
    Clarify a related error message.
2020-05-20 07:33:25 +10:00
Zoltan Somogyi
1b56de1dc2 Prevent error exit without error messages.
Code I introduced on 2020 march 13 to filter out error_specs if the option
--print-errors-warnings-when-generating-interface is not set could
delete error messages *after* the exit status of the compiler has been set
to a nonzero value. This would mean that an invocation of mmc with
--make-interface would fail to create the requested interface file,
which would mean that the mmake invocation that invoked mmc would stop,
but would give absolutely no indication of *why* it stopped.

This diff fixes that bug. The diff has two main parts.

compiler/error_util.m:
    The first part is: make filter_interface_generation_specs test
    the exit status, and refuse to filter out any error messages
    if the exit status has already been set to indicate failure.

compiler/write_module_interface_files.m:
    In the predicates that generate and write out the contents of the
    module qualified interface files, go back to their previous structure.
    (The errors that we could omit to issue were errors discovered during
    module qualification).

    This diff is not the ideal fix, but the ideal fix requires changes
    to how we test errors generated while making interface files, and thus
    is left for future work.

compiler/mercury_compile_main.m:
    Conform to the change in error_util.m.

compiler/hlds_error_util.m:
    Update a comment after the renaming of a predicate.
2020-03-25 11:56:57 +11:00
Zoltan Somogyi
57d937943b Put type_repn items into .int3 files.
compiler/comp_unit_interface.m:
    We used to put type_repn items for the simple types defined in a module
    into the module's .int3 file if the option --experiment1 was given.
    Change this to *always* do this. (What makes a type "simple" is defined
    in du_type_layout.m; roughly a type is simple if its representation
    does not depend on the representation of any other type.) It is
    from these type_repn items in .int3 files that we should be able to
    compute type representations for *all* types when creating a module's
    .int file.

    Don't take a globals parameter, as it is no longer needed.

compiler/write_module_interface_files.m:
    Don't pass a globals parameter.

compiler/options.m:
    Add a synonym for --compiler-sufficiently-recent, so a later diff
    to configure.ac can test for the presence of this functionality
    in the installed compiler.
2020-03-22 10:56:10 +11:00
Zoltan Somogyi
ac466a3531 Avoid almost all conversions to parse_tree_int.
For a while now, we have been generating interface files into
file-kind-specific parse trees, but we then converted those into
the generic parse_tree_int type for further processing, because
the old code for doing that further processing operated on
parse_tree_ints. This diff replaces most of that "further processing"
code with code that operates on int-file-kind-specific parse trees,
thus avoiding the cost of conversion. The main cost is not performance
(the effect of this change on compiler speed is negligible), but loss
of code clarity through the loss of invariants.

compiler/comp_unit_interface.m:
    After generating a parse_tree_intN for N in {0,1,2,3}, do NOT convert it
    to a generic parse_tree_int.

    When constructing a parse_tree_int3 from a raw_compilation_unit,
    guard against a situation where we could unintentionally create
    a duplicate of an abstract type definition. The solution here
    is not the most elegant possible, but a better one requires other
    changes that don't really fit in this diff.

compiler/canonicalize_interface.m:
    Put two large blocks of code (dealing with the canonicalization
    of pred and mode declarations respectively) into their own predicates.

    Expose a mechanism to standardize not the contents of a whole
    interface file, but just the predicate and mode declarations in it.
    (These are the tricky kinds of items to canonicalize, because sorting
    a set of mode declarations can change the meaning of the program.)
    This mechanism is just a specialized version of the existing machinery.

    Since this diff leaves the existing machinery unused, the next diff
    will delete it, and since that will probably leave a too-small module,
    delete the whole module, after moving the remaining small piece of code
    to the only module that uses it, parse_tree_out.m.

compiler/parse_module.m:
    Instead of implementing actually_read_module_intN in terms of
    actually_read_module_int, do the reverse, which avoids the unnecessary
    conversion from parse_tree_intN to parse_tree_int.

compiler/parse_tree_out.m:
    Provide predicates, output_parse_tree_intN, to write out parse_tree_intN,
    not just a generic parse_tree_int.

    Change the existing service routines, mercury_output_parse_tree_intN,
    that do all the work of the above new predicates to write out the contents
    of each parse tree in the canonical order, using the newly exposed
    functionality in canonicalize_interface.m where needed. (Previously,
    write_module_interface_files.m invoked canonicalize_interface.m to do this
    before calling parse_tree_out.m.)

    These service routines were originally written for debugging; make them
    suitable for creating proper .intN files by removing unneeded output.

compiler/write_module_interface_files.m:
    Call the newly provided predicates in parse_tree_out.m to output
    interface files. To make this possible, break up the old predicate
    actually_write_interface_file into smaller predicates, so that the
    new predicates actually_write_interface_fileN can each call whichever
    of these are relevant for the given interface-file-kind. (Some pieces
    are relevant to only a subset of these kinds.)

compiler/convert_parse_tree.m:
    Note the reason why this diff is not deleting the functions that convert
    parse_tree_intN to parse_tree_int.

compiler/read_modules.m:
    Note the reason why this diff is not deleting the functions that read in
    generic parse_tree_ints.
2020-03-20 22:00:41 +11:00
Zoltan Somogyi
9789375cc5 Make pre-HLDS passes use file-kind-specific parse trees.
Replacing item blocks file-kind-specific kinds of section markers with
file-kind-specific parse trees has several benefits.

- It allows us to encode the structural invariants of each kind of file
  we read in within the type of its representation. This makes the detection
  of any accidental violations of those invariants trivial.

- Since each file-kind-specific parse tree has separate lists for separate
  kinds of items, code that wants to operate on one or a few kinds of items
  can just operate on those kinds of items, without having to traverse
  item blocks containing many other kinds of items as well. The most
  important consequence of this is not the improved efficiency, though
  that is nice, but the increased clarity of the code.

- The new design is much more flexible. For example, it should be possible
  to record that e.g. an interface file we read in as a indirect dependency
  (i.e. a file we read not because its module was imported by the module
  we are compiling, but because its module was imported by *another* imported
  module) should be used *only* for the purpose it was read in for. This should
  avoid situations where deleting an import of A from a module, because it
  is not needed anymore, leads the compiler to generate an error message
  about a missing import of module B. This can happen if (a) module B
  always *should* have been imported, since it is used, but (b) module A's
  import of module B lead to module B's interface being available *without*
  an import of B.

  Specifically, this flexibility should enable us to establish each module's
  .int file as the single source of truth about how values of each type
  defined in that module should be represented. When compiling each source
  file, this approach requires the compiler to read in that module's .int file
  but using only the type_repn items from that .int file, and nothing else.

- By recording a single parse tree for each file we have read, instead of
  a varying number of item blocks, it should be significantly easier to
  derive the contents of .d files directly from the records of those
  parse trees, *without* having to maintain a separate set of fields
  in the module_and_imports structure for that purpose. We could also
  trivially avoid any possibility of inconsistencies between these two
  different sources of truth. (We currently fill in the fields used to
  drive the generation of .d files using two different pieces of code,
  one used for --generate-dependencies and one used for all other invocations,
  and these two *definitely* generate inconsistent results, as the significant
  differences in .d files between (a) just after an invocation of
  --generate-dependencies and (b) just after any other compiler invocation
  can witness.)

This change is big and therefore hard to review. Therefore in many files,
this change adds "XXX CLEANUP" comments to draw attention to places that
have issues that should be fixed, but whose fixes should come later, in
separate diffs.

compiler/module_imports.m:
    The compiler uses the module_and_imports structure defined here
    to go from a raw compilation unit (essentially a module to be compiled)
    to an augmented compilation unit (a raw compilation unit together
    with all the interface and optimization files its compilation needs).
    We used to store the contents of both the source file and of
    the interface and optimization files in the module_and_imports structure
    as item blocks. This diff replaces all those item blocks with
    file-kind-specific parse trees, for the reasons mentioned above.

    Separate out the .int0 files of ancestors modules from the .intN
    files for N>0 of directly imported modules. (Their item blocks
    used to be stored in the same list.)

    Maintain a database of the source, interface and optimization files
    we have read in so far. We use it to avoid reading in interface files
    if we have already read in a file for the same module that contains
    strictly more information (either an interface file with a smaller
    number as a suffix, or the source file itself).

    Shorten some field names.

compiler/prog_item.m:
    Define data structures for storing information about include_module,
    import_module and use_module declarations, both in a form that allows
    the representation of possibly erroneous code in actual source files,
    and in checked-and-cleaned-up form which is guaranteed to be free
    of the relevant kinds of errors. Add a block comment at the start
    of the module about the need for this distinction.

    Define parse_tree_module_src, a data structure for representing
    the source code of a single module. This is different from the existing
    parse_tree_src type, which represents the contents of a single source file
    but which may contain *more* than one module, and also different from
    a raw_compilation_unit, which is based on item blocks and is thus
    unable to express to invariants such as "no clauses in the interface".

    Modify the existing parse_tree_intN types to express the distinction
    mentioned just above, and to unify them "culturally", i.e. if they
    store the same information, make them store it using the same types.

    Fix a mistake by allowing promises to appear in .opt files.
    I originally ruled them out because the code that generates .opt files
    does not have any code to write out promises, but some of the predicates
    whose clauses it writes out have goal_type_promise, which means that
    they originated as promises, and get written out as promises.

    Split the existing pragma item kind into three item kinds, which have
    different invariants applying to them.

    - The decl (short for declarative) pragmas give the compiler some
      information, such as that a predicate is obsolete or that we
      want to type specialize some predicate or function, that is in effect
      part of the module's interface. Decl pragmas may appear in module
      interfaces, and the compiler may put them into interface files;
      neither statement is true of the other two kinds of pragmas.

    - The impl (short for implementation) pragmas are named so
      precisely because they may appear only in implementation sections.
      They give the compiler information that is private to that module.
      Examples include foreign_decls, foreign_codes, foreign_procs,
      and promises of clause equivalence, and requests for inlining,
      tabling etc. These will never be put into interface files,
      though some of them can affect the compilation of other modules
      by being included in .opt files.

    - The gen (short for generated) pragmas can never (legally) appear
      in source files at all. They record the results of compiler
      analyses e.g. about which arguments of a predicate are unused,
      or what exceptions a function can throw, and accordingly they
      should only ever occur in compiler-generated interface files.

    Use the new type differences between the three kinds of pragmas
    to encode the above invariants about which kinds of pragmas can appear
    where into the various kinds of parse trees.

    Make the augmented compilation unit, which is computed from
    the final module_and_imports structure, likewise switch from
    storing item blocks to storing the whole parse trees of the
    files that went into its construction. With each such parse tree,
    record *why* we read it, since this controls what permissions
    the source module being compiled has for access to the entities
    in the parse tree.

    Simplify the contains_foreign_code type, since one of three
    function symbols was equivalent to one possible use of another
    function symbol.

    Provide a way to record which method of which class a compiler-generated
    predicate is for. (See hlds_pred.m below.)

    Move the code of almost all utility operations to item_util.m
    (which is imported by many fewer modules than prog_item.m),
    keeping just the most "popular" ones.

compiler/item_util.m:
    Move most of the previously-existing utility operations here from
    prog_item.m, most in a pretty heavily modified form.

    Add a whole bunch of other utility operations that are needed
    in more than one other module.

compiler/convert_parse_tree.m:
    Provide predicates to convert from raw compilation units to
    parse_tree_module_srcs, and vice versa (though the reverse
    shouldn't be needed much longer).

    Update the conversion operations between the general parse_tree_int
    and the specific parse_tree_intN forms for the changes in prog_item.m
    mentioned above. In doing so, use a consistent approach, based on
    new operations in item_util.m, to detect errors such as duplicate
    include_module and import/use_module declarations in all kinds
    of parse trees.

    Enforce the invariants that the types of parse trees of various kinds
    can now express in types, generating error messages for their violations.

    Delete some utility operations that have been moved to item_util.m
    because now they are also needed by other modules.

compiler/grab_modules.m:
    Delete code that did tests on raw compilation units that are now done
    when that raw compilation unit is converted to a parse_tree_module_src.
    Use the results of the checks done during that conversion to decide
    which modules are imported/used and in which module section.

    Record a single reason for why we reading in each interface and
    optimization file. The code of make_hlds_separate_items.m will use
    this reason to set up the appropriate permissions for each item
    in those files.

    Use separate code for handling different kinds of interface and
    optimization files. Using generic traversal code was acceptable economy
    when we used the same data structure for every kind of interface file,
    but now that we *can* express different invariants for different kinds
    of interface and optimization file, we want to execute not just different
    code for each kind of file, but the data structures we want to work on
    are also of different types. Using file-kind-specific code is a bit
    longer, but it is significantly simpler and more robust, and it is
    *much* easier to read and understand.

    Delete the code that separates the parts of the implementation section
    that are exported to submodules, and the part that isn't, since that task
    is now done in make_hlds_separate_items.m.

    Pass a database of the files we have read through the relevant predicates.

    Give some predicates more meaningful names.

compiler/notes/interface_files.html:
    Note a problem with the current operation of grab_modules.

compiler/get_dependencies.m:
    Add operations to gather implicit references to builtin modules
    (which have to be made available even without an explicit import_module
    or use_module declaration) in all kinds of parse trees. These have
    more code overall, but will be at runtime, since we need only look at
    the item kinds that may *have* such implicit references.

    Add a mechanism to record the result of these gathering operations
    in import_and_or_use_maps.

    Give some types, function symbols, predicates and variables
    more meaningful names.

compiler/make_hlds_separate_items.m:
    When we stored the contents of the source module and the
    interface and optimization files we read in to augment it
    in the module_and_imports structure as a bunch of item blocks,
    the job of this module was to separate out the different kinds of items
    in the item blocks, returning a single list of each kind of item,
    with each such item being packaged up with its status (which encodes
    a set of permissions saying what the source module is allowed
    to do with it).

    Now that the module_and_imports structure stores this info in
    file-kind-specific parse trees, all of which have separate lists
    for each kind of item and none of which contain item blocks,
    the job of this module has changed. Now its job is to convert
    the reason why each file was read in into the (one or more) statuses
    that apply to the different kinds of items stored in it, wrap up
    each item with its status, and return the resulting overall list
    of status/item pairs for each kind of item.

compiler/read_modules.m:
    Add predicates that, when reading an interface file, return its contents
    in the tightest possible file-kind-specific parse tree.

    Refine the database of files we have read to allow us to store
    more file-kind-specific parse trees.

    Don't require that files in the database have associated timestamps,
    since in some cases, we read files we can put into the database
    *without* getting their timestamps.

    Allow the database to record that an attempt to read a file failed.

compiler/split_parse_tree_src.m:
    Rearchitect how this module separates out nested submodules from within
    the main module in a file.

    Another of the jobs of this module is to generate error messages for
    when module A includes module B twice, whether via nesting or via
    include_module declarations, with one special exception for the case
    where A's interface contains nested submodule A.B's interface,
    and A's implementation contains nested submodule A.B's implementation.
    The problem ironically was that while it reported duplicate include_module
    declarations as errors, split_parse_tree_src.m also *generated*
    duplicate include_module declarations. Since it replaced each nested
    submodule occurrence with an include_module declaration, in the scenario
    above, it generated two include_module declarations for A.B. Even worse,
    the interface incarnation of submodule A.B could contain
    (the interface of) its own nested submodule A.B.C, while its
    implementation incarnation could contain (the implementation section of)
    A.B.C. Each occurrence of A.B.C would be its only occurrence in the
    including part of its parent A.B, which means local tests for duplicates
    do not work. (I found this out the hard way.)

    The solution we now adopt adds include_module declarations to the
    parents of any submodule only once the parse tree of the entire
    file has been processed, since only then do we know all the
    includer/included relationships among nested modules. Until then,
    we just record such relationships in a database as we discover them,
    reporting duplicates when needed (e.g. when A includes B twice
    *in the same section*), but not reporting duplicates when not needed
    (e.g. when A.B includes A.B.C in *different* sections).

compiler/prog_data.m:
    Add a new type, pf_sym_name_and_arity, that exactly specifies
    a predicate or function. It is a clone of the existing simple_call_id
    type, but its name does NOT imply that the predicate or function
    is being called.

    Add XXXs that call for some other improvements in type names.

compiler/prog_data_foreign.m:
    Give a type, and the operations on that type, a more specific name.

compiler/error_util.m:
    Add an id field to all error_specs, which by convention should be
    filled in with $pred. Print out the value in this field if the compiler
    is invoked with the developer-only option --print-error-spec-id.
    This allows a person debugging the compiler find out where in the code
    an undesired error message is coming from significantly easier
    than was previously possible.

    Most of the modules that have changes only "to conform to the changes
    above" will be for this change. In many cases, the updated code
    will also simplify the creation of the affected error_specs.

    Fix a bug that looked for a phase in only one kind of error_spec.

    Add some utility operations needed by other parts of this change.

    Delete a previously internal function that has been moved to
    mdbcomp/prim_data.m to make it accessible in other modules as well.

compiler/Mercury.options:
    Ask the compiler to warn about dead predicates in every module
    touched by this change (at least in one its earlier versions).

compiler/add_foreign_enum.m:
    Replace a check for an inappropriately placed foreign_enum declaration
    with a sanity check, since with this diff, the error should be caught
    earlier.

compiler/add_mutable_aux_preds.m:
    Delete a check for an inappropriately placed mutable declaration,
    since with this diff, the error should be caught earlier.

compiler/add_pragma.m:
    Instead of adding pass2 and pass3 pragmas, add decl and impl and
    generated pragmas.

    Delete the tests for generated pragma occurring anywhere except
    .opt files, since those tests are now done earlier.

    Shorten some too-long predicate names.

compiler/comp_unit_interface.m:
    Operate on as specific kinds of parse trees as the interface of this
    module will allow. (We could operate on more specific parse trees
    if we changed the interface, but that is future work).

    Use the same predicates for handling duplicate include_module,
    import_module and use_module declarations as everywhere else.

    Delete the code of an experiment that shouldn't be needed anymore.

compiler/equiv_type.m:
    Replace code that operated on item blocks with code that operates
    on various kinds of parse trees.

    Move a giant block of comments to the front, where it belongs.

compiler/hlds_module.m:
    Add a field to the module_info that lets us avoid generating
    misleading error messages above missing definitions of predicates
    or functions when those definitions were present but were not
    added to the HLDS because they had errors.

    Give a field and its access predicates a more specific name.

    Mark a spot where an existing type cannot express everything
    it is supposed to.

compiler/hlds_pred.m:
    For predicates which the compiler creates to represent a class method
    (the virtual function, in OOP terms), record not just this fact,
    but the id of the class and of the method. Using this extra info
    in progress messages (with mmc -V) prevents the compiler from printing e.g.

        % Checking typeclass constraints on class method
        % Checking typeclass constraints on class method
        % Checking typeclass constraints on class method

    when checking three such predicates.

compiler/make.m:
    Provide a slot in the make_info structure to allow the database
    of the files we have read in to be passed around.

compiler/make_hlds_error.m:
    Delete predicates that are needed in just one other module,
    and have therefore been moved there.

compiler/make_hlds_passes.m:
    Add decl, impl and generated pragma separately, instead of adding
    pass2 and pass3 pragmas separately.

    Do not generate error messages for clauses, initialises or finalises
    in module interfaces, since with this diff, such errors should be
    caught earlier.

compiler/mercury_compile_main.m:
compiler/recompilation.check.m:
    Explicitly pass around the expanded database of parse trees
    of files that have been read in.

compiler/module_qual.collect_mq_info.m:
compiler/module_qual.m:
compiler/module_qual.qualify_items.m:
    Collect module qualification information, and do module qualification
    respectively on parse trees of various kinds, not item blocks.
    Take information about what the module may do with the contents
    of each interface or optimization file from the record of why
    we read that file, not from the section markers in item blocks.

    Break up some too-large predicates by carving smaller ones out of them.

compiler/options.m:
    Add an option to control whether errors and/or warnings detecting
    when deciding what should go into a .intN file be printed,
    thus (potentially) preventing the creation of that file.

    Add commented-out documentation for a previously totally undocumented
    option.

doc/user_guide.texi:
    Document the new option.

NEWS:
    Announce the new option.

    Mention that we now generate warnings for unused import_module and
    use_module declarations in the interface even if the module has
    submodules.

compiler/write_module_interface_files.m:
    Let the new option control whether we filter out any messages generated
    when deciding what should go into a .intN file.

compiler/parse_item.m:
    Delete actually_read_module_opt, since it is no longer needed;
    its callers now call actually_read_module_{plain,trans}_opt instead.

    Delete unneeded arguments from some predicates.

compiler/parse_module.m:
    Delete some long unused predicates.

compiler/parse_pragma.m:
    When parsing pragmas, wrap them up in the new decl, impl or generated
    pragma kinds.

compiler/parse_tree_out.m:
    Add predicates to write out each of the file-kind-specific parse trees.

compiler/parse_tree_out_pragma.m:
    Add predicates to write out decl, impl and generated pragmas.

compiler/polymorphism.m:
    Add a conditionally-enabled progress message, which can be useful
    in tracking down problems.

compiler/prog_item_stats.m:
    Conform NOT to the changes above beyond what is needed to let this module
    compile. Let that work be done the next time the functionality of
    this module is needed, by which time the affected data structures
    maybe have changed further.

compiler/typecheck.m:
    Fix a performance problem. With intermodule optimization, we read in
    .opt files, some of which (e.g. list.opt and int.opt) contain promises.
    These promises are read in as predicates with goal_type_promise,
    but they do not have declarations of the types of their arguments
    (since promises do not have declarations as such). Those argument types
    therefore have to be inferred. That inference replaces the original
    "I don't know" argument types with their actual types.

    The performance problem is that when we change the recorded argument types
    of a predicate, we require another loop over all the predicates in the
    module, so that any calls to this predicate can be checked against
    the updated types. This is as it should be for callable predicates,
    but promises are not callable. So if all the *only* predicates whose
    recorded argument types change during the first iteration to fixpoint
    are promises, then a second iteration is not needed, yet we used to do it.

    The fix is to replace the "Have the recorded types of this predicate
    changed?" boolean flag with a bespoke enum that says "Did the checking
    of this predicate discover a need for another iteration", and not
    setting it when processing predicates whose type is goal_type_promise.

compiler/typecheck_errors.m:
    Do not generate an error message for a predicate missing its clauses
    is the clauses existed but were not added to the HLDS because they were
    in the interface section.

    When reporting on ambiguities (when a call can match more than one
    predicate or function), sort the possible matches before reporting
    them.

compiler/accumulator.m:
compiler/add_class.m:
compiler/add_clause.m:
compiler/add_foreign_proc.m:
compiler/add_mode.m:
compiler/add_pragma_tabling.m:
compiler/add_pragma_type_spec.m:
compiler/add_pred.m:
compiler/add_type.m:
compiler/canonicalize_interface.m:
compiler/check_for_missing_type_defns.m:
compiler/check_parse_tree_type_defns.m:
compiler/check_promise.m:
compiler/check_raw_comp_unit.m:
compiler/check_typeclass.m:
compiler/common.m:
compiler/compile_target_code.m:
compiler/compiler_util.m:
compiler/dead_proc_elim.m:
compiler/deps_map.m:
compiler/det_analysis.m:
compiler/det_report.m:
compiler/du_type_layout.m:
compiler/field_access.m:
compiler/find_module.m:
compiler/float_regs.m:
compiler/format_call.m:
compiler/goal_expr_to_goal.m:
compiler/handle_options.m:
compiler/hlds_out_module.m:
compiler/hlds_out_pred.m:
compiler/hlds_out_util.m:
compiler/inst_check.m:
compiler/intermod.m:
compiler/introduce_parallelism.m:
compiler/layout_out.m:
compiler/make.dependencies.m:
compiler/make.module_dep_file.m:
compiler/make_hlds_warn.m:
compiler/mark_tail_calls.m:
compiler/mercury_compile_llds_back_end.m:
compiler/ml_top_gen.m:
compiler/mmakefiles.m:
compiler/mode_errors.m:
compiler/mode_robdd.equiv_vars.m:
compiler/modes.m:
compiler/module_qual.qual_errors.m:
compiler/oisu_check.m:
compiler/old_type_constraints.m:
compiler/options_file.m:
compiler/parse_class.m:
compiler/parse_dcg_goal.m:
compiler/parse_goal.m:
compiler/parse_inst_mode_defn.m:
compiler/parse_inst_mode_name.m:
compiler/parse_mutable.m:
compiler/parse_sym_name.m:
compiler/parse_type_defn.m:
compiler/parse_type_name.m:
compiler/parse_type_repn.m:
compiler/parse_types.m:
compiler/parse_util.m:
compiler/parse_vars.m:
compiler/post_term_analysis.m:
compiler/post_typecheck.m:
compiler/prog_event.m:
compiler/prog_mode.m:
compiler/purity.m:
compiler/qual_info.m:
compiler/recompilation.version.m:
compiler/resolve_unify_functor.m:
compiler/simplify_goal.m:
compiler/simplify_goal_call.m:
compiler/simplify_goal_disj.m:
compiler/simplify_goal_ite.m:
compiler/simplify_proc.m:
compiler/state_var.m:
compiler/stratify.m:
compiler/style_checks.m:
compiler/superhomogeneous.m:
compiler/table_gen.m:
compiler/term_constr_errors.m:
compiler/term_errors.m:
compiler/termination.m:
compiler/trace_params.m:
compiler/unused_args.m:
compiler/unused_imports.m:
compiler/write_deps_file.m:
compiler/xml_documentation.m:
    Conform to the changes above.

mdbcomp/prim_data.m:
    Move a utility function on pred_or_funcs here from a compiler module,
    to make it available to other compiler modules as well.

scripts/compare_s1s2_lib:
    A new script that helped debug this diff, and may help debug
    similar diffs the future. It can compare (a) .int* files, (b) .*opt
    files, (c) .mh/.mih files or (d) .c files between the stage 1 and
    stage 2 library directories. The reason for the restriction
    to the library directory is that any problems affecting the
    generation of any of these kinds of files are likely to manifest
    themselves in the library directory, and if they do, the bootcheck
    won't go on to compile any of the other stage 2 directories.

tests/debugger/breakpoints.a.m:
tests/debugger/breakpoints.b.m:
    Move import_module declarations to the implementation section
    when they are not used in the interface. Until now, the compiler
    has ignored this, but this diff causes the compiler to generate
    a warning for such misplaced import_module declarations even modules
    that have submodules. The testing of such warnings is not the point
    of the breakpoints test.

tests/invalid/Mercury.options:
    Since the missing_interface_import test case tests error messages
    generated during an invocation of mmc --make-interface, add the
    new option that *allows* that invocation to generate error messages.

tests/invalid/ambiguous_overloading_error.err_exp:
tests/invalid/max_error_line_width.err_exp:
tests/warnings/ambiguous_overloading.exp:
    Expect the updated error messages for ambiguity, in which
    the possible matches are sorted.

tests/invalid/bad_finalise_decl.m:
tests/invalid/bad_initialise_decl.m:
    Fix programming style.

tests/invalid/bad_item_in_interface.err_exp:
    Expect an error message for a foreign_export_enum item in the interface,
    where it should not be.

tests/invalid/errors.err_exp:
    Expect the expanded wording of a warning message.

tests/invalid/foreign_enum_invalid.err_exp:
    Expect a different wording for an error message. It is more "standard"
    but slightly less informative.

tests/invalid_submodules/children2.m:
    Move a badly placed import_module declaration, to avoid having
    the message the compiler now generates for it from affecting the test.

tests/submodules/parent2.m:
    Move a badly placed import_module declaration, to avoid having
    the message the compiler now generates for it from affecting the test.

    Update programming style.
2020-03-13 12:58:33 +11:00
Zoltan Somogyi
fc347fb6a1 Stop generating interface files only in the presence of errors.
compiler/write_module_interface_files.m:
    If the code that computes the contents of an interface file also
    generates messages to the user, don't take the mere existence of
    these messages as grounds for not writing out the interface file;
    instead, require them to report errors, or (if --halt-at-warn is enabled)
    warnings.

compiler/error_util.m:
    Provide a predicate for checking the above condition.

compiler/check_raw_comp_unit.m:
    Fix, in two different ways, the warning that indicated the need for
    the change to write_module_interface_files. First, if the module
    exports nothing, then generate a warning only if warn_nothing_exported
    is set, instead of always generating a warning that is conditional
    on warn_nothing_exported being set. The latter doesn't just tickle
    the problem mentioned above, it also uses more complex code. Second,
    fix the wording of the warning.

tests/invalid/bigtest.err_exp:
tests/invalid/duplicate_modes.err_exp:
tests/invalid/empty_interface.err_exp:
tests/invalid/errors.err_exp:
tests/invalid/errors1.err_exp:
tests/invalid/errors2.err_exp:
tests/invalid/funcs_as_preds.err_exp:
tests/invalid/inst_list_dup.err_exp:
tests/invalid/no_exports.err_exp:
tests/invalid/occurs.err_exp:
tests/invalid/prog_io_erroneous.err_exp:
tests/invalid/type_inf_loop.err_exp:
tests/invalid/typeclass_missing_det_3.err_exp:
tests/invalid/typeclass_no_param.err_exp:
tests/invalid/typeclass_test_11.err_exp:
tests/invalid/types.err_exp:
tests/invalid/undef_inst.err_exp:
tests/invalid/undef_mode.err_exp:
tests/invalid/undef_type.err_exp:
tests/invalid/unicode1.err_exp:
tests/invalid/unicode2.err_exp:
tests/invalid/vars_in_wrong_places.err_exp:
    Expect the improved wording of the warning.
2019-10-27 01:25:11 +11:00
Zoltan Somogyi
19ef98b66f Check the type definitions in a module for consistency ...
... when generating interface files.

compiler/check_parse_tree_type_defns.m:
    A new module for doing the checking.

compiler/parse_tree.m:
    Add the new module.

compiler/comp_unit_interface.m:
    When computing what should go into a .intN file, use the new module
    to check for inconsistencies in the source code of the module.

    Making the above possible requires retaining some information
    (such as contexts in foreign enum definitions, and the presence
    of definitions, as opposed to declarations, for some types)
    for longer than was needed until now.

    Give some predicates more descriptive names.

    Move the record_foreign_enum_spec predicate near where it is used.

compiler/write_module_interface_files.m:
    If comp_unit_interface.m reports any errors when generating the
    would-be contents of an interface file, do NOT create that
    interface file.

    The errors comp_unit_interface.m and check_parse_tree_type_defns.m
    now generate are the kinds of inconsistencies whose resolution requires
    everything depending on this module to be recompiled anyway,
    so stopping the compilation process *without* producing
    the would-be-erroneous interface file should be a net win
    almost all the time.

compiler/prog_item.m:
    Factor out some commonalities in data structures.

    When converting the generic parse_tree_int we get from reading
    in a .intN file to the parse tree type specific to that N,
    check the definitions we read in for consistency.

    As in comp_unit_interface, making the above possible requires
    retaining some information for longer than was needed until now.

    Never output two or more declarations of the same type_ctor
    in the same section of an interface file, since the code
    *reading* interface files now reports such redundancies
    (but see the change to options.m).

compiler/options.m:
    Disable the printing of error messages for problems found in
    interface files, since the new code in check_parse_tree_type_defns.m
    now finds problems in interface files generated by currently
    installed compilers. Specifically, many contain duplicate
    declarations for Mercury types. One declaration is written
    by the programmer, the other is the type's actual definition
    turned into the redundant declaration by the compiler.

compiler/parse_pragma.m:
    When parsing foreign enum pragmas, implicitly qualify the
    name of the type constructor they are for with the current module name.
    This is semantically sound, since foreign_enum pragmas *must* be
    for a type constructor defined in the same module. It is desirable
    because type_ctors in type definitions are module qualified.
    The code of check_parse_tree_type_defns.m needs to process
    the type definitions and foreign enums for a type_ctor at the
    same time, and doing so would be more needlessly complicated
    if the type ctor keys in the "type_ctor to type definitions"
    and "type_ctor to foreign enums" maps were incompatible due to
    the difference in qualification.

    The type_ctor in a foreign enum definition may still be manually
    module qualified by programmers; it just happens that any qualification
    other than the default is a semantic error.

compiler/module_qual.qual_errors.m:
    When printing error messages about undefined type_ctors in
    foreign enum definitions, do not print this implicitly
    added qualification, since it adds only clutter.

compiler/error_util.m:
    Provide a mechanism for printing type_ctors directly,
    i.e. without converting them to a sym_name/arity pair.

    Put a qual_ prefix in front of top_ctor_of_type, since it
    always prints module qualification. (This is to remind people
    who may use this component about that qualification.)

    Factor out some common code.

compiler/det_analysis.m:
    Conform to the change in error_util.m.

compiler/parse_module.m:
    Fix too-long lines.

compiler/Mercury.options:
    Require some of the modules above to have no dead predicates.

library/erlang_rtti_implementation.m:
library/io.m:
    Delete declarations of types that are unneeded because the types
    also have definitions.

tests/typeclasses/unqualified_method2.m:
    Move an import out of the interface, since we now print the warning
    generated for its improper location.
2019-09-28 02:57:00 +10:00
Zoltan Somogyi
c45719f5cb Don't report errors when creating interface files.
When deciding the contents of interface files, we used to generate
messages for these three kinds of errors:

- clauses in the module interface
- not-allowed-in-the-interface pragmas in the module interface
- empty module interface

None of these affected the interface file we were building, and their
reporting can easily be delayed until we are compiling the affected module
to target language code. At that time, the error messages get put into
.err files, whereas any messages printed when creating interface files
tends to get lost in the overall sea of compiler invocation messages
when running mmake.

compiler/comp_unit_interface.m:
    Do not generate error messages when building the contents of interface
    files.

compiler/write_module_interface_files.m:
    Do not print the error messages that are no longer being generated.
    We *do* still print any of the other kinds of error messages we have
    been printing, such as for those when a needed .int3 file cannot be read,
    or when ambiguity prevents full module qualification. (The contents of
    .int and .int2 files must be fully module qualified to do their jobs.)

compiler/add_pragma.m:
    Report an error when processing a pass 3 pragma that occurs in the
    interface but is not allowed in the interface.

    add_pragma.m has already been doing this for all other kinds of pragmas,
    and in general, compiler invocations that generate target language code
    do generate the messages for all the conditions whose messages we are
    no longer printing when generating interface files.

tests/invalid/bad_item_in_interface.{m,err_exp}:
tests/invalid/ft_examples:
    A new test case to test the above assertion with respect to
    inappropriate items in the interface of a module.

tests/invalid/empty_interface.{m,err_exp}:
    A new test case to test the above assertion with respect to
    the interface of a module being empty.

tests/invalid/Mercury.options:
    Specify the right options for the empty_interface test case.

tests/invalid/Mmakefile:
    Enable the new test cases.
2019-08-27 06:06:17 +10:00
Zoltan Somogyi
5b43964b06 Create and use bespoke parse trees for .int and .int2 files.
For now, we construct the contents of those files using the new bespoke types,
but convert them to the general parse_tree_int type when that is done.

Later changes should extend the use of the bespoke types to the later
processing of interface file contents (computation of version number maps,
standardization, and writing out) as well as their processing when
being read in.

compiler/prog_item.m:
    Create the bespoke types parse_tree_int1 and parse_tree_int2
    for the parse trees of .int and .int2 files respectively.
    These types make explicit significantly more of the structural invariants
    required of the contents of these interface files than the old general
    parse_tre_int type.

    Add conversion functions from the new bespoke types to that general type.

    Create a bespoke type for representing version number fields,
    mainly to provide a central place for documenting how such fields
    are filled in.

    Make predicate definition order match predicate declaration order.

compiler/comp_unit_interface.m:
    When generating .int and .int2 files, create their contents as values
    of the new bespoke types, converting them to the general parse_tree_int
    type when that is done.

compiler/module_imports.m:
compiler/parse_module.m:
compiler/parse_tree_out.m:
compiler/recompilation.check.m:
compiler/recompilation.version.m:
compiler/write_module_interface_files.m:
    Conform to the change in version number representation.
2019-08-23 15:57:43 +10:00
Zoltan Somogyi
19034b3b1a Start using a bespoke type for .int3 parse trees.
compiler/prog_item.m:
    Define parse_tree_int3, a type that represents the parse trees
    of .int3 files. It enforces several structural invariants that
    the old parse_tree_int type (which was designed to represent the
    contents of *all* interface files) does not: the absence of anything
    in the implementation section, the absence of use_module declarations,
    the absence of repeated include_module or import_module declarations,
    and the absence of any items other than the (a) the definitions of
    types, insts, modes, typeclasses and instances, and later
    (b) type_repn items.

    Define a function to convert from parse_tree_int3s to parse_tree_ints.

    Define auxiliaty predicates and functions now needed by
    comp_unit_interface.m, and similar predicates and functions
    for related types, that may be needed later.

compiler/comp_unit_interface.m:
    Compute the contents of .int3 files into parse_tree_int3s.
    Translate these to parse_tree_ints for the rest of the compiler,
    which does use the new type yet.

compiler/decide_type_repn.m:
    Return type representation information for .int3 files
    in the format required by the updated code in comp_unit_interface.m.

compiler/write_module_interface_files.m:
    Ignore the new parse_tree_int3 returned by comp_unit_interface.m
    until the rest of this module has been taught to use it.
2019-08-04 20:47:03 +02:00
Zoltan Somogyi
b063063713 Generate .int2 files using the mechanism for .int3 files, experimentally.
compiler/write_module_interface_files.m:
    If the right experiment option is set, when we generate m.int2,
    generate m.int2.via3 as well, using the same algorithm we use
    to generate .int3 files. This new file is not used anywhere.
    It is purely an experiment, one that allows us to compare the
    advertised nature of .int2 files (as fully module qualified versions
    of .int3 files) with the reality.

    Modify a service predicate to make the above possible, and to delete
    a long-unused argument.

compiler/options.m:
    Add a bunch of boolean experiment options, to control this and other
    (upcoming) experiments. Update the documentation of the experiment
    options.

compiler/comp_unit_interface.m:
    Add a predicate for generating .int2 files using the algorithm we use
    for .int3 files.

    Switch to using one the new, bool experiment options for an existing
    experiment.
2019-08-04 14:31:36 +02:00
Zoltan Somogyi
7ea66c40e2 Rename modules.m to grab_modules.m.
compiler/grab_modules.m:
    As above. The new name indicates the module's function
    much more clearly.

compiler/parse_tree.m:
    Rename the module in its containing package.

compiler/mercury_compile_main.m:
compiler/write_module_interface_files.m:
    Conform to the change above.

compiler/notes/compiler_design.html:
    Document the change. Put the description of the related modules
    into a more logical order.
2019-07-19 11:45:38 +02:00
Zoltan Somogyi
b6a5a97335 Module qualify .int3 files just once.
compiler/module_qual.m:
    Rename module_qualify_parse_tree_int as module_qualify_parse_tree_int3,
    since it is used only to module qualify .int3 files.

    Resolve an old XXX inside it about the treatment of implementation sections
    by requiring the implementation section to be empty (which it always is
    in .int3 files).

compiler/comp_unit_interface.m:
    Conform to the change above.

compiler/write_module_interface_files.m:
    Don't call module_qualify_parse_tree_int3, since the predicate in
    comp_unit_interface we just called has already done that.
2019-06-30 12:16:50 +02:00
Zoltan Somogyi
6241ca9c19 Add a pointer to notes/interface_files.html. 2019-06-26 15:06:40 +02:00
Zoltan Somogyi
12371148b5 Give two predicates more expressive names, ...
compiler/modules.m:
    ... and document them better.

compiler/mercury_compile_main.m:
compiler/write_module_interface_files.m:
    Use the new names.
2019-04-13 08:36:39 +10:00
Zoltan Somogyi
83e2f21a81 Ignore NumErrors in one central place.
compiler/error_util.m:
    For each predicate that writes out one or more error_specs, provide
    a version that does not return the numbers of warnings and errors
    written out.

compiler/deps_map.m:
compiler/find_module.m:
compiler/generate_dep_d_files.m:
compiler/handle_options.m:
compiler/make.module_dep_file.m:
compiler/mercury_compile_llds_back_end.m:
compiler/mercury_compile_main.m:
compiler/modes.m:
compiler/options_file.m:
compiler/pd_util.m:
compiler/recompilation.check.m:
compiler/typecheck.m:
compiler/write_module_interface_files.m:
    Replace calls write_error_{spec,specs} that ignored NumWarnings
    and NumErrors with calls to the versions that do not return those
    counts in the first place.
2019-04-11 07:00:57 +10:00
Zoltan Somogyi
c544da19f3 Fix indentation. 2019-03-29 13:03:38 +11:00
Zoltan Somogyi
590927fdeb Define, output and parse type representation items.
My plan to ensure that the compilation of every module sees a consistent
set of type representations requires these representations to be put
into interface files. This diff is the first step in the implementation
of that plan. It defines the structure of the items that hold type
representation information, and adds code both to write out such items,
and to read them in.

compiler/prog_item.m:
    Add the kinds of tc_repn items that I expect my plan will need.

compiler/parse_tree_out.m:
    Add code to output the new kinds of tc_repn items.

    Give the exported predicates more meaningful names.

compiler/parse_type_repn.m:
    Add code to parse the new kinds of tc_repn items.

compiler/du_type_layout.m:
    Add sanity checks that prohibits arity zero function symbols
    having any existential type constraints (since they have no arguments
    whose types could contain constrainable type variables).

    Don't pass around an ultimately unused argument.

    Delete some obsolete comments. Improve some other comments.

compiler/globals.m:
    Add a predicate version of an existing functions, so we can give it
    a reverse mode (which is needed by parsing code above).

compiler/mercury_compile_main.m:
compiler/module_qual.qualify_items.m:
compiler/write_module_interface_files.m:
    Conform to the change in the names of the predicates exported
    by parse_tree_out.m.

compiler/prog_data.m:
    Clarify a comment,
2019-03-12 06:44:17 +11:00
Zoltan Somogyi
4d1c969141 Delete the old code generating .int and ,int2 files.
compiler/write_module_interface_files.m:
    On feb 15, I committed two diffs that added new, much more direct ways
    to generate the contents of .int and .int2 files. I kept the old code
    that generated those contents in a much more convoluted way, but kept
    a sanity check that required the outputs of the old and new algorithms
    to agree.

    This diff deletes the old algorithms, along with the sanity check.
2019-02-23 00:42:42 +11:00
Zoltan Somogyi
0b36114595 Group related predicates together.
compiler/comp_unit_interface.m:
    Move the code to generate the contents of .int0 files here from
    write_module_interface_files.m. (I originally left it in its
    original file to make reviewing its diff easier.)

    Put predicates into generating interface files into a standard order:
    code for .int3, code for .int0, and then code for .int/.int2. This is
    in ascending order of complexity.

    Put at the end of the module the predicates that we shouldn't need
    once we modify the code for --generate-dependencies to build the
    module_and_imports structure the standard way.

    Improve some predicate names.

compiler/write_module_interface_files.m:
    Put predicates into writing out interface files into a standard order:
    code for .int3, code for .int0, and then code for .int/.int2.

    Put at the end of the module the predicates that we shouldn't need
    once we delete the sanity checks comparing the outputs of the
    old and new algorithms for generating contents for interface files.
2019-02-18 18:03:59 +11:00
Zoltan Somogyi
509aba2782 Separate two only sort-of related algorithms.
compiler/comp_unit_interface.m:
    The get_interface predicate used to do one of two jobs, which the caller
    selecting the job via a flag parameter. Replace this setup with two
    predicates, one for each job.

    Since the old flag would now always have the same value inside
    the call tree of get_interface, simplify the code by assuming that value.

compiler/module_imports.m:
compiler/modules.m:
compiler/write_module_interface_files.m:
    Conform to the changed signatures of predicates in comp_unit_interface.m.
2019-02-18 15:25:48 +11:00