Reduce the overhead of all forms of tabling by eliminating in many cases

Estimated hours taken: 32
Branches: main

Reduce the overhead of all forms of tabling by eliminating in many cases
the overhead of transferring data across the C/Mercury boundary. These
involve lots of control transfers as well as assignments to and from
Mercury abstract machine registers, which are not real machine registers
on x86 machines. Benchmarking in Uppsala revealed this overhead to be
a real problem.

The way we do that is by changing the tabling transformation so that instead
of generating sequences of calls to predicates from library/table_builtin.m,
we generate sequences of calls to C macros from runtime/mercury_tabling_pred.h,
and emit the resulting code string as the body of a foreign_proc goal.
(The old transformation is still available via a new option,
--no-tabling-via-extra-args.)

Since the number of inputs and outputs of the resulting C code sequences
are not always fixed (they can depend on the number of input or output
arguments of predicate being transformed), implementing this required
adding to foreign_procs a new field that allows the specification of extra
arguments to be passed to and from the given foreign code fragment. For now,
this mechanism is implemented only by the C backends, since it is needed
only by the C backends. (We don't support yet tabling on other backends.)

To simplify the new implementation of the field on foreign_procs, consolidate
three existing fields into one. Each of these fields was a list with one
element per argument, so turning them into a single list with a combined record
per argument should also improve reliability, since it reduces the likelyhood
of updates leaving the data structure inconsistent.

The goal paths of components of a tabled predicate depend on whether
-no-tabling-via-extra-args was specified. To enable the expected outputs
of the debugger test cases testing tabling, we add a new mdb command,
goal_paths, that controls whether goal paths are printed by the debugger
at events, and turn off the printing of events in the relevant test cases.

Also, prepare for a future change to optimize the trie structure for
user-defined types by handling type_infos (and once we support them,
typeclass_infos) specially.

compiler/table_gen.m:
	Change the tabling transformation along the lines described above.

	To allow us to factor out as much of the new code as possible,
	we change the meaning of the call_table_tip variable for minimal
	model subgoals: instead of the trie node at the end of the answer
	table, it is not now the subgoal reachable from it. This change
	has no effect as yet, because we use call_table_tip variables
	only to perform resets across retries in the debugger, and we
	don't do retries across calls to minimal model tabled predicates.

	Put predicates into logical groups.

library/table_builtin.m:
runtime/mercury_tabling_preds.h:
	When the new transformations in table_gen.m generate foreign_procs
	with variable numbers of arguments, the interfaces of those
	foreign_procs often do not match the interfaces of the existing
	library predicates at their core: they frequently have one more
	or one fewer argument. To prevent any possible confusion, in such
	cases we add a new variant of the predicate. These predicates
	have the suffix _shortcut in their name. Their implementations
	are dummy macros that do nothing; they serve merely as placeholders
	before or after which the macros that actually do the work are
	inserted.

	Move the definitions of the lookup, save and restore predicates
	into mercury_tabling_preds.h. Make the naming scheme of their
	arguments more regular.

runtime/mercury_minimal_model.c:
runtime/mercury_tabling_preds.h:
	Move the definition of a predicate from mercury_minimal_model.c
	to mercury_tabling_preds.h, since the compiler now needs to be
	able to generate an inlined version of it.

compiler/hlds_goal.m:
	Replace the three existing fields describing the arguments of
	foreign_procs with one, and add a new field describing the extra
	arguments that may be inserted by table_gen.m.

	Add utility predicates for processing the arguments of foreign_procs.

	Change the order of some existing groups of declarations make it
	more logical.

compiler/hlds_pred.m:
runtime/mercury_stack_layout.h:
	Extend the data structures recording the structure of tabling tries
	to allow the representation of trie steps for type_infos and
	typeclass_infos.

runtime/mercury_tabling_macros.c:
	Fix a bug regarding the tabling of typeclass_infos, which is now
	required for a clean compile.

compiler/pragma_c_gen.m:
compiler/ml_code_gen.m:
	Modify the generation of code for foreign_procs to handle extra
	arguments, and to conform to the new data structures for foreign_proc
	arguments.

compiler/llds.m:
	The tabling transformations can now generate significantly sized
	foreign_procs bodies, which the LLDS code generator translates to
	pragma_c instructions. Duplicating these by jump optimization
	may lose more by worsening locality than it gains in avoiding jumps,
	so we add an extra field to pragma_c instructions that tells jumpopt
	not to duplicate code sequences containing such pragma_cs.

compiler/jumpopt.m:
	Respect the new flag on pragma_cs.

compiler/goal_util.m:
	Add a predicate to create foreign_procs with specified contents,
	modelled on the existing predicate to create calls.

	Change the order of the arguments of that existing predicate
	to make it more logical.

compiler/polymorphism.m:
	Conform to the new definition of foreign_procs. Try to simplify
	the mechanism for generating the type_info and typeclass_info
	arguments of foreign_proc goals, but it is not clear that this
	code is even ever executed.

compiler/aditi_builtin_ops.m:
compiler/assertion.m:
compiler/bytecode_gen.m:
compiler/clause_to_proc.m:
compiler/code_gen.m:
compiler/code_info.m:
compiler/code_util.m:
compiler/constraint.m:
compiler/deep_profiling.m:
compiler/deforest.m:
compiler/delay_construct.m:
compiler/dependency_graph.m:
compiler/det_analysis.m:
compiler/det_report.m:
compiler/dnf.m:
compiler/dupelim.m:
compiler/equiv_type_hlds.m:
compiler/exprn_aux.m:
compiler/follow_code.m:
compiler/follow_vars.m:
compiler/frameopt.m:
compiler/goal_form.m:
compiler/goal_path.m:
compiler/higher_order.m:
compiler/higher_order.m:
compiler/hlds_module.m:
compiler/hlds_out.m:
compiler/inlining.m:
compiler/ite_gen.m:
compiler/layout_out.m:
compiler/livemap.m:
compiler/liveness.m:
compiler/llds_out.m:
compiler/loop_inv.m:
compiler/magic.m:
compiler/make_hlds.m:
compiler/mark_static_terms.m:
compiler/middle_rec.m:
compiler/modes.m:
compiler/modules.m:
compiler/opt_debug.m:
compiler/pd_cost.m:
compiler/prog_rep.m:
compiler/purity.m:
compiler/quantification.m:
compiler/reassign.m:
compiler/rl_exprn.m:
compiler/saved_vars.m:
compiler/simplify.m:
compiler/size_prof.m:
compiler/store_alloc.m:
compiler/stratify.m:
compiler/switch_detection.m:
compiler/term_pass1.m:
compiler/term_traversal.m:
compiler/termination.m:
compiler/trace.m:
compiler/typecheck.m:
compiler/unify_proc.m:
compiler/unique_modes.m:
compiler/unneeed_code.m:
compiler/unused_args.m:
compiler/use_local_vars.m:
	Conform to the new definition of foreign_procs, pragma_cs and/or
	table trie steps, or to changed argument orders.

compiler/add_heap_ops.m:
compiler/add_trail_ops.m:
compiler/cse_detection.m:
compiler/dead_proc_elim.m:
compiler/equiv_type.m:
compiler/intermod.m:
compiler/lambda.m:
compiler/lco.m:
compiler/module_util.m:
compiler/opt_util.m:
compiler/stack_opt.m:
compiler/trans_opt.m:
	Conform to the new definition of foreign_procs.

	Bring these modules up to date with our current code style guidelines,
	using predmode declarations, state variable syntax and unification
	expressions as appropriate.

compiler/mercury_compile.m:
	Conform to the changed argument order of a predicate in trans_opt.m.

compiler/options.m:
	Add the --no-tabling-via-extra-args option, but leave the
	documentation commented out since the option is for developers only.

doc/user_guide.texi:
	Document --no-tabling-via-extra-args option, though leave the
	documentation commented out since the option is for developers only.

doc/user_guide.texi:
doc/mdb_categories:
	Document the new goal_paths mdb command.

trace/mercury_trace_internals.c:
	Implement the new goal_paths mdb command.

tests/debugger/completion.exp:
	Conform to the presence of the goal_paths mdb command.

tests/debugger/mdb_command_test.inp:
	Test the existence of documentation for the goal_paths mdb command.

tests/debugger/print_table.{inp,exp*}:
tests/debugger/retry.{inp,exp*}:
	Use the goal_paths command to avoid having the expected output
	depend on the presence or absence of --tabling-via-extra-args.

tests/tabling/table_foreign_output.{m,exp}:
	Add a new test case to test the save/restore of arguments of foreign
	types.

tests/tabling/Mmakefile:
	Enable the new test case.

tests/tabling/test_tabling:
	Make this script more robust.

	Add an option for testing only the standard model forms of tabling.

compiler/delay_construct.m

@@ -159,7 +159,7 @@ delay_construct_in_goal(GoalExpr0 - GoalInfo0, InstMap0, DelayInfo, Goal) :-
 		GoalExpr0 = unify(_, _, _, _, _),
 		Goal = GoalExpr0 - GoalInfo0
 	;
-		GoalExpr0 = foreign_proc(_, _, _, _, _, _, _),
+		GoalExpr0 = foreign_proc(_, _, _, _, _, _),
 		Goal = GoalExpr0 - GoalInfo0
 	;
 		GoalExpr0 = shorthand(_),