Commit Graph

632 Commits

Author SHA1 Message Date
Zoltan Somogyi
03d6c5f436 Get higher insts from types as well as modes.
compiler/modecheck_call.m:
    As above. This fixes Mantis bug #529.

compiler/options.m:
    Allow configure to test whether the bug is fixed, so we can delete
    any now-redundant explicit higher order insts in mode declarations.

tests/hard_coded/exist_cons_ho_arg.{m,exp}:
    A test case for the bug. This is a strengthened version of the
    Mantis test case.

tests/hard_coded/Mmakefile:
    Enable the new test case.
2021-02-24 19:57:37 +11:00
Zoltan Somogyi
63dabcfcf8 Fix filling in partial terms that use direct_arg tags.
This fix uses the approach discussed on m-dev 2020 nov 16/17 for fixing
github issue #72, whose core problem is a need for information flow
back to a the caller from a callee when the callee fills in the
argument of a function symbol whose representation is a direct_arg tag.
In most cases when the callee fills in the value of an argument,
the caller can see it because the argument is in a word on the heap,
but when the function symbol uses a direct_arg tag, that is not the case.

compiler/direct_arg_in_out.m:
    A new module that implements the transformation proposed on m-dev.
    It creates a fresh clone variable every time an argument of a direct_arg
    tag function symbol is (or may be) updated. This can happen several
    times if a type has more than one function symbol with a direct_arg tag.
    Since the affected variable can be bound to only one function symbol
    at the start, its argument can be filled in only once, but the
    compiler cannot know in advance what function symbol the variable
    contains, and therefore which of the possibly several fill-in sites
    (which fill in the arguments of different function symbols) executed
    in sequence will actually update the variable.

    The transformation ensures that once a variable is cloned, it is
    never referred to again. It also ensures that in a branched control
    structure (if-then-else, disjunction or switch), all branches will use
    the *same* variable to represent the latest version of each cloned
    variable at the end, so that following code has a consistent view
    regardless of through which branch execution has reached it.

    There are three situations that the transformation cannot and does not
    handle.

    1. Situations in which the mode of an argument is either an inst variable,
       or an abstract inst. In either case, the pass cannot know whether
       it should apply its transformation to the argument.

    2. Situations where a procedure that has such an argument is
       exported to C code as a function. In that case, the C signature
       of the function we would generate would be different from what
       the user would normally expect. We could modify the documentation
       of the export pragma, but I don't think there much point due to
       lack of demand. (The problem cannot arise when targeting any language
       other than C, because we use direct_arg tags only with the low level
       data representation, which we only use for C.)

    3. Situations where a procedure that has such an argument is defined
       by foreign_proc. Again, dealing with the problem would require
       nontrivial changes to the documented interface between code in
       foreign_procs and the surrounding Mercury code, and I see no demand
       for code that could benefit from that.

    In these cases, this module generates error messages.

compiler/transform_hlds.m:
    Include the new module in the transform_hlds package.

    Delete unnecessary module qualification on some existing inclusions.
    Put some existing inclusions into a more meaningful order.

compiler/notes/compiler_design.html:
    Document the new pass. Fix some nearby prose.

compiler/lambda.m:
compiler/simplify_proc.m:
    Use a predicate exported by direct_arg_in_out.m to test, for each
    procedure, whether the procedure has any argument positions that are
    subject to the problem that direct_arg_in_out.m addresses.
    simplify_proc.m does this for all procedures it processes;
    lambda.m does this for all the procedures it creates from
    lambda expressions.

    Give a predicate in simplify_proc.m a better name.

    Sort a list of predicate names.

compiler/hlds_module.m:
    Add a field to the module_info that simplify_proc.m and lambda.m
    can use to tell direct_arg_in_out.m what work (if any) it needs to do.

compiler/mercury_compile_middle_passes.m:
    Invoke direct_arg_in_out.m if the new field in the HLDS indicates
    that it has some work to do. (In the vast majority of compiler invocations,
    it won't have any.)

compiler/hlds_pred.m:
    The new code in direct_arg_in_out.m creates a clone of each procedure
    affected by the problem, before deleting the originals (to make sure that
    no references to the unfixed versions of now-fixed procedures remain.)
    Make it possible to create exact clones of both predicates and procedures
    by adding two pairs of predicates, {pred,proc}_prepare_to_clone and
    {pred,proc}_create.

    Add the direct_arg_in_out transformation as a possible source
    of transformed predicates.

library/private_builtin.m:
    Add a new builtin operation, partial_inst_copy, that the new module
    generates calls to.

configure.ac:
    Require the installed compiler to recognize partial_inst_copy
    as a no_type_info builtin.

compiler/builtin_ops.m:
    Recognize the new builtin. (This was committed before the rest; the diff
    to private_builtin.m can be done only once the change to builtin_ops.m
    is part of the installed compiler.)

compiler/options.m:
    Add a way to test whether the builtin_ops.m in the installed compiler
    recognizes the new builtin.

compiler/dead_proc_elim.m:
    Do not delete the new primitive before direct_arg_in_out.m has had
    a chance to generate calls to it.

    Add an XXX.

compiler/error_util.m:
    Recognize the new module as a source of error messages.

compiler/pred_table.m:
    Add a pair of utility predicates to be used when looking up
    builtin predicates, for which the compiler writer knows that
    there should be exactly one match. These are used in direct_arg_in_out.m.

compiler/simplify_goal_call.m:
    Replace some existing code with calls to the new predicates
    in pred_table.m.

compiler/hlds_goal.m:
    Add modes to rename_vars_in_goal_expr that express the fact
    that when an atomic goal_expr has some variables renamed inside it,
    it does not suddenly become some *other* kind of goal_expr.
    New code in direct_arg_in_out.m relies on this.

compiler/hlds_out_goal.m:
    When the HLDS we are dumping out is malformed because it contains
    calls to predicates that have been deleted, the compiler used to abort
    at such calls. (I ran into this while debugging direct_arg_in_out.m.)

    Fix this. When such calls are encountered, we now print out as much
    information we can about the call, and prefix the call with an
    unmistakable prefix to draw attention to the problem.

compiler/inst_util.m:
    Fix a bug that prevented direct_arg_in_out.m from even being invoked
    on some test code for it.

    The bug was in code that we use to unify a headvar's initial inst
    with its final inst. When the initial inst was a non-ground bound_inst
    such as the ones used in tests/hard_coded/gh72.m, and the final inst
    was simply "ground", this code quite properly returned a bound_inst
    (which, unlike ground, can show the exact set of function symbols
    that the headvar could be bound to). The problem was that it
    reused the original bound_inst's test results, including the one
    that said the final inst is NOT ground, which of course is wrong
    for any inst unified with ground. Fix two instances of this bug.

compiler/modes.m:
    Make some of the code I had to traverse to find the bug in inst_util.m
    easier to read and understand.

    Replace some uses of booleans with bespoke enum types.

    Change the argument lists of some predicates to put related arguments
    next to each other.

    Give some variables more descriptive names.

compiler/layout_out.m:
    Conform to the change in hlds_pred.m.

compiler/var_locn.m:
    Fix a code generation bug. When filling-in the value of the argument
    of a function symbol represented by a direct_arg tag, the code we
    generated for it worked only if the direct_arg tag used 0
    as its ptag value. In the test cases we initially used for
    github issue 72, that was the case, but the new tests/hard_coded/gh72.m
    has direct_tag args that use other ptag values as well.

    Document the reason why the updated code works.

compiler/term_constr_initial.m:
    Add the new primitive predicate added to private_builtin.m,
    partial_inst_copy, to a table of builtins that do not take type_infos,
    even though their signatures contain type variables.

    Fix a bunch of old bugs: most other such primitives were not listed
    either.

mdbcomp/program_representation.m:
    Add partial_inst_copy to the master list of builtins that do not take
    type_infos even though their signatures contain type variables. (Done
    by an earlier commit.)

    Document the fact that any updates here require updates to
    term_constr_initial.m.

library/multi_map.m:
    We have long had multi_map.add and multi_map.set as synonyms,
    but we only had multi_map.reverse_set. Add multi_map.reverse_add
    as a synonym for it.

    Define the "set" versions in terms of the "add" versions,
    instead of vice versa.

NEWS:
    Document the new predicates in multi_map.m.

tests/hard_coded/gh72a.m:
    Fix typo.

tests/hard_coded/gh72.{m,exp}:
    A new, much more comprehensive test case than gh72a.m.
    This one tries to tickle github issue 72 in as many forms of code
    as I can think of.

tests/invalid/gh72_errors.{m,err_exp}:
    A test case for testing the generation of error messages for
    two out of the three kinds of situations that direct_arg_in_out.m
    cannot handle. (Proposals for how to test the third category welcome.)

tests/hard_coded/Mmakefile:
tests/invalid/Mmakefile:
    Enable the two new test cases, as well as two old ones, gh72[ab].m,
    that previously we didn't pass.

tests/invalid/Mercury.option:
    Do not compile gh72_error.m with --errorcheck-only, since its errors
    are reported by a pass that --errorcheck-only does not invoke.
2021-01-13 05:35:40 +11:00
Julien Fischer
f9f2e5e112 Enable a test case in the C# and Java grades.
test/hard_coded/Mmakefile:
    Run the pragma_export test in C# and Java grades.

tests/hard_coded/pragma_export.m:
    Add the foreign_proc and foreign_export pragmas required for
    this to work in the Java grade.
2021-01-07 14:08:04 +11:00
Julien Fischer
52b31f5089 Add uint64 to string conversion for bases 8 and 16.
library/string.m:
     Add functions for converting uint64s to strings of base 8 or base 16
     digits. For most integer types we can cast to a uint and then use the
     uint versions of these operations but for 64-bit types we cannot since
     on some of our supported platforms uints are 32-bit.

NEWS:
     Announce the additions.

tests/hard_coded/Mmakefile:
tests/hard_coded/uint64_string_conv.{m,exp}:
     Add a test of the new functions.
2020-12-15 22:45:31 +11:00
Julien Fischer
f8e65add3a Format uints directly.
Currently, the Mercury implementation of string formatting handles uints by
casting them to ints and then using the code for formatting signed integers as
unsigned values.  Add an implementation that works directly on uints and make
the code that formats signed integers as unsigned integers use that instead.
The new implementation is simpler and avoids unnecessary conversions to
arbitrary precision integers.

Add new functions for converting uint values directly to octal and hexadecimal
strings that use functionality provided by the underlying platforms; replace
the Mercury code that previously did that with calls to these new functions.

library/string.m:
    Add the functions uint_to_hex_string/1, uint_to_uc_hex_string/1 and
    uint_to_octal_string/1.

library/string.format.m:
    Make format_uint/6 operate directly on uints instead of casting the value
    to a signed int and calling format_unsigned_int/6.

    Make format_unsigned_int/6 cast the int value to a uint and then call
    format_uint/6.

    Delete predicates and functions used to convert ints to octal and
    hexadecimal strings.  We now just use the functions exported by
    the string module.

NEWS:
    Announce the additions to the string module.

tests/hard_coded/Mmakefile:
tests/hard_coded/uint_string_conv.{m,exp*}:
     Add a test of uint string conversion.
2020-11-20 23:07:52 +11:00
Peter Wang
88047bbb45 Delete Erlang from tests.
tests/general/float_test.exp3:
tests/general/float_test.m:
tests/general/read_dir_regression.exp4:
tests/general/read_dir_regression.m:
tests/hard_coded/remove_file.exp2:
tests/hard_coded/remove_file.m:
    Delete Erlang backend specific expected outputs.

tests/hard_coded/Mmakefile:
tests/hard_coded/erlang_deconstruct.exp:
tests/hard_coded/erlang_deconstruct.m:
tests/hard_coded/existential_list.exp:
tests/hard_coded/existential_list.m:
tests/valid/Mmakefile:
tests/valid/erl_ite_vars.m:
tests/valid/zf_erlang_bug.m:
    Delete erlang target specific tests.

tests/*:
    Delete Erlang foreign procs and foreign types.
2020-10-27 11:10:11 +11:00
Peter Wang
524f4d72e2 Delete references to Erlang backend in makefiles.
Mmake.workspace:
Mmakefile:
*/Mmakefile:
tests/*/Mmakefile:
tests/valid/Mmake.valid.common:
trace/Mmakefile:
    As above.
2020-10-27 11:10:11 +11:00
Julien Fischer
3d3378881b Do not run hard_coded/shift_test in deep profiling grades.
tests/hard_coded/Mmakefile:
     As above, the test catches exceptions and that isn't (yet) supported in
     deep profiling grades.
2020-08-24 00:30:58 +10:00
Zoltan Somogyi
dc47131aea Put foreign_codes before foreign_procs for LLDS.
This fixes a bug reported by Keri Harris on m-users.

compiler/llds_out_file.m:
    Output foreign_codes before the C modules that may contain
    foreign_procs, since those foreign_procs may need to refer
    to entities defined in the foreign_codes.

compiler/mercury_compile_llds_back_end.m:
    Improve some comments.

tests/hard_coded/foreign_code_before_proc.{m,exp}:
    A regression test for this bug.

tests/hard_coded/Mmakefile:
    Enable the regression test.
2020-07-26 21:39:11 +10:00
Zoltan Somogyi
4987544ab4 Add test case for newly reported bug. 2020-07-15 22:34:30 +10:00
Zoltan Somogyi
1f45f91886 Make "mmake runtests" work again.
My commit afe2887882 broke the ability
to run the test suite outside of a bootcheck by executing "mmake runtests"
in the tests directory. This diff fixes that.

tests/Mmake.common:
    Don't define "TESTS_DIR = ..". While every single tests/*/Mmakefile
    defined it as such, I overlooked the fact that tests/Mmakefile itself
    defined it ".", referring to the same directory from a different starting
    point. Document this easily-overlooked fact.

    Rename the old runtests target, which after afe2887 runs the tests
    in a single directory, as runtests_dir, to leave the target name
    "runtests" itself free for tests/Mmakefile to use.

tests/Mmakefile:
    Define "TESTS_DIR = .", and add a target "runtests" which invokes
    "mmake runtests_dir" in each test directory.

tools/bootcheck:
    Invoke "mmake runtests_dir" instead of "mmake runtests" in each
    test directory.

    Initialize a variable just before it is used.

tests/*/Mmakefile:
    Add back the definition "TESTS_DIR = .."
2020-06-10 01:05:15 +10:00
Zoltan Somogyi
1e64d0ed0e Declare private_builtin.unsigned_{lt,le}.
library/private_builtin.m:
    As above. The code recognizing them as builtins was added a few weeks ago.

configure.ac:
    Require the installed compiled to contain that code.

compiler/options.m:
    Provide a way to test whether the compiler has *this* diff.

library/int.m:
    Delete int.unsigned_lt, and use private_builtin.unsigned_lt instead.

tests/hard_coded/unsigned_lt_le.{m,exp}:
    A test case for the two operations.

tests/hard_coded/Mmakefile:
    Enable the new test case.
2020-05-25 16:50:46 +10:00
Zoltan Somogyi
afe2887882 Remove stale references to test subdirs.
A long time ago, test directories such as hard_coded had subdirectories
such as hard_coded/typeclasses. These have since been flattened out
(e.g. hard_coded/typeclasses is now just typeclasses), but there were
still remnants of the old approach. This diff deletes those remnants.

tests/*/Mmakefile:
    Delete the TESTS_DIR and the SUBDIRS mmake variables; TESTS_DIR
    was always set to "..", and SUBDIRS to the empty string.

    Delete any references to the make variable NOT_WORKING, since
    it is never used.

tests/Mmake.common:
    Document that Mmakefiles in test directories don't have to set
    TESTS_DIR and SUBDIRS anymore. Fix the formatting of the documentation
    of the make variables they do still have to set.

    Delete the targets and actions for handling subdirectories of
    test directories, since there aren't any.

tests/Mmakefile:
    Simplify some code.
2020-04-14 11:23:12 +10:00
Zoltan Somogyi
578cb606b4 Test one_or_more.m further. 2020-03-10 02:41:04 +11:00
Zoltan Somogyi
36c2000516 Add the one_or_more and one_or_more_map modules to the library.
library/one_or_more.m:
    We used to have a type named one_or_more in the list module representing
    nonempty lists. It had literally just two predicates and two functions
    defined on it, three of which did conversions to and from lists, which
    limited their usefulness.

    This new module is the new home of the one_or_more type, together with
    a vastly expanded set of utility predicates and functions. Specifically,
    it implements every operation in list.m which makes sense for nonempty
    lists.

library/list.m:
    Delete the code moved over to one_or_more.m.

library/one_or_more_map.m:
    This new module is a near copy of multi_map.m, with the difference being
    that while the multi_map type defined in multi_map.m maps each key
    to a list(V) of values (a list that happens to always be nonempty),
    the one_or_more_map type defined in one_or_more_map.m maps each key
    to a one_or_more(V) of values (which enforces the presence of at least
    one value for each key in the type).

library/map.m:
    Mention the existence of one_or_more_map.m as well as multi_map.m.

library/MODULES_DOC:
library/library.m:
    List the new modules as belonging to the standard library.

NEWS:
    Mention the new modules, and the non-backwards-compatible changes to
    list.m.

compiler/*.m:
    Import the one_or_more module when needed.

tests/hard_coded/test_one_or_more_chunk.{m,exp}:
    Test the one predicate in one_or_more.m that is non-trivially different
    from the corresponding predicate in list.m: the chunk predicate.

tests/hard_coded/Mmakefile:
    Enable the new test case.
2020-02-28 14:29:05 +11:00
Peter Wang
36e1e26b33 Add test for thread.closeable_channel.
tests/hard_coded/Mmakefile:
tests/hard_coded/closeable_channel_test.exp:
tests/hard_coded/closeable_channel_test.exp2:
tests/hard_coded/closeable_channel_test.m:
    Add new test.
2020-02-18 11:07:34 +11:00
Peter Wang
0f9e0fe873 Skip string_append_pieces test in profdeep grades.
tests/hard_coded/Mmakefile:
    Skip string_append_pieces in profdeep grades as it needs to
    catch exceptions.
2019-11-19 14:24:57 +11:00
Peter Wang
f71b5f20ed Define behaviour of string.set_char etc on ill-formed sequences.
library/string.m:
    Define behaviour of set_char, det_set_char and unsafe_set_char on
    ill-formed sequences. Also define them to throw an exception on an
    attempt to set a null character or surrogate code point in a UTF-8
    string.

    Delete claim that unsafe_set_char is constant time. That would only
    be true for the destructive mode of unsafe_set_char, and that mode
    has been disabled for a long time.

    Implement the defined behaviour for C and C# versions of
    unsafe_set_char. The Java version already behaved as defined.

    Use unsafe_set_char to implement set_char instead of duplicating
    foreign code.

    Replace a couple of uses of strcpy with MR_memcpy as it was
    convenient to do so. (On OpenBSD, the linker issues a warning
    whenever strcpy is used. Avoiding the warning is not high priority
    but we might still like to eliminate all uses of strcpy eventually.)

tests/hard_coded/Mmakefile:
tests/hard_coded/string_set_char_ilseq.exp:
tests/hard_coded/string_set_char_ilseq.exp2:
tests/hard_coded/string_set_char_ilseq.m:
    Add test case.
2019-11-12 17:16:34 +11:00
Peter Wang
f2e0998651 Add string.append_string_pieces.
library/string.m:
    Add append_string_pieces/2 predicate.

library/io.m:
    Add a comment about a potential future change.

tests/hard_coded/Mmakefile:
tests/hard_coded/string_append_pieces.exp:
tests/hard_coded/string_append_pieces.m:
    Add test case.

NEWS:
    Announce addition.
2019-11-08 14:23:06 +11:00
Peter Wang
adbf4c51c8 Tighten up string.from_code_unit_list et al.
library/string.m:
    Document that from_code_unit_list fails if the result string would
    contain a null character, and enforce that in the Java and C#
    implementations. It was already enforced in the C implementation.

    Make from_code_unit_list fail if the code unit list contains an
    invalid value (negative or >0xff or >0xffff).

    Document that from_utf{8,16}_code_unit_list fails if the result
    string would contain a null character.

    Make from_utf8_code_unit_list call semidet_from_rev_char_list rather
    than from_rev_char_list so that it fails as documented instead of
    throwing an exception if the code unit list correctly encodes a list
    of code points, but the code points cannot be encoded into a string.

    Similarly for from_utf16_code_unit_list.

tests/hard_coded/Mmakefile:
tests/hard_coded/string_from_code_unit_list.exp:
tests/hard_coded/string_from_code_unit_list.exp2:
tests/hard_coded/string_from_code_unit_list.m:
    Add test case.
2019-11-06 13:43:54 +11:00
Peter Wang
389d973a1b Skip two tests in profdeep grades.
tests/hard_coded/Mmakefile:
    Skip two tests in profdeep grades that need to catch exceptions.
2019-11-06 13:35:27 +11:00
Peter Wang
658c8a5ad5 Define behaviour of string.char_to_string on edge cases.
library/string.m:
    Define behaviour of char_to_string when the string is not
    well-formed or if the char is a surrogate code point.

    Implement char_to_string/2 using multiple clauses
    as the described behaviour doesn't match to_char_list/2.

tests/hard_coded/Mmakefile:
tests/hard_coded/char_to_string.exp:
tests/hard_coded/char_to_string.exp2:
tests/hard_coded/char_to_string.m:
    Add test case.
2019-10-30 11:21:02 +11:00
Peter Wang
56687d235e Define behaviour of string.first_char/3 on edge cases.
library/string.m:
    Define first_char/3 to fail if the input string begins with an
    ill-formed code unit sequence.

    Define the reverse mode to throw an exception on an attempt to
    encode a null character or surrogate code point in the output
    string.

    Reimplement first_char/3 in Mercury.

hard_coded/Mmakefile:
hard_coded/string_first_char_ilseq.exp:
hard_coded/string_first_char_ilseq.m:
    Add test case.
2019-10-30 11:21:02 +11:00
Peter Wang
025bee0549 Check for surrogates when converting list of char to string.
library/string.m:
    Make from_char_list, from_rev_char_list, to_char_list throw an
    exception if the list of chars includes a surrogate code point that
    cannot be encoded in a UTF-8 string.

    Make semidet_from_char_list, semidet_from_rev_char_list,
    to_char_list fail if the list of chars includes a surrogate code
    point that cannot be encoded in a UTF-8 string.

runtime/mercury_string.h:
    Document return value of MR_utf8_width.

tests/hard_coded/Mmakefile:
tests/hard_coded/string_from_char_list_ilseq.exp:
tests/hard_coded/string_from_char_list_ilseq.exp2:
tests/hard_coded/string_from_char_list_ilseq.m:
    Add test case.

tests/hard_coded/null_char.exp:
    Expect new message in exceptions thrown by from_char_list,
    from_rev_char_list.

tests/hard_coded/string_hash.m:
    Don't generate surrogate code points in random strings.
2019-10-30 11:21:02 +11:00
Peter Wang
cd899271c6 Make string.append(out, out, in) work with ill-formed sequences.
library/string.m:
    Simplify string.append(out, out, in) and make it work sensibly in
    the presence of ill-formed code unit sequences, breaking the input
    string after each code point or code unit in an ill-formed sequence.

tests/hard_coded/Mmakefile:
tests/hard_coded/string_append_ooi_ilseq.exp:
tests/hard_coded/string_append_ooi_ilseq.exp2:
tests/hard_coded/string_append_ooi_ilseq.m:
    Add test case.
2019-10-24 12:31:29 +11:00
Peter Wang
bf1f624632 Add string.compare_substrings and unsafe_compare_substrings.
library/string.m:
    Add the new predicates.

tests/hard_coded/Mmakefile:
tests/hard_coded/string_compare_substrings.exp:
tests/hard_coded/string_compare_substrings.m:
    Add test case.

NEWS:
    Announce additions.
2019-10-24 12:31:29 +11:00
Peter Wang
7350e7f0b6 Define behaviour of string.codepoint_offset on ill-formed sequences.
library/string.m:
    Define how string.codepoint_offset counts code units in ill-formed
    sequences.

    Delete C and C# foreign implementations in favour of the Mercury
    implementation that has the intended behaviour.
    (The Java implementation uses String.offsetByCodePoints which
    also matches our intended behaviour.)

tests/hard_coded/Mmakefile:
tests/hard_coded/string_codepoint_offset_ilseq.exp2:
tests/hard_coded/string_codepoint_offset_ilseq.m:
    Add test case.
2019-10-24 09:22:13 +11:00
Peter Wang
edfbeb1d9a Define behaviour of string.foldl etc on ill-formed sequences.
library/string.m:
    As above.

tests/hard_coded/Mmakefile:
tests/hard_coded/string_fold_ilseq.exp:
tests/hard_coded/string_fold_ilseq.exp2:
tests/hard_coded/string_fold_ilseq.m:
    Add test case.
2019-10-24 09:22:13 +11:00
Peter Wang
250b5bcc2e Define behaviour of string.count_codepoints with ill-formed sequences.
library/string.m:
    Make each code unit in an ill-formed sequence contribute one
    to the value of string.count_codepoints.

tests/hard_coded/Mmakefile:
tests/hard_coded/string_count_codepoints_ilseq.exp:
tests/hard_coded/string_count_codepoints_ilseq.exp2:
tests/hard_coded/string_count_codepoints_ilseq.m:
    Add test case.
2019-10-24 09:14:46 +11:00
Peter Wang
9b25e167e1 Define behaviour of string.to_char_list (and rev) on ill-formed sequences.
library/string.m:
    Define string.to_char_list and string.to_rev_char_list to either
    replace code units in ill-formed sequences with U+FFFD or return
    unpaired surrogate code points.

    Use Mercury version of do_to_char_list instead of updating
    the foreign language implementations.

tests/hard_coded/Mmakefile:
tests/hard_coded/string_char_list_ilseq.exp:
tests/hard_coded/string_char_list_ilseq.exp2:
tests/hard_coded/string_char_list_ilseq.m:
    Add test case.
2019-10-24 09:14:46 +11:00
Peter Wang
0c9bdf2587 Define behaviour of string.prev_index on ill-formed sequences.
library/string.m:
    Make string.prev_index and string.unsafe_prev_index
    return either U+FFFD or an unpaired surrogate code point
    when an ill-formed code unit sequence is detected.

tests/hard_coded/Mmakefile:
tests/hard_coded/string_prev_index_ilseq.exp:
tests/hard_coded/string_prev_index_ilseq.exp2:
tests/hard_coded/string_prev_index_ilseq.m:
    Add test case.
2019-10-24 09:14:46 +11:00
Peter Wang
d055627fd2 Define behaviour of string.index_next on ill-formed sequences.
library/string.m:
    Make string.index_next and string.unsafe_index_next
    return either U+FFFD or an unpaired surrogate code point
    when an ill-formed code unit sequence is detected.

tests/hard_coded/Mmakefile:
tests/hard_coded/string_index_next_ilseq.exp:
tests/hard_coded/string_index_next_ilseq.exp2:
tests/hard_coded/string_index_next_ilseq.m:
    Add test case.
2019-10-24 09:14:46 +11:00
Peter Wang
47d0f70ea4 Define behaviour of string.index on ill-formed sequences.
library/string.m:
    Make string.index/3 and string.unsafe_index/3
    return either U+FFFD or an unpaired surrogate code point
    when an ill-formed code unit sequence is detected.

tests/hard_coded/Mmakefile:
tests/hard_coded/string_index_ilseq.exp:
tests/hard_coded/string_index_ilseq.exp2:
tests/hard_coded/string_index_ilseq.m:
    Add test case.
2019-10-24 09:14:46 +11:00
Zoltan Somogyi
17b1b0725d Optimize the tests for builtin types when printing terms.
library/stream.string_writer.m:
    When printing terms to a stream, we special case the handling
    of a whole bunch of builtin types. We used to test for these one by one,
    in sequence, which can be very slow.

    Switch to an approach where we first get the identity of the term's type,
    and we test for a type for one of these types *if, and only if*,
    the identity says that it is one of the types we want to special-case
    in one of the modules that has to-be-special-cased types.

    This code is significantly longer than the original, but
    (a) it should be significantly faster (it speeds up a microbenchmark
    by a bit more than a third), and (b) it should not slow down
    if we want to special-case the treatment of more types in the future.

tests/hard_coded/stream_string_writer_types.{m,exp}:
    A test to ensure that the assumptions that the new code in
    stream.string_writer.m makes about which builtin type is defined where
    are still valid.

tests/hard_coded/Mmakefile:
    Enable the new test case.
2019-10-17 17:56:14 +11:00
Julien Fischer
63648fc73b Add a regression test for the bug fix in commit 3e3dbab.
tests/hard_coded/Mmakefile:
tests/hard_coded/read_bitmap_size.{m,exp}:
   As above.
2019-09-20 16:51:00 +10:00
Peter Wang
ee001d38b4 Add string.is_well_formed predicate.
library/string.m:
    Add predicate to test if a string is in UTF-8 or UTF-16,
    depending on the target language.

NEWS:
    Announce the addition.

tests/hard_coded/string_well_formed.exp:
tests/hard_coded/string_well_formed.m:
    Add basic test case.

tests/hard_coded/string_well_formed_utf8.exp:
tests/hard_coded/string_well_formed_utf8.exp2:
tests/hard_coded/string_well_formed_utf8.exp3:
tests/hard_coded/string_well_formed_utf8.inp:
tests/hard_coded/string_well_formed_utf8.m:
    Add more thorough test for UTF-8. The input file is from
    https://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-test.txt

tests/hard_coded/Mmakefile:
    Enable the tests.
2019-09-13 15:51:02 +10:00
Mark Brown
3a83702562 Changes per the discussion on the users list.
library/random.m:
	Add predicates shuffle_list/{4,5} and shuffle_array/{4,5}.

library/random.m:
library/random.sfc*.m:
	Improve comments.

tests/hard_coded/Mmakefile:
tests/hard_coded/random_shuffle*.{m,exp}:
	Test the new predicates.
2019-09-06 15:51:02 +10:00
Julien Fischer
3a8bca2a8f Re-enable random module tests in the C# grade.
tests/hard_coded/Mmakefile:
    As above.  (The underlying problem was fixed in commit 65f683e.)
2019-09-05 16:52:02 +10:00
Julien Fischer
476c28933e Add a test case for a recent bug fix.
Add a regression test for the bug fixed in commit 65f683e.  There were multiple
places that had the same underlying issue; the new test case covers those as
well.

tests/hard_coded/Mmakefile:
tests/hard_coded/array_primitives.{m,exp}:
    Add the new test case.
2019-09-02 17:02:59 +10:00
Mark Brown
97ad133625 Disable tests of random module in C# and Erlang grades. 2019-08-28 00:52:17 +10:00
Mark Brown
c8780cdfa7 Address more of Julien's review comments.
library/random.m:
    Move the new code to the existing random module. Update names and
    typeclass methods.

    Add adaptors to attach any ground or shared generator to the I/O
    state.

library/random.sfc{16,32,64}.m:
    Move sfc generators to here. Update for changes to interface.

extras/README:
extras/random/*.m:
    Move unused generators to a new directory under extras.

library/uint32.m:
    Add cast_from_uint/1.

library/MODULES_DOC:
library/library.m:
    Update for the module changes.

tests/hard_coded/*:
    Rename test cases to correspond with the library module name.

    Test the I/O adaptor.
2019-08-28 00:52:17 +10:00
Mark Brown
7721516520 Implement SFC random number generators, in various sizes
library/rng.sfc.m:
	New module that implements the above.

library/MODULES_DOC:
library/library.m:
library/rng.m:
	Add the new library module.

library/uint16.m:
	Add functions to cast to/from uint64.

tests/hard_coded/Mmakefile:
tests/hard_coded/*.*:
	Update/extend tests.
2019-08-28 00:52:17 +10:00
Mark Brown
1d0bde74f4 Some RNGs for the standard library, plus typeclass interfaces.
library/rng.m:
    Top-level module containing typeclasses and generic routines,
    as well as overall documentation.

library/rng.binfile.m:
    A generator that reads from a file.

library/rng.marsaglia.m:
    Fast, simple generator. Diehard results for default seed:
        PASSED: 109
        WEAK:   2
        FAILED: 3

library/rng.tausworthe.m:
    Combined Tausworthe generators. Diehard results for default seed:

    generator T3
        PASSED: 113
        WEAK:   1
        FAILED: 0

    generator T4
        PASSED: 112
        WEAK:   2
        FAILED: 0

library/library.m:
library/MODULES_DOC:
    Add the new modules to the public interface.

library/float.m:
    Add functions to convert from (u)int32/64 to float.

library/uint32.m:
    Add function to convert from uint64 to uint32.

tests/hard_coded/Mmakefile:
tests/hard_coded/rng1.{m,exp}:
tests/hard_coded/rng2.{m,exp}:
    Test the generators. The expected outputs are derived from
    the output of the C reference implementations.
2019-08-28 00:52:17 +10:00
Zoltan Somogyi
4b147be3f2 Add test cases for Github issue 72.
Disable the test_generic_ref test case.
2019-08-27 11:26:07 +10:00
Zoltan Somogyi
4b2a4a88b5 Fix handling "int used, imp imported" modules when making interface files.
This fixes the second incarnation of Mantis bug #401. (The first incarnation
was the handling of such modules when generating target code.)

compiler/grab_modules.m:
    Compute the set of modules that have a use_module declaration in the
    interface section and an import_module declaration in the implementation
    section, and set the status of the items imported from that module
    accordingly.

compiler/module_qual.id_set.m:
    If module A defines an entity (such as a type) that module B refers to
    without the required qualification (which can happen if B has a use_module,
    not import_module for A), then do NOT report A as unused in B; it IS used,
    just not properly.

tests/hard_coded/int_impl_imports.exp:
tests/hard_coded/int_impl_imports.m:
tests/hard_coded/int_impl_imports_2.m:
    A new test case to see whether we can compile a module that has
    "int used, imp imported" references to another module, and has
    unqualified references to an imported entity in the implementation.

tests/invalid/int_impl_imports.err_exp:
tests/invalid/int_impl_imports.m:
tests/invalid/int_impl_imports_2.m:
    A new test case to see whether we can generate the right error message
    for a module that has "int used, imp imported" references to another
    module, and has an unqualified references to an imported entity
    in the interface.

tests/hard_coded/Mmakefile:
tests/invalid/Mmakefile:
    Enable the two new test cases.

tests/valid_seq/int_impl_imports.m:
tests/valid_seq/int_impl_imports_2.m:
    Add vim mode lines.
2019-08-02 10:36:59 +02:00
Zoltan Somogyi
f79b951baa Compute change hunks from diff sequences.
library/edit_seq.m:
    Extend this module with code to compute change hunk sequences
    from diff sequences. Each change hunk contains one or more
    insertions and/or deletions surrounded by a few lines of context.

compiler/style_checks.m:
    Print any differences between declaration and definition order
    using change hunks. This omits the boring parts of the diff.

tests/hard_coded/change_hunk_test.{m,exp}:
    A new test case for the new functionality in edit_seq.m.

tests/hard_coded/Mmakefile:
    Enable the new test case.

tests/hard_coded/edit_seq_test.m:
    Fix the copyright year. (The file was written in 2018, but committed
    in 2019.)

tests/warnings/inconsistent_pred_order.exp:
    Expect the updated output of style_checks.m.
2019-06-28 15:59:40 +02:00
Zoltan Somogyi
1afe6d2594 Add map.select_unselect and map.select_unselect_sorted_list.
library/map.m:
    As above.

NEWS:
    Announce the new predicates.

tests/hard_coded/map_select_test.{m,err_exp}:
    A test case for the new code.

tests/hard_coded/Mmakefile:
    Enable the new test case.
2019-06-05 17:25:49 +02:00
Julien Fischer
3b7289d04b Improve deconstruction (and printing) of foreign types in C# and Java grades.
Currently, we return a placeholder string ("foreignxx") if we attempt to look
up the functor of a foreign type in the C# or Java grades.  For C, we return a
string of the form:

   <<foreign(Name, Rep)>>

where Name is the type's Mercury name and Rep is its "value" as results from
casting it to (void *) and then using sprintf's %p conversion specifier.  Do
something similar for both the C# and Java grades, except that for Rep we
return the output of the calling the toString() (ToString()) method.  If the
object in question is a null reference then we return "null" for Rep.

Document how the predicates in the deconstruct module handle foreign types.

library/rtti_implementation.m:
   Return more information about foreign type functors.

library/deconstruct.m:
   Make the above changes to the documentation.

tests/hard_coded/csharp_print_foreign.{m,exp}:
tests/hard_coded/java_print_foreign.{m,exp}:
   Tests of the new functionality in the C# and Java grades.

tests/hard_coded/Mmakefile:
    Add the new tests.

    Add a new category of test programs that are only run in the C# grades.
2019-03-13 05:57:46 +00:00
Julien Fischer
ae90cecc5c Reading multibyte integers from binary file streams.
Add predicates for reading multibyte integers from binary file streams.  As
with the corresponding predicates for writing multibyte integers to binary file
streams, we provide versions for big-endian, little-endian and native byte
orders.

library/io.m:
    Add a new type that represents possibly incomplete results when reading
    from binary file streams.

    Add the new predicates.

tests/hard_coded/Mmakefile:
tests/hard_coded/read_binary_{int,uint}{16,32,64}.{m,exp}:
    Add tests of the new predicates.
2019-01-18 08:38:52 +00:00
Zoltan Somogyi
4aed1a57e4 Print an arg type list diff for arg lists with wrong arity.
When a user writes a clause for a predicate (or function) that does not exist
with that arity, but does exist with one or more other arities, report not
just the list of the other arity/arities, but, for each such other arity,
a diff between the declared arg types and the inferred arg types.

After this diff, we generate output like this:

bad_pred_arity.m:027: Error: clause for predicate `bad_pred_arity.p'/4
bad_pred_arity.m:027:   without corresponding `:- pred' declaration.
bad_pred_arity.m:027:   However, predicates of that name do exist with arities
bad_pred_arity.m:027:   3 and 5.
bad_pred_arity.m:027: Inferred :- pred p(int, string, int, string).
bad_pred_arity.m:027:   The argument list difference from the arity 3 version
bad_pred_arity.m:027:   is
bad_pred_arity.m:027:     pred(
bad_pred_arity.m:027:         int,
bad_pred_arity.m:027:   +     string,
bad_pred_arity.m:027:         int,
bad_pred_arity.m:027:         string
bad_pred_arity.m:027:     )
bad_pred_arity.m:027:   The argument list difference from the arity 5 version
bad_pred_arity.m:027:   is
bad_pred_arity.m:027:     pred(
bad_pred_arity.m:027:         int,
bad_pred_arity.m:027:   -     float,
bad_pred_arity.m:027:         string,
bad_pred_arity.m:027:         int,
bad_pred_arity.m:027:         string
bad_pred_arity.m:027:     )

compiler/typecheck_errors.m:
    Generate the diff part of the message above.

compiler/typecheck.m:
    Invoke typecheck_errors.m when relevant.

compiler/error_util.m:
    When comparing two error_specs, switch from a two-level comparison
    (first the contexts of error_msgs, then everything else) to three levels
    first the contexts of error_msgs, then their error_msg_components,
    then everything else). This is needed to allow the error message from
    make_hlds_error.m (which reports the error and mentions the arities
    with which the named predicate or function does exist) come out before
    the informational message from typecheck.m that prints the inferred
    arg types and their differences from the other arities. (With the old
    comparison, the difference in severity would trump the invisible order
    components that this diff includes in both specs to force the desire
    order.)

    Base the code comparing error_specs on the code for comparing error_msgs.
    Move the two previously separate pieces code for those tasks next to each
    other.

compiler/make_hlds_error.m:
    Add the invisble ordering component.

    When we see clauses with two or more wrong arities for a given predicate
    or function, don't list the automatically created pred declaration
    for an *earlier* wrong-arity clause as a real declaration whose arity
    is to be listed in the error messages we generate for *later* wrong-arity
    clauses.

    Add some documentation.

compiler/add_pred.m:
    Factor out some common code.

library/edit_seq.m:
    A new module for computing diffs.

library/library.m:
library/MODULES_DOC:
    Add the new module to the standard library.

tests/hard_coded/edit_seq_test.{m,exp}:
    A new test case for the diff algorithm.

tests/invalid/bad_pred_arity.{m,err_exp}:
    A new test case for the new error message.

tests/hard_coded/Mmakefile:
tests/invalid/Mmakefile:
    Enable the new test cases.

tests/invalid/bigtest.err_exp:
tests/invalid/bug197.err_exp:
tests/invalid/bug278.err_exp:
tests/invalid/errors2.err_exp:
tests/invalid/invalid_binary_literal.err_exp:
tests/invalid/invalid_float_literal.err_exp:
tests/invalid/invalid_hex_literal.err_exp:
tests/invalid/invalid_main.err_exp:
tests/invalid/invalid_octal_literal.err_exp:
tests/invalid/multimode_dcg.err_exp:
tests/invalid/multisoln_func.err_exp:
tests/invalid/null_char.err_exp:
tests/invalid/state_vars_test3.err_exp:
tests/invalid/try_detism.err_exp2:
tests/invalid/typeclass_test_5.err_exp:
tests/invalid/typeclass_test_8.err_exp:
tests/invalid/unsatisfiable_constraint.err_exp:
tests/invalid_purity/impure_func_t3.err_exp:
    Update these files to expect error messages in the new order.

samples/diff/*.m:
    Fix comments, mostly by moving them to where our programming style
    wants them.
2019-01-03 08:57:20 +11:00