Previously, we created _detached_ threads to run work-stealing engines.
The only reason for using detached threads instead of joinable threads
was because the code for thread creation was originally designed for
creating Mercury threads (the interface exported by the thread.m module
expects detached threads).
When the program is about to end, the main thread notifies the engines
to shut down, then waits on a semaphore that is incremented when an
engine is shut down. But an engine can only increment the semaphore
BEFORE its thread terminates. That is, while the semaphore indicates
that the engine has shut down (no longer responding), the thread that
it was running on may continue for an indeterminate amount of time
before it is terminated. The main thread may think that it is safe
to proceed, even while some of the engine threads are still running.
I found that that on a Linux/glibc system, with a statically linked
binary, this setup could sometimes cause an "Aborted" error message
at program exit (after Mercury main/2). From backtraces, I believe the
problem is as described: the main thread is already in a exit() call
while engine threads are still performing their own cleanup, leading to
an abort() call.
The solution is to do what we should have done to begin with: run
work-stealing engines in non-detached threads, and call pthread_join()
to wait for engine threads to terminate before allowing the main thread
to continue with program termination.
runtime/mercury_context.c:
Delete references to shutdown_ws_semaphore.
runtime/mercury_thread.c:
runtime/mercury_thread.h:
Make MR_create_worksteal_thread create a non-detached thread.
runtime/mercury_wrapper.c:
In mercury_runtime_init, record the IDs of the threads created for
running work-stealing engines in an array.
In mercury_runtime_terminate, after notifying each work-stealing
engine to shut down, wait for the engine threads to terminate
by calling pthread_join().
Sample backtrace:
Thread 1 (Thread 0x7f6dcafb46c0 (LWP 11122) (Exiting)):
#0 0x000000000093c40c in pthread_kill ()
#1 0x00000000009219ce in raise ()
#2 0x00000000004013b2 in abort ()
#3 0x0000000000401bd4 in uw_init_context_1[cold] ()
#4 0x00000000009cebda in _Unwind_ForcedUnwind ()
#5 0x0000000000940044 in __pthread_unwind ()
#6 0x000000000093b9e0 in pthread_exit ()
#7 0x00000000009022ad in GC_pthread_exit ()
#8 0x00000000008bd062 in action_shutdown_ws_engine ()
#9 0x00000000008be0ea in scheduler_module_idle_sleep ()
#10 0x00000000008ca58d in MR_call_engine ()
#11 0x00000000008e5d05 in MR_init_thread_inner ()
#12 0x00000000008e5e8c in MR_create_worksteal_thread_2 ()
#13 0x00000000009048d9 in GC_inner_start_routine ()
#14 0x00000000008f46fe in GC_call_with_stack_base ()
#15 0x000000000093a85c in start_thread ()
#16 0x000000000096875c in clone3 ()
Thread 2 (Thread 0x1c533c0 (LWP 11092)):
#0 0x000000000096656d in write ()
#1 0x0000000000933f35 in _IO_new_file_write ()
#2 0x00000000009321c1 in _IO_new_do_write ()
#3 0x0000000000936906 in _IO_flush_all ()
#4 0x0000000000936f2d in _IO_cleanup ()
#5 0x000000000092236e in __run_exit_handlers ()
#6 0x00000000009223be in exit ()
#7 0x0000000000917d0f in __libc_start_call_main ()
#8 0x0000000000919ed0 in __libc_start_main_impl ()
#9 0x0000000000401d85 in _start ()
Thread 3 (Thread 0x7f6dd97d16c0 (LWP 11093) (Exiting)):
#0 0x0000000000941d19 in alloc_new_heap ()
#1 0x00000000009421f2 in arena_get2.part ()
#2 0x0000000000943fb9 in tcache_init.part ()
#3 0x00000000009448c4 in malloc ()
#4 0x00000000009d2141 in _Unwind_Find_FDE ()
#5 0x00000000009cda7d in uw_frame_state_for ()
#6 0x00000000009ce0d3 in uw_init_context_1 ()
#7 0x00000000009cebda in _Unwind_ForcedUnwind ()
#8 0x0000000000940044 in __pthread_unwind ()
#9 0x000000000093b9e0 in pthread_exit ()
#10 0x00000000009022ad in GC_pthread_exit ()
#11 0x00000000008bd062 in action_shutdown_ws_engine ()
#12 0x00000000008be0ea in scheduler_module_idle_sleep ()
#13 0x00000000008ca58d in MR_call_engine ()
#14 0x00000000008e5d05 in MR_init_thread_inner ()
#15 0x00000000008e5e8c in MR_create_worksteal_thread_2 ()
#16 0x00000000009048d9 in GC_inner_start_routine ()
#17 0x00000000008f46fe in GC_call_with_stack_base ()
#18 0x000000000093a85c in start_thread ()
#19 0x000000000096875c in clone3 ()
f3e5419499