Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attempting to avoid segfault in OctoTiger during initialization #6415

Merged
merged 1 commit into from
Jan 14, 2024

Conversation

hkaiser
Copy link
Member

@hkaiser hkaiser commented Jan 13, 2024

Fixes #6414

@G-071 could you please verify whether this fixes the segfault you reported?

@G-071
Copy link
Member

G-071 commented Jan 13, 2024

The PR helps, but it appears that there was more than one issue. I am still encountering segmentation faults, they are just occurring at a different place now:

#0  0x00007ffff730b68a in std::default_delete<hpx::util::extra_data_member_base>::operator() (__ptr=0x7fffd52a1900, this=<optimized out>) at /usr/include/c++/12/bits/unique_ptr.h:89
#1  std::unique_ptr<hpx::util::extra_data_member_base, std::default_delete<hpx::util::extra_data_member_base> >::~unique_ptr (this=0x7fffc8dee7f0, __in_chrg=<optimized out>) at /usr/include/c++/12/bits/unique_ptr.h:396
#2  hpx::util::extra_data_node::~extra_data_node (this=0x7fffc8dee7f0, __in_chrg=<optimized out>) at /home/daissgr/workshop/KokkosTiger/src/hpx
/libs/core/type_support/include/hpx/type_support/extra_data.hpp:61
#3  hpx::util::extra_data::~extra_data (this=0x7fffc8dee7f0, __in_chrg=<optimized out>) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/core/type_support/include/hpx/type_support/extra_data.hpp:144
#4  hpx::lcos::detail::future_data<hpx::id_type>::~future_data (this=0x7fffc8dee780, __in_chrg=<optimized out>) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/full/components/include/hpx/components/client_base.hpp:227
#5  hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void>::~future_data_allocator (this=0x7fffc8dee780, __in_chrg=<optimized out>) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/core/futures/include/hpx/futures/detail/future_data.hpp:730
#6  0x00007ffff7319380 in std::__new_allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> >::destroy<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > (__p=<optimized out>, this=<optimized out>) at /usr/include/c++/12/bits/new_allocator.h:181
#7  std::allocator_traits<std::allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > >::destroy<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > (__p=<optimized out>, __a=...) at /usr/include/c++/12/bits/alloc_traits.h:535
#8  hpx::util::thread_local_caching_allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void>, std::allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > >::destroy<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > (p=<optimized out>, this=<optimized out>) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/core/allocator_support/include/hpx/allocator_support/thread_local_caching_allocator.hpp:183
#9  std::allocator_traits<hpx::util::thread_local_caching_allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void>, std::allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > > >::_S_destroy<hpx::util::thread_local_caching_allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void>, std::allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > >, hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > (__p=<optimized out>, __a=...) at /usr/include/c++/12/bits/alloc_traits.h:272
#10 std::allocator_traits<hpx::util::thread_local_caching_allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void>, std::allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > > >::destroy<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > (__p=<optimized out>, __a=...) at /usr/include/c++/12/bits/alloc_traits.h:378
#11 hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void>::destroy (this=0x7fffc8dee780) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/core/futures/include/hpx/futures/detail/future_data.hpp:777
#12 0x00007ffff78a5ed6 in hpx::lcos::detail::intrusive_ptr_release (p=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/detail/future_data.hpp:145
#13 hpx::lcos::detail::intrusive_ptr_release (p=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/detail/future_data.hpp:141
#14 hpx::intrusive_ptr<hpx::lcos::detail::future_data_base<hpx::id_type> >::~intrusive_ptr (this=<optimized out>, __in_chrg=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/memory/intrusive_ptr.hpp:88
#15 hpx::intrusive_ptr<hpx::lcos::detail::future_data_base<hpx::id_type> >::reset (this=0x7fffd9d96e08) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/memory/intrusive_ptr.hpp:152
#16 hpx::future<hpx::id_type>::invalidate::~invalidate (this=0x7fffd9d96d38, __in_chrg=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/future.hpp:560
#17 hpx::future<hpx::id_type>::get (this=0x7fffd9d96e08) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/future.hpp:698
#18 0x00007ffff78d3da2 in operator() (__closure=<optimized out>, fut=...) at /home/daissgr/workshop/KokkosTiger/src/octotiger/src/node_server_actions_2.cpp:641
#19 operator() (__closure=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/packaged_continuation.hpp:69
#20 hpx::detail::try_catch_exception_ptr<hpx::lcos::detail::invoke_continuation_nounwrap<node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, hpx::future<long unsigned int>, continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*> >(node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>&, hpx::future<long unsigned int>&&, continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*>&)::<lambda()>, hpx::lcos::detail::invoke_continuation_nounwrap<node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, hpx::future<long unsigned int>, continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*> >(node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>&, hpx::future<long unsigned int>&&, continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*>&)::<lambda(std::__exception_ptr::exception_ptr)> > (c=..., t=...) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/errors/try_catch_exception_ptr.hpp:37
#21 hpx::lcos::detail::invoke_continuation_nounwrap<node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, hpx::future<long unsigned int>, hpx::lcos::detail::continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*> >(struct {...} &, hpx::future<unsigned long> &&, hpx::lcos::detail::continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*> &) (func=..., future=..., cont=warning: RTTI symbol not found for class 'hpx::lcos::detail::continuation_allocator<hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, hpx::future<unsigned long>, node_client::get_ptr() const::{lambda(hpx::future<unsigned long>&&)#1}, node_server*>'
...) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/packaged_continuation.hpp:57
#22 0x00007ffff78cbe3e in hpx::lcos::detail::invoke_continuation<node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, hpx::future<long unsigned int>, hpx::lcos::detail::continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*> > (cont=..., future=..., func=...) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/packaged_continuation.hpp:85
#23 hpx::lcos::detail::continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*>::run_impl<true> (f=..., this=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/packaged_continuation.hpp:215
#24 operator() (__closure=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/packaged_continuation.hpp:242
#25 hpx::threads::detail::thread_function_nullary<hpx::lcos::detail::continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*>::async<true, hpx::lcos::detail::post_policy_spawner>(hpx::traits::detail::shared_state_ptr_for_t<hpx::future<long unsigned int> >&&, hpx::lcos::detail::post_policy_spawner&&)::<lambda()> >::operator() (this=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/threading_base/register_thread.hpp:50
#26 hpx::util::detail::callable_vtable<std::pair<hpx::threads::thread_schedule_state, hpx::threads::thread_id>(hpx::threads::thread_restart_state)>::_invoke<hpx::threads::detail::thread_function_nullary<hpx::lcos::detail::continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*>::async<true, hpx::lcos::detail::post_policy_spawner>(hpx::traits::detail::shared_state_ptr_for_t<hpx::future<long unsigned int> >&&, hpx::lcos::detail::post_policy_spawner&&)::<lambda()> > >(void *, hpx::threads::thread_restart_state &&) (f=<optimized out>, vs#0=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/functional/detail/vtable/callable_vtable.hpp:88
#27 0x00007ffff6f28db9 in hpx::util::detail::basic_function<std::pair<hpx::threads::thread_schedule_state, hpx::threads::thread_id> (hpx::threads::thread_restart_state), false, false>::operator()(hpx::threads::thread_restart_state) const (vs#0=<optimized out>, this=0x7fffe37cf378) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/core/functional/include/hpx/functional/detail/basic_function.hpp:236
#28 hpx::threads::coroutines::detail::coroutine_impl::operator() (this=0x7fffe37cf310) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/core/coroutines/src/detail/coroutine_impl.cpp:77
#29 0x00007ffff6f2821f in hpx::threads::coroutines::detail::lx::trampoline<hpx::threads::coroutines::detail::coroutine_impl> (fun=<optimized out>) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/core/coroutines/include/hpx/coroutines/detail/context_linux_x86.hpp:181
#30 0x0000000000000000 in ?? ()

@hkaiser
Copy link
Member Author

hkaiser commented Jan 13, 2024

@G-071 could you please try one more time?

@G-071
Copy link
Member

G-071 commented Jan 13, 2024

Still not quite it, I get a segfault at

std::unique_ptr<hpx::util::extra_data_member_base, std::default_delete<hpx::util::extra_data_member_base> >::~unique_ptr (this=0x7fffeaa41170, __in_chrg=<optimized out>) at /usr/include/c++/12/bits/unique_ptr.h:396
396		 get_deleter()(std::move(__ptr));

Full backtrace:

#0  std::unique_ptr<hpx::util::extra_data_member_base, std::default_delete<hpx::util::extra_data_member_base> >::~unique_ptr (this=0x7fffeaa41170, __in_chrg=<optimized out>) at /usr/include/c++/12/bits/unique_ptr.h:396
#1  hpx::util::extra_data_node::~extra_data_node (this=0x7fffeaa41170, __in_chrg=<optimized out>) at /home/daissgr/workshop/KokkosTiger/src/hpx
/libs/core/type_support/include/hpx/type_support/extra_data.hpp:59
#2  hpx::util::extra_data::~extra_data (this=0x7fffeaa41170, __in_chrg=<optimized out>) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/core/type_support/include/hpx/type_support/extra_data.hpp:145
#3  hpx::lcos::detail::future_data<hpx::id_type>::~future_data (this=0x7fffeaa41100, __in_chrg=<optimized out>) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/full/components/include/hpx/components/client_base.hpp:228
#4  hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void>::~future_data_allocator (this=0x7fffeaa41100, __in_chrg=<optimized out>) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/core/futures/include/hpx/futures/detail/future_data.hpp:730
#5  0x00007ffff7319370 in std::__new_allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> >::destroy<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > (__p=<optimized out>, this=<optimized out>) at /usr/include/c++/12/bits/new_allocator.h:181
#6  std::allocator_traits<std::allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > >::destroy<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > (__p=<optimized out>, __a=...) at /usr/include/c++/12/bits/alloc_traits.h:535
#7  hpx::util::thread_local_caching_allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void>, std::allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > >::destroy<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > (p=<optimized out>, this=<optimized out>) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/core/allocator_support/include/hpx/allocator_support/thread_local_caching_allocator.hpp:183
#8  std::allocator_traits<hpx::util::thread_local_caching_allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void>, std::allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > > >::_S_destroy<hpx::util::thread_local_caching_allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void>, std::allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > >, hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > (__p=<optimized out>, __a=...) at /usr/include/c++/12/bits/alloc_traits.h:272
#9  std::allocator_traits<hpx::util::thread_local_caching_allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void>, std::allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > > >::destroy<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > (__p=<optimized out>, __a=...) at /usr/include/c++/12/bits/alloc_traits.h:378
#10 hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void>::destroy (this=0x7fffeaa41100) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/core/futures/include/hpx/futures/detail/future_data.hpp:777
#11 0x00007ffff78a5ed6 in hpx::lcos::detail::intrusive_ptr_release (p=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/detail/future_data.hpp:145
#12 hpx::lcos::detail::intrusive_ptr_release (p=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/detail/future_data.hpp:141
#13 hpx::intrusive_ptr<hpx::lcos::detail::future_data_base<hpx::id_type> >::~intrusive_ptr (this=<optimized out>, __in_chrg=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/memory/intrusive_ptr.hpp:88
#14 hpx::intrusive_ptr<hpx::lcos::detail::future_data_base<hpx::id_type> >::reset (this=0x7ffff64aee08) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/memory/intrusive_ptr.hpp:152
#15 hpx::future<hpx::id_type>::invalidate::~invalidate (this=0x7ffff64aed38, __in_chrg=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/future.hpp:560
#16 hpx::future<hpx::id_type>::get (this=0x7ffff64aee08) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/future.hpp:698
#17 0x00007ffff78d3da2 in operator() (__closure=<optimized out>, fut=...) at /home/daissgr/workshop/KokkosTiger/src/octotiger/src/node_server_actions_2.cpp:641
#18 operator() (__closure=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/packaged_continuation.hpp:69
#19 hpx::detail::try_catch_exception_ptr<hpx::lcos::detail::invoke_continuation_nounwrap<node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, hpx::future<long unsigned int>, continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*> >(node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>&, hpx::future<long unsigned int>&&, continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*>&)::<lambda()>, hpx::lcos::detail::invoke_continuation_nounwrap<node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, hpx::future<long unsigned int>, continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*> >(node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>&, hpx::future<long unsigned int>&&, continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*>&)::<lambda(std::__exception_ptr::exception_ptr)> > (c=..., t=...) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/errors/try_catch_exception_ptr.hpp:37
#20 hpx::lcos::detail::invoke_continuation_nounwrap<node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, hpx::future<long unsigned int>, hpx::lcos::detail::continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*> >(struct {...} &, hpx::future<unsigned long> &&, hpx::lcos::detail::continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*> &) (func=..., future=..., cont=warning: RTTI symbol not found for class 'hpx::lcos::detail::continuation_allocator<hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, hpx::future<unsigned long>, node_client::get_ptr() const::{lambda(hpx::future<unsigned long>&&)#1}, node_server*>'
...) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/packaged_continuation.hpp:57
#21 0x00007ffff78cbe3e in hpx::lcos::detail::invoke_continuation<node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, hpx::future<long unsigned int>, hpx::lcos::detail::continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*> > (cont=..., future=..., func=...) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/packaged_continuation.hpp:85
#22 hpx::lcos::detail::continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*>::run_impl<true> (f=..., this=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/packaged_continuation.hpp:215
#23 operator() (__closure=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/packaged_continuation.hpp:242
#24 hpx::threads::detail::thread_function_nullary<hpx::lcos::detail::continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*>::async<true, hpx::lcos::detail::post_policy_spawner>(hpx::traits::detail::shared_state_ptr_for_t<hpx::future<long unsigned int> >&&, hpx::lcos::detail::post_policy_spawner&&)::<lambda()> >::operator() (this=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/threading_base/register_thread.hpp:50
#25 hpx::util::detail::callable_vtable<std::pair<hpx::threads::thread_schedule_state, hpx::threads::thread_id>(hpx::threads::thread_restart_state)>::_invoke<hpx::threads::detail::thread_function_nullary<hpx::lcos::detail::continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*>::async<true, hpx::lcos::detail::post_policy_spawner>(hpx::traits::detail::shared_state_ptr_for_t<hpx::future<long unsigned int> >&&, hpx::lcos::detail::post_policy_spawner&&)::<lambda()> > >(void *, hpx::threads::thread_restart_state &&) (f=<optimized out>, vs#0=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/functional/detail/vtable/callable_vtable.hpp:88
#26 0x00007ffff6f28dc9 in hpx::util::detail::basic_function<std::pair<hpx::threads::thread_schedule_state, hpx::threads::thread_id> (hpx::threads::thread_restart_state), false, false>::operator()(hpx::threads::thread_restart_state) const (vs#0=<optimized out>, this=0x7fffe481f4f8) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/core/functional/include/hpx/functional/detail/basic_function.hpp:236
#27 hpx::threads::coroutines::detail::coroutine_impl::operator() (this=0x7fffe481f490) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/core/coroutines/src/detail/coroutine_impl.cpp:77
#28 0x00007ffff6f2822f in hpx::threads::coroutines::detail::lx::trampoline<hpx::threads::coroutines::detail::coroutine_impl> (fun=<optimized out>) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/core/coroutines/include/hpx/coroutines/detail/context_linux_x86.hpp:181
#29 0x0000000000000000 in ?? ()

@hkaiser
Copy link
Member Author

hkaiser commented Jan 13, 2024

Still not quite it, I get a segfault at

std::unique_ptr<hpx::util::extra_data_member_base, std::default_delete<hpx::util::extra_data_member_base> >::~unique_ptr (this=0x7fffeaa41170, __in_chrg=<optimized out>) at /usr/include/c++/12/bits/unique_ptr.h:396
396		 get_deleter()(std::move(__ptr));

Full backtrace:

#0  std::unique_ptr<hpx::util::extra_data_member_base, std::default_delete<hpx::util::extra_data_member_base> >::~unique_ptr (this=0x7fffeaa41170, __in_chrg=<optimized out>) at /usr/include/c++/12/bits/unique_ptr.h:396
#1  hpx::util::extra_data_node::~extra_data_node (this=0x7fffeaa41170, __in_chrg=<optimized out>) at /home/daissgr/workshop/KokkosTiger/src/hpx
/libs/core/type_support/include/hpx/type_support/extra_data.hpp:59
#2  hpx::util::extra_data::~extra_data (this=0x7fffeaa41170, __in_chrg=<optimized out>) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/core/type_support/include/hpx/type_support/extra_data.hpp:145
#3  hpx::lcos::detail::future_data<hpx::id_type>::~future_data (this=0x7fffeaa41100, __in_chrg=<optimized out>) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/full/components/include/hpx/components/client_base.hpp:228
#4  hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void>::~future_data_allocator (this=0x7fffeaa41100, __in_chrg=<optimized out>) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/core/futures/include/hpx/futures/detail/future_data.hpp:730
#5  0x00007ffff7319370 in std::__new_allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> >::destroy<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > (__p=<optimized out>, this=<optimized out>) at /usr/include/c++/12/bits/new_allocator.h:181
#6  std::allocator_traits<std::allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > >::destroy<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > (__p=<optimized out>, __a=...) at /usr/include/c++/12/bits/alloc_traits.h:535
#7  hpx::util::thread_local_caching_allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void>, std::allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > >::destroy<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > (p=<optimized out>, this=<optimized out>) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/core/allocator_support/include/hpx/allocator_support/thread_local_caching_allocator.hpp:183
#8  std::allocator_traits<hpx::util::thread_local_caching_allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void>, std::allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > > >::_S_destroy<hpx::util::thread_local_caching_allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void>, std::allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > >, hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > (__p=<optimized out>, __a=...) at /usr/include/c++/12/bits/alloc_traits.h:272
#9  std::allocator_traits<hpx::util::thread_local_caching_allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void>, std::allocator<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > > >::destroy<hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void> > (__p=<optimized out>, __a=...) at /usr/include/c++/12/bits/alloc_traits.h:378
#10 hpx::lcos::detail::future_data_allocator<hpx::id_type, hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, void>::destroy (this=0x7fffeaa41100) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/core/futures/include/hpx/futures/detail/future_data.hpp:777
#11 0x00007ffff78a5ed6 in hpx::lcos::detail::intrusive_ptr_release (p=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/detail/future_data.hpp:145
#12 hpx::lcos::detail::intrusive_ptr_release (p=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/detail/future_data.hpp:141
#13 hpx::intrusive_ptr<hpx::lcos::detail::future_data_base<hpx::id_type> >::~intrusive_ptr (this=<optimized out>, __in_chrg=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/memory/intrusive_ptr.hpp:88
#14 hpx::intrusive_ptr<hpx::lcos::detail::future_data_base<hpx::id_type> >::reset (this=0x7ffff64aee08) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/memory/intrusive_ptr.hpp:152
#15 hpx::future<hpx::id_type>::invalidate::~invalidate (this=0x7ffff64aed38, __in_chrg=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/future.hpp:560
#16 hpx::future<hpx::id_type>::get (this=0x7ffff64aee08) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/future.hpp:698
#17 0x00007ffff78d3da2 in operator() (__closure=<optimized out>, fut=...) at /home/daissgr/workshop/KokkosTiger/src/octotiger/src/node_server_actions_2.cpp:641
#18 operator() (__closure=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/packaged_continuation.hpp:69
#19 hpx::detail::try_catch_exception_ptr<hpx::lcos::detail::invoke_continuation_nounwrap<node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, hpx::future<long unsigned int>, continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*> >(node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>&, hpx::future<long unsigned int>&&, continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*>&)::<lambda()>, hpx::lcos::detail::invoke_continuation_nounwrap<node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, hpx::future<long unsigned int>, continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*> >(node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>&, hpx::future<long unsigned int>&&, continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*>&)::<lambda(std::__exception_ptr::exception_ptr)> > (c=..., t=...) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/errors/try_catch_exception_ptr.hpp:37
#20 hpx::lcos::detail::invoke_continuation_nounwrap<node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, hpx::future<long unsigned int>, hpx::lcos::detail::continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*> >(struct {...} &, hpx::future<unsigned long> &&, hpx::lcos::detail::continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*> &) (func=..., future=..., cont=warning: RTTI symbol not found for class 'hpx::lcos::detail::continuation_allocator<hpx::util::thread_local_caching_allocator<char, std::allocator<char> >, hpx::future<unsigned long>, node_client::get_ptr() const::{lambda(hpx::future<unsigned long>&&)#1}, node_server*>'
...) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/packaged_continuation.hpp:57
#21 0x00007ffff78cbe3e in hpx::lcos::detail::invoke_continuation<node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, hpx::future<long unsigned int>, hpx::lcos::detail::continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*> > (cont=..., future=..., func=...) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/packaged_continuation.hpp:85
#22 hpx::lcos::detail::continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*>::run_impl<true> (f=..., this=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/packaged_continuation.hpp:215
#23 operator() (__closure=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/futures/packaged_continuation.hpp:242
#24 hpx::threads::detail::thread_function_nullary<hpx::lcos::detail::continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*>::async<true, hpx::lcos::detail::post_policy_spawner>(hpx::traits::detail::shared_state_ptr_for_t<hpx::future<long unsigned int> >&&, hpx::lcos::detail::post_policy_spawner&&)::<lambda()> >::operator() (this=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/threading_base/register_thread.hpp:50
#25 hpx::util::detail::callable_vtable<std::pair<hpx::threads::thread_schedule_state, hpx::threads::thread_id>(hpx::threads::thread_restart_state)>::_invoke<hpx::threads::detail::thread_function_nullary<hpx::lcos::detail::continuation<hpx::future<long unsigned int>, node_client::get_ptr() const::<lambda(future<long unsigned int>&&)>, node_server*>::async<true, hpx::lcos::detail::post_policy_spawner>(hpx::traits::detail::shared_state_ptr_for_t<hpx::future<long unsigned int> >&&, hpx::lcos::detail::post_policy_spawner&&)::<lambda()> > >(void *, hpx::threads::thread_restart_state &&) (f=<optimized out>, vs#0=<optimized out>) at /home/daissgr/workshop/KokkosTiger/build/hpx/include/hpx/functional/detail/vtable/callable_vtable.hpp:88
#26 0x00007ffff6f28dc9 in hpx::util::detail::basic_function<std::pair<hpx::threads::thread_schedule_state, hpx::threads::thread_id> (hpx::threads::thread_restart_state), false, false>::operator()(hpx::threads::thread_restart_state) const (vs#0=<optimized out>, this=0x7fffe481f4f8) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/core/functional/include/hpx/functional/detail/basic_function.hpp:236
#27 hpx::threads::coroutines::detail::coroutine_impl::operator() (this=0x7fffe481f490) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/core/coroutines/src/detail/coroutine_impl.cpp:77
#28 0x00007ffff6f2822f in hpx::threads::coroutines::detail::lx::trampoline<hpx::threads::coroutines::detail::coroutine_impl> (fun=<optimized out>) at /home/daissgr/workshop/KokkosTiger/src/hpx/libs/core/coroutines/include/hpx/coroutines/detail/context_linux_x86.hpp:181
#29 0x0000000000000000 in ?? ()

Hmmm, still the same as before - not sure what's going on :/

@hkaiser
Copy link
Member Author

hkaiser commented Jan 13, 2024

@G-071 does it happen in Debug mode as well?

@hkaiser
Copy link
Member Author

hkaiser commented Jan 13, 2024

@G-071 Unrelated comment: the function node_client::get_ptr() (https://github.com/STEllAR-GROUP/octotiger/blob/5803961fd5334acef5665b8337bec0d16e165e9e/src/node_server_actions_2.cpp#L639-L647), i.e. the function where things are blowing up,

could be implemented as

future<std::shared_ptr<node_server>> node_client::get_ptr() const {
    return hpx::get_ptr<node_server>(get_unmanaged_gid());
}

(note the changed return type). Not sure if this would change anything, though.

At least, rewrite

    if (hpx::find_here() != hpx::get_colocation_id(get_gid()).get())

as

    if (hpx::get_locality_id() != hpx::naming::get_locality_id_from_id(get_unmanaged_id()))

as this check can be done locally (without invoking any possibly remote operations).

@hkaiser
Copy link
Member Author

hkaiser commented Jan 13, 2024

@G-071 I have added a couple of (somewhat desparate) changes here. I however think that the changes to OctoTiger I suggest above would be more appropriate.

@StellarBot
Copy link

Performance test report

HPX Performance

Comparison

BENCHMARKFORK_JOIN_EXECUTORPARALLEL_EXECUTORSCHEDULER_EXECUTOR
For Each(=)??-

Info

PropertyBeforeAfter
HPX Datetime2023-05-10T12:07:53+00:002024-01-13T19:42:48+00:00
HPX Commitdcb5415c95b972
Datetime2023-05-10T14:50:18.616050-05:002024-01-13T13:54:06.876970-06:00
Envfile
Clusternamerostamrostam
Hostnamemedusa08.rostam.cct.lsu.edumedusa08.rostam.cct.lsu.edu
Compiler/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1

Comparison

BENCHMARKNO-EXECUTOR
Future Overhead - Create Thread Hierarchical - Latch(=)

Info

PropertyBeforeAfter
HPX Datetime2023-05-10T12:07:53+00:002024-01-13T19:42:48+00:00
HPX Commitdcb5415c95b972
Datetime2023-05-10T14:52:35.047119-05:002024-01-13T13:56:19.887665-06:00
Envfile
Clusternamerostamrostam
Hostnamemedusa08.rostam.cct.lsu.edumedusa08.rostam.cct.lsu.edu
Compiler/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1

Comparison

BENCHMARKFORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATORPARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATORSCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATOR
Stream Benchmark - Add(=)(=)(=)
Stream Benchmark - Scale+(=)=
Stream Benchmark - Triad(=)=(=)
Stream Benchmark - Copy(=)(=)(=)

Info

PropertyBeforeAfter
HPX Datetime2023-05-10T12:07:53+00:002024-01-13T19:42:48+00:00
HPX Commitdcb5415c95b972
Datetime2023-05-10T14:52:52.237641-05:002024-01-13T13:56:36.821748-06:00
Envfile
Clusternamerostamrostam
Hostnamemedusa08.rostam.cct.lsu.edumedusa08.rostam.cct.lsu.edu
Compiler/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1

Explanation of Symbols

SymbolMEANING
=No performance change (confidence interval within ±1%)
(=)Probably no performance change (confidence interval within ±2%)
(+)/(-)Very small performance improvement/degradation (≤1%)
+/-Small performance improvement/degradation (≤5%)
++/--Large performance improvement/degradation (≤10%)
+++/---Very large performance improvement/degradation (>10%)
?Probably no change, but quite large uncertainty (confidence interval with ±5%)
??Unclear result, very large uncertainty (±10%)
???Something unexpected…

@hkaiser hkaiser force-pushed the fixing_6414 branch 3 times, most recently from 4d2cca5 to 9f23e8a Compare January 13, 2024 20:47
@G-071
Copy link
Member

G-071 commented Jan 13, 2024

@G-071 Unrelated comment: the function node_client::get_ptr() (https://github.com/STEllAR-GROUP/octotiger/blob/5803961fd5334acef5665b8337bec0d16e165e9e/src/node_server_actions_2.cpp#L639-L647), i.e. the function where things are blowing up,

could be implemented as

future<std::shared_ptr<node_server>> node_client::get_ptr() const {
    return hpx::get_ptr<node_server>(get_unmanaged_gid());
}

(note the changed return type). Not sure if this would change anything, though.

I am not opposed to this, however, I would need to check what other parts I would need to change for this. This part of Octo-Tiger pre-dates my involvement in the project by quite a bit, so I am not too familiar with it.

At least, rewrite

    if (hpx::find_here() != hpx::get_colocation_id(get_gid()).get())

as

    if (hpx::get_locality_id() != hpx::naming::get_locality_id_from_id(get_unmanaged_id()))

as this check can be done locally (without invoking any possibly remote operations).

Did you mean get_unmanaged_gid here instead of get_unmanaged_id here? (that's what the compiler suggested when it could not find get_unmanaged_id)

Replacing the branch condition with

if (hpx::get_locality_id() != hpx::naming::get_locality_id_from_id(get_unmanaged_gid())) {

seems to actually resolve the issue entirely (at least on my local machine) and all Octo-Tiger tests pass! This even seems to work without this PR.

@hkaiser
Copy link
Member Author

hkaiser commented Jan 13, 2024

Replacing the branch condition with

if (hpx::get_locality_id() != hpx::naming::get_locality_id_from_id(get_unmanaged_gid())) {

seems to actually resolve the issue entirely (at least on my local machine) and all Octo-Tiger tests pass! This even seems to work without this PR.

Yah, that works around the issue as it removes the failing code. Let's go with that for the time being. I'll merge the PR anyways as it resolves at least one (other) real issue.

Copy link

codacy-production bot commented Jan 13, 2024

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation Diff coverage
-0.03% 80.95%
Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (b8cc430) 198612 169125 85.15%
Head commit (a55a844) 190679 (-7933) 162308 (-6817) 85.12% (-0.03%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#6415) 21 17 80.95%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

You may notice some variations in coverage metrics with the latest Coverage engine update. For more details, visit the documentation

@hkaiser hkaiser merged commit cf70bf1 into master Jan 14, 2024
62 of 70 checks passed
@hkaiser hkaiser deleted the fixing_6414 branch January 14, 2024 17:44
@hkaiser hkaiser added this to the 1.10.0 milestone Jan 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Current HPX master causes segfaults within Octo-Tiger
3 participants