SegFault for 1 thread case with kokkos openmp

[weilewei]

Setting value OMP_NUM_THREADS to 2 and more runs fine, but 1 thread case failed due to segfault. Would it be possible to take a look? I tried debugging for a while but did not find much luck... Thanks.

wwei@login38:~/src/kokkos-miniapps/lulesh-2.0/kokkos-minimal-cpu/build> OMP_NUM_THREADS=1 OMP_PROC_BIND=spread OMP_PLACES=threads ./kokkosPar
Running problem size 30^3 per domain until completion
Num processors: 1
Num threads: 1
Total number of elements: 27000

To run other sizes, use -s .
To run a fixed number of iterations, use -i .
To run a more or less balanced region set, use -b .
To change the relative costs of regions, use -c .
To print out progress, use -p
To write an output file for VisIt, use -v
See help (-h) for more options

Segmentation fault
wwei@login38:~/src/kokkos-miniapps/lulesh-2.0/kokkos-minimal-cpu/build> OMP_NUM_THREADS=1 OMP_PROC_BIND=spread OMP_PLACES=threads gdb ./kokkosPar
GNU gdb (GDB; SUSE Linux Enterprise 15) 12.1
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-suse-linux".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
http://bugs.opensuse.org/.
Find the GDB manual and other documentation resources online at:
http://www.gnu.org/software/gdb/documentation/.

--Type for more, q to quit, c to continue without paging--
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./kokkosPar...
(gdb) r
Starting program: /global/u2/w/wwei/src/kokkos-miniapps/lulesh-2.0/kokkos-minimal-cpu/build/kokkosPar
Missing separate debuginfos, use: zypper install glibc-debuginfo-2.31-150300.41.1.x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Running problem size 30^3 per domain until completion
Num processors: 1
Num threads: 1
Total number of elements: 27000

To run other sizes, use -s .
To run a fixed number of iterations, use -i .
To run a more or less balanced region set, use -b .
To change the relative costs of regions, use -c .
To print out progress, use -p
To write an output file for VisIt, use -v
See help (-h) for more options

Program received signal SIGSEGV, Segmentation fault.
0x00001555540c0d21 in __strlen_avx2 () from /lib64/libc.so.6
Missing separate debuginfos, use: zypper install libstdc++6-debuginfo-12.2.1+git416-150000.1.5.1.x86_64
(gdb) bt
#0 0x00001555540c0d21 in __strlen_avx2 () from /lib64/libc.so.6
#1 0x0000155553fbf8cf in __vfprintf_internal () from /lib64/libc.so.6
#2 0x0000155553fc0896 in buffered_vfprintf () from /lib64/libc.so.6
#3 0x0000155553fbd9d4 in __vfprintf_internal () from /lib64/libc.so.6
#4 0x0000155553faa506 in fprintf () from /lib64/libc.so.6
#5 0x00000000004703bd in Kokkos::Impl::SharedAllocationRecord<void, void>::decrement(Kokkos::Impl::SharedAllocationRecord<void, void>) ()
#6 0x000000000042dafc in Kokkos::Impl::SharedAllocationTracker::~SharedAllocationTracker (
this=0x7fffffffa158, __in_chrg=)
at /global/u2/w/wwei/install/kokkos_gcc_openmp/include/impl/Kokkos_SharedAlloc.hpp:447
#7 Kokkos::Impl::ViewTracker<Kokkos::View<double> >::~ViewTracker (this=0x7fffffffa158,
__in_chrg=)
at /global/u2/w/wwei/install/kokkos_gcc_openmp/include/impl/Kokkos_ViewTracker.hpp:67
--Type for more, q to quit, c to continue without paging--
#8 0x000000000042db18 in Kokkos::View<double*>::~View (this=0x7fffffffa158, __in_chrg=)
at /global/u2/w/wwei/install/kokkos_gcc_openmp/include/Kokkos_View.hpp:1297
#9 0x0000000000409fcc in ~(void) (this=0x7fffffff8fd0, __in_chrg=)
at /global/homes/w/wwei/src/kokkos-miniapps/lulesh-2.0/kokkos-minimal-cpu/lulesh.cc:343
#10 0x000000000040a412 in IntegrateStressForElems (domain=..., sigxx=0x155555357010, sigyy=0x155553973010,
sigzz=0x15555393e010, determ=0x155553909010, numElem=27000, numNode=29791)
at /global/homes/w/wwei/src/kokkos-miniapps/lulesh-2.0/kokkos-minimal-cpu/lulesh.cc:342
#11 0x000000000040cba3 in CalcVolumeForceForElems (domain=...)
at /global/homes/w/wwei/src/kokkos-miniapps/lulesh-2.0/kokkos-minimal-cpu/lulesh.cc:825
#12 0x000000000040cd3e in CalcForceForNodes (domain=...)
at /global/homes/w/wwei/src/kokkos-miniapps/lulesh-2.0/kokkos-minimal-cpu/lulesh.cc:861
#13 0x000000000040d8d7 in LagrangeNodal (domain=...)
at /global/homes/w/wwei/src/kokkos-miniapps/lulesh-2.0/kokkos-minimal-cpu/lulesh.cc:953
--Type for more, q to quit, c to continue without paging--
#14 0x000000000041542d in LagrangeLeapFrog (domain=...)
at /global/homes/w/wwei/src/kokkos-miniapps/lulesh-2.0/kokkos-minimal-cpu/lulesh.cc:2241
#15 0x00000000004156eb in main (argc=1, argv=0x7fffffffc7d8)
at /global/homes/w/wwei/src/kokkos-miniapps/lulesh-2.0/kokkos-minimal-cpu/lulesh.cc:2348

[rgayatri23]

I got a seg fault each time with the clang compiler. Which gcc version are you using?

[weilewei]

With clang, do you get seg fault for every thread value or just 1 thread? I am using gcc 11.2.0.

[masterleinad]

diff --git a/lulesh-2.0/kokkos-minimal-cpu/lulesh.cc b/lulesh-2.0/kokkos-minimal-cpu/lulesh.cc
index 94a0a33..cca461e 100644
--- a/lulesh-2.0/kokkos-minimal-cpu/lulesh.cc
+++ b/lulesh-2.0/kokkos-minimal-cpu/lulesh.cc
@@ -359,8 +359,8 @@ static inline void IntegrateStressForElems(Domain &domain, Real_t *sigxx,
                                   &fx_elem[k * 8], &fy_elem[k * 8],
                                   &fz_elem[k * 8]);
     } else {
-      SumElemStressesToNodeForces(B, sigxx[k], sigyy[k], sigzz[k], &fx_local[8],
-                                  &fy_local[8], &fz_local[8]);
+      SumElemStressesToNodeForces(B, sigxx[k], sigyy[k], sigzz[k], fx_local.data(),
+                                  fy_local.data(), fz_local.data());

fixes it.

[weilewei]

can confirm this fix works. Feel free to close it if needed. Thanks!