[Llvm-bgq-discuss] Does bgclang++11 implement thread_local

Sat Aug 13 09:08:23 CDT 2016

Hi Patrick,

Thanks again for reporting this issue. I've added an implementation of __cxa_thread_atexit to bgclang, it is now in the nightly builds, and will be in the next release soon.

In case you're curious, I ended up writing an implementation for our libc++ similar to the implementation in libstdc++ 4.8.0. libc++abi has an implementation, but it depends on __cxa_thread_atexit_impl in glibc, but that was not added until glibc 2.18 (we're on 2.17). I looked into backporting the implementation from 2.18 into our glibc, but that would have changed the structure used by the dynamic loader which would have been an ABI break for existing dynamically-linked bgclang-compiled executables. __cxa_thread_atexit was added to libstdc++ in 4.8, and libstdc++ was later improved to use glibc's __cxa_thread_atexit_impl when available (which it is not in this case). Only implementations based on __cxa_thread_atexit_impl handle dynamic-library unloading correctly (i.e. when you use dlclose). Otherwise, like the implementation in libstdc++ 4.8, this should work well.

Please let me know if you have any problems with it.

 -Hal

----- Original Message -----
> From: "Hal Finkel" <hfinkel at anl.gov>
> To: "Patrick E Gartung" <gartung at fnal.gov>
> Cc: llvm-bgq-discuss at lists.alcf.anl.gov
> Sent: Thursday, July 14, 2016 9:11:48 PM
> Subject: Re: [Llvm-bgq-discuss] Does bgclang++11 implement thread_local
> 
> Hi Patrick,
> 
> Thanks! Indeed; shipping a newer libstdc++ with bgclang is also an
> option ;)
> 
>  -Hal
> 
> ----- Original Message -----
> > From: "Patrick E Gartung" <gartung at fnal.gov>
> > To: "Hal Finkel" <hfinkel at anl.gov>
> > Cc: llvm-bgq-discuss at lists.alcf.anl.gov
> > Sent: Thursday, July 14, 2016 9:04:42 PM
> > Subject: Re: [Llvm-bgq-discuss] Does bgclang++11 implement
> > thread_local
> > 
> > Hi Hal,
> > 
> > I managed to get things compiled with gcc4.8.4 in the $MPICH_CC
> > wrapper. This is what I have in ~/.soft
> > 
> > #@default
> > +accttools
> > +os_paths
> > +trackdeps
> > +user_paths
> > +utility_paths
> > +git-2.3.0
> > +cmake-3.5.1
> > +python-2.7.9
> > +bgqdriver-V1R2M2
> > +mpiwrapper-gcc
> > +bgqtoolchain-gcc484
> > 
> > 
> > 
> > Patrick
> > 
> > 
> > 
> > 
> > On 7/14/16, 9:01 PM, "Hal Finkel" <hfinkel at anl.gov> wrote:
> > 
> > >----- Original Message -----
> > >> From: "Patrick E Gartung" <gartung at fnal.gov>
> > >> To: "Hal Finkel" <hfinkel at anl.gov>
> > >> Cc: llvm-bgq-discuss at lists.alcf.anl.gov
> > >> Sent: Thursday, July 14, 2016 8:53:17 PM
> > >> Subject: Re: [Llvm-bgq-discuss] Does bgclang++11 implement
> > >> thread_local
> > >> 
> > >> Hi Hal,
> > >> 
> > >> Can you add libc++abi to the bgclang build? It is supposed to
> > >> provide
> > >> thread local for clang.
> > >> http://libcxxabi.llvm.org/
> > >
> > >Not directly (integration here is tricky because of the need to
> > >support statically linking against g++-compiled code - like the
> > >BG/Q PAMI implementation), however, I plan to try extracting the
> > >implementation from there if practical.
> > >
> > >> As you can see from the link line I also tried to use libsupc++
> > >> for
> > >> this.
> > >
> > >Yep; the system libraries on the BG/Q are old. That's why bgclang
> > >has to supplement in many circumstances.
> > >
> > >Thanks again,
> > >Hal
> > >
> > >> 
> > >> Patrick
> > >> 
> > >> 
> > >> 
> > >> 
> > >> On 7/14/16, 8:28 PM, "Hal Finkel" <hfinkel at anl.gov> wrote:
> > >> 
> > >> >Hi Patrick,
> > >> >
> > >> >Thanks for the report. I suspect this is another case where
> > >> >bgclang++11 needs to supplement the base system libraries; I'll
> > >> >need to see whether that is possible. thread_local should work
> > >> >on
> > >> >plain data types; I suspect this problem is tied to the need to
> > >> >call a destructor on this variable when the thread terminates.
> > >> >
> > >> > -Hal
> > >> >
> > >> >----- Original Message -----
> > >> >> From: "Patrick E Gartung" <gartung at fnal.gov>
> > >> >> To: llvm-bgq-discuss at lists.alcf.anl.gov
> > >> >> Sent: Tuesday, July 12, 2016 11:25:54 PM
> > >> >> Subject: [Llvm-bgq-discuss] Does bgclang++11 implement
> > >> >> thread_local
> > >> >> 
> > >> >> Hi
> > >> >> 
> > >> >> I am trying to link code that uses thread_local and I get
> > >> >> this
> > >> >> error:
> > >> >> 
> > >> >> /soft/compilers/bgclang/nightly/wbin/bgclang++11   -g -fPIC
> > >> >> -fno-vectorize -fno-slp-vectorize -std=c++11 -std=c++14
> > >> >>  -dynamic
> > >> >> -lsupc++ CMakeFiles/TBBDemo.dir/main.cpp.o  -o
> > >> >> ../BuildProducts/bin/TBBDemo -rdynamic
> > >> >> ../BuildProducts/lib64/libTBBFrameworkCore.a
> > >> >> ../BuildProducts/lib64/libTBBTestModules.so
> > >> >> ../BuildProducts/lib64/libTBBFrameworkCore.a
> > >> >> /home/gartung/tbb44_20160526oss/build/linux_bg_clang_cc4.4.7_libc2.12_kernel2.6.32_release/libtbbmalloc.a
> > >> >> /home/gartung/tbb44_20160526oss/build/linux_bg_clang_cc4.4.7_libc2.12_kernel2.6.32_release/libtbb.a
> > >> >> -Wl,-rpath,/home/gartung/build/BuildProducts/lib64:
> > >> >> ../BuildProducts/lib64/libTBBFrameworkCore.a(ModuleThreadStack.cpp.o):
> > >> >> In function `demo::ModuleThreadStack::privateStack()':
> > >> >> /home/gartung/toy-mt-framework/TBBProcessingDemo/TBBFrameworkCore/ModuleThreadStack.cpp:21:
> > >> >> undefined reference to `__cxa_thread_atexit’
> > >> >> 
> > >> >> This is line 21 of ModuleThreadStack.cpp
> > >> >> 
> > >> >>     thread_local std::vector<TransitionModuleID> s_stack;
> > >> >> 
> > >> >> Does bgclang++11 implement thread_local?
> > >> >> 
> > >> >> Patrick Gartung
> > >> >> _______________________________________________
> > >> >> llvm-bgq-discuss mailing list
> > >> >> llvm-bgq-discuss at lists.alcf.anl.gov
> > >> >> https://lists.alcf.anl.gov/mailman/listinfo/llvm-bgq-discuss
> > >> >> 
> > >> >
> > >> >--
> > >> >Hal Finkel
> > >> >Assistant Computational Scientist
> > >> >Leadership Computing Facility
> > >> >Argonne National Laboratory
> > >> 
> > >
> > >--
> > >Hal Finkel
> > >Assistant Computational Scientist
> > >Leadership Computing Facility
> > >Argonne National Laboratory
> > 
> 
> --
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory
> _______________________________________________
> llvm-bgq-discuss mailing list
> llvm-bgq-discuss at lists.alcf.anl.gov
> https://lists.alcf.anl.gov/mailman/listinfo/llvm-bgq-discuss
> 

-- 
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory