[Llvm-bgq-discuss] 2 quick questions about upgrading

Tue Feb 3 12:16:17 CST 2015

----- Original Message -----
> From: "John A. Biddiscombe" <biddisco at cscs.ch>
> To: "Hal Finkel" <hfinkel at anl.gov>
> Cc: llvm-bgq-discuss at alcf.anl.gov
> Sent: Tuesday, February 3, 2015 9:51:40 AM
> Subject: RE: [Llvm-bgq-discuss] 2 quick questions about upgrading
> 
> Thanks Hal,
> 
> I upgraded the bgclang and have two problems
> 
> 1) I had to remove the Xassembler  ma2q flag from the bgclang script
> (I think I also removed another but didn't keep the original so I've
> forgotten what it was, oops)
>        script_params="-Xassembler -ma2q $script_params"
> Does this matter for anything much?

No, are you using an old wrapper script? By default clang now uses its built-in assembler (and, thus, the assembler flags are no needed). However, it also should not add them by default, so I'm not sure what's going on here.

> 
> 2) compiling statically seems ok, c and c++ with and without mpi run
> as expected and produce standard hello world results, but I need to
> use -dynamic to pull in some libs for non simple projects and then
> things segfault on startup.
> (NB. using flag -std=c++11 - in the make verbose output I can see a
> stdlib=libstdc++ which I didn't put in there, so I assume the
> bgclang wrapper added for me)

Yes, I've seen this problem before, it comes from not picking up the correct dynamic loader. I think that, again, this is a result of you using an old wrapper script. If you're using the new one, then we'll need to investigate further. In short, when you run ldd, you should not see this:

 	/lib64/ld64.so.1 (0x0000000054d50000)

but should see a mapping path to the loader associated with the glibc version you're actually using (in V1R2M2_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib).

 -Hal

> 
> If you have any suggestions for the -dynamic, please say. I attach
> info below for completeness.
> 
> Many thanks in advance.
> 
> JB
> 
> I've put the base_4.7.2 tarball here
> /gpfs/bbp.cscs.ch/home/biddisco/clang/V1R2M2_base_4.7.2
> and the clang compiler is installed in
> /gpfs/bbp.cscs.ch/home/biddisco/apps/bgclang/r220548-20141024
> the only thing I'm getting from /bgsys/drivers is the comm mpi stuff
> AFAICT.
> 
> the segfault is in the kernel stuff
> #0  0x00000fffb7997b9c in _dl_vdso_vsym (name=0xfffb79aca28
> "__kernel_gettimeofday", vers=0xfffffffe650) at
> ../sysdeps/unix/sysv/linux/dl-vdso.c:40
> #1  0x00000fffb7875c50 in _libc_vdso_platform_setup (argc=Unhandled
> dwarf expression opcode 0xf3) at
> ../sysdeps/unix/sysv/linux/powerpc/init-first.c:37
> #2  _init (argc=Unhandled dwarf expression opcode 0xf3) at
> ../csu/init-first.c:86
> #3  0x00000080bd227bf0 in ._dl_init_internal () from /lib64/ld64.so.1
> #4  0x00000080bd2154dc in ._dl_start_user () from /lib64/ld64.so.1
> 
> and the libs being pulled in are (I've wiped LD_LIBRARY_PATH at this
> stage to make sure we don't get anything old and dodgy)
> bbpbg1:~/bgq/build/bgas$ ldd bin/hello
> 	linux-vdso64.so.1 =>  (0x00000fff8de20000)
> 	libc++.so.1 =>
> 	/gpfs/bbp.cscs.ch/home/biddisco/apps/bgclang/r220548-20141024/libc++/lib/libc++.so.1
> 	(0x00000fff8dd00000)
> 	libstdc++.so.6 =>
> 	/gpfs/bbp.cscs.ch/home/biddisco/clang/V1R2M2_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib/libstdc++.so.6
> 	(0x00000fff8dad0000)
> 	libm.so.6 =>
> 	/gpfs/bbp.cscs.ch/home/biddisco/clang/V1R2M2_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib/libm.so.6
> 	(0x00000fff8d9b0000)
> 	libgcc_s.so.1 =>
> 	/gpfs/bbp.cscs.ch/home/biddisco/clang/V1R2M2_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib/libgcc_s.so.1
> 	(0x00000fff8d8a0000)
> 	libc.so.6 =>
> 	/gpfs/bbp.cscs.ch/home/biddisco/clang/V1R2M2_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib/libc.so.6
> 	(0x00000fff8d670000)
> 	libpthread.so.0 =>
> 	/gpfs/bbp.cscs.ch/home/biddisco/clang/V1R2M2_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib/libpthread.so.0
> 	(0x00000fff8d550000)
> 	librt.so.1 =>
> 	/gpfs/bbp.cscs.ch/home/biddisco/clang/V1R2M2_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib/librt.so.1
> 	(0x00000fff8d430000)
> 	/lib64/ld64.so.1 (0x0000000054d50000)
> 
> which look ok to my untrained eye.
> 
> the make looks like this (this is from a project which includes some
> oost stuff etc using cmake so it's a bit bloated, but I can't see
> anything bad in there)
> 
>  [100%] Building CXX object cpptest/CMakeFiles/hello.dir/hello.cxx.o
> /gpfs/bbp.cscs.ch/home/biddisco/apps/bgclang/r220548-20141024/bin/clang++
> -Xclang -isystem -Xclang
> /gpfs/bbp.cscs.ch/home/biddisco/apps/bgclang/r220548-20141024/libc++/include
> -Xclang -isystem -Xclang
> /bgsys/drivers/V1R2M2/ppc64//comm/gcc/include -Xclang -isystem
> -Xclang /bgsys/drivers/V1R2M2/ppc64//comm/lib/gnu -Xclang -isystem
> -Xclang /bgsys/drivers/V1R2M2/ppc64/ -Xclang -isystem -Xclang
> /bgsys/drivers/V1R2M2/ppc64//comm/sys/include -Xclang -isystem
> -Xclang /bgsys/drivers/V1R2M2/ppc64//spi/include -Xclang -isystem
> -Xclang /bgsys/drivers/V1R2M2/ppc64//spi/include/kernel/cnk
> -I/gpfs/bbp.cscs.ch/home/biddisco/apps/bgclang/r220548-20141024/sleef/include
> -Xclang -isystem -Xclang
> /gpfs/bbp.cscs.ch/home/biddisco/apps/bgclang/r220548-20141024/libstdc++fixup/include
> -Xclang -isystem -Xclang
> /gpfs/bbp.cscs.ch/home/biddisco/clang/V1R2M2_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/include/c++/4.7.2
> -Xclang -isystem -Xclang
> /gpfs/bbp.cscs.ch/home/biddisco/clang/V1R2M2_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/include/c++/4.7.2/powerpc64-bgq-linux
> -Xclang -isystem -Xclang
> /gpfs/bbp.cscs.ch/home/biddisco/clang/V1R2M2_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/sys-include
> -target powerpc64-bgq-linux -gcc-toolchain
> /gpfs/bbp.cscs.ch/home/biddisco/clang/V1R2M2_base_4.7.2/gnu-linux-4.7.2
> -B/gpfs/bbp.cscs.ch/home/biddisco/apps/bgclang/r220548-20141024/binutils/bin
> -B/gpfs/bbp.cscs.ch/home/biddisco/clang/V1R2M2_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib
> -mcpu=a2q -ffp-contract=fast -fno-math-errno -stdlib=libc++ -static
> -D__bgclang__=1 -D__bgclang_version__="r220548-20141024" -mllvm
> -optimize-regalloc -mllvm -fast-isel=0 -D__BGQ__ -std=c++11 -g
> -I/gpfs/bbp.cscs.ch/home/biddisco/apps/bgq/boost_1_56_0/include/boost-1_56
> -I/bgsys/drivers/V1R2M2/ppc64/comm/include -o
> CMakeFiles/hello.dir/hello.cxx.o -c
> /gpfs/bbp.cscs.ch/home/biddisco/src/bgas/cpptest/hello.cxx
> Linking CXX executable ../bin/hello
> /gpfs/bbp.cscs.ch/home/biddisco/apps/bgclang/r220548-20141024/bin/clang++
> -target powerpc64-bgq-linux -gcc-toolchain
> /gpfs/bbp.cscs.ch/home/biddisco/clang/V1R2M2_base_4.7.2/gnu-linux-4.7.2
> -B/gpfs/bbp.cscs.ch/home/biddisco/apps/bgclang/r220548-20141024/binutils/bin
> -B/gpfs/bbp.cscs.ch/home/biddisco/clang/V1R2M2_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib
> -mcpu=a2q -ffp-contract=fast -fno-math-errno -stdlib=libc++
> -L/gpfs/bbp.cscs.ch/home/biddisco/apps/bgclang/r220548-20141024/libc++/lib
> -Wl,-rpath
> -Wl,/gpfs/bbp.cscs.ch/home/biddisco/apps/bgclang/r220548-20141024/libc++/lib
> -L/gpfs/bbp.cscs.ch/home/biddisco/clang/V1R2M2_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib
> -Wl,-rpath
> -Wl,/gpfs/bbp.cscs.ch/home/biddisco/clang/V1R2M2_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib
> -std=c++11 -g -dynamic CMakeFiles/hello.dir/hello.cxx.o -o
> ../bin/hello -rdynamic
> -L/gpfs/bbp.cscs.ch/home/biddisco/apps/bgclang/r220548-20141024/sleef/lib
> -Wl,-rpath
> -Wl,/gpfs/bbp.cscs.ch/home/biddisco/apps/bgclang/r220548-20141024/sleef/lib
> -Wl,--as-needed -lsleef -Wl,--no-as-needed
> [100%] Built target hello
> 
> 

-- 
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory