[Llvm-bgq-discuss] Dynamic linking failure

[Phil Miller]

I've compiled my application using bgclang/bgclang++ on Vesta, and the
process goes smoothly. When I use a static linked build of the system, it
runs cleanly.

I want to try out Address Sanitizer (aka 'asan', activated with
'-fsanitize=address'), which requires dynamic linking. Sadly, that lets me
compile and link, but fails to run. Here's what I'm seeing, again on Vesta:

===============
$ file check
check: ELF 64-bit MSB executable, 64-bit PowerPC or cisco 7500, version 1
(SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.4.21, not
stripped

$ echo $LD_LIBRARY_PATH  | tr : '\n'
/bgsys/drivers/ppcfloor/comm/lib
/bgsys/drivers/ppcfloor/comm/gcc/lib
/soft/compilers/ibmcmp-feb2014/vac/bg/12.1/bglib64
/soft/compilers/ibmcmp-feb2014/vacpp/bg/12.1/bglib64
/soft/compilers/ibmcmp-feb2014/xlf/bg/14.1/bglib64
/soft/compilers/ibmcmp-feb2014/xlmass/bg/7.3/bglib64
/soft/compilers/ibmcmp-feb2014/xlsmp/bg/3.1/bglib64
/dbhome/db2cat/sqllib/lib64
/dbhome/db2cat/sqllib/lib32

$ ldd check
    linux-vdso64.so.1 =>  (0x00000fff9ae40000)
    libdl.so.2 =>
/bgsys/drivers/toolchain/V1R2M1_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib/libdl.so.2
(0x00000fff9ad20000)
    libpami-gcc.so => /bgsys/drivers/ppcfloor/comm/lib/libpami-gcc.so
(0x00000fff9a7b0000)
    libpthread.so.0 =>
/bgsys/drivers/toolchain/V1R2M1_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib/libpthread.so.0
(0x00000fff9a690000)
    librt.so.1 =>
/bgsys/drivers/toolchain/V1R2M1_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib/librt.so.1
(0x00000fff9a560000)
    libm.so.6 =>
/bgsys/drivers/toolchain/V1R2M1_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib/libm.so.6
(0x00000fff9a440000)
    libstdc++.so.6 =>
/bgsys/drivers/toolchain/V1R2M1_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib/libstdc++.so.6
(0x00000fff9a210000)
    libgcc_s.so.1 =>
/bgsys/drivers/toolchain/V1R2M1_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib/libgcc_s.so.1
(0x00000fff9a100000)
    libc.so.6 =>
/bgsys/drivers/toolchain/V1R2M1_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib/libc.so.6
(0x00000fff99ed0000)
    /lib64/ld64.so.1 (0x000000003cc80000)

$ qsub -t 10 -A PARTS -n 1 --mode c1 ./check
190745

$ cat 190745.error
2014-05-08 18:45:16.828 (INFO ) [0x40000a3bc20] 27642:tatu.runjob.client:
scheduler job id is 190745
2014-05-08 18:45:16.829 (DEBUG) [0x40000a3bc20] 27642:tatu.runjob.client:
the environment variable COBALT_RESID did not contain a Cobalt reservation
id
2014-05-08 18:45:16.844 (INFO ) [0x400004034d0] 27642:tatu.runjob.monitor:
monitor started
2014-05-08 18:45:16.855 (INFO ) [0x40000a3bc20]
27642:ibm.runjob.AbstractOptions: using properties file
/bgsys/local/etc/bg.properties
2014-05-08 18:45:16.856 (INFO ) [0x40000a3bc20]
27642:ibm.runjob.AbstractOptions: max open file descriptors: 65536
2014-05-08 18:45:16.856 (INFO ) [0x40000a3bc20]
27642:ibm.runjob.AbstractOptions: core file limit: 18446744073709551615
2014-05-08 18:45:16.977 (INFO ) [0x400004034d0] 27642:tatu.runjob.monitor:
task record 645048 created
2014-05-08 18:45:16.978 (INFO ) [0x40000a3bc20]
VST-20420-31531-32:27642:ibm.runjob.client.options.Parser: set local socket
to runjob_mux from properties file
2014-05-08 18:45:17.782 (INFO ) [0x400004034d0] 27642:tatu.runjob.monitor:
tracklib completed
2014-05-08 18:45:19.162 (INFO ) [0x40000a3bc20]
VST-20420-31531-32:848093:ibm.runjob.client.Job: job 848093 started
/gpfs/vesta-home/phil/charm-6.6/pamilrts-bluegeneq-asan-clang/tests/util/./check:
error while loading shared libraries: libpami-gcc.so: cannot open shared
object file: No such file or directory
2014-05-08 18:45:20.952 (INFO ) [0x40000a3bc20]
VST-20420-31531-32:848093:ibm.runjob.client.Job: exited with status 127
2014-05-08 18:45:20.952 (WARN ) [0x40000a3bc20]
VST-20420-31531-32:848093:ibm.runjob.client.Job: normal termination with
status 127 from rank 0
2014-05-08 18:45:20.952 (INFO ) [0x40000a3bc20] tatu.runjob.client: task
exited with status 127
2014-05-08 18:45:20.952 (INFO ) [0x400004034d0] 27642:tatu.runjob.monitor:
monitor terminating
2014-05-08 18:45:20.956 (INFO ) [0x40000a3bc20] tatu.runjob.client: monitor
completed
=========

Have a missed a step in running dynamically linked binaries on BG/Q?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alcf.anl.gov/pipermail/llvm-bgq-discuss/attachments/20140508/30106863/attachment.html>