[Llvm-bgq-discuss] Dynamic linking failure

[Hal Finkel]

----- Original Message -----
> From: "Phil Miller" <mille121 at illinois.edu>
> To: llvm-bgq-discuss at lists.alcf.anl.gov
> Sent: Thursday, May 8, 2014 1:53:10 PM
> Subject: [Llvm-bgq-discuss] Dynamic linking failure
> 
> 
> 
> 
> I've compiled my application using bgclang/bgclang++ on Vesta, and
> the process goes smoothly. When I use a static linked build of the
> system, it runs cleanly.
> 
> I want to try out Address Sanitizer (aka 'asan', activated with
> '-fsanitize=address'), which requires dynamic linking. Sadly, that
> lets me compile and link, but fails to run. Here's what I'm seeing,
> again on Vesta:
> 
> ===============
> $ file check
> check: ELF 64-bit MSB executable, 64-bit PowerPC or cisco 7500,
> version 1 (SYSV), dynamically linked (uses shared libs), for
> GNU/Linux 2.4.21, not stripped
> 
> $ echo $LD_LIBRARY_PATH | tr : '\n'
> /bgsys/drivers/ppcfloor/comm/lib
> /bgsys/drivers/ppcfloor/comm/gcc/lib
> /soft/compilers/ibmcmp-feb2014/vac/bg/12.1/bglib64
> /soft/compilers/ibmcmp-feb2014/vacpp/bg/12.1/bglib64
> /soft/compilers/ibmcmp-feb2014/xlf/bg/14.1/bglib64
> /soft/compilers/ibmcmp-feb2014/xlmass/bg/7.3/bglib64
> /soft/compilers/ibmcmp-feb2014/xlsmp/bg/3.1/bglib64
> /dbhome/db2cat/sqllib/lib64
> /dbhome/db2cat/sqllib/lib32
> 
> $ ldd check
> linux-vdso64.so.1 => (0x00000fff9ae40000)
> libdl.so.2 =>
> /bgsys/drivers/toolchain/V1R2M1_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib/libdl.so.2
> (0x00000fff9ad20000)
> libpami-gcc.so => /bgsys/drivers/ppcfloor/comm/lib/libpami-gcc.so
> (0x00000fff9a7b0000)
> libpthread.so.0 =>
> /bgsys/drivers/toolchain/V1R2M1_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib/libpthread.so.0
> (0x00000fff9a690000)
> librt.so.1 =>
> /bgsys/drivers/toolchain/V1R2M1_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib/librt.so.1
> (0x00000fff9a560000)
> libm.so.6 =>
> /bgsys/drivers/toolchain/V1R2M1_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib/libm.so.6
> (0x00000fff9a440000)
> libstdc++.so.6 =>
> /bgsys/drivers/toolchain/V1R2M1_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib/libstdc++.so.6
> (0x00000fff9a210000)
> libgcc_s.so.1 =>
> /bgsys/drivers/toolchain/V1R2M1_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib/libgcc_s.so.1
> (0x00000fff9a100000)
> libc.so.6 =>
> /bgsys/drivers/toolchain/V1R2M1_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib/libc.so.6
> (0x00000fff99ed0000)
> /lib64/ld64.so.1 (0x000000003cc80000)
> 
> $ qsub -t 10 -A PARTS -n 1 --mode c1 ./check
> 190745
> 
> $ cat 190745.error
> 2014-05-08 18:45:16.828 (INFO ) [0x40000a3bc20]
> 27642:tatu.runjob.client: scheduler job id is 190745
> 2014-05-08 18:45:16.829 (DEBUG) [0x40000a3bc20]
> 27642:tatu.runjob.client: the environment variable COBALT_RESID did
> not contain a Cobalt reservation id
> 2014-05-08 18:45:16.844 (INFO ) [0x400004034d0]
> 27642:tatu.runjob.monitor: monitor started
> 2014-05-08 18:45:16.855 (INFO ) [0x40000a3bc20]
> 27642:ibm.runjob.AbstractOptions: using properties file
> /bgsys/local/etc/bg.properties
> 2014-05-08 18:45:16.856 (INFO ) [0x40000a3bc20]
> 27642:ibm.runjob.AbstractOptions: max open file descriptors: 65536
> 2014-05-08 18:45:16.856 (INFO ) [0x40000a3bc20]
> 27642:ibm.runjob.AbstractOptions: core file limit:
> 18446744073709551615
> 2014-05-08 18:45:16.977 (INFO ) [0x400004034d0]
> 27642:tatu.runjob.monitor: task record 645048 created
> 2014-05-08 18:45:16.978 (INFO ) [0x40000a3bc20]
> VST-20420-31531-32:27642:ibm.runjob.client.options.Parser: set local
> socket to runjob_mux from properties file
> 2014-05-08 18:45:17.782 (INFO ) [0x400004034d0]
> 27642:tatu.runjob.monitor: tracklib completed
> 2014-05-08 18:45:19.162 (INFO ) [0x40000a3bc20]
> VST-20420-31531-32:848093:ibm.runjob.client.Job: job 848093 started
> /gpfs/vesta-home/phil/charm-6.6/pamilrts-bluegeneq-asan-clang/tests/util/./check:
> error while loading shared libraries: libpami-gcc.so: cannot open
> shared object file: No such file or directory
> 2014-05-08 18:45:20.952 (INFO ) [0x40000a3bc20]
> VST-20420-31531-32:848093:ibm.runjob.client.Job: exited with status
> 127
> 2014-05-08 18:45:20.952 (WARN ) [0x40000a3bc20]
> VST-20420-31531-32:848093:ibm.runjob.client.Job: normal termination
> with status 127 from rank 0
> 2014-05-08 18:45:20.952 (INFO ) [0x40000a3bc20] tatu.runjob.client:
> task exited with status 127
> 2014-05-08 18:45:20.952 (INFO ) [0x400004034d0]
> 27642:tatu.runjob.monitor: monitor terminating
> 2014-05-08 18:45:20.956 (INFO ) [0x40000a3bc20] tatu.runjob.client:
> monitor completed
> =========
> 
> Have a missed a step in running dynamically linked binaries on BG/Q?

No, I don't think so. I see no reason why this should not work. Can you e-mail support about this?

 -Hal

> 
> _______________________________________________
> llvm-bgq-discuss mailing list
> llvm-bgq-discuss at lists.alcf.anl.gov
> https://lists.alcf.anl.gov/mailman/listinfo/llvm-bgq-discuss
> 

-- 
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory