[Llvm-bgq-discuss] mpic++11 Runtime Error on Cetus

[Hal Finkel]

----- Original Message -----
> From: "Michael Collins" <mikeliamcollins at gmail.com>
> To: "Hal Finkel" <hfinkel at anl.gov>
> Cc: llvm-bgq-discuss at lists.alcf.anl.gov, "Patrick E Gartung" <gartung at fnal.gov>
> Sent: Sunday, August 14, 2016 1:19:24 PM
> Subject: Re: [Llvm-bgq-discuss] mpic++11 Runtime Error on Cetus
> 
> 
> I was using mpiwrapper-bgclang-mpi3. Thanks for the help, Hal!
> 

Can you send me the command line you used to link the final executable? Or were you linking a shared library?

Regardless, can you run ldd on it and send me the output (you can do this off-list).

Thanks again,
Hal

> 
> On Sun, Aug 14, 2016 at 2:07 PM, Hal Finkel < hfinkel at anl.gov >
> wrote:
> 
> 
> ----- Original Message -----
> > From: "Michael Collins" < mikeliamcollins at gmail.com >
> > To: "Hal Finkel" < hfinkel at anl.gov >
> > Cc: llvm-bgq-discuss at lists.alcf.anl.gov , "Patrick E Gartung" <
> > gartung at fnal.gov >
> > Sent: Sunday, August 14, 2016 12:59:18 PM
> > Subject: Re: [Llvm-bgq-discuss] mpic++11 Runtime Error on Cetus
> > 
> > 
> > If I remember correctly, I had to manually add to LD_LIBRARY_PATH.
> 
> Understood. This shouldn't be necessary, however. Are you linking
> with bgclang, or one of its MPI wrapper scripts?
> 
> I'm worried that something you're doing (or whatever build system
> you're using is doing) is confusing the wrapper script so that it
> does not add the appropriate linker options -- in this case, I'd
> like to improve the wrapper script so it does not have this problem.
> 
> Thanks again,
> Hal
> 
> 
> 
> > 
> > 
> > On Sun, Aug 14, 2016 at 1:51 PM, Hal Finkel < hfinkel at anl.gov >
> > wrote:
> > 
> > 
> > Hi Michael,
> > 
> > How are you linking? The underlying bgclang wrapper script should
> > add
> > the necessary directories
> > (/bgsys/drivers/ppcfloor/gnu-linux-4.7.2/powerpc64-bgq-linux/lib in
> > this case) to your rpath so everything functions correctly.
> > 
> > -Hal
> > 
> > 
> > 
> > ----- Original Message -----
> > > From: "Michael Collins" < mikeliamcollins at gmail.com >
> > > To: "Patrick E Gartung" < gartung at fnal.gov >
> > > Cc: llvm-bgq-discuss at lists.alcf.anl.gov
> > > Sent: Monday, July 18, 2016 10:32:14 PM
> > > Subject: Re: [Llvm-bgq-discuss] mpic++11 Runtime Error on Cetus
> > > 
> > > 
> > > 
> > > Well, it took most of the day, but I finally found a libstdc++.so
> > > that works. The path is as follows:
> > > 
> > > 
> > > /bgsys/drivers/V1R2M1/ppc64/toolchain-4.7.2/gnu/build-powerpc64-bgq-linux/gcc-4.7.2-build/powerpc64-bgq-linux/libstdc++-v3/src/.libs/libstdc++.so.6
> > > 
> > > 
> > > Now, the program runs exactly as intended. Thanks so much for the
> > > help, Patrick.
> > > 
> > > 
> > > On Mon, Jul 18, 2016 at 1:25 PM, Patrick E Gartung <
> > > gartung at fnal.gov
> > > > wrote:
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > Probably this path or something similar
> > > 
> > > 
> > > 
> > > 
> > > /bgsys/drivers/toolchain/V1R2M4_base_4.7.2/gnu-linux-4.7.2/powerpc64-bgq-linux/lib/
> > > 
> > > 
> > > 
> > > 
> > > From: Michael Collins < mikeliamcollins at gmail.com >
> > > Date: Monday, July 18, 2016 at 1:11 PM
> > > 
> > > 
> > > To: Patrick Gartung < gartung at fnal.gov >
> > > Cc: " llvm-bgq-discuss at lists.alcf.anl.gov " <
> > > llvm-bgq-discuss at lists.alcf.anl.gov >
> > > Subject: Re: [Llvm-bgq-discuss] mpic++11 Runtime Error on Cetus
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > OK, I'll dig around a bit and see if I can find the libstdc++.so
> > > I
> > > need.
> > > 
> > > 
> > > On Mon, Jul 18, 2016 at 1:02 PM, Patrick E Gartung <
> > > gartung at fnal.gov
> > > > wrote:
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > My solution only worked because I compiled with gcc4.8.4 in the
> > > mpich_cpp wrapper. I think bgclang is compiled against a
> > > different
> > > libstdc++.so. This may be the cause of the segfault. You will
> > > have
> > > to determine the correct LD_LIBRARY_PATH to use with bgclang.
> > > 
> > > 
> > > 
> > > 
> > > From: Michael Collins < mikeliamcollins at gmail.com >
> > > Date: Monday, July 18, 2016 at 12:57 PM
> > > To: Patrick Gartung < gartung at fnal.gov >
> > > Cc: " llvm-bgq-discuss at lists.alcf.anl.gov " <
> > > llvm-bgq-discuss at lists.alcf.anl.gov >
> > > Subject: Re: [Llvm-bgq-discuss] mpic++11 Runtime Error on Cetus
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > That seems to have done the trick. My program's segfault-ing, but
> > > at
> > > least that means the library was loaded.
> > > 
> > > 
> > > On Mon, Jul 18, 2016 at 12:22 PM, Michael Collins <
> > > mikeliamcollins at gmail.com > wrote:
> > > 
> > > 
> > > 
> > > Thanks, Patrick. I'll give it a try and report back.
> > > 
> > > 
> > > 
> > > 
> > > On Mon, Jul 18, 2016 at 12:20 PM, Patrick E Gartung <
> > > gartung at fnal.gov > wrote:
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > Hi Michael
> > > 
> > > 
> > > I had the same problem on vesta. I had to add the envs option
> > > with
> > > LD_LIBRARY_PATH to the qsub command
> > > 
> > > 
> > > 
> > > 
> > > qsub -n 2 --mode c1 -t 120 --env
> > > LD_LIBRARY_PATH=/soft/compilers/gcc/4.8.4/lib/gcc/powerpc64-bgq-linux/4.8.4/:/soft/compilers/gcc/4.8.4/lib/gcc/:/soft/compilers/gcc/4.8.4/lib/gcc/powerpc64-bgq-linux/4.8.4/../../../../powerpc64-bgq-linux/lib/:$LD_LIBRARY_PATH
> > > 
> > > 
> > > 
> > > 
> > > From: < llvm-bgq-discuss-bounces at lists.alcf.anl.gov > on behalf
> > > of
> > > Michael Collins < mikeliamcollins at gmail.com >
> > > Date: Monday, July 18, 2016 at 12:14 PM
> > > To: " llvm-bgq-discuss at lists.alcf.anl.gov " <
> > > llvm-bgq-discuss at lists.alcf.anl.gov >
> > > Subject: [Llvm-bgq-discuss] mpic++11 Runtime Error on Cetus
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > Hello, all.
> > > 
> > > 
> > > I'm having some trouble running my Swift/T project on Cetus. I'm
> > > compiling a shared library with mpic++11 under the
> > > mpiwrapper-bclang-mpi3 wrapper, and my program is dynamically
> > > linked
> > > to it. At runtime, I'm getting the following error when my
> > > program
> > > tries to load the shared library:
> > > 
> > > 
> > > 
> > > stderr[1]: couldn't load file
> > > "/home/mlcoll/src/eq-c/tests/genmax/EQC/libeqc.so":
> > > /lib64/bgq/libstdc++.so.6: version `CXXABI_1.3.5' not found
> > > (required by
> > > /soft/compilers/bgclang/r266865-stable/libc++/lib/libc++.so.1)
> > > 
> > > 
> > > Any ideas on how to remedy this?
> > > 
> > > 
> > > Thanks,
> > > Mike Collins
> > > 
> > > 
> > > 
> > > 
> > > _______________________________________________
> > > llvm-bgq-discuss mailing list
> > > llvm-bgq-discuss at lists.alcf.anl.gov
> > > https://lists.alcf.anl.gov/mailman/listinfo/llvm-bgq-discuss
> > > 
> > 
> > --
> > Hal Finkel
> > Assistant Computational Scientist
> > Leadership Computing Facility
> > Argonne National Laboratory
> > 
> > 
> 
> --
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory
> 
> 

-- 
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory