[Llvm-bgq-discuss] Details behind MPI wrapper for bgclang++

Thu Feb 28 18:51:08 CST 2013

Thanks Hal, that helps clear things up a bit. I guess I should be a little
more clear about what exactly I'm doing. Since CMake's FindMPI module seems
to attempt to link to MPICH incorrectly if I directly specify the mpic++11,
I have to instead manually specify its behaviour (and thankfully Jeff took
care of this a month or two ago). One of my link commands generated by
CMake looks like this:

/home/projects/llvm/bin/bgclang++   -Wall -std=c++11  -O3 -stdlib=libc++
 -L/bgsys/drivers/ppcfloor/comm/gcc/lib
-L/bgsys/drivers/ppcfloor/comm/sys/lib -L/bgsys/drivers/ppcfloor/spi/lib
CMakeFiles/HypRadon-2d.dir/test/transform/HypRadon-2d.cpp.o  -o
bin/transform/HypRadon-2d -rdynamic libcmake-dummy-lib.a
-L/soft/libraries/alcf/current/gcc/LAPACK/lib -llapack
-L/soft/libraries/essl/current/essl/5.1/lib64 -lesslbg
-L/soft/compilers/ibmcmp-nov2012/xlf/bg/14.1/bglib64 -lxlf90_r
-L/soft/compilers/ibmcmp-nov2012/xlsmp/bg/3.1/bglib64 -lxlomp_ser
-L/soft/compilers/ibmcmp-nov2012/xlmass/bg/7.3/bglib64 -lmassv -lmass
-lxlopt -lxlfmath -lxl -lgfortran -lm -lpthread -ldl
-Wl,--allow-multiple-definition -lcxxmpich -lmpich -lopa -lmpl -ldl -lpami
-lSPI -lSPI_cnk -lpthread -lrt -lstdc++

Upon typing this, I noticed that both libc++ and libstdc++ are both being
used (which I assume is bad). However, I believe that libstdc++ is depended
upon by PAMI. I tried changing my build to replace the libstdc++ library
with /home/projects/llvm/libc++/lib/libc++.so and received a long list of
undefined references in PAMI to various entities in the std namespace. For
example,

/bgsys/drivers/ppcfloor/comm/sys/lib/libpami.a(component-pami--COLLECTIVESGEOM.cnk.o):(.toc+0x888):
undefined reference to `std::ios_base::Init::~Init()'
/bgsys/drivers/ppcfloor/comm/sys/lib/libpami.a(component-pami--PROGRESS_EXTENSION.cnk.o):
In function `__static_initialization_and_destruction_0':
/bgsys/drivers/toolchain/V1R2M0/gnu-linux/lib/gcc/powerpc64-bgq-linux/4.4.6/../../../../powerpc64-bgq-linux/include/c++/4.4.6/iostream:72:
undefined reference to `std::ios_base::Init::Init()'
/bgsys/drivers/ppcfloor/comm/sys/lib/libpami.a(component-pami--PROGRESS_EXTENSION.cnk.o):(.toc+0x38):
undefined reference to `std::ios_base::Init::~Init()'

Any ideas?

Jack

On Thu, Feb 28, 2013 at 4:36 PM, Hal Finkel <hfinkel at anl.gov> wrote:

>
>
> ----- Original Message -----
> > From: "Jack Poulson" <jack.poulson at gmail.com>
> > To: "Hal Finkel" <hfinkel at anl.gov>
> > Cc: "Jeff Hammond" <jhammond at alcf.anl.gov>,
> llvm-bgq-discuss at lists.alcf.anl.gov
> > Sent: Thursday, February 28, 2013 6:12:21 PM
> > Subject: Re: [Llvm-bgq-discuss] Details behind MPI wrapper for bgclang++
> >
> > Hi Hal,
> >
> >
> > As far as I can tell, there are only two distinct wrappers in the
> > /home/projects/llvm/mpi/clang/bin folder, mpicc and mpicxx, and all
> > of the others are simply symlinks to them. Is it possible that I am
> > misunderstanding?
>
> You're correct, but the wrapper scripts alter their behavior slightly
> depending on the invoking name. Specifically, if you use mpic++ then you
> get the GNU C++03 mode by default with libstdc++ as the primary C++ STL
> implementation (and you still get libstdc++ in C++11 mode if you specify
> -std=c++11, etc.). If you use the mpic++11 wrapper, then you get libc++ as
> the primary C++ STL implementation (and the GNU C++11 mode by default). The
> libc++ implementation, being much more recent than the system libstdc++,
> provides significantly better C++11 support.
>
> Can you give me access to the directory with the core files so that I can
> take a look?
>
> Thanks again,
> Hal
>
> >
> >
> > I am using the mpicxx wrapper.
> >
> >
> > Jack
> >
> >
> > On Thu, Feb 28, 2013 at 4:08 PM, Hal Finkel < hfinkel at anl.gov >
> > wrote:
> >
> >
> > Jack,
> >
> > What Jeff said is correct. Jeff, I did not specifically update the
> > wrapper scripts after the V1R2 transition. Is there anything that I
> > need to do for that?
> >
> > Also, Jack, are you using the mpic++ (with -std=c++11) wrapper or the
> > mpic++11 wrapper?
> >
> > -Hal
> >
> >
> >
> > ----- Original Message -----
> > > From: "Jeff Hammond" < jhammond at alcf.anl.gov >
> > > To: "Jack Poulson" < jack.poulson at gmail.com >
> > > Cc: llvm-bgq-discuss at lists.alcf.anl.gov
> > > Sent: Thursday, February 28, 2013 5:59:08 PM
> > > Subject: Re: [Llvm-bgq-discuss] Details behind MPI wrapper for
> > > bgclang++
> > >
> > > IBM does not provide MPI builds as part of the driver because they
> > > do
> > > not - and are not required to - provide LLVM builds of MPI.
> > >
> > > ALCF can provide /home/projects/pami/llvm/mpi.... if you want an
> > > MPI
> > > build using LLVM. I will do this next week during MiraCon as part
> > > of
> > > building MPICH2 1.5 + IBM patches from source.
> > >
> > > Best,
> > >
> > > Jeff
> > >
> > > On Thu, Feb 28, 2013 at 6:56 PM, Jack Poulson
> > > < jack.poulson at gmail.com > wrote:
> > > > Hello All,
> > > >
> > > > I am trying to run a C++11 code on BGQ and have been running into
> > > > strange
> > > > sporadic segfaults with non-helpful stack traces. I started
> > > > looking
> > > > through
> > > > the details of the driver
> > > > /home/projects/llvm/mpi/clang/bin/mpicxx and was
> > > > surprised by the 31st line:
> > > > prefix=/bgsys/drivers/ppcfloor/comm/gcc
> > > > When I look in the /bgsys/drivers/ppcfloor/comm folder, I notice
> > > > that there
> > > > are directories for gcc and xl, but none for Clang. Is there some
> > > > clear
> > > > explanation for this?
> > > >
> > > > Thanks!
> > > > Jack
> > > >
> > > > _______________________________________________
> > > > llvm-bgq-discuss mailing list
> > > > llvm-bgq-discuss at lists.alcf.anl.gov
> > > > https://lists.alcf.anl.gov/mailman/listinfo/llvm-bgq-discuss
> > > >
> > >
> > >
> > >
> > > --
> > > Jeff Hammond
> > > Argonne Leadership Computing Facility
> > > University of Chicago Computation Institute
> > > jhammond at alcf.anl.gov / (630) 252-5381
> > > http://www.linkedin.com/in/jeffhammond
> > > https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond
> > > _______________________________________________
> > > llvm-bgq-discuss mailing list
> > > llvm-bgq-discuss at lists.alcf.anl.gov
> > > https://lists.alcf.anl.gov/mailman/listinfo/llvm-bgq-discuss
> > >
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alcf.anl.gov/pipermail/llvm-bgq-discuss/attachments/20130228/869b33dc/attachment.html>