<html><head><style>body{font-family:Helvetica,Arial;font-size:13px}</style></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">Oops, hit the “send” instead of “attach” button by mistake. Here we go.</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">The output with xlc at all optimization levels, and clang at -O0 is "v: 1 2 3 4”.</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">With clang and -O3 I get “v: 0.0078125 32 0 0”</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">Cheers,</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">E.</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><br></div> <div class="airmail_ext_on" style="color:black"><br>From: <span style="color:black">Erik Lindahl</span> <a href="mailto:erik.lindahl@gmail.com"><erik.lindahl@gmail.com></a><br>Reply: <span style="color:black">Erik Lindahl</span> <a href="mailto:erik.lindahl@gmail.com"><erik.lindahl@gmail.com>></a><br>Date: <span style="color:black">10 May 2015 at 17:40:02</span><br>To: <span style="color:black">Hal Finkel</span> <a href="mailto:hfinkel@anl.gov"><hfinkel@anl.gov>></a><br>Cc: <span style="color:black">Mark Abraham</span> <a href="mailto:mark.j.abraham@gmail.com"><mark.j.abraham@gmail.com>></a>, <span style="color:black">llvm-bgq-discuss@lists.alcf.anl.gov</span> <a href="mailto:llvm-bgq-discuss@lists.alcf.anl.gov"><llvm-bgq-discuss@lists.alcf.anl.gov>></a><br>Subject: <span style="color:black"> Re: [Llvm-bgq-discuss] New bgclang nighty builds (and other updates) <br></span></div><br> <blockquote type="cite" class="clean_bq"><span><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div></div><div>
<title></title>
<div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">
Hi Hal,</div>
<div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">
<br></div>
<div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">
See attachment!</div>
<div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">
<br></div>
<div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">
<br></div>
<div class="airmail_ext_on" style="color:black"><br>
From: <span style="color:black">Hal Finkel</span> <a href="mailto:hfinkel@anl.gov"><hfinkel@anl.gov></a><br>
Reply: <span style="color:black">Hal Finkel</span> <a href="mailto:hfinkel@anl.gov"><hfinkel@anl.gov>></a><br>
Date: <span style="color:black">10 May 2015 at
17:22:52</span><br>
To: <span style="color:black">Erik Lindahl</span> <a href="mailto:erik.lindahl@gmail.com"><erik.lindahl@gmail.com>></a><br>
Cc: <span style="color:black">llvm-bgq-discuss@lists.alcf.anl.gov</span> <a href="mailto:llvm-bgq-discuss@lists.alcf.anl.gov"><llvm-bgq-discuss@lists.alcf.anl.gov>></a>,
<span style="color:black">Mark Abraham</span> <a href="mailto:mark.j.abraham@gmail.com"><mark.j.abraham@gmail.com>></a><br>
Subject: <span style="color:black">Re: [Llvm-bgq-discuss] New
bgclang nighty builds (and other updates)<br></span></div>
<br>
<blockquote type="cite" class="clean_bq">
<div>
<div><span>Hi Erik,<br>
<br>
If you could send this trivial piece of code, that would be greatly
appreciated.<br>
<br>
Thanks again,<br>
Hal<br>
<br>
----- Original Message -----<br>
> From: "Erik Lindahl" <erik.lindahl@gmail.com><br>
> To: "Hal Finkel" <hfinkel@anl.gov>, "Mark Abraham"
<mark.j.abraham@gmail.com><br>
> Cc: llvm-bgq-discuss@lists.alcf.anl.gov<br>
> Sent: Sunday, May 10, 2015 9:25:34 AM<br>
> Subject: Re: [Llvm-bgq-discuss] New bgclang nighty builds (and
other updates)<br>
><br>
><br>
> Hi,<br>
><br>
><br>
> Super-brief summary:<br>
><br>
><br>
> The bug occurs at high optimization (-O3) with the vec_ld()
or<br>
> vec_st() functions when calling the (overloaded) version
with<br>
> single-precision arguments (i.e., float pointers).<br>
><br>
><br>
> This is reproducible even with a trivial piece of code that
tries to<br>
> load from single precision memory and then write out the
contents of<br>
> the (double precision) vector4doubles by storing to a
double<br>
> variable.<br>
><br>
><br>
> - Everything works fine with the double-precision
functions,<br>
> regardless of optimization<br>
> - It works fine with clang in single precision without
optimization<br>
> - It works fine with xlc.<br>
><br>
><br>
> Cheers,<br>
><br>
><br>
> Erik<br>
><br>
><br>
><br>
><br>
> From: Mark Abraham <mark.j.abraham@gmail.com><br>
> Reply: Mark Abraham <mark.j.abraham@gmail.com>><br>
> Date: 10 May 2015 at 16:04:51<br>
> To: Hal Finkel <hfinkel@anl.gov>><br>
> Cc: llvm-bgq-discuss@lists.alcf.anl.gov<br>
> <llvm-bgq-discuss@lists.alcf.anl.gov>> , Erik
Lindahl<br>
> <erik.lindahl@gmail.com>><br>
> Subject: Re: [Llvm-bgq-discuss] New bgclang nighty builds (and
other<br>
> updates)<br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
> Hi,<br>
><br>
> Bump. Erik Lindahl has also observed this on bgclang on
JUQUEEN. Is<br>
> it helpful if I produce a small piece of code that reproduces
some<br>
> of the issues on bgclang 3.6?<br>
><br>
><br>
><br>
> Mark<br>
><br>
> On Wed, Mar 25, 2015 at 4:16 PM Mark Abraham <<br>
> mark.j.abraham@gmail.com > wrote:<br>
><br>
><br>
><br>
><br>
><br>
> On Tue, Mar 24, 2015 at 7:29 PM, Hal Finkel <
hfinkel@anl.gov ><br>
> wrote:<br>
><br>
><br>
> ----- Original Message -----<br>
> > From: "Mark Abraham" < mark.j.abraham@gmail.com
><br>
> > To: "Hal Finkel" < hfinkel@anl.gov ><br>
> > Cc: llvm-bgq-discuss@lists.alcf.anl.gov<br>
> > Sent: Tuesday, March 24, 2015 12:35:20 PM<br>
> > Subject: Re: [Llvm-bgq-discuss] New bgclang nighty builds
(and<br>
> > other updates)<br>
> ><br>
> ><br>
> > Hi Hal,<br>
> ><br>
> ><br>
> > Thanks very much for the update & effort.<br>
> ><br>
><br>
> You're very welcome.<br>
><br>
> ><br>
> > I tried out the default bgclang 3.6.0 on vesta, but found
a bunch<br>
> > of<br>
> > the GROMACS SIMD-layer unit tests failing. These need
correct QPX<br>
> > vector intrinsics available. From memory, things worked
fine with<br>
> > bgclang in ~August last year, but I no longer have those
results.<br>
> > Is<br>
> > there a simple way I can compile on vesta with older
bgclang to see<br>
> > where a problem might lie? Otherwise / depending what I
learn, I'll<br>
> > break out a debugger.<br>
><br>
> The old builds are still all installed. Just use the MPI
wrappers<br>
> from:<br>
><br>
> /home/projects/llvm/<whatever>/mpi/bgclang/bin -- I
don't know<br>
> exactly which build you were using in August of last
year,<br>
> r209570-20140527 maybe?<br>
><br>
><br>
><br>
><br>
><br>
><br>
> Yes, that was it, looking at some cruft in my former build
script. In<br>
> any case, the latest GROMACS SIMD single-precision unit tests
pass<br>
> on that old compiler version (3.5.0), and many of them fail in
a<br>
> release build on the default 3.6.0 bgclang on vesta. For some
of the<br>
> cases, it looks like some junk memory gets loaded, somehow. A
few<br>
> tests pass, but no theme for passing or failing tests leaps
out at<br>
> me. Double precision unit tests are OK, though.<br>
><br>
><br>
> Naturally, in debug mode most of the tests pass. However, the
two I<br>
> looked at in ddt were fine up until they called vec_extract(x,
0)<br>
> e.g. on lines<br>
>
https://github.com/gromacs/gromacs/blob/master/src/gromacs/simd/impl_ibm_qpx/impl_ibm_qpx.h#L315<br>
> (where a SIMD vector of 1 2 3 1 was summed to 3, rather than
7) and<br>
>
https://github.com/gromacs/gromacs/blob/master/src/gromacs/simd/impl_ibm_qpx/impl_ibm_qpx.h#L462<br>
> (where a SIMD vector dot product of the first 3 lanes returns
a<br>
> garbage answer). So maybe that is a productive lead?<br>
><br>
><br>
> I couldn't inspect the disassembly in ddt, so I'm not sure
where we<br>
> can take this from here, Hal. Tarball of test results
attached.<br>
><br>
><br>
> Thanks,<br>
><br>
><br>
> Mark<br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
> -Hal<br>
><br>
><br>
><br>
> ><br>
> ><br>
> > Thanks,<br>
> ><br>
> ><br>
> > Mark<br>
> ><br>
> ><br>
> ><br>
> ><br>
> > On Fri, Mar 20, 2015 at 12:15 AM, Hal Finkel <
hfinkel@anl.gov ><br>
> > wrote:<br>
> ><br>
> ><br>
> > Hello everyone,<br>
> ><br>
> > First, let me apologize to everyone, this is a few months
late...<br>
> > but, hopefully, this will never be a problem
again...<br>
> ><br>
> > I now have a system setup which automatically pulls in
upstream<br>
> > changes and tries to merge those with the
bgclang-specific patches,<br>
> > and then builds the resulting suite of bgclang RPMs. When
this<br>
> > succeeds, the RPMs should be posted automatically
to:<br>
> ><br>
> > http://www.mcs.anl.gov/~hfinkel/bgclang/<br>
> > (note that installing a build from here now also requires
both the<br>
> > 'stage1' and 'stage2' RPMs as well)<br>
> ><br>
> > The first such nightly build, r232720-20150319, has been
posted to<br>
> > that page.<br>
> ><br>
> > And, for the curious, the local repositories used for
version<br>
> > control<br>
> > are now mirrored to github:<br>
> ><br>
> > https://github.com/hfinkel/clang-bgq<br>
> > https://github.com/hfinkel/llvm-bgq<br>
> > https://github.com/hfinkel/bgclang-aux<br>
> > https://github.com/hfinkel/compiler-rt-bgq<br>
> > https://github.com/hfinkel/libcxx-bgq<br>
> > https://github.com/hfinkel/openmp-bgq<br>
> > https://github.com/hfinkel/sleef-bgq<br>
> ><br>
> > Compared to the latest "released" version
(r220548-20141024), the<br>
> > most-recent nightly build does show some performance
regressions,<br>
> > and there are a few things I've not even tested yet (LTO,
ASan,<br>
> > etc.), but it also contains a number of bug fixes and
improvements,<br>
> > so feel free to test on your applications.<br>
> ><br>
> > One particular noteworthy improvement is that our OpenMP
runtime<br>
> > library now has affinity support enabled. This means that
all of<br>
> > the<br>
> > OpenMP 4 affinity features should work, and also that the
default<br>
> > thread<->core bindings are now sensible.<br>
> ><br>
> > The bgclang wrapper script no longer disables
'fast-isel'<br>
> > instruction<br>
> > selection at -O0, so your debug builds should now be
faster too.<br>
> > Also, the automated vectorization of math functions using
our SLEEF<br>
> > library adaptation is controlled using the new -fveclib
flag (so<br>
> > the<br>
> > wrapper script contains -fveclib=SLEEF, and you can
add<br>
> > -fveclib=none to turn it off if desired for whatever
reason).<br>
> ><br>
> > Also, the core QPX support has been contributed upstream
(although<br>
> > not yet the Clang-level intrinsics support); so if you're
using<br>
> > LLVM<br>
> > as a library, and want to just build from upstream
sources instead<br>
> > of depending on the bgclang builds, that is now
possible.<br>
> ><br>
> > Thanks again everyone, and please let me know if you
experience any<br>
> > difficulties,<br>
> > Hal<br>
> ><br>
> > --<br>
> > Hal Finkel<br>
> > Assistant Computational Scientist<br>
> > Leadership Computing Facility<br>
> > Argonne National Laboratory<br>
> > _______________________________________________<br>
> > llvm-bgq-discuss mailing list<br>
> > llvm-bgq-discuss@lists.alcf.anl.gov<br>
> >
https://lists.alcf.anl.gov/mailman/listinfo/llvm-bgq-discuss<br>
> ><br>
> ><br>
><br>
> --<br>
> Hal Finkel<br>
> Assistant Computational Scientist<br>
> Leadership Computing Facility<br>
> Argonne National Laboratory<br>
><br>
><br>
> --<br>
> Erik Lindahl < erik.lindahl@gmail.com ><br>
> Professor of Biophysics, Dept. Biochemistry & Biophysics,
Stockholm<br>
> University<br>
> Professor of Theoretical biophysics, Dept. Theoretical
Physics, Royal<br>
> Inst. Technology<br>
> Science for Life Laboratory, Box 1031, 17121 Solna,
Sweden<br>
<br>
--<br>
Hal Finkel<br>
Assistant Computational Scientist<br>
Leadership Computing Facility<br>
Argonne National Laboratory<br></span></div>
</div>
</blockquote>
<div id="bloop_sign_1431272391362680832" class="bloop_sign">
<div style="font-family:helvetica,arial;font-size:13px">
-- <br>
<span style="font-family: 'helvetica Neue', helvetica;">Erik
Lindahl <</span><a href="mailto:erik.lindahl@gmail.com" style="font-family: 'helvetica Neue', helvetica;">erik.lindahl@gmail.com</a><span style="font-family: 'helvetica Neue', helvetica;">> </span><br style="font-family: 'helvetica Neue', helvetica;">
<span style="font-family: 'helvetica Neue', helvetica;">Professor
of Biophysics, Dept. Biochemistry & Biophysics,
Stockholm </span><span style="font-family: 'helvetica Neue', helvetica;">University </span><br style="font-family: 'helvetica Neue', helvetica;">
<span style="font-family: 'helvetica Neue', helvetica;">Professor
of Theoretical biophysics, Dept. Theoretical Physics, Royal
Inst. </span><span style="font-family: 'helvetica Neue', helvetica;">Technology </span><br style="font-family: 'helvetica Neue', helvetica;">
<span style="font-family: 'helvetica Neue', helvetica;">Science for
Life Laboratory, Box 1031, 17121 Solna, Sweden</span></div>
</div>
</div></div></span></blockquote> <div id="bloop_sign_1431272408321490944" class="bloop_sign"><div style="font-family:helvetica,arial;font-size:13px">-- <br><span style="font-family: 'helvetica Neue', helvetica;">Erik Lindahl <</span><a href="mailto:erik.lindahl@gmail.com" style="font-family: 'helvetica Neue', helvetica;">erik.lindahl@gmail.com</a><span style="font-family: 'helvetica Neue', helvetica;">> </span><br style="font-family: 'helvetica Neue', helvetica;"><span style="font-family: 'helvetica Neue', helvetica;">Professor of Biophysics, Dept. Biochemistry & Biophysics, Stockholm </span><span style="font-family: 'helvetica Neue', helvetica;">University </span><br style="font-family: 'helvetica Neue', helvetica;"><span style="font-family: 'helvetica Neue', helvetica;">Professor of Theoretical biophysics, Dept. Theoretical Physics, Royal Inst. </span><span style="font-family: 'helvetica Neue', helvetica;">Technology </span><br style="font-family: 'helvetica Neue', helvetica;"><span style="font-family: 'helvetica Neue', helvetica;">Science for Life Laboratory, Box 1031, 17121 Solna, Sweden</span></div></div></body></html>