<div dir="ltr"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);display:inline!important;float:none">Hi all,</span><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
<br></div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
I'm the development manager for GROMACS, which will offer new SIMD support for BlueGene/Q in its impending 4.6.4 release. Following some off-list discussion with Jeff Hammond and Hal Finkel, I was happy to explore compiling with clang for BlueGene/Q. Today I tried the version installed on JUQUEEN (r190771-20130914), as I had trouble logging into Vesta (support request lodged).</div>
<div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
<br></div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
In debug mode, everything went great. clang even warned about some MPI_Alltoall calls that could have had some explicit pointer casts to reassure the reader, which I've now patched.</div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
<br></div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
I even used qpxmath.h for a small handful of SIMD trig functions we'd want - that worked perfectly.</div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
<br></div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
In release mode, there was a fatal error from clang when compiling the "plain C" version of the code for which I've now written SIMD kernels. This kernel is compiled and built into mdrun as a fallback. My guess would be that auto-vectorization is choking, but hopefully you guys are better judges of that than me! I'm happy to pass this upstream to LLVM if that's the correct place for this report. The .c and .sh files to reproduce the issue can be found at</div>
<div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
<br></div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
<a href="https://docs.google.com/file/d/0B0H2SbsMc3_qTnVvcTI1OTNFMFE/edit?usp=sharing" target="_blank" style="color:rgb(17,85,204)">https://docs.google.com/file/d/0B0H2SbsMc3_qTnVvcTI1OTNFMFE/edit?usp=sharing</a></div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
<a href="https://docs.google.com/file/d/0B0H2SbsMc3_qenZBX05KSEg1TnM/edit?usp=sharing" target="_blank" style="color:rgb(17,85,204)">https://docs.google.com/file/d/0B0H2SbsMc3_qenZBX05KSEg1TnM/edit?usp=sharing</a></div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
<br></div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
The crash trace follows:</div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
<br></div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
<div>clang: /gpfs/vesta-home/hfinkel/rpmbuild/BUILD/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp:630: llvm::SDValue<unnamed>::DAGCombiner::CombineTo(llvm::SDNode*, const llvm::SDValue*, unsigned int, bool): Assertion `N->getNumValues() == NumTo && "Broken CombineTo call!"' failed.</div>
<div>0 <a href="http://libllvm-3.4svn.so/" target="_blank" style="color:rgb(17,85,204)">libLLVM-3.4svn.so</a><span class=""> </span>0x00000fff7ec34a9c llvm::sys::PrintStackTrace(_IO_FILE*) + 4281424836</div><div>1 <a href="http://libllvm-3.4svn.so/" target="_blank" style="color:rgb(17,85,204)">libLLVM-3.4svn.so</a><span class=""> </span>0x00000fff7ec34d00</div>
<div>2 <a href="http://libllvm-3.4svn.so/" target="_blank" style="color:rgb(17,85,204)">libLLVM-3.4svn.so</a><span class=""> </span>0x00000fff7ec35ba4</div><div>3 0x00000fff7f980418 __kernel_sigtramp_rt64 + 0</div>
<div>4 libc.so.6 0x00000080c3766ef8 abort + 4293479848</div><div>5 libc.so.6 0x00000080c375b98c</div><div>6 libc.so.6 0x00000080c375baa4 __assert_fail + 4293437492</div><div>7 <a href="http://libllvm-3.4svn.so/" target="_blank" style="color:rgb(17,85,204)">libLLVM-3.4svn.so</a><span class=""> </span>0x00000fff7ea0a94c</div>
<div>8 <a href="http://libllvm-3.4svn.so/" target="_blank" style="color:rgb(17,85,204)">libLLVM-3.4svn.so</a><span class=""> </span>0x00000fff7ea0adfc</div><div>9 <a href="http://libllvm-3.4svn.so/" target="_blank" style="color:rgb(17,85,204)">libLLVM-3.4svn.so</a><span class=""> </span>0x00000fff7ea2de20</div>
<div>10<span class=""> </span><a href="http://libllvm-3.4svn.so/" target="_blank" style="color:rgb(17,85,204)">libLLVM-3.4svn.so</a><span class=""> </span>0x00000fff7ea43554</div><div>11<span class=""> </span><a href="http://libllvm-3.4svn.so/" target="_blank" style="color:rgb(17,85,204)">libLLVM-3.4svn.so</a><span class=""> </span>0x00000fff7ea46ecc</div>
<div>12<span class=""> </span><a href="http://libllvm-3.4svn.so/" target="_blank" style="color:rgb(17,85,204)">libLLVM-3.4svn.so</a><span class=""> </span>0x00000fff7ea49c70 llvm::SelectionDAG::Combine(llvm::CombineLevel, llvm::AliasAnalysis&, llvm::CodeGenOpt::Level) + 4279456680</div>
<div>13<span class=""> </span><a href="http://libllvm-3.4svn.so/" target="_blank" style="color:rgb(17,85,204)">libLLVM-3.4svn.so</a><span class=""> </span>0x00000fff7eb8fba8 llvm::SelectionDAGISel::CodeGenAndEmitDAG() + 4280770368</div>
<div>14<span class=""> </span><a href="http://libllvm-3.4svn.so/" target="_blank" style="color:rgb(17,85,204)">libLLVM-3.4svn.so</a><span class=""> </span>0x00000fff7eb909f8 llvm::SelectionDAGISel::SelectBasicBlock(llvm::ilist_iterator<llvm::Instruction const>, llvm::ilist_iterator<llvm::Instruction const>, bool&) + 4280774016</div>
<div>15<span class=""> </span><a href="http://libllvm-3.4svn.so/" target="_blank" style="color:rgb(17,85,204)">libLLVM-3.4svn.so</a><span class=""> </span>0x00000fff7eb92dec llvm::SelectionDAGISel::SelectAllBasicBlocks(llvm::Function const&) + 4280783188</div>
<div>16<span class=""> </span><a href="http://libllvm-3.4svn.so/" target="_blank" style="color:rgb(17,85,204)">libLLVM-3.4svn.so</a><span class=""> </span>0x00000fff7eb93fbc llvm::SelectionDAGISel::runOnMachineFunction(llvm::MachineFunction&) + 4280787732</div>
<div>17<span class=""> </span><a href="http://libllvm-3.4svn.so/" target="_blank" style="color:rgb(17,85,204)">libLLVM-3.4svn.so</a><span class=""> </span>0x00000fff7e84d9c8</div><div>18<span class=""> </span><a href="http://libllvm-3.4svn.so/" target="_blank" style="color:rgb(17,85,204)">libLLVM-3.4svn.so</a><span class=""> </span>0x00000fff7e1e26cc llvm::MachineFunctionPass::runOnFunction(llvm::Function&) + 4270867196</div>
<div>19<span class=""> </span><a href="http://libllvm-3.4svn.so/" target="_blank" style="color:rgb(17,85,204)">libLLVM-3.4svn.so</a><span class=""> </span>0x00000fff7e4529b8 llvm::FPPassManager::runOnFunction(llvm::Function&) + 4273352968</div>
<div>20<span class=""> </span><a href="http://libllvm-3.4svn.so/" target="_blank" style="color:rgb(17,85,204)">libLLVM-3.4svn.so</a><span class=""> </span>0x00000fff7e452afc llvm::FPPassManager::runOnModule(llvm::Module&) + 4273353276</div>
<div>21<span class=""> </span><a href="http://libllvm-3.4svn.so/" target="_blank" style="color:rgb(17,85,204)">libLLVM-3.4svn.so</a><span class=""> </span>0x00000fff7e4522bc llvm::MPPassManager::runOnModule(llvm::Module&) + 4273351228</div>
<div>22<span class=""> </span><a href="http://libllvm-3.4svn.so/" target="_blank" style="color:rgb(17,85,204)">libLLVM-3.4svn.so</a><span class=""> </span>0x00000fff7e4525e4 llvm::PassManagerImpl::run(llvm::Module&) + 4273352020</div>
<div>23<span class=""> </span><a href="http://libllvm-3.4svn.so/" target="_blank" style="color:rgb(17,85,204)">libLLVM-3.4svn.so</a><span class=""> </span>0x00000fff7e4526f4 llvm::PassManager::run(llvm::Module&) + 4273352276</div>
<div>24 clang 0x00000000103ae874</div><div>25 clang 0x00000000103af7f8 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::Module*, clang::BackendAction, llvm::raw_ostream*) + 4272665128</div>
</div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
<div>26 clang 0x00000000103ab4a4</div><div>27 clang 0x000000001059f230 clang::ParseAST(clang::Sema&, bool, bool) + 4274649152</div><div>28 clang 0x00000000101e4b64 clang::ASTFrontendAction::ExecuteAction() + 4270836484</div>
<div>29 clang 0x00000000103a9b00 clang::CodeGenAction::ExecuteAction() + 4272641808</div><div>30 clang 0x00000000101e4fb4 clang::FrontendAction::Execute() + 4270837524</div><div>31 clang 0x00000000101be154 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) + 4270679924</div>
<div>32 clang 0x000000001019f894 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) + 4270560804</div><div>33 clang 0x00000000101959d8 cc1_main(char const**, char const**, char const*, void*) + 4270520648</div>
<div>34 clang 0x000000001019d540 main + 4270551792</div><div>35 libc.so.6 0x00000080c374bcf8</div><div>36 libc.so.6 0x00000080c374bef0 __libc_start_main + 4293374496</div><div>Stack dump:</div>
<div>0.<span style="white-space:pre-wrap"> </span>Program arguments: /usr/local/bg_soft/clang/llvm.r190771/r190771-20130914/bin/clang -cc1 -fopenmp -triple powerpc64-bgq-linux -S -disable-free -main-file-name nbnxn_kernel_ref.c -static-define -mrelocation-model static -mdisable-fp-elim -ffp-contract=fast -mconstructor-aliases -target-cpu a2q -target-linker-version 2.20.51.0.2 -coverage-file /tmp/nbnxn_kernel_ref-bb4750.s -resource-dir /usr/local/bg_soft/clang/llvm.r190771/r190771-20130914/bin/../lib/clang/3.4 -D __bgclang__=1 -D __bgclang_version__="r000000-00000000" -D HAVE_CONFIG_H -D md_EXPORTS -D NDEBUG -I /bgsys/local/clang/llvm.r190771/r190771-20130914/sleef/include -I /bgsys/local/clang/llvm.r190771/r190771-20130914/omp/include -I /bgsys/drivers/V1R2M1/ppc64/comm/include -I /bgsys/drivers/V1R2M1/ppc64/comm/lib/gnu -I /bgsys/drivers/V1R2M1/ppc64 -I /bgsys/drivers/V1R2M1/ppc64/comm/sys/include -I /bgsys/drivers/V1R2M1/ppc64/spi/include -I /bgsys/drivers/V1R2M1/ppc64/spi/include/kernel/cnk -I /homeb/zdv518/zdv518/git/bluegene-dev-r46/build-cmake-clang/src -I /homeb/zdv518/zdv518/git/bluegene-dev-r46/build-cmake-clang/include -I /homeb/zdv518/zdv518/git/bluegene-dev-r46/include -I /homeb/zdv518/zdv518/progs/bgsys-clang/include -I /bgsys/drivers/V1R2M1/ppc64/comm/include -internal-isystem /usr/local/include -internal-isystem /usr/local/bg_soft/clang/llvm.r190771/r190771-20130914/bin/../lib/clang/3.4/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -O3 -Wall -Wno-unused -Wunused-value -fno-dwarf-directory-asm -fdebug-compilation-dir /homeb/zdv518/zdv518/git/bluegene-dev-r46/build-cmake-clang/src/mdlib -ferror-limit 19 -fmessage-length 108 -mstackrealign -fno-signed-char -fobjc-runtime=gcc -fobjc-default-synthesize-properties -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -isystem /bgsys/drivers/V1R2M1/ppc64/gnu-linux/powerpc64-bgq-linux/sys-include -mllvm -optimize-regalloc -mllvm -fast-isel=0 -o /tmp/nbnxn_kernel_ref-bb4750.s -x c /homeb/zdv518/zdv518/git/bluegene-dev-r46/src/mdlib/nbnxn_kernels/nbnxn_kernel_ref.c </div>
<div>1.<span style="white-space:pre-wrap"> </span><eof> parser at end of file</div><div>2.<span style="white-space:pre-wrap"> </span>Code generation</div><div>3.<span style="white-space:pre-wrap"> </span>Running pass 'Function Pass Manager' on module '/homeb/zdv518/zdv518/git/bluegene-dev-r46/src/mdlib/nbnxn_kernels/nbnxn_kernel_ref.c'.</div>
<div>4.<span style="white-space:pre-wrap"> </span>Running pass 'PowerPC DAG->DAG Pattern Instruction Selection' on function '@nbnxn_kernel_ref_rf_noener'</div></div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
<div>clang: error: unable to execute command: Aborted (core dumped)</div><div>clang: error: clang frontend command failed due to signal (use -v to see invocation)</div><div>clang version 3.4 (trunk)</div><div>Target: powerpc64-bgq-linux</div>
<div>Thread model: posix</div><div>clang: note: diagnostic msg: PLEASE submit a bug report to<span class=""> </span><a href="http://llvm.org/bugs/" target="_blank" style="color:rgb(17,85,204)">http://llvm.org/bugs/</a><span class=""> </span>and include the crash backtrace, preprocessed source, and associated run script.</div>
<div>clang: note: diagnostic msg: </div><div>********************</div><div><br></div><div>PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:</div><div>Preprocessed source(s) and associated run script(s) are located at:</div>
<div>clang: note: diagnostic msg: /tmp/nbnxn_kernel_ref-96ac7c.c</div><div>clang: note: diagnostic msg: /tmp/nbnxn_kernel_ref-96ac7c.sh</div><div>clang: note: diagnostic msg: </div><div><br></div><div>********************</div>
</div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
<br></div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
I tried to check that the .sh file would reproduce the above, but it failed with</div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
<br></div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
<div>In file included from <built-in>:167:</div><div><command line>:6:10: fatal error: 'qpxintrin.h' file not found</div><div>#include "qpxintrin.h"</div><div> </div></div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
Hope that is useful - do let me know if I can be of further help!</div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
<br></div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
Cheers,</div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
<br></div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">
Mark</div></div>