Hi Jeff,<br><br>I think I could have been clearer in my last message. I am skeptical that Darshan is the issue but would like to spend a minute looking through the code for MPI_Gather on BGQ. Is this accessible?<br><br>Jack<br>
<br><div class="gmail_quote">On Fri, Mar 1, 2013 at 11:01 AM, Jeff Hammond <span dir="ltr"><<a href="mailto:jhammond@alcf.anl.gov" target="_blank">jhammond@alcf.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<a href="https://www.alcf.anl.gov/resource-guides/darshan" target="_blank">https://www.alcf.anl.gov/resource-guides/darshan</a> doesn't touch the MPI<br>
source. It's an IO profiling library that is interposed by the MPI<br>
wrappers. If you don't see , then it might not be included.<br>
<br>
As you know, I'm at SIAM, but I'll try to look at this next week<br>
during MiraCon (which I know you are not attending).<br>
<span class="HOEnZb"><font color="#888888"><br>
Jeff<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
On Fri, Mar 1, 2013 at 1:52 PM, Jack Poulson <<a href="mailto:jack.poulson@gmail.com">jack.poulson@gmail.com</a>> wrote:<br>
> Hi Jeff,<br>
><br>
> Yes, this is on Vesta. There doesn't seem to be anything in your .soft file<br>
> different from mine, other than you specifying the Nov 2012 IBM compilers.<br>
><br>
> Is it possible for me to browse through the current source for the BGQ MPICH<br>
> modifications?<br>
><br>
> Jack<br>
><br>
><br>
> On Fri, Mar 1, 2013 at 10:30 AM, Jeff Hammond <<a href="mailto:jhammond@alcf.anl.gov">jhammond@alcf.anl.gov</a>> wrote:<br>
>><br>
>> This is BGQ @ ALCF, right? Might be MPI calls inside of Darshan. I<br>
>> disable it because of issues like this.<br>
>><br>
>> I think my ~/.soft is world-readable. Use the @mpi-wrappers script<br>
>> and try to verify that you aren't getting Darshan in your build.<br>
>><br>
>> Jeff<br>
>><br>
>> On Fri, Mar 1, 2013 at 11:16 AM, Jack Poulson <<a href="mailto:jack.poulson@gmail.com">jack.poulson@gmail.com</a>><br>
>> wrote:<br>
>> > On Thu, Feb 28, 2013 at 10:15 PM, Hal Finkel <<a href="mailto:hfinkel@anl.gov">hfinkel@anl.gov</a>> wrote:<br>
>> >><br>
>> >><br>
>> >> Not a problem! Thanks for being a beta tester :) I've updated the<br>
>> >> installed libc++ libraries to use CLOCK_REALTIME instead of<br>
>> >> CLOCK_MONOTONIC.<br>
>> >> Please try again.<br>
>> >><br>
>> >> -Hal<br>
>> >><br>
>> ><br>
>> > One more problem taken care of it seems. Unfortunately my program now<br>
>> > segfaults in an MPI_Gather call (and the trace still seems a bit<br>
>> > corrupted,<br>
>> > see core.13). There is really only one instance in my program where<br>
>> > MPI_Gather is called, and it looks like this:<br>
>> ><br>
>> > vector<int> myCoords(d), coords(1);<br>
>> > // <fill myCoords here><br>
>> > if( commRank == 0 )<br>
>> > coords.resize( d*commSize );<br>
>> > MPI_Gather( &myCoords[0], d, MPI_INT, &coords[0], d, MPI_INT, 0, comm );<br>
>> ><br>
>> > In the above snippet, 'd' is the dimension of the domain, which is two<br>
>> > for<br>
>> > the executable in question, and space for storing every process's<br>
>> > coordinates is only allocated on the root process. This is pretty<br>
>> > straightforward MPI in my opinion, so I am skeptical that I have a bug<br>
>> > here.<br>
>> ><br>
>> > Jack<br>
>><br>
>><br>
>><br>
>> --<br>
>> Jeff Hammond<br>
>> Argonne Leadership Computing Facility<br>
>> University of Chicago Computation Institute<br>
>> <a href="mailto:jhammond@alcf.anl.gov">jhammond@alcf.anl.gov</a> / <a href="tel:%28630%29%20252-5381" value="+16302525381">(630) 252-5381</a><br>
>> <a href="http://www.linkedin.com/in/jeffhammond" target="_blank">http://www.linkedin.com/in/jeffhammond</a><br>
>> <a href="https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond" target="_blank">https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond</a><br>
><br>
><br>
<br>
<br>
<br>
--<br>
Jeff Hammond<br>
Argonne Leadership Computing Facility<br>
University of Chicago Computation Institute<br>
<a href="mailto:jhammond@alcf.anl.gov">jhammond@alcf.anl.gov</a> / <a href="tel:%28630%29%20252-5381" value="+16302525381">(630) 252-5381</a><br>
<a href="http://www.linkedin.com/in/jeffhammond" target="_blank">http://www.linkedin.com/in/jeffhammond</a><br>
<a href="https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond" target="_blank">https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond</a><br>
</div></div></blockquote></div><br>