[dcmf] New DCMF Readme

[Joseph Ratterman]

I've been working on a new README file for the comm stack.  Please take a 
look at it, since I hope that it will answer some of the recent questions. 
 It is a little rough, but I wanted to get it out fast.

Thanks,
Joe Ratterman
jratt at us.ibm.com


About the DCMF messaging stack
    This is the DCMF messaging stack, currently for use on the IBM Blue
    Gene/P platform. The package does contain more than just the DCMF
    libraries. Everything is collected together to ensure that it works
    correctly.

    This file is generated from the POD in README.pod. Please only change
    that file.

    DCMF
        DCMF stands for *Deep Computing Messaging Framework*. It forms the
        basis of the active message layer in use, and is the prefix on all 
C
        API symbols.

    CCMI
        CCMI stands for *Common Collective Messaging Interface*. It is a
        general system for writing cross-platform collectives. Internally,
        it uses the CCMI namespace, but the external C API still uses the
        DCMF prefix.

Directory layout
    The directories are designed to organize components effectively. There
    is some historical carry-over, but the layout is intended to group
    related items.

  tools/
    This is the simplest directory. It contains programs and scripts that
    run on the FENs and help make your life better.

  sys/
    The sys directory contains low-level software written entirely by IBM.
    It is generally assumed to be for software that talks to the HW/SPIs 
or
    implements a new external API.

    sys/include/
        The API header files exist in this directory. They are installed 
by
        the build process of the tool that owns them.

    sys/messaging/
        This is the home of the DCMF layer. This directory contains a
        "configure" script that can be used to setup the build with 
various
        options. The defaults are generally good for our setup, but other
        people may have different goals. It is not necessary to run
        "configure" from inside the messaging directory. This allows
        multiple different builds from the same source.

        There are also a number of older tools in this directory that 
should
        be moved in the future.

    sys/collectives/
        This is the home of the CCMI layer. This directory also contains a
        configure script that works very much like the one in
        sys/messaging/.

    sys/tests/
        This is a small (but growing) test repository, able to test many
        different layers.

    sys/examples/
        These files are examples of how to use DCMF C API documentation.
        They are also included in the documentation that is generated by
        doxygen.

    sys/build-*/
        These directories are the build paths for the source directories.
        Since it is so easy to avoid building in the src path, we try to
        avoid mixing the source and objects--it simplifies cleaning up and
        building with multiple compilers.

  lib/
    This is generally for imported external packages.

    lib/mpi/
        This is "beefed up" version of the MPICH2 distribution from ANL. 
It
        contains a modified version of MPICH2 1.0.4p1 in lib/mpi/mpich2/.
        Mostly, the modifications consist of a new MPID device, a new 
MPI-IO
        FS, and some changes to the test bucket scripts.

        The m4 and autoconf directories are from when MPICH2 required a
        version of those tools that was not installed on the build 
systems.
        They are not currently used on the BG/P platform.

        The sowing directory contains the source for the doctext tool. It 
is
        used to create the manual and html pages to document the MPI2 API.

        The build and stage dirs are for object files created by the
        building the components. Stage is the install location for
        sowing/doctext and, when used, m4 and autoconf. There are two
        version of the MPICH2 build under lib/mpi/build/mpich2/. The 
*dcmf*
        version is the standard version compiled with GNU compiler suite 
and
        shipped. The *dcmf-fast* version, which isn't actually known to be
        faster, is compiled with the IBM XL compiler suite.

    lib/ga/
        This is the GA-ARMCI distribution from PNNL. It has been slightly
        modified to work with the DCMF API. This is a reference
        implementation and has been released to the GA community.

Build system
    The build system in use for the messaging stack is a bit of a kludging
    together of the more sane build systems found in most of the
    sub-components.

  Make files
    Generally, one should only really have to look at a few specific files
    to see how certain sections are built:

    Makefile
        This is top-level make file and is designed to help build 
individual
        components.

    Make.rules*
        Make.rules is a symlink that is intended to point to another
        Make.rules.* file. To aid in porting to other build systems, the
        Make.rules file is included (eventually) by every other make file 
in
        the system--they must first define a ROOTDIR variable to store the
        path that contains Make.rules. The Make.rules.floor contains 
logical
        defaults for building everything based on the installed floor on 
the
        FEN. Make.rules.ibm contains commands to get the same information
        from the internal build system.

    lib/mpi/Makefile
        This file accommodates the large number of options that must be
        passed to the MPICH2 configure. Unlike the DCMF and CCMI parts,
        MPICH2 is a cross-platform system that has no concept of the BG/P
        install layout. Obviously MPICH2 can be configured on it's own, 
but
        it's quite hard to get the options correct.

  Make targets
    The build system is designed to detect the presence or lack of the IBM
    XL compiler suite. To force the build system to avoid XL, add "XL=" to
    your make command.

    default & install
        This is the default, and builds everything below except for 
"tests".

    all This builds everything including "tests".

    docs
        DCMF, CCMI, and the DCMF MPID device in MPICH2 all contain 
extensive
        documentation. This rule builds and installs it.

    dcmf[-fast]
        These are two different versions of the DCMF library. The -fast
        version is build without asset macros.

    ccmi[-fast]
        Again, two versions of the CCMI library. The -fast version is
        compiled by the IBM XL compilers without assert macros. It is not
        known if it is actually faster.

    mpi This simply runs the makefile in lib/mpi/.

    ga  Like mpi, this simply runs the makefile in lib/ga/.

    tests
        This builds the tests in sys/tests/.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alcf.anl.gov/pipermail/dcmf/attachments/20080206/9030a086/attachment.htm>