[music-rfc] MUSIC

[ben mitch]

Hi Örjan

Örjan Ekeberg wrote:
 > The primary focus of the MUSIC library is to provide an efficient way
 > for large scale parallel simulators to cooperate during runtime.  As you
 > point out, this might be useful as a data transfer mechanism within
 > BRAHMS.  Still, I guess that MUSIC would only be useful for some types
 > of communication; especially when large amounts of data and processes
 > are involved.  Note that we are primarily targeting simulations on
 > supercomputers and large clusters, often on thousands of processors.

Right, I hadn't appreciated that targeting. One of BRAHMS's guiding
principles is that it targets all deployment environments, from embedded
through to compute cluster (thousands of processors is more than I ever
considered, though, wow :) ) with minimal user effort. For clusters,
BRAHMS currently offers two inter-process communication layers (based on
TCP/IP and MPI, respectively), both currently in alpha. Does MUSIC offer
us something we won't be getting by using MPI natively, perhaps
particularly in large-scale compute environments? (If you feel we're
heading off-topic here, please do email me directly.)

 > One critical component shared by BRAHMS and MUSIC is that we both depend
 > on that a number of applications are adapted to the communication
 > interface.  For the benefit of the users, it would be nice if the same
 > adapted application could be used for incorporation in both MUSIC and
 > BRAHMS.  We have tried to keep the API in MUSIC to a minimum, and I
 > guess that BRAHMS will require more adaptation.  Perhaps we can find a
 > way such that a BRAHMS-adapted program can directly make use of MUSIC?

To reiterate our support: we strongly agree that the main point here is
integration, not bifurcation :). One more paragraph of BRAHMS trivia
before I address your comment, if you'll indulge me...

The primary aim of BRAHMS is to offer the researcher the ability to
write compute software that integrates with everything already out there
in the BRAHMS world, with very minimal development overhead (i.e with
inter-software communication abstracted as far as possible). Therefore,
we expect most BRAHMS development to be fairly simple tightly-targeted
modules built from scratch rather than modifications to existing
software. The primary aim of MUSIC, in my current understanding, is to
facilitate interoperation of existing applications (notwithstanding that
they could be built for MUSIC from the outset). We are aware of the need
to have BRAHMS talk to existing software (NEURON, GENESIS, Webots,
BRIAN, etc.), of course - oh, I've just noticed that BRIAN is a play on
BRAIN, dash it I'm slow. For software that offers a compute "engine", we
anticipate a BRAHMS module to "wrap" that engine. For software that does
not, there are other routes, you can imagine (inter-process comms, or a
BRAHMS front-end for the third-party application). That's our vision,
currently.

In reply to your first point, a (minimal) BRAHMS application consists of
very little indeed, since all services are provided by the framework
(stock preamble aside, a simple but functional BRAHMS app can be written
in five to ten lines of code). It seems impractical to suggest to those
developing software for BRAHMS that they write the rest of the app
required to run it standalone, or likewise to suggest to those modifying
an app for MUSIC that they modify the rest of their app in such a way
that you can "strip it off" to fit the fundamental computational core
into a BRAHMS module. In other words, it seems to me that the two things
are sufficiently different that you can't easily do them "both at once"
(either modifying existing software or developing new) - rather,
users/developers might choose to modify/develop for BRAHMS, or for
MUSIC, or for both (but separately). I'm not sure on this, please do let
me know if you think I'm being short-sighted, because it would certainly
be easier if this were possible.

To your second point, I strongly agree that we need to connect the two
things here. Given my comment above, I suggest that the route might be
to assume that there will be some BRAHMS software and some MUSIC
software, and, therefore, to develop a BRAHMS "MUSIC" module that forms
a bridge between the BRAHMS interface and the MUSIC interface. Thus,
software wrapped in a BRAHMS module or software offering a MUSIC
interface could interoperate. An example of the "software bridge" 
discussed by Howell et al. [1]. The cost would be an additional 
transport layer where one isn't strictly needed, but wherever the 
development cost was seen to be less than the run-time cost, users would 
be free to do the native wrapping of a piece of software for BRAHMS or 
MUSIC, as appropriate.

I suspect that users would generally choose to work in one or other
universe, for the sake of simplicity, but the ability to knit the two
together in a fairly straightforward way would seem to offer a benefit
to both.

Look forward to your further comments.

Cheers
Ben Mitch

--------------------------------------

[1] "Linking computational neuroscience simulation tools—a pragmatic
approach to component-based development", Howell et al.

http://dx.doi.org/10.1016/S0925-2312(02)00781-6