6. Contributors Guide

6.1 The ICENI Module Structure

6.1.1 Introduction

ICENI is composed of a collection of functional modules. These modules are mostly self-contained services or applications that collectively define useful profiles of ICENI environment that fulfills the common needs of various Grid scenarios.


Figure 6.41. Build Dependencies of ICENI Modules

ICENI module is a functional unit of code or application that might express build dependencies to other modules. Build Dependencies refer to the need to import foreign classes or binaries from a foreign module in order to compile and build the code. More importantly, it does not define the functional dependencies of the module, although build dependency often entails functional dependency.

ICENI modules are required to satisfy a set of naming standard, directory structure and build environment in order to standardise the development effort across modules. Requirements and recommended practices for developing a new ICENI module are described in the coming sections. See Section 6.2 for creating a new ICENI module from scratch using the standard template.

6.1.2 Structure of an ICENI Module

An ICENI module MUST contain a set of required files and directories. This is a synthesised guideline of many well-tested open-source projects, such as Apache, Jakarta and Netbeans. By defining a `best practice' approach and maintaining a set of naming conventions, we can promote increased clarity and familiarity of the source organization that everyone can modify and build.

An ICENI module from CVS is known as a source module , the binary module is a product of the source module after the source code is being build.

6.1.3 Source Module

Required Directory Structure

Recommended Directory Structure

6.1.4 Source Module Build System

A coherent build process will reduce the cost of maintenance and also decentralising the expertise of incorporating a particular module into a system as a whole. The build process described here is a minimal set of commands based on the Jakarta ANT system to build a distributable module from source code. The aim is to separate the binary and runtime scripts from the source code .

The ANT build system (http://jakarta.apache.org/ant ) consists of a cross-platform runtime executable ant (similar to make, but written in Java, so available in all Java Platforms), and the build script (similar to Makefile in XML format) for each module. ICENI modules MUST only use features in ANT version 1.5.1 and the the ant-contrib (http://ant-contrib.sourceforge.net ). See [?] for a description of iceni-build and the bundled ANT system.

To invoke ant on a given build script,

ant -f /iceni/somemodule/build.xml [target]

The `target' specifies the command you would like to perform provided by the given build script. If the build.xml script is in the current directory, you can simply invoke

ant [target]

To examine the list of targets available in a build script, invoke

ant -projecthelp

A target is a collection of commands specified in the build script. A`compile' target might invoke the `javac' command on all the *.java files in the src/ directory and jar the classfiles into a jar in the build/ directory. Please consult the ANT user manual (http://jakarta.apache.org/ant/manual/index.html ) for examples and tutorials for writing build scripts.

6.1.5 Source Module Build Process

As described in Section 6.1.3, each ICENI source module MUST have a build.xml and build.properties files, which can be used to build the module. The build.xml script MUST have at least the following targets.

6.1.6 Binary Module

A binary module is a set of directories and files that are product of a source module. It is free from any source-code and can be classified as Non-executable or Executable .

Required Directory Structure

An Executable is an extension of Non-Executable. It has the same directory structures with the following additions

6.1.7 ICENI Core Module Dependencies

dependency diagram goes here...

6.2 Creating a New Module

6.2.1 Description

 In order to standardise on a common source module build environment and ease the effort in developing and maintaining module. The iceni-build module provides the necessary tools and templates to create a new module from scratch which can be easily integrated into the overall build process of the ICENI system.

6.2.2 Pre-requisites

6.2.3 Actions
  1. Execute the following commands 
    cvs -d:pserver:<username>@cvs.icpc.doc.ic.ac.uk:/cvs/iceni-core login
cvs -d:pserver:<username>@cvs.icpc.doc.ic.ac.uk:/cvs/iceni-core co iceni-build
cd iceni-build
source bin/setenv.csh (or source bin/setenv.sh)

ant newmodule

  2. Follow the prompt to create a new module

6.2.4 Usage

The new module is created with the required directory structure as described in Section 6.1.3. You are free to modify the build.xml to suit your needs, however, the list of targets must satisfy 6.1.5.

You can customise your module in the following ways

6.3 Creating a New Project

ICENI Modules can be grouped in project. A compilation or installation of a project will result in the compilation or the installation of the list of all the modules of this project.

6.3.1 Defining a New Project

 The project is mainly defined by its name, its version, and a list of modules. The following file needs to be present in the directory iceni-build/data/projects 
#******************************************************************************************
# Properties file defining the set of properties for the virtual project
#******************************************************************************************

#******************************************************************************************
# Mandatory properties
#******************************************************************************************

# the name of the virtual project
project.name=test

# the name of the virtual project
project.version=latest

# the set of modules to be built
# syntax: comma-separated list of module names
modules=iceni-bootstrap,iceni-external,iceni-services,iceni-services-jini,iceni-utils,iceni-xml

# indicate whether test should be performed after the build
performtest=true

#******************************************************************************************
# Optional properties
#******************************************************************************************

# for each module, a CVS tag can be specified
# syntax: module.{module-name}.tag=cvstag


# for each module, test can be optionally performed, default to the value of the performtest property
# syntax: module.{module-name}.performtest=[true|false]

6.3.2 The Nightly Build Project

The nightly build project is a predefined project representing all the ICENI modules, that is automatically build and uploaded on a web page every night.

To add a new module in this project, one needs to:

6.3.3 Downloading a Project

A tool is available in the module iceni-build to checkout all the modules belonging to a specific project, it is also possible to specify the CVS tag of the version to checkout.

The syntax of the tool is:

./cvsCheckout.sh <directory> <project file> [<cvs user>] [-useproxy] [<cvs tag>]

where

For example, if you wish to download the main version of the modules of the nightly build project: 

cvs co iceni-build
cd iceni-build/bin
./cvsCheckout.sh $HOME/iceni ../data/projects/nightly-build.properties

Or if you wish to download the latest version of the branch iceni-authentication-branch of the modules of the nightly build project: 

cvs co -r iceni-authentication-branch iceni-build
cd iceni-build/bin
./cvsCheckout.sh $HOME/iceni ../data/projects/nightly-build.properties $USER iceni-authentication-branch

6.3.4 Compiling a Project

 The nightly build project is compiled automatically at frequent interval, the result of the build being reported on the LeSC Web Server (http://www.lesc.imperial.ac.uk/iceni/dashboard.jsp ). However, it is possible to run by hand the compilation of a specific project (by default the nightly build project). The module iceni-nightly in the iceni-core repository contains scripts to build and deploy ICENI.
  1. nightly-build.sh

    Syntax:

    nightly-build.sh <compilation directory> [-webstart <webstart directory>] [-dest <destination directory>] [-server <machine name>]

    [-project <project file>] [-title <title>] [-nojavadoc] [-cvstag <cvs tag>] [-release <release name>]

    where

    • compilation directory is the directory where the compilation will take place;

    • -webstart specifies the directory where the webstart application file will be installed (Should only be used if the result of the nightly build is going to be uploaded on the LeSC web server )

    • -dest specifies the directory where the result of the build, i.e the binary modules, the Java documentation (unless the option -nojavadoc is specified, and the webstart configuration file (if the option -webstart is specified) will be uploaded. If the option -dest is not specified, none of the result files will be uploaded.

    • -server specifies the name of the machine where to upload the files. The default value is saturn.icpc.doc.ic.ac.uk . The files will be uploaded to the directory specified with -dest sitting on this machine.

    • -project specifies the name of the file defining the modules to compile, the default value is data/projects/nightly-build.properties . The file should be present in the iceni-build module that will get downloaded as a first step of the build process.

    • -title specifies the title of the window for the JAVA API Documentation.

    • -nojavadoc indicates the Java documentation should not be generated

    • -cvstag specifies the CVS tag to use when checking out the modules of the CVS repository.

    • -release specifies the name of the release that is built, this data is purely informational, and is only used on the web page that is created during the build process

    For example, if you wish to compile the latest version of the branch iceni-authentication-branch but do not wish to upload the binary modules on the web server, the following command can be used:

    ./nightly-build.sh /tmp/ICENI -cvstag iceni-authentication-branch

    If you wish to make the binary modules available on the LeSC web server, the following command can be used:

    ./nightly-build.sh /tmp/ICENI -dest /homes/iceni/web/downloads/authentication -title Authentication -cvstag iceni-authentication-branch

    The binary modules will then be accessible from the address http://www.lesc.imperial.ac.uk/iceni/downloads/authentication/

    If you wish to make the binary modules available from your own web server www.server.org , the following command can be used:

    ./nightly-build.sh /tmp/ICENI -dest /iceni -server www.server.org -title Authentication -cvstag iceni-authentication-branch

6.4 Accessing the CVS Repository

To access the repository from a console you will have to set your CVSROOT environment variable:

setenv CVSROOT :pserver:username@cvs.icpc.doc.ic.ac.uk:/cvs/repositoryName

where username is your CVS username access and repositoryName the name of the repository you wish to access. You can also use the -d flag on the CVS command. This is needed when checking out the initial sources.

To access the CVS repository from outside DoC, you first need to use the following command that will start up a proxy for 60 minutes. The command has to be called on the machine you wish to access the CVS repository from. The private key should have been sent to you with your login informations. More complete information on how to proceed are given at the following address http://www.lesc.imperial.ac.uk/iceni/developers/cvsSSH.html .

ssh -i <private-key-file> -l cvs-ssh saturn.icpc.doc.ic.ac.uk -L 2401:localhost:2401 proxy

You also need to use the following CVSROOT environment variable:

setenv CVSROOT :pserver:<username>@localhost:/cvs/<repositoryName>

6.4.1 Getting Started

To gain access to the repository you will have to login with the CVS login command. After you have finished, you are advised to logout using the command CVS logout command. To check out the module named component :

cvs co component

The directory structure will be created in the directory component . You are now free to edit or add to these files as they are now resident on your local file space.

6.4.2 Finding out about the changes

Other people will be working on the code while you are. To view any changes that have been made to the source code you can use:

cvs -n -q update

This command compares the state of the central repository with your local copy and shows the files that are not up-to-date (-q option). With the -n option no changes are actually made to your local source. By doing

cvs update

all changes in the repository are merged with your local code. For any new directories created in the repository to appear in your local file space, add a -d to the previous command. This command will work recursively from your current directory to produce a list of files with a defined state.

State Meaning
U Your local file was updated from a later version that exists in the repository.
A The file has been added to your local copy and will be added to the repository after a commit.
R The file has been removed from your local copy and will be removed from the repository after a commit.
M You have modified your local file or their are modifications in the repository that do not conflict with your changes.
C You have modified your local file and their are modifications in the repository that conflict with your changes. The conflicts will be left in the file and will have to be resolved by hand.
? The local file has no corresponding partner in the repository

6.4.3 Contributing your changes

Adding new files or directories into the repository is a two stage process. The files are added to the repository and then committed. If you have made changes to any existing files then these changes have to be committed into the repository before they will appear there for others to use.

The add command will not work recursively.

cvs add file

To get the add command to work "recursively", one can use the following command and call it until it generates an error message.

cvs add `cvs -n -q update|grep '?'|awk '{print $2}'`

The commit command will work recursively. A message needs to be specified to identify the changes made to the files. If none is provided you will be asked to enter some information through an editor. Alternatively the information can be provided on the command line with the option -m "message"

cvs commit file

Return to Contents

Copyright © January 2005 ICENI Research Group
Generated on: 05-Jan-2005 12:04:25