Using the google gadget API it was easy to embed an application into a web page, here is a sample gadget .xml file. The Google gadget developer tools and API are at

http://code.google.com/apis/gadgets/docs/tools.html#GGE

It is useful to observe that when attempting to publish the app from the gadget editor the following error is reported

“Gadget XML should have home view and type html.”

The way around this problem is to put the gadget.xml file in a suitable web location and then to click add gadgets on the igoogle page, in the bottom left of this page is a link to add a feed or gadget this requires the user to add the URL of the gadget.xml file.

The key feature of the script is the following line
<Content type=”url” href=”http://organisation.org.uk/iogatest/iogatest1.php” view=”home,canvas”/>

Module>
−
<ModulePrefs thumbnail=”http://organisation.org.uk/iogatest/iogathumbnail.jpg” screenshot=”http://organisation.org.uk/iogatest/iogagadget.jpg” title=”iogs” title_url=”http://www.iome.org.uk/” description=”Client used for accessing IOME based web services” scrolling=”true” singleton=”true” height=”400″ width=”400″ author=”Michael Griffiths” author_email=”myemail@org.ac.uk” author_aboutme=”Organisation.”>
<Require feature=”dynamic-height”/>
<Require feature=”settitle”/>
</ModulePrefs>
<Content type=”url” href=”http://organisation.org.uk/iogatest/iogatest1.php” view=”home,canvas”/>
</Module>

Some examples of java based SOAP clients are given below

http://www.guru4.net/articoli/javascript-soap-client/en/

http://www.codeproject.com/KB/ajax/JavaScriptSOAPClient.aspx

http://stackoverflow.com/questions/124269/simplest-soap-example-using-javascript

One of the problems is that we require the google gadget to communicate directly with the web service due to firewall restrictions this may not be possible.

We succesfully submitted an IOME task using a URL of the form

http://webaddress.ac.uk/iogatest/submitiogajobt1.php?useremail=username@institution.org.uk&equation=y=n+sum(x.^3-cos(2*pi*x));&populationnumber=40&generationnumber=60&crossoverprobability=0.7&mutationprobability=0.5

Note the maximum line length for such a request is approximately 2000 characters, notes about using PHP $_GET is shown below.

http://www.w3schools.com/php/php_get.asp

The script submitiogajobt1.php used to make the request and on the web server is shown below

<?php
require_once(‘iome/iome.php’);

$myioservice = new ioservice;
$myioservice->server = ‘suilven.shef.ac.uk’;
$myioservice->port = ’60001′;
$myioservice->id = 0;
$myioservice->method = 2;

//need correct write permission on target path
$currenttime = time();
//$target_path=”/var/www/html/uploads/”;
$target_path=”/webusers/cs6grid/public_html/iogatest/results/$currenttime”;
//echo $target_path ;
//echo getcwd();

//$target_path = $target_path.basename( $_FILES['userfile']['name']);
//echo $target_path ;
/*The following lines read the job template xml file*/
/*The %items%  in the jobfile content are replaced with
the variable parameters*/
/*we use the str_replace function to acheive this*/
/*str_replace — Replace all occurrences of the search string with the replacement string*/

$useremail=”";
$jobfile = file_get_contents( “iogaphp.xml”);
$jobref=$currenttime;
$populationnumber=”30″;
$generationnumber=”50″;
$crossoverprobability=”0.9″;
$mutationprobability=”0.6″;
$equation=”y=n+sum(x.^2-cos(2*pi*x));”;
$parameternumber=”2″;

if(!empty($_GET['useremail'])){
$useremail = $_GET['useremail'];
}

if(!empty($_GET['populationnumber'])){
$populationnumber = $_GET['populationnumber'];
}

if(!empty($_GET['crossoverprobability'])){
$crossoverprobability = $_GET['crossoverprobability'];
}

if(!empty($_GET['mutationprobability'])){
$mutationprobability = $_GET['mutationprobability'];
}

if(!empty($_GET['equation'])){
$equation = $_GET['equation'];
}

if(!empty($_GET['parameternumber'])){
$parameternumber = $_GET['parameternumber'];
}

$tempjobfile = str_replace(“%useremail%”, $useremail, $jobfile);
$temptjobfile = str_replace(“%populationnumber%”, $populationnumber, $tempjobfile);
$tempjobfile = str_replace(“%generationnumber%”, $generationnumber, $temptjobfile);
$temptjobfile = str_replace(“%crossoverprobability%”, $crossoverprobability, $tempjobfile);
$tempjobfile = str_replace(“%mutationprobability%”, $mutationprobability, $temptjobfile);
$temptjobfile = str_replace(“%equation%”, $equation, $tempjobfile);
$tempjobfile = str_replace(“%jobref%”, $jobref, $temptjobfile);

$jobfile = str_replace(“%parameternumber%”, $parameternumber, $tempjobfile);

//submit the job to the server
//when the job runs it will pull the binary image file from the ftp server
echo “populationnumber: “.$populationnumber;
echo “generationnumber: “.$generationnumber;
echo “crossoverprobability: “.$crossoverprobability;
echo “mutationprobability: “.$mutationprobability;
echo “equation: “.$equation;
echo “parameternumber: “.$parameternumber;

$outfile=”newsimresults.xml”;
mkdir($target_path,0777,true);
chmod($target_path,0777);  //this line essential to ensure folder set with correct permissions

//$result=(string)setparamdouble($name,(float)$_GET['floatval'],$myioservice);

//running the job in non interactive mode
$result= submitsimulation($jobfile,$myioservice);

//Running the job interactively
//$result= runsimulation($jobfile,$outfile,$myioservice);
//  $myFile = $target_path.”/testFile.xml”;
// $fh = fopen($myFile, ‘w’);
// fwrite($fh, $result);
// fclose($fh);

//during testing we use the request to set up the job
//this will not delete the directory when the job has completed
//$result = requestsimulation($jobfile,$myioservice);
echo $result;
$resultlink=”http://webaddress.ac.uk/iogatest/results/”.$currenttime;
echo “Result will be available at:”;
echo(“<a href=\”".$resultlink.”\”>click here to access the results directory</a>”);
//echo $resultlink;
//$myioservice -> id = $result;
//runrequestedsimulation($myioservice);

?>

Using the google gadget API

http://code.google.com/apis/gadgets/

gadgets may be edited using the google gadget editor at

http://code.google.com/apis/gadgets/docs/tools.html#GGE

A simple example shown below is as follows

<?xml version=”1.0″ encoding=”UTF-8″ ?>
<Module>
<ModulePrefs title=”Set Title Example”>
<Require feature=”settitle”/>
</ModulePrefs>
<Content type=”html”>
<![CDATA[
<script type="text/javascript">
function changeTitle(form) {
var newTitle = form.inputbox.value;
gadgets.window.setTitle(newTitle);
}
</script>
<FORM NAME="myform" ACTION="" METHOD="GET">Change the Gadget Title: <BR><BR>
<INPUT TYPE="text" NAME="inputbox" VALUE=""><BR><BR>
<INPUT TYPE="button" NAME="button" Value="Change Title" onClick="changeTitle(this.form)">
</FORM>
<div id="content_div"></div>
]]>
</Content>
</Module>

Now from the code examples we can see below a function which may be adapted to make a request to the IOME server using the php client.

function getContacts() {
    var params = {};
    params[gadgets.io.RequestParameters.CONTENT_TYPE] = gadgets.io.ContentType.TEXT;
    var url = "http://gadget-doc-examples.googlecode.com/svn/trunk/opensocial-gadgets/Contacts.csv";
    gadgets.io.makeRequest(url, response, params);
  };

This uses the gadgets.io.makeRequest method described below

http://code.google.com/apis/gadgets/docs/reference/#gadgets.io.makeRequest

a more complete example is shown below

http://code.google.com/apis/gadgets/docs/remote-content.html#Fetch_text

Further ideas

Using the twitter php and python libraries to notify users of results may be achieved using the following API’s

http://dev.twitter.com/pages/libraries#php

http://dev.twitter.com/pages/libraries#python

The application consists of the following components

1. An instance of the IOME service storing licence information. The licence manager servers will update the licence data at regular intervals.
2. A service monitor script which checks that the licence monitor IOME service is up and running.
3. Scripts an each licence manager node which will query each of the licence servers and update the IOME licence monitor service with the licence information.
4. A php script on a web server (with the IOME php client tools). This is used to query the data, when the user clicks the web page the php script queries the IOME server.

The licence usage information is stored on an IOME server started using the following script.

#!/bin/bash
IOME_SIMNAME=”iolicencemonitor”

./startserver.sh &

sleep 2
IOME_WSPORT=`cat ${IOME_SIMNAME}0_port.txt`
echo port is $IOME_WSPORT

sleep 20

iogs addparam int ansys_research_inuse 0 7 0 $IOME_WSPORT localhost
iogs addparam int ansys_teaching_inuse 0 7 0 $IOME_WSPORT localhost
iogs addparam int matlab_inuse 0 7 0 $IOME_WSPORT localhost
iogs addparam int abaqus_inuse 0 7 0 $IOME_WSPORT localhost
iogs addparam int ansys_research_total 0 7 0 $IOME_WSPORT localhost
iogs addparam int ansys_teaching_total 0 7 0 $IOME_WSPORT localhost
iogs addparam int matlab_total 0 7 0 $IOME_WSPORT localhost
iogs addparam int abaqus_total 0 7 0 $IOME_WSPORT localhost

./servicemonitor.sh &

The servicemonitor checks the service status and alerts, by e-mail the administrator of any diffculties.

#!/bin/bash
IOME_SIMNAME=”iolicencemonitor”
sleep 2
IOME_WSPORT=`cat ${IOME_SIMNAME}0_port.txt`
echo port is $IOME_WSPORT

iogs addparam int servicestatus 1 7 0 $IOME_WSPORT localhost

ADMINEMAIL=”myemail@institution.ac.uk”

#loop around and get status every minute update IOME server
SERVICESTATUS=`iogs getparam int servicestatus 0 $IOME_WSPORT localhost`
echo “Service status is $SERVICESTATUS\n” > temp.txt
cat mes.txt temp.txt > newmes.txt
mail -s “message from iolicencemonitor” myemail@institution@email.ac.uk < newmes.txt

#if get status param from webv service fails email admin
#and infomr that web service has failed

loopvar=”testing”
loopdelay=”120″

while [ "$loopvar" != "end" ]
do
sleep $loopdelay
#check iome server
SERVICESTATUS=`iogs getparam int servicestatus 0 $IOME_WSPORT localhost`
if [ $SERVICESTATUS != "1" ]
then
loopdelay=10200
echo “Service status is $SERVICESTATUS IOME licence monitor server not responding” > temp.txt
cat mes.txt temp.txt > newmes.txt
mail -s “message from iolicencemonitor” $ADMINEMAIL < newmes.txt
fi

done

The licence monitor script running on each of the licence managers is shown below. There are separate functions for each package licence which update the IOME server

#!/bin/bash

function extractifl1()
{
# echo “Ansys_Research” $6  ${11}  $IOME_WSPORT
iogs setparam int ansys_research_total $6  $IOME_VARS
iogs setparam int ansys_research_inuse ${11} $IOME_VARS
}

function extractifl2()
{
# echo “Ansys_Teaching” $6  ${11}
iogs setparam int ansys_teaching_total $6  $IOME_VARS
iogs setparam int ansys_teaching_inuse ${11} $IOME_VARS
}

function extractmt()
{
#  echo “Matlab_Total” $6  ${11}
iogs setparam int matlab_total $6  $IOME_VARS
iogs setparam int matlab_inuse ${11} $IOME_VARS
}

# IOME_SIMNAME=”iolicencemonitor”
sleep 2
IOME_VARS=”0  60002  suilven.shef.ac.uk”

loopvar=”testing”
loopdelay=”5s”

#
while [ "$loopvar" != "end" ]
do

sleep $loopdelay

# ansys_research
tmpv=`(/usr/local/packages/ansys_inc/shared_files/licensing/linx64/lmutil lmstat -a -c /usr/local/packages//ansys_inc/shared_files/licensing/license.dat  | grep ‘Total’ ) | grep ‘aa_r:’`
# echo “var= $tmpv”
extractifl1 $tmpv

#ansys_teaching
tmpv=`(/usr/local/packages/ansys_inc/shared_files/licensing/linx64/lmutil lmstat -a -c /usr/local/packages//ansys_inc/shared_files/licensing/license.dat  | grep ‘Total’ ) | grep ‘t_a:’`
# echo “var= $tmpv”
extractifl2 $tmpv

#matlab_total
tmpv=`(/usr/local/packages/matlab/etc/lmstat -a -c /usr/local/packages/matlab/etc/license.dat  | grep ‘Total’ ) | grep ‘MATLAB:’`
#  echo $tmpv
extractmt $tmpv

done
[flexlm@licserv ~]$c/license.dat  | grep ‘Total’ ) | grep ‘MATLAB:’`
#  echo $tmpv
extractmt $tmpv

The php script on the web server is

<html>
<head>
<title>IOME PHP Licence monitor test</title>
</head>
<body>

<?php echo ‘<p>This is the IOME licence monitor test.</p>’;
require_once(‘iome/iome.php’);
$ios = new ioservice;
$ios->server = “machine.institution.ac.uk”;
$ios->port = XXXXXX;
$ios->id = 0;
$ios->method = 2;
$client = $ios->client();

print(“ansys_research_inuse/total “.getparamint(“ansys_research_inuse”, $ios).” “.getparamint(“ansys_research_total”, $ios).”\n”);
?>
<br />

<?php
print(“ansys_teaching_inuse/total “.getparamint(“ansys_teaching_inuse”, $ios).” “.getparamint(“ansys_teaching_total”, $ios).”\n”);
?>
<br />
<?php
print(“abaqus_inuse/total “.getparamint(“abaqus_inuse”, $ios).” “.getparamint(“abaqus_total”, $ios).”\n”);
?>
<br />

<?php
print(“matlab_inuse/total “.getparamint(“matlab_inuse”, $ios).” “.getparamint(“matlab_total”, $ios).”\n”);
?>
<br />
</html>

The installation of the service required the following tasks

• Installation of IOME on each server
• Preparation of the web server
• Setting firewalls to allow each server to make requests to the IOME licence monitor service.

Using IOME the complete installation was a quick process taking only hour.

Given the complexity of the problems that are modelled using magnetohydrodynamics it is a challenge for researchers to provide a method for repeating such computations. This is due to the number of processing stages and the parametric spaces which might be explored.

We have developed a system which enables computational MHD practicioners to generate metadata enabling

1. Reconstruction and repetition of  computational models (rather like a workflow e.g. taverna)
2. Automated generation of XML descriptions of individual models, which maybe be tranformed using an appropriate XSLT to present the salient data in a useful way for the researcher.

The automated metadata capture technique is similar to that used by the Cambridge based e-Minerals project in the National institute of environemnetal e-Science. The basic approach is as follows

The researcher provides a series of template files used to describe the initial system, the physical problem, pre-processing scripts to generate additional data required by the model. A driver script for the simulation is set up and this is used to create models. When each model is run it generates and IOME-ML based description of the model and example of invoking iome is as follows

runmodeliome mod mymodel.sh

The mod keyword lets IOME know that we need to generate and run a model, the basic description of which is described in the mymodel.sh script file.

A model output file is generated e.g. mymodel1.xml this may be used to re run the model as follows

runmodeliome mymodel.xml mymodelname

The scripting enables different actions to be switched on and off and specified  parameters to be modified rather like a parametric sweep.

The implementation has been tested with the Versatile advection code for MHD. A couple of additional points should be noted. This is a bespoke tool for MHD and thus the full complexity offered by workflow tools such as taverna was not required. It appears the MHD researchers already have a markup at their disposal e.g. the FSML it is believed that this level of detail is not currentlyrequired. Following along the lines fo NIEES’s automated metadata capture method it was found that one could easily develop the metadata capture tool for Solar Science using the IOME toolkit.

• MHD Using the Versatile advection code
• MHD fusion markup language
• Taverna
• OPM provenance model
• Automatic metadata capture

One of the features of the complex modelling systems is to guarantee the correct files for describing the full model such version control/provenance management may be acheive using some thing like the Open provenance markup.

A further advantage of our method for collecting information about MHD modelling runs is that it enables these runs to be easily managed through a browser accessible interface and additional feature is the possibility of tuning model parameters  as a computational steered application. At this point some form provenance management is required to maintain a track of revisions to the steering state of the model. Our approach is to use the IOME-ML script to summarise the main characteristics and job dependencies.

The problem we have set up is a genetic algorithm service for which users may provide a function and a genetic algorithm is used to determine the global minima. The results are returned via e-mail and via ftp to a results page hosted by the job service.

The example we have provided highlights a number of techniques for

• Testing the implementation (compute server side and web server side)
• Transferring data to and from web server and compute server using CURL and ftp
• Submitting a simulation from the web server (using interactive, batch submit or request test method)

Here are some of the problems.

• Sending results back to a secured and managed ftp service it was found that the php script creating a results folder had to use both mkdir(), and chmod functions and explicitly set permissions to 777. This is due to the fact that the scripts run as the web server apache user and any folders created are owned by that user. Attempts to ftp via a bash script or curl script result in permissions errors.
• During testing the submitsimulation method deletes the job folder upon completion of the job. So we lose any output. During testing we therefore simply copy the folder contents back to a testing folder.
• When running the matlab job on the compute cluster and via the scheduler (sun grid engine) matlab does not successfully print the generated figures. (see posting by Loren). There are two solutions to this problem. The first is to simply run the job on the server and not through the scheduler. The second method is to run two matlab jobs one which is computationally intensive and is submitted to the scheduler. The other application runs upon job completion and runs matlab on the server to generate the figure output.

A good success is that this is the first run for which we have submitted an application test interactively. With theRunSimulation command the server request waits for job completion. This is excellent because we may now extract relevant results direct from an application response.

Even with this method there is a minor problm which is described at

Source forge tracker it is hoped that this will be resolved fairly rapidly. The workaround is to make use of the ftp method to email the results file back to the web server, which we have successfully tested.

The next stage of our testing is to test the php parser routines for processing the IOME XML data returned by the remote web service. Further improvements include breaking the matlab job into two parts.

• Compute intensive and runs on compute cluster (may use parallel matlab to distribute matlab problem)
• Matlab job which runs on compute server and generates figures for the results

See proposed developments at

http://iomes.wordpress.com/2010/03/24/prioritisation-for-future-development-of-iome-and-caiman/

The following are tasks for completion in the short term

• PRIORITY review CAIMAN user recommendation
• IMPLEMENTAION COMPLETE NEED TO TESTComplete GA example as a web service using IOME (completed php client, but requires additional example which processes the result in realtime and returns xml results immediately)
• DONE UPDATED TO VER 2 and sychronised repository Test the new development version
• DONE need to document Test metadata generation for VAC
• REQUIRED NOT STARTED Update fortran implementation steerable vac model (shallow water problem)
• DONE Generate optimisation based service from user reecommendation

All this can either be built into the core IOME libraries or even better use an already existing and reliable tool such as CURL. I would prefer the second option to avoid reinventing the wheel.

The following scenarios exemplify the need of dealing with data/results files when steering a remote application.

Example 1:  Remote steering a CFD of FE package such as Fluent or Abaqus.

A small steering file is sent to the remote worker.

Remote worker sends back a large results file.

Example 2:  The results file generated above is then used as a start-up file.

Therefore it becomes a large data file to be sent from the steering client to the worker client.  The job produces a new file or modifies the input  file and sends it back.

The numbers of files to be used for data or results can vary in numbers or sizes but the basic picture remains the same which necessitates the need to deal with file transfers in a systematic manner.  The metadata definition of IOME will already have information about these I/O files which it can use to initiate, for example CURL, transfers at appropriate times. Or the job script itself can contain file transfer calls as and when needed without the need to code these data dependencies into IOME.  Further enhancements can be; to define some of the data-files as ‘transfer-only-if-it-does-not-exist-on-worker” and some of the output files as “transfer-if-exists-on-the-worker”

The results are usually a collection of a few small output files and optionally perhaps a few large application specify results files.

Once a batch job starts, there are no options to stear or control the computation apart from via the already provided stearing file.

Currently, for an average user the job submission involves login into a power server, preparing a script and submitting it to the batch processing system specific to that HPC cluster.

If there was a way of submitting such jobs via a web-form or via a custom-GUI and receiving the results back automatically,  life would be so much more easier for the application users who are not necessarily interested in the inner workings of an HPC cluster.

GUI interface builder of MATLAB used in association with the Matlab IOME toolkit is one way forward for achieving such an objective.   Results can also be emailed to the user  ‘in a similar manner to it is implemented in  CAIMAN’  as a series of attached files.

Apart from the ease of use, another advantage of this method of job submission would be the ability to submit jobs to any participating HPC cluster without the penalty of overheads associated with such an action in terms of user effort.

Adding such a capability may not always   be necessary or needed.  But doing so can achieve CONDOR-like capabilities without having to run a Condor Pool.

By removing these early features it is hoped that

• We will be able to improve the performance and quality of the next release distribution
• Make it easier to understand and to provide good documentation

The original source distribution has been saved as part of the first release of IOME i.e. version 1.o.

This major modification occured at revision 457,458 and 459 to the source forge dev branch

Future developements will be now be found in version 1.1.

The following procedure applies for a root user installing iome in the /usr/local directory

• change directory to /usr/local
• Download the correct binary distribution (e.g. by visiting the project web site or using wget as follows) sudo wget http://iome.googlecode.com/files/iorel1_bin_linux_ubuntu_amd64.tgz.gz
• Extract the distribution using sudo tar -zxvf  iorel1_bin_linux_ubuntu_amd64.tgz.gz
• sudo mv ioubuntulin_bin_amd64 iome
• cd iome
• sudo cp iome.sh /etc/profile.d

The iome.sh contains the following lines

The procedure is easily adapted for use installation in the home area. The distribution does not yet contain the iome.sh script which will be generalised for windows and other linux ditributions and also for final installation of the source distribution.