Class MPIExec
- java.lang.Object
-
- edu.isi.pegasus.planner.cluster.aggregator.Abstract
-
- edu.isi.pegasus.planner.cluster.aggregator.MPIExec
-
- All Implemented Interfaces:
JobAggregator
public class MPIExec extends Abstract
This class aggregates the smaller jobs in a manner such that they are launched at remote end, by mpiexec on n nodes where n is the nodecount associated with the aggregated job that is being lauched by mpiexec. The executable mpiexec is a Pegasus tool distributed in the Pegasus worker package, and can be usually found at $PEGASUS_HOME/bin/mpiexec.- Version:
- $Revision$
- Author:
- Karan Vahi vahi@isi.edu
-
-
Field Summary
Fields Modifier and Type Field Description static java.lang.String
COLLAPSE_LOGICAL_NAME
The logical name of the transformation that is able to run multiple jobs via mpi.static java.lang.String
EXECUTABLE_BASENAME
The basename of the executable that is able to run multiple jobs via mpi.-
Fields inherited from class edu.isi.pegasus.planner.cluster.aggregator.Abstract
CLUSTERED_JOB_PREFIX, DERIVATION_NAMESPACE, DERIVATION_VERSION, mBag, mClusteredADag, mDirectory, mGridStartFactory, mLogger, MONITORD_COMMENT_MARKER, mProps, mSiteStore, mTCHandle, TRANSFORMATION_NAMESPACE, TRANSFORMATION_VERSION
-
Fields inherited from interface edu.isi.pegasus.planner.cluster.JobAggregator
VERSION
-
-
Constructor Summary
Constructors Constructor Description MPIExec()
The default constructor.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description boolean
abortOnFristJobFailure()
Returns a boolean indicating whether to fail the aggregated job on detecting the first failure during execution of constituent jobs.java.lang.String
aggregatedJobArguments(AggregatedJob job)
Returns the arguments with which theAggregatedJob
needs to be invoked with.private void
complain(java.lang.String message, java.lang.String id, java.lang.String value)
Complains for invalid values passed in profilesboolean
entryNotInTC(java.lang.String site)
Determines whether there is NOT an entry in the transformation catalog for the job aggregator executable on a particular site.java.io.File
generatePMCInputFile(Graph job, java.lang.String name, boolean isClustered)
Writes out the input file for the aggregated jobjava.lang.String
getClusterExecutableBasename()
Returns the executable basename of the clustering executable used.java.lang.String
getClusterExecutableLFN()
Returns the logical name of the transformation that is used to collapse the jobs.java.lang.String
getCPURequirementsArgument(Job job)
Looks at the profile keys associated with the job to generate the argument string fragment containing the cpu required for the job.java.lang.String
getExtraArguments(Job job)
Looks at the profile key for pegasus::pmc_task_arguments to determine if extra arguments are required for the task.java.lang.String
getMemoryRequirementsArgument(Job job)
Looks at the profile keys associated with the job to generate the argument string fragment containing the memory required for the job.java.lang.String
getPriorityArgument(Job job)
Looks at the profile keys associated with the job to generate the argument string fragment containing the priority to be associated for the job.void
initialize(ADag dag, PegasusBag bag)
Initializes the JobAggregator impelementationvoid
makeAbstractAggregatedJobConcrete(AggregatedJob job)
Enables the abstract clustered job for execution and converts it to it's executable form.void
setAbortOnFirstJobFailure(boolean fail)
Setter method to indicate , failure on first consitutent job should result in the abort of the whole aggregated job.boolean
topologicalOrderingRequired()
A boolean indicating whether ordering is important while traversing through the aggregated job.protected java.io.File
writeOutInputFileForJobAggregator(AggregatedJob job)
Writes out the input file for the aggregated job-
Methods inherited from class edu.isi.pegasus.planner.cluster.aggregator.Abstract
constructAbstractAggregatedJob, constructAbstractAggregatedJob, entryNotInTC, getCommentString, getCompleteTranformationName, getTCEntry, setDirectory
-
-
-
-
Field Detail
-
COLLAPSE_LOGICAL_NAME
public static final java.lang.String COLLAPSE_LOGICAL_NAME
The logical name of the transformation that is able to run multiple jobs via mpi.- See Also:
- Constant Field Values
-
EXECUTABLE_BASENAME
public static java.lang.String EXECUTABLE_BASENAME
The basename of the executable that is able to run multiple jobs via mpi.
-
-
Method Detail
-
initialize
public void initialize(ADag dag, PegasusBag bag)
Initializes the JobAggregator impelementation- Specified by:
initialize
in interfaceJobAggregator
- Overrides:
initialize
in classAbstract
- Parameters:
dag
- the workflow that is being clustered.bag
- the bag of objects that is useful for initialization.
-
makeAbstractAggregatedJobConcrete
public void makeAbstractAggregatedJobConcrete(AggregatedJob job)
Enables the abstract clustered job for execution and converts it to it's executable form. Also associates the post script that should be invoked for the AggregatedJob- Specified by:
makeAbstractAggregatedJobConcrete
in interfaceJobAggregator
- Overrides:
makeAbstractAggregatedJobConcrete
in classAbstract
- Parameters:
job
- the abstract clustered job
-
writeOutInputFileForJobAggregator
protected java.io.File writeOutInputFileForJobAggregator(AggregatedJob job)
Writes out the input file for the aggregated job- Overrides:
writeOutInputFileForJobAggregator
in classAbstract
- Parameters:
job
- the aggregated job- Returns:
- path to the input file
-
generatePMCInputFile
public java.io.File generatePMCInputFile(Graph job, java.lang.String name, boolean isClustered)
Writes out the input file for the aggregated job- Parameters:
job
- the aggregated jobname
- the name of PMC file to be generatedisClustered
- a boolean indicating whetehre the graph belongs to a clustered job or not.- Returns:
- path to the input file
-
getClusterExecutableLFN
public java.lang.String getClusterExecutableLFN()
Returns the logical name of the transformation that is used to collapse the jobs.- Returns:
- the the logical name of the collapser executable.
- See Also:
COLLAPSE_LOGICAL_NAME
-
getClusterExecutableBasename
public java.lang.String getClusterExecutableBasename()
Returns the executable basename of the clustering executable used.- Returns:
- the executable basename.
- See Also:
EXECUTABLE_BASENAME
-
entryNotInTC
public boolean entryNotInTC(java.lang.String site)
Determines whether there is NOT an entry in the transformation catalog for the job aggregator executable on a particular site.- Parameters:
site
- the site at which existence check is required.- Returns:
- boolean true if an entry does not exists, false otherwise.
-
aggregatedJobArguments
public java.lang.String aggregatedJobArguments(AggregatedJob job)
Returns the arguments with which theAggregatedJob
needs to be invoked with. At present any empty argument string is returned.- Specified by:
aggregatedJobArguments
in classAbstract
- Parameters:
job
- theAggregatedJob
for which the arguments have to be constructed.- Returns:
- argument string
-
setAbortOnFirstJobFailure
public void setAbortOnFirstJobFailure(boolean fail)
Setter method to indicate , failure on first consitutent job should result in the abort of the whole aggregated job. Ignores any value passed, as MPIExec does not handle it for time being.- Parameters:
fail
- indicates whether to abort or not .
-
abortOnFristJobFailure
public boolean abortOnFristJobFailure()
Returns a boolean indicating whether to fail the aggregated job on detecting the first failure during execution of constituent jobs.- Returns:
- boolean indicating whether to fail or not.
-
topologicalOrderingRequired
public boolean topologicalOrderingRequired()
A boolean indicating whether ordering is important while traversing through the aggregated job.- Returns:
- false
-
getCPURequirementsArgument
public java.lang.String getCPURequirementsArgument(Job job)
Looks at the profile keys associated with the job to generate the argument string fragment containing the cpu required for the job.- Parameters:
job
- the Job for which memory requirements has to be determined.- Returns:
- the arguments fragment else empty string
-
getMemoryRequirementsArgument
public java.lang.String getMemoryRequirementsArgument(Job job)
Looks at the profile keys associated with the job to generate the argument string fragment containing the memory required for the job.- Parameters:
job
- the Job for which memory requirements has to be determined.- Returns:
- the arguments fragment else empty string
-
getPriorityArgument
public java.lang.String getPriorityArgument(Job job)
Looks at the profile keys associated with the job to generate the argument string fragment containing the priority to be associated for the job. Negative values are allowed- Parameters:
job
- the Job for which memory requirements has to be determined.- Returns:
- the arguments fragment else empty string
-
getExtraArguments
public java.lang.String getExtraArguments(Job job)
Looks at the profile key for pegasus::pmc_task_arguments to determine if extra arguments are required for the task.- Parameters:
job
- the constitunt job for which extra arguments need to be determined- Returns:
- the arguments if they are present, or an empty string
-
complain
private void complain(java.lang.String message, java.lang.String id, java.lang.String value)
Complains for invalid values passed in profiles- Parameters:
message
- the string describing the error messageid
- id of the jobvalue
- value of the CPU passed
-
-