Package edu.isi.pegasus.planner.cluster
Class Abstract
- java.lang.Object
-
- edu.isi.pegasus.planner.cluster.Abstract
-
- All Implemented Interfaces:
Clusterer
- Direct Known Subclasses:
Vertical
public abstract class Abstract extends java.lang.Object implements Clusterer
An abstract clusterer that the other clusterers can extend. The abstract implementation treats each partition as a single cluster. It has callouts to determine the ordering of the jobs in the cluster, and the input/output files for the clustered jobs.- Version:
- $Revision$
- Author:
- Karan Vahi
-
-
Field Summary
Fields Modifier and Type Field Description protected java.util.Collection<PCRelation>
mClusteredRelations
The collection of relations, that is constructed for the clustered workflow.protected JobAggregatorInstanceFactory
mJobAggregatorFactory
The handle to the job aggregator factory.protected LogManager
mLogger
The handle to the logger object.protected java.util.Map<java.lang.String,Job>
mPartitionClusterMap
A Map that indexes the partition ID to the clustered job.protected PegasusProperties
mProps
The handle to the properties object holding all the properties.protected ADag
mScheduledDAG
ADag object containing the jobs that have been scheduled by the site selector.protected java.util.Map<java.lang.String,Job>
mSubInfoMap
A Map to store all the job(Job) objects indexed by their logical ID found in the dax.
-
Constructor Summary
Constructors Constructor Description Abstract()
The Abstract constructor.
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description protected void
addJob(Job job)
Adds jobs to the internal map of jobs that is maintained by the clusterer.protected void
associate(Partition p, Job job)
Maps the partition to the corresponding clustered job.protected Job
clusteredJob(Partition p)
Returns the job corresponding to a partition.protected Job
clusteredJob(java.lang.String id)
Returns the job corresponding to a partition.protected java.lang.String
constructClusteredJobID(Partition partition)
Returns the ID for the clustered job corresponding to a partition.void
determineClusters(Partition partition)
It creates a single clustered job for the partition.abstract void
determineInputOutputFiles(AggregatedJob job, java.util.List<Job> orderedJobs)
Determine the input and output files of the job on the basis of the order of the constituent jobs in the AggregatedJob.ADag
getClusteredDAG()
Returns the clustered workflow.protected Job
getJob(java.lang.String id)
Returns the job object corresponding to the id of the job.protected java.lang.String
getLogicalNameForJobs(java.util.List<Job> jobs)
Returns the logical names for the jobs.void
initialize(ADag dag, PegasusBag bag)
Initializes the Clusterer impelementationabstract java.util.List<java.lang.String>
order(Partition p)
Returns the nodes in the partition as a List in a particular order.void
parents(java.lang.String partitionID, java.util.List parents)
Associates the relations between the partitions with the corresponding relations between the clustered jobs that are created for each Partition.-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface edu.isi.pegasus.planner.cluster.Clusterer
description
-
-
-
-
Field Detail
-
mSubInfoMap
protected java.util.Map<java.lang.String,Job> mSubInfoMap
A Map to store all the job(Job) objects indexed by their logical ID found in the dax. This should actually be in the ADag structure.
-
mPartitionClusterMap
protected java.util.Map<java.lang.String,Job> mPartitionClusterMap
A Map that indexes the partition ID to the clustered job.
-
mLogger
protected LogManager mLogger
The handle to the logger object.
-
mProps
protected PegasusProperties mProps
The handle to the properties object holding all the properties.
-
mJobAggregatorFactory
protected JobAggregatorInstanceFactory mJobAggregatorFactory
The handle to the job aggregator factory.
-
mClusteredRelations
protected java.util.Collection<PCRelation> mClusteredRelations
The collection of relations, that is constructed for the clustered workflow.
-
mScheduledDAG
protected ADag mScheduledDAG
ADag object containing the jobs that have been scheduled by the site selector.
-
-
Method Detail
-
order
public abstract java.util.List<java.lang.String> order(Partition p) throws ClustererException
Returns the nodes in the partition as a List in a particular order. The iterator of the list returns the nodes in the order determined by the clusterer.- Parameters:
p
- the partition whose nodes have to be ordered.- Returns:
- an ordered List of
String
objects that are the ID's of the nodes. - Throws:
ClustererException
- in case of error.
-
determineInputOutputFiles
public abstract void determineInputOutputFiles(AggregatedJob job, java.util.List<Job> orderedJobs)
Determine the input and output files of the job on the basis of the order of the constituent jobs in the AggregatedJob.- Parameters:
job
- theAggregatedJob
orderedJobs
- the List of Jobs that is ordered as determined by the clustererr- Throws:
ClustererException
- in case of error.
-
initialize
public void initialize(ADag dag, PegasusBag bag) throws ClustererException
Initializes the Clusterer impelementation- Specified by:
initialize
in interfaceClusterer
- Parameters:
dag
- the workflow that is being clustered.bag
- the bag of objects that is useful for initialization.- Throws:
ClustererException
- in case of error.
-
determineClusters
public void determineClusters(Partition partition) throws ClustererException
It creates a single clustered job for the partition. If there is only one job in the partition, then no clustering happens.- Specified by:
determineClusters
in interfaceClusterer
- Parameters:
partition
- the partition for which the clusters need to be determined.- Throws:
ClustererException
- if the clustering executable is not installed on the remote site or if all the jobs in the partition are not scheduled on the same site.
-
getLogicalNameForJobs
protected java.lang.String getLogicalNameForJobs(java.util.List<Job> jobs)
Returns the logical names for the jobs. It picks the first job in the list and uses that to construct the name- Parameters:
jobs
- List of jobs- Returns:
- name
-
parents
public void parents(java.lang.String partitionID, java.util.List parents) throws ClustererException
Associates the relations between the partitions with the corresponding relations between the clustered jobs that are created for each Partition.- Specified by:
parents
in interfaceClusterer
- Parameters:
partitionID
- the id of a partition.parents
- the list ofString
objects that contain the id's of the parents of the partition.- Throws:
ClustererException
- in case of clustered job not being found for a partition.
-
getClusteredDAG
public ADag getClusteredDAG() throws ClustererException
Returns the clustered workflow.- Specified by:
getClusteredDAG
in interfaceClusterer
- Returns:
- the
ADag
object corresponding to the clustered workflow. - Throws:
ClustererException
- in case of error.
-
constructClusteredJobID
protected java.lang.String constructClusteredJobID(Partition partition)
Returns the ID for the clustered job corresponding to a partition.- Parameters:
partition
- the partition.- Returns:
- the ID of the clustered job
-
addJob
protected void addJob(Job job)
Adds jobs to the internal map of jobs that is maintained by the clusterer.- Parameters:
job
- the job being added
-
getJob
protected Job getJob(java.lang.String id)
Returns the job object corresponding to the id of the job.- Parameters:
id
- the id of the job- Returns:
- the corresponding job.
-
associate
protected void associate(Partition p, Job job)
Maps the partition to the corresponding clustered job.- Parameters:
p
- the partition being clustered.job
- the corresponding clustered job.
-
clusteredJob
protected Job clusteredJob(Partition p)
Returns the job corresponding to a partition.- Parameters:
p
- the partition for which the clustered job is reqd.- Returns:
- the corresponding job, else null in case of job is not found.
-
clusteredJob
protected Job clusteredJob(java.lang.String id)
Returns the job corresponding to a partition.- Parameters:
id
- the partition id.- Returns:
- the corresponding job, else null in case of job is not found.
-
-