Package edu.isi.pegasus.planner.transfer
Class AbstractRefiner
- java.lang.Object
-
- edu.isi.pegasus.planner.transfer.AbstractRefiner
-
- Direct Known Subclasses:
MultipleFTPerXFERJobRefiner
,SingleFTPerXFERJobRefiner
public abstract class AbstractRefiner extends java.lang.Object implements Refiner
An abstract implementation that implements some of the common functions in the Refiner Interface and member variables that are required by all the refiners.- Version:
- $Revision$
- Author:
- Karan Vahi
-
-
Field Summary
Fields Modifier and Type Field Description protected ADag
mDAG
The ADag object associated with the Dag.protected LogManager
mLogger
The logging object which is used to log all the messages.protected PlannerOptions
mPOptions
The options passed to the planner at runtime.protected PegasusProperties
mProps
The handle to the properties object holding the properties relevant to Pegasus.protected RemoteTransfer
mRemoteTransfers
The handle to the Remote Transfers State machinery.protected TPT
mTPT
The handle to the Third Party State machinery.protected Implementation
mTXInterImplementation
The inter transfer implementation that the refiner requires.protected Implementation
mTXStageInImplementation
The stage-in transfer implementation that the refiner requires.protected Implementation
mTXStageOutImplementation
The stage-out transfer implementation that the refiner requires.protected Implementation
mTXSymbolicLinkImplementation
The stage-in symbolic link transfer implementation that refiner requires.protected XMLProducer
mXMLStore
The XML Producer object that records the actions.-
Fields inherited from interface edu.isi.pegasus.planner.transfer.Refiner
INTER_POOL_PREFIX, LOCAL_PREFIX, REGISTER_PREFIX, REMOTE_PREFIX, STAGE_IN_PREFIX, STAGE_OUT_PREFIX
-
-
Constructor Summary
Constructors Constructor Description AbstractRefiner(ADag dag, PegasusBag bag)
The overloaded constructor.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
addStageInXFERNodes(Job job, java.util.Collection<FileTransfer> files)
Default behaviour to preserve backward compatibility when the stage in and symbolic link jobs were not separated.void
addStageInXFERNodes(Job job, java.util.Collection<FileTransfer> files, java.util.Collection<FileTransfer> symlinkFiles)
Default behaviour to preserve backward compatibility when the stage in and symbolic link jobs were not separated.ADag
getWorkflow()
Returns a reference to the workflow that is being refined by the refiner.XMLProducer
getXMLProducer()
Returns a reference to the XMLProducer, that generates the XML fragment capturing the actions of the refiner.boolean
isSiteThirdParty(java.lang.String site, int type)
Returns whether a Site is third party enabled or not.protected void
logConfigMessages()
Logs configuration messages regarding the type of implementations loaded for various type of transfer node creations.boolean
refinerPreferenceForLocalTransferJobs(int type)
Boolean indicating Refiner preference for transfer jobs to run locally.boolean
refinerPreferenceForTransferJobLocation()
Boolean indicating whether the Transfer Refiner has a preference for where a transfer job is run.boolean
runTPTOnRemoteSite(java.lang.String site, int type)
Returns whether the third party transfers for a particular site are to be run on the remote site or the submit host.boolean
runTransferRemotely(java.lang.String site, int type)
Returns whether a Site prefers transfers to be run on it i.e remote transfers enabled.-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface edu.isi.pegasus.planner.transfer.Refiner
addInterSiteTXNodes, addJob, addRelation, addRelation, addStageOutXFERNodes, addStageOutXFERNodes, done, getDescription, loadImplementations
-
-
-
-
Field Detail
-
mTXStageInImplementation
protected Implementation mTXStageInImplementation
The stage-in transfer implementation that the refiner requires.
-
mTXSymbolicLinkImplementation
protected Implementation mTXSymbolicLinkImplementation
The stage-in symbolic link transfer implementation that refiner requires.
-
mTXInterImplementation
protected Implementation mTXInterImplementation
The inter transfer implementation that the refiner requires.
-
mTXStageOutImplementation
protected Implementation mTXStageOutImplementation
The stage-out transfer implementation that the refiner requires.
-
mDAG
protected ADag mDAG
The ADag object associated with the Dag. This is the object to which the transfer nodes are added. This object is initialised in the TransferEngine.
-
mProps
protected PegasusProperties mProps
The handle to the properties object holding the properties relevant to Pegasus.
-
mPOptions
protected PlannerOptions mPOptions
The options passed to the planner at runtime.
-
mLogger
protected LogManager mLogger
The logging object which is used to log all the messages.
-
mTPT
protected TPT mTPT
The handle to the Third Party State machinery.
-
mRemoteTransfers
protected RemoteTransfer mRemoteTransfers
The handle to the Remote Transfers State machinery.
-
mXMLStore
protected XMLProducer mXMLStore
The XML Producer object that records the actions.
-
-
Constructor Detail
-
AbstractRefiner
public AbstractRefiner(ADag dag, PegasusBag bag)
The overloaded constructor.- Parameters:
dag
- the workflow to which transfer nodes need to be added.bag
- the bag of initialization objects.
-
-
Method Detail
-
getWorkflow
public ADag getWorkflow()
Returns a reference to the workflow that is being refined by the refiner.- Specified by:
getWorkflow
in interfaceRefiner
- Returns:
- ADAG object.
-
addStageInXFERNodes
public void addStageInXFERNodes(Job job, java.util.Collection<FileTransfer> files, java.util.Collection<FileTransfer> symlinkFiles)
Default behaviour to preserve backward compatibility when the stage in and symbolic link jobs were not separated. The symlink transfer files are added back into the files collection and passed onto legacy implementations. Refiners that want to distinguish between symlink and stagein jobs should over ride this method.- Specified by:
addStageInXFERNodes
in interfaceRefiner
- Parameters:
job
-Job
object corresponding to the node to which the files are to be transferred to.files
- Collection ofFileTransfer
objects containing the information about source and destURL's.symlinkFiles
- Collection ofFileTransfer
objects containing source and destination file url's for symbolic linking on compute site.
-
addStageInXFERNodes
public void addStageInXFERNodes(Job job, java.util.Collection<FileTransfer> files)
Default behaviour to preserve backward compatibility when the stage in and symbolic link jobs were not separated.- Parameters:
job
-Job
object corresponding to the node to which the files are to be transferred to.files
- Collection ofFileTransfer
objects containing the information about source and destURL's.
-
getXMLProducer
public XMLProducer getXMLProducer()
Returns a reference to the XMLProducer, that generates the XML fragment capturing the actions of the refiner. This is used for provenace purposes.- Specified by:
getXMLProducer
in interfaceRefiner
- Returns:
- XMLProducer
-
refinerPreferenceForTransferJobLocation
public boolean refinerPreferenceForTransferJobLocation()
Boolean indicating whether the Transfer Refiner has a preference for where a transfer job is run. By default, Refiners dont advertise any preference as to where transfer jobs run.- Specified by:
refinerPreferenceForTransferJobLocation
in interfaceRefiner
- Returns:
- false
-
refinerPreferenceForLocalTransferJobs
public boolean refinerPreferenceForLocalTransferJobs(int type)
Boolean indicating Refiner preference for transfer jobs to run locally. This method should be called only if refinerPreferenceForTransferJobLocation is true for a refiner.- Specified by:
refinerPreferenceForLocalTransferJobs
in interfaceRefiner
- Parameters:
type
- the type of transfer job for which the URL is being constructed. Should be one of the following: stage-in stage-out inter-pool transfer- Returns:
- boolean refiner preference for transfer job to run locally or not.
-
runTransferRemotely
public boolean runTransferRemotely(java.lang.String site, int type)
Returns whether a Site prefers transfers to be run on it i.e remote transfers enabled.- Specified by:
runTransferRemotely
in interfaceRefiner
- Parameters:
site
- the name of the site.type
- the type of transfer job for which the URL is being constructed. Should be one of the following: stage-in stage-out inter-pool transfer- Returns:
- true if site is setup for remote transfers
- See Also:
Job.STAGE_IN_JOB
,Job.INTER_POOL_JOB
,Job.STAGE_OUT_JOB
-
isSiteThirdParty
public boolean isSiteThirdParty(java.lang.String site, int type)
Returns whether a Site is third party enabled or not.- Specified by:
isSiteThirdParty
in interfaceRefiner
- Parameters:
site
- the name of the site.type
- the type of transfer job for which the URL is being constructed. Should be one of the following: stage-in stage-out inter-pool transfer- Returns:
- true pool is third party enabled false pool is not third party enabled.
- Throws:
java.lang.IllegalArgumentException
- See Also:
Job.STAGE_IN_JOB
,Job.INTER_POOL_JOB
,Job.STAGE_OUT_JOB
-
runTPTOnRemoteSite
public boolean runTPTOnRemoteSite(java.lang.String site, int type)
Returns whether the third party transfers for a particular site are to be run on the remote site or the submit host.- Specified by:
runTPTOnRemoteSite
in interfaceRefiner
- Parameters:
site
- the name of the site.type
- the type of transfer job for which the URL is being constructed. Should be one of the following: stage-in stage-out inter-pool transfer- Returns:
- true if the transfers are to be run on remote site, else false.
- See Also:
Job.STAGE_IN_JOB
,Job.INTER_POOL_JOB
,Job.STAGE_OUT_JOB
-
logConfigMessages
protected void logConfigMessages()
Logs configuration messages regarding the type of implementations loaded for various type of transfer node creations.
-
-