Class Abstract
- java.lang.Object
-
- edu.isi.pegasus.planner.transfer.implementation.Abstract
-
- All Implemented Interfaces:
Implementation
- Direct Known Subclasses:
AbstractMultipleFTPerXFERJob
,AbstractSingleFTPerXFERJob
public abstract class Abstract extends java.lang.Object implements Implementation
An abstract implementation that implements some of the common functions in the Implementation Interface that are required by all the implementations.- Version:
- $Revision$
- Author:
- Karan Vahi
-
-
Field Summary
Fields Modifier and Type Field Description static java.lang.String
CHANGE_XBIT_TRANSFORMATION
The logical name of the transformation that creates directories on the remote execution pools.protected boolean
mAddNodesForSettingXBit
A boolean indicating whether chmod jobs should be created that set the xbit in case of executable staging.protected boolean
mChmodDisabledForAllSites
A boolean indicating whether chmod is disabled for all sites.protected java.util.Set
mDisabledChmodSites
The set of sites for which chmod job creation has to be disabled while doing executable staging.protected java.lang.String
mLocalIrodsEnv
The path to the irodsEnv file on the submit host (local pool).protected java.lang.String
mLocalIrodsEnvBasename
The basename of the user irodsEnv fileprotected java.lang.String
mLocalS3cfg
The path to the s3cfg file on the submit host (local pool).protected java.lang.String
mLocalS3cfgBasename
The basename of the user s3cfg fileprotected java.lang.String
mLocalUserProxy
The path to the user proxy on the submit host (local pool), that is picked up for use in transfer of proxies.protected java.lang.String
mLocalUserProxyBasename
The basename of the user proxy , that is picked up for use in transfer of proxies.protected LogManager
mLogger
The logging object which is used to log all the messages.protected PlannerOptions
mPOptions
Contains the various options to the Planner as passed by the user at runtime.protected PegasusProperties
mProps
The handle to the properties object holding the properties relevant to Pegasus.protected Refiner
mRefiner
The handle to the refiner that loaded this implementation.protected SiteStore
mSiteStore
The handle to the Pool Info Provider.protected TransformationCatalog
mTCHandle
The handle to the Transformation Catalog.static java.lang.String
NOOP_PREFIX
The prefix for the NoOP jobs that are created.static java.lang.String
SET_XBIT_PREFIX
The prefix for the jobs which are added to set X bit for the staged executables.static java.lang.String
XBIT_DERIVATION_NS
The derivation namespace for the setXBit jobs.static java.lang.String
XBIT_DERIVATION_VERSION
The version number for the derivations for setXBit jobs.static java.lang.String
XBIT_EXECUTABLE_BASENAME
The basename of the pegasus dirmanager executable.static java.lang.String
XBIT_TRANSFORMATION_NS
The transformation namespace for the setXBit jobs.static java.lang.String
XBIT_TRANSFORMATION_VERSION
The version number for the derivations for setXBit jobs.-
Fields inherited from interface edu.isi.pegasus.planner.transfer.Implementation
TRANSFER_UNIVERSE, VERSION
-
-
Constructor Summary
Constructors Constructor Description Abstract(PegasusBag bag)
The overloaded constructor, that is called by the Factory to load the class.
-
Method Summary
All Methods Instance Methods Concrete Methods Deprecated Methods Modifier and Type Method Description protected boolean
addSetXBitJobs(Job computeJob, Job txJob, java.util.Collection execFiles)
Adds the dirmanager to the workflow, that do a chmod on the files being staged.boolean
addSetXBitJobs(Job computeJob, java.lang.String txJobName, java.util.Collection execFiles, int transferClass)
Adds the dirmanager job to the workflow, that do a chmod on the files being staged.boolean
addSetXBitJobs(Job computeJob, java.lang.String txJobName, java.util.Collection execFiles, int transferClass, int xbitIndex)
Adds the dirmanager job to the workflow, that do a chmod on the files being staged.void
applyPriority(TransferJob job)
Applies priorities to the transfer jobs if a priority is specified in the properties file.boolean
checkAndTransferIrodsEnvFile(TransferJob job)
Deprecated.boolean
checkAndTransferProxy(TransferJob job)
Deprecated.protected void
construct(Job job, java.lang.String key, java.lang.String value)
Constructs a condor variable in the condor profile namespace associated with the job.Job
createNoOPJob(java.lang.String name)
It creates a NoOP job that runs on the submit host.protected Job
createSetXBitJob(FileTransfer file, java.lang.String name)
Creates a dirmanager job, that does a chmod on the file being staged.Job
createSetXBitJob(Job computeJob, java.util.Collection<FileTransfer> execFiles, int transferClass, int xbitIndex)
Adds the dirmanager job to the workflow, that do a chmod on the files being staged.protected Job
createSetXBitJob(java.util.Collection<FileTransfer> files, java.lang.String name, java.lang.String site)
Creates a dirmanager job, that does a chmod on the file being staged.private TransformationCatalogEntry
defaultXBitTCEntry(java.lang.String site)
Returns a default TC entry to be used in case entry is not found in the transformation catalog.protected java.util.Set
determineDisabledChmodSites(java.lang.String sites)
Builds up a set of disabled chmod sitesprotected boolean
disableChmodJobCreation(java.lang.String site)
Returns a boolean indicating whether to disable chmod job creation for a site or not.java.lang.String
getNOOPJobName(java.lang.String name, int counter)
Generates the name of the noop job , that is unique for the given workflow.protected java.lang.String
getPriority(TransferJob job)
Returns the priority for the transfer job as specified in the properties file.java.lang.String
getSetXBitJobName(java.lang.String name, int counter)
Generates the name of the setXBitJob , that is unique for the given workflow.void
setRefiner(Refiner refiner)
Sets the callback to the refiner, that has loaded this implementation.-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface edu.isi.pegasus.planner.transfer.Implementation
createTransferJob, doesPreserveXBit, getDescription, getTransformationCatalogEntry, useThirdPartyTransferAlways
-
-
-
-
Field Detail
-
CHANGE_XBIT_TRANSFORMATION
public static final java.lang.String CHANGE_XBIT_TRANSFORMATION
The logical name of the transformation that creates directories on the remote execution pools.- See Also:
- Constant Field Values
-
XBIT_EXECUTABLE_BASENAME
public static final java.lang.String XBIT_EXECUTABLE_BASENAME
The basename of the pegasus dirmanager executable.- See Also:
- Constant Field Values
-
XBIT_TRANSFORMATION_NS
public static final java.lang.String XBIT_TRANSFORMATION_NS
The transformation namespace for the setXBit jobs.- See Also:
- Constant Field Values
-
XBIT_TRANSFORMATION_VERSION
public static final java.lang.String XBIT_TRANSFORMATION_VERSION
The version number for the derivations for setXBit jobs.
-
XBIT_DERIVATION_NS
public static final java.lang.String XBIT_DERIVATION_NS
The derivation namespace for the setXBit jobs.- See Also:
- Constant Field Values
-
XBIT_DERIVATION_VERSION
public static final java.lang.String XBIT_DERIVATION_VERSION
The version number for the derivations for setXBit jobs.
-
SET_XBIT_PREFIX
public static final java.lang.String SET_XBIT_PREFIX
The prefix for the jobs which are added to set X bit for the staged executables.- See Also:
- Constant Field Values
-
NOOP_PREFIX
public static final java.lang.String NOOP_PREFIX
The prefix for the NoOP jobs that are created.- See Also:
- Constant Field Values
-
mLocalUserProxy
protected java.lang.String mLocalUserProxy
The path to the user proxy on the submit host (local pool), that is picked up for use in transfer of proxies.
-
mLocalUserProxyBasename
protected java.lang.String mLocalUserProxyBasename
The basename of the user proxy , that is picked up for use in transfer of proxies.
-
mLocalS3cfg
protected java.lang.String mLocalS3cfg
The path to the s3cfg file on the submit host (local pool).
-
mLocalS3cfgBasename
protected java.lang.String mLocalS3cfgBasename
The basename of the user s3cfg file
-
mLocalIrodsEnv
protected java.lang.String mLocalIrodsEnv
The path to the irodsEnv file on the submit host (local pool).
-
mLocalIrodsEnvBasename
protected java.lang.String mLocalIrodsEnvBasename
The basename of the user irodsEnv file
-
mProps
protected PegasusProperties mProps
The handle to the properties object holding the properties relevant to Pegasus.
-
mPOptions
protected PlannerOptions mPOptions
Contains the various options to the Planner as passed by the user at runtime.
-
mSiteStore
protected SiteStore mSiteStore
The handle to the Pool Info Provider. It is instantiated in this class
-
mTCHandle
protected TransformationCatalog mTCHandle
The handle to the Transformation Catalog. It must be instantiated in the implementing class
-
mRefiner
protected Refiner mRefiner
The handle to the refiner that loaded this implementation.
-
mLogger
protected LogManager mLogger
The logging object which is used to log all the messages.
-
mDisabledChmodSites
protected java.util.Set mDisabledChmodSites
The set of sites for which chmod job creation has to be disabled while doing executable staging.
-
mChmodDisabledForAllSites
protected boolean mChmodDisabledForAllSites
A boolean indicating whether chmod is disabled for all sites.
-
mAddNodesForSettingXBit
protected boolean mAddNodesForSettingXBit
A boolean indicating whether chmod jobs should be created that set the xbit in case of executable staging.
-
-
Constructor Detail
-
Abstract
public Abstract(PegasusBag bag)
The overloaded constructor, that is called by the Factory to load the class.- Parameters:
bag
- the bag of initialization objects.
-
-
Method Detail
-
applyPriority
public void applyPriority(TransferJob job)
Applies priorities to the transfer jobs if a priority is specified in the properties file.- Specified by:
applyPriority
in interfaceImplementation
- Parameters:
job
- the transfer job .
-
checkAndTransferProxy
public boolean checkAndTransferProxy(TransferJob job)
Deprecated.Determines if there is a need to transfer proxy for the transfer job or not. If there is a need to transfer proxy, then the job is modified to create the correct condor commands to transfer the proxy. Proxy is usually transferred if the Pegasus profile TRANSFER_PROXY is set, or the job is being run in the condor vanilla universe. The proxy is transferred from the submit host (i.e site local). The location is determined from the value of the X509_USER_PROXY profile key associated in the env namespace.- Specified by:
checkAndTransferProxy
in interfaceImplementation
- Parameters:
job
- the transfer job .- Returns:
- boolean true job was modified to transfer the proxy, else false when job is not modified.
-
checkAndTransferIrodsEnvFile
public boolean checkAndTransferIrodsEnvFile(TransferJob job)
Deprecated.Determines if there is a need to transfer the irodsEnvFile for the transfer job or not. If there is a need to transfert the file, then the job is modified to create the correct condor commands to transfer the file. The file is transferred from the submit host (i.e site local).- Parameters:
job
- the transfer job .- Returns:
- boolean true job was modified to transfer the irodsEnvFile, else false when job is not modified.
-
setRefiner
public void setRefiner(Refiner refiner)
Sets the callback to the refiner, that has loaded this implementation.- Specified by:
setRefiner
in interfaceImplementation
- Parameters:
refiner
- the transfer refiner that loaded the implementation.
-
addSetXBitJobs
protected boolean addSetXBitJobs(Job computeJob, Job txJob, java.util.Collection execFiles)
Adds the dirmanager to the workflow, that do a chmod on the files being staged.- Parameters:
computeJob
- the computeJob for which the files are being staged.txJob
- the transfer job that is staging the files.execFiles
- the executable files that are being staged.- Returns:
- boolean indicating whether any XBitJobs were succesfully added or not.
-
addSetXBitJobs
public boolean addSetXBitJobs(Job computeJob, java.lang.String txJobName, java.util.Collection execFiles, int transferClass)
Adds the dirmanager job to the workflow, that do a chmod on the files being staged.- Parameters:
computeJob
- the computeJob for which the files are being staged.txJobName
- the name of the transfer job that is staging the files.execFiles
- the executable files that are being staged.transferClass
- the class of transfer job- Returns:
- boolean indicating whether any XBitJobs were succesfully added or not.
-
createSetXBitJob
public Job createSetXBitJob(Job computeJob, java.util.Collection<FileTransfer> execFiles, int transferClass, int xbitIndex)
Adds the dirmanager job to the workflow, that do a chmod on the files being staged.- Specified by:
createSetXBitJob
in interfaceImplementation
- Parameters:
computeJob
- the computeJob for which the files are being staged.execFiles
- the executable files that are being staged.transferClass
- the class of transfer jobxbitIndex
- index to be used for creating the name of XBitJob.- Returns:
- the job object for the xBitJob
-
addSetXBitJobs
public boolean addSetXBitJobs(Job computeJob, java.lang.String txJobName, java.util.Collection execFiles, int transferClass, int xbitIndex)
Adds the dirmanager job to the workflow, that do a chmod on the files being staged.- Specified by:
addSetXBitJobs
in interfaceImplementation
- Parameters:
computeJob
- the computeJob for which the files are being staged.txJobName
- the name of the transfer job that is staging the files.execFiles
- the executable files that are being staged.transferClass
- the class of transfer jobxbitIndex
- index to be used for creating the name of XBitJob.- Returns:
- boolean indicating whether any XBitJobs were succesfully added or not.
-
getSetXBitJobName
public java.lang.String getSetXBitJobName(java.lang.String name, int counter)
Generates the name of the setXBitJob , that is unique for the given workflow.- Specified by:
getSetXBitJobName
in interfaceImplementation
- Parameters:
name
- the name of the compute jobcounter
- the index for the setXBit job.- Returns:
- the name of the setXBitJob .
-
getNOOPJobName
public java.lang.String getNOOPJobName(java.lang.String name, int counter)
Generates the name of the noop job , that is unique for the given workflow.- Parameters:
name
- the name of the compute jobcounter
- the index for the noop job.- Returns:
- the name of the setXBitJob .
-
createNoOPJob
public Job createNoOPJob(java.lang.String name)
It creates a NoOP job that runs on the submit host.- Parameters:
name
- the name to be assigned to the noop job- Returns:
- the noop job.
-
createSetXBitJob
protected Job createSetXBitJob(java.util.Collection<FileTransfer> files, java.lang.String name, java.lang.String site)
Creates a dirmanager job, that does a chmod on the file being staged. The file being staged should be of type executable. Though no explicit check is made for that. The staged file is the one whose X bit would be set on execution of this job. The site at which job is executed, is determined from the site associated with the destination URL.- Parameters:
files
- the collectionFileTransfer
containing the file that has to be X Bit Set.name
- the name that has to be assigned to the job.site
- the site at which the job has to be created- Returns:
- the chmod job, else null if it is not able to be created for some reason.
-
createSetXBitJob
protected Job createSetXBitJob(FileTransfer file, java.lang.String name)
Creates a dirmanager job, that does a chmod on the file being staged. The file being staged should be of type executable. Though no explicit check is made for that. The staged file is the one whose X bit would be set on execution of this job. The site at which job is executed, is determined from the site associated with the destination URL.- Parameters:
file
- theFileTransfer
containing the file that has to be X Bit Set.name
- the name that has to be assigned to the job.- Returns:
- the chmod job, else null if it is not able to be created for some reason.
-
defaultXBitTCEntry
private TransformationCatalogEntry defaultXBitTCEntry(java.lang.String site)
Returns a default TC entry to be used in case entry is not found in the transformation catalog.- Parameters:
site
- the site for which the default entry is required.- Returns:
- the default entry.
-
determineDisabledChmodSites
protected java.util.Set determineDisabledChmodSites(java.lang.String sites)
Builds up a set of disabled chmod sites- Parameters:
sites
- comma separated list of sites.- Returns:
- a Set containing the site names.
-
disableChmodJobCreation
protected boolean disableChmodJobCreation(java.lang.String site)
Returns a boolean indicating whether to disable chmod job creation for a site or not.- Parameters:
site
- the name of the site- Returns:
- boolean
-
getPriority
protected java.lang.String getPriority(TransferJob job)
Returns the priority for the transfer job as specified in the properties file.- Parameters:
job
- the Transfer job.- Returns:
- the priority of the job as determined from properties, can be null if invalid value passed or property not set.
-
construct
protected void construct(Job job, java.lang.String key, java.lang.String value)
Constructs a condor variable in the condor profile namespace associated with the job. Overrides any preexisting key values.- Parameters:
job
- contains the job description.key
- the key of the profile.value
- the associated value.
-
-