Class Abstract

    • Field Summary

      Fields 
      Modifier and Type Field Description
      static java.lang.String CHANGE_XBIT_TRANSFORMATION
      The logical name of the transformation that creates directories on the remote execution pools.
      protected boolean mAddNodesForSettingXBit
      A boolean indicating whether chmod jobs should be created that set the xbit in case of executable staging.
      protected boolean mChmodDisabledForAllSites
      A boolean indicating whether chmod is disabled for all sites.
      protected java.util.Set mDisabledChmodSites
      The set of sites for which chmod job creation has to be disabled while doing executable staging.
      protected java.lang.String mLocalIrodsEnv
      The path to the irodsEnv file on the submit host (local pool).
      protected java.lang.String mLocalIrodsEnvBasename
      The basename of the user irodsEnv file
      protected java.lang.String mLocalS3cfg
      The path to the s3cfg file on the submit host (local pool).
      protected java.lang.String mLocalS3cfgBasename
      The basename of the user s3cfg file
      protected java.lang.String mLocalUserProxy
      The path to the user proxy on the submit host (local pool), that is picked up for use in transfer of proxies.
      protected java.lang.String mLocalUserProxyBasename
      The basename of the user proxy , that is picked up for use in transfer of proxies.
      protected LogManager mLogger
      The logging object which is used to log all the messages.
      protected PlannerOptions mPOptions
      Contains the various options to the Planner as passed by the user at runtime.
      protected PegasusProperties mProps
      The handle to the properties object holding the properties relevant to Pegasus.
      protected Refiner mRefiner
      The handle to the refiner that loaded this implementation.
      protected SiteStore mSiteStore
      The handle to the Pool Info Provider.
      protected TransformationCatalog mTCHandle
      The handle to the Transformation Catalog.
      static java.lang.String NOOP_PREFIX
      The prefix for the NoOP jobs that are created.
      static java.lang.String SET_XBIT_PREFIX
      The prefix for the jobs which are added to set X bit for the staged executables.
      static java.lang.String XBIT_DERIVATION_NS
      The derivation namespace for the setXBit jobs.
      static java.lang.String XBIT_DERIVATION_VERSION
      The version number for the derivations for setXBit jobs.
      static java.lang.String XBIT_EXECUTABLE_BASENAME
      The basename of the pegasus dirmanager executable.
      static java.lang.String XBIT_TRANSFORMATION_NS
      The transformation namespace for the setXBit jobs.
      static java.lang.String XBIT_TRANSFORMATION_VERSION
      The version number for the derivations for setXBit jobs.
    • Constructor Summary

      Constructors 
      Constructor Description
      Abstract​(PegasusBag bag)
      The overloaded constructor, that is called by the Factory to load the class.
    • Method Summary

      All Methods Instance Methods Concrete Methods Deprecated Methods 
      Modifier and Type Method Description
      protected boolean addSetXBitJobs​(Job computeJob, Job txJob, java.util.Collection execFiles)
      Adds the dirmanager to the workflow, that do a chmod on the files being staged.
      boolean addSetXBitJobs​(Job computeJob, java.lang.String txJobName, java.util.Collection execFiles, int transferClass)
      Adds the dirmanager job to the workflow, that do a chmod on the files being staged.
      boolean addSetXBitJobs​(Job computeJob, java.lang.String txJobName, java.util.Collection execFiles, int transferClass, int xbitIndex)
      Adds the dirmanager job to the workflow, that do a chmod on the files being staged.
      void applyPriority​(TransferJob job)
      Applies priorities to the transfer jobs if a priority is specified in the properties file.
      boolean checkAndTransferIrodsEnvFile​(TransferJob job)
      Deprecated. 
      boolean checkAndTransferProxy​(TransferJob job)
      Deprecated. 
      protected void construct​(Job job, java.lang.String key, java.lang.String value)
      Constructs a condor variable in the condor profile namespace associated with the job.
      Job createNoOPJob​(java.lang.String name)
      It creates a NoOP job that runs on the submit host.
      protected Job createSetXBitJob​(FileTransfer file, java.lang.String name)
      Creates a dirmanager job, that does a chmod on the file being staged.
      Job createSetXBitJob​(Job computeJob, java.util.Collection<FileTransfer> execFiles, int transferClass, int xbitIndex)
      Adds the dirmanager job to the workflow, that do a chmod on the files being staged.
      protected Job createSetXBitJob​(java.util.Collection<FileTransfer> files, java.lang.String name, java.lang.String site)
      Creates a dirmanager job, that does a chmod on the file being staged.
      private TransformationCatalogEntry defaultXBitTCEntry​(java.lang.String site)
      Returns a default TC entry to be used in case entry is not found in the transformation catalog.
      protected java.util.Set determineDisabledChmodSites​(java.lang.String sites)
      Builds up a set of disabled chmod sites
      protected boolean disableChmodJobCreation​(java.lang.String site)
      Returns a boolean indicating whether to disable chmod job creation for a site or not.
      java.lang.String getNOOPJobName​(java.lang.String name, int counter)
      Generates the name of the noop job , that is unique for the given workflow.
      protected java.lang.String getPriority​(TransferJob job)
      Returns the priority for the transfer job as specified in the properties file.
      java.lang.String getSetXBitJobName​(java.lang.String name, int counter)
      Generates the name of the setXBitJob , that is unique for the given workflow.
      void setRefiner​(Refiner refiner)
      Sets the callback to the refiner, that has loaded this implementation.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Field Detail

      • CHANGE_XBIT_TRANSFORMATION

        public static final java.lang.String CHANGE_XBIT_TRANSFORMATION
        The logical name of the transformation that creates directories on the remote execution pools.
        See Also:
        Constant Field Values
      • XBIT_EXECUTABLE_BASENAME

        public static final java.lang.String XBIT_EXECUTABLE_BASENAME
        The basename of the pegasus dirmanager executable.
        See Also:
        Constant Field Values
      • XBIT_TRANSFORMATION_NS

        public static final java.lang.String XBIT_TRANSFORMATION_NS
        The transformation namespace for the setXBit jobs.
        See Also:
        Constant Field Values
      • XBIT_TRANSFORMATION_VERSION

        public static final java.lang.String XBIT_TRANSFORMATION_VERSION
        The version number for the derivations for setXBit jobs.
      • XBIT_DERIVATION_NS

        public static final java.lang.String XBIT_DERIVATION_NS
        The derivation namespace for the setXBit jobs.
        See Also:
        Constant Field Values
      • XBIT_DERIVATION_VERSION

        public static final java.lang.String XBIT_DERIVATION_VERSION
        The version number for the derivations for setXBit jobs.
      • SET_XBIT_PREFIX

        public static final java.lang.String SET_XBIT_PREFIX
        The prefix for the jobs which are added to set X bit for the staged executables.
        See Also:
        Constant Field Values
      • NOOP_PREFIX

        public static final java.lang.String NOOP_PREFIX
        The prefix for the NoOP jobs that are created.
        See Also:
        Constant Field Values
      • mLocalUserProxy

        protected java.lang.String mLocalUserProxy
        The path to the user proxy on the submit host (local pool), that is picked up for use in transfer of proxies.
      • mLocalUserProxyBasename

        protected java.lang.String mLocalUserProxyBasename
        The basename of the user proxy , that is picked up for use in transfer of proxies.
      • mLocalS3cfg

        protected java.lang.String mLocalS3cfg
        The path to the s3cfg file on the submit host (local pool).
      • mLocalS3cfgBasename

        protected java.lang.String mLocalS3cfgBasename
        The basename of the user s3cfg file
      • mLocalIrodsEnv

        protected java.lang.String mLocalIrodsEnv
        The path to the irodsEnv file on the submit host (local pool).
      • mLocalIrodsEnvBasename

        protected java.lang.String mLocalIrodsEnvBasename
        The basename of the user irodsEnv file
      • mProps

        protected PegasusProperties mProps
        The handle to the properties object holding the properties relevant to Pegasus.
      • mPOptions

        protected PlannerOptions mPOptions
        Contains the various options to the Planner as passed by the user at runtime.
      • mSiteStore

        protected SiteStore mSiteStore
        The handle to the Pool Info Provider. It is instantiated in this class
      • mTCHandle

        protected TransformationCatalog mTCHandle
        The handle to the Transformation Catalog. It must be instantiated in the implementing class
      • mRefiner

        protected Refiner mRefiner
        The handle to the refiner that loaded this implementation.
      • mLogger

        protected LogManager mLogger
        The logging object which is used to log all the messages.
      • mDisabledChmodSites

        protected java.util.Set mDisabledChmodSites
        The set of sites for which chmod job creation has to be disabled while doing executable staging.
      • mChmodDisabledForAllSites

        protected boolean mChmodDisabledForAllSites
        A boolean indicating whether chmod is disabled for all sites.
      • mAddNodesForSettingXBit

        protected boolean mAddNodesForSettingXBit
        A boolean indicating whether chmod jobs should be created that set the xbit in case of executable staging.
    • Constructor Detail

      • Abstract

        public Abstract​(PegasusBag bag)
        The overloaded constructor, that is called by the Factory to load the class.
        Parameters:
        bag - the bag of initialization objects.
    • Method Detail

      • applyPriority

        public void applyPriority​(TransferJob job)
        Applies priorities to the transfer jobs if a priority is specified in the properties file.
        Specified by:
        applyPriority in interface Implementation
        Parameters:
        job - the transfer job .
      • checkAndTransferProxy

        public boolean checkAndTransferProxy​(TransferJob job)
        Deprecated.
        Determines if there is a need to transfer proxy for the transfer job or not. If there is a need to transfer proxy, then the job is modified to create the correct condor commands to transfer the proxy. Proxy is usually transferred if the Pegasus profile TRANSFER_PROXY is set, or the job is being run in the condor vanilla universe. The proxy is transferred from the submit host (i.e site local). The location is determined from the value of the X509_USER_PROXY profile key associated in the env namespace.
        Specified by:
        checkAndTransferProxy in interface Implementation
        Parameters:
        job - the transfer job .
        Returns:
        boolean true job was modified to transfer the proxy, else false when job is not modified.
      • checkAndTransferIrodsEnvFile

        public boolean checkAndTransferIrodsEnvFile​(TransferJob job)
        Deprecated.
        Determines if there is a need to transfer the irodsEnvFile for the transfer job or not. If there is a need to transfert the file, then the job is modified to create the correct condor commands to transfer the file. The file is transferred from the submit host (i.e site local).
        Parameters:
        job - the transfer job .
        Returns:
        boolean true job was modified to transfer the irodsEnvFile, else false when job is not modified.
      • setRefiner

        public void setRefiner​(Refiner refiner)
        Sets the callback to the refiner, that has loaded this implementation.
        Specified by:
        setRefiner in interface Implementation
        Parameters:
        refiner - the transfer refiner that loaded the implementation.
      • addSetXBitJobs

        protected boolean addSetXBitJobs​(Job computeJob,
                                         Job txJob,
                                         java.util.Collection execFiles)
        Adds the dirmanager to the workflow, that do a chmod on the files being staged.
        Parameters:
        computeJob - the computeJob for which the files are being staged.
        txJob - the transfer job that is staging the files.
        execFiles - the executable files that are being staged.
        Returns:
        boolean indicating whether any XBitJobs were succesfully added or not.
      • addSetXBitJobs

        public boolean addSetXBitJobs​(Job computeJob,
                                      java.lang.String txJobName,
                                      java.util.Collection execFiles,
                                      int transferClass)
        Adds the dirmanager job to the workflow, that do a chmod on the files being staged.
        Parameters:
        computeJob - the computeJob for which the files are being staged.
        txJobName - the name of the transfer job that is staging the files.
        execFiles - the executable files that are being staged.
        transferClass - the class of transfer job
        Returns:
        boolean indicating whether any XBitJobs were succesfully added or not.
      • createSetXBitJob

        public Job createSetXBitJob​(Job computeJob,
                                    java.util.Collection<FileTransfer> execFiles,
                                    int transferClass,
                                    int xbitIndex)
        Adds the dirmanager job to the workflow, that do a chmod on the files being staged.
        Specified by:
        createSetXBitJob in interface Implementation
        Parameters:
        computeJob - the computeJob for which the files are being staged.
        execFiles - the executable files that are being staged.
        transferClass - the class of transfer job
        xbitIndex - index to be used for creating the name of XBitJob.
        Returns:
        the job object for the xBitJob
      • addSetXBitJobs

        public boolean addSetXBitJobs​(Job computeJob,
                                      java.lang.String txJobName,
                                      java.util.Collection execFiles,
                                      int transferClass,
                                      int xbitIndex)
        Adds the dirmanager job to the workflow, that do a chmod on the files being staged.
        Specified by:
        addSetXBitJobs in interface Implementation
        Parameters:
        computeJob - the computeJob for which the files are being staged.
        txJobName - the name of the transfer job that is staging the files.
        execFiles - the executable files that are being staged.
        transferClass - the class of transfer job
        xbitIndex - index to be used for creating the name of XBitJob.
        Returns:
        boolean indicating whether any XBitJobs were succesfully added or not.
      • getSetXBitJobName

        public java.lang.String getSetXBitJobName​(java.lang.String name,
                                                  int counter)
        Generates the name of the setXBitJob , that is unique for the given workflow.
        Specified by:
        getSetXBitJobName in interface Implementation
        Parameters:
        name - the name of the compute job
        counter - the index for the setXBit job.
        Returns:
        the name of the setXBitJob .
      • getNOOPJobName

        public java.lang.String getNOOPJobName​(java.lang.String name,
                                               int counter)
        Generates the name of the noop job , that is unique for the given workflow.
        Parameters:
        name - the name of the compute job
        counter - the index for the noop job.
        Returns:
        the name of the setXBitJob .
      • createNoOPJob

        public Job createNoOPJob​(java.lang.String name)
        It creates a NoOP job that runs on the submit host.
        Parameters:
        name - the name to be assigned to the noop job
        Returns:
        the noop job.
      • createSetXBitJob

        protected Job createSetXBitJob​(java.util.Collection<FileTransfer> files,
                                       java.lang.String name,
                                       java.lang.String site)
        Creates a dirmanager job, that does a chmod on the file being staged. The file being staged should be of type executable. Though no explicit check is made for that. The staged file is the one whose X bit would be set on execution of this job. The site at which job is executed, is determined from the site associated with the destination URL.
        Parameters:
        files - the collection FileTransfer containing the file that has to be X Bit Set.
        name - the name that has to be assigned to the job.
        site - the site at which the job has to be created
        Returns:
        the chmod job, else null if it is not able to be created for some reason.
      • createSetXBitJob

        protected Job createSetXBitJob​(FileTransfer file,
                                       java.lang.String name)
        Creates a dirmanager job, that does a chmod on the file being staged. The file being staged should be of type executable. Though no explicit check is made for that. The staged file is the one whose X bit would be set on execution of this job. The site at which job is executed, is determined from the site associated with the destination URL.
        Parameters:
        file - the FileTransfer containing the file that has to be X Bit Set.
        name - the name that has to be assigned to the job.
        Returns:
        the chmod job, else null if it is not able to be created for some reason.
      • defaultXBitTCEntry

        private TransformationCatalogEntry defaultXBitTCEntry​(java.lang.String site)
        Returns a default TC entry to be used in case entry is not found in the transformation catalog.
        Parameters:
        site - the site for which the default entry is required.
        Returns:
        the default entry.
      • determineDisabledChmodSites

        protected java.util.Set determineDisabledChmodSites​(java.lang.String sites)
        Builds up a set of disabled chmod sites
        Parameters:
        sites - comma separated list of sites.
        Returns:
        a Set containing the site names.
      • disableChmodJobCreation

        protected boolean disableChmodJobCreation​(java.lang.String site)
        Returns a boolean indicating whether to disable chmod job creation for a site or not.
        Parameters:
        site - the name of the site
        Returns:
        boolean
      • getPriority

        protected java.lang.String getPriority​(TransferJob job)
        Returns the priority for the transfer job as specified in the properties file.
        Parameters:
        job - the Transfer job.
        Returns:
        the priority of the job as determined from properties, can be null if invalid value passed or property not set.
      • construct

        protected void construct​(Job job,
                                 java.lang.String key,
                                 java.lang.String value)
        Constructs a condor variable in the condor profile namespace associated with the job. Overrides any preexisting key values.
        Parameters:
        job - contains the job description.
        key - the key of the profile.
        value - the associated value.