Class Abstract

  • All Implemented Interfaces:
    JobAggregator
    Direct Known Subclasses:
    MPIExec, SeqExec

    public abstract class Abstract
    extends java.lang.Object
    implements JobAggregator
    An abstract implementation of the JobAggregator interface, which the other implementations can choose to extend.
    Version:
    $Revision$
    Author:
    Karan Vahi vahi@isi.edu
    • Field Detail

      • CLUSTERED_JOB_PREFIX

        public static final java.lang.String CLUSTERED_JOB_PREFIX
        The prefix that is assigned to the jobname of the collapsed jobs to get the jobname for the fat job.
        See Also:
        Constant Field Values
      • TRANSFORMATION_NAMESPACE

        public static final java.lang.String TRANSFORMATION_NAMESPACE
        The transformation namespace for the cluster jobs.
        See Also:
        Constant Field Values
      • TRANSFORMATION_VERSION

        public static final java.lang.String TRANSFORMATION_VERSION
        The version number for the derivations for cluster jobs
      • DERIVATION_NAMESPACE

        public static final java.lang.String DERIVATION_NAMESPACE
        The derivation namespace for the cluster jobs.
        See Also:
        Constant Field Values
      • DERIVATION_VERSION

        public static final java.lang.String DERIVATION_VERSION
        The version number for the derivations for cluster jobs.
        See Also:
        Constant Field Values
      • MONITORD_COMMENT_MARKER

        public static final java.lang.String MONITORD_COMMENT_MARKER
        The marker to designate a line in the input file reserved for monitord purposes.
        See Also:
        Constant Field Values
      • mDirectory

        protected java.lang.String mDirectory
        The directory, where the stdin file of the fat jobs are created. It should be the submit file directory that the user mentions at runtime.
      • mProps

        protected PegasusProperties mProps
        The object holding all the properties pertaining to Pegasus.
      • mLogger

        protected LogManager mLogger
        The handle to the LogManager that logs all the messages.
      • mSiteStore

        protected SiteStore mSiteStore
        Handle to the site catalog store
      • mClusteredADag

        protected ADag mClusteredADag
        The handle to the ADag object that contains the workflow being clustered.
      • mGridStartFactory

        protected GridStartFactory mGridStartFactory
        The handle to the GridStart Factory.
      • mBag

        protected PegasusBag mBag
        Bag of initialization objects.
    • Constructor Detail

      • Abstract

        public Abstract()
        The default constructor.
    • Method Detail

      • getCompleteTranformationName

        public static java.lang.String getCompleteTranformationName​(java.lang.String name)
        A convenience method to return the complete transformation name being used to construct jobs in this class.
        Parameters:
        name - the name of the transformation
        Returns:
        the complete transformation name
      • initialize

        public void initialize​(ADag dag,
                               PegasusBag bag)
        Initializes the JobAggregator impelementation
        Specified by:
        initialize in interface JobAggregator
        Parameters:
        dag - the workflow that is being clustered.
        bag - the bag of objects that is useful for initialization.
      • aggregatedJobArguments

        public abstract java.lang.String aggregatedJobArguments​(AggregatedJob job)
        Returns the arguments with which the AggregatedJob needs to be invoked with.
        Parameters:
        job - the AggregatedJob for which the arguments have to be constructed.
        Returns:
        argument string
      • constructAbstractAggregatedJob

        public AggregatedJob constructAbstractAggregatedJob​(java.util.List jobs,
                                                            java.lang.String name,
                                                            java.lang.String id)
        Constructs a new aggregated job that contains all the jobs passed to it. The new aggregated job, appears as a single job in the workflow and replaces the jobs it contains in the workflow.
        Specified by:
        constructAbstractAggregatedJob in interface JobAggregator
        Parameters:
        jobs - the list of Job objects that need to be collapsed. All the jobs being collapsed should be scheduled at the same pool, to maintain correct semantics.
        name - the logical name of the jobs in the list passed to this function.
        id - the id that is given to the new job.
        Returns:
        the Job object corresponding to the aggregated job containing the jobs passed as List in the input, null if the list of jobs is empty
      • constructAbstractAggregatedJob

        public AggregatedJob constructAbstractAggregatedJob​(java.util.List jobs,
                                                            java.lang.String name,
                                                            java.lang.String id,
                                                            java.lang.String mergeLFN)
        Constructs an abstract aggregated job that has a handle to the appropriate JobAggregator that will be used to aggregate the jobs.
        Parameters:
        jobs - the list of SubInfo objects that need to be collapsed. All the jobs being collapsed should be scheduled at the same pool, to maintain correct semantics.
        name - the logical name of the jobs in the list passed to this function.
        id - the id that is given to the new job.
        mergeLFN - the logical name for the aggregated job that has to be constructed.
        Returns:
        the SubInfo object corresponding to the aggregated job containing the jobs passed as List in the input, null if the list of jobs is empty
      • makeAbstractAggregatedJobConcrete

        public void makeAbstractAggregatedJobConcrete​(AggregatedJob job)
        Enables the abstract clustered job for execution and converts it to it's executable form
        Specified by:
        makeAbstractAggregatedJobConcrete in interface JobAggregator
        Parameters:
        job - the abstract clustered job
      • getCommentString

        protected java.lang.String getCommentString​(Job job,
                                                    int taskid)
        Generates the comment string for the job . It generates a comment of the format # task_id transformation derivation.
        Parameters:
        job - the job for which
        taskid - the task id to put in.
        Returns:
        the comment invocation
      • getTCEntry

        protected TransformationCatalogEntry getTCEntry​(Job job)
        Helper method to get an entry from the transformation catalog for an installed executable. It does the traversal from the list of entries to return a single TransformationCatalogEntry object, and dies with an appropriate error message if the object is not found. The pool and the name are retrieved from job object.
        Parameters:
        job - the job whose corresponding TransformationCatalogEntry you want.
        Returns:
        the TransformationCatalogEntry corresponding to the entry in the TC.
      • defaultTCEntry

        private TransformationCatalogEntry defaultTCEntry​(java.lang.String name,
                                                          java.lang.String executableBasename,
                                                          java.lang.String site)
        Returns a default TC entry to be used in case entry is not found in the transformation catalog.
        Parameters:
        name - the logical name for the clustering transformation.
        executableBasename - the basename for the executable in the bin directory of a Pegasus installation
        site - the site for which the default entry is required.
        Returns:
        the default entry.
      • entryNotInTC

        protected boolean entryNotInTC​(java.lang.String namespace,
                                       java.lang.String name,
                                       java.lang.String version,
                                       java.lang.String executableBasename,
                                       java.lang.String site)
        Determines whether there is NOT an entry in the transformation catalog for a particular transformation on a particular site.
        Parameters:
        namespace - the logical namespace of the transformation.
        name - the logical name of the transformation.
        version - the version of the transformation.
        executableBasename - basename of the executable that does the clustering.
        site - the site at which existence check is required.
        Returns:
        boolean true if an entry does not exists, false otherwise.
      • setDirectory

        protected void setDirectory​(java.lang.String directory)
        Sets the directory where the stdin files are to be generated.
        Parameters:
        directory - the path to the directory to which it needs to be set.
      • writeOutInputFileForJobAggregator

        protected java.io.File writeOutInputFileForJobAggregator​(AggregatedJob job)
        Writes out the input file for the aggregated job
        Parameters:
        job - the aggregated job
        Returns:
        path to the input file