Class DAX2CDAG

  • All Implemented Interfaces:
    Callback

    public class DAX2CDAG
    extends java.lang.Object
    implements Callback
    This creates a dag corresponding to one particular partition of the whole abstract plan. The partition can be as big as the whole abstract graph or can be as small as a single job. The partitions are determined by the Partitioner.
    Version:
    $Revision$
    Author:
    Karan Vahi
    • Field Summary

      Fields 
      Modifier and Type Field Description
      private boolean mAddDataDependencies
      To track whether we auto detect data dependancies or not.
      protected java.util.Map<java.lang.String,​CompoundTransformation> mCompoundTransformations
      Map of Compound Transformations indexed by complete name of the compound transformation.
      private ADag mDag
      The ADag object which contains information corresponding to the ADag in the XML file.
      private boolean mDone
      A flag to specify whether the graph has been generated for the partition or not.
      private java.util.Map<java.lang.String,​Job> mFileCreationMap
      Map to track a LFN with the job that creates the file corresponding to the LFN.
      private java.util.Map<java.lang.String,​java.lang.String> mJobMap
      The mapping of the idrefs of a job to the job name.
      private LogManager mLogger
      The handle to the logger
      private Notifications mNotifications
      All the notifications associated with the adag
      private PegasusProperties mProps
      The handle to the properties object.
      protected ReplicaStore mReplicaStore
      Handle to the replica store that stores the replica catalog user specifies in the DAX
      protected TransformationStore mTransformationStore
      Handle to the transformation store that stores the transformation catalog user specifies in the DAX
      • Fields inherited from interface edu.isi.pegasus.planner.parser.dax.Callback

        VERSION
    • Constructor Summary

      Constructors 
      Constructor Description
      DAX2CDAG()  
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      private void addDataDependencies()
      Goes through the ADag and computes any data dependencies.
      void cbCompoundTransformation​(CompoundTransformation compoundTransformation)
      Callback when a compound transformation is encountered in the DAX
      void cbDocument​(java.util.Map attributes)
      Callback when the opening tag was parsed.
      void cbDone()
      Callback when the parsing of the document is done.
      void cbExecutable​(TransformationCatalogEntry tce)
      Callback when a transformation catalog entry is encountered in the DAX
      void cbFile​(ReplicaLocation rl)
      Callback when a replica catalog entry is encountered in the DAX
      void cbJob​(Job job)
      Callback for the job from section 2 jobs.
      void cbParents​(java.lang.String child, java.util.List<PCRelation> parents)
      Callback for child and parentID relationships from section 3.
      void cbWfInvoke​(Invoke invoke)
      Callback when a invoke entry is encountered in the top level inside the adag element in DAX.
      java.lang.Object getConstructedObject()
      Returns an ADag object corresponding to the abstract plan it has generated.
      void initialize​(PegasusBag bag, java.lang.String dax)
      The overloaded constructor.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Field Detail

      • mDag

        private ADag mDag
        The ADag object which contains information corresponding to the ADag in the XML file.
      • mJobMap

        private java.util.Map<java.lang.String,​java.lang.String> mJobMap
        The mapping of the idrefs of a job to the job name. e.g ID0000001 -> preprocess_ID000001
      • mDone

        private boolean mDone
        A flag to specify whether the graph has been generated for the partition or not.
      • mReplicaStore

        protected ReplicaStore mReplicaStore
        Handle to the replica store that stores the replica catalog user specifies in the DAX
      • mTransformationStore

        protected TransformationStore mTransformationStore
        Handle to the transformation store that stores the transformation catalog user specifies in the DAX
      • mCompoundTransformations

        protected java.util.Map<java.lang.String,​CompoundTransformation> mCompoundTransformations
        Map of Compound Transformations indexed by complete name of the compound transformation.
      • mNotifications

        private Notifications mNotifications
        All the notifications associated with the adag
      • mAddDataDependencies

        private boolean mAddDataDependencies
        To track whether we auto detect data dependancies or not.
      • mFileCreationMap

        private java.util.Map<java.lang.String,​Job> mFileCreationMap
        Map to track a LFN with the job that creates the file corresponding to the LFN.
      • mLogger

        private LogManager mLogger
        The handle to the logger
    • Constructor Detail

      • DAX2CDAG

        public DAX2CDAG()
    • Method Detail

      • initialize

        public void initialize​(PegasusBag bag,
                               java.lang.String dax)
        The overloaded constructor.
        Specified by:
        initialize in interface Callback
        Parameters:
        bag - the bag of initialization objects containing the properties and the logger
        dax - the path to the DAX file.
      • cbDocument

        public void cbDocument​(java.util.Map attributes)
        Callback when the opening tag was parsed. This contains all attributes and their raw values within a map. It ends up storing the attributes with the adag element in the internal memory structure.
        Specified by:
        cbDocument in interface Callback
        Parameters:
        attributes - is a map of attribute key to attribute value
      • cbWfInvoke

        public void cbWfInvoke​(Invoke invoke)
        Callback when a invoke entry is encountered in the top level inside the adag element in DAX.
        Specified by:
        cbWfInvoke in interface Callback
        Parameters:
        invoke - the invoke object
      • cbJob

        public void cbJob​(Job job)
        Callback for the job from section 2 jobs. These jobs are completely assembled, but each is passed separately.
        Specified by:
        cbJob in interface Callback
        Parameters:
        job - the Job object storing the job information gotten from parser.
      • cbParents

        public void cbParents​(java.lang.String child,
                              java.util.List<PCRelation> parents)
        Callback for child and parentID relationships from section 3.
        Specified by:
        cbParents in interface Callback
        Parameters:
        child - is the IDREF of the child element.
        parents - is a list of IDREFs of the included parents.
      • cbDone

        public void cbDone()
        Callback when the parsing of the document is done. It sets the flag that the parsing has been done, that is used to determine whether the ADag object has been fully generated or not.
        Specified by:
        cbDone in interface Callback
      • getConstructedObject

        public java.lang.Object getConstructedObject()
        Returns an ADag object corresponding to the abstract plan it has generated. It throws a runtime exception if the method is called before the object has been created fully.
        Specified by:
        getConstructedObject in interface Callback
        Returns:
        ADag object containing the abstract plan referred in the dax.
      • cbCompoundTransformation

        public void cbCompoundTransformation​(CompoundTransformation compoundTransformation)
        Callback when a compound transformation is encountered in the DAX
        Specified by:
        cbCompoundTransformation in interface Callback
        Parameters:
        compoundTransformation - the compound transformation
      • cbFile

        public void cbFile​(ReplicaLocation rl)
        Callback when a replica catalog entry is encountered in the DAX
        Specified by:
        cbFile in interface Callback
        Parameters:
        rl - the ReplicaLocation object
      • cbExecutable

        public void cbExecutable​(TransformationCatalogEntry tce)
        Callback when a transformation catalog entry is encountered in the DAX
        Specified by:
        cbExecutable in interface Callback
        Parameters:
        tce - the transformation catalog entry object.
      • addDataDependencies

        private void addDataDependencies()
        Goes through the ADag and computes any data dependencies. For example if Job A creates an output file X and job B consumes it, then it automatically adds a dependency between A -> B if it does not exist already.