Class AbstractFileFactoryBasedMapper

  • All Implemented Interfaces:
    OutputMapper
    Direct Known Subclasses:
    Flat, Hashed

    public abstract class AbstractFileFactoryBasedMapper
    extends java.lang.Object
    implements OutputMapper
    The abstract class that serves as the base class for the File Factory based mappers.
    Author:
    Karan Vahi
    • Field Summary

      Fields 
      Modifier and Type Field Description
      protected org.griphyn.vdl.euryale.FileFactory mFactory  
      protected LogManager mLogger
      The handle to the logger.
      protected java.lang.String mOutputSite
      The output site where the data needs to be placed.
      protected SiteStore mSiteStore
      Handle to the Site Catalog contents.
      protected Directory mStageoutDirectory
      The stage out directory where the outputs are staged to.
    • Method Summary

      All Methods Instance Methods Abstract Methods Concrete Methods 
      Modifier and Type Method Description
      protected void complainForStorageFileServer​(FileServerType.OPERATION operation, java.lang.String site)
      Complains for a missing head node storage file server on a site for a job
      protected java.lang.String constructURL​(FileServer server, java.lang.String addOn)
      Returns the full path on remote output site, where the lfn will reside, using the FileServer passed.
      abstract java.lang.String createAndGetAddOn​(java.lang.String lfn, java.lang.String site, boolean existing)
      Returns the addOn part that is retrieved from the File Factory.
      protected java.lang.String getErrorMessagePrefix()
      Returns the prefix message to be attached to an error message
      abstract java.lang.String getShortName()
      Returns the short name for the implementation class.
      void initialize​(PegasusBag bag, ADag workflow)
      Initializes the mappers.
      abstract org.griphyn.vdl.euryale.FileFactory instantiateFileFactory​(PegasusBag bag, ADag workflow)
      Method that instantiates the FileFactory
      protected Directory lookupStorageDirectory​(java.lang.String site)
      Looks up the site catalog to return the storage directory for a site
      java.lang.String map​(java.lang.String lfn, java.lang.String site, FileServerType.OPERATION operation)
      Maps a LFN to a location on the filsystem of a site and returns a single externally accessible URL corresponding to that location.
      java.lang.String map​(java.lang.String lfn, java.lang.String site, FileServerType.OPERATION operation, boolean existing)
      Maps a LFN to a location on the filsystem of a site and returns a single externally accessible URL corresponding to that location.
      java.util.List<java.lang.String> mapAll​(java.lang.String lfn, java.lang.String site, FileServerType.OPERATION operation)
      Maps a LFN to a location on the filsystem of a site and returns all the possible equivalent externally accessible URL corresponding to that location.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Field Detail

      • mLogger

        protected LogManager mLogger
        The handle to the logger.
      • mFactory

        protected org.griphyn.vdl.euryale.FileFactory mFactory
      • mSiteStore

        protected SiteStore mSiteStore
        Handle to the Site Catalog contents.
      • mOutputSite

        protected java.lang.String mOutputSite
        The output site where the data needs to be placed.
      • mStageoutDirectory

        protected Directory mStageoutDirectory
        The stage out directory where the outputs are staged to.
    • Constructor Detail

      • AbstractFileFactoryBasedMapper

        public AbstractFileFactoryBasedMapper()
        The default constructor.
    • Method Detail

      • initialize

        public void initialize​(PegasusBag bag,
                               ADag workflow)
                        throws MapperException
        Initializes the mappers.
        Specified by:
        initialize in interface OutputMapper
        Parameters:
        bag - the bag of objects that is useful for initialization.
        workflow - the workflow refined so far.
        Throws:
        MapperException - if unable to construct URL for any reason
      • instantiateFileFactory

        public abstract org.griphyn.vdl.euryale.FileFactory instantiateFileFactory​(PegasusBag bag,
                                                                                   ADag workflow)
        Method that instantiates the FileFactory
        Parameters:
        bag - the bag of objects that is useful for initialization.
        workflow - the workflow refined so far.
        Returns:
        the handle to the File Factory to use
      • getShortName

        public abstract java.lang.String getShortName()
        Returns the short name for the implementation class.
        Returns:
      • createAndGetAddOn

        public abstract java.lang.String createAndGetAddOn​(java.lang.String lfn,
                                                           java.lang.String site,
                                                           boolean existing)
        Returns the addOn part that is retrieved from the File Factory. It creates a new file in the factory for the LFN and returns it.
        Parameters:
        lfn - the LFN to be used
        site - the site at which the LFN resides
        existing - indicates whether to create a new location/placement for a file, or rely on existing placement on the site.
        Returns:
      • constructURL

        protected java.lang.String constructURL​(FileServer server,
                                                java.lang.String addOn)
                                         throws MapperException
        Returns the full path on remote output site, where the lfn will reside, using the FileServer passed. This method creates a new File in the FileFactory space.
        Parameters:
        server - the file server to use
        addOn - the addOn part containing the LFN
        Returns:
        the URL for the LFN
        Throws:
        MapperException
      • map

        public java.lang.String map​(java.lang.String lfn,
                                    java.lang.String site,
                                    FileServerType.OPERATION operation)
                             throws MapperException
        Maps a LFN to a location on the filsystem of a site and returns a single externally accessible URL corresponding to that location. If the storage directory for the site has multiple file servers associated with it, it selects one randomly and returns a URL accessible from that FileServer
        Specified by:
        map in interface OutputMapper
        Parameters:
        lfn - the lfn
        site - the output site
        operation - whether we want a GET or a PUT URL
        Returns:
        the URL to file that was mapped
        Throws:
        MapperException - if unable to construct URL for any reason
      • map

        public java.lang.String map​(java.lang.String lfn,
                                    java.lang.String site,
                                    FileServerType.OPERATION operation,
                                    boolean existing)
                             throws MapperException
        Maps a LFN to a location on the filsystem of a site and returns a single externally accessible URL corresponding to that location.
        Specified by:
        map in interface OutputMapper
        Parameters:
        lfn - the lfn
        site - the output site
        operation - whether we want a GET or a PUT URL
        existing - indicates whether to create a new location/placement for a file, or rely on existing placement on the site.
        Returns:
        externally accessible URL to the mapped file.
        Throws:
        MapperException - if unable to construct URL for any reason
      • mapAll

        public java.util.List<java.lang.String> mapAll​(java.lang.String lfn,
                                                       java.lang.String site,
                                                       FileServerType.OPERATION operation)
                                                throws MapperException
        Maps a LFN to a location on the filsystem of a site and returns all the possible equivalent externally accessible URL corresponding to that location. For example, if a file on the filesystem is accessible via multiple file servers it should return externally accessible URL's from all the File Servers on the site.
        Specified by:
        mapAll in interface OutputMapper
        Parameters:
        lfn - the lfn
        site - the output site
        operation - whether we want a GET or a PUT URL
        Returns:
        List of externally accessible URLs to the mapped file.
        Throws:
        MapperException - if unable to construct URL for any reason
      • lookupStorageDirectory

        protected Directory lookupStorageDirectory​(java.lang.String site)
                                            throws MapperException
        Looks up the site catalog to return the storage directory for a site
        Parameters:
        site - the site
        Returns:
        Throws:
        MapperException
      • complainForStorageFileServer

        protected void complainForStorageFileServer​(FileServerType.OPERATION operation,
                                                    java.lang.String site)
        Complains for a missing head node storage file server on a site for a job
        Parameters:
        operation - the file server operation
        site - the site
      • getErrorMessagePrefix

        protected java.lang.String getErrorMessagePrefix()
        Returns the prefix message to be attached to an error message
        Returns: