org.apache.flink.runtime.execution.librarycache

Class BlobLibraryCacheManager

java.lang.Object

java.util.TimerTask

org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager

All Implemented Interfaces:

Runnable, LibraryCacheManager

public final class BlobLibraryCacheManager
extends TimerTask
implements LibraryCacheManager

For each job graph that is submitted to the system the library cache manager maintains a set of libraries (typically JAR files) which the job requires to run. The library cache manager caches library files in order to avoid unnecessary retransmission of data. It is based on a singleton programming pattern, so there exists at most on library manager at a time.

This class is thread-safe.

Constructor Summary

Constructors

Constructor and Description
BlobLibraryCacheManager(BlobService blobService, Configuration configuration)

Method Summary

Modifier and Type	Method and Description
int	getBlobServerPort()
ClassLoader	getClassLoader(JobID id) Returns the class loader to the specified vertex.
File	getFile(BlobKey blobKey) Returns a file handle to the file identified by the blob key.
void	register(JobID id, Collection<BlobKey> requiredJarFiles) Registers a job ID with a set of library paths that are required to run the job.
void	run() Cleans up blobs which are not referenced anymore
void	shutdown() Shutdown method
void	unregister(JobID id) Unregisters a job ID and releases the resources associated with it.

Methods inherited from class java.util.TimerTask

cancel, scheduledExecutionTime

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BlobLibraryCacheManager

public BlobLibraryCacheManager(BlobService blobService,
                               Configuration configuration)

Method Detail

register

public void register(JobID id,
                     Collection<BlobKey> requiredJarFiles)
              throws IOException

Registers a job ID with a set of library paths that are required to run the job. For every registered job the library cache manager creates a class loader that is used to instantiate the vertex's environment later on.

Specified by:

register in interface LibraryCacheManager

Parameters:

id - the ID of the job to be registered.

requiredJarFiles - the client path's of the required libraries

Throws:

IOException - thrown if one of the requested libraries is not in the cache

unregister

public void unregister(JobID id)

Unregisters a job ID and releases the resources associated with it.

Specified by:

unregister in interface LibraryCacheManager

Parameters:

id - the job ID to unregister

getClassLoader

public ClassLoader getClassLoader(JobID id)

Returns the class loader to the specified vertex.

Specified by:

getClassLoader in interface LibraryCacheManager

Parameters:

id - the ID of the job to return the class loader for

Returns:

the class loader of requested vertex or null if no class loader has been registered with the given ID.

getFile

public File getFile(BlobKey blobKey)
             throws IOException

Description copied from interface: LibraryCacheManager

Returns a file handle to the file identified by the blob key.

Specified by:

getFile in interface LibraryCacheManager

Parameters:

blobKey - identifying the requested file

Returns:

File handle

Throws:

IOException

getBlobServerPort

public int getBlobServerPort()

shutdown

public void shutdown()
              throws IOException

Description copied from interface: LibraryCacheManager

Shutdown method

Specified by:

shutdown in interface LibraryCacheManager

Throws:

IOException

run

public void run()

Cleans up blobs which are not referenced anymore

Specified by:

run in interface Runnable

Specified by:

run in class TimerTask