org.apache.flink.runtime.jobmanager.scheduler

Class DefaultScheduler

java.lang.Object

org.apache.flink.runtime.jobmanager.scheduler.DefaultScheduler

All Implemented Interfaces:

ExecutionStageListener, JobStatusListener, InstanceListener

public class DefaultScheduler
extends Object
implements InstanceListener, JobStatusListener, ExecutionStageListener

The default scheduler for Nephele. While Nephele's JobManager is responsible for requesting the required instances for the job at the InstanceManager, the scheduler is in charge of assigning the individual tasks to the instances.

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.apache.commons.logging.Log	LOG The LOG object to report events within the scheduler.

Constructor Summary

Constructors

Constructor and Description
DefaultScheduler(DeploymentManager deploymentManager, InstanceManager instanceManager) Constructs a new abstract scheduler.

Method Summary

Methods

Modifier and Type	Method and Description
void	allocatedResourcesDied(JobID jobID, List<AllocatedResource> allocatedResources) Called if one or more allocated resources assigned to at least one job have died unexpectedly.
void	checkAndReleaseAllocatedResource(ExecutionGraph executionGraph, AllocatedResource allocatedResource) Checks if the given AllocatedResource is still required for the execution of the given execution graph.
void	deployAssignedInputVertices(ExecutionGraph executionGraph) Collects all execution vertices with the state ASSIGNED starting from the input vertices of the current execution stage and deploys them on the assigned AllocatedResource objects.
void	deployAssignedPipeline(ExecutionPipeline pipeline) Collects all execution vertices with the state ASSIGNED from the given pipeline and deploys them on the assigned AllocatedResource objects.
void	deployAssignedVertices(Collection<ExecutionVertex> startVertices) Collects all execution vertices with the state ASSIGNED starting from the given collection of start vertices and deploys them on the assigned AllocatedResource objects.
void	deployAssignedVertices(ExecutionVertex startVertex) Collects all execution vertices with the state ASSIGNED starting from the given start vertex and deploys them on the assigned AllocatedResource objects.
ExecutionGraph	getExecutionGraphByID(JobID jobID) Returns the execution graph which is associated with the given job ID.
InstanceManager	getInstanceManager() Returns the InstanceManager object which is used by the current scheduler.
void	jobStatusHasChanged(ExecutionGraph executionGraph, InternalJobStatus newJobStatus, String optionalMessage) Called when the status of the job with the given {@li
void	nextExecutionStageEntered(JobID jobID, ExecutionStage executionStage) This method is called to indicate that the job with the given JobID has finished its previous ExecutionStage and has entered the next ExecutionStage.
protected void	replayCheckpointsFromPreviousStage(ExecutionGraph executionGraph)
protected void	requestInstances(ExecutionStage executionStage) Collects the instances required to run the job from the given ExecutionStage and requests them at the loaded instance manager.
void	resourcesAllocated(JobID jobID, List<AllocatedResource> allocatedResources) Called if one or more requested resources have become available.
void	scheduleJob(ExecutionGraph executionGraph) Adds a job represented by an ExecutionGraph object to the scheduler.
void	shutdown() Shuts the scheduler down.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

protected static final org.apache.commons.logging.Log LOG

The LOG object to report events within the scheduler.

Constructor Detail

DefaultScheduler

public DefaultScheduler(DeploymentManager deploymentManager,
                InstanceManager instanceManager)

Constructs a new abstract scheduler.

Parameters:

deploymentManager - the deployment manager assigned to this scheduler

instanceManager - the instance manager to be used with this scheduler

Method Detail

scheduleJob

public void scheduleJob(ExecutionGraph executionGraph)
                 throws SchedulingException

Adds a job represented by an ExecutionGraph object to the scheduler. The job is then executed according to the strategies of the concrete scheduler implementation.

Parameters:

executionGraph - the job to be added to the scheduler

Throws:

SchedulingException - thrown if an error occurs and the scheduler does not accept the new job

getExecutionGraphByID

public ExecutionGraph getExecutionGraphByID(JobID jobID)

Returns the execution graph which is associated with the given job ID.

Parameters:

jobID - the job ID to search the execution graph for

Returns:

the execution graph which belongs to the given job ID or null

shutdown

public void shutdown()

Shuts the scheduler down. After shut down no jobs can be added to the scheduler.

jobStatusHasChanged

public void jobStatusHasChanged(ExecutionGraph executionGraph,
                       InternalJobStatus newJobStatus,
                       String optionalMessage)

Description copied from interface: JobStatusListener

Called when the status of the job with the given {@li

Specified by:

jobStatusHasChanged in interface JobStatusListener

Parameters:

executionGraph - the executionGraph representing the job the event belongs to

newJobStatus - the new job status

optionalMessage - an optional message (possibly null) that can be attached to the state change

nextExecutionStageEntered

public void nextExecutionStageEntered(JobID jobID,
                             ExecutionStage executionStage)

Description copied from interface: ExecutionStageListener

This method is called to indicate that the job with the given JobID has finished its previous ExecutionStage and has entered the next ExecutionStage. Note that a notification is not sent when the job has entered its initial execution stage.

Specified by:

nextExecutionStageEntered in interface ExecutionStageListener

Parameters:

jobID - the ID of the job the notification belongs to

executionStage - the next execution stage that has just been entered

getInstanceManager

public InstanceManager getInstanceManager()

Returns the InstanceManager object which is used by the current scheduler.

Returns:

the InstanceManager object which is used by the current scheduler

requestInstances

protected void requestInstances(ExecutionStage executionStage)
                         throws InstanceException

Collects the instances required to run the job from the given ExecutionStage and requests them at the loaded instance manager.

Parameters:

executionStage - the execution stage to collect the required instances from

Throws:

InstanceException - thrown if the given execution graph is already processing its final stage

deployAssignedVertices

public void deployAssignedVertices(ExecutionVertex startVertex)

Collects all execution vertices with the state ASSIGNED starting from the given start vertex and deploys them on the assigned AllocatedResource objects.

Parameters:

startVertex - the execution vertex to start the deployment from

deployAssignedPipeline

public void deployAssignedPipeline(ExecutionPipeline pipeline)

Collects all execution vertices with the state ASSIGNED from the given pipeline and deploys them on the assigned AllocatedResource objects.

Parameters:

pipeline - the execution pipeline to be deployed

deployAssignedVertices

public void deployAssignedVertices(Collection<ExecutionVertex> startVertices)

Collects all execution vertices with the state ASSIGNED starting from the given collection of start vertices and deploys them on the assigned AllocatedResource objects.

Parameters:

startVertices - the collection of execution vertices to start the deployment from

deployAssignedInputVertices

public void deployAssignedInputVertices(ExecutionGraph executionGraph)

Collects all execution vertices with the state ASSIGNED starting from the input vertices of the current execution stage and deploys them on the assigned AllocatedResource objects.

Parameters:

executionGraph - the execution graph to collect the vertices from

resourcesAllocated

public void resourcesAllocated(JobID jobID,
                      List<AllocatedResource> allocatedResources)

Description copied from interface: InstanceListener

Called if one or more requested resources have become available.

Specified by:

resourcesAllocated in interface InstanceListener

Parameters:

jobID - the ID of the job the initial request has been triggered for

allocatedResources - the resources which have been allocated as a response to the initial request

checkAndReleaseAllocatedResource

public void checkAndReleaseAllocatedResource(ExecutionGraph executionGraph,
                                    AllocatedResource allocatedResource)

Checks if the given AllocatedResource is still required for the execution of the given execution graph. If the resource is no longer assigned to a vertex that is either currently running or about to run the given resource is returned to the instance manager for deallocation.

Parameters:

executionGraph - the execution graph the provided resource has been used for so far

allocatedResource - the allocated resource to check the assignment for

replayCheckpointsFromPreviousStage

protected void replayCheckpointsFromPreviousStage(ExecutionGraph executionGraph)

allocatedResourcesDied

public void allocatedResourcesDied(JobID jobID,
                          List<AllocatedResource> allocatedResources)

Description copied from interface: InstanceListener

Called if one or more allocated resources assigned to at least one job have died unexpectedly.

Specified by:

allocatedResourcesDied in interface InstanceListener

Parameters:

jobID - the ID of the job the instance is used for

allocatedResources - the allocated resources which are affected by the instance death