org.apache.tez.dag.app
Class DAGAppMaster

java.lang.Object
  org.apache.hadoop.service.AbstractService
      org.apache.tez.dag.app.DAGAppMaster

All Implemented Interfaces:

Closeable, org.apache.hadoop.service.Service

public class DAGAppMaster
extends org.apache.hadoop.service.AbstractService

The Map-Reduce Application Master. The state machine is encapsulated in the implementation of Job interface. All state changes happens via Job interface. Each event results in a Finite State Transition in Job. MR AppMaster is the composition of loosely coupled services. The services interact with each other via events. The components resembles the Actors model. The component acts on received event and send out the events to other components. This keeps it highly concurrent with no or minimal synchronization needs. The events are dispatched by a central Dispatch mechanism. All components register to the Dispatcher. The information is shared across different components using AppContext.

Nested Class Summary

class

DAGAppMaster.DAGClientHandler

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

org.apache.hadoop.service.Service.STATE

Field Summary

protected boolean	isLastAMRetry
static int	SHUTDOWN_HOOK_PRIORITY Priority of the DAGAppMaster shutdown hook.

Constructor Summary

DAGAppMaster(org.apache.hadoop.yarn.api.records.ApplicationAttemptId applicationAttemptId, org.apache.hadoop.yarn.api.records.ContainerId containerId, String nmHost, int nmPort, int nmHttpPort, org.apache.hadoop.yarn.util.Clock clock, long appSubmitTime, boolean isSession)

DAGAppMaster(org.apache.hadoop.yarn.api.records.ApplicationAttemptId applicationAttemptId, org.apache.hadoop.yarn.api.records.ContainerId containerId, String nmHost, int nmPort, int nmHttpPort, long appSubmitTime, boolean isSession)

Method Summary

protected void	addIfService(Object object, boolean addDispatcher)
protected void	addIfServiceDependency(Object object, Object dependency)
protected ContainerHeartbeatHandler	createContainerHeartbeatHandler(AppContext context, org.apache.hadoop.conf.Configuration conf)
protected ContainerLauncher	createContainerLauncher(AppContext context)
protected ContainerSignatureMatcher	createContainerSignatureMatcher()
protected DAG	createDAG(org.apache.tez.dag.api.records.DAGProtos.DAGPlan dagPB)
protected org.apache.hadoop.yarn.event.Dispatcher	createDispatcher()
protected TaskAttemptListener	createTaskAttemptListener(AppContext context, TaskHeartbeatHandler thh, ContainerHeartbeatHandler chh)
protected TaskHeartbeatHandler	createTaskHeartbeatHandler(AppContext context, org.apache.hadoop.conf.Configuration conf)
org.apache.hadoop.yarn.api.records.ContainerId	getAppContainerId()
org.apache.hadoop.yarn.api.records.ApplicationId	getAppID()
String	getAppNMHost()
int	getAppNMHttpPort()
int	getAppNMPort()
org.apache.hadoop.yarn.api.records.ApplicationAttemptId	getAttemptID()
ContainerLauncher	getContainerLauncher()
AppContext	getContext()
List<String>	getDiagnostics()
org.apache.hadoop.yarn.event.Dispatcher	getDispatcher()
float	getProgress()
int	getStartCount()
DAGAppMasterState	getState()
TaskAttemptListener	getTaskAttemptListener()
protected static void	initAndStartAppMaster(DAGAppMaster appMaster, org.apache.hadoop.conf.Configuration conf, String jobUserName)
boolean	isSession()
static void	main(String[] args)
void	serviceInit(org.apache.hadoop.conf.Configuration conf)
void	serviceStart()
void	serviceStop()
void	setCurrentDAG(DAG currentDAG)
protected void	sysexit() Exit call.

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

SHUTDOWN_HOOK_PRIORITY

public static final int SHUTDOWN_HOOK_PRIORITY

Priority of the DAGAppMaster shutdown hook.

See Also:

Constant Field Values

isLastAMRetry

protected boolean isLastAMRetry

Constructor Detail

DAGAppMaster

public DAGAppMaster(org.apache.hadoop.yarn.api.records.ApplicationAttemptId applicationAttemptId,
                    org.apache.hadoop.yarn.api.records.ContainerId containerId,
                    String nmHost,
                    int nmPort,
                    int nmHttpPort,
                    long appSubmitTime,
                    boolean isSession)

DAGAppMaster

public DAGAppMaster(org.apache.hadoop.yarn.api.records.ApplicationAttemptId applicationAttemptId,
                    org.apache.hadoop.yarn.api.records.ContainerId containerId,
                    String nmHost,
                    int nmPort,
                    int nmHttpPort,
                    org.apache.hadoop.yarn.util.Clock clock,
                    long appSubmitTime,
                    boolean isSession)

Method Detail

serviceInit

public void serviceInit(org.apache.hadoop.conf.Configuration conf)
                 throws Exception

Overrides:

serviceInit in class org.apache.hadoop.service.AbstractService

Throws:

Exception

createContainerSignatureMatcher

protected ContainerSignatureMatcher createContainerSignatureMatcher()

createDispatcher

protected org.apache.hadoop.yarn.event.Dispatcher createDispatcher()

sysexit

protected void sysexit()

Exit call. Just in a function call to enable testing.

setCurrentDAG

public void setCurrentDAG(DAG currentDAG)

createDAG

protected DAG createDAG(org.apache.tez.dag.api.records.DAGProtos.DAGPlan dagPB)

addIfService

protected void addIfService(Object object,
                            boolean addDispatcher)

addIfServiceDependency

protected void addIfServiceDependency(Object object,
                                      Object dependency)

createTaskAttemptListener

protected TaskAttemptListener createTaskAttemptListener(AppContext context,
                                                        TaskHeartbeatHandler thh,
                                                        ContainerHeartbeatHandler chh)

createTaskHeartbeatHandler

protected TaskHeartbeatHandler createTaskHeartbeatHandler(AppContext context,
                                                          org.apache.hadoop.conf.Configuration conf)

createContainerHeartbeatHandler

protected ContainerHeartbeatHandler createContainerHeartbeatHandler(AppContext context,
                                                                    org.apache.hadoop.conf.Configuration conf)

createContainerLauncher

protected ContainerLauncher createContainerLauncher(AppContext context)

getAppID

public org.apache.hadoop.yarn.api.records.ApplicationId getAppID()

getAttemptID

public org.apache.hadoop.yarn.api.records.ApplicationAttemptId getAttemptID()

getStartCount

public int getStartCount()

getContext

public AppContext getContext()

getDispatcher

public org.apache.hadoop.yarn.event.Dispatcher getDispatcher()

getContainerLauncher

public ContainerLauncher getContainerLauncher()

getTaskAttemptListener

public TaskAttemptListener getTaskAttemptListener()

getAppContainerId

public org.apache.hadoop.yarn.api.records.ContainerId getAppContainerId()

getAppNMHost

public String getAppNMHost()

getAppNMPort

public int getAppNMPort()

getAppNMHttpPort

public int getAppNMHttpPort()

getState

public DAGAppMasterState getState()

getDiagnostics

public List<String> getDiagnostics()

getProgress

public float getProgress()

serviceStart

public void serviceStart()
                  throws Exception

Overrides:

serviceStart in class org.apache.hadoop.service.AbstractService

Throws:

Exception

serviceStop

public void serviceStop()
                 throws Exception

Overrides:

serviceStop in class org.apache.hadoop.service.AbstractService

Throws:

Exception

isSession

public boolean isSession()

main

public static void main(String[] args)

initAndStartAppMaster

protected static void initAndStartAppMaster(DAGAppMaster appMaster,
                                            org.apache.hadoop.conf.Configuration conf,
                                            String jobUserName)
                                     throws IOException,
                                            InterruptedException

Throws:

IOException

InterruptedException