org.apache.tez.runtime.api
Class OutputCommitter

java.lang.Object
  org.apache.tez.runtime.api.OutputCommitter

public abstract class OutputCommitter
extends Object

OutputCommitter to "finalize" the output and make it user-visible if needed. The OutputCommitter is allowed only on a terminal Output.

Constructor Summary

OutputCommitter()

Method Summary

abstract void	abortOutput(VertexStatus.State finalState) For aborting an output.
abstract void	commitOutput() For committing the output after successful completion of tasks that write the output.
abstract void	initialize(OutputCommitterContext context) Setup up the Output committer.
boolean	isTaskRecoverySupported() Whether the OutputCommitter supports recovery of output from a Task that completed in a previous DAG attempt
void	recoverTask(int taskIndex, int previousDAGAttempt) Recover task output from a previous DAG attempt
abstract void	setupOutput() For the framework to setup the output during initialization.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OutputCommitter

public OutputCommitter()

Method Detail

initialize

public abstract void initialize(OutputCommitterContext context)
                         throws Exception

Setup up the Output committer.

Parameters:

context - Context of the output that is being acted upon

Throws:

Exception

setupOutput

public abstract void setupOutput()
                          throws Exception

For the framework to setup the output during initialization. This is called from the application master process for the vertex. This will be called multiple times, once per dag attempt.

Throws:

Exception - if setup fails

commitOutput

public abstract void commitOutput()
                           throws Exception

For committing the output after successful completion of tasks that write the output. Note that this is invoked for the outputs of vertices whose tasks have successfully completed. This is called from the application master process. Based on user configuration, commit is called at the end of the DAG execution for all outputs or immediately upon completion of all the tasks that produced the output. This is guaranteed to only be called once.

Throws:

Exception

abortOutput

public abstract void abortOutput(VertexStatus.State finalState)
                          throws Exception

For aborting an output. Note that this is invoked for vertices with a final non-successful state. This is also called to abort a previously committed output in the case of a post-commit failure. This is called from the application master process. This may be called multiple times.

Parameters:

finalState - final run-state of the vertex

Throws:

Exception

isTaskRecoverySupported

public boolean isTaskRecoverySupported()

Whether the OutputCommitter supports recovery of output from a Task that completed in a previous DAG attempt

Returns:

True if recovery supported

recoverTask

public void recoverTask(int taskIndex,
                        int previousDAGAttempt)
                 throws Exception

Recover task output from a previous DAG attempt

Parameters:

taskIndex - Index of task to be recovered

previousDAGAttempt - Previous DAG Attempt Number

Throws:

Exception