org.apache.flink.runtime.source.coordinator

类 SourceCoordinator<SplitT extends org.apache.flink.api.connector.source.SourceSplit,EnumChkT>

java.lang.Object

org.apache.flink.runtime.source.coordinator.SourceCoordinator<SplitT,EnumChkT>

所有已实现的接口:

AutoCloseable, org.apache.flink.api.common.state.CheckpointListener, OperatorCoordinator

@Internal
public class SourceCoordinator<SplitT extends org.apache.flink.api.connector.source.SourceSplit,EnumChkT>
extends Object
implements OperatorCoordinator

The default implementation of the OperatorCoordinator for the Source.

The SourceCoordinator provides an event loop style thread model to interact with the Flink runtime. The coordinator ensures that all the state manipulations are made by its event loop thread. It also helps keep track of the necessary split assignments history per subtask to simplify the SplitEnumerator implementation.

The coordinator maintains a SplitEnumeratorContxt and shares it with the enumerator. When the coordinator receives an action request from the Flink runtime, it sets up the context, and calls corresponding method of the SplitEnumerator to take actions.

嵌套类概要

从接口继承的嵌套类/接口 org.apache.flink.runtime.operators.coordination.OperatorCoordinator

OperatorCoordinator.Context, OperatorCoordinator.Provider, OperatorCoordinator.SubtaskGateway

字段概要

从接口继承的字段 org.apache.flink.runtime.operators.coordination.OperatorCoordinator

NO_CHECKPOINT

构造器概要

构造器

构造器和说明
SourceCoordinator(String operatorName, org.apache.flink.api.connector.source.Source<?,SplitT,EnumChkT> source, SourceCoordinatorContext<SplitT> context, CoordinatorStore coordinatorStore)
SourceCoordinator(String operatorName, org.apache.flink.api.connector.source.Source<?,SplitT,EnumChkT> source, SourceCoordinatorContext<SplitT> context, CoordinatorStore coordinatorStore, org.apache.flink.api.common.eventtime.WatermarkAlignmentParams watermarkAlignmentParams)

方法概要

限定符和类型	方法和说明
void	checkpointCoordinator(long checkpointId, CompletableFuture<byte[]> result) Takes a checkpoint of the coordinator.
void	close() This method is called when the coordinator is disposed.
void	handleEventFromOperator(int subtask, OperatorEvent event) Hands an OperatorEvent coming from a parallel Operator instances (one of the parallel subtasks).
void	notifyCheckpointAborted(long checkpointId) We override the method here to remove the checked exception.
void	notifyCheckpointComplete(long checkpointId) We override the method here to remove the checked exception.
void	resetToCheckpoint(long checkpointId, byte[] checkpointData) Resets the coordinator to the given checkpoint.
void	start() Starts the coordinator.
void	subtaskFailed(int subtaskId, Throwable reason) Called when one of the subtasks of the task running the coordinated operator goes through a failover (failure / recovery cycle).
void	subtaskReady(int subtask, OperatorCoordinator.SubtaskGateway gateway) This is called when a subtask of the Operator becomes ready to receive events, both after initial startup and after task failover.
void	subtaskReset(int subtaskId, long checkpointId) Called if a task is recovered as part of a partial failover, meaning a failover handled by the scheduler's failover strategy (by default recovering a pipelined region).

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

SourceCoordinator

public SourceCoordinator(String operatorName,
                         org.apache.flink.api.connector.source.Source<?,SplitT,EnumChkT> source,
                         SourceCoordinatorContext<SplitT> context,
                         CoordinatorStore coordinatorStore)

SourceCoordinator

public SourceCoordinator(String operatorName,
                         org.apache.flink.api.connector.source.Source<?,SplitT,EnumChkT> source,
                         SourceCoordinatorContext<SplitT> context,
                         CoordinatorStore coordinatorStore,
                         org.apache.flink.api.common.eventtime.WatermarkAlignmentParams watermarkAlignmentParams)

方法详细资料

start

public void start()
           throws Exception

从接口复制的说明: OperatorCoordinator

Starts the coordinator. This method is called once at the beginning, before any other methods.

指定者:

start 在接口中 OperatorCoordinator

抛出:

Exception - Any exception thrown from this method causes a full job failure.

close

public void close()
           throws Exception

从接口复制的说明: OperatorCoordinator

This method is called when the coordinator is disposed. This method should release currently held resources. Exceptions in this method do not cause the job to fail.

指定者:

close 在接口中 AutoCloseable

指定者:

close 在接口中 OperatorCoordinator

抛出:

Exception

handleEventFromOperator

public void handleEventFromOperator(int subtask,
                                    OperatorEvent event)

从接口复制的说明: OperatorCoordinator

Hands an OperatorEvent coming from a parallel Operator instances (one of the parallel subtasks).

指定者:

handleEventFromOperator 在接口中 OperatorCoordinator

subtaskFailed

public void subtaskFailed(int subtaskId,
                          @Nullable
                          Throwable reason)

从接口复制的说明: OperatorCoordinator

Called when one of the subtasks of the task running the coordinated operator goes through a failover (failure / recovery cycle).

This method is called every time there is a failover of a subtasks, regardless of whether there it is a partial failover or a global failover.

指定者:

subtaskFailed 在接口中 OperatorCoordinator

subtaskReset

public void subtaskReset(int subtaskId,
                         long checkpointId)

从接口复制的说明: OperatorCoordinator

Called if a task is recovered as part of a partial failover, meaning a failover handled by the scheduler's failover strategy (by default recovering a pipelined region). The method is invoked for each subtask involved in that partial failover.

In contrast to this method, the OperatorCoordinator.resetToCheckpoint(long, byte[]) method is called in the case of a global failover, which is the case when the coordinator (JobManager) is recovered.

指定者:

subtaskReset 在接口中 OperatorCoordinator

subtaskReady

public void subtaskReady(int subtask,
                         OperatorCoordinator.SubtaskGateway gateway)

从接口复制的说明: OperatorCoordinator

This is called when a subtask of the Operator becomes ready to receive events, both after initial startup and after task failover. The given SubtaskGateway can be used to send events to the executed subtask.

The given SubtaskGateway is bound to that specific execution attempt that became ready. All events sent through the gateway target that execution attempt; if the attempt is no longer running by the time the event is sent, then the events are failed.

指定者:

subtaskReady 在接口中 OperatorCoordinator

checkpointCoordinator

public void checkpointCoordinator(long checkpointId,
                                  CompletableFuture<byte[]> result)

从接口复制的说明: OperatorCoordinator

Takes a checkpoint of the coordinator. The checkpoint is identified by the given ID.

To confirm the checkpoint and store state in it, the given CompletableFuture must be completed with the state. To abort or dis-confirm the checkpoint, the given CompletableFuture must be completed exceptionally. In any case, the given CompletableFuture must be completed in some way, otherwise the checkpoint will not progress.

Exactly-once Semantics

The semantics are defined as follows:

• The point in time when the checkpoint future is completed is considered the point in time when the coordinator's checkpoint takes place.
• The OperatorCoordinator implementation must have a way of strictly ordering the sending of events and the completion of the checkpoint future (for example the same thread does both actions, or both actions are guarded by a mutex).
• Every event sent before the checkpoint future is completed is considered before the checkpoint.
• Every event sent after the checkpoint future is completed is considered to be after the checkpoint.

指定者:

checkpointCoordinator 在接口中 OperatorCoordinator

notifyCheckpointComplete

public void notifyCheckpointComplete(long checkpointId)

从接口复制的说明: OperatorCoordinator

We override the method here to remove the checked exception. Please check the Java docs of CheckpointListener.notifyCheckpointComplete(long) for more detail semantic of the method.

指定者:

notifyCheckpointComplete 在接口中 org.apache.flink.api.common.state.CheckpointListener

指定者:

notifyCheckpointComplete 在接口中 OperatorCoordinator

notifyCheckpointAborted

public void notifyCheckpointAborted(long checkpointId)

从接口复制的说明: OperatorCoordinator

We override the method here to remove the checked exception. Please check the Java docs of CheckpointListener.notifyCheckpointAborted(long) for more detail semantic of the method.

指定者:

notifyCheckpointAborted 在接口中 org.apache.flink.api.common.state.CheckpointListener

指定者:

notifyCheckpointAborted 在接口中 OperatorCoordinator

resetToCheckpoint

public void resetToCheckpoint(long checkpointId,
                              @Nullable
                              byte[] checkpointData)
                       throws Exception

从接口复制的说明: OperatorCoordinator

Resets the coordinator to the given checkpoint. When this method is called, the coordinator can discard all other in-flight working state. All subtasks will also have been reset to the same checkpoint.

This method is called in the case of a global failover of the system, which means a failover of the coordinator (JobManager). This method is not invoked on a partial failover; partial failovers call the OperatorCoordinator.subtaskReset(int, long) method for the involved subtasks.

This method is expected to behave synchronously with respect to other method calls and calls to Context methods. For example, Events being sent by the Coordinator after this method returns are assumed to take place after the checkpoint that was restored.

This method is called with a null state argument in the following situations:

• There is a recovery and there was no completed checkpoint yet.
• There is a recovery from a completed checkpoint/savepoint but it contained no state for the coordinator.

In both cases, the coordinator should reset to an empty (new) state.

Restoring implicitly notifies of Checkpoint Completion

Restoring to a checkpoint is a way of confirming that the checkpoint is complete. It is safe to commit side-effects that are predicated on checkpoint completion after this call.

Even if no call to OperatorCoordinator.notifyCheckpointComplete(long) happened, the checkpoint can still be complete (for example when a system failure happened directly after committing the checkpoint, before calling the OperatorCoordinator.notifyCheckpointComplete(long) method).

指定者:

resetToCheckpoint 在接口中 OperatorCoordinator

抛出:

Exception