org.apache.flink.runtime.minicluster

类 MiniCluster

java.lang.Object

org.apache.flink.runtime.minicluster.MiniCluster

所有已实现的接口:

AutoCloseable, org.apache.flink.util.AutoCloseableAsync

public class MiniCluster
extends Object
implements org.apache.flink.util.AutoCloseableAsync

MiniCluster to execute Flink jobs locally.

嵌套类概要

嵌套类

限定符和类型	类和说明
protected static class	MiniCluster.CommonRpcServiceFactory Factory which returns always the common RpcService.
protected class	MiniCluster.DedicatedRpcServiceFactory Factory which creates and registers new RpcService.
static class	MiniCluster.HaServices HA Services to use.
protected static interface	MiniCluster.RpcServiceFactory Internal factory for RpcService.

构造器概要

构造器

构造器和说明
MiniCluster(MiniClusterConfiguration miniClusterConfiguration) Creates a new Flink mini cluster based on the given configuration.
MiniCluster(MiniClusterConfiguration miniClusterConfiguration, java.util.function.Supplier<org.apache.flink.util.Reference<org.apache.flink.runtime.rpc.RpcSystem>> rpcSystemSupplier)

方法概要

限定符和类型	方法和说明
CompletableFuture<Acknowledge>	cancelJob(org.apache.flink.api.common.JobID jobId)
CompletableFuture<Void>	closeAsync() Shuts down the mini cluster, failing all currently executing jobs.
protected DispatcherResourceManagerComponentFactory	createDispatcherResourceManagerComponentFactory()
protected Collection<? extends DispatcherResourceManagerComponent>	createDispatcherResourceManagerComponents(org.apache.flink.configuration.Configuration configuration, MiniCluster.RpcServiceFactory rpcServiceFactory, BlobServer blobServer, HeartbeatServices heartbeatServices, MetricRegistry metricRegistry, MetricQueryServiceRetriever metricQueryServiceRetriever, org.apache.flink.runtime.rpc.FatalErrorHandler fatalErrorHandler)
protected HighAvailabilityServices	createHighAvailabilityServices(org.apache.flink.configuration.Configuration configuration, Executor executor)
protected org.apache.flink.runtime.rpc.RpcService	createLocalRpcService(org.apache.flink.configuration.Configuration configuration, org.apache.flink.runtime.rpc.RpcSystem rpcSystem) Factory method to instantiate the local RPC service.
protected MetricRegistryImpl	createMetricRegistry(org.apache.flink.configuration.Configuration config, long maximumMessageSizeInBytes) Factory method to create the metric registry for the mini cluster.
protected org.apache.flink.runtime.rpc.RpcService	createRemoteRpcService(org.apache.flink.configuration.Configuration configuration, String bindAddress, int bindPort, org.apache.flink.runtime.rpc.RpcSystem rpcSystem) Factory method to instantiate the remote RPC service.
protected org.apache.flink.runtime.rpc.RpcService	createRemoteRpcService(org.apache.flink.configuration.Configuration configuration, String externalAddress, String externalPortRange, String bindAddress, org.apache.flink.runtime.rpc.RpcSystem rpcSystem) Factory method to instantiate the remote RPC service.
CompletableFuture<CoordinationResponse>	deliverCoordinationRequestToCoordinator(org.apache.flink.api.common.JobID jobId, OperatorID operatorId, org.apache.flink.util.SerializedValue<CoordinationRequest> serializedRequest)
CompletableFuture<Acknowledge>	disposeSavepoint(String savepointPath)
org.apache.flink.api.common.JobExecutionResult	executeJobBlocking(JobGraph job) This method runs a job in blocking mode.
CompletableFuture<ArchivedExecutionGraph>	getArchivedExecutionGraph(org.apache.flink.api.common.JobID jobId)
ClusterInformation	getClusterInformation()
org.apache.flink.configuration.Configuration	getConfiguration()
protected CompletableFuture<DispatcherGateway>	getDispatcherGatewayFuture()
CompletableFuture<? extends AccessExecutionGraph>	getExecutionGraph(org.apache.flink.api.common.JobID jobId)
Optional<HaLeadershipControl>	getHaLeadershipControl() Returns HaLeadershipControl if enabled.
protected HighAvailabilityServices	getHaServices()
protected Executor	getIOExecutor()
CompletableFuture<org.apache.flink.api.common.JobStatus>	getJobStatus(org.apache.flink.api.common.JobID jobId)
CompletableFuture<ResourceOverview>	getResourceOverview()
CompletableFuture<URI>	getRestAddress()
boolean	isRunning() Checks if the mini cluster was started and is running.
CompletableFuture<Collection<JobStatusMessage>>	listJobs()
void	overrideRestoreModeForChangelogStateBackend()
CompletableFuture<ClusterOverview>	requestClusterOverview()
CompletableFuture<JobResult>	requestJobResult(org.apache.flink.api.common.JobID jobId)
void	runDetached(JobGraph job) This method executes a job in detached mode.
void	start() Starts the mini cluster, based on the configured properties.
void	startTaskManager() Starts additional TaskManager process.
CompletableFuture<String>	stopWithSavepoint(org.apache.flink.api.common.JobID jobId, String targetDirectory, boolean terminate, org.apache.flink.core.execution.SavepointFormatType formatType)
CompletableFuture<org.apache.flink.api.common.JobSubmissionResult>	submitJob(JobGraph jobGraph)
CompletableFuture<Void>	terminateTaskManager(int index) Terminates a TaskManager with the given index.
CompletableFuture<String>	triggerCheckpoint(org.apache.flink.api.common.JobID jobID)
CompletableFuture<String>	triggerSavepoint(org.apache.flink.api.common.JobID jobId, String targetDirectory, boolean cancelJob, org.apache.flink.core.execution.SavepointFormatType formatType)
protected boolean	useLocalCommunication()

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.flink.util.AutoCloseableAsync

close

构造器详细资料

MiniCluster

public MiniCluster(MiniClusterConfiguration miniClusterConfiguration)

Creates a new Flink mini cluster based on the given configuration.

参数:

miniClusterConfiguration - The configuration for the mini cluster

MiniCluster

public MiniCluster(MiniClusterConfiguration miniClusterConfiguration,
                   java.util.function.Supplier<org.apache.flink.util.Reference<org.apache.flink.runtime.rpc.RpcSystem>> rpcSystemSupplier)

方法详细资料

getRestAddress

public CompletableFuture<URI> getRestAddress()

getClusterInformation

public ClusterInformation getClusterInformation()

getIOExecutor

protected Executor getIOExecutor()

isRunning

public boolean isRunning()

Checks if the mini cluster was started and is running.

start

public void start()
           throws Exception

Starts the mini cluster, based on the configured properties.

抛出:

Exception - This method passes on any exception that occurs during the startup of the mini cluster.

createDispatcherResourceManagerComponents

@VisibleForTesting
protected Collection<? extends DispatcherResourceManagerComponent> createDispatcherResourceManagerComponents(org.apache.flink.configuration.Configuration configuration,
                                                                                                                                MiniCluster.RpcServiceFactory rpcServiceFactory,
                                                                                                                                BlobServer blobServer,
                                                                                                                                HeartbeatServices heartbeatServices,
                                                                                                                                MetricRegistry metricRegistry,
                                                                                                                                MetricQueryServiceRetriever metricQueryServiceRetriever,
                                                                                                                                org.apache.flink.runtime.rpc.FatalErrorHandler fatalErrorHandler)
                                                                                                                         throws Exception

抛出:

Exception

createDispatcherResourceManagerComponentFactory

protected DispatcherResourceManagerComponentFactory createDispatcherResourceManagerComponentFactory()

createHighAvailabilityServices

@VisibleForTesting
protected HighAvailabilityServices createHighAvailabilityServices(org.apache.flink.configuration.Configuration configuration,
                                                                                     Executor executor)
                                                                              throws Exception

抛出:

Exception

getHaLeadershipControl

public Optional<HaLeadershipControl> getHaLeadershipControl()

Returns HaLeadershipControl if enabled.

HaLeadershipControl allows granting and revoking leadership of HA components, e.g. JobManager. The method return Optional.empty() if the control is not enabled in MiniClusterConfiguration.

Enabling this feature disables HighAvailabilityOptions.HA_MODE option.

getHaServices

protected HighAvailabilityServices getHaServices()

closeAsync

public CompletableFuture<Void> closeAsync()

Shuts down the mini cluster, failing all currently executing jobs. The mini cluster can be started again by calling the start() method again.

This method shuts down all started services and components, even if an exception occurs in the process of shutting down some component.

指定者:

closeAsync 在接口中 org.apache.flink.util.AutoCloseableAsync

返回:

Future which is completed once the MiniCluster has been completely shut down

startTaskManager

public void startTaskManager()
                      throws Exception

Starts additional TaskManager process.

When the MiniCluster starts up, it always starts MiniClusterConfiguration.getNumTaskManagers() TaskManagers. All TaskManagers are indexed from 0 to the number of TaskManagers, started so far, minus one. This method starts a TaskManager with the next index which is the number of TaskManagers, started so far. The index always increases with each new started TaskManager. The indices of terminated TaskManagers are not reused after terminateTaskManager(int).

抛出:

Exception

useLocalCommunication

@VisibleForTesting
protected boolean useLocalCommunication()

getConfiguration

@VisibleForTesting
public org.apache.flink.configuration.Configuration getConfiguration()

overrideRestoreModeForChangelogStateBackend

@Internal
public void overrideRestoreModeForChangelogStateBackend()

terminateTaskManager

public CompletableFuture<Void> terminateTaskManager(int index)

Terminates a TaskManager with the given index.

See startTaskManager() to understand how TaskManagers are indexed. This method terminates a TaskManager with a given index but it does not clear the index. The index stays occupied for the lifetime of the MiniCluster and its TaskManager stays terminated. The index is not reused if more TaskManagers are started with startTaskManager().

参数:

index - index of the TaskManager to terminate

返回:

CompletableFuture of the given TaskManager termination

getArchivedExecutionGraph

public CompletableFuture<ArchivedExecutionGraph> getArchivedExecutionGraph(org.apache.flink.api.common.JobID jobId)

listJobs

public CompletableFuture<Collection<JobStatusMessage>> listJobs()

getJobStatus

public CompletableFuture<org.apache.flink.api.common.JobStatus> getJobStatus(org.apache.flink.api.common.JobID jobId)

cancelJob

public CompletableFuture<Acknowledge> cancelJob(org.apache.flink.api.common.JobID jobId)

triggerSavepoint

public CompletableFuture<String> triggerSavepoint(org.apache.flink.api.common.JobID jobId,
                                                  String targetDirectory,
                                                  boolean cancelJob,
                                                  org.apache.flink.core.execution.SavepointFormatType formatType)

triggerCheckpoint

public CompletableFuture<String> triggerCheckpoint(org.apache.flink.api.common.JobID jobID)

stopWithSavepoint

public CompletableFuture<String> stopWithSavepoint(org.apache.flink.api.common.JobID jobId,
                                                   String targetDirectory,
                                                   boolean terminate,
                                                   org.apache.flink.core.execution.SavepointFormatType formatType)

disposeSavepoint

public CompletableFuture<Acknowledge> disposeSavepoint(String savepointPath)

getExecutionGraph

public CompletableFuture<? extends AccessExecutionGraph> getExecutionGraph(org.apache.flink.api.common.JobID jobId)

deliverCoordinationRequestToCoordinator

public CompletableFuture<CoordinationResponse> deliverCoordinationRequestToCoordinator(org.apache.flink.api.common.JobID jobId,
                                                                                       OperatorID operatorId,
                                                                                       org.apache.flink.util.SerializedValue<CoordinationRequest> serializedRequest)

getResourceOverview

public CompletableFuture<ResourceOverview> getResourceOverview()

runDetached

public void runDetached(JobGraph job)
                 throws JobExecutionException,
                        InterruptedException

This method executes a job in detached mode. The method returns immediately after the job has been added to the

参数:

job - The Flink job to execute

抛出:

JobExecutionException - Thrown if anything went amiss during initial job launch, or if the job terminally failed.

InterruptedException

executeJobBlocking

public org.apache.flink.api.common.JobExecutionResult executeJobBlocking(JobGraph job)
                                                                  throws JobExecutionException,
                                                                         InterruptedException

This method runs a job in blocking mode. The method returns only after the job completed successfully, or after it failed terminally.

参数:

job - The Flink job to execute

返回:

The result of the job execution

抛出:

JobExecutionException - Thrown if anything went amiss during initial job launch, or if the job terminally failed.

InterruptedException

submitJob

public CompletableFuture<org.apache.flink.api.common.JobSubmissionResult> submitJob(JobGraph jobGraph)

requestJobResult

public CompletableFuture<JobResult> requestJobResult(org.apache.flink.api.common.JobID jobId)

requestClusterOverview

public CompletableFuture<ClusterOverview> requestClusterOverview()

getDispatcherGatewayFuture

@VisibleForTesting
protected CompletableFuture<DispatcherGateway> getDispatcherGatewayFuture()

createMetricRegistry

protected MetricRegistryImpl createMetricRegistry(org.apache.flink.configuration.Configuration config,
                                                  long maximumMessageSizeInBytes)

Factory method to create the metric registry for the mini cluster.

参数:

config - The configuration of the mini cluster

maximumMessageSizeInBytes - the maximum message size

createRemoteRpcService

protected org.apache.flink.runtime.rpc.RpcService createRemoteRpcService(org.apache.flink.configuration.Configuration configuration,
                                                                         String bindAddress,
                                                                         int bindPort,
                                                                         org.apache.flink.runtime.rpc.RpcSystem rpcSystem)
                                                                  throws Exception

Factory method to instantiate the remote RPC service.

参数:

configuration - Flink configuration.

bindAddress - The address to bind the RPC service to.

bindPort - The port range to bind the RPC service to.

rpcSystem -

返回:

The instantiated RPC service

抛出:

Exception

createRemoteRpcService

protected org.apache.flink.runtime.rpc.RpcService createRemoteRpcService(org.apache.flink.configuration.Configuration configuration,
                                                                         String externalAddress,
                                                                         String externalPortRange,
                                                                         String bindAddress,
                                                                         org.apache.flink.runtime.rpc.RpcSystem rpcSystem)
                                                                  throws Exception

Factory method to instantiate the remote RPC service.

参数:

configuration - Flink configuration.

externalAddress - The external address to access the RPC service.

externalPortRange - The external port range to access the RPC service.

bindAddress - The address to bind the RPC service to.

rpcSystem -

返回:

The instantiated RPC service

抛出:

Exception

createLocalRpcService

protected org.apache.flink.runtime.rpc.RpcService createLocalRpcService(org.apache.flink.configuration.Configuration configuration,
                                                                        org.apache.flink.runtime.rpc.RpcSystem rpcSystem)
                                                                 throws Exception

Factory method to instantiate the local RPC service.

参数:

configuration - Flink configuration.

rpcSystem -

返回:

The instantiated RPC service

抛出:

Exception