org.apache.flink.client.program

Class MiniClusterClient

java.lang.Object

org.apache.flink.client.program.ClusterClient<org.apache.flink.client.program.MiniClusterClient.MiniClusterId>

org.apache.flink.client.program.MiniClusterClient

All Implemented Interfaces:

NewClusterClient

public class MiniClusterClient
extends ClusterClient<org.apache.flink.client.program.MiniClusterClient.MiniClusterId>
implements NewClusterClient

Client to interact with a MiniCluster.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.client.program.ClusterClient

ClusterClient.LazyActorSystemLoader

Field Summary

Fields inherited from class org.apache.flink.client.program.ClusterClient

actorSystemLoader, flinkConfig, highAvailabilityServices, lastJobExecutionResult, log, MAX_SLOTS_UNKNOWN, timeout

Constructor Summary

Constructors

Constructor and Description
MiniClusterClient(org.apache.flink.configuration.Configuration configuration, org.apache.flink.runtime.minicluster.MiniCluster miniCluster)

Method Summary

Modifier and Type	Method and Description
void	cancel(org.apache.flink.api.common.JobID jobId) Cancels a job identified by the job id.
String	cancelWithSavepoint(org.apache.flink.api.common.JobID jobId, String savepointDirectory) Cancels a job identified by the job id and triggers a savepoint.
CompletableFuture<org.apache.flink.runtime.messages.Acknowledge>	disposeSavepoint(String savepointPath)
Map<String,org.apache.flink.util.OptionalFailure<Object>>	getAccumulators(org.apache.flink.api.common.JobID jobID) Requests and returns the accumulators for the given job identifier.
Map<String,org.apache.flink.util.OptionalFailure<Object>>	getAccumulators(org.apache.flink.api.common.JobID jobID, ClassLoader loader) Requests and returns the accumulators for the given job identifier.
org.apache.flink.runtime.util.LeaderConnectionInfo	getClusterConnectionInfo() Gets the current cluster connection info (may change in case of a HA setup).
org.apache.flink.client.program.MiniClusterClient.MiniClusterId	getClusterId() Returns the cluster id identifying the cluster to which the client is connected.
org.apache.flink.runtime.clusterframework.messages.GetClusterStatusResponse	getClusterStatus() Returns the latest cluster status, with number of Taskmanagers and slots.
CompletableFuture<org.apache.flink.runtime.jobgraph.JobStatus>	getJobStatus(org.apache.flink.api.common.JobID jobId) Requests the JobStatus of the job with the given JobID.
int	getMaxSlots() The client may define an upper limit on the number of slots to use.
List<String>	getNewMessages() May return new messages from the cluster.
String	getWebInterfaceURL() Returns an URL (as a string) to the JobManager web interface.
boolean	hasUserJarsInClassPath(List<URL> userJarFiles) Returns true if the client already has the user jar and providing it again would result in duplicate uploading of the jar.
CompletableFuture<Collection<org.apache.flink.runtime.client.JobStatusMessage>>	listJobs() Lists the currently running and finished jobs on the cluster.
CompletableFuture<org.apache.flink.runtime.jobmaster.JobResult>	requestJobResult(org.apache.flink.api.common.JobID jobId) Request the JobResult for the given JobID.
void	shutdown() Shuts down the client.
void	stop(org.apache.flink.api.common.JobID jobId) Stops a program on Flink cluster whose job-manager is configured in this client's configuration.
CompletableFuture<org.apache.flink.api.common.JobSubmissionResult>	submitJob(org.apache.flink.runtime.jobgraph.JobGraph jobGraph) Submit the given JobGraph to the cluster.
org.apache.flink.api.common.JobSubmissionResult	submitJob(org.apache.flink.runtime.jobgraph.JobGraph jobGraph, ClassLoader classLoader) Calls the subclasses' submitJob method.
CompletableFuture<String>	triggerSavepoint(org.apache.flink.api.common.JobID jobId, String savepointDirectory) Triggers a savepoint for the job identified by the job id.
void	waitForClusterToBeReady() Blocks until the client has determined that the cluster is ready for Job submission.

Methods inherited from class org.apache.flink.client.program.ClusterClient

connectToJob, endSession, endSessions, getFlinkConfiguration, getJobGraph, getJobGraph, getJobManagerGateway, getOptimizedPlan, getOptimizedPlan, getOptimizedPlanAsJson, getPrintStatusDuringExecution, isDetached, logAndSysout, rescaleJob, retrieveJob, run, run, run, run, run, run, runDetached, setDetached, setPrintStatusDuringExecution, shutDownCluster

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MiniClusterClient

public MiniClusterClient(@Nonnull
                         org.apache.flink.configuration.Configuration configuration,
                         @Nonnull
                         org.apache.flink.runtime.minicluster.MiniCluster miniCluster)

Method Detail

shutdown

public void shutdown()
              throws Exception

Description copied from class: ClusterClient

Shuts down the client. This stops the internal actor system and actors.

Overrides:

shutdown in class ClusterClient<org.apache.flink.client.program.MiniClusterClient.MiniClusterId>

Throws:

Exception

submitJob

public org.apache.flink.api.common.JobSubmissionResult submitJob(org.apache.flink.runtime.jobgraph.JobGraph jobGraph,
                                                                 ClassLoader classLoader)
                                                          throws ProgramInvocationException

Description copied from class: ClusterClient

Calls the subclasses' submitJob method. It may decide to simply call one of the run methods or it may perform some custom job submission logic.

Specified by:

submitJob in class ClusterClient<org.apache.flink.client.program.MiniClusterClient.MiniClusterId>

Parameters:

jobGraph - The JobGraph to be submitted

Returns:

JobSubmissionResult

Throws:

ProgramInvocationException

submitJob

public CompletableFuture<org.apache.flink.api.common.JobSubmissionResult> submitJob(@Nonnull
                                                                                    org.apache.flink.runtime.jobgraph.JobGraph jobGraph)

Description copied from interface: NewClusterClient

Submit the given JobGraph to the cluster.

Specified by:

submitJob in interface NewClusterClient

Parameters:

jobGraph - to submit

Returns:

Future which is completed with the JobSubmissionResult

requestJobResult

public CompletableFuture<org.apache.flink.runtime.jobmaster.JobResult> requestJobResult(@Nonnull
                                                                                        org.apache.flink.api.common.JobID jobId)

Description copied from interface: NewClusterClient

Request the JobResult for the given JobID.

Specified by:

requestJobResult in interface NewClusterClient

Parameters:

jobId - for which to request the JobResult

Returns:

Future which is completed with the JobResult

cancel

public void cancel(org.apache.flink.api.common.JobID jobId)
            throws Exception

Description copied from class: ClusterClient

Cancels a job identified by the job id.

Overrides:

cancel in class ClusterClient<org.apache.flink.client.program.MiniClusterClient.MiniClusterId>

Parameters:

jobId - the job id

Throws:

Exception - In case an error occurred.

cancelWithSavepoint

public String cancelWithSavepoint(org.apache.flink.api.common.JobID jobId,
                                  @Nullable
                                  String savepointDirectory)
                           throws Exception

Description copied from class: ClusterClient

Cancels a job identified by the job id and triggers a savepoint.

Overrides:

cancelWithSavepoint in class ClusterClient<org.apache.flink.client.program.MiniClusterClient.MiniClusterId>

Parameters:

jobId - the job id

savepointDirectory - directory the savepoint should be written to

Returns:

path where the savepoint is located

Throws:

Exception - In case an error cocurred.

stop

public void stop(org.apache.flink.api.common.JobID jobId)
          throws Exception

Description copied from class: ClusterClient

Stops a program on Flink cluster whose job-manager is configured in this client's configuration. Stopping works only for streaming programs. Be aware, that the program might continue to run for a while after sending the stop command, because after sources stopped to emit data all operators need to finish processing.

Overrides:

stop in class ClusterClient<org.apache.flink.client.program.MiniClusterClient.MiniClusterId>

Parameters:

jobId - the job ID of the streaming program to stop

Throws:

Exception - If the job ID is invalid (ie, is unknown or refers to a batch job) or if sending the stop signal failed. That might be due to an I/O problem, ie, the job-manager is unreachable.

triggerSavepoint

public CompletableFuture<String> triggerSavepoint(org.apache.flink.api.common.JobID jobId,
                                                  @Nullable
                                                  String savepointDirectory)

Description copied from class: ClusterClient

Triggers a savepoint for the job identified by the job id. The savepoint will be written to the given savepoint directory, or CheckpointingOptions.SAVEPOINT_DIRECTORY if it is null.

Overrides:

triggerSavepoint in class ClusterClient<org.apache.flink.client.program.MiniClusterClient.MiniClusterId>

Parameters:

jobId - job id

savepointDirectory - directory the savepoint should be written to

Returns:

path future where the savepoint is located

disposeSavepoint

public CompletableFuture<org.apache.flink.runtime.messages.Acknowledge> disposeSavepoint(String savepointPath)

Overrides:

disposeSavepoint in class ClusterClient<org.apache.flink.client.program.MiniClusterClient.MiniClusterId>

listJobs

public CompletableFuture<Collection<org.apache.flink.runtime.client.JobStatusMessage>> listJobs()

Description copied from class: ClusterClient

Lists the currently running and finished jobs on the cluster.

Overrides:

listJobs in class ClusterClient<org.apache.flink.client.program.MiniClusterClient.MiniClusterId>

Returns:

future collection of running and finished jobs

getAccumulators

public Map<String,org.apache.flink.util.OptionalFailure<Object>> getAccumulators(org.apache.flink.api.common.JobID jobID)
                                                                          throws Exception

Description copied from class: ClusterClient

Requests and returns the accumulators for the given job identifier. Accumulators can be requested while a is running or after it has finished. The default class loader is used to deserialize the incoming accumulator results.

Overrides:

getAccumulators in class ClusterClient<org.apache.flink.client.program.MiniClusterClient.MiniClusterId>

Parameters:

jobID - The job identifier of a job.

Returns:

A Map containing the accumulator's name and its value.

Throws:

Exception

getAccumulators

public Map<String,org.apache.flink.util.OptionalFailure<Object>> getAccumulators(org.apache.flink.api.common.JobID jobID,
                                                                                 ClassLoader loader)
                                                                          throws Exception

Description copied from class: ClusterClient

Requests and returns the accumulators for the given job identifier. Accumulators can be requested while a is running or after it has finished.

Overrides:

getAccumulators in class ClusterClient<org.apache.flink.client.program.MiniClusterClient.MiniClusterId>

Parameters:

jobID - The job identifier of a job.

loader - The class loader for deserializing the accumulator results.

Returns:

A Map containing the accumulator's name and its value.

Throws:

Exception

getJobStatus

public CompletableFuture<org.apache.flink.runtime.jobgraph.JobStatus> getJobStatus(org.apache.flink.api.common.JobID jobId)

Description copied from class: ClusterClient

Requests the JobStatus of the job with the given JobID.

Overrides:

getJobStatus in class ClusterClient<org.apache.flink.client.program.MiniClusterClient.MiniClusterId>

getClusterId

public org.apache.flink.client.program.MiniClusterClient.MiniClusterId getClusterId()

Description copied from class: ClusterClient

Returns the cluster id identifying the cluster to which the client is connected.

Specified by:

getClusterId in class ClusterClient<org.apache.flink.client.program.MiniClusterClient.MiniClusterId>

Returns:

cluster id of the connected cluster

getClusterConnectionInfo

public org.apache.flink.runtime.util.LeaderConnectionInfo getClusterConnectionInfo()
                                                                            throws org.apache.flink.runtime.leaderretrieval.LeaderRetrievalException

Description copied from class: ClusterClient

Gets the current cluster connection info (may change in case of a HA setup).

Overrides:

getClusterConnectionInfo in class ClusterClient<org.apache.flink.client.program.MiniClusterClient.MiniClusterId>

Returns:

The the connection info to the leader component of the cluster

Throws:

org.apache.flink.runtime.leaderretrieval.LeaderRetrievalException - if the leader could not be retrieved

waitForClusterToBeReady

public void waitForClusterToBeReady()

Description copied from class: ClusterClient

Blocks until the client has determined that the cluster is ready for Job submission.

This is delayed until right before job submission to report any other errors first (e.g. invalid job definitions/errors in the user jar)

Specified by:

waitForClusterToBeReady in class ClusterClient<org.apache.flink.client.program.MiniClusterClient.MiniClusterId>

getWebInterfaceURL

public String getWebInterfaceURL()

Description copied from class: ClusterClient

Returns an URL (as a string) to the JobManager web interface.

Specified by:

getWebInterfaceURL in class ClusterClient<org.apache.flink.client.program.MiniClusterClient.MiniClusterId>

getClusterStatus

public org.apache.flink.runtime.clusterframework.messages.GetClusterStatusResponse getClusterStatus()

Description copied from class: ClusterClient

Returns the latest cluster status, with number of Taskmanagers and slots.

Specified by:

getClusterStatus in class ClusterClient<org.apache.flink.client.program.MiniClusterClient.MiniClusterId>

getNewMessages

public List<String> getNewMessages()

Description copied from class: ClusterClient

May return new messages from the cluster. Messages can be for example about failed containers or container launch requests.

Specified by:

getNewMessages in class ClusterClient<org.apache.flink.client.program.MiniClusterClient.MiniClusterId>

getMaxSlots

public int getMaxSlots()

Description copied from class: ClusterClient

The client may define an upper limit on the number of slots to use.

Specified by:

getMaxSlots in class ClusterClient<org.apache.flink.client.program.MiniClusterClient.MiniClusterId>

Returns:

-1 (ClusterClient.MAX_SLOTS_UNKNOWN) if unknown

hasUserJarsInClassPath

public boolean hasUserJarsInClassPath(List<URL> userJarFiles)

Description copied from class: ClusterClient

Returns true if the client already has the user jar and providing it again would result in duplicate uploading of the jar.

Specified by:

hasUserJarsInClassPath in class ClusterClient<org.apache.flink.client.program.MiniClusterClient.MiniClusterId>