org.apache.flink.runtime.resourcemanager.active

类 ActiveResourceManager<WorkerType extends ResourceIDRetrievable>

java.lang.Object

org.apache.flink.runtime.rpc.RpcEndpoint

org.apache.flink.runtime.rpc.FencedRpcEndpoint<ResourceManagerId>

org.apache.flink.runtime.resourcemanager.ResourceManager<WorkerType>

org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager<WorkerType>

所有已实现的接口:

AutoCloseable, BlocklistListener, ClusterPartitionManager, ResourceEventHandler<WorkerType>, ResourceManagerGateway, org.apache.flink.runtime.rpc.FencedRpcGateway<ResourceManagerId>, org.apache.flink.runtime.rpc.RpcGateway, DelegationTokenManager.Listener, org.apache.flink.util.AutoCloseableAsync

public class ActiveResourceManager<WorkerType extends ResourceIDRetrievable>
extends ResourceManager<WorkerType>
implements ResourceEventHandler<WorkerType>

An active implementation of ResourceManager.

This resource manager actively requests and releases resources from/to the external resource management frameworks. With different ResourceManagerDriver provided, this resource manager can work with various frameworks.

嵌套类概要

从类继承的嵌套类/接口 org.apache.flink.runtime.rpc.RpcEndpoint

org.apache.flink.runtime.rpc.RpcEndpoint.MainThreadExecutor

字段概要

字段

限定符和类型	字段和说明
protected org.apache.flink.configuration.Configuration	flinkConfig

从类继承的字段 org.apache.flink.runtime.resourcemanager.ResourceManager

blocklistHandler, ioExecutor, RESOURCE_MANAGER_NAME, resourceManagerMetricGroup

从类继承的字段 org.apache.flink.runtime.rpc.RpcEndpoint

log, rpcServer

构造器概要

构造器

构造器和说明

ActiveResourceManager(ResourceManagerDriver<WorkerType> resourceManagerDriver, org.apache.flink.configuration.Configuration flinkConfig, org.apache.flink.runtime.rpc.RpcService rpcService, UUID leaderSessionId, ResourceID resourceId, HeartbeatServices heartbeatServices, DelegationTokenManager delegationTokenManager, SlotManager slotManager, ResourceManagerPartitionTrackerFactory clusterPartitionTrackerFactory, BlocklistHandler.Factory blocklistHandlerFactory, JobLeaderIdService jobLeaderIdService, ClusterInformation clusterInformation, org.apache.flink.runtime.rpc.FatalErrorHandler fatalErrorHandler, ResourceManagerMetricGroup resourceManagerMetricGroup, ThresholdMeter startWorkerFailureRater, java.time.Duration retryInterval, java.time.Duration workerRegistrationTimeout, java.time.Duration previousWorkerRecoverTimeout, Executor ioExecutor)

方法概要

限定符和类型	方法和说明
void	declareResourceNeeded(Collection<ResourceDeclaration> resourceDeclarations)
CompletableFuture<Void>	getReadyToServeFuture() Get the ready to serve future of the resource manager.
protected ResourceAllocator	getResourceAllocator()
protected Optional<WorkerType>	getWorkerNodeIfAcceptRegistration(ResourceID resourceID) Get worker node if the worker resource is accepted.
protected void	initialize() Initializes the framework specific components.
protected void	internalDeregisterApplication(ApplicationStatus finalStatus, String optionalDiagnostics) The framework specific code to deregister the application.
void	onError(Throwable exception) Notifies that an error has occurred that the process cannot proceed.
void	onPreviousAttemptWorkersRecovered(Collection<WorkerType> recoveredWorkers) Notifies that workers of previous attempt have been recovered from the external resource manager.
protected void	onWorkerRegistered(WorkerType worker, WorkerResourceSpec workerResourceSpec)
void	onWorkerTerminated(ResourceID resourceId, String diagnostics) Notifies that the worker has been terminated.
protected void	registerMetrics()
void	requestNewWorker(WorkerResourceSpec workerResourceSpec) Allocates a resource using the worker resource specification.
protected void	terminate() Terminates the framework specific components.

从类继承的方法 org.apache.flink.runtime.resourcemanager.ResourceManager

closeJobManagerConnection, closeTaskManagerConnection, declareRequiredResources, deregisterApplication, disconnectJobManager, disconnectTaskManager, getClusterPartitionsShuffleDescriptors, getInstanceIdByResourceId, getNumberOfRegisteredTaskManagers, getStartedFuture, getWorkerByInstanceId, heartbeatFromJobManager, heartbeatFromTaskManager, jobLeaderLostLeadership, listDataSets, notifyNewBlockedNodes, notifySlotAvailable, onFatalError, onNewTokensObtained, onStart, onStop, registerJobMaster, registerTaskExecutor, releaseClusterPartitions, removeJob, reportClusterPartitions, requestResourceOverview, requestTaskExecutorThreadInfoGateway, requestTaskManagerDetailsInfo, requestTaskManagerFileUploadByName, requestTaskManagerFileUploadByType, requestTaskManagerInfo, requestTaskManagerLogList, requestTaskManagerMetricQueryServiceAddresses, requestThreadDump, sendSlotReport, setFailUnfulfillableRequest, stopWorkerIfSupported

从类继承的方法 org.apache.flink.runtime.rpc.FencedRpcEndpoint

getFencingToken

从类继承的方法 org.apache.flink.runtime.rpc.RpcEndpoint

callAsync, closeAsync, getAddress, getEndpointId, getHostname, getMainThreadExecutor, getRpcService, getSelfGateway, getTerminationFuture, internalCallOnStart, internalCallOnStop, isRunning, registerResource, runAsync, scheduleRunAsync, scheduleRunAsync, start, stop, unregisterResource, validateRunsInMainThread

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.flink.runtime.rpc.FencedRpcGateway

getFencingToken

从接口继承的方法 org.apache.flink.runtime.rpc.RpcGateway

getAddress, getHostname

从接口继承的方法 org.apache.flink.util.AutoCloseableAsync

close

字段详细资料

flinkConfig

protected final org.apache.flink.configuration.Configuration flinkConfig

构造器详细资料

ActiveResourceManager

public ActiveResourceManager(ResourceManagerDriver<WorkerType> resourceManagerDriver,
                             org.apache.flink.configuration.Configuration flinkConfig,
                             org.apache.flink.runtime.rpc.RpcService rpcService,
                             UUID leaderSessionId,
                             ResourceID resourceId,
                             HeartbeatServices heartbeatServices,
                             DelegationTokenManager delegationTokenManager,
                             SlotManager slotManager,
                             ResourceManagerPartitionTrackerFactory clusterPartitionTrackerFactory,
                             BlocklistHandler.Factory blocklistHandlerFactory,
                             JobLeaderIdService jobLeaderIdService,
                             ClusterInformation clusterInformation,
                             org.apache.flink.runtime.rpc.FatalErrorHandler fatalErrorHandler,
                             ResourceManagerMetricGroup resourceManagerMetricGroup,
                             ThresholdMeter startWorkerFailureRater,
                             java.time.Duration retryInterval,
                             java.time.Duration workerRegistrationTimeout,
                             java.time.Duration previousWorkerRecoverTimeout,
                             Executor ioExecutor)

方法详细资料

initialize

protected void initialize()
                   throws ResourceManagerException

从类复制的说明: ResourceManager

Initializes the framework specific components.

指定者:

initialize 在类中 ResourceManager<WorkerType extends ResourceIDRetrievable>

抛出:

ResourceManagerException - which occurs during initialization and causes the resource manager to fail.

terminate

protected void terminate()
                  throws ResourceManagerException

从类复制的说明: ResourceManager

Terminates the framework specific components.

指定者:

terminate 在类中 ResourceManager<WorkerType extends ResourceIDRetrievable>

抛出:

ResourceManagerException

internalDeregisterApplication

protected void internalDeregisterApplication(ApplicationStatus finalStatus,
                                             @Nullable
                                             String optionalDiagnostics)
                                      throws ResourceManagerException

从类复制的说明: ResourceManager

The framework specific code to deregister the application. This should report the application's final status and shut down the resource manager cleanly.

This method also needs to make sure all pending containers that are not registered yet are returned.

指定者:

internalDeregisterApplication 在类中 ResourceManager<WorkerType extends ResourceIDRetrievable>

参数:

finalStatus - The application status to report.

optionalDiagnostics - A diagnostics message or null.

抛出:

ResourceManagerException - if the application could not be shut down.

getWorkerNodeIfAcceptRegistration

protected Optional<WorkerType> getWorkerNodeIfAcceptRegistration(ResourceID resourceID)

从类复制的说明: ResourceManager

Get worker node if the worker resource is accepted.

指定者:

getWorkerNodeIfAcceptRegistration 在类中 ResourceManager<WorkerType extends ResourceIDRetrievable>

参数:

resourceID - The worker resource id

declareResourceNeeded

@VisibleForTesting
public void declareResourceNeeded(Collection<ResourceDeclaration> resourceDeclarations)

onWorkerRegistered

protected void onWorkerRegistered(WorkerType worker,
                                  WorkerResourceSpec workerResourceSpec)

覆盖:

onWorkerRegistered 在类中 ResourceManager<WorkerType extends ResourceIDRetrievable>

registerMetrics

protected void registerMetrics()

覆盖:

registerMetrics 在类中 ResourceManager<WorkerType extends ResourceIDRetrievable>

onPreviousAttemptWorkersRecovered

public void onPreviousAttemptWorkersRecovered(Collection<WorkerType> recoveredWorkers)

从接口复制的说明: ResourceEventHandler

Notifies that workers of previous attempt have been recovered from the external resource manager.

指定者:

onPreviousAttemptWorkersRecovered 在接口中 ResourceEventHandler<WorkerType extends ResourceIDRetrievable>

参数:

recoveredWorkers - Collection of worker nodes, in the deployment specific type.

onWorkerTerminated

public void onWorkerTerminated(ResourceID resourceId,
                               String diagnostics)

从接口复制的说明: ResourceEventHandler

Notifies that the worker has been terminated.

See also ResourceManagerDriver.requestResource(org.apache.flink.runtime.clusterframework.TaskExecutorProcessSpec).

指定者:

onWorkerTerminated 在接口中 ResourceEventHandler<WorkerType extends ResourceIDRetrievable>

参数:

resourceId - Identifier of the terminated worker.

diagnostics - Diagnostic message about the worker termination.

onError

public void onError(Throwable exception)

从接口复制的说明: ResourceEventHandler

Notifies that an error has occurred that the process cannot proceed.

指定者:

onError 在接口中 ResourceEventHandler<WorkerType extends ResourceIDRetrievable>

参数:

exception - Exception that describes the error.

requestNewWorker

@VisibleForTesting
public void requestNewWorker(WorkerResourceSpec workerResourceSpec)

Allocates a resource using the worker resource specification.

参数:

workerResourceSpec - workerResourceSpec specifies the size of the to be allocated resource

getReadyToServeFuture

public CompletableFuture<Void> getReadyToServeFuture()

从类复制的说明: ResourceManager

Get the ready to serve future of the resource manager.

指定者:

getReadyToServeFuture 在类中 ResourceManager<WorkerType extends ResourceIDRetrievable>

返回:

The ready to serve future of the resource manager, which indicated whether it is ready to serve.

getResourceAllocator

protected ResourceAllocator getResourceAllocator()

指定者:

getResourceAllocator 在类中 ResourceManager<WorkerType extends ResourceIDRetrievable>