public class DefaultDeclarativeSlotPool extends Object implements DeclarativeSlotPool
DeclarativeSlotPool implementation.
The implementation collects the current resource requirements and declares them at the ResourceManager. Whenever new slots are offered, the slot pool compares the offered slots to the set of available and required resources and only accepts those slots which are required.
Slots which are released won't be returned directly to their owners. Instead, the slot pool implementation will only return them after the idleSlotTimeout has been exceeded by a free slot.
The slot pool will call newSlotsListener whenever newly offered slots are accepted or
if an allocated slot should become free after it is being freed.
This class expects 1 of 2 access patterns for changing requirements, which should not be mixed:
1) the legacy approach (used by the DefaultScheduler) tightly couples requirements to reserved
slots. When a slot is requested it increases the requirements, when the slot is freed they are
decreased again. In the general case what happens is that requirements are increased, a free slot
is reserved, (the slot is used for a bit,) the slot is freed, the requirements are reduced. To
this end freeReservedSlot(org.apache.flink.runtime.clusterframework.types.AllocationID, java.lang.Throwable, long), releaseSlot(org.apache.flink.runtime.clusterframework.types.AllocationID, java.lang.Exception) and releaseSlots(org.apache.flink.runtime.clusterframework.types.ResourceID, java.lang.Exception) return a
ResourceCounter describing which requirement the slot(s) were fulfilling, with the
expectation that the scheduler will subsequently decrease the requirements by that amount.
2) The declarative approach (used by the AdaptiveScheduler) in contrast derives requirements exclusively based on what a given job currently requires. It may repeatedly reserve/free slots without any modifications to the requirements.
DeclarativeSlotPool.NewSlotsListener, DeclarativeSlotPool.NoOpNewSlotsListener| 构造器和说明 |
|---|
DefaultDeclarativeSlotPool(org.apache.flink.api.common.JobID jobId,
AllocatedSlotPool slotPool,
java.util.function.Consumer<? super Collection<ResourceRequirement>> notifyNewResourceRequirements,
org.apache.flink.api.common.time.Time idleSlotTimeout,
org.apache.flink.api.common.time.Time rpcTimeout) |
| 限定符和类型 | 方法和说明 |
|---|---|
boolean |
containsFreeSlot(AllocationID allocationId)
Checks whether the slot pool contains a slot with the given
AllocationID and if it is
free. |
boolean |
containsSlots(ResourceID owner)
Returns whether the slot pool has a slot registered which is owned by the given TaskExecutor.
|
void |
decreaseResourceRequirementsBy(ResourceCounter decrement)
Decreases the resource requirements by decrement.
|
ResourceCounter |
freeReservedSlot(AllocationID allocationId,
Throwable cause,
long currentTime)
Frees the reserved slot identified by the given allocationId.
|
Collection<? extends SlotInfo> |
getAllSlotsInformation()
Returns the slot information for all slots (free and allocated slots).
|
Collection<SlotInfoWithUtilization> |
getFreeSlotsInformation()
Returns the slot information for all free slots (slots which can be allocated from the slot
pool).
|
Collection<ResourceRequirement> |
getResourceRequirements()
Returns the current resource requirements.
|
void |
increaseResourceRequirementsBy(ResourceCounter increment)
Increases the resource requirements by increment.
|
Collection<SlotOffer> |
offerSlots(Collection<? extends SlotOffer> offers,
TaskManagerLocation taskManagerLocation,
TaskManagerGateway taskManagerGateway,
long currentTime)
Offers slots to this slot pool.
|
void |
registerNewSlotsListener(DeclarativeSlotPool.NewSlotsListener newSlotsListener)
Registers a listener which is called whenever new slots become available.
|
void |
registerSlots(Collection<? extends SlotOffer> slots,
TaskManagerLocation taskManagerLocation,
TaskManagerGateway taskManagerGateway,
long currentTime)
Registers the given set of slots at the slot pool.
|
void |
releaseIdleSlots(long currentTimeMillis)
Releases slots which have exceeded the idle slot timeout and are no longer needed to fulfill
the resource requirements.
|
ResourceCounter |
releaseSlot(AllocationID allocationId,
Exception cause)
Releases the slot specified by allocationId if one exists.
|
ResourceCounter |
releaseSlots(ResourceID owner,
Exception cause)
Releases all slots belonging to the owning TaskExecutor if it has been registered.
|
PhysicalSlot |
reserveFreeSlot(AllocationID allocationId,
ResourceProfile requiredSlotProfile)
Reserves the free slot identified by the given allocationId and maps it to the given
requiredSlotProfile.
|
void |
setResourceRequirements(ResourceCounter resourceRequirements)
Sets the resource requirements to the given resourceRequirements.
|
public DefaultDeclarativeSlotPool(org.apache.flink.api.common.JobID jobId,
AllocatedSlotPool slotPool,
java.util.function.Consumer<? super Collection<ResourceRequirement>> notifyNewResourceRequirements,
org.apache.flink.api.common.time.Time idleSlotTimeout,
org.apache.flink.api.common.time.Time rpcTimeout)
public void increaseResourceRequirementsBy(ResourceCounter increment)
DeclarativeSlotPoolincreaseResourceRequirementsBy 在接口中 DeclarativeSlotPoolincrement - increment by which to increase the resource requirementspublic void decreaseResourceRequirementsBy(ResourceCounter decrement)
DeclarativeSlotPooldecreaseResourceRequirementsBy 在接口中 DeclarativeSlotPooldecrement - decrement by which to decrease the resource requirementspublic void setResourceRequirements(ResourceCounter resourceRequirements)
DeclarativeSlotPoolsetResourceRequirements 在接口中 DeclarativeSlotPoolresourceRequirements - new resource requirementspublic Collection<ResourceRequirement> getResourceRequirements()
DeclarativeSlotPoolgetResourceRequirements 在接口中 DeclarativeSlotPoolpublic Collection<SlotOffer> offerSlots(Collection<? extends SlotOffer> offers, TaskManagerLocation taskManagerLocation, TaskManagerGateway taskManagerGateway, long currentTime)
DeclarativeSlotPoolofferSlots 在接口中 DeclarativeSlotPooloffers - offers containing the list of slots offered to this slot pooltaskManagerLocation - taskManagerLocation is the location of the offering TaskExecutortaskManagerGateway - taskManagerGateway is the gateway to talk to the offering
TaskExecutorcurrentTime - currentTime is the time the slots are being offeredpublic void registerSlots(Collection<? extends SlotOffer> slots, TaskManagerLocation taskManagerLocation, TaskManagerGateway taskManagerGateway, long currentTime)
DeclarativeSlotPoolregisterSlots 在接口中 DeclarativeSlotPoolslots - slots to registertaskManagerLocation - taskManagerLocation is the location of the offering TaskExecutortaskManagerGateway - taskManagerGateway is the gateway to talk to the offering
TaskExecutorcurrentTime - currentTime is the time the slots are being offeredpublic PhysicalSlot reserveFreeSlot(AllocationID allocationId, ResourceProfile requiredSlotProfile)
DeclarativeSlotPoolreserveFreeSlot 在接口中 DeclarativeSlotPoolallocationId - allocationId identifies the free slot to allocaterequiredSlotProfile - requiredSlotProfile specifying the resource requirementpublic ResourceCounter freeReservedSlot(AllocationID allocationId, @Nullable Throwable cause, long currentTime)
DeclarativeSlotPoolWhether the freed slot is returned to the owning TaskExecutor is implementation dependent.
freeReservedSlot 在接口中 DeclarativeSlotPoolallocationId - allocationId identifying the slot to releasecause - cause for releasing the slot; can be nullcurrentTime - currentTime when the slot was releasedpublic void registerNewSlotsListener(DeclarativeSlotPool.NewSlotsListener newSlotsListener)
DeclarativeSlotPoolregisterNewSlotsListener 在接口中 DeclarativeSlotPoolnewSlotsListener - which is called whenever new slots become availablepublic ResourceCounter releaseSlots(ResourceID owner, Exception cause)
DeclarativeSlotPoolreleaseSlots 在接口中 DeclarativeSlotPoolowner - owner identifying the owning TaskExecutorcause - cause for failing the slotspublic ResourceCounter releaseSlot(AllocationID allocationId, Exception cause)
DeclarativeSlotPoolreleaseSlot 在接口中 DeclarativeSlotPoolallocationId - allocationId identifying the slot to failcause - cause for failing the slotpublic void releaseIdleSlots(long currentTimeMillis)
DeclarativeSlotPoolreleaseIdleSlots 在接口中 DeclarativeSlotPoolcurrentTimeMillis - current timepublic Collection<SlotInfoWithUtilization> getFreeSlotsInformation()
DeclarativeSlotPoolgetFreeSlotsInformation 在接口中 DeclarativeSlotPoolpublic Collection<? extends SlotInfo> getAllSlotsInformation()
DeclarativeSlotPoolgetAllSlotsInformation 在接口中 DeclarativeSlotPoolpublic boolean containsFreeSlot(AllocationID allocationId)
DeclarativeSlotPoolAllocationID and if it is
free.containsFreeSlot 在接口中 DeclarativeSlotPoolallocationId - allocationId specifies the slot to check fortrue if the slot pool contains a free slot registered under the given
allocation id; otherwise falsepublic boolean containsSlots(ResourceID owner)
DeclarativeSlotPoolcontainsSlots 在接口中 DeclarativeSlotPoolowner - owner identifying the TaskExecutor for which to check whether the slot pool has
some slots registeredCopyright © 2014–2022 The Apache Software Foundation. All rights reserved.