ResultPartition (Flink : Runtime 1.15.0 API)

java.lang.Object
- org.apache.flink.runtime.io.network.partition.ResultPartition

所有已实现的接口:

AutoCloseable, AvailabilityProvider, ResultPartitionWriter

直接已知子类:

BufferWritingResultPartition, SortMergeResultPartition
```
public abstract class ResultPartition
extends Object
implements ResultPartitionWriter
```
A result partition for data produced by a single task.
This class is the runtime part of a logical IntermediateResultPartition. Essentially, a result partition is a collection of Buffer instances. The buffers are organized in one or more ResultSubpartition instances or in a joint structure which further partition the data depending on the number of consuming tasks and the data DistributionPattern.
Tasks, which consume a result partition have to request one of its subpartitions. The request happens either remotely (see RemoteInputChannel) or locally (see LocalInputChannel)
Life-cycle

The life-cycle of each result partition has three (possibly overlapping) phases:
1. Produce:
2. Consume:
3. Release:
Buffer management

State management

嵌套类概要
- 从接口继承的嵌套类/接口 org.apache.flink.runtime.io.AvailabilityProvider
  AvailabilityProvider.AvailabilityHelper

字段概要

字段
限定符和类型	字段和说明
`protected BufferCompressor`	`bufferCompressor` Used to compress buffer to reduce IO.
`protected BufferPool`	`bufferPool`
`protected static org.slf4j.Logger`	`LOG`
`protected org.apache.flink.metrics.Counter`	`numBuffersOut`
`protected org.apache.flink.metrics.Counter`	`numBytesOut`
`protected org.apache.flink.metrics.Counter`	`numBytesProduced` The difference with `numBytesOut` : numBytesProduced represents the number of bytes actually produced, and numBytesOut represents the number of bytes sent to downstream tasks.
`protected int`	`numSubpartitions`
`protected ResultPartitionID`	`partitionId`
`protected ResultPartitionManager`	`partitionManager`
`protected ResultPartitionType`	`partitionType` Type of this partition.

从接口继承的字段 org.apache.flink.runtime.io.AvailabilityProvider
AVAILABLE

构造器概要

构造器
构造器和说明
`ResultPartition(String owningTaskName, int partitionIndex, ResultPartitionID partitionId, ResultPartitionType partitionType, int numSubpartitions, int numTargetKeyGroups, ResultPartitionManager partitionManager, BufferCompressor bufferCompressor, org.apache.flink.util.function.SupplierWithException<BufferPool,IOException> bufferPoolFactory)`

方法概要

所有方法实例方法抽象方法具体方法
限定符和类型	方法和说明
`protected boolean`	`canBeCompressed(Buffer buffer)` Whether the buffer can be compressed or not.
`protected void`	`checkInProduceState()`
`void`	`close()` Closes the partition writer which releases the allocated resource, for example the buffer pool.
`void`	`fail(Throwable throwable)` Fail the production of the partition.
`void`	`finish()` Finishes the result partition.
`CompletableFuture<Void>`	`getAllDataProcessedFuture()` Gets the future indicating whether all the records has been processed by the downstream tasks.
`CompletableFuture<?>`	`getAvailableFuture()`
`BufferPool`	`getBufferPool()`
`Throwable`	`getFailureCause()`
`abstract int`	`getNumberOfQueuedBuffers()` Returns the total number of queued buffers of all subpartitions.
`abstract int`	`getNumberOfQueuedBuffers(int targetSubpartition)` Returns the number of queued buffers of the given target subpartition.
`int`	`getNumberOfSubpartitions()`
`int`	`getNumTargetKeyGroups()`
`String`	`getOwningTaskName()`
`ResultPartitionID`	`getPartitionId()`
`int`	`getPartitionIndex()`
`ResultPartitionManager`	`getPartitionManager()`
`ResultPartitionType`	`getPartitionType()` Returns the type of this result partition.
`abstract long`	`getSizeOfQueuedBuffersUnsafe()` Returns the total size in bytes of queued buffers of all subpartitions.
`boolean`	`isFinished()`
`boolean`	`isReleased()` Whether this partition is released.
`void`	`notifyEndOfData(StopMode mode)` Notifies the downstream tasks that this `ResultPartitionWriter` have emitted all the user records.
`void`	`onSubpartitionAllDataProcessed(int subpartition)` The subpartition notifies that the corresponding downstream task have processed all the user records.
`void`	`release()`
`void`	`release(Throwable cause)` Releases the partition writer which releases the produced data and no reader can consume the partition any more.
`protected abstract void`	`releaseInternal()` Releases all produced data including both those stored in memory and persisted on disk.
`void`	`setMetricGroup(TaskIOMetricGroup metrics)` Sets the metric group for the `ResultPartitionWriter`.
`void`	`setup()` Registers a buffer pool with this result partition.
`String`	`toString()`

从类继承的方法 java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

从接口继承的方法 org.apache.flink.runtime.io.network.api.writer.ResultPartitionWriter
broadcastEvent, broadcastRecord, createSubpartitionView, emitRecord, flush, flushAll

从接口继承的方法 org.apache.flink.runtime.io.AvailabilityProvider
and, isApproximatelyAvailable, isAvailable, or

- 字段详细资料
  - LOG
```
protected static final org.slf4j.Logger LOG
```
  - partitionId
```
protected final ResultPartitionID partitionId
```
  - partitionType
```
protected final ResultPartitionType partitionType
```
    Type of this partition. Defines the concrete subpartition implementation to use.
  - partitionManager
```
protected final ResultPartitionManager partitionManager
```
  - numSubpartitions
```
protected final int numSubpartitions
```
  - bufferPool
```
protected BufferPool bufferPool
```
  - bufferCompressor
```
@Nullable
protected final BufferCompressor bufferCompressor
```
    Used to compress buffer to reduce IO.
  - numBytesOut
```
protected org.apache.flink.metrics.Counter numBytesOut
```
  - numBuffersOut
```
protected org.apache.flink.metrics.Counter numBuffersOut
```
  - numBytesProduced
```
protected org.apache.flink.metrics.Counter numBytesProduced
```
    The difference with numBytesOut : numBytesProduced represents the number of bytes actually produced, and numBytesOut represents the number of bytes sent to downstream tasks. In unicast scenarios, these two values should be equal. In broadcast scenarios, numBytesOut should be (N * numBytesProduced), where N refers to the number of subpartitions.
- 构造器详细资料
  - ResultPartition
```
public ResultPartition(String owningTaskName,
                       int partitionIndex,
                       ResultPartitionID partitionId,
                       ResultPartitionType partitionType,
                       int numSubpartitions,
                       int numTargetKeyGroups,
                       ResultPartitionManager partitionManager,
                       @Nullable
                       BufferCompressor bufferCompressor,
                       org.apache.flink.util.function.SupplierWithException<BufferPool,IOException> bufferPoolFactory)
```
- 方法详细资料
  - setup
```
public void setup()
           throws IOException
```
    Registers a buffer pool with this result partition.
    There is one pool for each result partition, which is shared by all its sub partitions.
    The pool is registered with the partition *after* it as been constructed in order to conform to the life-cycle of task registrations in the TaskExecutor.
    
    指定者:
    
    setup 在接口中 ResultPartitionWriter
    
    抛出:
    
    IOException
  - getOwningTaskName
```
public String getOwningTaskName()
```
  - getPartitionId
```
public ResultPartitionID getPartitionId()
```
    指定者:
    
    getPartitionId 在接口中 ResultPartitionWriter
  - getPartitionIndex
```
public int getPartitionIndex()
```
  - getNumberOfSubpartitions
```
public int getNumberOfSubpartitions()
```
    指定者:
    
    getNumberOfSubpartitions 在接口中 ResultPartitionWriter
  - getBufferPool
```
public BufferPool getBufferPool()
```
  - getNumberOfQueuedBuffers
```
public abstract int getNumberOfQueuedBuffers()
```
    Returns the total number of queued buffers of all subpartitions.
  - getSizeOfQueuedBuffersUnsafe
```
public abstract long getSizeOfQueuedBuffersUnsafe()
```
    Returns the total size in bytes of queued buffers of all subpartitions.
  - getNumberOfQueuedBuffers
```
public abstract int getNumberOfQueuedBuffers(int targetSubpartition)
```
    Returns the number of queued buffers of the given target subpartition.
  - getPartitionType
```
public ResultPartitionType getPartitionType()
```
    Returns the type of this result partition.
    
    返回:
    
    result partition type
  - notifyEndOfData
```
public void notifyEndOfData(StopMode mode)
                     throws IOException
```
    从接口复制的说明: ResultPartitionWriter
    
    Notifies the downstream tasks that this ResultPartitionWriter have emitted all the user records.
    
    指定者:
    
    notifyEndOfData 在接口中 ResultPartitionWriter
    
    参数:
    
    mode - tells if we should flush all records or not (it is false in case of stop-with-savepoint (--no-drain))
    
    抛出:
    
    IOException
  - getAllDataProcessedFuture
```
public CompletableFuture<Void> getAllDataProcessedFuture()
```
    从接口复制的说明: ResultPartitionWriter
    
    Gets the future indicating whether all the records has been processed by the downstream tasks.
    
    指定者:
    
    getAllDataProcessedFuture 在接口中 ResultPartitionWriter
  - onSubpartitionAllDataProcessed
```
public void onSubpartitionAllDataProcessed(int subpartition)
```
    The subpartition notifies that the corresponding downstream task have processed all the user records.
    
    参数:
    
    subpartition - The index of the subpartition sending the notification.
    
    另请参阅:
    
    EndOfData
  - finish
```
public void finish()
            throws IOException
```
    Finishes the result partition.
    After this operation, it is not possible to add further data to the result partition.
    For BLOCKING results, this will trigger the deployment of consuming tasks.
    
    指定者:
    
    finish 在接口中 ResultPartitionWriter
    
    抛出:
    
    IOException
  - isFinished
```
public boolean isFinished()
```
    指定者:
    
    isFinished 在接口中 ResultPartitionWriter
  - release
```
public void release()
```
  - release
```
public void release(Throwable cause)
```
    从接口复制的说明: ResultPartitionWriter
    
    Releases the partition writer which releases the produced data and no reader can consume the partition any more.
    
    指定者:
    
    release 在接口中 ResultPartitionWriter
  - releaseInternal
```
protected abstract void releaseInternal()
```
    Releases all produced data including both those stored in memory and persisted on disk.
  - close
```
public void close()
```
    从接口复制的说明: ResultPartitionWriter
    
    Closes the partition writer which releases the allocated resource, for example the buffer pool.
    
    指定者:
    
    close 在接口中 AutoCloseable
    
    指定者:
    
    close 在接口中 ResultPartitionWriter
  - fail
```
public void fail(@Nullable
                 Throwable throwable)
```
    从接口复制的说明: ResultPartitionWriter
    
    Fail the production of the partition.
    This method propagates non-null failure causes to consumers on a best-effort basis. This call also leads to the release of all resources associated with the partition. Closing of the partition is still needed afterwards if it has not been done before.
    
    指定者:
    
    fail 在接口中 ResultPartitionWriter
    
    参数:
    
    throwable - failure cause
  - getFailureCause
```
public Throwable getFailureCause()
```
  - getNumTargetKeyGroups
```
public int getNumTargetKeyGroups()
```
    指定者:
    
    getNumTargetKeyGroups 在接口中 ResultPartitionWriter
  - setMetricGroup
```
public void setMetricGroup(TaskIOMetricGroup metrics)
```
    从接口复制的说明: ResultPartitionWriter
    
    Sets the metric group for the ResultPartitionWriter.
    
    指定者:
    
    setMetricGroup 在接口中 ResultPartitionWriter
  - isReleased
```
public boolean isReleased()
```
    Whether this partition is released.
    A partition is released when each subpartition is either consumed and communication is closed by consumer or failed. A partition is also released if task is cancelled.
    
    指定者:
    
    isReleased 在接口中 ResultPartitionWriter
  - getAvailableFuture
```
public CompletableFuture<?> getAvailableFuture()
```
    指定者:
    
    getAvailableFuture 在接口中 AvailabilityProvider
    
    返回:
    
    a future that is completed if the respective provider is available.
  - toString
```
public String toString()
```
    覆盖:
    
    toString 在类中 Object
  - checkInProduceState
```
protected void checkInProduceState()
                            throws IllegalStateException
```
    抛出:
    
    IllegalStateException
  - getPartitionManager
```
@VisibleForTesting
public ResultPartitionManager getPartitionManager()
```
  - canBeCompressed
```
protected boolean canBeCompressed(Buffer buffer)
```
    Whether the buffer can be compressed or not. Note that event is not compressed because it is usually small and the size can become even larger after compression.

类 ResultPartition

Life-cycle

Buffer management

State management

嵌套类概要

从接口继承的嵌套类/接口 org.apache.flink.runtime.io.AvailabilityProvider

字段概要

从接口继承的字段 org.apache.flink.runtime.io.AvailabilityProvider

构造器概要

方法概要

从类继承的方法 java.lang.Object

从接口继承的方法 org.apache.flink.runtime.io.network.api.writer.ResultPartitionWriter

从接口继承的方法 org.apache.flink.runtime.io.AvailabilityProvider

字段详细资料

LOG

partitionId

partitionType

partitionManager

numSubpartitions

bufferPool

bufferCompressor

numBytesOut

numBuffersOut

numBytesProduced

构造器详细资料

ResultPartition

方法详细资料

setup

getOwningTaskName

getPartitionId

getPartitionIndex

getNumberOfSubpartitions

getBufferPool

getNumberOfQueuedBuffers

getSizeOfQueuedBuffersUnsafe

getNumberOfQueuedBuffers

getPartitionType

notifyEndOfData

getAllDataProcessedFuture

onSubpartitionAllDataProcessed

finish

isFinished

release

release

releaseInternal

close

fail

getFailureCause

getNumTargetKeyGroups

setMetricGroup

isReleased

getAvailableFuture

toString

checkInProduceState

getPartitionManager

canBeCompressed