org.apache.flink.runtime.io.network.partition.hybrid

类 HsFileDataIndexSpilledRegionManagerImpl

java.lang.Object

org.apache.flink.runtime.io.network.partition.hybrid.HsFileDataIndexSpilledRegionManagerImpl

所有已实现的接口:

AutoCloseable, HsFileDataIndexSpilledRegionManager

public class HsFileDataIndexSpilledRegionManagerImpl
extends Object
implements HsFileDataIndexSpilledRegionManager

Default implementation of HsFileDataIndexSpilledRegionManager. This manager will handle and spill regions in the following way:

• All regions will be written to the same file, namely index file.
• Multiple regions belonging to the same subpartition form a segment.
• The regions in the same segment have no special relationship, but are only related to the order in which they are spilled.
• Each segment is independent. Even if the previous segment is not full, the next segment can still be allocated.
• If a region has been written to the index file already, spill it again will overwrite the previous region.
• The very large region will monopolize a single segment.

The relationships between index file and segment are shown below.

         - - - - - - - - - Index File - - — - - - - - - - - -
        |                                                     |
        | - - — -Segment1 - - - -   - - - - Segment2- - - -   |
        ||SP1 R1｜｜SP1 R2｜ Free | |SP2 R3| SP2 R1| SP2 R2 |  |
        | - - - - - - - - - - - -   - - - - - - - - - - - -   |
        |                                                     |
        | - - - - - - - -Segment3 - - - - - - -               |
        ||              Big Region             |              |
        | - - - - - - - - - - - - - - - - - - -               |
         - - - - - - - - - - - - - - - - - - - - - -- - - - -
 

嵌套类概要

嵌套类

限定符和类型	类和说明
static class	HsFileDataIndexSpilledRegionManagerImpl.Factory Factory of HsFileDataIndexSpilledRegionManager.

构造器概要

构造器

构造器和说明
HsFileDataIndexSpilledRegionManagerImpl(int numSubpartitions, Path indexFilePath, int segmentSizeInBytes, long maxCacheCapacity, java.util.function.BiConsumer<Integer,org.apache.flink.runtime.io.network.partition.hybrid.HsFileDataIndexImpl.InternalRegion> cacheRegionConsumer)

方法概要

限定符和类型	方法和说明
void	appendOrOverwriteRegion(int subpartition, org.apache.flink.runtime.io.network.partition.hybrid.HsFileDataIndexImpl.InternalRegion newRegion) Write this region to index file.
void	close() Close this spilled region manager.
long	findRegion(int subpartition, int bufferIndex, boolean loadToCache) Find the region contains target bufferIndex and belong to target subpartition.

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

HsFileDataIndexSpilledRegionManagerImpl

public HsFileDataIndexSpilledRegionManagerImpl(int numSubpartitions,
                                               Path indexFilePath,
                                               int segmentSizeInBytes,
                                               long maxCacheCapacity,
                                               java.util.function.BiConsumer<Integer,org.apache.flink.runtime.io.network.partition.hybrid.HsFileDataIndexImpl.InternalRegion> cacheRegionConsumer)

方法详细资料

findRegion

public long findRegion(int subpartition,
                       int bufferIndex,
                       boolean loadToCache)

从接口复制的说明: HsFileDataIndexSpilledRegionManager

Find the region contains target bufferIndex and belong to target subpartition.

指定者:

findRegion 在接口中 HsFileDataIndexSpilledRegionManager

参数:

subpartition - the subpartition id that target region belong to.

bufferIndex - the buffer index that target region contains.

loadToCache - whether to load the found region into the cache.

返回:

if target region can be founded, return it's offset in index file. Otherwise, return -1.

appendOrOverwriteRegion

public void appendOrOverwriteRegion(int subpartition,
                                    org.apache.flink.runtime.io.network.partition.hybrid.HsFileDataIndexImpl.InternalRegion newRegion)
                             throws IOException

从接口复制的说明: HsFileDataIndexSpilledRegionManager

Write this region to index file. If target region already spilled, overwrite it.

指定者:

appendOrOverwriteRegion 在接口中 HsFileDataIndexSpilledRegionManager

参数:

subpartition - the subpartition id of this region.

newRegion - the region to be spilled to index file.

抛出:

IOException

close

public void close()
           throws IOException

从接口复制的说明: HsFileDataIndexSpilledRegionManager

Close this spilled region manager.

指定者:

close 在接口中 AutoCloseable

指定者:

close 在接口中 HsFileDataIndexSpilledRegionManager

抛出:

IOException