org.apache.flink.runtime.operators.hash

类 CompactingHashTable<T>

java.lang.Object

org.apache.flink.runtime.operators.hash.AbstractMutableHashTable<T>

org.apache.flink.runtime.operators.hash.CompactingHashTable<T>

类型参数:

T - Record type stored in hash table

public class CompactingHashTable<T>
extends AbstractMutableHashTable<T>

A hash table that uses Flink's managed memory and supports replacement of records or updates to records. For an overview of the general data structure of the hash table, please refer to the description of the MutableHashTable.

The hash table is internally divided into two parts: The hash index, and the partition buffers that store the actual records. When records are inserted or updated, the hash table appends the records to its corresponding partition, and inserts or updates the entry in the hash index. In the case that the hash table runs out of memory, it compacts a partition by walking through the hash index and copying all reachable elements into a fresh partition. After that, it releases the memory of the partition to compact.

嵌套类概要

嵌套类

限定符和类型	类和说明
class	CompactingHashTable.EntryIterator Iterator that traverses the whole hash table once If entries are inserted during iteration they may be overlooked by the iterator
class	CompactingHashTable.HashTableProber<PT>

字段概要

从类继承的字段 org.apache.flink.runtime.operators.hash.AbstractMutableHashTable

buildSideComparator, buildSideSerializer, closed, stateLock

构造器概要

构造器

构造器和说明
CompactingHashTable(org.apache.flink.api.common.typeutils.TypeSerializer<T> buildSideSerializer, org.apache.flink.api.common.typeutils.TypeComparator<T> buildSideComparator, List<org.apache.flink.core.memory.MemorySegment> memorySegments)
CompactingHashTable(org.apache.flink.api.common.typeutils.TypeSerializer<T> buildSideSerializer, org.apache.flink.api.common.typeutils.TypeComparator<T> buildSideComparator, List<org.apache.flink.core.memory.MemorySegment> memorySegments, int avgRecordLen)

方法概要

限定符和类型	方法和说明
void	abort()
void	buildTableWithUniqueKey(org.apache.flink.util.MutableObjectIterator<T> input)
void	close() Closes the hash table.
org.apache.flink.util.MutableObjectIterator<T>	getEntryIterator()
List<org.apache.flink.core.memory.MemorySegment>	getFreeMemory()
<PT> CompactingHashTable.HashTableProber<PT>	getProber(org.apache.flink.api.common.typeutils.TypeComparator<PT> probeSideComparator, org.apache.flink.api.common.typeutils.TypePairComparator<PT,T> pairComparator)
void	insert(T record)
void	insertOrReplaceRecord(T record) Replaces record in hash table if record already present or append record if not.
void	open() Initialize the hash table

从类继承的方法 org.apache.flink.runtime.operators.hash.AbstractMutableHashTable

getBuildSideComparator, getBuildSideSerializer

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

CompactingHashTable

public CompactingHashTable(org.apache.flink.api.common.typeutils.TypeSerializer<T> buildSideSerializer,
                           org.apache.flink.api.common.typeutils.TypeComparator<T> buildSideComparator,
                           List<org.apache.flink.core.memory.MemorySegment> memorySegments)

CompactingHashTable

public CompactingHashTable(org.apache.flink.api.common.typeutils.TypeSerializer<T> buildSideSerializer,
                           org.apache.flink.api.common.typeutils.TypeComparator<T> buildSideComparator,
                           List<org.apache.flink.core.memory.MemorySegment> memorySegments,
                           int avgRecordLen)

方法详细资料

open

public void open()

Initialize the hash table

指定者:

open 在类中 AbstractMutableHashTable<T>

close

public void close()

Closes the hash table. This effectively releases all internal structures and closes all open files and removes them. The call to this method is valid both as a cleanup after the complete inputs were properly processed, and as an cancellation call, which cleans up all resources that are currently held by the hash join. If another process still access the hash table after close has been called no operations will be performed.

指定者:

close 在类中 AbstractMutableHashTable<T>

abort

public void abort()

指定者:

abort 在类中 AbstractMutableHashTable<T>

getFreeMemory

public List<org.apache.flink.core.memory.MemorySegment> getFreeMemory()

指定者:

getFreeMemory 在类中 AbstractMutableHashTable<T>

buildTableWithUniqueKey

public void buildTableWithUniqueKey(org.apache.flink.util.MutableObjectIterator<T> input)
                             throws IOException

抛出:

IOException

insert

public final void insert(T record)
                  throws IOException

指定者:

insert 在类中 AbstractMutableHashTable<T>

抛出:

IOException

insertOrReplaceRecord

public void insertOrReplaceRecord(T record)
                           throws IOException

Replaces record in hash table if record already present or append record if not. May trigger expensive compaction.

指定者:

insertOrReplaceRecord 在类中 AbstractMutableHashTable<T>

参数:

record - record to insert or replace

抛出:

IOException

getProber

public <PT> CompactingHashTable.HashTableProber<PT> getProber(org.apache.flink.api.common.typeutils.TypeComparator<PT> probeSideComparator,
                                                              org.apache.flink.api.common.typeutils.TypePairComparator<PT,T> pairComparator)

指定者:

getProber 在类中 AbstractMutableHashTable<T>

getEntryIterator

public org.apache.flink.util.MutableObjectIterator<T> getEntryIterator()

指定者:

getEntryIterator 在类中 AbstractMutableHashTable<T>

返回:

Iterator over hash table

另请参阅:

CompactingHashTable.EntryIterator