| Modifier and Type | Method and Description |
|---|---|
IOManager |
RuntimeEnvironment.getIOManager() |
IOManager |
Environment.getIOManager()
Returns the current
IOManager. |
| Constructor and Description |
|---|
RuntimeEnvironment(TaskDeploymentDescriptor tdd,
MemoryManager memoryManager,
IOManager ioManager,
InputSplitProvider inputSplitProvider,
AccumulatorProtocol accumulatorProtocolProxy,
Map<String,FutureTask<Path>> cpTasks)
Constructs a runtime environment from a task deployment description.
|
| Constructor and Description |
|---|
ChannelReaderInputViewIterator(IOManager ioAccess,
Channel.ID channel,
LinkedBlockingQueue<MemorySegment> returnQueue,
List<MemorySegment> segments,
List<MemorySegment> freeMemTarget,
TypeSerializer<E> accessors,
int numBlocks) |
ChannelReaderInputViewIterator(IOManager ioAccess,
Channel.ID channel,
List<MemorySegment> segments,
List<MemorySegment> freeMemTarget,
TypeSerializer<E> accessors,
int numBlocks) |
SpillingBuffer(IOManager ioManager,
MemorySegmentSource memSource,
int segmentSize) |
| Constructor and Description |
|---|
SerializedUpdateBuffer(List<MemorySegment> memSegments,
int segmentSize,
IOManager ioManager) |
| Modifier and Type | Method and Description |
|---|---|
IOManager |
RegularPactTask.getIOManager() |
IOManager |
PactTaskContext.getIOManager() |
| Constructor and Description |
|---|
TempBarrier(AbstractInvokable owner,
MutableObjectIterator<T> input,
TypeSerializerFactory<T> serializerFactory,
MemoryManager memManager,
IOManager ioManager,
int numPages) |
| Modifier and Type | Field and Description |
|---|---|
protected IOManager |
MutableHashTable.ioManager
The I/O manager used to instantiate writers for the spilled partitions.
|
| Modifier and Type | Method and Description |
|---|---|
void |
HashPartition.finalizeBuildPhase(IOManager ioAccess,
Channel.Enumerator probeChannelEnumerator,
LinkedBlockingQueue<MemorySegment> bufferReturnQueue) |
<BT,PT> MutableHashTable<BT,PT> |
BuildSecondReOpenableHashMatchIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction) |
<BT,PT> MutableHashTable<BT,PT> |
BuildSecondHashMatchIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction) |
<BT,PT> MutableHashTable<BT,PT> |
BuildFirstReOpenableHashMatchIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction) |
<BT,PT> MutableHashTable<BT,PT> |
BuildFirstHashMatchIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction) |
void |
HashPartition.prepareProbePhase(IOManager ioAccess,
Channel.Enumerator probeChannelEnumerator,
LinkedBlockingQueue<MemorySegment> bufferReturnQueue) |
int |
HashPartition.spillPartition(List<MemorySegment> target,
IOManager ioAccess,
Channel.ID targetChannel,
LinkedBlockingQueue<MemorySegment> bufferReturnQueue)
Spills this partition to disk and sets it up such that it continues spilling records that are added to
it.
|
| Constructor and Description |
|---|
SpillingResettableIterator(Iterator<T> input,
TypeSerializer<T> serializer,
MemoryManager memoryManager,
IOManager ioManager,
int numPages,
AbstractInvokable parentTask) |
SpillingResettableIterator(Iterator<T> input,
TypeSerializer<T> serializer,
MemoryManager memoryManager,
IOManager ioManager,
List<MemorySegment> memory) |
SpillingResettableMutableObjectIterator(MutableObjectIterator<T> input,
TypeSerializer<T> serializer,
MemoryManager memoryManager,
IOManager ioManager,
int numPages,
AbstractInvokable parentTask) |
SpillingResettableMutableObjectIterator(MutableObjectIterator<T> input,
TypeSerializer<T> serializer,
MemoryManager memoryManager,
IOManager ioManager,
List<MemorySegment> memory) |
| Modifier and Type | Field and Description |
|---|---|
protected IOManager |
UnilateralSortMerger.SpillingThread.ioManager |
| Modifier and Type | Method and Description |
|---|---|
protected UnilateralSortMerger.ThreadBase<E> |
UnilateralSortMerger.getSpillingThread(ExceptionHandler<IOException> exceptionHandler,
UnilateralSortMerger.CircularQueues<E> queues,
AbstractInvokable parentTask,
MemoryManager memoryManager,
IOManager ioManager,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
List<MemorySegment> sortReadMemory,
List<MemorySegment> writeMemory,
int maxFileHandles) |
protected UnilateralSortMerger.ThreadBase<E> |
CombiningUnilateralSortMerger.getSpillingThread(ExceptionHandler<IOException> exceptionHandler,
UnilateralSortMerger.CircularQueues<E> queues,
AbstractInvokable parentTask,
MemoryManager memoryManager,
IOManager ioManager,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
List<MemorySegment> sortReadMemory,
List<MemorySegment> writeMemory,
int maxFileHandles) |
| Constructor and Description |
|---|
CombiningUnilateralSortMerger.CombiningSpillingThread(ExceptionHandler<IOException> exceptionHandler,
UnilateralSortMerger.CircularQueues<E> queues,
AbstractInvokable parentTask,
MemoryManager memManager,
IOManager ioManager,
TypeSerializer<E> serializer,
TypeComparator<E> comparator,
List<MemorySegment> sortReadMemory,
List<MemorySegment> writeMemory,
int maxNumFileHandles) |
CombiningUnilateralSortMerger(FlatCombineFunction<E> combineStub,
MemoryManager memoryManager,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
double memoryFraction,
int maxNumFileHandles,
float startSpillingFraction)
Creates a new sorter that reads the data from a given reader and provides an iterator returning that
data in a sorted manner.
|
CombiningUnilateralSortMerger(FlatCombineFunction<E> combineStub,
MemoryManager memoryManager,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
double memoryFraction,
int numSortBuffers,
int maxNumFileHandles,
float startSpillingFraction)
Creates a new sorter that reads the data from a given reader and provides an iterator returning that
data in a sorted manner.
|
MergeMatchIterator(MutableObjectIterator<T1> input1,
MutableObjectIterator<T2> input2,
TypeSerializer<T1> serializer1,
TypeComparator<T1> comparator1,
TypeSerializer<T2> serializer2,
TypeComparator<T2> comparator2,
TypePairComparator<T1,T2> pairComparator,
MemoryManager memoryManager,
IOManager ioManager,
int numMemoryPages,
AbstractInvokable parentTask) |
UnilateralSortMerger.SpillingThread(ExceptionHandler<IOException> exceptionHandler,
UnilateralSortMerger.CircularQueues<E> queues,
AbstractInvokable parentTask,
MemoryManager memManager,
IOManager ioManager,
TypeSerializer<E> serializer,
TypeComparator<E> comparator,
List<MemorySegment> sortReadMemory,
List<MemorySegment> writeMemory,
int maxNumFileHandles)
Creates the spilling thread.
|
UnilateralSortMerger(MemoryManager memoryManager,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
double memoryFraction,
int maxNumFileHandles,
float startSpillingFraction)
Creates a new sorter that reads the data from a given reader and provides an iterator returning that
data in a sorted manner.
|
UnilateralSortMerger(MemoryManager memoryManager,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
double memoryFraction,
int numSortBuffers,
int maxNumFileHandles,
float startSpillingFraction)
Creates a new sorter that reads the data from a given reader and provides an iterator returning that
data in a sorted manner.
|
UnilateralSortMerger(MemoryManager memoryManager,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
double memoryFraction,
int numSortBuffers,
int maxNumFileHandles,
float startSpillingFraction,
boolean noSpillingMemory)
Internal constructor and constructor for subclasses that want to circumvent the spilling.
|
Copyright © 2014 The Apache Software Foundation. All rights reserved.