| Modifier and Type | Method and Description |
|---|---|
AbstractInvokable |
RuntimeEnvironment.getInvokable()
Returns the invokable object that represents the Nephele task.
|
| Constructor and Description |
|---|
AbstractSingleGateRecordReader(AbstractInvokable invokable) |
BufferWriter(AbstractInvokable invokable) |
MutableRecordReader(AbstractInvokable taskBase)
Constructs a new mutable record reader and registers a new input gate with the application's environment.
|
RecordReader(AbstractInvokable taskBase,
Class<T> recordType)
Constructs a new record reader and registers a new input gate with the application's environment.
|
RecordWriter(AbstractInvokable invokable) |
RecordWriter(AbstractInvokable invokable,
ChannelSelector<T> channelSelector) |
| Modifier and Type | Class and Description |
|---|---|
class |
AbstractIterativePactTask<S extends Function,OT>
The base class for all tasks able to participate in an iteration.
|
class |
IterationHeadPactTask<X,Y,S extends Function,OT>
The head is responsible for coordinating an iteration and can run a
PactDriver inside. |
class |
IterationIntermediatePactTask<S extends Function,OT>
An intermediate iteration task, which runs a Driver}inside.
|
class |
IterationSynchronizationSinkTask
The task responsible for synchronizing all iteration heads, implemented as an output task.
|
class |
IterationTailPactTask<S extends Function,OT>
An iteration tail, which runs a driver inside.
|
| Modifier and Type | Method and Description |
|---|---|
Class<? extends AbstractInvokable> |
AbstractJobVertex.getInvokableClass(ClassLoader cl)
Returns the invokable class which represents the task of this vertex
|
| Modifier and Type | Method and Description |
|---|---|
void |
AbstractJobVertex.setInvokableClass(Class<? extends AbstractInvokable> invokable) |
| Modifier and Type | Method and Description |
|---|---|
List<MemorySegment> |
DefaultMemoryManager.allocatePages(AbstractInvokable owner,
int numPages) |
List<MemorySegment> |
MemoryManager.allocatePages(AbstractInvokable owner,
int numPages) |
void |
DefaultMemoryManager.allocatePages(AbstractInvokable owner,
List<MemorySegment> target,
int numPages) |
void |
MemoryManager.allocatePages(AbstractInvokable owner,
List<MemorySegment> target,
int numPages) |
void |
DefaultMemoryManager.releaseAll(AbstractInvokable owner) |
void |
MemoryManager.releaseAll(AbstractInvokable task)
Releases all memory segments for the given task.
|
| Modifier and Type | Class and Description |
|---|---|
class |
DataSinkTask<IT>
DataSinkTask which is executed by a task manager.
|
class |
DataSourceTask<OT>
DataSourceTask which is executed by a task manager.
|
class |
RegularPactTask<S extends Function,OT>
The abstract base class for all tasks.
|
| Modifier and Type | Method and Description |
|---|---|
AbstractInvokable |
PactTaskContext.getOwningNepheleTask() |
AbstractInvokable |
RegularPactTask.getOwningNepheleTask() |
| Modifier and Type | Method and Description |
|---|---|
static void |
RegularPactTask.closeChainedTasks(List<ChainedDriver<?,?>> tasks,
AbstractInvokable parent)
Closes all chained tasks, in the order as they are stored in the array.
|
static String |
RegularPactTask.constructLogString(String message,
String taskName,
AbstractInvokable parent)
Utility function that composes a string for logging purposes.
|
static <T> Collector<T> |
RegularPactTask.getOutputCollector(AbstractInvokable task,
TaskConfig config,
ClassLoader cl,
List<BufferWriter> eventualOutputs,
int numOutputs)
Creates the
Collector for the given task, as described by the given configuration. |
static <T> Collector<T> |
RegularPactTask.initOutputs(AbstractInvokable nepheleTask,
ClassLoader cl,
TaskConfig config,
List<ChainedDriver<?,?>> chainedTasksTarget,
List<BufferWriter> eventualOutputs)
Creates a writer for each output.
|
static void |
RegularPactTask.logAndThrowException(Exception ex,
AbstractInvokable parent)
Prints an error message and throws the given exception.
|
static void |
RegularPactTask.openChainedTasks(List<ChainedDriver<?,?>> tasks,
AbstractInvokable parent)
Opens all chained tasks, in the order as they are stored in the array.
|
| Constructor and Description |
|---|
TempBarrier(AbstractInvokable owner,
MutableObjectIterator<T> input,
TypeSerializerFactory<T> serializerFactory,
MemoryManager memManager,
IOManager ioManager,
int numPages) |
| Modifier and Type | Method and Description |
|---|---|
void |
ChainedTerminationCriterionDriver.setup(AbstractInvokable parent) |
void |
ChainedCollectorMapDriver.setup(AbstractInvokable parent) |
abstract void |
ChainedDriver.setup(AbstractInvokable parent) |
void |
ChainedFlatMapDriver.setup(AbstractInvokable parent) |
void |
SynchronousChainedCombineDriver.setup(AbstractInvokable parent) |
void |
ChainedMapDriver.setup(AbstractInvokable parent) |
void |
ChainedDriver.setup(TaskConfig config,
String taskName,
Collector<OT> outputCollector,
AbstractInvokable parent,
ClassLoader userCodeClassLoader) |
| Modifier and Type | Method and Description |
|---|---|
<BT,PT> MutableHashTable<BT,PT> |
BuildFirstHashMatchIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction) |
<BT,PT> MutableHashTable<BT,PT> |
BuildSecondHashMatchIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction) |
<BT,PT> MutableHashTable<BT,PT> |
BuildSecondReOpenableHashMatchIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction) |
<BT,PT> MutableHashTable<BT,PT> |
BuildFirstReOpenableHashMatchIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction) |
| Constructor and Description |
|---|
BlockResettableIterator(MemoryManager memoryManager,
Iterator<T> input,
TypeSerializer<T> serializer,
int numPages,
AbstractInvokable ownerTask) |
BlockResettableIterator(MemoryManager memoryManager,
TypeSerializer<T> serializer,
int numPages,
AbstractInvokable ownerTask) |
BlockResettableMutableObjectIterator(MemoryManager memoryManager,
MutableObjectIterator<T> input,
TypeSerializer<T> serializer,
int numMemoryPages,
AbstractInvokable ownerTask) |
SpillingResettableIterator(Iterator<T> input,
TypeSerializer<T> serializer,
MemoryManager memoryManager,
IOManager ioManager,
int numPages,
AbstractInvokable parentTask) |
SpillingResettableMutableObjectIterator(MutableObjectIterator<T> input,
TypeSerializer<T> serializer,
MemoryManager memoryManager,
IOManager ioManager,
int numPages,
AbstractInvokable parentTask) |
| Constructor and Description |
|---|
CombiningSpillingThread(ExceptionHandler<IOException> exceptionHandler,
UnilateralSortMerger.CircularQueues<E> queues,
AbstractInvokable parentTask,
MemoryManager memManager,
IOManager ioManager,
TypeSerializer<E> serializer,
TypeComparator<E> comparator,
List<MemorySegment> sortReadMemory,
List<MemorySegment> writeMemory,
int maxNumFileHandles) |
CombiningUnilateralSortMerger(FlatCombineFunction<E> combineStub,
MemoryManager memoryManager,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
double memoryFraction,
int maxNumFileHandles,
float startSpillingFraction)
Creates a new sorter that reads the data from a given reader and provides an iterator returning that
data in a sorted manner.
|
CombiningUnilateralSortMerger(FlatCombineFunction<E> combineStub,
MemoryManager memoryManager,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
double memoryFraction,
int numSortBuffers,
int maxNumFileHandles,
float startSpillingFraction)
Creates a new sorter that reads the data from a given reader and provides an iterator returning that
data in a sorted manner.
|
MergeMatchIterator(MutableObjectIterator<T1> input1,
MutableObjectIterator<T2> input2,
TypeSerializer<T1> serializer1,
TypeComparator<T1> comparator1,
TypeSerializer<T2> serializer2,
TypeComparator<T2> comparator2,
TypePairComparator<T1,T2> pairComparator,
MemoryManager memoryManager,
IOManager ioManager,
int numMemoryPages,
AbstractInvokable parentTask) |
ReadingThread(ExceptionHandler<IOException> exceptionHandler,
MutableObjectIterator<E> reader,
UnilateralSortMerger.CircularQueues<E> queues,
E readTarget,
AbstractInvokable parentTask,
long startSpillingBytes)
Creates a new reading thread.
|
SortingThread(ExceptionHandler<IOException> exceptionHandler,
UnilateralSortMerger.CircularQueues<E> queues,
AbstractInvokable parentTask)
Creates a new sorting thread.
|
SpillingThread(ExceptionHandler<IOException> exceptionHandler,
UnilateralSortMerger.CircularQueues<E> queues,
AbstractInvokable parentTask,
MemoryManager memManager,
IOManager ioManager,
TypeSerializer<E> serializer,
TypeComparator<E> comparator,
List<MemorySegment> sortReadMemory,
List<MemorySegment> writeMemory,
int maxNumFileHandles)
Creates the spilling thread.
|
ThreadBase(ExceptionHandler<IOException> exceptionHandler,
String name,
UnilateralSortMerger.CircularQueues<E> queues,
AbstractInvokable parentTask)
Creates a new thread.
|
UnilateralSortMerger(MemoryManager memoryManager,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
double memoryFraction,
int maxNumFileHandles,
float startSpillingFraction)
Creates a new sorter that reads the data from a given reader and provides an iterator returning that
data in a sorted manner.
|
UnilateralSortMerger(MemoryManager memoryManager,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
double memoryFraction,
int numSortBuffers,
int maxNumFileHandles,
float startSpillingFraction)
Creates a new sorter that reads the data from a given reader and provides an iterator returning that
data in a sorted manner.
|
UnilateralSortMerger(MemoryManager memoryManager,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
double memoryFraction,
int numSortBuffers,
int maxNumFileHandles,
float startSpillingFraction,
boolean noSpillingMemory)
Internal constructor and constructor for subclasses that want to circumvent the spilling.
|
Copyright © 2015 The Apache Software Foundation. All rights reserved.