org.apache.flink.runtime.io.network.gates

Class InputGate<T extends IOReadableWritable>

java.lang.Object

org.apache.flink.runtime.io.network.gates.Gate<T>

org.apache.flink.runtime.io.network.gates.InputGate<T>

Type Parameters:

T - The type of record that can be transported through this gate.

All Implemented Interfaces:

BufferProvider, LocalBufferPoolOwner

public class InputGate<T extends IOReadableWritable>
extends Gate<T>
implements BufferProvider, LocalBufferPoolOwner

Input gates are a specialization of general gates and connect input channels and record readers. As channels, input gates are always parameterized to a specific type of record which they can transport. In contrast to output gates input gates can be associated with a DistributionPattern object which dictates the concrete wiring between two groups of vertices.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.runtime.io.network.bufferprovider.BufferProvider

BufferProvider.BufferAvailabilityRegistration

Constructor Summary

Constructors

Constructor and Description
InputGate(JobID jobID, GateID gateID, int index) Constructs a new runtime input gate.

Method Summary

Methods

Modifier and Type	Method and Description
InputChannel<T>[]	channels()
void	clearLocalBufferPool() Clears the local buffer pool and returns all buffers to the global buffer pool.
void	close() Immediately closes the input gate and all its input channels.
int	getBufferSize() Returns the size of buffers (in bytes) available at this buffer provider.
AbstractTaskEvent	getCurrentEvent()
InputChannel<T>	getInputChannel(int pos) Returns the input channel from position pos of the gate's internal channel list.
int	getNumberOfChannels() Returns the number of byte-buffered channels that will retrieve their buffers from the local buffer pool.
int	getNumberOfInputChannels() Returns the number of input channels associated with this input gate.
void	initializeChannels(GateDeploymentDescriptor inputGateDescriptor)
boolean	isClosed() Checks if the gate is closed.
boolean	isInputGate() Checks if the considered gate is an input gate.
void	logBufferUtilization() Logs the current status of the local buffer pool.
void	notifyDataUnitConsumed(int channelIndex) Notify the gate that is has consumed a data unit from the channel with the given index
void	notifyRecordIsAvailable(int channelIndex) Notify the gate that the channel with the given index has at least one record available.
void	publishEvent(AbstractEvent event) Publishes an event.
InputChannelResult	readRecord(T target) Reads a record from one of the associated input channels.
BufferProvider.BufferAvailabilityRegistration	registerBufferAvailabilityListener(BufferAvailabilityListener listener) Registers the given BufferAvailabilityListener with an empty buffer pool.
void	registerGlobalBufferPool(GlobalBufferPool globalBufferPool)
void	registerRecordAvailabilityListener(RecordAvailabilityListener<T> listener) Registers a RecordAvailabilityListener with this input gate.
void	releaseAllChannelResources() Releases the allocated resources (particularly buffer) of all channels attached to this gate.
void	reportAsynchronousEvent() Reports an asynchronous event and interrupts each blocking method of this buffer provider in order to allow the blocked thread to respond to the event.
Buffer	requestBuffer(int minBufferSize) Requests a buffer with a minimum size of minBufferSize.
Buffer	requestBufferBlocking(int minBufferSize) Requests a buffer with a minimum size of minBufferSize.
void	setDesignatedNumberOfBuffers(int numBuffers) Sets the designated number of buffers the local buffer pool owner is allowed to fetch from the global buffer pool and manage locally by means of the LocalBufferPool.
String	toString()
int	waitForAnyChannelToBecomeAvailable() This method returns the index of a channel which has at least one record available.

Methods inherited from class org.apache.flink.runtime.io.network.gates.Gate

deliverEvent, getChannelType, getEventNotificationManager, getGateID, getIndex, getJobID, setChannelType, subscribeToEvent, unsubscribeFromEvent

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

InputGate

public InputGate(JobID jobID,
         GateID gateID,
         int index)

Constructs a new runtime input gate.

Parameters:

jobID - the ID of the job this input gate belongs to

gateID - the ID of the gate

index - the index assigned to this input gate at the Environment object

Method Detail

initializeChannels

public void initializeChannels(GateDeploymentDescriptor inputGateDescriptor)

isInputGate

public boolean isInputGate()

Description copied from class: Gate

Checks if the considered gate is an input gate.

Specified by:

isInputGate in class Gate<T extends IOReadableWritable>

Returns:

true if the considered gate is an input gate, false if it is an output gate

getNumberOfInputChannels

public int getNumberOfInputChannels()

Returns the number of input channels associated with this input gate.

Returns:

the number of input channels associated with this input gate

getInputChannel

public InputChannel<T> getInputChannel(int pos)

Returns the input channel from position pos of the gate's internal channel list.

Parameters:

pos - the position to retrieve the channel from

Returns:

the channel from the given position or null if such position does not exist.

channels

public InputChannel<T>[] channels()

readRecord

public InputChannelResult readRecord(T target)
                              throws IOException,
                                     InterruptedException

Reads a record from one of the associated input channels. Channels are read such that one buffer from a channel is consecutively consumed. The buffers in turn are consumed in the order in which they arrive. Note that this method is not guaranteed to return a record, because the currently available channel data may not always constitute an entire record, when events or partial records are part of the data. When called even though no data is available, this call will block until data is available, so this method should be called when waiting is desired (such as when synchronously consuming a single gate) or only when it is known that data is available (such as when reading a union of multiple input gates).

Parameters:

target - The record object into which to construct the complete record.

Returns:

The result indicating whether a complete record is available, a event is available, only incomplete data is available (NONE), or the gate is exhausted.

Throws:

IOException - Thrown when an error occurred in the network stack relating to this channel.

InterruptedException - Thrown, when the thread working on this channel is interrupted.

getCurrentEvent

public AbstractTaskEvent getCurrentEvent()

notifyRecordIsAvailable

public void notifyRecordIsAvailable(int channelIndex)

Notify the gate that the channel with the given index has at least one record available.

Parameters:

channelIndex - the index of the channel which has at least one record available

waitForAnyChannelToBecomeAvailable

public int waitForAnyChannelToBecomeAvailable()
                                       throws InterruptedException

This method returns the index of a channel which has at least one record available. The method may block until at least one channel has become ready.

Returns:

the index of the channel which has at least one record available

Throws:

InterruptedException

isClosed

public boolean isClosed()
                 throws IOException,
                        InterruptedException

Description copied from class: Gate

Checks if the gate is closed. The gate is closed if all this associated channels are closed.

Specified by:

isClosed in class Gate<T extends IOReadableWritable>

Returns:

true if the gate is closed, false otherwise

Throws:

IOException - thrown if any error occurred while closing the gate

InterruptedException - thrown if the gate is interrupted while waiting for this operation to complete

close

public void close()
           throws IOException,
                  InterruptedException

Immediately closes the input gate and all its input channels. The corresponding output channels are notified. Any remaining records in any buffers or queue is considered irrelevant and is discarded.

Throws:

IOException - thrown if an I/O error occurs while closing the gate

InterruptedException - thrown if the thread is interrupted while waiting for the gate to be closed

toString

public String toString()

Overrides:

toString in class Gate<T extends IOReadableWritable>

publishEvent

public void publishEvent(AbstractEvent event)
                  throws IOException,
                         InterruptedException

Description copied from class: Gate

Publishes an event.

Specified by:

publishEvent in class Gate<T extends IOReadableWritable>

Parameters:

event - the event to be published

Throws:

IOException - thrown if an error occurs while transmitting the event

InterruptedException - thrown if the thread is interrupted while waiting for the event to be published

releaseAllChannelResources

public void releaseAllChannelResources()

Description copied from class: Gate

Releases the allocated resources (particularly buffer) of all channels attached to this gate. This method should only be called after the respected task has stopped running.

Specified by:

releaseAllChannelResources in class Gate<T extends IOReadableWritable>

registerRecordAvailabilityListener

public void registerRecordAvailabilityListener(RecordAvailabilityListener<T> listener)

Registers a RecordAvailabilityListener with this input gate.

Parameters:

listener - the listener object to be registered

notifyDataUnitConsumed

public void notifyDataUnitConsumed(int channelIndex)

Notify the gate that is has consumed a data unit from the channel with the given index

Parameters:

channelIndex - the index of the channel from which a data unit has been consumed

requestBuffer

public Buffer requestBuffer(int minBufferSize)
                     throws IOException

Description copied from interface: BufferProvider

Requests a buffer with a minimum size of minBufferSize. The method returns immediately, even if the request could not be fulfilled.

Specified by:

requestBuffer in interface BufferProvider

Parameters:

minBufferSize - minimum size of the requested buffer (in bytes)

Returns:

buffer with at least the requested size or null if no such buffer is currently available

Throws:

IOException

requestBufferBlocking

public Buffer requestBufferBlocking(int minBufferSize)
                             throws IOException,
                                    InterruptedException

Description copied from interface: BufferProvider

Requests a buffer with a minimum size of minBufferSize. The method blocks until the request has been fulfilled or BufferProvider.reportAsynchronousEvent() has been called.

Specified by:

requestBufferBlocking in interface BufferProvider

Parameters:

minBufferSize - minimum size of the requested buffer (in bytes)

Returns:

buffer with at least the requested size

Throws:

IOException

InterruptedException

getBufferSize

public int getBufferSize()

Description copied from interface: BufferProvider

Returns the size of buffers (in bytes) available at this buffer provider.

Specified by:

getBufferSize in interface BufferProvider

Returns:

size of buffers (in bytes) available at this buffer provider

getNumberOfChannels

public int getNumberOfChannels()

Description copied from interface: LocalBufferPoolOwner

Returns the number of byte-buffered channels that will retrieve their buffers from the local buffer pool.

Specified by:

getNumberOfChannels in interface LocalBufferPoolOwner

Returns:

the number of byte-buffered channels that will retrieve their buffers from the local buffer pool

setDesignatedNumberOfBuffers

public void setDesignatedNumberOfBuffers(int numBuffers)

Description copied from interface: LocalBufferPoolOwner

Sets the designated number of buffers the local buffer pool owner is allowed to fetch from the global buffer pool and manage locally by means of the LocalBufferPool.

Specified by:

setDesignatedNumberOfBuffers in interface LocalBufferPoolOwner

Parameters:

numBuffers - the numBuffers the local buffer pool owner is allowed to fetch from the global buffer pool

clearLocalBufferPool

public void clearLocalBufferPool()

Description copied from interface: LocalBufferPoolOwner

Clears the local buffer pool and returns all buffers to the global buffer pool.

Specified by:

clearLocalBufferPool in interface LocalBufferPoolOwner

registerGlobalBufferPool

public void registerGlobalBufferPool(GlobalBufferPool globalBufferPool)

Specified by:

registerGlobalBufferPool in interface LocalBufferPoolOwner

logBufferUtilization

public void logBufferUtilization()

Description copied from interface: LocalBufferPoolOwner

Logs the current status of the local buffer pool. This method is intended mainly for debugging purposes.

Specified by:

logBufferUtilization in interface LocalBufferPoolOwner

reportAsynchronousEvent

public void reportAsynchronousEvent()

Description copied from interface: BufferProvider

Reports an asynchronous event and interrupts each blocking method of this buffer provider in order to allow the blocked thread to respond to the event.

Specified by:

reportAsynchronousEvent in interface BufferProvider

Specified by:

reportAsynchronousEvent in interface LocalBufferPoolOwner

registerBufferAvailabilityListener

public BufferProvider.BufferAvailabilityRegistration registerBufferAvailabilityListener(BufferAvailabilityListener listener)

Description copied from interface: BufferProvider

Registers the given BufferAvailabilityListener with an empty buffer pool.

The registration only succeeds, if the buffer pool is empty and has not been destroyed yet.

The registered listener will receive a notification when at least one buffer has become available again. After the notification, the listener will be unregistered.

Specified by:

registerBufferAvailabilityListener in interface BufferProvider

Parameters:

listener - the listener to be registered

Returns:

true if the registration has been successful; false if the registration failed, because the buffer pool was not empty or has already been destroyed