org.apache.flink.runtime.io.disk.iomanager

Class IOManager

java.lang.Object

org.apache.flink.runtime.io.disk.iomanager.IOManager

Direct Known Subclasses:

IOManagerAsync

public abstract class IOManager
extends Object

The facade for the provided I/O manager services.

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.slf4j.Logger	LOG Logging

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	IOManager(String[] paths) Constructs a new IOManager.

Method Summary

Methods

Modifier and Type	Method and Description
BlockChannelReader	createBlockChannelReader(FileIOChannel.ID channelID) Creates a block channel reader that reads blocks from the given channel.
abstract BlockChannelReader	createBlockChannelReader(FileIOChannel.ID channelID, LinkedBlockingQueue<MemorySegment> returnQueue) Creates a block channel reader that reads blocks from the given channel.
BlockChannelWriter	createBlockChannelWriter(FileIOChannel.ID channelID) Creates a block channel writer that writes to the given channel.
abstract BlockChannelWriter	createBlockChannelWriter(FileIOChannel.ID channelID, LinkedBlockingQueue<MemorySegment> returnQueue) Creates a block channel writer that writes to the given channel.
abstract BlockChannelWriterWithCallback	createBlockChannelWriter(FileIOChannel.ID channelID, RequestDoneCallback callback) Creates a block channel writer that writes to the given channel.
abstract BulkBlockChannelReader	createBulkBlockChannelReader(FileIOChannel.ID channelID, List<MemorySegment> targetSegments, int numBlocks) Creates a block channel reader that reads all blocks from the given channel directly in one bulk.
FileIOChannel.ID	createChannel() Creates a new FileIOChannel.ID in one of the temp directories.
FileIOChannel.Enumerator	createChannelEnumerator() Creates a new FileIOChannel.Enumerator, spreading the channels in a round-robin fashion across the temporary file directories.
protected int	getNextPathNum()
abstract boolean	isProperlyShutDown() Utility method to check whether the IO manager has been properly shut down.
abstract void	shutdown() Close method, marks the I/O manager as closed.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

protected static final org.slf4j.Logger LOG

Logging

Constructor Detail

IOManager

protected IOManager(String[] paths)

Constructs a new IOManager.

Parameters:

paths - the basic directory paths for files underlying anonymous channels.

Method Detail

shutdown

public abstract void shutdown()

Close method, marks the I/O manager as closed.

isProperlyShutDown

public abstract boolean isProperlyShutDown()

Utility method to check whether the IO manager has been properly shut down.

Returns:

True, if the IO manager has properly shut down, false otherwise.

createChannel

public FileIOChannel.ID createChannel()

Creates a new FileIOChannel.ID in one of the temp directories. Multiple invocations of this method spread the channels evenly across the different directories.

Returns:

A channel to a temporary directory.

createChannelEnumerator

public FileIOChannel.Enumerator createChannelEnumerator()

Creates a new FileIOChannel.Enumerator, spreading the channels in a round-robin fashion across the temporary file directories.

Returns:

An enumerator for channels.

createBlockChannelWriter

public BlockChannelWriter createBlockChannelWriter(FileIOChannel.ID channelID)
                                            throws IOException

Creates a block channel writer that writes to the given channel. The writer adds the written segment to its return-queue afterwards (to allow for asynchronous implementations).

Parameters:

channelID - The descriptor for the channel to write to.

Returns:

A block channel writer that writes to the given channel.

Throws:

IOException - Thrown, if the channel for the writer could not be opened.

createBlockChannelWriter

public abstract BlockChannelWriter createBlockChannelWriter(FileIOChannel.ID channelID,
                                          LinkedBlockingQueue<MemorySegment> returnQueue)
                                                     throws IOException

Creates a block channel writer that writes to the given channel. The writer adds the written segment to the given queue (to allow for asynchronous implementations).

Parameters:

channelID - The descriptor for the channel to write to.

returnQueue - The queue to put the written buffers into.

Returns:

A block channel writer that writes to the given channel.

Throws:

IOException - Thrown, if the channel for the writer could not be opened.

createBlockChannelWriter

public abstract BlockChannelWriterWithCallback createBlockChannelWriter(FileIOChannel.ID channelID,
                                                      RequestDoneCallback callback)
                                                                 throws IOException

Creates a block channel writer that writes to the given channel. The writer calls the given callback after the I/O operation has been performed (successfully or unsuccessfully), to allow for asynchronous implementations.

Parameters:

channelID - The descriptor for the channel to write to.

callback - The callback to be called for

Returns:

A block channel writer that writes to the given channel.

Throws:

IOException - Thrown, if the channel for the writer could not be opened.

createBlockChannelReader

public BlockChannelReader createBlockChannelReader(FileIOChannel.ID channelID)
                                            throws IOException

Creates a block channel reader that reads blocks from the given channel. The reader pushed full memory segments (with the read data) to its "return queue", to allow for asynchronous read implementations.

Parameters:

channelID - The descriptor for the channel to write to.

Returns:

A block channel reader that reads from the given channel.

Throws:

IOException - Thrown, if the channel for the reader could not be opened.

createBlockChannelReader

public abstract BlockChannelReader createBlockChannelReader(FileIOChannel.ID channelID,
                                          LinkedBlockingQueue<MemorySegment> returnQueue)
                                                     throws IOException

Creates a block channel reader that reads blocks from the given channel. The reader pushes the full segments to the given queue, to allow for asynchronous implementations.

Parameters:

channelID - The descriptor for the channel to write to.

returnQueue - The queue to put the full buffers into.

Returns:

A block channel reader that reads from the given channel.

Throws:

IOException - Thrown, if the channel for the reader could not be opened.

createBulkBlockChannelReader

public abstract BulkBlockChannelReader createBulkBlockChannelReader(FileIOChannel.ID channelID,
                                                  List<MemorySegment> targetSegments,
                                                  int numBlocks)
                                                             throws IOException

Creates a block channel reader that reads all blocks from the given channel directly in one bulk. The reader draws segments to read the blocks into from a supplied list, which must contain as many segments as the channel has blocks. After the reader is done, the list with the full segments can be obtained from the reader.

If a channel is not to be read in one bulk, but in multiple smaller batches, a BlockChannelReader should be used.

Parameters:

channelID - The descriptor for the channel to write to.

targetSegments - The list to take the segments from into which to read the data.

numBlocks - The number of blocks in the channel to read.

Returns:

A block channel reader that reads from the given channel.

Throws:

IOException - Thrown, if the channel for the reader could not be opened.

getNextPathNum

protected int getNextPathNum()