org.apache.flink.runtime.io.disk.iomanager

Class BlockChannelWriter

java.lang.Object

org.apache.flink.runtime.io.disk.iomanager.ChannelAccess<MemorySegment,R>

org.apache.flink.runtime.io.disk.iomanager.BlockChannelAccess<org.apache.flink.runtime.io.disk.iomanager.WriteRequest,LinkedBlockingQueue<MemorySegment>>

org.apache.flink.runtime.io.disk.iomanager.BlockChannelWriter

public class BlockChannelWriter
extends BlockChannelAccess<org.apache.flink.runtime.io.disk.iomanager.WriteRequest,LinkedBlockingQueue<MemorySegment>>

A writer that writes data in blocks to a file channel. The writer receives the data blocks in the form of MemorySegment, which it writes entirely to the channel, regardless of how space in the segment is used. The writing happens in an asynchronous fashion. That is, a write request is not processed by the thread that issues it, but by an asynchronous writer thread. Once the request is done, the asynchronous writer adds the MemorySegment to a return queue where it can be popped by the worker thread, to be reused. The return queue is in this case a LinkedBlockingQueue, such that the working thread blocks until the request has been served, if the request is still pending when the it requires the segment back.

Typical write behind is realized, by having a small set of segments in the return queue at all times. When a memory segment must be written, the request is issued to the writer and a new segment is immediately popped from the return queue. Once too many requests have been issued and the I/O thread cannot keep up, the working thread naturally blocks until another segment is available again.

Field Summary

Fields inherited from class org.apache.flink.runtime.io.disk.iomanager.BlockChannelAccess

closed, closeLock, requestsNotReturned, returnBuffers

Fields inherited from class org.apache.flink.runtime.io.disk.iomanager.ChannelAccess

exception, fileChannel, id, requestQueue

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	BlockChannelWriter(Channel.ID channelID, RequestQueue<org.apache.flink.runtime.io.disk.iomanager.WriteRequest> requestQueue, LinkedBlockingQueue<MemorySegment> returnSegments, int numRequestsToBundle) Creates a new block channel writer for the given channel.

Method Summary

Methods

Modifier and Type	Method and Description
MemorySegment	getNextReturnedSegment() Gets the next memory segment that has been written and is available again.
void	writeBlock(MemorySegment segment) Issues a asynchronous write request to the writer.

Methods inherited from class org.apache.flink.runtime.io.disk.iomanager.BlockChannelAccess

close, closeAndDelete, getReturnQueue, isClosed, returnBuffer

Methods inherited from class org.apache.flink.runtime.io.disk.iomanager.ChannelAccess

checkErroneous, deleteChannel, getChannelID

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BlockChannelWriter

protected BlockChannelWriter(Channel.ID channelID,
                  RequestQueue<org.apache.flink.runtime.io.disk.iomanager.WriteRequest> requestQueue,
                  LinkedBlockingQueue<MemorySegment> returnSegments,
                  int numRequestsToBundle)
                      throws IOException

Creates a new block channel writer for the given channel.

Parameters:

channelID - The ID of the channel to write to.

requestQueue - The request queue of the asynchronous writer thread, to which the I/O requests are added.

returnSegments - The return queue, to which the processed Memory Segments are added.

Throws:

IOException - Thrown, if the underlying file channel could not be opened exclusively.

Method Detail

writeBlock

public void writeBlock(MemorySegment segment)
                throws IOException

Issues a asynchronous write request to the writer.

Parameters:

segment - The segment to be written.

Throws:

IOException - Thrown, when the writer encounters an I/O error. Due to the asynchronous nature of the writer, the exception thrown here may have been caused by an earlier write request.

getNextReturnedSegment

public MemorySegment getNextReturnedSegment()
                                     throws IOException

Gets the next memory segment that has been written and is available again. This method blocks until such a segment is available, or until an error occurs in the writer, or the writer is closed.

WARNING: If this method is invoked without any segment ever returning (for example, because the writeBlock(MemorySegment) method has not been invoked appropriately), the method may block forever.

Returns:

The next memory segment from the writers's return queue.

Throws:

IOException - Thrown, if an I/O error occurs in the writer while waiting for the request to return.