org.apache.flink.formats.parquet.utils

Class ParquetRecordReader<T>

java.lang.Object

org.apache.flink.formats.parquet.utils.ParquetRecordReader<T>

public class ParquetRecordReader<T>
extends Object

Customized ParquetRecordReader that support start read from particular position.

Constructor Summary

Constructors

Constructor and Description
ParquetRecordReader(org.apache.parquet.hadoop.api.ReadSupport<T> readSupport, org.apache.parquet.schema.MessageType readSchema)
ParquetRecordReader(org.apache.parquet.hadoop.api.ReadSupport<T> readSupport, org.apache.parquet.schema.MessageType readSchema, org.apache.parquet.filter2.compat.FilterCompat.Filter filter)

Method Summary

Modifier and Type	Method and Description
void	close()
org.apache.flink.api.java.tuple.Tuple2<Long,Long>	getCurrentReadPosition() Returns the current read position in the split, i.e., the current block and the number of records that were returned from that block.
void	initialize(org.apache.parquet.hadoop.ParquetFileReader reader, org.apache.hadoop.conf.Configuration configuration)
T	nextRecord() Returns the next record.
boolean	reachEnd() Checks if the record reader returned all records.
void	seek(long block, long recordInBlock) Moves the reading position to the given block and seeks to and reads the given record.
void	setSkipCorruptedRecord(boolean skipCorruptedRecord)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ParquetRecordReader

public ParquetRecordReader(org.apache.parquet.hadoop.api.ReadSupport<T> readSupport,
                           org.apache.parquet.schema.MessageType readSchema,
                           org.apache.parquet.filter2.compat.FilterCompat.Filter filter)

ParquetRecordReader

public ParquetRecordReader(org.apache.parquet.hadoop.api.ReadSupport<T> readSupport,
                           org.apache.parquet.schema.MessageType readSchema)

Method Detail

setSkipCorruptedRecord

public void setSkipCorruptedRecord(boolean skipCorruptedRecord)

initialize

public void initialize(org.apache.parquet.hadoop.ParquetFileReader reader,
                       org.apache.hadoop.conf.Configuration configuration)

seek

public void seek(long block,
                 long recordInBlock)
          throws IOException

Moves the reading position to the given block and seeks to and reads the given record.

Parameters:

block - The block to seek to.

recordInBlock - The number of the record in the block to return next.

Throws:

IOException

getCurrentReadPosition

public org.apache.flink.api.java.tuple.Tuple2<Long,Long> getCurrentReadPosition()

Returns the current read position in the split, i.e., the current block and the number of records that were returned from that block.

Returns:

The current read position in the split.

reachEnd

public boolean reachEnd()
                 throws IOException

Checks if the record reader returned all records. This method must be called before a record can be returned.

Returns:

False if there are more records to be read. True if all records have been returned.

Throws:

IOException

nextRecord

@CheckReturnValue(when=NEVER)
public T nextRecord()

Returns the next record. Note that the reachedEnd() method must be called before.

Returns:

The next record.

close

public void close()
           throws IOException

Throws:

IOException