org.apache.flink.formats.parquet

Class ParquetRowInputFormat

java.lang.Object

org.apache.flink.api.common.io.RichInputFormat<OT,org.apache.flink.core.fs.FileInputSplit>

org.apache.flink.api.common.io.FileInputFormat<E>

org.apache.flink.formats.parquet.ParquetInputFormat<org.apache.flink.types.Row>

org.apache.flink.formats.parquet.ParquetRowInputFormat

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.io.CheckpointableInputFormat<org.apache.flink.core.fs.FileInputSplit,org.apache.flink.api.java.tuple.Tuple2<Long,Long>>, org.apache.flink.api.common.io.InputFormat<org.apache.flink.types.Row,org.apache.flink.core.fs.FileInputSplit>, org.apache.flink.api.java.typeutils.ResultTypeQueryable<org.apache.flink.types.Row>, org.apache.flink.core.io.InputSplitSource<org.apache.flink.core.fs.FileInputSplit>

public class ParquetRowInputFormat
extends ParquetInputFormat<org.apache.flink.types.Row>
implements org.apache.flink.api.java.typeutils.ResultTypeQueryable<org.apache.flink.types.Row>

An implementation of ParquetInputFormat to read Row records from Parquet files.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.FileInputFormat

org.apache.flink.api.common.io.FileInputFormat.FileBaseStatistics, org.apache.flink.api.common.io.FileInputFormat.InputSplitOpenThread

Field Summary

Fields inherited from class org.apache.flink.formats.parquet.ParquetInputFormat

PARQUET_SKIP_CORRUPTED_RECORD, PARQUET_SKIP_WRONG_SCHEMA_SPLITS

Fields inherited from class org.apache.flink.api.common.io.FileInputFormat

currentSplit, ENUMERATE_NESTED_FILES_FLAG, enumerateNestedFiles, filePath, INFLATER_INPUT_STREAM_FACTORIES, minSplitSize, numSplits, openTimeout, READ_WHOLE_SPLIT_FLAG, splitLength, splitStart, stream, unsplittable

Constructor Summary

Constructors

Constructor and Description
ParquetRowInputFormat(org.apache.flink.core.fs.Path path)
ParquetRowInputFormat(org.apache.flink.core.fs.Path path, org.apache.parquet.schema.MessageType messageType)

Method Summary

Modifier and Type	Method and Description
protected org.apache.flink.types.Row	convert(org.apache.flink.types.Row row) This ParquetInputFormat read parquet record as Row by default.
org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.types.Row>	getProducedType()

Methods inherited from class org.apache.flink.formats.parquet.ParquetInputFormat

close, configure, getCurrentState, getFieldNames, getFieldTypes, getPredicate, nextRecord, open, reachedEnd, reopen, selectFields, setFilterPredicate

Methods inherited from class org.apache.flink.api.common.io.FileInputFormat

acceptFile, createInputSplits, decorateInputStream, extractFileExtension, getFilePath, getFilePaths, getFileStats, getFileStats, getInflaterInputStreamFactory, getInputSplitAssigner, getMinSplitSize, getNestedFileEnumeration, getNumSplits, getOpenTimeout, getSplitLength, getSplitStart, getStatistics, registerInflaterInputStreamFactory, setFilePath, setFilePath, setFilePaths, setFilePaths, setFilesFilter, setMinSplitSize, setNestedFileEnumeration, setNumSplits, setOpenTimeout, supportsMultiPaths, testForUnsplittable, toString

Methods inherited from class org.apache.flink.api.common.io.RichInputFormat

closeInputFormat, getRuntimeContext, openInputFormat, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

ParquetRowInputFormat

public ParquetRowInputFormat(org.apache.flink.core.fs.Path path,
                             org.apache.parquet.schema.MessageType messageType)

ParquetRowInputFormat

public ParquetRowInputFormat(org.apache.flink.core.fs.Path path)

Method Detail

getProducedType

public org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.types.Row> getProducedType()

Specified by:

getProducedType in interface org.apache.flink.api.java.typeutils.ResultTypeQueryable<org.apache.flink.types.Row>

convert

protected org.apache.flink.types.Row convert(org.apache.flink.types.Row row)

Description copied from class: ParquetInputFormat

This ParquetInputFormat read parquet record as Row by default. Sub classes of it can extend this method to further convert row to other types, such as POJO, Map or Tuple.

Specified by:

convert in class ParquetInputFormat<org.apache.flink.types.Row>

Parameters:

row - row read from parquet file

Returns:

E target result type