com.twitter.elephantbird.mapreduce.input
Class RCFileProtobufInputFormat.ProtobufReader

java.lang.Object
  org.apache.hadoop.mapreduce.RecordReader<K,V>
      com.twitter.elephantbird.mapreduce.input.FilterRecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Writable>
          com.twitter.elephantbird.mapreduce.input.RCFileProtobufInputFormat.ProtobufReader

All Implemented Interfaces:

Closeable

Direct Known Subclasses:

RCFileProtobufTupleInputFormat.TupleReader

Enclosing class:

RCFileProtobufInputFormat

public class RCFileProtobufInputFormat.ProtobufReader
extends FilterRecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Writable>

Field Summary

protected ArrayList<Integer>	columnsBeingRead
protected List<com.google.protobuf.Descriptors.FieldDescriptor>	knownRequiredFields
protected com.google.protobuf.Message.Builder	msgBuilder
protected com.google.protobuf.Message	msgInstance
protected ProtobufWritable<com.google.protobuf.Message>	protoWritable
protected boolean	readUnknownsColumn

Fields inherited from class com.twitter.elephantbird.mapreduce.input.FilterRecordReader

reader

Constructor Summary

RCFileProtobufInputFormat.ProtobufReader(org.apache.hadoop.mapreduce.RecordReader reader)
The reader is expected to be a RecordReader< LongWritable, BytesRefArrayWritable >.

Method Summary

org.apache.hadoop.hive.serde2.columnar.BytesRefArrayWritable	getCurrentBytesRefArrayWritable() returns super.getCurrentValue()
com.google.protobuf.Message	getCurrentProtobufValue() Builds protobuf message from the raw bytes returned by RCFile reader.
org.apache.hadoop.io.Writable	getCurrentValue()
void	initialize(org.apache.hadoop.mapreduce.InputSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext ctx)
boolean	isReadingUnknonwsColumn() is valid only after initialize() is called

Methods inherited from class com.twitter.elephantbird.mapreduce.input.FilterRecordReader

close, getCurrentKey, getProgress, nextKeyValue

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

msgInstance

protected com.google.protobuf.Message msgInstance

msgBuilder

protected com.google.protobuf.Message.Builder msgBuilder

readUnknownsColumn

protected boolean readUnknownsColumn

knownRequiredFields

protected List<com.google.protobuf.Descriptors.FieldDescriptor> knownRequiredFields

columnsBeingRead

protected ArrayList<Integer> columnsBeingRead

protoWritable

protected ProtobufWritable<com.google.protobuf.Message> protoWritable

Constructor Detail

RCFileProtobufInputFormat.ProtobufReader

public RCFileProtobufInputFormat.ProtobufReader(org.apache.hadoop.mapreduce.RecordReader reader)

The reader is expected to be a RecordReader< LongWritable, BytesRefArrayWritable >.

Method Detail

isReadingUnknonwsColumn

public boolean isReadingUnknonwsColumn()

is valid only after initialize() is called

initialize

public void initialize(org.apache.hadoop.mapreduce.InputSplit split,
                       org.apache.hadoop.mapreduce.TaskAttemptContext ctx)
                throws IOException,
                       InterruptedException

Overrides:

initialize in class FilterRecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Writable>

Throws:

IOException

InterruptedException

getCurrentValue

public org.apache.hadoop.io.Writable getCurrentValue()
                                              throws IOException,
                                                     InterruptedException

Overrides:

getCurrentValue in class FilterRecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Writable>

Throws:

IOException

InterruptedException

getCurrentBytesRefArrayWritable

public org.apache.hadoop.hive.serde2.columnar.BytesRefArrayWritable getCurrentBytesRefArrayWritable()
                                                                                             throws IOException,
                                                                                                    InterruptedException

returns super.getCurrentValue()

Throws:

IOException

InterruptedException

getCurrentProtobufValue

public com.google.protobuf.Message getCurrentProtobufValue()
                                                    throws IOException,
                                                           InterruptedException

Builds protobuf message from the raw bytes returned by RCFile reader.

Throws:

IOException

InterruptedException