com.facebook.hive.orc.lazy

Class LazyTreeReader

java.lang.Object

com.facebook.hive.orc.lazy.LazyTreeReader

Direct Known Subclasses:

LazyBinaryTreeReader, LazyBooleanTreeReader, LazyByteTreeReader, LazyDoubleTreeReader, LazyFloatTreeReader, LazyIntTreeReader, LazyListTreeReader, LazyMapTreeReader, LazyStringDictionaryTreeReader, LazyStringTreeReader, LazyStructTreeReader, LazyTimestampTreeReader, LazyUnionTreeReader

public abstract class LazyTreeReader
extends Object

Field Summary

Fields

Modifier and Type	Field and Description
protected int	columnId
protected long	previousPresentRow
protected long	previousRow
protected int	previousRowIndexEntry
protected long	rowBaseInStripe
protected long	rowIndexStride
protected boolean	valuePresent

Constructor Summary

Constructors

Constructor and Description
LazyTreeReader(int columnId, long rowIndexStride)

Method Summary

Modifier and Type	Method and Description
void	close()
protected int	computeRowIndexEntry(long row) Computes the number of the row index entry that immediately precedes row
protected long	countNonNulls(long rows) in the next rows values, returns the number of values which are not null
Object	createWritableFromLatest(Object previous) When a client asks for a writable object after materializing a primitive, create a writable from previous value read from stream.
Object	get(long currentRow, Object previous) Returns the value corresponding to currentRow, suitable for calling from outside
boolean	getBoolean(long currentRow)
double	getDouble(long currentRow)
float	getFloat(long currentRow)
Object	getInComplexType(Object previous, long row) Should be called only from containers (lists, maps, structs, unions) since for these tree readers the number of rows does not correspond to the number of values (e.g.
int	getInt(long currentRow)
long	getLong(long currentRow)
short	getShort(long currentRow)
int	loadIndeces(List<OrcProto.RowIndexEntry> rowIndexEntries, int startIndex) Read any indeces that will be needed and return a startIndex after the values that have been read.
protected abstract Object	next(Object previous) Should only be called from within the tree reader once it's been checked that the next object isn't null
boolean	nextBoolean(boolean readStream)
double	nextDouble(boolean readStream)
float	nextFloat(boolean readStream)
int	nextInt(boolean readStream)
boolean	nextIsNull(long currentRow) Returns whether or not the value corresponding to currentRow is null, suitable for calling from outside
boolean	nextIsNullInComplexType() Should be called only from containers (lists, maps, structs, unions) since for these tree readers the number of rows does not correspond to the number of values (e.g.
long	nextLong(boolean readStream)
short	nextShort(boolean readStream)
protected abstract void	seek(int index)
protected void	seek(int rowIndexEntry, boolean backwards) Adjust all streams to the beginning of the row index entry specified, backwards means that a previous value is being read and forces the index entry to be restarted, otherwise, has no effect if we're already in the current index entry
protected void	seekToPresentRow(long currentRow) Shifts only the present stream so that next will return the value for currentRow
protected boolean	seekToRow(long currentRow) Adjusts all streams so that the next value returned corresponds to the value for currentRow, suitable for calling from outside
protected abstract void	skipRows(long numNonNullValues) Should only be called from within the tree reader once numNonNulls has been set correctly
void	skipRowsInComplexType(long numRows) Should be called only from containers (lists, maps, structs, unions) since for these tree readers the number of rows does not correspond to the number of values (e.g.
void	startStripe(Map<StreamName,InStream> streams, List<OrcProto.ColumnEncoding> encodings, OrcProto.RowIndex[] indexes, long rowBaseInStripe)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

rowIndexStride

protected long rowIndexStride

previousRow

protected long previousRow

columnId

protected final int columnId

rowBaseInStripe

protected long rowBaseInStripe

valuePresent

protected boolean valuePresent

previousPresentRow

protected long previousPresentRow

previousRowIndexEntry

protected int previousRowIndexEntry

Constructor Detail

LazyTreeReader

public LazyTreeReader(int columnId,
                      long rowIndexStride)

Method Detail

seek

protected abstract void seek(int index)
                      throws IOException

Throws:

IOException

countNonNulls

protected long countNonNulls(long rows)
                      throws IOException

in the next rows values, returns the number of values which are not null

Parameters:

rows -

Returns:

Throws:

IOException

skipRows

protected abstract void skipRows(long numNonNullValues)
                          throws IOException

Should only be called from within the tree reader once numNonNulls has been set correctly

Throws:

IOException

skipRowsInComplexType

public void skipRowsInComplexType(long numRows)
                           throws IOException

Should be called only from containers (lists, maps, structs, unions) since for these tree readers the number of rows does not correspond to the number of values (e.g. there may be many more if it contains the entries in lists with more than one element, or much less if it is the fields in a struct where every other struct is null)

Parameters:

numRows -

Throws:

IOException

next

protected abstract Object next(Object previous)
                        throws IOException

Should only be called from within the tree reader once it's been checked that the next object isn't null

Parameters:

previous -

Returns:

Throws:

IOException

getInComplexType

public Object getInComplexType(Object previous,
                               long row)
                        throws IOException

Should be called only from containers (lists, maps, structs, unions) since for these tree readers the number of rows does not correspond to the number of values (e.g. there may be many more if it contains the entries in lists with more than one element, or much less if it is the fields in a struct where every other struct is null)

Parameters:

previous -

Returns:

Throws:

IOException

nextIsNull

public boolean nextIsNull(long currentRow)
                   throws IOException

Returns whether or not the value corresponding to currentRow is null, suitable for calling from outside

Parameters:

currentRow -

Returns:

Throws:

IOException

nextIsNullInComplexType

public boolean nextIsNullInComplexType()
                                throws IOException

Should be called only from containers (lists, maps, structs, unions) since for these tree readers the number of rows does not correspond to the number of values (e.g. there may be many more if it contains the entries in lists with more than one element, or much less if it is the fields in a struct where every other struct is null)

Returns:

Throws:

IOException

computeRowIndexEntry

protected int computeRowIndexEntry(long row)

Computes the number of the row index entry that immediately precedes row

Parameters:

row -

Returns:

seekToPresentRow

protected void seekToPresentRow(long currentRow)
                         throws IOException

Shifts only the present stream so that next will return the value for currentRow

Parameters:

currentRow -

Throws:

IOException

get

public Object get(long currentRow,
                  Object previous)
           throws IOException

Returns the value corresponding to currentRow, suitable for calling from outside

Parameters:

currentRow -

previous -

Returns:

Throws:

IOException

createWritableFromLatest

public Object createWritableFromLatest(Object previous)
                                throws IOException

When a client asks for a writable object after materializing a primitive, create a writable from previous value read from stream.

Throws:

IOException

getDouble

public double getDouble(long currentRow)
                 throws IOException

Throws:

IOException

getFloat

public float getFloat(long currentRow)
               throws IOException

Throws:

IOException

getBoolean

public boolean getBoolean(long currentRow)
                   throws IOException

Throws:

IOException

getLong

public long getLong(long currentRow)
             throws IOException

Throws:

IOException

getInt

public int getInt(long currentRow)
           throws IOException

Throws:

IOException

getShort

public short getShort(long currentRow)
               throws IOException

Throws:

IOException

nextDouble

public double nextDouble(boolean readStream)
                  throws IOException

Throws:

IOException

nextFloat

public float nextFloat(boolean readStream)
                throws IOException

Throws:

IOException

nextBoolean

public boolean nextBoolean(boolean readStream)
                    throws IOException

Throws:

IOException

nextLong

public long nextLong(boolean readStream)
              throws IOException

Throws:

IOException

nextInt

public int nextInt(boolean readStream)
            throws IOException

Throws:

IOException

nextShort

public short nextShort(boolean readStream)
                throws IOException

Throws:

IOException

seek

protected void seek(int rowIndexEntry,
                    boolean backwards)
             throws IOException

Adjust all streams to the beginning of the row index entry specified, backwards means that a previous value is being read and forces the index entry to be restarted, otherwise, has no effect if we're already in the current index entry

Parameters:

rowIndexEntry -

backwards -

Throws:

IOException

seekToRow

protected boolean seekToRow(long currentRow)
                     throws IOException

Adjusts all streams so that the next value returned corresponds to the value for currentRow, suitable for calling from outside

Parameters:

currentRow -

Returns:

Throws:

IOException

loadIndeces

public int loadIndeces(List<OrcProto.RowIndexEntry> rowIndexEntries,
                       int startIndex)

Read any indeces that will be needed and return a startIndex after the values that have been read.

startStripe

public void startStripe(Map<StreamName,InStream> streams,
                        List<OrcProto.ColumnEncoding> encodings,
                        OrcProto.RowIndex[] indexes,
                        long rowBaseInStripe)
                 throws IOException

Throws:

IOException

close

public void close()
           throws IOException

Throws:

IOException