Packages

c

org.apache.spark.sql.execution.datasources.parquet

VectorizedRleValuesReader

final class VectorizedRleValuesReader extends ValuesReader with VectorizedValuesReader

A values reader for Parquet's run-length encoded data. This is based off of the version in parquet-mr with these changes:

  • Supports the vectorized interface.
  • Works on byte arrays(byte[]) instead of making byte streams.

This encoding is used in multiple places:

  • Definition/Repetition levels
  • Dictionary ids.
Linear Supertypes
VectorizedValuesReader, ValuesReader, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. VectorizedRleValuesReader
  2. VectorizedValuesReader
  3. ValuesReader
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new VectorizedRleValuesReader(bitWidth: Int, readLength: Boolean)
  2. new VectorizedRleValuesReader(bitWidth: Int)
  3. new VectorizedRleValuesReader()

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  10. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  11. def initFromPage(valueCount: Int, in: ByteBufferInputStream): Unit
    Definition Classes
    VectorizedRleValuesReader → ValuesReader
  12. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  13. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  14. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  15. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  16. def readBinary(len: Int): Binary
    Definition Classes
    VectorizedRleValuesReader → VectorizedValuesReader
  17. def readBinary(total: Int, c: WritableColumnVector, rowId: Int): Unit
    Definition Classes
    VectorizedRleValuesReader → VectorizedValuesReader
  18. def readBinarys(total: Int, c: WritableColumnVector, rowId: Int, level: Int, data: VectorizedValuesReader): Unit
  19. def readBoolean(): Boolean
    Definition Classes
    VectorizedRleValuesReader → VectorizedValuesReader → ValuesReader
  20. def readBooleans(total: Int, c: WritableColumnVector, rowId: Int): Unit
    Definition Classes
    VectorizedRleValuesReader → VectorizedValuesReader
  21. def readBooleans(total: Int, c: WritableColumnVector, rowId: Int, level: Int, data: VectorizedValuesReader): Unit
  22. def readByte(): Byte
    Definition Classes
    VectorizedRleValuesReader → VectorizedValuesReader
  23. def readBytes(total: Int, c: WritableColumnVector, rowId: Int): Unit
    Definition Classes
    VectorizedRleValuesReader → VectorizedValuesReader
  24. def readBytes(total: Int, c: WritableColumnVector, rowId: Int, level: Int, data: VectorizedValuesReader): Unit
  25. def readBytes(): Binary
    Definition Classes
    ValuesReader
  26. def readDouble(): Double
    Definition Classes
    ValuesReader
  27. def readDoubles(total: Int, c: WritableColumnVector, rowId: Int): Unit
    Definition Classes
    VectorizedRleValuesReader → VectorizedValuesReader
  28. def readDoubles(total: Int, c: WritableColumnVector, rowId: Int, level: Int, data: VectorizedValuesReader): Unit
  29. def readFloat(): Float
    Definition Classes
    ValuesReader
  30. def readFloats(total: Int, c: WritableColumnVector, rowId: Int): Unit
    Definition Classes
    VectorizedRleValuesReader → VectorizedValuesReader
  31. def readFloats(total: Int, c: WritableColumnVector, rowId: Int, level: Int, data: VectorizedValuesReader): Unit
  32. def readInteger(): Int
    Definition Classes
    VectorizedRleValuesReader → VectorizedValuesReader → ValuesReader
  33. def readIntegers(total: Int, c: WritableColumnVector, rowId: Int): Unit
    Definition Classes
    VectorizedRleValuesReader → VectorizedValuesReader
  34. def readIntegers(total: Int, values: WritableColumnVector, nulls: WritableColumnVector, rowId: Int, level: Int, data: VectorizedValuesReader): Unit

    Decoding for dictionary ids.

    Decoding for dictionary ids. The IDs are populated into values and the nullability is populated into nulls.

  35. def readIntegers(total: Int, c: WritableColumnVector, rowId: Int, level: Int, data: VectorizedValuesReader): Unit

    Reads total ints into c filling them in starting at c[rowId].

    Reads total ints into c filling them in starting at c[rowId]. This reader reads the definition levels and then will read from data for the non-null values. If the value is null, c will be populated with nullValue. Note that nullValue is only necessary for readIntegers because we also use it to decode dictionaryIds and want to make sure it always has a value in range.

    This is a batched version of this logic: if (this.readInt() == level) { c[rowId] = data.readInteger(); } else { c[rowId] = null; }

  36. def readIntegersWithRebase(total: Int, c: WritableColumnVector, rowId: Int, failIfRebase: Boolean): Unit
    Definition Classes
    VectorizedRleValuesReader → VectorizedValuesReader
  37. def readIntegersWithRebase(total: Int, c: WritableColumnVector, rowId: Int, level: Int, data: VectorizedValuesReader, failIfRebase: Boolean): Unit
  38. def readLong(): Long
    Definition Classes
    ValuesReader
  39. def readLongs(total: Int, c: WritableColumnVector, rowId: Int): Unit
    Definition Classes
    VectorizedRleValuesReader → VectorizedValuesReader
  40. def readLongs(total: Int, c: WritableColumnVector, rowId: Int, level: Int, data: VectorizedValuesReader): Unit
  41. def readLongsWithRebase(total: Int, c: WritableColumnVector, rowId: Int, failIfRebase: Boolean): Unit
    Definition Classes
    VectorizedRleValuesReader → VectorizedValuesReader
  42. def readLongsWithRebase(total: Int, c: WritableColumnVector, rowId: Int, level: Int, data: VectorizedValuesReader, failIfRebase: Boolean): Unit
  43. def readShorts(total: Int, c: WritableColumnVector, rowId: Int, level: Int, data: VectorizedValuesReader): Unit
  44. def readValueDictionaryId(): Int
    Definition Classes
    VectorizedRleValuesReader → ValuesReader
  45. def skip(): Unit
    Definition Classes
    VectorizedRleValuesReader → ValuesReader
  46. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  47. def toString(): String
    Definition Classes
    AnyRef → Any
  48. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  49. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  50. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from VectorizedValuesReader

Inherited from ValuesReader

Inherited from AnyRef

Inherited from Any

Ungrouped