Packages

o

org.apache.spark.sql.catalyst.expressions

FileSourceMetadataAttribute

object FileSourceMetadataAttribute

The internal representation of the FileSourceMetadataAttribute, it sets metadata_col and file_source_metadata_col to true in AttributeReference's metadata. This is a super type of FileSourceConstantMetadataAttribute and FileSourceGeneratedMetadataAttribute.

- apply() will create a file source metadata attribute reference - unapply() will check if an attribute reference is any file source metadata attribute reference

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. FileSourceMetadataAttribute
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. val FILE_SOURCE_METADATA_COL_ATTR_KEY: String
  5. def apply(name: String, dataType: DataType, nullable: Boolean = false): AttributeReference
  6. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  7. def cleanupFileSourceMetadataInformation(field: StructField): StructField

    Removes the internal field metadata.

  8. def cleanupFileSourceMetadataInformation(attr: Attribute): Attribute

    Removes the internal field metadata.

  9. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  10. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  11. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  12. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  13. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  14. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  15. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  16. def isSupportedType(dataType: DataType): Boolean

    True if the given data type is supported in file source metadata attributes.

    True if the given data type is supported in file source metadata attributes.

    The set of supported types is limited by ColumnVectorUtils.populate, which the constant file metadata implementation relies on. In general, types that can be partition columns are supported (including most primitive types). Notably unsupported types include ObjectType, UserDefinedType, and the complex types (StructType, MapType, ArrayType).

  17. def isValid(metadata: Metadata): Boolean
  18. def metadata(name: String): Metadata
  19. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  20. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  21. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  22. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  23. def toString(): String
    Definition Classes
    AnyRef → Any
  24. def unapply(attr: AttributeReference): Option[AttributeReference]

    Matches if attr is any File source metadata attribute (including constant and generated).

  25. def validateType(dataType: DataType): DataType

    Returns the type unchanged if valid; otherwise throws IllegalArgumentException.

  26. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  27. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  28. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from AnyRef

Inherited from Any

Ungrouped