Packages

  • package root
    Definition Classes
    root
  • package org
    Definition Classes
    root
  • package apache
    Definition Classes
    org
  • package spark
    Definition Classes
    apache
  • package sql
    Definition Classes
    spark
  • package catalyst

    Catalyst is a library for manipulating relational query plans.

    Catalyst is a library for manipulating relational query plans. All classes in catalyst are considered an internal API to Spark SQL and are subject to change between minor releases.

    Definition Classes
    sql
  • package dsl

    A collection of implicit conversions that create a DSL for constructing catalyst data structures.

    A collection of implicit conversions that create a DSL for constructing catalyst data structures.

    scala> import org.apache.spark.sql.catalyst.dsl.expressions._

// Standard operators are added to expressions.
scala> import org.apache.spark.sql.catalyst.expressions.Literal
scala> Literal(1) + Literal(1)
res0: org.apache.spark.sql.catalyst.expressions.Add = (1 + 1)

// There is a conversion from 'symbols to unresolved attributes.
scala> 'a.attr
res1: org.apache.spark.sql.catalyst.analysis.UnresolvedAttribute = 'a

// These unresolved attributes can be used to create more complicated expressions.
scala> 'a === 'b
res2: org.apache.spark.sql.catalyst.expressions.EqualTo = ('a = 'b)

// SQL verbs can be used to construct logical query plans.
scala> import org.apache.spark.sql.catalyst.plans.logical._
scala> import org.apache.spark.sql.catalyst.dsl.plans._
scala> LocalRelation('key.int, 'value.string).where('key === 1).select('value).analyze
res3: org.apache.spark.sql.catalyst.plans.logical.LogicalPlan =
Project [value#3]
 Filter (key#2 = 1)
  LocalRelation [key#2,value#3], []
    Definition Classes
    catalyst
  • trait ExpressionConversions extends AnyRef
    Definition Classes
    dsl
  • DslAttr
  • DslAttribute
  • DslExpression
  • DslString
  • DslSymbol
  • ImplicitAttribute
  • StringToAttributeConversionHelper

implicit class DslAttr extends ImplicitAttribute

Linear Supertypes
ImplicitAttribute, ImplicitOperators, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. DslAttr
  2. ImplicitAttribute
  3. ImplicitOperators
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new DslAttr(attr: UnresolvedAttribute)

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. def %(other: Expression): Expression
    Definition Classes
    ImplicitOperators
  4. def &(other: Expression): Expression
    Definition Classes
    ImplicitOperators
  5. def &&(other: Expression): Predicate
    Definition Classes
    ImplicitOperators
  6. def *(other: Expression): Expression
    Definition Classes
    ImplicitOperators
  7. def +(other: Expression): Expression
    Definition Classes
    ImplicitOperators
  8. def -(other: Expression): Expression
    Definition Classes
    ImplicitOperators
  9. def /(other: Expression): Expression
    Definition Classes
    ImplicitOperators
  10. def <(other: Expression): Predicate
    Definition Classes
    ImplicitOperators
  11. def <=(other: Expression): Predicate
    Definition Classes
    ImplicitOperators
  12. def <=>(other: Expression): Predicate
    Definition Classes
    ImplicitOperators
  13. def =!=(other: Expression): Predicate
    Definition Classes
    ImplicitOperators
  14. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  15. def ===(other: Expression): Predicate
    Definition Classes
    ImplicitOperators
  16. def >(other: Expression): Predicate
    Definition Classes
    ImplicitOperators
  17. def >=(other: Expression): Predicate
    Definition Classes
    ImplicitOperators
  18. def ^(other: Expression): Expression
    Definition Classes
    ImplicitOperators
  19. def array(arrayType: ArrayType): AttributeReference
    Definition Classes
    ImplicitAttribute
  20. def array(dataType: DataType): AttributeReference

    Creates a new AttributeReference of type array

    Creates a new AttributeReference of type array

    Definition Classes
    ImplicitAttribute
  21. def as(alias: Symbol): NamedExpression
    Definition Classes
    ImplicitOperators
  22. def as(alias: String): NamedExpression
    Definition Classes
    ImplicitOperators
  23. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  24. def asc: SortOrder
    Definition Classes
    ImplicitOperators
  25. def asc_nullsLast: SortOrder
    Definition Classes
    ImplicitOperators
  26. def attr: UnresolvedAttribute
    Definition Classes
    ImplicitAttribute
  27. def binary: AttributeReference

    Creates a new AttributeReference of type binary

    Creates a new AttributeReference of type binary

    Definition Classes
    ImplicitAttribute
  28. def boolean: AttributeReference

    Creates a new AttributeReference of type boolean

    Creates a new AttributeReference of type boolean

    Definition Classes
    ImplicitAttribute
  29. def byte: AttributeReference

    Creates a new AttributeReference of type byte

    Creates a new AttributeReference of type byte

    Definition Classes
    ImplicitAttribute
  30. def cast(to: DataType): Expression
    Definition Classes
    ImplicitOperators
  31. def castNullable(): Expression
    Definition Classes
    ImplicitOperators
  32. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  33. def contains(other: Expression): Expression
    Definition Classes
    ImplicitOperators
  34. def date: AttributeReference

    Creates a new AttributeReference of type date

    Creates a new AttributeReference of type date

    Definition Classes
    ImplicitAttribute
  35. def dayTimeInterval(): AttributeReference
    Definition Classes
    ImplicitAttribute
  36. def dayTimeInterval(startField: Byte, endField: Byte): AttributeReference

    Creates a new AttributeReference of the day-time interval type

    Creates a new AttributeReference of the day-time interval type

    Definition Classes
    ImplicitAttribute
  37. def decimal(precision: Int, scale: Int): AttributeReference

    Creates a new AttributeReference of type decimal

    Creates a new AttributeReference of type decimal

    Definition Classes
    ImplicitAttribute
  38. def decimal: AttributeReference

    Creates a new AttributeReference of type decimal

    Creates a new AttributeReference of type decimal

    Definition Classes
    ImplicitAttribute
  39. def desc: SortOrder
    Definition Classes
    ImplicitOperators
  40. def desc_nullsFirst: SortOrder
    Definition Classes
    ImplicitOperators
  41. def distinctFunction(exprs: Expression*): UnresolvedFunction
    Definition Classes
    ImplicitAttribute
  42. def div(other: Expression): Expression
    Definition Classes
    ImplicitOperators
  43. def double: AttributeReference

    Creates a new AttributeReference of type double

    Creates a new AttributeReference of type double

    Definition Classes
    ImplicitAttribute
  44. def endsWith(other: Expression): Expression
    Definition Classes
    ImplicitOperators
  45. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  46. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  47. def expr: UnresolvedAttribute
    Definition Classes
    ImplicitAttribute → ImplicitOperators
  48. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  49. def float: AttributeReference

    Creates a new AttributeReference of type float

    Creates a new AttributeReference of type float

    Definition Classes
    ImplicitAttribute
  50. def function(exprs: Expression*): UnresolvedFunction

    Create a function.

    Create a function.

    Definition Classes
    ImplicitAttribute
  51. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  52. def getField(fieldName: String): UnresolvedExtractValue
    Definition Classes
    ImplicitOperators
  53. def getItem(ordinal: Expression): UnresolvedExtractValue
    Definition Classes
    ImplicitOperators
  54. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  55. def ilike(other: Expression, escapeChar: Char = '\\'): Expression
    Definition Classes
    ImplicitOperators
  56. def in(list: Expression*): Expression
    Definition Classes
    ImplicitOperators
  57. def int: AttributeReference

    Creates a new AttributeReference of type int

    Creates a new AttributeReference of type int

    Definition Classes
    ImplicitAttribute
  58. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  59. def isNotNull: Predicate
    Definition Classes
    ImplicitOperators
  60. def isNull: Predicate
    Definition Classes
    ImplicitOperators
  61. def like(other: Expression, escapeChar: Char = '\\'): Expression
    Definition Classes
    ImplicitOperators
  62. def likeAll(others: Expression*): Expression
    Definition Classes
    ImplicitOperators
  63. def likeAny(others: Expression*): Expression
    Definition Classes
    ImplicitOperators
  64. def long: AttributeReference

    Creates a new AttributeReference of type long

    Creates a new AttributeReference of type long

    Definition Classes
    ImplicitAttribute
  65. def map(mapType: MapType): AttributeReference
    Definition Classes
    ImplicitAttribute
  66. def map(keyType: DataType, valueType: DataType): AttributeReference

    Creates a new AttributeReference of type map

    Creates a new AttributeReference of type map

    Definition Classes
    ImplicitAttribute
  67. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  68. def notLikeAll(others: Expression*): Expression
    Definition Classes
    ImplicitOperators
  69. def notLikeAny(others: Expression*): Expression
    Definition Classes
    ImplicitOperators
  70. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  71. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  72. def obj(cls: Class[_]): AttributeReference

    Creates a new AttributeReference of object type

    Creates a new AttributeReference of object type

    Definition Classes
    ImplicitAttribute
  73. def rlike(other: Expression): Expression
    Definition Classes
    ImplicitOperators
  74. def s: String
    Definition Classes
    DslAttr → ImplicitAttribute
  75. def short: AttributeReference

    Creates a new AttributeReference of type short

    Creates a new AttributeReference of type short

    Definition Classes
    ImplicitAttribute
  76. def startsWith(other: Expression): Expression
    Definition Classes
    ImplicitOperators
  77. def string: AttributeReference

    Creates a new AttributeReference of type string

    Creates a new AttributeReference of type string

    Definition Classes
    ImplicitAttribute
  78. def struct(attrs: AttributeReference*): AttributeReference
    Definition Classes
    ImplicitAttribute
  79. def struct(structType: StructType): AttributeReference

    Creates a new AttributeReference of type struct

    Creates a new AttributeReference of type struct

    Definition Classes
    ImplicitAttribute
  80. def substr(pos: Expression, len: Expression = Literal(Int.MaxValue)): Expression
    Definition Classes
    ImplicitOperators
  81. def substring(pos: Expression, len: Expression = Literal(Int.MaxValue)): Expression
    Definition Classes
    ImplicitOperators
  82. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  83. def timestamp: AttributeReference

    Creates a new AttributeReference of type timestamp

    Creates a new AttributeReference of type timestamp

    Definition Classes
    ImplicitAttribute
  84. def toString(): String
    Definition Classes
    AnyRef → Any
  85. def unary_!: Predicate
    Definition Classes
    ImplicitOperators
  86. def unary_+: Expression
    Definition Classes
    ImplicitOperators
  87. def unary_-: Expression
    Definition Classes
    ImplicitOperators
  88. def unary_~: Expression
    Definition Classes
    ImplicitOperators
  89. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  90. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  91. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  92. def yearMonthInterval(): AttributeReference
    Definition Classes
    ImplicitAttribute
  93. def yearMonthInterval(startField: Byte, endField: Byte): AttributeReference

    Creates a new AttributeReference of the year-month interval type

    Creates a new AttributeReference of the year-month interval type

    Definition Classes
    ImplicitAttribute
  94. def |(other: Expression): Expression
    Definition Classes
    ImplicitOperators
  95. def ||(other: Expression): Predicate
    Definition Classes
    ImplicitOperators

Inherited from ImplicitAttribute

Inherited from ImplicitOperators

Inherited from AnyRef

Inherited from Any

Ungrouped