Value Members

1.  final def !=(arg0: Any): Boolean

   Definition Classes

   AnyRef → Any

2.  final def ##(): Int

   Definition Classes

   AnyRef → Any

3.  final def ==(arg0: Any): Boolean

   Definition Classes

   AnyRef → Any

4.  final def asInstanceOf[T0]: T0

   Definition Classes

   Any

5.  def canCreatePartitioning: Boolean

   Whether this shuffle spec can be used to create partitionings for the other children.

   Whether this shuffle spec can be used to create partitionings for the other children.

   Definition Classes

   KeyGroupedShuffleSpec → ShuffleSpec

6.  def clone(): AnyRef

   Attributes

   protected[lang]

   Definition Classes

   AnyRef

   Annotations

   @throws( ... ) @native()

7.  def createPartitioning(clustering: Seq[Expression]): Partitioning

   Creates a partitioning that can be used to re-partition the other side with the given clustering expressions.

   Creates a partitioning that can be used to re-partition the other side with the given clustering expressions.

   This will only be called when:

   • isCompatibleWith returns false on the side where the clustering is from.

   Definition Classes

   ShuffleSpec

8.  val distribution: ClusteredDistribution
9.  final def eq(arg0: AnyRef): Boolean

   Definition Classes

   AnyRef

10.  def finalize(): Unit

    Attributes

    protected[lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( classOf[java.lang.Throwable] )

11.  final def getClass(): Class[_]

    Definition Classes

    AnyRef → Any

    Annotations

    @native()

12.  def isCompatibleWith(other: ShuffleSpec): Boolean

    Returns true iff this spec is compatible with the provided shuffle spec.

    Returns true iff this spec is compatible with the provided shuffle spec.

    A true return value means that the data partitioning from this spec can be seen as co-partitioned with the other, and therefore no shuffle is required when joining the two sides.

    Note that Spark assumes this to be reflexive, symmetric and transitive.

    Definition Classes

    KeyGroupedShuffleSpec → ShuffleSpec

13.  final def isInstanceOf[T0]: Boolean

    Definition Classes

    Any

14.  lazy val keyPositions: Seq[BitSet]

    A sequence where each element is a set of positions of the partition expression to the cluster keys.

    A sequence where each element is a set of positions of the partition expression to the cluster keys. For instance, if cluster keys are [a, b, b] and partition expressions are [bucket(4, a), years(b)], the result will be [(0), (1, 2)].

    Note that we only allow each partition expression to contain a single partition key. Therefore the mapping here is very similar to that from HashShuffleSpec.

15.  final def ne(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

16.  final def notify(): Unit

    Definition Classes

    AnyRef

    Annotations

    @native()

17.  final def notifyAll(): Unit

    Definition Classes

    AnyRef

    Annotations

    @native()

18.  def numPartitions: Int

    Returns the number of partitions of this shuffle spec

    Returns the number of partitions of this shuffle spec

    Definition Classes

    KeyGroupedShuffleSpec → ShuffleSpec

19.  val partitioning: KeyGroupedPartitioning
20.  final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes

    AnyRef

21.  final def wait(): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

22.  final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

23.  final def wait(arg0: Long): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... ) @native()