org.apache.flink.optimizer.dag

Class TwoInputNode

java.lang.Object

org.apache.flink.optimizer.dag.OptimizerNode

org.apache.flink.optimizer.dag.TwoInputNode

All Implemented Interfaces:

EstimateProvider, DumpableNode<OptimizerNode>, org.apache.flink.util.Visitable<OptimizerNode>

Direct Known Subclasses:

BinaryUnionNode, CoGroupNode, CoGroupRawNode, CrossNode, JoinNode, OuterJoinNode, SinkJoiner, WorksetIterationNode, WorksetIterationNode.SingleRootJoiner

public abstract class TwoInputNode
extends OptimizerNode

A node in the optimizer plan that represents an operator with a two different inputs, such as Join, Cross, CoGroup, or Union. The two inputs are not substitutable in their sides.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.optimizer.dag.OptimizerNode

OptimizerNode.UnclosedBranchDescriptor

Field Summary

Fields

Modifier and Type	Field and Description
protected DagConnection	input1
protected DagConnection	input2
protected org.apache.flink.api.common.operators.util.FieldList	keys1
protected org.apache.flink.api.common.operators.util.FieldList	keys2

Fields inherited from class org.apache.flink.optimizer.dag.OptimizerNode

cachedPlans, closedBranchingNodes, costWeight, estimatedNumRecords, estimatedOutputSize, hereJoinedBranches, id, MAX_DYNAMIC_PATH_COST_WEIGHT, onDynamicPath, openBranches, uniqueFields

Constructor Summary

Constructors

Constructor and Description
TwoInputNode(org.apache.flink.api.common.operators.DualInputOperator<?,?,?,?> operator) Creates a new two input node for the optimizer plan, representing the given operator.

Method Summary

Modifier and Type	Method and Description
void	accept(org.apache.flink.util.Visitor<OptimizerNode> visitor) This method implements the visit of a depth-first graph traversing visitor.
protected void	addLocalCandidates(Channel template1, Channel template2, List<Set<? extends NamedChannel>> broadcastPlanChannels, RequestedGlobalProperties rgps1, RequestedGlobalProperties rgps2, List<PlanNode> target, OperatorDescriptorDual.LocalPropertiesPair[] validLocalCombinations, CostEstimator estimator)
void	computeInterestingPropertiesForInputs(CostEstimator estimator) Tells the node to compute the interesting properties for its inputs.
void	computeUnclosedBranchStack() This method causes the node to compute the description of open branches in its sub-plan.
List<PlanNode>	getAlternativePlans(CostEstimator estimator) Computes the plan alternatives for this node, an implicitly for all nodes that are children of this node.
DagConnection	getFirstIncomingConnection() Gets the DagConnection through which this node receives its first input.
OptimizerNode	getFirstPredecessorNode()
List<DagConnection>	getIncomingConnections() Gets all incoming connections of this node.
org.apache.flink.api.common.operators.DualInputOperator<?,?,?,?>	getOperator() Gets the operator represented by this optimizer node.
protected abstract List<OperatorDescriptorDual>	getPossibleProperties()
DagConnection	getSecondIncomingConnection() Gets the DagConnection through which this node receives its second input.
OptimizerNode	getSecondPredecessorNode()
org.apache.flink.api.common.operators.SemanticProperties	getSemanticProperties()
protected org.apache.flink.api.common.operators.SemanticProperties	getSemanticPropertiesForGlobalPropertyFiltering()
protected org.apache.flink.api.common.operators.SemanticProperties	getSemanticPropertiesForLocalPropertyFiltering()
protected void	instantiate(OperatorDescriptorDual operator, Channel in1, Channel in2, List<Set<? extends NamedChannel>> broadcastPlanChannels, List<PlanNode> target, CostEstimator estimator, RequestedGlobalProperties globPropsReq1, RequestedGlobalProperties globPropsReq2, RequestedLocalProperties locPropsReq1, RequestedLocalProperties locPropsReq2)
protected void	placePipelineBreakersIfNecessary(org.apache.flink.runtime.operators.DriverStrategy strategy, Channel in1, Channel in2)
void	setInput(Map<org.apache.flink.api.common.operators.Operator<?>,OptimizerNode> contractToNode, org.apache.flink.api.common.ExecutionMode defaultExecutionMode) This function connects the predecessors to this operator.

Methods inherited from class org.apache.flink.optimizer.dag.OptimizerNode

addBroadcastConnection, addClosedBranch, addClosedBranches, addOutgoingConnection, areBranchCompatible, clearInterestingProperties, computeOperatorSpecificDefaultEstimates, computeOutputEstimates, computeUnclosedBranchStackForBroadcastInputs, computeUnionOfInterestingPropertiesFromSuccessors, getBranchesForParent, getBroadcastConnectionNames, getBroadcastConnections, getClosedBranchingNodes, getCostWeight, getDumpableInputs, getEstimatedAvgWidthPerOutputRecord, getEstimatedNumRecords, getEstimatedOutputSize, getId, getInterestingProperties, getMaxDepth, getMinimalMemoryAcrossAllSubTasks, getOpenBranches, getOperatorName, getOptimizerNode, getOutgoingConnections, getParallelism, getPlanNode, getPredecessors, getUniqueFields, hasUnclosedBranches, haveAllOutputConnectionInterestingProperties, identifyDynamicPath, initId, isBranching, isOnDynamicPath, markAllOutgoingConnectionsAsPipelineBreaking, mergeLists, prunePlanAlternatives, prunePlanAlternativesWithCommonBranching, readStubAnnotations, readUniqueFieldsAnnotation, removeClosedBranches, setBroadcastInputs, setEstimatedNumRecords, setEstimatedOutputSize, setParallelism, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

keys1

protected final org.apache.flink.api.common.operators.util.FieldList keys1

keys2

protected final org.apache.flink.api.common.operators.util.FieldList keys2

input1

protected DagConnection input1

input2

protected DagConnection input2

Constructor Detail

TwoInputNode

public TwoInputNode(org.apache.flink.api.common.operators.DualInputOperator<?,?,?,?> operator)

Creates a new two input node for the optimizer plan, representing the given operator.

Parameters:

operator - The operator that the optimizer DAG node should represent.

Method Detail

getOperator

public org.apache.flink.api.common.operators.DualInputOperator<?,?,?,?> getOperator()

Description copied from class: OptimizerNode

Gets the operator represented by this optimizer node.

Overrides:

getOperator in class OptimizerNode

Returns:

This node's operator.

getFirstIncomingConnection

public DagConnection getFirstIncomingConnection()

Gets the DagConnection through which this node receives its first input.

Returns:

The first input connection.

getSecondIncomingConnection

public DagConnection getSecondIncomingConnection()

Gets the DagConnection through which this node receives its second input.

Returns:

The second input connection.

getFirstPredecessorNode

public OptimizerNode getFirstPredecessorNode()

getSecondPredecessorNode

public OptimizerNode getSecondPredecessorNode()

getIncomingConnections

public List<DagConnection> getIncomingConnections()

Description copied from class: OptimizerNode

Gets all incoming connections of this node. This method needs to be overridden by subclasses to return the children.

Specified by:

getIncomingConnections in class OptimizerNode

Returns:

The list of incoming connections.

setInput

public void setInput(Map<org.apache.flink.api.common.operators.Operator<?>,OptimizerNode> contractToNode,
                     org.apache.flink.api.common.ExecutionMode defaultExecutionMode)

Description copied from class: OptimizerNode

This function connects the predecessors to this operator.

Specified by:

setInput in class OptimizerNode

Parameters:

contractToNode - The map from program operators to optimizer nodes.

defaultExecutionMode - The data exchange mode to use, if the operator does not specify one.

getPossibleProperties

protected abstract List<OperatorDescriptorDual> getPossibleProperties()

computeInterestingPropertiesForInputs

public void computeInterestingPropertiesForInputs(CostEstimator estimator)

Description copied from class: OptimizerNode

Tells the node to compute the interesting properties for its inputs. The interesting properties for the node itself must have been computed before. The node must then see how many of interesting properties it preserves and add its own.

Specified by:

computeInterestingPropertiesForInputs in class OptimizerNode

Parameters:

estimator - The CostEstimator instance to use for plan cost estimation.

getAlternativePlans

public List<PlanNode> getAlternativePlans(CostEstimator estimator)

Description copied from class: OptimizerNode

Computes the plan alternatives for this node, an implicitly for all nodes that are children of this node. This method must determine for each alternative the global and local properties and the costs. This method may recursively call getAlternatives() on its children to get their plan alternatives, and build its own alternatives on top of those.

Specified by:

getAlternativePlans in class OptimizerNode

Parameters:

estimator - The cost estimator used to estimate the costs of each plan alternative.

Returns:

A list containing all plan alternatives.

addLocalCandidates

protected void addLocalCandidates(Channel template1,
                                  Channel template2,
                                  List<Set<? extends NamedChannel>> broadcastPlanChannels,
                                  RequestedGlobalProperties rgps1,
                                  RequestedGlobalProperties rgps2,
                                  List<PlanNode> target,
                                  OperatorDescriptorDual.LocalPropertiesPair[] validLocalCombinations,
                                  CostEstimator estimator)

instantiate

protected void instantiate(OperatorDescriptorDual operator,
                           Channel in1,
                           Channel in2,
                           List<Set<? extends NamedChannel>> broadcastPlanChannels,
                           List<PlanNode> target,
                           CostEstimator estimator,
                           RequestedGlobalProperties globPropsReq1,
                           RequestedGlobalProperties globPropsReq2,
                           RequestedLocalProperties locPropsReq1,
                           RequestedLocalProperties locPropsReq2)

placePipelineBreakersIfNecessary

protected void placePipelineBreakersIfNecessary(org.apache.flink.runtime.operators.DriverStrategy strategy,
                                                Channel in1,
                                                Channel in2)

computeUnclosedBranchStack

public void computeUnclosedBranchStack()

Description copied from class: OptimizerNode

This method causes the node to compute the description of open branches in its sub-plan. An open branch describes, that a (transitive) child node had multiple outputs, which have not all been re-joined in the sub-plan. This method needs to set the openBranches field to a stack of unclosed branches, the latest one top. A branch is considered closed, if some later node sees all of the branching node's outputs, no matter if there have been more branches to different paths in the meantime.

Specified by:

computeUnclosedBranchStack in class OptimizerNode

getSemanticProperties

public org.apache.flink.api.common.operators.SemanticProperties getSemanticProperties()

Specified by:

getSemanticProperties in class OptimizerNode

getSemanticPropertiesForLocalPropertyFiltering

protected org.apache.flink.api.common.operators.SemanticProperties getSemanticPropertiesForLocalPropertyFiltering()

getSemanticPropertiesForGlobalPropertyFiltering

protected org.apache.flink.api.common.operators.SemanticProperties getSemanticPropertiesForGlobalPropertyFiltering()

accept

public void accept(org.apache.flink.util.Visitor<OptimizerNode> visitor)

Description copied from class: OptimizerNode

This method implements the visit of a depth-first graph traversing visitor. Implementers must first call the preVisit() method, then hand the visitor to their children, and finally call the postVisit() method.

Specified by:

accept in interface org.apache.flink.util.Visitable<OptimizerNode>

Specified by:

accept in class OptimizerNode

Parameters:

visitor - The graph traversing visitor.

See Also:

Visitable.accept(org.apache.flink.util.Visitor)