org.apache.flink.cep

Class PatternStream<T>

java.lang.Object

org.apache.flink.cep.PatternStream<T>

Type Parameters:

T - Type of the events

public class PatternStream<T>
extends Object

Stream abstraction for CEP pattern detection. A pattern stream is a stream which emits detected pattern sequences as a map of events associated with their names. The pattern is detected using a NFA. In order to process the detected sequences, the user has to specify a PatternSelectFunction or a PatternFlatSelectFunction.

Additionally it allows to handle partially matched event patterns which have timed out. For this the user has to specify a PatternTimeoutFunction or a PatternFlatTimeoutFunction.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	PatternStream.CoMapTimeout<R,L> Used for joining results from timeout side-output for API backward compatibility.

Method Summary

Modifier and Type	Method and Description
<L,R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<R>	flatSelect(org.apache.flink.util.OutputTag<L> timedOutPartialMatchesTag, PatternFlatTimeoutFunction<T,L> patternFlatTimeoutFunction, PatternFlatSelectFunction<T,R> patternFlatSelectFunction) Applies a flat select function to the detected pattern sequence.
<L,R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<R>	flatSelect(org.apache.flink.util.OutputTag<L> timedOutPartialMatchesTag, PatternFlatTimeoutFunction<T,L> patternFlatTimeoutFunction, org.apache.flink.api.common.typeinfo.TypeInformation<R> outTypeInfo, PatternFlatSelectFunction<T,R> patternFlatSelectFunction) Applies a flat select function to the detected pattern sequence.
<R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<R>	flatSelect(PatternFlatSelectFunction<T,R> patternFlatSelectFunction) Applies a flat select function to the detected pattern sequence.
<R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<R>	flatSelect(PatternFlatSelectFunction<T,R> patternFlatSelectFunction, org.apache.flink.api.common.typeinfo.TypeInformation<R> outTypeInfo) Applies a flat select function to the detected pattern sequence.
<L,R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<org.apache.flink.types.Either<L,R>>	flatSelect(PatternFlatTimeoutFunction<T,L> patternFlatTimeoutFunction, PatternFlatSelectFunction<T,R> patternFlatSelectFunction) Deprecated. Use flatSelect(OutputTag, PatternFlatTimeoutFunction, PatternFlatSelectFunction) that returns timed out events as a side-output
PatternStream<T>	inEventTime() Sets the time characteristic to event time.
PatternStream<T>	inProcessingTime() Sets the time characteristic to processing time.
<R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<R>	process(PatternProcessFunction<T,R> patternProcessFunction) Applies a process function to the detected pattern sequence.
<R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<R>	process(PatternProcessFunction<T,R> patternProcessFunction, org.apache.flink.api.common.typeinfo.TypeInformation<R> outTypeInfo) Applies a process function to the detected pattern sequence.
<L,R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<R>	select(org.apache.flink.util.OutputTag<L> timedOutPartialMatchesTag, PatternTimeoutFunction<T,L> patternTimeoutFunction, PatternSelectFunction<T,R> patternSelectFunction) Applies a select function to the detected pattern sequence.
<L,R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<R>	select(org.apache.flink.util.OutputTag<L> timedOutPartialMatchesTag, PatternTimeoutFunction<T,L> patternTimeoutFunction, org.apache.flink.api.common.typeinfo.TypeInformation<R> outTypeInfo, PatternSelectFunction<T,R> patternSelectFunction) Applies a select function to the detected pattern sequence.
<R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<R>	select(PatternSelectFunction<T,R> patternSelectFunction) Applies a select function to the detected pattern sequence.
<R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<R>	select(PatternSelectFunction<T,R> patternSelectFunction, org.apache.flink.api.common.typeinfo.TypeInformation<R> outTypeInfo) Applies a select function to the detected pattern sequence.
<L,R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<org.apache.flink.types.Either<L,R>>	select(PatternTimeoutFunction<T,L> patternTimeoutFunction, PatternSelectFunction<T,R> patternSelectFunction) Deprecated. Use select(OutputTag, PatternTimeoutFunction, PatternSelectFunction) that returns timed out events as a side-output
PatternStream<T>	sideOutputLateData(org.apache.flink.util.OutputTag<T> lateDataOutputTag) Send late arriving data to the side output identified by the given OutputTag.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

sideOutputLateData

public PatternStream<T> sideOutputLateData(org.apache.flink.util.OutputTag<T> lateDataOutputTag)

Send late arriving data to the side output identified by the given OutputTag. A record is considered late after the watermark has passed its timestamp.

You can get the stream of late data using SingleOutputStreamOperator.getSideOutput(OutputTag) on the SingleOutputStreamOperator resulting from the pattern processing operations.

inProcessingTime

public PatternStream<T> inProcessingTime()

Sets the time characteristic to processing time.

inEventTime

public PatternStream<T> inEventTime()

Sets the time characteristic to event time.

process

public <R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<R> process(PatternProcessFunction<T,R> patternProcessFunction)

Applies a process function to the detected pattern sequence. For each pattern sequence the provided PatternProcessFunction is called. In order to process timed out partial matches as well one can use TimedOutPartialMatchHandler as additional interface.

Type Parameters:

R - Type of the resulting elements

Parameters:

patternProcessFunction - The pattern process function which is called for each detected pattern sequence.

Returns:

DataStream which contains the resulting elements from the pattern process function.

process

public <R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<R> process(PatternProcessFunction<T,R> patternProcessFunction,
                                                                                           org.apache.flink.api.common.typeinfo.TypeInformation<R> outTypeInfo)

Applies a process function to the detected pattern sequence. For each pattern sequence the provided PatternProcessFunction is called. In order to process timed out partial matches as well one can use TimedOutPartialMatchHandler as additional interface.

Type Parameters:

R - Type of the resulting elements

Parameters:

patternProcessFunction - The pattern process function which is called for each detected pattern sequence.

outTypeInfo - Explicit specification of output type.

Returns:

DataStream which contains the resulting elements from the pattern process function.

select

public <R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<R> select(PatternSelectFunction<T,R> patternSelectFunction)

Applies a select function to the detected pattern sequence. For each pattern sequence the provided PatternSelectFunction is called. The pattern select function can produce exactly one resulting element.

Type Parameters:

R - Type of the resulting elements

Parameters:

patternSelectFunction - The pattern select function which is called for each detected pattern sequence.

Returns:

DataStream which contains the resulting elements from the pattern select function.

select

public <R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<R> select(PatternSelectFunction<T,R> patternSelectFunction,
                                                                                          org.apache.flink.api.common.typeinfo.TypeInformation<R> outTypeInfo)

Applies a select function to the detected pattern sequence. For each pattern sequence the provided PatternSelectFunction is called. The pattern select function can produce exactly one resulting element.

Type Parameters:

R - Type of the resulting elements

Parameters:

patternSelectFunction - The pattern select function which is called for each detected pattern sequence.

outTypeInfo - Explicit specification of output type.

Returns:

DataStream which contains the resulting elements from the pattern select function.

select

public <L,R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<R> select(org.apache.flink.util.OutputTag<L> timedOutPartialMatchesTag,
                                                                                            PatternTimeoutFunction<T,L> patternTimeoutFunction,
                                                                                            PatternSelectFunction<T,R> patternSelectFunction)

Applies a select function to the detected pattern sequence. For each pattern sequence the provided PatternSelectFunction is called. The pattern select function can produce exactly one resulting element.

Applies a timeout function to a partial pattern sequence which has timed out. For each partial pattern sequence the provided PatternTimeoutFunction is called. The pattern timeout function can produce exactly one resulting element.

You can get the stream of timed-out data resulting from the SingleOutputStreamOperator.getSideOutput(OutputTag) on the SingleOutputStreamOperator resulting from the select operation with the same OutputTag.

Type Parameters:

L - Type of the resulting timeout elements

R - Type of the resulting elements

Parameters:

timedOutPartialMatchesTag - OutputTag that identifies side output with timed out patterns

patternTimeoutFunction - The pattern timeout function which is called for each partial pattern sequence which has timed out.

patternSelectFunction - The pattern select function which is called for each detected pattern sequence.

Returns:

DataStream which contains the resulting elements with the resulting timeout elements in a side output.

select

public <L,R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<R> select(org.apache.flink.util.OutputTag<L> timedOutPartialMatchesTag,
                                                                                            PatternTimeoutFunction<T,L> patternTimeoutFunction,
                                                                                            org.apache.flink.api.common.typeinfo.TypeInformation<R> outTypeInfo,
                                                                                            PatternSelectFunction<T,R> patternSelectFunction)

Applies a select function to the detected pattern sequence. For each pattern sequence the provided PatternSelectFunction is called. The pattern select function can produce exactly one resulting element.

Applies a timeout function to a partial pattern sequence which has timed out. For each partial pattern sequence the provided PatternTimeoutFunction is called. The pattern timeout function can produce exactly one resulting element.

You can get the stream of timed-out data resulting from the SingleOutputStreamOperator.getSideOutput(OutputTag) on the SingleOutputStreamOperator resulting from the select operation with the same OutputTag.

Type Parameters:

L - Type of the resulting timeout elements

R - Type of the resulting elements

Parameters:

timedOutPartialMatchesTag - OutputTag that identifies side output with timed out patterns

patternTimeoutFunction - The pattern timeout function which is called for each partial pattern sequence which has timed out.

outTypeInfo - Explicit specification of output type.

patternSelectFunction - The pattern select function which is called for each detected pattern sequence.

Returns:

DataStream which contains the resulting elements with the resulting timeout elements in a side output.

select

@Deprecated
public <L,R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<org.apache.flink.types.Either<L,R>> select(PatternTimeoutFunction<T,L> patternTimeoutFunction,
                                                                                                                                         PatternSelectFunction<T,R> patternSelectFunction)

Deprecated. Use select(OutputTag, PatternTimeoutFunction, PatternSelectFunction) that returns timed out events as a side-output

Applies a select function to the detected pattern sequence. For each pattern sequence the provided PatternSelectFunction is called. The pattern select function can produce exactly one resulting element.

Applies a timeout function to a partial pattern sequence which has timed out. For each partial pattern sequence the provided PatternTimeoutFunction is called. The pattern timeout function can produce exactly one resulting element.

Type Parameters:

L - Type of the resulting timeout elements

R - Type of the resulting elements

Parameters:

patternTimeoutFunction - The pattern timeout function which is called for each partial pattern sequence which has timed out.

patternSelectFunction - The pattern select function which is called for each detected pattern sequence.

Returns:

DataStream which contains the resulting elements or the resulting timeout elements wrapped in an Either type.

flatSelect

public <R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<R> flatSelect(PatternFlatSelectFunction<T,R> patternFlatSelectFunction)

Applies a flat select function to the detected pattern sequence. For each pattern sequence the provided PatternFlatSelectFunction is called. The pattern flat select function can produce an arbitrary number of resulting elements.

Type Parameters:

R - Type of the resulting elements

Parameters:

patternFlatSelectFunction - The pattern flat select function which is called for each detected pattern sequence.

Returns:

DataStream which contains the resulting elements from the pattern flat select function.

flatSelect

public <R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<R> flatSelect(PatternFlatSelectFunction<T,R> patternFlatSelectFunction,
                                                                                              org.apache.flink.api.common.typeinfo.TypeInformation<R> outTypeInfo)

Applies a flat select function to the detected pattern sequence. For each pattern sequence the provided PatternFlatSelectFunction is called. The pattern flat select function can produce an arbitrary number of resulting elements.

Type Parameters:

R - Type of the resulting elements

Parameters:

patternFlatSelectFunction - The pattern flat select function which is called for each detected pattern sequence.

outTypeInfo - Explicit specification of output type.

Returns:

DataStream which contains the resulting elements from the pattern flat select function.

flatSelect

public <L,R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<R> flatSelect(org.apache.flink.util.OutputTag<L> timedOutPartialMatchesTag,
                                                                                                PatternFlatTimeoutFunction<T,L> patternFlatTimeoutFunction,
                                                                                                PatternFlatSelectFunction<T,R> patternFlatSelectFunction)

Applies a flat select function to the detected pattern sequence. For each pattern sequence the provided PatternFlatSelectFunction is called. The pattern select function can produce exactly one resulting element.

Applies a timeout function to a partial pattern sequence which has timed out. For each partial pattern sequence the provided PatternFlatTimeoutFunction is called. The pattern timeout function can produce exactly one resulting element.

You can get the stream of timed-out data resulting from the SingleOutputStreamOperator.getSideOutput(OutputTag) on the SingleOutputStreamOperator resulting from the select operation with the same OutputTag.

Type Parameters:

L - Type of the resulting timeout elements

R - Type of the resulting elements

Parameters:

timedOutPartialMatchesTag - OutputTag that identifies side output with timed out patterns

patternFlatTimeoutFunction - The pattern timeout function which is called for each partial pattern sequence which has timed out.

patternFlatSelectFunction - The pattern select function which is called for each detected pattern sequence.

Returns:

DataStream which contains the resulting elements with the resulting timeout elements in a side output.

flatSelect

public <L,R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<R> flatSelect(org.apache.flink.util.OutputTag<L> timedOutPartialMatchesTag,
                                                                                                PatternFlatTimeoutFunction<T,L> patternFlatTimeoutFunction,
                                                                                                org.apache.flink.api.common.typeinfo.TypeInformation<R> outTypeInfo,
                                                                                                PatternFlatSelectFunction<T,R> patternFlatSelectFunction)

Applies a flat select function to the detected pattern sequence. For each pattern sequence the provided PatternFlatSelectFunction is called. The pattern select function can produce exactly one resulting element.

Applies a timeout function to a partial pattern sequence which has timed out. For each partial pattern sequence the provided PatternFlatTimeoutFunction is called. The pattern timeout function can produce exactly one resulting element.

You can get the stream of timed-out data resulting from the SingleOutputStreamOperator.getSideOutput(OutputTag) on the SingleOutputStreamOperator resulting from the select operation with the same OutputTag.

Type Parameters:

L - Type of the resulting timeout elements

R - Type of the resulting elements

Parameters:

timedOutPartialMatchesTag - OutputTag that identifies side output with timed out patterns

patternFlatTimeoutFunction - The pattern timeout function which is called for each partial pattern sequence which has timed out.

patternFlatSelectFunction - The pattern select function which is called for each detected pattern sequence.

outTypeInfo - Explicit specification of output type.

Returns:

DataStream which contains the resulting elements with the resulting timeout elements in a side output.

flatSelect

@Deprecated
public <L,R> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<org.apache.flink.types.Either<L,R>> flatSelect(PatternFlatTimeoutFunction<T,L> patternFlatTimeoutFunction,
                                                                                                                                             PatternFlatSelectFunction<T,R> patternFlatSelectFunction)

Deprecated. Use flatSelect(OutputTag, PatternFlatTimeoutFunction, PatternFlatSelectFunction) that returns timed out events as a side-output

Applies a flat select function to the detected pattern sequence. For each pattern sequence the provided PatternFlatSelectFunction is called. The pattern flat select function can produce an arbitrary number of resulting elements.

Applies a timeout function to a partial pattern sequence which has timed out. For each partial pattern sequence the provided PatternFlatTimeoutFunction is called. The pattern timeout function can produce an arbitrary number of resulting elements.

Type Parameters:

L - Type of the resulting timeout events

R - Type of the resulting events

Parameters:

patternFlatTimeoutFunction - The pattern flat timeout function which is called for each partial pattern sequence which has timed out.

patternFlatSelectFunction - The pattern flat select function which is called for each detected pattern sequence.

Returns:

DataStream which contains the resulting events from the pattern flat select function or the resulting timeout events from the pattern flat timeout function wrapped in an Either type.