kafka.tier.tools

Class TierTopicMaterializationUtils

java.lang.Object

kafka.tier.tools.TierTopicMaterializationUtils

public class TierTopicMaterializationUtils
extends Object

TierTopicMaterializationUtils provides utils for dumping materialization events from tier topic for the given source topic(or all) for a give source partition(or all). Additionally it can recompute the materialized state and dump it too. This utils will be used by various tools for debugging, validation and recovering of tier states.

Field Summary

Fields

Modifier and Type	Field and Description
org.apache.kafka.clients.consumer.KafkaConsumer<byte[],byte[]>	consumer

Constructor Summary

Constructors

Constructor and Description
TierTopicMaterializationUtils(TierTopicMaterializationToolConfig config, Properties additionalConsumerProps, Map<TopicIdPartition,Long> offsetMap, kafka.utils.Scheduler scheduler)
TierTopicMaterializationUtils(TierTopicMaterializationToolConfig config, Properties additionalConsumerProps, kafka.utils.Scheduler scheduler)

Method Summary

Modifier and Type	Method and Description
long	getStartOffset(org.apache.kafka.common.TopicPartition tp) getStartOffset will return the beginning offset of a given TopicPartition 'tp'.
Path	getTierStateFile(TopicIdPartition id)
void	run()
void	setupConsumer(TierTopicMaterializationToolConfig config)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

consumer

public org.apache.kafka.clients.consumer.KafkaConsumer<byte[],byte[]> consumer

Constructor Detail

TierTopicMaterializationUtils

public TierTopicMaterializationUtils(TierTopicMaterializationToolConfig config,
                                     Properties additionalConsumerProps,
                                     Map<TopicIdPartition,Long> offsetMap,
                                     kafka.utils.Scheduler scheduler)

TierTopicMaterializationUtils

public TierTopicMaterializationUtils(TierTopicMaterializationToolConfig config,
                                     Properties additionalConsumerProps,
                                     kafka.utils.Scheduler scheduler)

Method Detail

setupConsumer

public void setupConsumer(TierTopicMaterializationToolConfig config)

run

public void run()
         throws IOException

Throws:

IOException

getTierStateFile

public Path getTierStateFile(TopicIdPartition id)

getStartOffset

public long getStartOffset(org.apache.kafka.common.TopicPartition tp)

getStartOffset will return the beginning offset of a given TopicPartition 'tp'. If unable to fetch the offset due to any reason, the method will return UNKNOWN_OFFSET.

Parameters:

tp - is TopicPartition

Returns:

beginning offset.