ai.grakn.engine.attribute.deduplicator

Class AttributeDeduplicatorDaemon

java.lang.Object

ai.grakn.engine.attribute.deduplicator.AttributeDeduplicatorDaemon

public class AttributeDeduplicatorDaemon
extends Object

This class is responsible for de-duplicating attributes. It is done to ensure that every attribute in Grakn stays unique. Marking an attribute for deduplication: When the EmbeddedGraknTx.commit() is invoked, it will trigger the markForDeduplication(Keyspace, String, ConceptId) which inserts the attribute to an internal queue for deduplication. De-duplicating attributes in the de-duplicator daemon: The de-duplicator daemon is an always-on background thread which performs deduplication on incoming attributes. When a new attribute is inserted, it will immediately trigger the deduplicate operation, meaning that duplicates are merged in almost real-time speed. The daemon is started and stopped with startDeduplicationDaemon() and stopDeduplicationDaemon() Fault tolerance: The de-duplicator daemon is fault-tolerant and will re-process incoming attributes if Grakn crashes in the middle of a deduplication.

Author:

Ganeshwara Herawan Hananda

Constructor Summary

Constructors

Constructor and Description
AttributeDeduplicatorDaemon(GraknConfig config, EngineGraknTxFactory txFactory) Instantiates AttributeDeduplicatorDaemon

Method Summary

Modifier and Type	Method and Description
void	markForDeduplication(ai.grakn.Keyspace keyspace, String index, ai.grakn.concept.ConceptId conceptId) Marks an attribute for deduplication.
CompletableFuture<Void>	startDeduplicationDaemon() Starts a daemon which performs deduplication on incoming attributes in real-time.
void	stopDeduplicationDaemon() Stops the attribute uniqueness daemon

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AttributeDeduplicatorDaemon

public AttributeDeduplicatorDaemon(GraknConfig config,
                                   EngineGraknTxFactory txFactory)

Instantiates AttributeDeduplicatorDaemon

Parameters:

config - a reference to an instance of GraknConfig which is initialised from a grakn.properties.

txFactory - an EngineGraknTxFactory instance which provides access to write into the database

Method Detail

markForDeduplication

public void markForDeduplication(ai.grakn.Keyspace keyspace,
                                 String index,
                                 ai.grakn.concept.ConceptId conceptId)

Marks an attribute for deduplication. The attribute will be inserted to an internal queue to be processed by the de-duplicator daemon in real-time. The attribute must have been inserted to the database, prior to calling this method.

Parameters:

keyspace - keyspace of the attribute

index - the value of the attribute

conceptId - the concept id of the attribute

startDeduplicationDaemon

public CompletableFuture<Void> startDeduplicationDaemon()

Starts a daemon which performs deduplication on incoming attributes in real-time. The thread listens to the RocksDbQueue queue for incoming attributes and applies the AttributeDeduplicator.deduplicate(EngineGraknTxFactory, KeyspaceIndexPair) algorithm.

stopDeduplicationDaemon

public void stopDeduplicationDaemon()

Stops the attribute uniqueness daemon