ai.grakn.engine.attribute.deduplicator

Class AttributeDeduplicatorDaemonImpl

java.lang.Object

ai.grakn.engine.attribute.deduplicator.AttributeDeduplicatorDaemonImpl

All Implemented Interfaces:

AttributeDeduplicatorDaemon

public class AttributeDeduplicatorDaemonImpl
extends Object
implements AttributeDeduplicatorDaemon

This class is responsible for de-duplicating attributes. It is done to ensure that every attribute in Grakn stays unique. Marking an attribute for deduplication: When the EmbeddedGraknTx.commit() is invoked, it will trigger the markForDeduplication(Keyspace, String, ConceptId) which inserts the attribute to an internal queue for deduplication. De-duplicating attributes in the de-duplicator daemon: The de-duplicator daemon is an always-on background thread which performs deduplication on incoming attributes. When a new attribute is inserted, it will immediately trigger the deduplicate operation, meaning that duplicates are merged in almost real-time speed. The daemon is started and stopped with startDeduplicationDaemon() and stopDeduplicationDaemon() Fault tolerance: The de-duplicator daemon is fault-tolerant and will re-process incoming attributes if Grakn crashes in the middle of a deduplication.

Author:

Ganeshwara Herawan Hananda

Constructor Summary

Constructors

Constructor and Description
AttributeDeduplicatorDaemonImpl(GraknConfig config, EngineGraknTxFactory txFactory) Instantiates AttributeDeduplicatorDaemonImpl

Method Summary

Modifier and Type	Method and Description
void	markForDeduplication(ai.grakn.Keyspace keyspace, String index, ai.grakn.concept.ConceptId conceptId) Marks an attribute for deduplication.
CompletableFuture<Void>	startDeduplicationDaemon() Starts a daemon which performs deduplication on incoming attributes in real-time.
void	stopDeduplicationDaemon() Stops the attribute uniqueness daemon

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AttributeDeduplicatorDaemonImpl

public AttributeDeduplicatorDaemonImpl(GraknConfig config,
                                       EngineGraknTxFactory txFactory)

Instantiates AttributeDeduplicatorDaemonImpl

Parameters:

config - a reference to an instance of GraknConfig which is initialised from a grakn.properties.

txFactory - an EngineGraknTxFactory instance which provides access to write into the database

Method Detail

markForDeduplication

public void markForDeduplication(ai.grakn.Keyspace keyspace,
                                 String index,
                                 ai.grakn.concept.ConceptId conceptId)

Marks an attribute for deduplication. The attribute will be inserted to an internal queue to be processed by the de-duplicator daemon in real-time. The attribute must have been inserted to the database, prior to calling this method.

Specified by:

markForDeduplication in interface AttributeDeduplicatorDaemon

Parameters:

keyspace - keyspace of the attribute

index - the value of the attribute

conceptId - the concept id of the attribute

startDeduplicationDaemon

public CompletableFuture<Void> startDeduplicationDaemon()

Starts a daemon which performs deduplication on incoming attributes in real-time. The thread listens to the RocksDbQueue queue for incoming attributes and applies the AttributeDeduplicator.deduplicate(EmbeddedGraknTx, KeyspaceIndexPair) algorithm.

Specified by:

startDeduplicationDaemon in interface AttributeDeduplicatorDaemon

stopDeduplicationDaemon

public void stopDeduplicationDaemon()

Stops the attribute uniqueness daemon

Specified by:

stopDeduplicationDaemon in interface AttributeDeduplicatorDaemon