org.apache.druid.metadata

Class SQLMetadataSegmentManager

java.lang.Object

org.apache.druid.metadata.SQLMetadataSegmentManager

All Implemented Interfaces:

MetadataSegmentManager

public class SQLMetadataSegmentManager
extends Object
implements MetadataSegmentManager

Constructor Summary

Constructors

Constructor and Description
SQLMetadataSegmentManager(com.fasterxml.jackson.databind.ObjectMapper jsonMapper, com.google.common.base.Supplier<MetadataSegmentManagerConfig> config, com.google.common.base.Supplier<MetadataStorageTablesConfig> dbTables, SQLMetadataConnector connector)

Method Summary

Modifier and Type	Method and Description
Collection<ImmutableDruidDataSource>	getImmutableDataSourcesWithAllUsedSegments() Returns a set of ImmutableDruidDataSource objects containing information about all used segments.
ImmutableDruidDataSource	getImmutableDataSourceWithUsedSegments(String dataSourceName) If there are used segments belonging to the given data source this method returns them as an ImmutableDruidDataSource object.
Set<SegmentId>	getOvershadowedSegments() Returns a set of overshadowed segment ids.
DataSourcesSnapshot	getSnapshotOfDataSourcesWithAllUsedSegments() Returns a snapshot of DruidDataSources and overshadowed segments
List<org.joda.time.Interval>	getUnusedSegmentIntervals(String dataSource, org.joda.time.DateTime maxEndTime, int limit) Returns top N unused segment intervals with the end time no later than the specified maxEndTime when ordered by segment start time, end time.
boolean	isPollingDatabasePeriodically()
Iterable<DataSegment>	iterateAllUsedSegments() Returns an iterable to go over all segments in all data sources.
int	markAsUnusedAllSegmentsInDataSource(String dataSource) Returns the number of segment entries in the database whose state was changed as the result of this call (that is, the segments were marked as unused).
int	markAsUnusedSegmentsInInterval(String dataSourceName, org.joda.time.Interval interval)
int	markAsUsedAllNonOvershadowedSegmentsInDataSource(String dataSource) Returns the number of segment entries in the database whose state was changed as the result of this call (that is, the segments were marked as used).
int	markAsUsedNonOvershadowedSegments(String dataSource, Set<String> segmentIds)
int	markAsUsedNonOvershadowedSegmentsInInterval(String dataSource, org.joda.time.Interval interval)
boolean	markSegmentAsUnused(String segmentId) This method does not update dataSourcesSnapshot, see the comments in doPoll() about snapshot update.
boolean	markSegmentAsUsed(String segmentId) Returns true if the state of the segment entry is changed in the database as the result of this call (that is, the segment was marked as used), false otherwise.
int	markSegmentsAsUnused(String dataSourceName, Set<String> segmentIds)
void	poll()
Set<String>	retrieveAllDataSourceNames() Retrieves all data source names for which there are segment in the database, regardless of whether those segments are used or not.
void	start() Don't confuse this method with startPollingDatabasePeriodically().
void	startPollingDatabasePeriodically()
void	stop() Don't confuse this method with stopPollingDatabasePeriodically().
void	stopPollingDatabasePeriodically()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SQLMetadataSegmentManager

@Inject
public SQLMetadataSegmentManager(com.fasterxml.jackson.databind.ObjectMapper jsonMapper,
                                         com.google.common.base.Supplier<MetadataSegmentManagerConfig> config,
                                         com.google.common.base.Supplier<MetadataStorageTablesConfig> dbTables,
                                         SQLMetadataConnector connector)

Method Detail

start

public void start()

Don't confuse this method with startPollingDatabasePeriodically(). This is a lifecycle starting method to be executed just once for an instance of SqlSegmentsMetadata.

stop

public void stop()

Don't confuse this method with stopPollingDatabasePeriodically(). This is a lifecycle stopping method to be executed just once for an instance of SqlSegmentsMetadata.

startPollingDatabasePeriodically

public void startPollingDatabasePeriodically()

Specified by:

startPollingDatabasePeriodically in interface MetadataSegmentManager

isPollingDatabasePeriodically

public boolean isPollingDatabasePeriodically()

Specified by:

isPollingDatabasePeriodically in interface MetadataSegmentManager

stopPollingDatabasePeriodically

public void stopPollingDatabasePeriodically()

Specified by:

stopPollingDatabasePeriodically in interface MetadataSegmentManager

markSegmentAsUsed

public boolean markSegmentAsUsed(String segmentId)

Description copied from interface: MetadataSegmentManager

Returns true if the state of the segment entry is changed in the database as the result of this call (that is, the segment was marked as used), false otherwise. If the call results in a database error, an exception is relayed to the caller.

Specified by:

markSegmentAsUsed in interface MetadataSegmentManager

markAsUsedAllNonOvershadowedSegmentsInDataSource

public int markAsUsedAllNonOvershadowedSegmentsInDataSource(String dataSource)

Description copied from interface: MetadataSegmentManager

Returns the number of segment entries in the database whose state was changed as the result of this call (that is, the segments were marked as used). If the call results in a database error, an exception is relayed to the caller.

Specified by:

markAsUsedAllNonOvershadowedSegmentsInDataSource in interface MetadataSegmentManager

markAsUsedNonOvershadowedSegmentsInInterval

public int markAsUsedNonOvershadowedSegmentsInInterval(String dataSource,
                                                       org.joda.time.Interval interval)

Specified by:

markAsUsedNonOvershadowedSegmentsInInterval in interface MetadataSegmentManager

markAsUsedNonOvershadowedSegments

public int markAsUsedNonOvershadowedSegments(String dataSource,
                                             Set<String> segmentIds)
                                      throws UnknownSegmentIdException

Specified by:

markAsUsedNonOvershadowedSegments in interface MetadataSegmentManager

Throws:

UnknownSegmentIdException

markAsUnusedAllSegmentsInDataSource

public int markAsUnusedAllSegmentsInDataSource(String dataSource)

Description copied from interface: MetadataSegmentManager

Returns the number of segment entries in the database whose state was changed as the result of this call (that is, the segments were marked as unused). If the call results in a database error, an exception is relayed to the caller.

Specified by:

markAsUnusedAllSegmentsInDataSource in interface MetadataSegmentManager

markSegmentAsUnused

public boolean markSegmentAsUnused(String segmentId)

This method does not update dataSourcesSnapshot, see the comments in doPoll() about snapshot update. The update of the segment's state will be reflected after the next DatabasePoll.

Specified by:

markSegmentAsUnused in interface MetadataSegmentManager

markSegmentsAsUnused

public int markSegmentsAsUnused(String dataSourceName,
                                Set<String> segmentIds)

Specified by:

markSegmentsAsUnused in interface MetadataSegmentManager

markAsUnusedSegmentsInInterval

public int markAsUnusedSegmentsInInterval(String dataSourceName,
                                          org.joda.time.Interval interval)

Specified by:

markAsUnusedSegmentsInInterval in interface MetadataSegmentManager

getImmutableDataSourceWithUsedSegments

@Nullable
public ImmutableDruidDataSource getImmutableDataSourceWithUsedSegments(String dataSourceName)

Description copied from interface: MetadataSegmentManager

If there are used segments belonging to the given data source this method returns them as an ImmutableDruidDataSource object. If there are no used segments belonging to the given data source this method returns null.

Specified by:

getImmutableDataSourceWithUsedSegments in interface MetadataSegmentManager

getImmutableDataSourcesWithAllUsedSegments

public Collection<ImmutableDruidDataSource> getImmutableDataSourcesWithAllUsedSegments()

Description copied from interface: MetadataSegmentManager

Returns a set of ImmutableDruidDataSource objects containing information about all used segments. ImmutableDruidDataSource objects in the returned collection are unique. If there are no used segments, this method returns an empty collection.

Specified by:

getImmutableDataSourcesWithAllUsedSegments in interface MetadataSegmentManager

getOvershadowedSegments

public Set<SegmentId> getOvershadowedSegments()

Description copied from interface: MetadataSegmentManager

Returns a set of overshadowed segment ids.

Specified by:

getOvershadowedSegments in interface MetadataSegmentManager

getSnapshotOfDataSourcesWithAllUsedSegments

public DataSourcesSnapshot getSnapshotOfDataSourcesWithAllUsedSegments()

Description copied from interface: MetadataSegmentManager

Returns a snapshot of DruidDataSources and overshadowed segments

Specified by:

getSnapshotOfDataSourcesWithAllUsedSegments in interface MetadataSegmentManager

iterateAllUsedSegments

public Iterable<DataSegment> iterateAllUsedSegments()

Description copied from interface: MetadataSegmentManager

Returns an iterable to go over all segments in all data sources. The order in which segments are iterated is unspecified. Note: the iteration may not be as trivially cheap as, for example, iteration over an ArrayList. Try (to some reasonable extent) to organize the code so that it iterates the returned iterable only once rather than several times.

Specified by:

iterateAllUsedSegments in interface MetadataSegmentManager

retrieveAllDataSourceNames

public Set<String> retrieveAllDataSourceNames()

Description copied from interface: MetadataSegmentManager

Retrieves all data source names for which there are segment in the database, regardless of whether those segments are used or not. If there are no segments in the database, returns an empty set. Performance warning: this method makes a query into the database. This method might return a different set of data source names than may be observed via MetadataSegmentManager.getImmutableDataSourcesWithAllUsedSegments() method. This method will include a data source name even if there are no used segments belonging to it, while MetadataSegmentManager.getImmutableDataSourcesWithAllUsedSegments() won't return such a data source.

Specified by:

retrieveAllDataSourceNames in interface MetadataSegmentManager

poll

public void poll()

Specified by:

poll in interface MetadataSegmentManager

getUnusedSegmentIntervals

public List<org.joda.time.Interval> getUnusedSegmentIntervals(String dataSource,
                                                              org.joda.time.DateTime maxEndTime,
                                                              int limit)

Description copied from interface: MetadataSegmentManager

Returns top N unused segment intervals with the end time no later than the specified maxEndTime when ordered by segment start time, end time.

Specified by:

getUnusedSegmentIntervals in interface MetadataSegmentManager