Package org.nd4j.linalg.dataset.api.iterator.fetcher

Class BaseDataFetcher

java.lang.Object

org.nd4j.linalg.dataset.api.iterator.fetcher.BaseDataFetcher

All Implemented Interfaces:

Serializable, DataSetFetcher

public abstract class BaseDataFetcher
extends Object
implements DataSetFetcher

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field	Description
protected DataSet	curr
protected int	cursor
protected int	inputColumns
protected static org.slf4j.Logger	log
protected int	numOutcomes
protected File	topLevelDir
protected int	totalExamples

Constructor Summary

Constructors

Constructor	Description
BaseDataFetcher()

Method Summary

Modifier and Type	Method	Description
protected INDArray	createInputMatrix(int numRows)	Creates a feature vector
protected INDArray	createOutputMatrix(int numRows)
protected INDArray	createOutputVector(int outcomeLabel)	Creates an output label matrix
int	cursor()	Direct access to a number represenative of iterating through a dataset
boolean	hasMore()	Whether the dataset has more to load
protected void	initializeCurrFromList(List<DataSet> examples)	Initializes this data transform fetcher from the passed in datasets
int	inputColumns()	The length of a feature vector for an individual example
DataSet	next()	Returns the next data applyTransformToDestination
void	reset()	Returns the fetcher back to the beginning of the dataset
void	setTopLevelDir(File topLevelDir)	For file download datasets this sets a top level directory outside the default
File	topLevelDir()	The top level directory to download datasets from This will usually be the default dl4j resources directory at user home/.deeplearning4j
int	totalExamples()	The total number of examples
int	totalOutcomes()	The number of labels for a dataset

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.nd4j.linalg.dataset.api.iterator.fetcher.DataSetFetcher

fetch

Field Detail

log

protected static final org.slf4j.Logger log

cursor

protected int cursor

numOutcomes

protected int numOutcomes

inputColumns

protected int inputColumns

curr

protected DataSet curr

totalExamples

protected int totalExamples

topLevelDir

protected File topLevelDir

Constructor Detail

BaseDataFetcher

public BaseDataFetcher()

Method Detail

topLevelDir

public File topLevelDir()

Description copied from interface: DataSetFetcher

The top level directory to download datasets from This will usually be the default dl4j resources directory at user home/.deeplearning4j

Specified by:

topLevelDir in interface DataSetFetcher

Returns:

setTopLevelDir

public void setTopLevelDir(File topLevelDir)

Description copied from interface: DataSetFetcher

For file download datasets this sets a top level directory outside the default

Specified by:

setTopLevelDir in interface DataSetFetcher

Parameters:

topLevelDir - the top level directory to use

createInputMatrix

protected INDArray createInputMatrix(int numRows)

Creates a feature vector

Parameters:

numRows - the number of examples

Returns:

a feature vector

createOutputVector

protected INDArray createOutputVector(int outcomeLabel)

Creates an output label matrix

Parameters:

outcomeLabel - the outcome label to use

Returns:

a binary vector where 1 is transform to the index specified by outcomeLabel

createOutputMatrix

protected INDArray createOutputMatrix(int numRows)

initializeCurrFromList

protected void initializeCurrFromList(List<DataSet> examples)

Initializes this data transform fetcher from the passed in datasets

Parameters:

examples - the examples to use

hasMore

public boolean hasMore()

Description copied from interface: DataSetFetcher

Whether the dataset has more to load

Specified by:

hasMore in interface DataSetFetcher

Returns:

whether the data applyTransformToDestination has more to load

next

public DataSet next()

Description copied from interface: DataSetFetcher

Returns the next data applyTransformToDestination

Specified by:

next in interface DataSetFetcher

Returns:

the next dataset

totalOutcomes

public int totalOutcomes()

Description copied from interface: DataSetFetcher

The number of labels for a dataset

Specified by:

totalOutcomes in interface DataSetFetcher

Returns:

the number of labels for a dataset

inputColumns

public int inputColumns()

Description copied from interface: DataSetFetcher

The length of a feature vector for an individual example

Specified by:

inputColumns in interface DataSetFetcher

Returns:

the length of a feature vector for an individual example

totalExamples

public int totalExamples()

Description copied from interface: DataSetFetcher

The total number of examples

Specified by:

totalExamples in interface DataSetFetcher

Returns:

the total number of examples

reset

public void reset()

Description copied from interface: DataSetFetcher

Returns the fetcher back to the beginning of the dataset

Specified by:

reset in interface DataSetFetcher

cursor

public int cursor()

Description copied from interface: DataSetFetcher

Direct access to a number represenative of iterating through a dataset

Specified by:

cursor in interface DataSetFetcher

Returns:

a cursor similar to an index