deepnetts.data

Class DataSets

java.lang.Object

deepnetts.data.DataSets

public class DataSets
extends Object

Data set utility methods ans constants.

Author:

Zoran Sevarac

Field Summary

Fields

Modifier and Type	Field and Description
static String	DELIMITER_COMMA
static String	DELIMITER_SEMICOLON
static String	DELIMITER_SPACE
static String	DELIMITER_TAB

Constructor Summary

Constructors

Constructor and Description
DataSets()

Method Summary

Modifier and Type	Method and Description
static CsvFormat	detectCsvFormat(String fileName)
static float[]	oneHotEncode(String hotLabel, String[] allLabels) Returns one hot encoded vector for the given label.
static TabularDataSet	readCsv(File csvFile, int numInputs, int numOutputs, boolean hasColumnNames, String delimiter) Creates and returns data set from specified CSV file.
static javax.visrec.ml.data.DataSet	readCsv(String fileName, int numInputs, int numOutputs) Create data set from CSV file, using coma (,) as default delimiter and no header (column names) in first row.
static TabularDataSet	readCsv(String fileName, int numInputs, int numOutputs, boolean hasColumnNames)
static TabularDataSet	readCsv(String fileName, int numInputs, int numOutputs, boolean hasColumnNames, String delimiter)
static TabularDataSet	readCsv(String fileName, int numInputs, int numOutputs, String delimiter)
static MaxScaler	scaleToMax(javax.visrec.ml.data.DataSet dataSet)
static MinMaxScaler	scaleToMinMax(javax.visrec.ml.data.DataSet dataSet)
static TrainTestPair	trainTestSplit(javax.visrec.ml.data.DataSet<?> dataSet, double split)

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DELIMITER_SPACE

public static final String DELIMITER_SPACE

See Also:

Constant Field Values

DELIMITER_COMMA

public static final String DELIMITER_COMMA

See Also:

Constant Field Values

DELIMITER_SEMICOLON

public static final String DELIMITER_SEMICOLON

See Also:

Constant Field Values

DELIMITER_TAB

public static final String DELIMITER_TAB

See Also:

Constant Field Values

Constructor Detail

DataSets

public DataSets()

Method Detail

readCsv

public static TabularDataSet readCsv(File csvFile,
                                     int numInputs,
                                     int numOutputs,
                                     boolean hasColumnNames,
                                     String delimiter)
                              throws FileNotFoundException,
                                     IOException

Creates and returns data set from specified CSV file. Empty lines are skipped

Parameters:

csvFile - CSV file

numInputs - number of input values in a row

numOutputs - number of output values in a row

hasColumnNames - true if first row contains column names

delimiter - delimiter character used to separate values in a row

Returns:

instance of data set with values loaded from file

Throws:

FileNotFoundException - if file was not found

IOException - if there was an error reading file TODO: Detect if there are labels in the first line, if there are no labels, set class1, class2, class3 in classifier evaluation! and detect type of attributes Move this method to some factory class or something? or as a default method in data set? TODO: Autodetetect delimiter; column type

readCsv

public static TabularDataSet readCsv(String fileName,
                                     int numInputs,
                                     int numOutputs,
                                     boolean hasColumnNames,
                                     String delimiter)
                              throws IOException

Throws:

IOException

readCsv

public static TabularDataSet readCsv(String fileName,
                                     int numInputs,
                                     int numOutputs,
                                     boolean hasColumnNames)
                              throws IOException

Throws:

IOException

readCsv

public static TabularDataSet readCsv(String fileName,
                                     int numInputs,
                                     int numOutputs,
                                     String delimiter)
                              throws IOException

Throws:

IOException

readCsv

public static javax.visrec.ml.data.DataSet readCsv(String fileName,
                                                   int numInputs,
                                                   int numOutputs)
                                            throws IOException

Create data set from CSV file, using coma (,) as default delimiter and no header (column names) in first row.

Parameters:

fileName - Name of the CSV file

numInputs - Number of input columns

numOutputs - Number of output columns

Returns:

Throws:

IOException

detectCsvFormat

public static CsvFormat detectCsvFormat(String fileName)
                                 throws FileNotFoundException,
                                        IOException

Throws:

FileNotFoundException

IOException

scaleToMax

public static MaxScaler scaleToMax(javax.visrec.ml.data.DataSet dataSet)

scaleToMinMax

public static MinMaxScaler scaleToMinMax(javax.visrec.ml.data.DataSet dataSet)

oneHotEncode

public static float[] oneHotEncode(String hotLabel,
                                   String[] allLabels)

Returns one hot encoded vector for the given label. One-Hot encoded vector is a binary array in which each position corresponds to one label, and all elements are zero, except the one which corresponds to hotLabel which has value of one. Index of hotLabel in allLabels array, determines which position in vector should be one. Vector size equals to the number of labels in allLabels array.

Parameters:

hotLabel - one label to encode

allLabels - all labels (used to determine size and hot position of encoded vector)

Returns:

one hot encoded vector for given label

trainTestSplit

public static TrainTestPair trainTestSplit(javax.visrec.ml.data.DataSet<?> dataSet,
                                           double split)