How to tutorials

Cloud Classification using Deep Learning in Java

03 Nov

Cloud classification is the task of assigning cloud images to corresponding types/categories. Cloud image classification is important for many applications, including climate modeling, weather prediction, solar energy production, satellite imaging, satellite communication, and many others. This tutorial will guide you on how to create an advanced deep learning model for image classification, even if you’re not an expert in image analysis and deep learning.

Data Set

SWIMCAT dataset (Singapore Whole sky IMaging CATegories Database) contains 784 images of sky/cloud patches, categorized into 5 distinct categories: 1) clear sky, 2) patterned clouds, 3) thick dark clouds, 4) thick white clouds, and 5) veil clouds. Representative sample images from each category are shown below.

Cloud Classification

More details about the dataset can be found on http://vintage.winklerbros.net/swimcat.html and the original paper: S. Dev, Y. H. Lee, S. Winkler. Categorization of cloud image patches using an improved texton-based approach. Proc. IEEE International Conference on Image Processing (ICIP), Québec City, Canada, Sep. 27-30, 2015. The dataset is released under a creative commons license

Convolutional Neural Network

The cloud image classification is performed using convolutional neural networks. A convolutional neural network is a specific deep learning technique that is commonly used for image classification. Convolutional network classifies clouds into one of the five categories, based on the patterns detected in image pixels, that are used as inputs for the neural network. The great thing about using convolutional networks is that patterns in image pixels can be automatically discovered (learned) during the so-called training process. You can find more details about how convolutional networks work in this tutorial.

Step by step How To

  1. Download the Deep Netts. Unpack the downloaded zip file.
  2. Download the clouds image data set Unpack the downloaded zip file
  3. Start the Deep Netts IDE (for details look at Getting Started tutorial in the downloaded package)
  4. Create a new Deep Netts project by clicking [File > New Project] in the main menu, enter the project name, and click [Next]
  5. Choose [Classify Images] option and click [Next]
  6. Choose [Multi Class Classification] option (since there are more than two (five) categories of images). Click [Next]
  7. Specify the image directory to the unpacked downloaded image set. Wizard will automatically detect or generate category labels image index. Click [Next].
  8. Set image data preprocessing option: uncheck invert pixels, check Standardize (Zero Mean) option, and Upsample to balance classes option (since there is a different number of images in each class). Note that these settings specify how to prepare images for training and can have a significant impact on the results. Click [Next]
  9. Specify small convolutional neural network architecture (add 2 convolutional layers with 6 and 12 channels that perform image filtering and feature detection, max-pooling layer after each convolutional layer that scales down an image, and two fully connected layers with width 48 that helps with classification). Other settings have been preselected by the expert wizard for this type of task. Click [Next]
  10. Set main training parameters that include network architecture and training/test set split options (you can accept settings provided by the expert wizard). Click [Next].
  11. Set additional training algorithm parameters (or accept commonly used default settings provided by the expert wizard ). Click Next.
  12. Run Training by right-clicking the training file and then select [Run Training File] pop-up menu option. If you wish you can check [Visualize Layers] in the training dialog option to be able to observe layer activity during the training. You can also observe network outputs, error, and accuracy graphs and inspect detailed logs for each training iteration (aka epoch). These visualizations can be helpful to debug network training.
  13. At the end of the training, you can analyze the training process and test metrics in log (confusion matrix, accuracy, precision, recall). An entire log file will be available for later inspection and comparison in [Training Logs] project folder.
  14. The trained convolutional network is available as a serialized object in [Trained Networks] project folder (>)
    You can also generate .jar file which contains a trained neural network using the Create Jar Option from the right click menu.
  15. To test the classification of the trained model with the entire data set use [Tools > Test Image Set] in the main menu
  16. You can use the trained network in your Java application using the Deep Netts library which is also available as a part of the download package.

To use the trained convolutional network in your Java app, use the following Java code:

// Load trained convolutional network
ConvolutionalNetwork trainedNet=  FileIO.createFromFile("trainedNetwork.dnet", ConvolutionalNetwork.class);
// turn on image preprocessing used during the training
((ImagePreprocessing)trainedNet.getPreprocessing()).setEnabled(true);	
// Create image classifier based on trained network and VisRec API
ImageClassifierNetwork imageClassifier = new ImageClassifierNetwork (trainedNet);
// load image to classify
BufferedImage image = ImageIO.read(new File("someImage.jpg"));
// Classify new image and get probabilities for all possible classes
Map<String, Float> results = imageClassifier.classify(image);

Why Deep Netts

Deep Netts is a perfect fit for mid and small-scale machine learning problems (in terms of data) with large-scale on-premise deployment on heterogeneous devices. It is easy to integrate, distribute, and maintain which enables sustainable and cost-effective innovation of Java applications. Along with it’s deep learning IDE, it provides a great user experience for users with a software development background.

Resources

Learn More