Here we will demonstrate the working of Actual Artificial Neural Networks using classic examples of digit image recognition, also known as the “hello world” of Neural Network. We will specify the working process by building a model with already existing data named MNIST, training the model with it, and then predicting the output with a set of new data. As you might be familiar that when it comes to neural networks everything is all about those layers, So our main area of focus will be these layers as they are the core process and are as follows:

Architecture of Artificial Neural Network

Role of Layers

Input Layer: Consider the following that for digit 8, we have 25*25 images with 625 pixels in total. This represents that for input there has to be huge data but the output is going to be a single digit. Each pixel has a number of grayscale values that range from 0 to 1, similarly to neurons that hold a number as values. Values with activations 1 are white, whereas values with activations 0 are black. When its activation value is high, as shown below, the image is illuminated, which collectively represents the number 8. In this case, the input layer is simply the total amount of pixels in the input digit image 8, which is 625 pixels in this case. This layer will activate specific pixels based on the user input layer’s gradient scale values to the hidden layer.

In our case when the 625 pixels are fed to input layers some pixels are black and some are white. Those white pixels have some edges that associate with the hidden layer in order to generate some sort of patterns which further can correspond with these sub components to detect some shapes. At the final layer the brightest neuron is considered to be as predicted outcome and possibly closest to the actual output.

Input Image 25*25 px

Hidden layer: This layer is considered the heart of networks, since it does the majority of the processing. So If you recall how neurons work, where some specific groups of neurons fire that cause the chain reaction to fire certain other neurons which are somehow related. In our case in input layers we have all the 625 pixels of digit 8 which initiated the process by activating similar pixels that can somehow be related to digit 8. If you notice what these layers are actually doing, they are trying to find out the similarities from pixels in between given input 8 and hidden layers. For this it takes help of edges of the input, patterns of it and tries to find out the shape associated with the output that makes the hidden layers progress further in the same manner. In order to get to know how those middle layers work we need to go deep into our minds and its working. Have you ever wondered, How our mind comes to the conclusion that the given image is something, in this case it’s 8. Well it simply breaks down the layers of abstraction followed by feature extraction that we will see in details below.

Feature extraction

The goal of our neural network should be to find a way of approaching to find out the features mentioned below through some process. The best way to do this is to assign weights to the networking chain of all the permutations and combinations between layers. So in this case we must assign some weights to those connections between pixels.

 

Weights

The weights simply reflects the strong connection between the perceptron /nodes. This actually represents the impact factor that one perceptron has on another or not having at all to start the chain reaction for the operation. In case we haven’t got the desire output it these weights who are then  fined tuned in order to get things on track.

 

Biased

 

It is possible to think of bias in neural networks as analogous to a constant in a linear function, where the line is effectively transposed by the constant value. The input to the activation function in the absence of bias is ‘x’ multiplied by the connection weight ‘w0’.

Now we have values of gradient scales on pixels and weights assigned to the network in order to compare and find out the neighboring associated pixels for the following processes.

 

Edges:

Consider this as a hidden layer and let’s call it an edge hidden layer where the neural network works in such a way that it finds the most promising associate neighbor pixel with the help of weights and biases that are assigned to it that we have seen earlier. If You observe carefully you can technically look into the working of the brain in terms of this specific content. This works similarly to neuron as it will collect the fragmented data and try to get them back together in order to find some related content in it. It does it too many times until it doesn’t get close to something sensible.

 

Patterns:

Here the process is similar as the previous only difference is that it is taking the input data from the previous edge hidden layer. As below you can see that there is some pattern which is detectible partially. The output that you are looking at came from the edge layer by putting all the associated inputs collectively together. If we are on the same page we can see those edges combined together are making some kind of loop which is remarkable as without considering the case we want to predict our input 8. As you know, eight digit shares of two loops are put together below each other.

Shapes:

By following the same procedure and taking input from the pattern layer to perform the neural network on these layers you will clearly see that with this we can easily detect the digit 8 by collaboration of all the output from the patter layer with the help of weights assigned to them.

With results like this in the final layer it becomes easy for the neural network to predict the final outcome. It just had to activate the final neuron which is likely to be more close to the actual outcome with minimum error rate and idle accuracy rate.

 

Output Layer: The Output layer is so obvious that it ranges from 0-9 and already has been fed to the model with input data. The outputs are also assigned those same grayscale values in order to let the system think how similar the generated output is with actual output with minimum error called residual. The most bright gradient lit up pixels are going to be considered as the idle output by the neural networks. The last layer will fire the neurons to one of the below mentioned outputs which was prior known and fed to the module. Once the prediction is done we will get the output as 8 finally.

 

The final output layer

This particular section of code focuses on calculating the Accuracy named: AccuracyCalculator.kt. we will calculate the accuracy by performing numbers of iterations using ML/DL algorithms here

				
					// package has been imported and initialised
package gaurav.cnn
// all the libraries that are required has been imported here
// some of them are classes created and deep learning 4j libraries with ML and DL
import java.lang.Thread
import kotlin.Throws
import java.io.IOException
import kotlin.jvm.JvmStatic
import gaurav.cnn.EdgeDetection
import java.awt.image.BufferedImage
import javax.imageio.ImageIO
import gaurav.cnn.MnistExample
import org.nd4j.linalg.dataset.api.iterator.DataSetIterator
import org.deeplearning4j.datasets.iterator.impl.MnistDataSetIterator
import org.deeplearning4j.nn.conf.MultiLayerConfiguration
import org.deeplearning4j.nn.conf.NeuralNetConfiguration
import org.deeplearning4j.nn.conf.LearningRatePolicy
import org.deeplearning4j.nn.weights.WeightInit
import org.deeplearning4j.nn.api.OptimizationAlgorithm
import org.nd4j.linalg.lossfunctions.LossFunctions
import org.deeplearning4j.nn.multilayer.MultiLayerNetwork
import org.deeplearning4j.optimize.listeners.ScoreIterationListener
import org.deeplearning4j.util.ModelSerializer
import org.deeplearning4j.earlystopping.scorecalc.ScoreCalculator
import gaurav.cnn.AccuracyCalculator
import gaurav.cnn.ConvolutionalNeuralNetwork
import gaurav.data.LabeledImage
import org.nd4j.linalg.factory.Nd4j
import org.deeplearning4j.earlystopping.EarlyStoppingConfiguration
import org.deeplearning4j.earlystopping.termination.MaxEpochsTerminationCondition
import org.deeplearning4j.earlystopping.termination.MaxTimeIterationTerminationCondition
import java.util.concurrent.TimeUnit
import org.deeplearning4j.earlystopping.saver.LocalFileModelSaver
import org.deeplearning4j.earlystopping.trainer.EarlyStoppingTrainer
import org.deeplearning4j.earlystopping.EarlyStoppingResult
import org.slf4j.LoggerFactory

// here class is created to calculate the accuracy from the dataset with the iterations and once it is done
// we will calculate the score in order to get the output using multilayer network
class AccuracyCalculator(private val dataSetIterator: MnistDataSetIterator) : ScoreCalculator {
    var a = 0
    override fun calculateScore(network: MultiLayerNetwork): Double {
        val evaluation = network.evaluate(dataSetIterator)
        val accuracies = evaluation.accuracy()
        log.error("Accuracy " + a++ + " " + accuracies)
        return 1 - evaluation.accuracy()
    }

    companion object {
        private val log = LoggerFactory.getLogger(AccuracyCalculator::class.java)
    }
				
			

This file represents the convolutional approach in order for generations for the structure. In this section we are applying number of iterations in order to take the input image as pixel in form of arrays

				
					package gaurav.cnn
// package and import statements has been called here
import java.lang.Thread
import kotlin.Throws
import java.io.IOException
import kotlin.jvm.JvmStatic
import gaurav.cnn.EdgeDetection
import java.awt.image.BufferedImage
import javax.imageio.ImageIO
import gaurav.cnn.MnistExample
import org.nd4j.linalg.dataset.api.iterator.DataSetIterator
import org.deeplearning4j.datasets.iterator.impl.MnistDataSetIterator
import org.deeplearning4j.nn.conf.MultiLayerConfiguration
import org.deeplearning4j.nn.conf.NeuralNetConfiguration
import org.deeplearning4j.nn.conf.LearningRatePolicy
import org.deeplearning4j.nn.weights.WeightInit
import org.deeplearning4j.nn.api.OptimizationAlgorithm
import org.nd4j.linalg.lossfunctions.LossFunctions
import org.deeplearning4j.nn.multilayer.MultiLayerNetwork
import org.deeplearning4j.optimize.listeners.ScoreIterationListener
import org.deeplearning4j.util.ModelSerializer
import org.deeplearning4j.earlystopping.scorecalc.ScoreCalculator
import gaurav.cnn.AccuracyCalculator
import gaurav.cnn.ConvolutionalNeuralNetwork
import gaurav.data.LabeledImage
import org.nd4j.linalg.factory.Nd4j
import org.deeplearning4j.earlystopping.EarlyStoppingConfiguration
import org.deeplearning4j.earlystopping.termination.MaxEpochsTerminationCondition
import org.deeplearning4j.earlystopping.termination.MaxTimeIterationTerminationCondition
import java.util.concurrent.TimeUnit
import org.deeplearning4j.earlystopping.saver.LocalFileModelSaver
import org.deeplearning4j.earlystopping.trainer.EarlyStoppingTrainer
import org.deeplearning4j.earlystopping.EarlyStoppingResult
import java.awt.Color

 // This code is for implementing convolutional part

class Convolution
/**
 *  This is the Default no-arg constructor used here.
 */
    : Thread() {
    /**
     * we supply as many iterations for input array by using convolution2D algorithm.

     * @param width        this is for the width of the image
     * @param height       this represents the height of the images
     * @param input        the 2D double array representing the image
     * @param iterations   the number of iterations to apply to the convolution
     * @param kernel       the 2D array representing the kernel
     * @param kernelWidth  its for the width of kernel
     * @param kernelHeight indicate the  height of kernel
     * @return              it represents the new image
     */

    // function is being used to structure the basic structure of the model
    fun convolutionType1(
        input: Array,
        width: Int, height: Int,
        kernel: Array,
        kernelWidth: Int, kernelHeight: Int,
        iterations: Int
    ): Array {
        var Width = width
        var Height = height
        var NewInput = input.clone()
        var theoutput = input.clone()
        for (a in 0 until iterations) {
            val SmallWidth = width - kernelWidth + 1
            val SmallHeight = height - kernelHeight + 1
            theoutput = convolution2D(
                NewInput, width, height,
                kernel, kernelWidth, kernelHeight
            )
            Width = SmallWidth
            Height = SmallHeight
            NewInput = theoutput.clone()
        }
        return theoutput
    }

    /**
     *
     * here we supply similar type of as many iterations for input array by using convolution2D padded algorithm.
     */
    fun convolutionType2(
        input: Array,
        width: Int, height: Int,
        kernel: Array,
        kernelWidth: Int, kernelHeight: Int,
        iterations: Int
    ): Array {
        var NewInput = input.clone()
        var Theoutput = input.clone()
        for (a in 0 until iterations) {
            Theoutput = Array(width) { DoubleArray(height) }
            Theoutput = convolution2DPadded(
                NewInput, width, height,
                kernel, kernelWidth, kernelHeight
            )

            NewInput = Theoutput.clone()
        }
        return Theoutput
    }

    companion object {
        /**
         * here in this section it takes the greyscale image, position and the kernel get applies ot
         * the convolution at same position and returns it with the new value of pixels.
         * some new parameters that are being used here
         * @param x            represents the X coordinate for the position of the convolution.
         * @param y            represents the Y coordinate for the position of the convolution
         * @param k            Representing the kernel.
         * @return returns the new pixel value after the convolution.
         */
        fun singlePixelConvolution(
            input: Array,
            x: Int, y: Int,
            k: Array,
            kernelWidth: Int,
            kernelHeight: Int
        ): Double {
            var Theoutput = 0.0
            for (a in 0 until kernelWidth) {
                for (b in 0 until kernelHeight) {
                    Theoutput = Theoutput + input[x + a][y + b] * k[a][b]
                }
            }
            return Theoutput
        }

        fun applyConvolution(
            input: Array,
            x: Int, y: Int,
            k: Array,
            kernelWidth: Int,
            kernelHeight: Int
        ): Int {
            var Theoutput = 0
            for (a in 0 until kernelWidth) {
                for (b in 0 until kernelHeight) {
                    Theoutput = Theoutput + Math.round(input[x + a][y + b] * k[a][b]).toInt()
                }
            }
            return Theoutput
        }

        /**
         * it takes the greyscale image of 2D array , position and the kernel get applies
         * of the convolution to over the area of the image represented by width and height.
         * @return the 2D array representing the new image
         */
        fun convolution2D(
            input: Array,
            width: Int, height: Int,
            kernel: Array,
            kernelWidth: Int,
            kernelHeight: Int
        ): Array {
            val SmallWidth = width - kernelWidth + 1
            val SmallHeight = height - kernelHeight + 1
            val Theoutput = Array(SmallWidth) { DoubleArray(SmallHeight) }
            for (a in 0 until SmallWidth) {
                for (b in 0 until SmallHeight) {
                    Theoutput[a][b] = 0.0
                }
            }
            for (a in 0 until SmallWidth) {
                for (b in 0 until SmallHeight) {
                    Theoutput[a][b] = singlePixelConvolution(
                        input, a, b, kernel,
                        kernelWidth, kernelHeight
                    )

                }
            }
            return Theoutput
        }

        /**
         * it takes the greyscale image of 2D array , position and the kernel get applies
         * of the convolution to over the area of the image represented by width and height.
         * @return the 2D array representing the final  image
         */
        fun convolution2DPadded(
            input: Array,
            width: Int, height: Int,
            kernel: Array,
            kernelWidth: Int,
            kernelHeight: Int
        ): Array {
            val SmallWidth = width - kernelWidth + 1
            val SmallHeight = height - kernelHeight + 1
            val Top = kernelHeight / 2
            val left = kernelWidth / 2
            var Small = Array(SmallWidth) { DoubleArray(SmallHeight) }
            Small = convolution2D(
                input, width, height,
                kernel, kernelWidth, kernelHeight
            )
            val Big = Array(width) { DoubleArray(height) }
            for (b in 0 until height) {
                for (a in 0 until width) {
                    Big[a][b] = 0.0
                }
            }
            for (b in 0 until SmallHeight) {
                for (a in 0 until SmallWidth) {
                    Big[a + left][b + Top] = Small[a][b]
                }
            }
            return Big
        }

        /**
         * it takes the greyscale image of 2D array , position and the kernel get applies
         * of the convolution to over the area of the image represented by width and height.
         * @return  1D array that represents new generated image
         */
        fun convolutionDouble(
            input: Array,
            width: Int, height: Int,
            kernel: Array,
            kernelWidth: Int, kernelHeight: Int
        ): DoubleArray {
            val SmallWidth = width - kernelWidth + 1
            val SmallHeight = height - kernelHeight + 1
            var Small = Array(SmallWidth) { DoubleArray(SmallHeight) }
            Small = convolution2D(input, width, height, kernel, kernelWidth, kernelHeight)
            val result = DoubleArray(SmallWidth * SmallHeight)
            for (b in 0 until SmallHeight) {
                for (a in 0 until SmallWidth) {
                    result[b * SmallWidth + a] = Small[a][b]
                }
            }
            return result
        }
        /**
         * it takes the greyscale image of 2D array , position and the kernel get applies
         * of the convolution to over the area of the image represented by width and height.
         * @return  1D array that represents new generated image
         */

        fun convolutionDoublePadded(
            input: Array,
            width: Int, height: Int,
            kernel: Array,
            kernelWidth: Int,
            kernelHeight: Int
        ): DoubleArray {
            var resultof2D = Array(width) { DoubleArray(height) }
            resultof2D = convolution2DPadded(
                input, width, height,
                kernel, kernelWidth, kernelHeight
            )
            val result = DoubleArray(width * height)
            for (b in 0 until height) {
                for (a in 0 until width) {
                    result[b * width + a] = resultof2D[a][b]

                }
            }
            return result
        }

        /**
         *Simply converts the greyscale array to pixel arrays
         * @return the 1D array of pixels.
         */
        fun doublesToValidPixels(greys: DoubleArray): IntArray {
            val theresult = IntArray(greys.size)
            var grey: Int
            for (a in greys.indices) {
                grey = if (greys[a] > 255) {
                    255
                } else if (greys[a] < 0) {
                    0
                } else {
                    Math.round(greys[a]).toInt()
                }
                theresult[a] = Color(grey, grey, grey).rgb
            }
            return theresult
        }

        /**
         * implements the convolution2DPadded algorithm with the use of scales and offset
         * @param scale      for applying the scale factor
         * @param offset     for applying the offset factor
         * @return the 1D array that represents the new image
         */

        fun convolution__image(
            input: IntArray, width: Int, height: Int,
            kernel: Array,
            kernelWidth: Int, kernelHeight: Int,
            scale: Double, offset: Double
        ): IntArray {
            val inputof2D = Array(width) { DoubleArray(height) }
            var theoutput = DoubleArray(width * height)
            for (b in 0 until height) {
                for (a in 0 until width) {
                    inputof2D[a][b] = Color(input[b * width + a]).red.toDouble()
                }
            }
            theoutput = convolutionDoublePadded(
                inputof2D, width, height,
                kernel, kernelWidth, kernelHeight
            )
            val outputInts = IntArray(width * height)
            for (a in outputInts.indices) {
                outputInts[a] = Math.round(theoutput[a] * scale + offset).toInt()
                if (outputInts[a] > 255) outputInts[a] = 255
                if (outputInts[a] < 0) outputInts[a] = 0
                val c = outputInts[a]
                outputInts[a] = Color(c, c, c).rgb
            }
            return outputInts
        }
    }
}
				
			

Here we are applying the actual CNN which will implement the model in order to generate the outcome with the help of kernel size and we are also applying regularization and generating the accuracy for CNN

				
					package gaurav.cnn

import java.lang.Thread
import kotlin.Throws
import java.io.IOException
import kotlin.jvm.JvmStatic
import gaurav.cnn.EdgeDetection
import java.awt.image.BufferedImage
import javax.imageio.ImageIO
import gaurav.cnn.MnistExample
import org.nd4j.linalg.dataset.api.iterator.DataSetIterator
import org.deeplearning4j.datasets.iterator.impl.MnistDataSetIterator
import org.deeplearning4j.nn.conf.MultiLayerConfiguration
import org.deeplearning4j.nn.conf.NeuralNetConfiguration
import org.deeplearning4j.nn.conf.LearningRatePolicy
import org.deeplearning4j.nn.weights.WeightInit
import org.deeplearning4j.nn.api.OptimizationAlgorithm
import org.nd4j.linalg.lossfunctions.LossFunctions
import org.deeplearning4j.nn.multilayer.MultiLayerNetwork
import org.deeplearning4j.optimize.listeners.ScoreIterationListener
import org.deeplearning4j.util.ModelSerializer
import org.deeplearning4j.earlystopping.scorecalc.ScoreCalculator
import gaurav.cnn.AccuracyCalculator
import gaurav.cnn.ConvolutionalNeuralNetwork
import gaurav.data.LabeledImage
import org.nd4j.linalg.factory.Nd4j
import org.deeplearning4j.earlystopping.EarlyStoppingConfiguration
import org.deeplearning4j.earlystopping.termination.MaxEpochsTerminationCondition
import org.deeplearning4j.earlystopping.termination.MaxTimeIterationTerminationCondition
import java.util.concurrent.TimeUnit
import org.deeplearning4j.earlystopping.saver.LocalFileModelSaver
import org.deeplearning4j.earlystopping.trainer.EarlyStoppingTrainer
import org.deeplearning4j.earlystopping.EarlyStoppingResult
import org.deeplearning4j.nn.api.Model
import org.deeplearning4j.nn.conf.Updater
import org.deeplearning4j.nn.conf.inputs.InputType
import org.deeplearning4j.nn.conf.layers.ConvolutionLayer
import org.deeplearning4j.nn.conf.layers.DenseLayer
import org.deeplearning4j.nn.conf.layers.OutputLayer
import org.deeplearning4j.nn.conf.layers.SubsamplingLayer
import org.nd4j.linalg.activations.Activation
import org.slf4j.LoggerFactory
import java.io.File
import java.lang.Exception
import kotlin.Any as Any1

class ConvolutionalNeuralNetwork {
    private var preTrainedModel: MultiLayerNetwork? = null
    @Throws(IOException::class)
    fun init() {
        preTrainedModel = ModelSerializer.restoreMultiLayerNetwork(File(TRAINED_MODEL_FILE))
    }

    fun prediction(labeledImage: LabeledImage): Int {
        val pixel = labeledImage.pixels
        for (a in pixel.indices) {
            pixel[a] = pixel[a] / 255.0
        }
        val prediction = preTrainedModel!!.predict(Nd4j.create(pixel))
        return prediction[0]
    }

    @Throws(IOException::class)
    fun train(trainDataSize: Int?, testDataSize: Int?) {
        val chanels = 1 // inputs channels
        val opnum = 10 // total number of outputs
        val sizeofbatch = 64
        val nEpochs = 20 // training epochs
        val iter = 1 // numbers of the training iterations mentioned
        val seed = 123
        // here in the next line we have set the binary segment to false and training and shuffling are set to be true.
        val mnistTraindata = MnistDataSetIterator(sizeofbatch, trainDataSize!!, false, true, true, 12345)
        val config = NeuralNetConfiguration.Builder()
            .seed(seed)
            .iterations(iter)
            .regularization(false)// using regularization
            .learningRate(0.01)// learning rate
            .weightInit(WeightInit.XAVIER)
            .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)
            .updater(Updater.NESTEROVS)
            .list()
            .layer(
                //below are some kernel sizes, strides, number of outputs ar mentioned again and again
                0, ConvolutionLayer.Builder(5, 5)// kernel size
                    .nIn(chanels)
                    .stride(1, 1)// strides
                    .nOut(20)// number of outputs
                    .activation(Activation.IDENTITY)
                    .build()
            )
            .layer(
                1, SubsamplingLayer.Builder(SubsamplingLayer.PoolingType.MAX)
                    .kernelSize(2, 2)
                    .stride(2, 2)
                    .build()
            )
            .layer(
                2, ConvolutionLayer.Builder(5, 5)// kernel size
                    .nIn(20)
                    .stride(1, 1)
                    .nOut(50)
                    .activation(Activation.IDENTITY)
                    .build()
            )
            .layer(
                3, SubsamplingLayer.Builder(SubsamplingLayer.PoolingType.MAX)
                    .kernelSize(2, 2)
                    .stride(2, 2)
                    .build()
            )
            .layer(
                4, DenseLayer.Builder().activation(Activation.RELU)
                    .nIn(800)
                    .nOut(128).build()
            )
            .layer(
                5, DenseLayer.Builder().activation(Activation.RELU)
                    .nIn(128)
                    .nOut(64).build()
            )
            .layer(
                6, OutputLayer.Builder(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD)
                    .nOut(opnum)
                    .activation(Activation.SOFTMAX)
                    .build()
            )
            .setInputType(InputType.convolutionalFlat(28, 28, 1)) //height,width and depth
            .backprop(true).pretrain(false).build()
        val esConf: Unit = EarlyStoppingConfiguration.Builder()
            .epochTerminationConditions(MaxEpochsTerminationCondition(nEpochs))
            .iterationTerminationConditions(MaxTimeIterationTerminationCondition(75, TimeUnit.MINUTES))// maximum time count
            .scoreCalculator(
                /* scoreCalculator = */ AccuracyCalculator(
                    MnistDataSetIterator(testDataSize!!, testDataSize, false, false, true, 12345)
                )
            )
            .evaluateEveryNEpochs(1)
            .modelSaver(LocalFileModelSaver(OUT_DIR))
            .build()
        val trainers = EarlyStoppingTrainer(esConf, config, mnistTraindata)
        val results: EarlyStoppingResult

    }

    private fun EarlyStoppingTrainer(
        esConf: Unit,
        conf: MultiLayerConfiguration?,
        mnistTrain: MnistDataSetIterator
    ): kotlin.Any {
        TODO("Not impelemented")
    }

    companion object {
        private const val OUT_DIR = "resources/cnnCurrentTrainingModels"
        private const val TRAINED_MODEL_FILE = "resources/cnnTrainedModels/bestModel.bin"
        private val LOG = LoggerFactory.getLogger(ConvolutionalNeuralNetwork::class.java)
        @Throws(Exception::class)
        @JvmStatic
        fun main(args: Array) {
            ConvolutionalNeuralNetwork().train(60000, 1000) // training and testing are mentioned
        }
    }
}

private fun Model.fit() {

}

private fun Unit.build() {
}


private fun Unit.modelSaver(localFileModelSaver: LocalFileModelSaver) {

}

private fun Unit.evaluateEveryNEpochs(i: Int) {

}

private fun  EarlyStoppingConfiguration.Builder.scoreCalculator(accuracyCalculator: AccuracyCalculator) {
}

				
			

In this block we used the edge detection approach to implement on the CNN model. Below we can see that we are trying to detect different forms of edge and trying to get the outcome by transforming image to arrays and also resetting the edge for the further process

				
					package gaurav.cnn

import java.lang.Thread
import kotlin.Throws
import java.io.IOException
import kotlin.jvm.JvmStatic
import gaurav.cnn.EdgeDetection
import java.awt.image.BufferedImage
import javax.imageio.ImageIO
import gaurav.cnn.MnistExample
import org.nd4j.linalg.dataset.api.iterator.DataSetIterator
import org.deeplearning4j.datasets.iterator.impl.MnistDataSetIterator
import org.deeplearning4j.nn.conf.MultiLayerConfiguration
import org.deeplearning4j.nn.conf.NeuralNetConfiguration
import org.deeplearning4j.nn.conf.LearningRatePolicy
import org.deeplearning4j.nn.weights.WeightInit
import org.deeplearning4j.nn.api.OptimizationAlgorithm
import org.nd4j.linalg.lossfunctions.LossFunctions
import org.deeplearning4j.nn.multilayer.MultiLayerNetwork
import org.deeplearning4j.optimize.listeners.ScoreIterationListener
import org.deeplearning4j.util.ModelSerializer
import org.deeplearning4j.earlystopping.scorecalc.ScoreCalculator
import gaurav.cnn.AccuracyCalculator
import gaurav.cnn.ConvolutionalNeuralNetwork
import gaurav.data.LabeledImage
import org.nd4j.linalg.factory.Nd4j
import org.deeplearning4j.earlystopping.EarlyStoppingConfiguration
import org.deeplearning4j.earlystopping.termination.MaxEpochsTerminationCondition
import org.deeplearning4j.earlystopping.termination.MaxTimeIterationTerminationCondition
import java.util.concurrent.TimeUnit
import org.deeplearning4j.earlystopping.saver.LocalFileModelSaver
import org.deeplearning4j.earlystopping.trainer.EarlyStoppingTrainer
import org.deeplearning4j.earlystopping.EarlyStoppingResult
import java.awt.Color
import java.io.File

// this part of code operates on edge detection
object EdgeDetection {
    private val verticle_filter =
        arrayOf(doubleArrayOf(1.0, 0.0, -1.0), doubleArrayOf(1.0, 0.0, -1.0), doubleArrayOf(1.0, 0.0, -1.0))
    private val horizontal_filter =
        arrayOf(doubleArrayOf(1.0, 1.0, 1.0), doubleArrayOf(0.0, 0.0, 0.0), doubleArrayOf(-1.0, -1.0, -1.0))
    private val sobel_filter =
        arrayOf(doubleArrayOf(1.0, 0.0, -1.0), doubleArrayOf(2.0, 0.0, -2.0), doubleArrayOf(1.0, 0.0, -1.0))
    private const val input_image = "resources/smallGirl.png"
    private var counts = 1
    @Throws(IOException::class)
    @JvmStatic
    //in this section we are initialising different edge detection method
    fun main(args: Array) {
        detectVerticalEdges()
        detectHorizontalEdges()
        detectSobelEdges()
    }
    // block of code for detecting the soble edges 
    @Throws(IOException::class)
    private fun detectSobelEdges() {
        val bufferedImages = ImageIO.read(File(input_image))
        val images = transformImageToArray(bufferedImages)
        val finalConv = applyConvolution(bufferedImages.width, bufferedImages.height, images, sobel_filter)
        reCreateOriginalImageFromMatrix(bufferedImages, finalConv)
    }
    // block of code for detecting the horizontal edges 

    @Throws(IOException::class)
    private fun detectHorizontalEdges() {
        val bufferedImages = ImageIO.read(File(input_image))
        val images = transformImageToArray(bufferedImages)
        val finalConvo = applyConvolution(bufferedImages.width, bufferedImages.height, images, horizontal_filter)
        reCreateOriginalImageFromMatrix(bufferedImages, finalConvo)
    }
    // block of code for detecting the verticle edges 

    @Throws(IOException::class)
    private fun detectVerticalEdges() {
        val bufferedImages = ImageIO.read(File(input_image))
        val images = transformImageToArray(bufferedImages)
        val finalConvo = applyConvolution(bufferedImages.width, bufferedImages.height, images, verticle_filter)
        reCreateOriginalImageFromMatrix(bufferedImages, finalConvo)
    }
    // here we are transforming the image to array in order to apply the CNN method

    private fun transformImageToArray(bufferedImage: BufferedImage): Array<Array> {
        val widths = bufferedImage.width
        val heights = bufferedImage.height
        return transformImageToArray(bufferedImage, widths, heights)
    }
    // now we can see the model is being approached from here
    private fun applyConvolution(
        width: Int,
        height: Int,
        image: Array<Array>,
        filter: Array
    ): Array {
        val convo = Convolution()
        val redConvo = convo.convolutionType2(image[0], height, width, filter, 3, 3, 1)
        val greenConvo = convo.convolutionType2(image[1], height, width, filter, 3, 3, 1)
        val blueConvo = convo.convolutionType2(image[2], height, width, filter, 3, 3, 1)
        val finalConvo = Array(redConvo!!.size) { DoubleArray(redConvo[0]!!.size) }
        for (i in redConvo.indices) {
            for (j in redConvo[i]!!.indices) {
                finalConvo[i][j] = redConvo[i]!![j] + greenConvo!![i]!![j] + blueConvo!![i]!![j]
            }
        }
        return finalConvo
    }
    // here we are transforming the image to array in order to apply the CNN method

    private fun transformImageToArray(
        bufferedImage: BufferedImage,
        width: Int,
        height: Int
    ): Array<Array> {
        val images = Array(3) { Array(height) { DoubleArray(width) } }
        for (a in 0 until height) {
            for (b in 0 until width) {
                val color = Color(bufferedImage.getRGB(b, a))
                images[0][a][b] = color.red.toDouble()
                images[1][a][b] = color.green.toDouble()
                images[2][a][b] = color.blue.toDouble()
            }
        }
        return images
    }
    // this helps us for recreation 
    @Throws(IOException::class)
    private fun reCreateOriginalImageFromMatrix(originalImage: BufferedImage, imageRGB: Array) {
        val BackImages = BufferedImage(originalImage.width, originalImage.height, BufferedImage.TYPE_INT_RGB)
        for (a in imageRGB.indices) {
            for (b in imageRGB[a].indices) {
                val colors = Color(
                    fixOutOfRangeRGBValues(imageRGB[a][a]),
                    fixOutOfRangeRGBValues(imageRGB[a][b]),
                    fixOutOfRangeRGBValues(imageRGB[a][b])
                )
                BackImages.setRGB(b, a, colors.rgb)
            }
        }
        val outputFiles = File("edges" + counts++ + ".png")
        ImageIO.write(BackImages, "png", outputFiles)
    }

    private fun fixOutOfRangeRGBValues(value: Double): Int {
        var values = value
        if (value  255) {
            255
        } else {
            value.toInt()
        }
    }
}
				
			

This is something that you all are familiar with it is the code of operation for the famous MNIST dataset which here simply does is that it loads the data into the project and further it build the model using this dataset and and hence than we started to train the model later than we evaluate them in order to make full use of this dataset example

				
					package gaurav.cnn
// all the libraries and class has been imported here in this secion
import java.lang.Thread
import kotlin.Throws
import java.io.IOException
import kotlin.jvm.JvmStatic
import gaurav.cnn.EdgeDetection
import java.awt.image.BufferedImage
import javax.imageio.ImageIO
import gaurav.cnn.MnistExample
import org.nd4j.linalg.dataset.api.iterator.DataSetIterator
import org.deeplearning4j.datasets.iterator.impl.MnistDataSetIterator
import org.deeplearning4j.nn.conf.MultiLayerConfiguration
import org.deeplearning4j.nn.conf.NeuralNetConfiguration
import org.deeplearning4j.nn.conf.LearningRatePolicy
import org.deeplearning4j.nn.weights.WeightInit
import org.deeplearning4j.nn.api.OptimizationAlgorithm
import org.nd4j.linalg.lossfunctions.LossFunctions
import org.deeplearning4j.nn.multilayer.MultiLayerNetwork
import org.deeplearning4j.optimize.listeners.ScoreIterationListener
import org.deeplearning4j.util.ModelSerializer
import org.deeplearning4j.earlystopping.scorecalc.ScoreCalculator
import gaurav.cnn.AccuracyCalculator
import gaurav.cnn.ConvolutionalNeuralNetwork
import gaurav.data.LabeledImage
import org.nd4j.linalg.factory.Nd4j
import org.deeplearning4j.earlystopping.EarlyStoppingConfiguration
import org.deeplearning4j.earlystopping.termination.MaxEpochsTerminationCondition
import org.deeplearning4j.earlystopping.termination.MaxTimeIterationTerminationCondition
import java.util.concurrent.TimeUnit
import org.deeplearning4j.earlystopping.saver.LocalFileModelSaver
import org.deeplearning4j.earlystopping.trainer.EarlyStoppingTrainer
import org.deeplearning4j.earlystopping.EarlyStoppingResult
import org.deeplearning4j.nn.conf.Updater
import org.deeplearning4j.nn.conf.inputs.InputType
import org.deeplearning4j.nn.conf.layers.ConvolutionLayer
import org.deeplearning4j.nn.conf.layers.DenseLayer
import org.deeplearning4j.nn.conf.layers.OutputLayer
import org.deeplearning4j.nn.conf.layers.SubsamplingLayer
import org.nd4j.linalg.activations.Activation
import org.slf4j.LoggerFactory
import java.io.File
import java.lang.Exception
import java.util.HashMap

object MnistExample {
    private val log = LoggerFactory.getLogger(MnistExample::class.java)
    @Throws(Exception::class)
    @JvmStatic
    fun main(args: Array) {
        val nChannels = 1 // number of input
        val outputNum = 10 // number of  outputs
        val batchSize = 64
        val nEpochs = 100 // training epochs
        val iterations = 1
        val seed = 123 //

        /*
            here we have Created an iterator using the batch size for single iteration.
         */log.info("Loading data....")
        val mnistTrain: DataSetIterator = MnistDataSetIterator(batchSize, true, 12345)

        /*
            here we are contructiong the neural network for the model
         */log.info("Building model....")

        // in this section we have scheduled learning rate that too in form of 
        val lrSchedule: MutableMap = HashMap()
        lrSchedule[0] = 0.01
        lrSchedule[1000] = 0.005
        lrSchedule[3000] = 0.001
        val conf = NeuralNetConfiguration.Builder()
            .seed(seed)
            .iterations(iterations) // we similarly start Training iterations here
            .regularization(true).l2(0.0005)
            .learningRate(.01) // here we have biased data iterations
            .learningRateDecayPolicy(LearningRatePolicy.Schedule)
            .learningRateSchedule(lrSchedule)
            .weightInit(WeightInit.XAVIER)
            .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)
            .updater(Updater.NESTEROVS) // here is to updata the new NESTEROVS value
            .list()
            .layer(
                0,
                ConvolutionLayer.Builder(
                    5,
                    5
                ) //nIn and nOut specify depth. nIn here is the nChannels and nOut is the number of filters to be applied
                    .nIn(nChannels)
                    .stride(1, 1)
                    .nOut(20)
                    .activation(Activation.IDENTITY)
                    .build()
            )
            .layer(
                1, SubsamplingLayer.Builder(SubsamplingLayer.PoolingType.MAX)
                    .kernelSize(2, 2)
                    .stride(2, 2)
                    .build()
            )
            .layer(
                2, ConvolutionLayer.Builder(5, 5) //nIn here is also here for future ref
                    .stride(1, 1)
                    .nOut(50)
                    .activation(Activation.IDENTITY)
                    .build()
            )
            .layer(
                3, SubsamplingLayer.Builder(SubsamplingLayer.PoolingType.MAX)
                    .kernelSize(2, 2)
                    .stride(2, 2)
                    .build()
            )
            .layer(
                4, DenseLayer.Builder().activation(Activation.RELU)
                    .nOut(500).build()
            )
            .layer(
                5, OutputLayer.Builder(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD)
                    .nOut(outputNum)
                    .activation(Activation.SOFTMAX)
                    .build()
            )
            .setInputType(InputType.convolutionalFlat(28, 28, 1))
            .backprop(true).pretrain(false).build()

        val model = MultiLayerNetwork(conf)
        model.init()
        log.info("Training model....")
        model.setListeners(ScoreIterationListener(1))
        var mnistTest: DataSetIterator? = null
        for (a in 0 until nEpochs) {
            model.fit(mnistTrain)
            log.info("epoch completed here", a)
            if (mnistTest == null) {
                mnistTest = MnistDataSetIterator(10000, false, 12345)
            }
            log.info("Evaluate model....")
            val eval = model.evaluate(mnistTest)

            log.info(eval.stats())
            mnistTest.reset()
        }
        log.info("done with example")
    }
}
				
			

From here on we are working on data specifically. In this particular code you can see that easily that data is being loaded, trained, and we got images from data in order to perform the necessary operation require for the model.

				
					package gaurav.data
// all the libraries/ classes has already been imported
import org.slf4j.LoggerFactory
import gaurav.data.LabeledImage
import gaurav.data.IdxReader
import java.io.FileInputStream
import java.lang.Exception
import java.lang.RuntimeException
import java.util.ArrayList

object IdxReader {
    private val LOGGER = LoggerFactory.getLogger(IdxReader::class.java)
    const val INPUT_IMAGE_PATH = "resources/train-images.idx3-ubyte"
    const val INPUT_LABEL_PATH = "resources/train-labels.idx1-ubyte"
    const val INPUT_IMAGE_PATH_TEST_DATA = "resources/t10k-images.idx3-ubyte"
    const val INPUT_LABEL_PATH_TEST_DATA = "resources/t10k-labels.idx1-ubyte"
    const val VECTOR_DIMENSION = 784 //total pixels square 28*28 from dataset
    // which leads to total number of arrays to 784

    //data is being loaded
    @JvmStatic
    fun loadData(size: Int): List {
        return getLabeledImages(INPUT_IMAGE_PATH, INPUT_LABEL_PATH, size)
    }
    // testing the data
       @JvmStatic
    fun loadTestData(size: Int): List {
        return getLabeledImages(INPUT_IMAGE_PATH_TEST_DATA, INPUT_LABEL_PATH_TEST_DATA, size)
    }
    // trying to get labelled images from dataset
    private fun getLabeledImages(
        inputImagePath: String,
        inputLabelPath: String,
        amountOfDataSet: Int
    ): List {
        val labeledImageArrayList: MutableList = ArrayList(amountOfDataSet)
        try {
            FileInputStream(inputImagePath).use { inImage ->
                FileInputStream(inputLabelPath).use { inLabel ->

                    // just look at the test and description from the datasets
                    inImage.skip(16)
                    inLabel.skip(8)
                    LOGGER.debug("Available bytes in inputImage stream after read: " + inImage.available())
                    LOGGER.debug("Available bytes in inputLabel stream after read: " + inLabel.available())

                    //empty array for 784 pixels - the image from 28x28 pixels in a single row
                    val imgPixels = DoubleArray(VECTOR_DIMENSION)
                    LOGGER.info("Creating ADT filed with Labeled Images ")
                    val start = System.currentTimeMillis()
                    for (i in 0 until amountOfDataSet) {
                        if (i % 1000 == 0) {
                            LOGGER.info("extracted images: $i")
                        }
                        //this section adds the pixels in the arrays
                        for (index in 0 until VECTOR_DIMENSION) {
                            imgPixels[index] = inImage.read().toDouble()
                        }
                        //than labels it
                        val label = inLabel.read()

                        //creating compound to add to data list
                        labeledImageArrayList.add(LabeledImage(label, imgPixels))
                    }
                    LOGGER.info("loading LabeledImages: " + (System.currentTimeMillis() - start) / 1000.0)
                }
            }
        } catch (e: Exception) {
            LOGGER.error("Something went wrong: \n$e")
            throw RuntimeException(e)
        }
        return labeledImageArrayList
    }
}
				
			

The second data is set to deal with the labelled image and we look for the handling the pixels and then finally converting it to string to get the final labelled output

				
					package gaurav.data

import org.apache.spark.ml.linalg.Vector
import org.apache.spark.ml.linalg.Vectors
import java.io.Serializable

// this part operates for the labelled images
class LabeledImage(labeled: Int, pixel: DoubleArray) : Serializable {
    private val meanNormalizedPixel: DoubleArray
    val pixels: DoubleArray
    var label: Double
    val features: Vector

    init {
        meanNormalizedPixel = meanNormalizeFeatures(pixel)
        this.pixels = pixel
        features = Vectors.dense(meanNormalizedPixel)
        this.label = labeled.toDouble()
    }

    private fun meanNormalizeFeatures(pixels: DoubleArray): DoubleArray {
        var min = Double.MAX_VALUE
        var max = Double.MIN_VALUE
        var sum = 0.0
        for (pixel in pixels) {
            sum = sum + pixel
            if (pixel > max) {
                max = pixel
            }
            if (pixel < min) {
                min = pixel
            }
        }
        val mean = sum / pixels.size
        val pixelsNorm = DoubleArray(pixels.size)
        for (i in pixels.indices) {
            pixelsNorm[i] = (pixels[i] - mean) / (max - min)
        }
        return pixelsNorm
    }

    override fun toString(): String {
        return "LabeledImage{" +
                "label=" + label +
                '}'
    }
}
				
			

Until here we were following the approach of CNN and lets now look into the operation of Neural Network and see what we get. 

				
					package gaurav.nn

import gaurav.data.IdxReader.loadData
import gaurav.data.IdxReader.loadTestData
import org.apache.spark.sql.SparkSession
import org.apache.spark.ml.classification.MultilayerPerceptronClassificationModel
import gaurav.data.LabeledImage
import gaurav.data.IdxReader
import org.apache.spark.sql.Dataset
import org.apache.spark.ml.classification.MultilayerPerceptronClassifier
import org.apache.spark.ml.evaluation.MulticlassClassificationEvaluator
import org.apache.spark.sql.Row
import org.slf4j.LoggerFactory

class NeuralNetwork {
    private var sparkSessions: SparkSession? = null
    private var models: MultilayerPerceptronClassificationModel? = null
    fun init() {
        initSparkSession()
        if (models == null) {
            LOGGER.info("Loading the Neural Network (NN)")
            models = MultilayerPerceptronClassificationModel.load("resources/nnTrainedModels/ModelWith60000")
            LOGGER.info("done loading")
        }
    }

    fun train(trainData: Int?, testFieldValue: Int?) {
        initSparkSession()
        val labeledImages: List = loadData(trainData!!)
        val testLabeledImages: List = loadTestData(testFieldValue!!)
        val training = sparkSessions!!.createDataFrame(labeledImages, LabeledImage::class.java).checkpoint()
        val testing = sparkSessions!!.createDataFrame(testLabeledImages, LabeledImage::class.java).checkpoint()
        val layers = intArrayOf(784, 128, 64, 10)
        val trainer = MultilayerPerceptronClassifier()
            .setLayers(layers)
            .setBlockSize(128)
            .setSeed(1234L)
            .setMaxIter(100)
        models = trainer.fit(training)
        evalOnTest(testing)
        evalOnTest(training)
    }

    private fun evalOnTest(test: Dataset) {
        val finalresult = models!!.transform(test)
        val predictionAndLabels = finalresult.select("prediction", "labels")
        val evaluator = MulticlassClassificationEvaluator()
            .setMetricName("The Accuracy")
        LOGGER.info("testing Accuracy = " + evaluator.evaluate(predictionAndLabels))
    }

    private fun initSparkSession() {
        if (sparkSessions == null) {
            sparkSessions = SparkSession.builder()
                .master("local[*]")
                .appName("Digit Recognizer")
                .orCreate
        }
        sparkSessions!!.sparkContext().setCheckpointDir("checkPoint")
    }

    fun predict(labeledImage: LabeledImage): LabeledImage {
        val prediction = models!!.predict(labeledImage.features)
        labeledImage.label = prediction
        return labeledImage
    }

    companion object {
        private val LOGGER = LoggerFactory.getLogger(NeuralNetwork::class.java)
    }
}
				
			

Lets dive into the User Interface part where we look into the following details. In this block we are designing a UI for user to draw the pattern that we will take it as a input and process further. 

				
					package gaurav.ui

import java.awt.*
import javax.swing.JComponent
import javax.swing.BorderFactory
import javax.swing.border.TitledBorder
import java.awt.event.MouseAdapter
import java.awt.event.MouseMotionAdapter
import java.awt.event.MouseEvent
// for drawing operations
class DrawArea : JComponent() {
    private val sansSerifBold = Font("Times New Roman", Font.BOLD, 18)

    // for drwaing image
    var image: Image? = null

    // grahics 2D will be used to draw on
    private var g2: Graphics2D? = null

    // to capture mouse position by detecting coordinates of it
    private var currentX = 0
    private var currentY = 0
    private var oldX = 0
    private var oldY = 0

    init {
        isDoubleBuffered = false
        border = BorderFactory.createTitledBorder(
            BorderFactory.createEtchedBorder(),
            "Draw any one Digit from 0-9",
            TitledBorder.LEFT,
            TitledBorder.TOP, sansSerifBold, Color.BLUE
        )
        addMouseListener(object : MouseAdapter() {
            override fun mousePressed(e: MouseEvent) {
                // When mouse is pressed it saves the X, Y chords
                oldX = e.x
                oldY = e.y
            }
        })
        addMouseMotionListener(object : MouseMotionAdapter() {
            override fun mouseDragged(e: MouseEvent) {
                // this section captures the X,Y chords when mouse is dragged
                currentX = e.x
                currentY = e.y
                if (g2 != null) {
                    g2!!.stroke = BasicStroke(10F)
                    // it draw the line if g2 context not null
                    g2!!.drawLine(oldX, oldY, currentX, currentY)
                    // reset the area for new shape
                    repaint()
                    // store current coords x,y as olds x,Y
                    oldX = currentX
                    oldY = currentY
                }
            }
        })
    }

    override fun paintComponent(g: Graphics) {
        if (image == null) {
            //image to draw null
            image = createImage(size.width, size.height)
            (image!!.getGraphics() as Graphics2D).also { g2 = it }
            // enable antialiasing
            g2!!.setRenderingHint(RenderingHints.KEY_ANTIALIASING, RenderingHints.VALUE_ANTIALIAS_ON)
            // it simply clears drawing area
            clear()
        }
        g.drawImage(image, 0, 0, null)
    }

    fun clear() {
        g2!!.paint = Color.white
        // it simply draws color white to clear the background
        g2!!.fillRect(0, 0, size.width, size.height)
        g2!!.paint = Color.black
        repaint()
    }
}
				
			

Once we are done with the project we will progress to run the project which will generate the progress bar to let the user know what the current status of project is after running it.

				
					package gaurav.ui

import javax.swing.JFrame
import javax.swing.JProgressBar
import javax.swing.SwingUtilities
import java.lang.Runnable
import java.awt.BorderLayout

class ProgressBar {
    private val mainFrame: JFrame
    private var progressBar: JProgressBar
    private var unDecoreate = false

    constructor(mainFrame: JFrame) {
        this.mainFrame = mainFrame
        progressBar = createProgressBar(mainFrame)
    }

    constructor(mainFrame: JFrame, unDecoreate: Boolean) {
        this.mainFrame = mainFrame
        progressBar = createProgressBar(mainFrame)
        this.unDecoreate = unDecoreate
    }

    fun showProgressBar(msg: String?) {
        SwingUtilities.invokeLater {
            if (unDecoreate) {
                mainFrame.setLocationRelativeTo(null)
                mainFrame.isUndecorated = true
            }
            progressBar = createProgressBar(mainFrame)
            progressBar.string = msg
            progressBar.isStringPainted = true
            progressBar.isIndeterminate = true
            progressBar.isVisible = true
            mainFrame.add(progressBar, BorderLayout.NORTH)
            if (unDecoreate) {
                mainFrame.pack()
                mainFrame.isVisible = true
            }
            mainFrame.repaint()
        }
    }

    private fun createProgressBar(mainFrame: JFrame): JProgressBar {
        val jProgressBar = JProgressBar(JProgressBar.HORIZONTAL)
        jProgressBar.isVisible = false
        mainFrame.add(jProgressBar, BorderLayout.NORTH)
        return jProgressBar
    }

    fun setVisible(visible: Boolean) {
        progressBar.isVisible = visible
    }
}
				
			

This part of code is what will user see and work on in order to put the input and choose the approach for the corrosponding outcomes and it also has the features for the user to train their own model for their own purpose

				
					package gaurav.ui
// all the libraries has been attached here
import com.mortennobel.imagescaling.ResampleFilters
import com.mortennobel.imagescaling.ResampleOp
import org.slf4j.LoggerFactory
import gaurav.cnn.ConvolutionalNeuralNetwork
import gaurav.data.LabeledImage
import gaurav.nn.NeuralNetwork
import gaurav.ui.UI
import java.awt.*
import java.awt.event.ActionEvent
import java.awt.event.WindowAdapter
import java.awt.event.WindowEvent
import java.awt.image.BufferedImage
import java.io.IOException
import java.util.concurrent.Executors
import javax.swing.*
import javax.swing.plaf.FontUIResource

class UI {
    private val neuralNetwork = NeuralNetwork()
    private val convolutionalNeuralNetwork = ConvolutionalNeuralNetwork()
    private var drawArea: DrawArea? = null
    private var mainFrame: JFrame? = null
    private var mainPanel: JPanel? = null
    private var drawAndDigitPredictionPanel: JPanel? = null
    private var modelTrainSize: SpinnerNumberModel? = null
    private var trainField: JSpinner? = null
    private val TRAIN_SIZE = 30000
    private val sansSerifBold = Font("Times New Roman", Font.BOLD, 21)
    private val TEST_SIZE = 10000
    private var modelTestSize: SpinnerNumberModel? = null
    private var testField: JSpinner? = null
    private var resultPanel: JPanel? = null

    init {
        UIManager.setLookAndFeel(UIManager.getSystemLookAndFeelClassName())
        UIManager.put("Button.font", FontUIResource(Font("Dialog", Font.BOLD, 18)))
        UIManager.put("ProgressBar.font", FontUIResource(Font("Dialog", Font.BOLD, 18)))
        neuralNetwork.init()
        convolutionalNeuralNetwork.init()
    }

    fun initUI() {
        // create main frame
        mainFrame = createMainFrame()
        mainPanel = JPanel()
        mainPanel!!.layout = BorderLayout()
        addTopPanel()
        drawAndDigitPredictionPanel = JPanel(GridLayout())
        addActionPanel()
        addDrawAreaAndPredictionArea()
        mainPanel!!.add(drawAndDigitPredictionPanel, BorderLayout.CENTER)
        addSignature()
        mainFrame!!.add(mainPanel, BorderLayout.CENTER)
        mainFrame!!.isVisible = true
    }

    private fun addActionPanel() {
        val recognize = JButton("Neural Network")
        val recognizeCNN = JButton("Convolutional Neural Network")
        recognize.addActionListener { e: ActionEvent? ->
            val drawImage = drawArea!!.image
            val sbi = toBufferedImage(drawImage)
            val scaled: Image = scale(sbi)
            val scaledBuffered = toBufferedImage(scaled)
            val scaledPixels = transformImageToOneDimensionalVector(scaledBuffered)
            val labeledImage = LabeledImage(0, scaledPixels)
            val predict = neuralNetwork.predict(labeledImage)
            val predictNumber = JLabel("" + predict.label.toInt())
            predictNumber.foreground = Color.RED
            predictNumber.font = Font("Times New Roman", Font.BOLD, 128)
            resultPanel!!.removeAll()
            resultPanel!!.add(predictNumber)
            resultPanel!!.updateUI()
        }
        recognizeCNN.addActionListener { e: ActionEvent? ->
            val drawImage = drawArea!!.image
            val sbi = toBufferedImage(drawImage)
            val scaled: Image = scale(sbi)
            val scaledBuffered = toBufferedImage(scaled)
            val scaledPixels = transformImageToOneDimensionalVector(scaledBuffered)
            val labeledImage = LabeledImage(0, scaledPixels)
            val predict = convolutionalNeuralNetwork.prediction(labeledImage)
            val predictNumber = JLabel("" + predict)
            predictNumber.foreground = Color.RED
            predictNumber.font = Font("Times New Roman", Font.BOLD, 128)
            resultPanel!!.removeAll()
            resultPanel!!.add(predictNumber)
            resultPanel!!.updateUI()
        }
        val clear = JButton("Reset")
        clear.addActionListener { e: ActionEvent? ->
            drawArea!!.image = null
            drawArea!!.repaint()
            drawAndDigitPredictionPanel!!.updateUI()
        }
        val actionPanel = JPanel(GridLayout(8, 1))
        actionPanel.add(recognizeCNN)
        actionPanel.add(recognize)
        actionPanel.add(clear)
        drawAndDigitPredictionPanel!!.add(actionPanel)
    }

    private fun addDrawAreaAndPredictionArea() {
        drawArea = DrawArea()
        drawAndDigitPredictionPanel!!.add(drawArea)
        resultPanel = JPanel()
        resultPanel!!.layout = GridBagLayout()
        drawAndDigitPredictionPanel!!.add(resultPanel)
    }

    private fun addTopPanel() {
        val topPanel = JPanel(FlowLayout())
        val trainNN = JButton("Train NN")
        trainNN.addActionListener { e: ActionEvent? ->
            val i = JOptionPane.showConfirmDialog(mainFrame, "Are you sure this may take while to train?")
            if (i == JOptionPane.OK_OPTION) {
                val progressBar = ProgressBar(mainFrame!!)
                SwingUtilities.invokeLater { progressBar.showProgressBar("Training may take one or two minutes...") }
                Executors.newCachedThreadPool().submit {
                    try {
                        LOGGER.info("Start to train Neural Network")
                        neuralNetwork.train(trainField!!.value as Int, testField!!.value as Int)
                        LOGGER.info("End of training")
                    } finally {
                        progressBar.setVisible(false)
                    }
                }
            }
        }
        val trainCNN = JButton("Training Convolutional NN")
        trainCNN.addActionListener { e: ActionEvent? ->
            val i = JOptionPane.showConfirmDialog(
                mainFrame,
                "Are you sure, training requires >10GB memory and more than 1 hour?"
            )
            if (i == JOptionPane.OK_OPTION) {
                val progressBar = ProgressBar(mainFrame!!)
                SwingUtilities.invokeLater { progressBar.showProgressBar("Training may take a while") }
                Executors.newCachedThreadPool().submit {
                    try {
                        LOGGER.info("Start to train Convolutional Neural Network")
                        convolutionalNeuralNetwork.train(trainField!!.value as Int, testField!!.value as Int)
                        LOGGER.info("End of the training")
                    } catch (e1: IOException) {
                        LOGGER.error("something went wrong CNN not trained $e1")
                        throw RuntimeException(e1)
                    } finally {
                        progressBar.setVisible(false)
                    }
                }
            }
        }
        topPanel.add(trainCNN)
        topPanel.add(trainNN)
        val tL = JLabel("Training the Data")
        tL.font = sansSerifBold
        topPanel.add(tL)
        modelTrainSize = SpinnerNumberModel(TRAIN_SIZE, 10000, 60000, 1000)
        trainField = JSpinner(modelTrainSize)
        trainField!!.font = sansSerifBold
        topPanel.add(trainField)
        val ttL = JLabel("Test the Data")
        ttL.font = sansSerifBold
        topPanel.add(ttL)
        modelTestSize = SpinnerNumberModel(TEST_SIZE, 1000, 10000, 500)
        testField = JSpinner(modelTestSize)
        testField!!.font = sansSerifBold
        topPanel.add(testField)
        mainPanel!!.add(topPanel, BorderLayout.NORTH)
    }

    private fun createMainFrame(): JFrame {
        val mainFrame = JFrame()
        mainFrame.title = "Digit Recognizer"
        mainFrame.defaultCloseOperation = WindowConstants.DISPOSE_ON_CLOSE
        mainFrame.setSize(FRAME_WIDTH, FRAME_HEIGHT)
        mainFrame.setLocationRelativeTo(null)
        mainFrame.addWindowListener(object : WindowAdapter() {
            override fun windowClosed(e: WindowEvent) {
                System.exit(0)
            }
        })
        val imageIcon = ImageIcon("icon.png")
        mainFrame.iconImage = imageIcon.image
        return mainFrame
    }

    private fun addSignature() {
        val signature = JLabel("NM-DEV", JLabel.HORIZONTAL)
        signature.font = Font(Font.SANS_SERIF, Font.ITALIC, 20)
        signature.foreground = Color.BLUE
        mainPanel!!.add(signature, BorderLayout.SOUTH)
    }

    companion object {
        private val LOGGER = LoggerFactory.getLogger(UI::class.java)
        private const val FRAME_WIDTH = 1200
        private const val FRAME_HEIGHT = 628
        private fun scale(imageToScale: BufferedImage): BufferedImage {
            val resizeOp = ResampleOp(28, 28)
            resizeOp.filter = ResampleFilters.getLanczos3Filter()
            return resizeOp.filter(imageToScale, null)
        }

        private fun toBufferedImage(img: Image?): BufferedImage {
            // for creating a buffered image
            val bimage = BufferedImage(img!!.getWidth(null), img.getHeight(null), BufferedImage.TYPE_INT_ARGB)

            // Draw the image on to the buffered image
            val bGr = bimage.createGraphics()
            bGr.drawImage(img, 0, 0, null)
            bGr.dispose()

            // Returned the buffered image
            return bimage
        }

        private fun transformImageToOneDimensionalVector(img: BufferedImage): DoubleArray {
            val imageGray = DoubleArray(28 * 28)
            val w = img.width
            val h = img.height
            var index = 0
            for (i in 0 until w) {
                for (j in 0 until h) {
                    val color = Color(img.getRGB(j, i), true)
                    val red = color.red
                    val green = color.green
                    val blue = color.blue
                    val v = 255 - (red + green + blue) / 3.0
                    imageGray[index] = v
                    index++
                }
            }
            return imageGray
        }
    }
}
				
			

This code will execute and will able to run the project which generate the tells the user to know what the process is going on and with the message displaying and to look into the process while running

				
					package gaurav

// the required libraries and classed has been attached to this class
import org.slf4j.LoggerFactory
import gaurav.ui.ProgressBar
import gaurav.ui.UI
import java.io.File
import java.util.concurrent.Executors
import javax.swing.JFrame

object Run {
    private val LOGGER = LoggerFactory.getLogger(Run::class.java)
    private val mainFrame = JFrame()
    @Throws(Exception::class)
    @JvmStatic
    fun main(args: Array) {
        LOGGER.info("Here We Go")
        setHadoopHomeEnvironmentVariable()
        val progressBar = ProgressBar(mainFrame, true)
        progressBar.showProgressBar("This may take a while")
        val ui = UI()
        Executors.newCachedThreadPool().submit {
            try {
                ui.initUI()
            } finally {
                progressBar.setVisible(false)
                mainFrame.dispose()
            }
        }
    }

    @Throws(Exception::class)
    private fun setHadoopHomeEnvironmentVariable() {
        val hadoopEnvSetUp = HashMap()
        hadoopEnvSetUp["HADOOP_HOME"] = File("resources/winutils-master/hadoop-2.8.1").absolutePath
        val processEnvironmentClass = Class.forName("java.lang.ProcessEnvironment")
        val theEnvironmentField = processEnvironmentClass.getDeclaredField("theEnvironment")
        theEnvironmentField.isAccessible = true
        val env = theEnvironmentField[null] as MutableMap
        env.clear()
        env.putAll(hadoopEnvSetUp)
        val theCaseInsensitiveEnvironmentField =
            processEnvironmentClass.getDeclaredField("theCaseInsensitiveEnvironment")
        theCaseInsensitiveEnvironmentField.isAccessible = true
        val cienv = theCaseInsensitiveEnvironmentField[null] as MutableMap
        cienv.clear()
        cienv.putAll(hadoopEnvSetUp)
    }
}
				
			

This part handles and operate the imaging open before and after in terms of size of it and changes in it after the execution of the process

				
					package digit.recogniser.data

import org.junit.Assert
import org.junit.Test
import gaurav.data.IdxReader.loadData
import gaurav.data.LabeledImage
import gaurav.data.IdxReader
import gaurav.data.IdxReader.INPUT_IMAGE_PATH
import gaurav.data.IdxReader.INPUT_LABEL_PATH
import java.io.FileInputStream
import java.io.IOException

class IdxReaderTest {
    @Test //small data
    fun loadDataOne_Thousand() {
        val labeledImages = loadData(100)
    }
    @Test
    fun testmagicNumbersForBigDataset() {
        try {
            FileInputStream(INPUT_IMAGE_PATH).use { inImage ->
                FileInputStream(INPUT_LABEL_PATH).use { inLabel ->

                    println("Available bytes before read: " + inImage.available())
                    val magicNumberImages: Int =
                        inImage.read() shl 24 or (inImage.read() shl 16) or (inImage.read() shl 8) or inImage.read()
                    val numberOfImages: Int =
                        inImage.read() shl 24 or (inImage.read() shl 16) or (inImage.read() shl 8) or inImage.read()
                    val numberOfRows: Int =
                        inImage.read() shl 24 or (inImage.read() shl 16) or (inImage.read() shl 8) or inImage.read()
                    val numberOfColumns: Int =
                        inImage.read() shl 24 or (inImage.read() shl 16) or (inImage.read() shl 8) or inImage.read()
                    Assert.assertTrue(magicNumberImages == 2051)
                    Assert.assertTrue(numberOfImages == 60000)
                    Assert.assertTrue(numberOfRows == 28)
                    Assert.assertTrue(numberOfColumns == 28)

                    println("Available bytes after read: " + inImage.available())
                    println("Available bytes before read: " + inLabel.available())
                    val magicNumberLabels: Int =
                        inLabel.read() shl 24 or (inLabel.read() shl 16) or (inLabel.read() shl 8) or inLabel.read()
                    val numberOfLabels: Int =
                        inLabel.read() shl 24 or (inLabel.read() shl 16) or (inLabel.read() shl 8) or inLabel.read()
                    Assert.assertTrue(magicNumberLabels == 2049)
                    Assert.assertTrue(numberOfLabels == 60000)
                    println("Available bytes after read: " + inLabel.available())
                }
            }
        } catch (e: IOException) {
            e.printStackTrace()
        }
    }
}
				
			

Here we can see that the final layer of output has been generated and it will finally compare itself with this layer to generate the final result as we can see below

Output