Anomaly Detection in TensorFlow and Keras Utilizing the Autoencoder Methodology | by Rashida Nasrin Sucky | Sep, 2023

A cutting-edge unsupervised technique for noise elimination, dimensionality discount, anomaly detection, and extra

All of the tutorials about TensorFlow and neural networks I’ve shared till now have been about supervised studying. This one shall be in regards to the Autoenocder which is an unsupervised studying approach. If I wish to categorical it merely, autoencoders scale back the noises from the info by compressing the enter knowledge, and encoding and reconstructing the info. That method autoencoders can scale back the dimensionality or the noise of the info and give attention to the true point of interest of the enter knowledge.

As you may see from the introduction to the autoencoders right here there’s multiple course of required.

1. First, a mannequin to compress the enter knowledge which is the encoder mannequin.
2. Then one other mannequin to reconstruct the compressed knowledge that must be as shut because the enter knowledge which is a decoder mannequin.

On this course of, it could actually take away the noise, scale back the dimensionality, and clear up the enter knowledge.

On this tutorial, I’ll clarify intimately how an autoencoder works with a working instance.

For this instance, I selected to make use of a public dataset (Apache License 2.0) named deep_weeds.

import tensorflow as tf
import tensorflow_datasets as tfds
ds = tfds.load('deep_weeds', break up='prepare', shuffle_files=True)

Information Preparation

We have to put together a dataset for this unsupervised anomaly detection instance. Just one class shall be taken as our fundamental class that shall be thought-about because the legitimate class. And I’ll put a number of knowledge from one other class as an anomaly. Then we’ll develop the mannequin to see if we will discover that few anomaly knowledge.

I selected class 5 because the legitimate class and sophistication 1 because the anomaly. Within the code block beneath, I’m taking all the info of lessons 5 and 1 first and creating lists of the pictures and their corresponding labels.

import numpy as np
images_main = []
images_anomaly = []
labels_main= []
labels_anomaly = []
ds = ds.prefetch(tf.knowledge.AUTOTUNE)
for instance in ds…