Deep Belief Networks (DBNs) are a type of artificial neural network that are composed of multiple layers of hidden units, specifically, they are a class of deep neural network which comprises several layers of latent variables or hidden units. DBNs have two main types of layers: visible layers and hidden layers. The visible layers are those that receive inputs from the outside world, while the hidden layers are those that process inputs received from either visible layers or other hidden layers.
How they work
DBNs are made up of several Restricted Boltzmann Machines (RBM) where each RBM layer communicates with both the previous and next layers, but the individual layers are not connected amongst themselves. They're stacked up in such a way that the output of each layer is the input of the next. The connections are bidirectional.
The learning process in DBNs consists of two main phases: an unsupervised pre-training phase and a supervised fine-tuning phase.
In the pre-training phase, a DBN learns to reconstruct its inputs in an unsupervised manner layer by layer. Each layer is trained as an RBM which tries to learn the structure in the input received from the previous layer and then gives out the output to the next layer.
After pre-training, the outputs of the final hidden layer are used as inputs for a standard supervised learning algorithm like logistic regression in the fine-tuning phase.
So, initially, the DBN attempts to understand the structure in the input by trying to model the input distribution, then it fine-tunes the parameters using a labelled dataset to perform tasks such as classification or regression.
The two-phase learning system makes DBNs quite effective at handling unlabeled data, as they can make useful abstractions from the data in the pre-training phase and then fine-tune using a smaller set of labeled data.
Download this guide to delve into the most common LLM security risks and ways to mitigate them.
Lakera Guard protects your LLM applications from cybersecurity risks with a single line of code. Get started in minutes. Become stronger every day.
Several people are typing about AI/ML security. Come join us and 1000+ others in a chat that’s thoroughly SFW.