- \n
- Introduction to Perceptrons<\/a><\/li>\n
- Perceptron Structure<\/a><\/li>\n
- Perceptrons in Machine Learning and Deep Learning<\/a><\/li>\n
- Frequently Asked Questions<\/a><\/li>\n<\/ul>\n
<\/a><\/p>\n
A Quick Introduction to Perceptrons<\/h2>\n
You’re probably familiar with deep learning and artificial neural networks or at least heard the terms.<\/p>\n
Perceptrons are related to deep learning and ANNs. <\/p>\n
In fact, perceptrons are one of the simplest types of artificial neuron, so we can think of them as the foundation of neural networks.<\/p>\n
Perceptrons are Based on Biological Neurons<\/h3>\n
Originally proposed in 1957, perceptrons were one of the earliest components of artificial neural networks. <\/p>\n
The structure of the perceptron is based on the anatomy of neurons.<\/p>\n
Neurons have several parts, but for our purposes, the most important parts are the dendrites, which receive inputs from other neurons, and the axon, which produces outputs. <\/p>\n
![\"An](\"https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/07\/neuron-anatomy.jpg\")
<\/p>\nThe outputs of an axon can then serve as the inputs to the dendrites of another neuron (which allow neurons to operate in networks, like in the human brain).<\/p>\n
Importantly, a neuron will produce an output only if it receives a sufficient number of input signals at the dendrites. Remember: neurons in a brain are organized into a network of many neurons. So the dendrites of a neuron typically receive inputs from many other neurons. <\/p>\n
Neuron Activation<\/h3>\n
Neurons “fire” \u2013 that is, produce an output \u2013\u00a0in an all or nothing way. The outputs of a neuron are essentially 0 or 1. On or off.<\/p>\n
A neuron will “fire” if the input signals at the dendrites are sufficiently large, collectively. If the amount of input signal at the dendrites is high enough, the neuron will “fire” an produce an output. But if the amount of input signal is insufficient, the neuron will not produce a output. Put simply, the neuron sums up the inputs, and if the collective input signals meet a certain threshold, then it will produce an output. If the collective input signals are under the threshold, it will not produce an output.<\/p>\n
This is, of course, a very simple explanation of how a neuron works (because they are very complex at the chemical level), but it’s roughly accurate.<\/p>\n
Why is this important?<\/p>\n
Because scientists suggested that we would be able to create artificial<\/em> neurons, modeled after animal neurons, which would be able to process input data in a similar way.<\/p>\n
<\/a><\/p>\n
How Perceptrons are Structured<\/h2>\n
Perceptrons are essentially artificial neurons.<\/p>\n
They have a very similar structure to biological neurons.<\/p>\n
![\"An](\"https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/07\/perceptron_structure_v1.png\")
<\/p>\nThere are inputs, the inputs are weighted, and if the weighted inputs collectively reach a certain threshold, then the perceptron produces an output.<\/p>\n
Let’s quickly discuss a few of these parts of a perceptron.<\/p>\n
Inputs<\/h3>\n
Perceptrons have multiple numeric input values,
![\"x_i\"](\"https:\/\/www.sharpsightlabs.com\/wp-content\/ql-cache\/quicklatex.com-3a7857b57d3ca30f32198442ddfde770_l3.png\" "\\"Rendered")
.<\/p>\nIn a perceptron, these input x values can be real numbers.<\/p>\n
However, in other artificial neuron models, the allowed input values are constrained. For example, in the McCulloch-Pitts neuron (which is slightly different from a perceptron), the input values are binary 0\/1 values.<\/p>\n
Weights<\/h3>\n
In a perceptron, every input value is multiplied by a weight,
![\"w_i\"](\"https:\/\/www.sharpsightlabs.com\/wp-content\/ql-cache\/quicklatex.com-eabb3273ce6c739949b910d293dd7b91_l3.png\" "\\"Rendered")
.<\/p>\nSo collectively, we have the inputs
, each multiplied by their respective weight, which produces the values ![\"x_iw_i\"](\"https:\/\/www.sharpsightlabs.com\/wp-content\/ql-cache\/quicklatex.com-3327d26be4e44d3b1abcb10fe9fd0c83_l3.png\" "\\"Rendered")
.<\/p>\nAlternatively, if you think of the input x values as a vector
![\"\textbf{x}\"](\"https:\/\/www.sharpsightlabs.com\/wp-content\/ql-cache\/quicklatex.com-8de909fe7225645a20fe5fa3623f76fc_l3.png\" "\\"Rendered")
, and the weights also as a vector ![\"\textbf{w}\"](\"https:\/\/www.sharpsightlabs.com\/wp-content\/ql-cache\/quicklatex.com-5a2d840d0a2671b7af64e522283b4fd3_l3.png\" "\\"Rendered")
, then the inputs multiplied by the vector can be thought of as the product ![\"\textbf{x}^\top\textbf{w}\"](\"https:\/\/www.sharpsightlabs.com\/wp-content\/ql-cache\/quicklatex.com-f0c50dfbd35586708fcd25dabc26ecf9_l3.png\" "\\"Rendered")
.<\/p>\nSummation<\/h3>\n
The perceptron takes the product of the inputs and the weights, and sums them up<\/em>.<\/p>\n
This can be represented with the equation:<\/p>\n
(1) <\/span> <\/span>
![\"\begin{equation*}](\"https:\/\/www.sharpsightlabs.com\/wp-content\/ql-cache\/quicklatex.com-0974add48e45a430012ce3ee78f0271d_l3.png\" "\\"Rendered")
<\/p>\nBut in a perceptron and other forms of artificial neurons, we typically apply a function to these summed weighted inputs, which brings us to the activation function.<\/p>\n
Step Function (i.e., Activation Function)<\/h3>\n
As you learn more about artificial neurons and neural networks, you’ll learn a lot more about activation functions<\/em>.<\/p>\n
An activation function is a function that’s applied to the summed inputs times weights,
![\"\sum_{i = 1}^{n}x_iw_i\"](\"https:\/\/www.sharpsightlabs.com\/wp-content\/ql-cache\/quicklatex.com-8c85fd9321d42da498dbef51d99bbb35_l3.png\" "\\"Rendered")
, to determine if the neuron should activate, and the exact form of the output if it does activate.<\/p>\nThere are actually a variety of different activation functions, but the simplest is arguably the step function.<\/p>\n
The step function, as the name implies, visually looks like a step.<\/p>\n
![](\"https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/07\/step-function_ggplot2_v2.png\")
<\/p>\nIn the step function visualized above, if the input value is greater than 0, then the output of the step function is 1.<\/p>\n
If the input value is less than 0 or equal to 0, then the output of the step function is 0.<\/p>\n
So the step function forces the output into a binary 0\/1 range.<\/p>\n
The step activation function is important for the final output, which we’ll see in the next section.<\/p>\n
Output<\/h3>\n
We can model the output of a perceptron like this:<\/p>\n
(2) <\/span> <\/span>
![\"\begin{equation*}](\"https:\/\/www.sharpsightlabs.com\/wp-content\/ql-cache\/quicklatex.com-9d7bc1e246525a1c697cc5eddd7ac266_l3.png\" "\\"Rendered")
<\/p>\nThat’s sort of an English-language version.<\/p>\n
A slightly more mathematical version is as follows:<\/p>\n
(3) <\/span> <\/span>
![\"\begin{equation*}](\"https:\/\/www.sharpsightlabs.com\/wp-content\/ql-cache\/quicklatex.com-666d5136bb41e49b3a35993042f6f7d4_l3.png\" "\\"Rendered")
<\/p>\nAgain though, this assumes that we’re strictly using a step function as our activation function (i.e., that we’re modeling a traditional perceptron).<\/p>\n
<\/a><\/p>\n
Perceptrons in Machine Learning and Deep Learning<\/h2>\n
So now that we’ve looked at the structure of a Perceptron, let’s talk about how we use them and where they sit in the overall discussion of machine learning and deep learning.<\/p>\n
Perceptrons are a type of Binary Linear Classifier<\/h3>\n
Perceptrons are a type of simple machine learning model<\/a>.<\/p>\n
- Perceptron Structure<\/a><\/li>\n
![\"An](\"https:\/\/www.sharpsightlabs.com\/wp-content\/uploads\/2023\/07\/multilayer-perceptron.png\")
<\/p>\n