-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy path1introtodeeplearning.html
240 lines (182 loc) · 13.4 KB
/
1introtodeeplearning.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<meta name="description" content="Deep Learning Tutorial using Keras">
<meta name="author" content="Lindsey M Kitchell">
<title>Intro to Deep Learning</title>
<!-- Bootstrap core CSS -->
<link href="vendor/bootstrap/css/bootstrap.min.css" rel="stylesheet">
<!-- Custom styles for this template -->
<link href="css/simple-sidebar.css" rel="stylesheet">
<!-- fonts -->
<link href="https://fonts.googleapis.com/css?family=Source+Sans+Pro:300,400,600,700&display=swap" rel="stylesheet">
</head>
<body>
<div class="d-flex" id="wrapper">
<!-- Sidebar -->
<div class="bg-light border-right" id="sidebar-wrapper">
<div class="sidebar-heading">Deep Learning With Keras</div>
<div class="list-group list-group-flush">
<a href="1introtodeeplearning.html" class="list-group-item list-group-item-action bg-light">1. Intro to Deep Learning</a>
<a href="2introtokeras.html" class="list-group-item list-group-item-action bg-light">2. Intro to Keras</a>
<a href="3mlpsinkeras.html" class="list-group-item list-group-item-action bg-light">3. MLPs in Keras</a>
<a href="4cnnsinkeras.html" class="list-group-item list-group-item-action bg-light">4. CNNs in Keras</a>
<a href="5activationfunctions.html" class="list-group-item list-group-item-action bg-light">5. Activation Functions</a>
<a href="6otherkerasfunctions.html" class="list-group-item list-group-item-action bg-light">6. Other Useful Keras Functions</a>
<a href="7lossfunctionsoptimizers.html" class="list-group-item list-group-item-action bg-light">7. Loss Functions and Optimizers</a>
<a href="8evaluatingnns.html" class="list-group-item list-group-item-action bg-light">8. Evaluating Neural Networks</a>
<a href="9datapreprocessing.html" class="list-group-item list-group-item-action bg-light">9. Data Preprocessing</a>
<a href="10regularization.html" class="list-group-item list-group-item-action bg-light">10. Regularization</a>
<a href="11hyperparametertuning.html" class="list-group-item list-group-item-action bg-light">11. Hyperparameter Tuning</a>
</div>
</div>
<!-- /#sidebar-wrapper -->
<!-- Page Content -->
<div id="page-content-wrapper">
<nav class="navbar navbar-expand-lg navbar-light bg-light border-bottom">
<button class="btn btn-primary" id="menu-toggle">Toggle Menu</button>
<button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
<span class="navbar-toggler-icon"></span>
</button>
<div class="collapse navbar-collapse" id="navbarSupportedContent">
<ul class="navbar-nav ml-auto mt-2 mt-lg-0">
<li class="nav-item active">
<a class="nav-link" href="index.html">Home <span class="sr-only">(current)</span></a>
</li>
<li class="nav-item">
<a class="nav-link" target="_blank" href="https://lindseykitchell.weebly.com/">About the Author</a>
</li>
<!--
<li class="nav-item dropdown">
<a class="nav-link dropdown-toggle" href="#" id="navbarDropdown" role="button" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
Dropdown
</a>
<div class="dropdown-menu dropdown-menu-right" aria-labelledby="navbarDropdown">
<a class="dropdown-item" href="#">Action</a>
<a class="dropdown-item" href="#">Another action</a>
<div class="dropdown-divider"></div>
<a class="dropdown-item" href="#">Something else here</a>
</div>
</li>
-->
</ul>
</div>
</nav>
<div class="container-fluid">
<h1>Introduction to Deep Learning</h1>
<hr>
<h2>Deep learning vs. Machine learning</h2>
<h3>Machine learning</h3>
<p>Machine learning is a subfield of artificial intelligence that focuses on teaching computers
how to learn without being specifically programmed for certain tasks. The idea is to create
algorithms that learn from data and make predictions on data. Machine learning can be split
into three categories:</p>
<ol>
<li><em>Supervised learning</em>: the computer is given training data and the desired output from that data (e.g. category labels) and it has to learn from the training data in order to make meaningful predictions based on new data.</li>
<li><em>Unsupervised learning</em>: the computer is given data only and it has to find meaningful structure or groups in the data by itself with no supervision.</li>
<li><em>Reinforcement learning</em>: the computer is interacting with the environment and learning via feedback which behaviors generate rewards. </li>
</ol>
<p> We will be using supervised learning for the task of classification.</p>
<h3>Deep learning</h3>
<p>Deep learning is a subset of machine learning methods that use artificial neural networks.
It is somewhat inspired by the structure of the neurons of the brain. The "deep" in deep
learning refers to the multiple hidden layers of the neural networks. Deep learning has
had great success with several domains, such as images, text, speech, and video. There
are many different types of neural networks. We will focus on the following two:</p>
<ol>
<li>Multi-layer Perceptron (Fully connected network)</li>
<li>Convolutional Neural Network</li>
</ol>
<h2>Why use deep learning?</h2>
<p>Deep learning is most appropriate when there are very complex patterns in the data you
want classified (and also lots of data). </p>
<h2> Perceptrons</h2>
<p>To begin to understand neural networks we should understand what a perceptron is.
A perceptron is a basic artificial neuron. It takes in multiple binary inputs and
produces a single binary output. The inputs are given weights and the output is
determined by whether the sum of the input weights are over or under a certain threshold.
Essentially, it is a device that makes decisions by weighing all the evidence. </p>
<img src="http://neuralnetworksanddeeplearning.com/images/tikz0.png" class="center img-fluid">
<p>Perceptrons can be combined together in layers to create a network. Each perceptron
(node) in a layer weighs the evidence of the perceptrons of the previous layer.
The first layer weighs the input evidence and makes a decision. The second layer weighs
the results of the first layer and makes a more complex/abstract decision, and so on, until
the final layer, where a final decision is made. </p>
<img src="http://neuralnetworksanddeeplearning.com/images/tikz1.png" class="center img-fluid">
<p>The weights of the inputs of these networks are learned through learning algorithms
that minimize the errors in the final output. Nowadays, perceptrons are not really used
because their inputs and outputs are binary. Instead sigmoid neurons or other more powerful
activation functions like ReLU are often used. Sigmoid neurons operate very similar to perceptrons,
however they can take input values between 0 and 1 as well as give output values that are between 0
and 1. More on this later. </p>
<h2>Structure of a Neural Network</h2>
<p>A neural network is a highly structured network that has multiple layers.
The first layer is called the input layer and the final layer is called the output layer.
Every layer in between is referred to as a hidden layer. Each layer is made up of multiple nodes
(e.g. the perceptrons/sigmoid neurons from above). Each node of the hidden and output layers has
its own classifier and the results of the classifiers are passed forward to the next layer. These
multiple layer networks are often called <b>Multi-layer Perceptrons</b> (although they are typically
not actually using perceptrons). Neural networks like these, where the output from one layer is the
input for another, are called <b>feedforward</b> or <b>forward propagation</b> networks. There are neural
networks, called <b>Recurrent Networks</b>, where feedback loops can be used and information is not only
fed forward, however these are less popular and will not be used in this tutorial.</p>
<img src="http://neuralnetworksanddeeplearning.com/images/tikz11.png" class="center img-fluid">
<h3>Input layer</h3>
<p>The number of nodes in the input layer is determined by the number of inputs. For example,
say we are doing image classification and the input is a 28 by 28 pixel image. The model
would have one input node for each pixel. There are 784 (28x28=784) pixels in the image,
so there would be 784 input nodes.</p>
<h3>Output layer</h3>
<p>The number of nodes in the output layer is determined by the number of categories the inputs
are being classified into. If it is a binary classification (2 options), the output layer has 1
node (the options are 0 or 1). If there are 5 possible categories, there are 5 nodes. </p>
<h3>Hidden layers</h3>
<p>The number of nodes in the hidden layers is chosen by the user. This is unfortunately not
determined by anything in particular and is one of the parameters that will need to be optimized.
It is best to test different amounts of nodes and see how it affects the outcome. </p>
<h2>Training a neural network</h2>
<p>In order to train a neural network we use a process called <b>back propagation</b>. It
is essentially a way of progressively correcting mistakes as soon as they are detected. Each layer
of a network has a set of weights that determines the output of that layer for a given set of inputs.
In the beginning those weights are randomly assigned. The network is activated and values are propagated
forward through the network. Since we know the actual value of what the output should be we can calculate
the error in prediction. We then propagate the error backwards through the network and use a gradient descent
algorithm to adjust the weights to minimize the error. This process repeats until the error reaches
below a certain threshold. </p>
<h2>One-hot encoding</h2>
<p>Deep learning requires that the categories used in classification be represented by
one-hot encoding. This is done by representing the category of each input as a vector
the length of the number of categories. The vector is filled with 0's except for the
specific category belonging to that input. For example, imagine I have 10 samples and
the first 5 are from female subjects and the second 5 are from male subjects. I might
normally represent this data in a list of strings: <code>labels = [female, female, female,
female, female, male, male, male, male, male]</code> or better yet a list of integers
where <code>1 = male</code> and <code>2 = female</code>: <code>labels = [2, 2, 2, 2, 2, 1, 1, 1, 1, 1]</code></p>
<p>One-hot encoding would change the represention from 1 and 2 to <code>male = [1, 0]</code>
and <code>female = [0, 1]</code> so that the ten subjects would be represented as:
<code>labels = [[0, 1], [0, 1], [0, 1], [0, 1], [0, 1], [1, 0], [1, 0], [1, 0], [1, 0]]</code>.
This can be further reduced to just <code>labels = [0,0,0,0,0,1,1,1,1,1]</code>. </p>
<p>Another example: If you had five categories: <code>[A, B, C, D, E]</code>, a sample belonging to
category 'C' would be represented by <code>[0, 0, 1, 0, 0]</code>, a sample belonging to category
A would be represented by <code>[1, 0, 0, 0, 0]</code>. </p>
<p><b>Please continue on to the next section <a href="2introtokeras.html">2. Intro to Keras</a>. Keras is the python library we will be using to create neural networks.</b></p>
<p>Figures are from this very helpful resource: <a href="http://neuralnetworksanddeeplearning.com/">neuralnetworksanddeeplearning.com</a></p>
</div>
</div>
<!-- /#page-content-wrapper -->
</div>
<!-- /#wrapper -->
<!-- Bootstrap core JavaScript -->
<script src="vendor/jquery/jquery.min.js"></script>
<script src="vendor/bootstrap/js/bootstrap.bundle.min.js"></script>
<!-- Menu Toggle Script -->
<script>
$("#menu-toggle").click(function(e) {
e.preventDefault();
$("#wrapper").toggleClass("toggled");
});
</script>
</body>
</html>