update

.gitignore

@@ -2,4 +2,5 @@
 .jupyter
 Manifest.toml
 _build
-notebooks/data/
+notebooks/data/
+.vscode

Project.toml

@@ -5,6 +5,7 @@ version = "0.1.0"
 
 [deps]
 CSV = "336ed68f-0bac-5ca0-87d4-7b16caf5d00b"
+CUDA = "052768ef-5323-5732-b1bb-66c8b64840ba"
 CairoMakie = "13f3f980-e62b-5c42-98c6-ff1f3baf88f0"
 DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0"
 Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f"

_toc.yml

@@ -35,6 +35,7 @@ parts:
     chapters:
     - file: notebooks/chapter_builders_guide/parameters.ipynb
     - file: notebooks/chapter_builders_guide/read-write.ipynb
+    - file: notebooks/chapter_builders_guide/use-gpu.ipynb
   - caption: Recurrent Neural Networks
     numbered: true
     chapters:

notebooks/chapter_builders_guide/use-gpu.ipynb

@@ -0,0 +1,147 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "vscode": {
+     "languageId": "julia"
+    }
+   },
+   "source": [
+    "# GPUs\n",
+    "\n",
+    "## Computing Devices\n",
+    "\n",
+    "We can query the number of available GPUs."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 29,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "CUDA.DeviceIterator() for 1 devices:\n",
+       "0. NVIDIA GeForce GTX 970"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "using CUDA,Flux\n",
+    "\n",
+    "CUDA.devices()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Vectors and GPUs\n",
+    "\n",
+    "### Storage on the GPU\n",
+    "\n",
+    "There are several ways to store a Vector on the GPU. For example, we can specify a storage device when creating a Vector. Next, we create the Vector variable X on the first gpu. The Vector created on a GPU only consumes the memory of this GPU. We can use the nvidia-smi command to view GPU memory usage. In general, we need to make sure that we do not create data that exceeds the GPU memory limit."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 30,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "3×2 CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}:\n",
+       " 1.0  1.0\n",
+       " 1.0  1.0\n",
+       " 1.0  1.0"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "X = cu(ones(3,2))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Neural Networks and GPUs\n",
+    "\n",
+    "Similarly, a neural network model can specify devices. The following code puts the model parameters on the GPU."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 31,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "Chain(\n",
+       "  Dense(3 => 1),                        \u001b[90m# 4 parameters\u001b[39m\n",
+       ") "
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "model = Chain(Dense(3=>1))\n",
+    "model = model |> gpu"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "For example, when the input is a tensor on the GPU, the model will calculate the result on the same GPU."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 32,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "1×2 CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}:\n",
+       " 0.0954295  0.0954295"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "model(X)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Julia 1.10.3",
+   "language": "julia",
+   "name": "julia-1.10"
+  },
+  "language_info": {
+   "file_extension": ".jl",
+   "mimetype": "application/julia",
+   "name": "julia",
+   "version": "1.10.3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}