tutorials/spyker_and_pytorch.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c34f5221",
   "metadata": {},
   "source": [
    "# Tutorial 1: Spyker and PyTorch\n",
    "In this tutorial we will see some usages of the library, and its interaction with PyTorch. We do this by creating a network to classify the MNIST dataset. First We start with importing the needed tools."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "90b670ac",
   "metadata": {},
   "outputs": [],
   "source": [
    "import spyker, torch\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.decomposition import PCA\n",
    "from torch.utils.data import TensorDataset, DataLoader"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8189d26",
   "metadata": {},
   "source": [
    "Then we load the MNIST dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4c3c3153",
   "metadata": {},
   "outputs": [],
   "source": [
    "batch, root = 64, './MNIST'\n",
    "trainx, trainy, testx, testy = spyker.read_mnist(\n",
    "    root+'/train-images-idx3-ubyte', root+'/train-labels-idx1-ubyte',\n",
    "    root+'/t10k-images-idx3-ubyte', root+'/t10k-labels-idx1-ubyte')\n",
    "trainx, trainy, testx, testy = spyker.to_torch(trainx, trainy, testx, testy)\n",
    "train = DataLoader(TensorDataset(trainx, trainy), batch_size=batch)\n",
    "test = DataLoader(TensorDataset(testx, testy), batch_size=batch)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49a53474",
   "metadata": {},
   "source": [
    "We need to transform the input images into spikes. This is where the `Transform` module comes in. We define this module here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "270c6353",
   "metadata": {},
   "outputs": [],
   "source": [
    "class Transform:\n",
    "    def __init__(self, device):\n",
    "        self.device = device\n",
    "        self.filter = spyker.LoG(3, [.5, 1, 1.5, 2], pad=3, device=device, dtype='f32')\n",
    "    \n",
    "    def __call__(self, input):\n",
    "        if self.device.kind == 'cuda': input = input.cuda()\n",
    "        return spyker.code(spyker.threshold(self.filter(input), .01), 15)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "25780fd9",
   "metadata": {},
   "source": [
    "Once we have our spikes, we need a network to process them. We define the `Network` module here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8cccc132",
   "metadata": {},
   "outputs": [],
   "source": [
    "class Network:\n",
    "    def __init__(self, device):\n",
    "        self.conv1 = spyker.Conv(8, 32, 5, pad=2, device=device, dtype='f32')\n",
    "        self.conv2 = spyker.Conv(32, 64, 3, pad=1, device=device, dtype='f32')\n",
    "        self.conv1.stdpconfig = [spyker.STDPConfig(.004, -.003)]\n",
    "        self.conv2.stdpconfig = [spyker.STDPConfig(.004, -.003)]\n",
    "    \n",
    "    def train1(self, input):\n",
    "        output = spyker.inhibit(spyker.threshold(self.conv1(input), 16))\n",
    "        self.conv1.stdp(input, spyker.convwta(output, 3, 5), spyker.fire(output))\n",
    "        \n",
    "    def train2(self, input):\n",
    "        input = spyker.pool(spyker.fire(self.conv1(input), 16), 2)\n",
    "        output = spyker.inhibit(spyker.threshold(self.conv2(input), 5))\n",
    "        self.conv2.stdp(input, spyker.convwta(output, 1, 8), spyker.fire(output))\n",
    "    \n",
    "    def __call__(self, input):\n",
    "        input = spyker.pool(spyker.fire(self.conv1(input), 16), 2)\n",
    "        input = spyker.pool(spyker.fire(self.conv2(input), 5), 3)\n",
    "        return (15 - spyker.gather(input)).flatten(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ced04fc9",
   "metadata": {},
   "source": [
    "We need to get the output of the network for the entire datatset in order to use scikit-learn's tools on it. We define the `Total` function that does this operation here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "567a92ca",
   "metadata": {},
   "outputs": [],
   "source": [
    "def Total(network, transform, dataset):\n",
    "    data_total, target_total = [], []\n",
    "    for data, target in dataset:\n",
    "        data_total.append(network(transform(data)).cpu())\n",
    "        target_total.append(target)\n",
    "    return torch.cat(data_total), torch.cat(target_total)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae57e434",
   "metadata": {},
   "source": [
    "We need to update the learning rates of our STDP configurations after training on a specified number of samples. the `Update` function, which we define here, does this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a8beda7b",
   "metadata": {},
   "outputs": [],
   "source": [
    "def Update(config):\n",
    "    rate = config.negative / config.positive\n",
    "    pos = min(config.positive * 2, .1)\n",
    "    config.positive, config.negative = pos, pos * rate"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b4aaaee",
   "metadata": {},
   "source": [
    "Spyker operations can be used on both CPUs and GPUs. Depending on the hardware of your machine and Spyker's build configuration, we need to specify the device to run our network on. We will use CUDA if it is avialable. If not, we fall back on the CPU instead."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cced9fcd",
   "metadata": {},
   "outputs": [],
   "source": [
    "device = spyker.device('cuda' if spyker.cuda_available() else 'cpu')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f491165",
   "metadata": {},
   "source": [
    "We need instances of the `Transform` and `Network` modules. We create them here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c2b49470",
   "metadata": {},
   "outputs": [],
   "source": [
    "transform = Transform(device)\n",
    "network = Network(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3b0d19e",
   "metadata": {},
   "source": [
    "Now that we have our instances, we train the layers of the network one by one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ca19cd3a",
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "from tqdm.notebook import tqdm\n",
    "\n",
    "for i, (data, _) in enumerate(tqdm(train, \"Training Layer 1\")):\n",
    "    if (i + 1) % 10 == 0: Update(network.conv1.stdpconfig[0])\n",
    "    network.train1(transform(data))\n",
    "spyker.quantize(network.conv1.kernel, 0, .5, 1)\n",
    "\n",
    "for i, (data, _) in enumerate(tqdm(train, \"Training Layer 2\")):\n",
    "    if (i + 1) % 10 == 0: Update(network.conv2.stdpconfig[0])\n",
    "    network.train2(transform(data))\n",
    "spyker.quantize(network.conv2.kernel, 0, .5, 1);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5273d5f8",
   "metadata": {},
   "source": [
    "Once we have trained our network, we get the output of the network for the training and testing sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c22fd2b",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_data, train_target = Total(network, transform, train)\n",
    "test_data, test_target = Total(network, transform, test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67f73f4f",
   "metadata": {},
   "source": [
    "We can reduce the number of features of the network output to improve classification speed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "50cafe6c",
   "metadata": {},
   "outputs": [],
   "source": [
    "pca = PCA(n_components=200).fit(train_data, train_target)\n",
    "train_data, test_data = pca.transform(train_data), pca.transform(test_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "539288b9",
   "metadata": {},
   "source": [
    "Finally, we will run the SVM classifier and print the accuracy we get."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "49306bcc",
   "metadata": {},
   "outputs": [],
   "source": [
    "target = SVC(C=2.4).fit(train_data, train_target).predict(test_data)\n",
    "accuracy = (torch.tensor(target) == test_target).sum() / len(test_target)\n",
    "print(f\"Final Accuracy: {accuracy * 100 :.2f}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1569746",
   "metadata": {},
   "source": [
    "As we can see, our network reaches a good accuracy (if everything goes well, should be a little higher than 99%)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}