merge updates

.gitignore

@@ -1,4 +1,4 @@
 __pycache__/
 .DS_Store
-mnist/
-cifar10/
+semiHebb_learning/mnist/
+semiHebb_learning/cifar10/

README.md

@@ -1,25 +1,40 @@
-# biologically plausible neural networks
+# Biologically plausible neural networks
 
-## Pretrain lighter vision models
+In the pursuit of artificial intelligence that mirrors the deftness of human cognition, the concept of biological plausibility stands as a beacon, guiding the design of neural networks toward the intricate workings of the human brain. A neural network that is considered biologically plausible emulates the structure and functions of the biological nervous system, often with the purpose of improving the performance of neural networks or gaining insights into processes of the biological brain. 
+
+while backpropagation (BP) is a cornerstone in training modern neural networks, it deviates from how biological neural systems function. Key differences include BP's reliance on inter-layer weight dynamics, unlike the local information transmission in biological neurons, its use of symmetric weights for both forward and backward passes which contrasts with the one-directional, asymmetric nature of biological synapses, and its continuous output neuron firing, as opposed to the all-or-none firing based on a threshold in biological neurons. 
+Recognizing these discrepancies, this project focuses on exploring neural network techniques that better mimic human brain functions. The aim is to investigate how these biologically inspired alternatives to backpropagation could enhance the performance and interpretability of neural networks. 
+
+A full version report can be found here: [LINK TO PDF]
+
+## Folder Explanation 
+
+- hybridBio_learning: A PyTorch implementation of “Unsupervised learning by competing hidden units” MNIST classifier, combining with Feedback alignment. Original descriptive documentation can be found at [here](https://github.com/clps1291-bioplausnn/hybrid-bioLearning).
+
+- semiHebb_learning
+
+### ## Pretrain lighter vision models
 Recognizing the need for more accessible alternatives to large pretrained vision models on imagenet, this repo aims to provide models pretrained on smaller datasets like MNIST and CIFAR10. These lighter and more manageable models are pretrained for easy import and utilization, facilitating quick experimentation and integration into projects where resources are limited. 
 
 The MNIST database contains 60,000 training images and 10,000 testing images.
 
 The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images.
+### Learning rules
 
-## Learning rules
+### Neural networks
 
-## Neural networks
-
-## Evaluation
+### Evaluation
 
 
 
 Reference:
 
-1.[PyTorch CIFAR10 by huyvnphan](https://github.com/huyvnphan/PyTorch_CIFAR10)
+1. [PyTorch CIFAR10 by huyvnphan](https://github.com/huyvnphan/PyTorch_CIFAR10)
+
+2. [MNIST_database](https://en.wikipedia.org/wiki/MNIST_database)
 
-2.https://en.wikipedia.org/wiki/MNIST_database
+3. [Unsupervised Bio Classifier](https://github.com/gatapia/unsupervised_bio_classifier)
 
+4. [Linear FA implementation](https://github.com/L0SG/feedback-alignment-pytorch)
 
 Except for torchvision models, [GluonCV](https://github.com/dmlc/gluon-cv/tree/master/gluoncv/model_zoo) includes many sota models in CV.

hybridBio_learning/0020_refactor.ipynb

@@ -0,0 +1,140 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!rm -r ../data/logs/src_tb_logs"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%load_ext autoreload\n",
+    "%autoreload 2\n",
+    "%matplotlib inline\n",
+    "%load_ext tensorboard\n",
+    "\n",
+    "import os\n",
+    "cpu='0'\n",
+    "os.environ['CUDA_VISIBLE_DEVICES']=cpu\n",
+    "\n",
+    "from bio_learn2 import *"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "train_data, test_data: torch.Size([60000, 784]) torch.Size([60000]) torch.Size([10000, 784]) torch.Size([10000])\n"
+     ]
+    }
+   ],
+   "source": [
+    "(train_X, train_y), (test_X, test_y) = get_data('train'), get_data('test')\n",
+    "print('train_data, test_data:', train_X.shape, train_y.shape, test_X.shape, test_y.shape)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "file = '../data/0020_unsupervised_weights.pkl'\n",
+    "if os.path.isfile(file): weights = torch.load(file)\n",
+    "else: \n",
+    "    weights = get_unsupervised_weights(train_X, n_hidden=2000, n_epochs=200, batch_size=3584)\n",
+    "    torch.save(weights, file)\n",
+    "tb.add_histogram('weights-hist', weights)\n",
+    "tb.add_figure('weights', draw_unsupervised_weights(weights, 10, 10, 28))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Final Accuracy: 0.92 Took: 12s\n"
+     ]
+    }
+   ],
+   "source": [
+    "n_classes = 10\n",
+    "model = BioClassifier(weights, n_classes).cuda()\n",
+    "tb.add_text('model-bio', 'BioClassifier')\n",
+    "tb.add_graph(model, input_to_model=torch.randn(1, 28*28).cuda())\n",
+    "run_test(train_X, train_y, test_X, test_y, model, epochs=10, loss=BioLoss(n_classes), batch_size=1024, lr=1e-4)  \n",
+    "# Final Accuracy: 0.97 Took: 55s"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model_file = '../data/0020_BioClassifier.pkl'\n",
+    "torch.save(model, model_file)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model2 = SimpleConvNet(n_classes)\n",
+    "tb.add_text('model-cnn', 'SimpleConvNet')\n",
+    "tb.add_graph(model2, input_to_model=torch.randn(1, 28*28).cuda())\n",
+    "run_test(train_X, train_y, test_X, test_y, model2, epochs=100, loss=torch.nn.NLLLoss(), batch_size=1024, lr=1e-4)  \n",
+    "# Final Accuracy: 0.98 Took: 95s"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model_file_2 = '../data/0020_SimpleConvClassifier.pkl'\n",
+    "torch.save(model2, model_file_2)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.6.7"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}