index.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Appended /home/blackhc/PycharmProjects/blackhc.batchbald/src to paths\n",
      "Switched to directory /home/blackhc/PycharmProjects/blackhc.batchbald\n",
      "%load_ext autoreload\n",
      "%autoreload 2\n"
     ]
    }
   ],
   "source": [
    "#hide\n",
    "import blackhc.project.script\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BatchBALD Redux\n",
    "\n",
    "> Clean reimplementation of \\\"BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning\\\"\n",
    "\n",
    "- toc: False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For an introduction & more information, see  http://batchbald.ml/. The paper can be found at http://arxiv.org/abs/1906.08158.\n",
    "\n",
    "The original implementation used in the paper is available at https://github.com/BlackHC/BatchBALD.\n",
    "\n",
    "We are grateful for fastai's [nbdev](https://nbdev.fast.ai/) which is powering this package.\n",
    "\n",
    "For more information, explore the sections and notebooks in the left-hand menu.\n",
    "The code is available on https://github.com/BlackHC/batchbald_redux, \n",
    "and the website on https://blackhc.github.io/batchbald_redux."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Install"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`pip install batchbald_redux`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Motivation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "BatchBALD is an algorithm and acquisition function for Active Learning in a Bayesian setting using BNNs and MC dropout.\n",
    "\n",
    "The aquisition function is the mutual information between the joint of a candidate batch and the model parameters $\\omega$:\n",
    "\n",
    "$$a_{\\text{BatchBALD}}((y_b)_B) = I[(y_b)_B;\\omega]$$\n",
    "\n",
    "The best candidate batch is one that maximizes this acquisition function. \n",
    "\n",
    "In the paper, we show that this function satisfies sub-modularity, which provides us an optimality guarantee for a greedy algorithm. The candidate batch is selected using greedy expansion.\n",
    "\n",
    "Joint entropies are hard to estimate and, for everything to work, one also has to use consistent MC dropout, which keeps a set of dropout masks fixed while scoring the pool set.\n",
    "\n",
    "To aid reproducibility and baseline reproduction, we provide this simpler and clearer reimplementation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Please cite us\n",
    "\n",
    "```\n",
    "@misc{kirsch2019batchbald,\n",
    "    title={BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning},\n",
    "    author={Andreas Kirsch and Joost van Amersfoort and Yarin Gal},\n",
    "    year={2019},\n",
    "    eprint={1906.08158},\n",
    "    archivePrefix={arXiv},\n",
    "    primaryClass={cs.LG}\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How to use\n",
    "\n",
    "We provide a simple example experiment that uses this package [here](https://blackhc.github.io/batchbald_redux/example_experiment/). \n",
    "\n",
    "To get a candidate batch using BatchBALD, we provide a simple API in [`batchbald_redux.batchbald`](https://blackhc.github.io/batchbald_redux/batchbald/):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "<h4 id=\"get_batchbald_batch\" class=\"doc_header\"><code>get_batchbald_batch</code><a href=\"https://github.com/blackhc/batchbald_redux/tree/master/batchbald_redux/batchbald.py#L67\" class=\"source_link\" style=\"float:right\">[source]</a></h4>\n",
       "\n",
       "> <code>get_batchbald_batch</code>(**`log_probs_N_K_C`**:`Tensor`, **`batch_size`**:`int`, **`num_samples`**:`int`, **`dtype`**=*`None`*, **`device`**=*`None`*)\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from nbdev.showdoc import *\n",
    "from batchbald_redux.batchbald import get_batchbald_batch\n",
    "\n",
    "show_doc(get_batchbald_batch)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also provide a simple implementation of consistent MC dropout in [`batchbald_redux.consistent_mc_dropout`](https://blackhc.github.io/batchbald_redux/consistent_mc_dropout/)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:active_learning]",
   "language": "python",
   "name": "conda-env-active_learning-py"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}