KUnet.jl is the beginnings of a deep learning package for Julia with emphasis on conciseness, clarity and easy extensibility. It started as a challenge to see how many lines of (readable) code were sufficient to express deep learning algorithms given the right language. A secondary concern was efficiency: being able to run the same code on GPU with minimal trouble. The latest version is less than 1000 lines of code and supports backprop in feedforward nets with convolution, pooling, and inner product layers with/without bias, relu, tanh, sigmoid activations, softmax and quadratic loss, optimization with sgd, momentum, nesterov, adagrad, dropout, L1-L2 regularization, on both CPU/GPU, with Float32/Float64 arrays of 1-5 dimensions. Its speed is competitive with Caffe (here is a benchmark), and I think recurrent and boltzmann nets can be added without too much effort.
You can send me suggestions for improvement (both in coding style and new functionality) using issues or pull requests on GitHub.
I tried to make the code (cpu/gpu) generic and close to how we think of these algorithms mathematically. Getting the same code working on the GPU and the CPU in Julia proved to be a bit challenging and showed that both a more standard treatment of CPU and GPU arrays, and a standard syntax for in-place operations would be welcome additions to the language. I'd like to thank Tim Holy (CUDArt), Nick Henderson (CUBLAS), and Simon Byrne (InplaceOps) for their generous help.
- Beginning deep learning with 500 lines of Julia: my neural net tutorial based on v0.0.1 of KUnet.
- Mocha.jl: a deep learning framework for Julia
- BackpropNeuralNet.jl: another neural net implementation
- UFLDL: deep learning tutorial
- deeplearning.net: resources and pointers to information about Deep Learning
- Some starting points for deep learning, and some more: my blog posts with links