Open Source Neural Network Library

I am looking for an open source neural network library. So far, I have looked at FANN, WEKA, and OpenNN. Are the others that I should look at? The criteria, of course, is documentation, examples, and ease of use.

Last update: 2020/03/24 (I will update this answer from time to time...)

Simple Implementations of Neural Networks

• FANN is a very popular implementation in C/C++ and has bindings for many other languages.
• In scikit-learn (Python) 0.18 (current developement version) there will be an implementation of feed-forward neural networks (API documentation).
• And I must mention my own project, which is called OpenANN (Documentation). It is written in C++ and has Python bindings.

Deep Learning

Because neural networks are quite popular in research and industry at the moment ("deep learning") there are many research libraries available. Most of them are kind of easy to set up, integrate, and use. Although not as easy as the libraries mentioned above. They provide leading edge functionality and high performance (with GPUs etc.). Most of these libraries also have automatic differentiation. You can easily specify new architectures, loss functions etc. and don't have to specify the backpropagation manually.

• TensorFlow from Google (C++/Python)
• PyTorch from Facebook, in Python, can be extended with C/C++
• mxnet (C++, Python, R, Scala, Julia, Matlab, Javascript)
• Deeplearning4j (Java)
• CNTK from Microsoft (training in Python / evaluation in C++/C#/Java/Python)
• Chainer (Python)
• PaddlePaddle from Baidu in CUDA/C++ with Python bindings
• NNabla from Sony in Cuda/C++11 with Python bindings

A performance comparison for GPU-accelerated libraries can be found here (a bit outdated unfortunately). A comparison of GPUs and library versions can be found here.

Inactive:

• Keras: It could use Tensorflow, Theano, and CNTK as a backend. (Now part of tensorflow as its high-level interface.)

• Caffe from Berkeley Vision and Learning Center in C++ with Python bindings

• Darknet: CNNs in C, known for the implementations of the YOLO object detector.

• Neon from Intel Nervana provides very efficient implementations (Python)

• MatConvNet (Matlab)

• Theano (Python) and its high-level APIs:

  • Pylearn 2
  • Theanets
  • scikit-neuralnetwork
  • Lasagne
  • Blocks based on Theano (Python)

• cuda-convnet2 in CUDA/C++ with Python bindings

• Hebel (Python)

• Caffe2 from Facebook in C++ with Python bindings; has been joined with PyTorch

• Neural Networks for Torch 7 (Lua, Torch 7 is a "Matlab-like environment", overview of machine learning algorithms in Torch)

• PyBrain (Python) contains different types of neural networks and training methods.

• Encog (Java and C#)

If you want flexibility in defining network configurations, like sharing parameters or creating different types of convolutional architectures, then you should look at the family of Torch libraries: http://www.torch.ch/.

I haven't gone through the documentation for Torch 7 yet, but documentation for the other versions was pretty decent and the code is very readable (in Lua and C++).

You can use accord.net framework. http://accord-framework.net/

It contains Neural learning algorithms such as Levenberg-Marquardt, Parallel Resilient Backpropagation, the Nguyen-Widrow initialization algorithm, Deep Belief Networks and Restrictured Boltzmann Machines, and many other neural network related items.

Netlab is a commonly used Matlab library. (free and open source)

> The Netlab toolbox is designed to provide the central tools necessary > for the simulation of theoretically well founded neural network > algorithms and related models for use in teaching, research and > applications development. It is extensively used in the MSc by > Research in the Mathematics of Complex Systems. > > The Netlab library includes software implementations of a wide range > of data analysis techniques, many of which are not yet available in > standard neural network simulation packages. Netlab works with Matlab > version 5.0 and higher but only needs core Matlab (i.e. no other > toolboxes are required). It is not compatible with earlier versions of > Matlab.