xrnn 1.1.1

EXtremely Rapid Neural Networks (xrnn) is a Python machine learning framework for building layers and neural networks by exposing an easy-to-use interface for building neural networks as a series of layers, similar to Keras, while being as lightweight, fast, compatible and extendable as possible.

Table of Contents

• The advantages of this package over existing machine learning frameworks
• Installation
• Examples
• Features
• Building from Source
• Testing
• Current Design Limitations
• Project Status
• License

The advantages of this package over existing machine learning frameworks

1. Works on any Python version 3.6 (released in 2016) and above.
2. Requires only one dependency, which is Numpy (you most likely already have it).
3. Lightweight in terms of size.
4. Very fast startup time, which is the main version behind developing this project, meaning that importing the package, building a network and straiting training takes less than a second (compared to Tensorflow for example which can take more than 10 seconds).
5. High performance, even on weak hardware, reached 90% validation accuracy on MNIST dataset using a CNN on a 2 core 2.7 GHZ cpu (i7-7500U) in 20 seconds.
6. Memory efficient, uses less RAM than Tensorflow (~25% less) for a full CNN training/inference pipeline.
7. Compatibility, there's no OS-specific code (OS and hardware independent), so the package can pretty much be built and run on any platform that has python >= 3.6 and any C compiler that has come out in the last 20 years.

Installation

Run the following command:

pip install xrnn

• Pre-built distributions (wheels) are provided for pretty much every platform, so the installation should be quick and error-free.
• The source distribution is also present if there isn't a wheel for the platform you are running on.

Examples

This example will show how to build a CNN for classification, add layers to it, train it on dummy data, validate it and use it for inference.

import numpy as np
# Create a dummy dataset, which contains 1000 images, where each image is 28 pixels in height and width and has 3 channels.
number_of_samples = 1000
height = 28
width = 28
channels = 3
number_of_classes = 9  # How many classes are in the dataset, for e.g., cat, car, dog, etc.
x_dummy = np.random.random((number_of_samples, height, width, channels))
y_dummy = np.random.randint(number_of_classes, size=(number_of_samples, ))

# Build the network.
batch_size = 64  # How many samples are in each batch (slice) of the data.
epochs = 2  # How many full iterations over the dataset to train the network for.

from xrnn.model import Model  # The neural network blueprint (houses the layers)
from xrnn.layers import Conv2D, BatchNormalization, Flatten, Dense, MaxPooling2D
from xrnn.activations import ReLU, Softmax
from xrnn.losses import CategoricalCrossentropy  # This loss is used for classification problems.
from xrnn.optimizers import Adam

model = Model()
model.add(Conv2D(16, 3, 2, 'same'))
model.add(ReLU())
model.add(BatchNormalization())
model.add(MaxPooling2D(2, 2, 'same'))
model.add(Flatten())
model.add(Dense(100))
model.add(ReLU())
model.add(Dense(number_of_classes))  # The output layer, has the same number of neurons as the number of unique classes in the dataset.
model.add(Softmax())

model.set(Adam(), CategoricalCrossentropy())
model.train(x_dummy, y_dummy, epochs=epochs, batch_size=batch_size, validation_split=0.1)  # Use 10% of the data for validation.

x_dummy_predict = np.random.random((batch_size, height, width, channels))
prediction = model.inference(x_dummy_predict)  # Same as model.predict(x_dummy_predict).

# Model predicts on batches, so even if one sample is provided, it's turned into a batch of 1, that's why we take
# the first sample.
prediction = prediction[0]

# The model returns a probability for each label, `np.argmax` returns the index with the largest probability.
label = np.argmax(prediction)

print(f"Prediction: {label} - Actual: {y_dummy[0]}.")

And that's it! You've built, trained and validated a convolutional neural network in just a few lines. It's true that the data is random therefor the model isn't going to learn, but this demonstrates how to use the package, just replace 'x_dummy' and 'y_dummy' with actual data and see how the magic happens!
A complete example that demonstrate the above with actual data can be found in example.py script that is bundled with the package. It trains a CNN on MNIST data set, just import the script using from xrnn import example and run example.mnist_example(). Alternatively, you can run it from the command line using python -m xrnn.example
Note that the script will download the MNIST dataset (~12 megabytes) and store it locally.

Features

• xrnn.layers: Implements Conv2D, Dropout, Dense, Max/AvgPool2D, Flatten and BatchNormalization layers.
• xrnn.optimizers: Implements Adam, SGD (with momentum support), RMSprop and Adagrad optimizers.
• xrnn.losees: Implements BinaryCrossentropy, CategoricalCrossentropy and MeanSquaredError (MSE) loss functions.
• xrnn.activations: Implements ReLU, LeakyReLU, Softmax, Sigmoid and Tanh activation functions.
• xrnn.models: Implements the Model class, which is similar to Keras Sequential model. and can be used to build, train, validate and use (inference) a neural network.

For more information on how to use each feature (like the Model class), look at said feature docstring (for example help(Conv2D.forward)). Nonetheless, if you are acquainted with Keras, it's pretty easy to get started with this package because it has almost the same interface as Keras, the only notable difference is that keras model.fit is equivalent to model.train in this package.

Building From Source

Running python -m build should suffice.

^{¹ The compiler used to build the package is in bold.
² You might encounter omp.h file not found error, to fix this, install libomp using sudo apt install libomp-dev.
³ You might encounter an error indicating that omp.h couldn't be found, to fix this, install libomp using Homebrew brew install libomp.}

To set the compiler you want to use for compilation, change the value of compiler under [tool.xrnn] in pyproject.tmol.
It can be a full path to the compiler executable or just the shortcut (gcc for e.g.) if it's in your path.

Testing

For testing the package, first you need to download pytest if you don't have it via:

pip install pytest

Then run:

pytest PATH/TO/TESTS -p xrnn

Note That you need to install the package first if you built it from source

Current Design Limitations

The current design philosophy is compatibility, being able to port/build this package on any OS or hardware, so only native Python/C code is used with no dependence on any third party libraries (except for numpy), this is great for compatibility but not so for performance, because the usage of optimized libraries like Eigen, Intel's oneDNN or cuda is prohibited, which in turn makes this machine learning framework unusable for large datasets and big models.

Project Status

This project still has a long way to go, and I'm currently polishing its API. I might add the following features in the future:

• Add Support for Cuda.
• Optimize CPU performance to match the mature frameworks like Pytorch and Tensorflow.
• Add support for automatic differentiation to make building custom layers easier.
• Add more layers, mainly recurrent, attention and other convolution (transpose, separable) layers.
• Add support for multiple inputs/outputs to the layers and models.

While keeping with the core vision of the project, which is to make it as easy to install, compatible with all platforms and extendable as possible

License

This project is licensed under the MIT license.