Python package containing all custom and SOTA mathematical backend algorithms used in Machine Learning.
Project description
Echo-AI
Python package containing all mathematical backend algorithms used in Machine Learning. The full documentation for Echo is provided here.
Table of Contents
About
Echo-AI Package is created to provide an implementation of the most promising mathematical algorithms, which are missing in the most popular deep learning libraries, such as PyTorch, Keras and TensorFlow.
Activation Functions
The package contains implementation for following activation functions (:white_check_mark: - implemented functions, :clock9: - functions to be implemented soon, :white_large_square: - function is implemented in the original deep learning package):
# | Function | Equation | Keras | PyTorch | TensorFlow |
---|---|---|---|---|---|
1 | Weighted Tanh | :white_check_mark: | :white_check_mark: | :clock9: | |
2 | Swish | :white_check_mark: | :white_check_mark: | :clock9: | |
3 | ESwish | :white_check_mark: | :white_check_mark: | :clock9: | |
4 | Aria2 | :white_check_mark: | :white_check_mark: | :clock9: | |
5 | ELiSH | :white_check_mark: | :white_check_mark: | :clock9: | |
6 | HardELiSH | :white_check_mark: | :white_check_mark: | :clock9: | |
7 | Mila | :white_check_mark: | :white_check_mark: | :clock9: | |
8 | SineReLU | :white_check_mark: | :white_check_mark: | :clock9: | |
9 | Flatten T-Swish | :white_check_mark: | :white_check_mark: | :clock9: | |
10 | SQNL | :white_check_mark: | :white_check_mark: | :clock9: | |
11 | ISRU | :white_check_mark: | :white_check_mark: | :clock9: | |
12 | ISRLU | :white_check_mark: | :white_check_mark: | :clock9: | |
13 | Bent's identity | :white_check_mark: | :white_check_mark: | :clock9: | |
14 | Soft Clipping | :white_check_mark: | :white_check_mark: | :clock9: | |
15 | SReLU | :white_check_mark: | :white_check_mark: | :clock9: | |
15 | BReLU | :clock9: | :white_check_mark: | :clock9: | |
16 | APL | :clock9: | :white_check_mark: | :clock9: | |
17 | Soft Exponential | :white_check_mark: | :white_check_mark: | :clock9: | |
18 | Maxout | :clock9: | :white_check_mark: | :clock9: | |
19 | Mish | - | :white_check_mark: | :white_check_mark: | :clock9: |
20 | Beta Mish | :white_check_mark: | :white_check_mark: | :clock9: | |
21 | RReLU | :clock9: | :white_large_square: | :clock9: | |
22 | CELU | :white_check_mark: | :white_large_square: | :clock9: | |
23 | ReLU6 | :white_check_mark: | :white_large_square: | :clock9: | |
24 | HardTanh | :white_check_mark: | :white_large_square: | :clock9: | |
25 | GLU | :clock9: | :white_large_square: | :clock9: | |
26 | LogSigmoid | :white_check_mark: | :white_large_square: | :clock9: | |
27 | TanhShrink | :white_check_mark: | :white_large_square: | :clock9: | |
28 | HardShrink | :white_check_mark: | :white_large_square: | :clock9: | |
29 | SoftShrink | :white_check_mark: | :white_large_square: | :clock9: | |
30 | SoftMin | :white_check_mark: | :white_large_square: | :clock9: | |
31 | LogSoftmax | :white_check_mark: | :white_large_square: | :clock9: | |
32 | Gumbel-Softmax | :clock9: | :white_large_square: | :clock9: |
Repository Structure
The repository has the following structure:
- echoAI # main package directory
| - Activation # sub-package containing activation functions implementation
| |- Torch # sub-package containing implementation for PyTorch
| | | - functional.py # script which contains implementation of activation functions
| | | - weightedTanh.py # activation functions wrapper class for PyTorch
| | | - ... # PyTorch activation functions wrappers
| |- Keras # sub-package containing implementation for Keras
| | | - custom_activations.py # script which contains implementation of activation functions
| - __init__.py
- Observations # Folder containing other assets
- docs # Sphinx documentation folder
- LICENSE # license file
- README.md
- setup.py # package setup file
- Smoke_tests # folder, which contains scripts with demonstration of activation functions usage
- Unit_tests # folder, which contains unit test scripts
Setup Instructions
To install Echo package follow the instructions below:
-
Clone or download GitHub repository.
-
Navigate to echoAI folder:
$ cd Echo
- Install the package with pip:
$ pip install .
To run the demo scripts:
- From root folder navigate to Smoke_tests folder:
$ cd Smoke_tests
- Run demo script:
$ python keras_activations_demo.py --activation aria2
To run unit tests:
- From root folder navigate to Unit_tests folder:
$ cd Unit_tests
- Run unit test script:
$ python -m unittest torch_activation_tests
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.