Neural networks are not only about real-valued neural network architectures. These network architectures require math knowledge that exceeds the stuff that is covered in a first year course ;). However, it seems like complex-valued neural networks can outperform real-valued NNs.

WIP

Literature on Quaternion Neural Networks

  • Comminiello et al. (2018): Quaternion Convolutional Neural Networks for Detection and Localization of 3D Sound Events. arXiv:1812.06811

  • Gaudet and Maida (2017): Deep Quaternion Networks. arXiv:1712.04604

  • Parcollet et al. (2018): Quaternion Recurrent Neural Networks. arXiv:1806.04418.
  • Parcollet et al. (2018): Quaternion Convolutional Neural Networks for End-to-End Automatic Speech Recognition. arXiv:1806.07789.
  • Parcollet et al. (2018): Quaternion Convolutional Neural Networks for Heterogeneous Image Processing. arXiv:1811.02656
  • Parcollet et al. (2018): Bidirectional Quaternion Long-Short Term Memory Recurrent Neural Networks for Speech Recognition. arXiv:1811.02566

  • Parcollet et al. (2018): Speech recognition with quaternion neural networks. arXiv:1811.09678

  • Trabelsi et al. (2017): Deep Complex Networks. arXiv:1705.09792.

  • Vecchi et al. (2019): Compressing deep quaternion neural networks with targeted regularization. arXiv:1907.11546

  • Zhang et al. (2019): Quaternion Knowledge Graph Embedding. arXiv:1904.10281
  • Zhu et al. (2019): Quaternion Convolutional Neural Networks. arXiv:1903.00658

There is more …

Well, if quaternions are still to simple, then have a look at octernions. It seems like they could outperform quaternions by a tiny bit. And if that is to complicated for you, then drop back to complex numbers ;).