Digital Reasoning Trains World’s Largest Neural Network
NASHVILLE, Tenn., July 7 -- Digital Reasoning, a leader in cognitive computing, today announced that it has trained the largest neural network in the world to date with a stunning 160 billion parameters. Google’s previous record was 11.2 billion, and Lawrence Livermore National Laboratory recently trained a neural network with 15 billion parameters.
The results of Digital Reasoning’s research with deep learning and neural networks is published in the Journal of Machine Learning and Arxiv alongside other notable companies like Google, Facebook, and Microsoft and will be presented at the prestigious 32nd International Conference on Machine Learning in Lille, France, July 6-11.
Neural Networks are computer systems that are modeled after the human brain. Like the human brain, these networks can gather new data, process it, and react to it. Digital Reasoning’s paper, titled “Modeling Order in Neural Word Embeddings at Scale,” details both the impressive scope of their neural network as well as the exponential improvement in quality.
In their research, Matthew Russell, Digital Reasoning’s Chief Technology Officer, and his team evaluated neural word embeddings on “word analogy” accuracy. Neural networks generate a vector of numbers for each word in a vocabulary. This allowed the Digital Reasoning team to do “word math.” For instance “king” minus “man” plus “woman” would yield a result of “queen.” There is an industry standard dataset of around 20,000 word analogies. Google's previous accuracy on this metric was a 76.2% accuracy rate. In other words, Google was able to get 76.2% of the word analogies "correct" in their system. Stanford's best score is a 75.0% accuracy. Digital Reasoning’s model achieves a score of 85.8% accuracy, which is a near 40% reduction in error over both Google and Stanford. This is a massive advancement in the state of the art.
“We are extremely proud of the results we have achieved, and the contribution we are making daily to the field of deep learning. This is a tremendous accomplishment for the company and marks an important milestone in putting a defensible stake in the ground towards our position as not just a thought leader in the space, but as an organization that is truly advancing the state of the art in a rigorous peer reviewed way,” said Russell.
About Digital Reasoning
Digital Reasoning delivers trusted cognitive computing for a better world. For more information go to http://www.
---
Source: Digital Reasoning
