What is a neural network, and how does it work?

Neural network definition

A neural network (NN), also known as an artificial neural network (ANN), is a type of machine learning that processes data in a similar way as a human brain does. Artificial neural networks, or neural nets, are adaptive systems that consist of layers of interconnected artificial neurons. These neurons, also called nodes, process input signals and produce output signals that they pass on to other nodes in the network. ANNs are trained to recognize patterns and relationships in data, making them a powerful tool for data classification, speech and image recognition, and predictive analytics.

How do neural networks work?

An artificial neural network is a layered structure of interconnected nodes that transmit information. Every neural network has an input layer, an output layer, and a hidden layer or layers in between. Each of those is composed of input, output, and hidden nodes, respectively. A node receives inputs, performs a simple computation, and produces an output that travels to other nodes in the next layer.

First, the raw data, also called the input data, is fed into the network in the form of texts, images, or digits. Input nodes receive the input data, encode it, and pass it on to the hidden nodes. These, in turn, perform a series of mathematical calculations on the data and pass it on to the output nodes. The output nodes produce the final output value or a set of values representing the network’s prediction or decision based on the raw input.

How to train neural networks

Data scientists train neural networks by feeding them with large labeled datasets that provide the correct answer in advance. After going through these examples, neural networks can process unknown input more accurately. For example, when a deep learning network is trained in facial recognition, it processes hundreds of thousands of images of human faces with descriptions of race, ethnicity, emotion, and other features.

Artificial neural networks with multiple hidden layers between the input and output layers are called deep neural networks. They lie at the core of deep learning and artificial intelligence. These networks use a form of supervised learning called backpropagation. Data scientists adjust the weights and biases of the neurons in a network based on the difference between its output and the desired output. During training, the neural network learns to generalize from the training data and make accurate predictions on new data. Deep neural networks are used in deep learning, a subfield of machine learning.

What are neural networks used for?

Neural networks are useful in a variety of fields and industries, from medicine to finance. Here is the list of the main applications of neural networks:

Computer vision

Computer vision is the ability of computers to interpret and understand visual data. They process and analyze images and videos to extract information from them. With the help of neural networks, computers can distinguish and recognize images similarly to how humans do. You can apply computer vision in the following areas:

• Content moderation. Computers recognize and remove inappropriate visual content.
• Facial recognition. It is used in the fields of policing and security.
• Autonomous vehicles. With computer vision, autonomous vehicles recognize traffic signs, lanes, and other road users.
• Robotics. Robots learn to recognize objects and the relationship between them, understand their surroundings, create maps, and guide their motions based on visual feedback.

Speech recognition

Neural networks analyze human speech and identify patterns that correspond to words. They adapt to different speech patterns, pitch, tone, or accent. This technology is used for:

• Classifying calls in call centers.
• Transcription — converting conversations into documents.
• Subtitling videos and recordings.
• Processing spoken commands given to virtual assistants and voice-activated systems in cars and homes.

Natural language processing (NLP)

Neural networks enable computers to understand, interpret, and generate human language. NLP is used for:

• Sentiment analysis.
• Language translation.
• Text summarization.
• Chatbots.
• Spoken language recognition.

Recommendation engines

Neural networks can be used to track user activity for custom recommendations. They can also analyze user behavior and then find and offer specific products and services that could interest them.

History of neural networks

The history of artificial neural networks goes back to the 1940s.That’s when scientists first came up with the idea of a thinking machine inspired by the structure and function of the human brain. They started researching how to break down the brain’s complex neural networks into a binary logic structure with true/false connections.

In the 1950s and 1960s, researchers developed mathematical models of neural networks and demonstrated how they could detect images and make inferences. The 1980s brought further exploration of backpropagation and new algorithms for training neural networks.

In the 1990s, neural networks were already used for speech and image recognition, language processing, and outcome prediction. However, their performance was limited by the lack of sufficient computational resources.

In the 2000s and 2010s, large datasets and computing resources led to significant advances in neural network technology. Developing deep learning algorithms for training deep neural networks resulted in technologies such as self-driving vehicles and virtual assistants.

Types of neural networks

There are many different types of neural networks, each designed for specific tasks and applications. Here are six examples of the most common types of neural networks:

Feedforward neural networks

Feedforward neural networks are the most basic type of neural networks. They consist of an input layer, one or more hidden layers, and an output layer. In feedforward networks, information flows in one direction only — from the input layer through the hidden layers to the output layer. But they can be trained to move in the opposite direction — from output to input. A feedforward neural network is often used for classification and regression tasks.

Recurrent neural networks

Unlike feedforward networks, recurrent neural networks (RNNs) have loops in their structure. That means they can store information from previous inputs. They are typically used for natural language processing and speech recognition tasks.

Convolutional neural networks

Convolutional neural networks (CNNs) are a type of feedforward neural networks designed to identify features in images and classify them by processing small areas of the image at a time. A convolutional neural network is commonly used in image and video recognition tasks.

Deconvolutional neural networks

Deconvolutional neural networks are convolutional neural networks that work in reverse. They strive to find features that were previously lost because they were not deemed important during CNN’s task. These networks are mostly used for image synthesis and analysis.

Modular neural networks

A modular neural network is a series of artificial neural networks moderated by an intermediary. Each independent neural network acts as a module and works on separate inputs. This way, each accomplishes a sub-task of a bigger task the whole network is trying to achieve.

Autoencoder neural networks

Autoencoder neural networks are used for data compression and dimensionality reduction tasks. First they encode input data into a lower-dimensional representation and then decode it back into the original format.

Advantages and disadvantages of neural networks

As efficient as neural networks are at accomplishing complex tasks, there is still room for improvement. Here are the main pros and cons of these machine learning models.

Advantages of neural networks

• Neural networks are more efficient than humans.
• They learn from prior outputs and can be trained to make predictions and foresee future outcomes for new inputs.
• They can process complex and non-linear patterns and relationships in data with many variables.
• If deployed in the cloud, they can mitigate risk compared to systems that rely only heavily on hardware.
• They can function even when some of the nodes are damaged or fail.

Disadvantages of neural networks

• They require a lot of time and resources to train and optimize, for example, to create an algorithm for a specific task.
• If presented with incomplete or otherwise faulty data, they can produce inaccurate or biased results.
• The hardware component is still required, which is complicated to set up and needs physical maintenance.
• They are difficult to audit and understand, which makes it challenging to diagnose and correct errors and biases. It can be difficult to analyze its weaknesses if the network lacks transparency on how it learns.
• Like any other software, they are hackable. Data poisoning, backdoor attacks, and private data leaks are common security vulnerabilities.

Neural networks vs. deep learning

The terms “neural networks” and “deep learning” are sometimes used interchangeably even though they are not the same thing. The word “deep” refers to the number of hidden layers in a neural network, or its depth. Neural networks with more than three layers are called deep neural networks, or deep learning networks. They are the ones used in deep learning.

Deep learning lies at the core of artificial intelligence, and while it’s common in a wide range of industries, its application raises ethical concerns. With the lack of regulation and international laws, AI can still be used with little accountability and possibly malicious purposes. For example, cities around the world employ machine learning for mass surveillance and facial recognition, which relies on biometric data. Using AI in manufacturing and personalization also gives corporations an upper hand compared to smaller companies that cannot afford this technology.

But the most worrisome is the issue of privacy. Internet service providers, governments, and businesses track their users and collect their data but don’t always declare how they store or use it. This makes our online security and privacy even more critical.