Deep Learning

Your constantly-updated definition of Deep Learning and collection of videos and articles

What is Deep Learning?

Show Hide video transcript
  1. Transcript loading...

Deep learning is a type of technology that allows computers to simulate how our brains work. It is a subset of machine learning, a branch of artificial intelligence.

Deep learning uses artificial neural networks that consist of multiple layers of nodes, also known as neurons, that can learn from large amounts of data. Deep learning can process unstructured data, such as images and text, and automatically extract important features and patterns from the data. Deep learning can also perform different types of learning, such as supervised, unsupervised, and reinforcement learning. Deep learning is the technology behind many popular AI applications, such as chatbots, virtual assistants, self-driving cars, and more.

Neural Networks in Deep Learning

Neural networks for deep learning have several hidden layers between input and output at the extremities. This is the origin of the term "deep".

© Interaction Design Foundation, CC BY-SA 4.0

Neural networks are the building blocks of deep learning algorithms. Artificial neural networks are inspired by the structure and function of the human brain and consist of interconnected layers of nodes that process and transmit information. Neural networks can have different architectures and parameters, depending on the problem they are trying to solve. Deep learning algorithms use multiple layers of neural networks to extract features from the data and make predictions based on them.

Differences Between Deep Learning and Machine Learning

While deep learning is a subset of machine learning, there are some important differences. Deep learning…

  • Requires large amounts of data to train, while machine learning can work with smaller data sets.

  • Learns independently from the environment and past mistakes, while machine learning requires more human intervention to correct and improve the algorithms.

  • Can process unstructured data, such as images and text, and automatically extract important features and patterns, while machine learning often needs pre-processing and feature engineering to handle such data.

  • Makes complex, non-linear correlations between inputs and outputs, while machine learning makes simple, linear correlations.

  • Needs a specialized GPU (graphics processing unit) to train, while machine learning can train on a CPU (central processing unit).

History of Deep Learning

Deep learning has a long history from the 1940s when McCulloch and Pitts proposed the first artificial neuron. However, deep learning did not gain much attention until the 1980s and 1990s, when some breakthroughs were made in backpropagation, convolutional neural networks, and recurrent neural networks. In the early 2000s, advances in speech recognition, image recognition and natural language processing were made possible by the advent of deep learning — a branch of machine learning that uses deep neural networks. The success of deep learning was also driven by the availability of large-scale data sets, such as ImageNet and Wikipedia, and the increase in computational power, such as GPUs and cloud computing.

Diagram of a human neuron. Signals are received through the dendrites (left) and sent out through the axon (right).

CC BY-SA 4.0 ShareAlike (https://creativecommons.org/licenses/by-sa/4.0/)

A graphic representation of the McCulloch-Pitts neuron. Signals are received by g (left). f (right) makes a decision whether to provide an output. All signals are binary (either 0 or 1).

© Interaction Design Foundation, CC BY-SA 4.0

Deep learning is now one of the most active and influential areas of AI research, with applications in various domains, such as healthcare, education, entertainment, finance, and more. Deep learning is constantly evolving and improving, with new architectures, algorithms, and techniques being developed and tested. Some current trends and challenges in deep learning include self-supervised learning, generative adversarial networks, transformers, explainable AI, adversarial attacks, and ethical issues.

Applications and Use Cases of Deep Learning

Chatbots

Deep learning is used in chatbots to create natural and human-like conversations with users. Deep learning chatbots can mimic real-life conversations through neural networks. They can learn from existing interactions between customers and support staff or from other data sources, such as movie dialogue or play scripts. They can also understand the intent and context of the user’s messages and generate appropriate and relevant responses. Deep learning chatbots can handle complex questions and scenarios and adapt to different situations and user preferences. Deep learning chatbots can also improve over time by learning from their mistakes and feedback.

A chatbot explaining how they learn.

Microsoft Copilot, Fair Use

Virtual Assistants

Amazon Echo Dot with Alexa virtual assistant.

Virtual assistants like Alexa, Google Assistant and Siri increasingly use deep learning to perform complex tasks. These include understanding natural language, recognizing speech and images, generating responses, and learning from user feedback. Some examples of how deep learning is used in voice-driven virtual assistants are:

  • Speech-to-text and text-to-speech: Deep learning algorithms enable virtual assistants to convert speech to text and vice versa, allowing users to interact with them using voice commands. For example, Siri uses deep learning to transcribe user queries and provide voice feedback.

  • Computer vision: Deep learning algorithms enable virtual assistants to analyze images and videos, and provide relevant information or actions based on them. For example, Google Lens uses deep learning to identify objects, landmarks, text, and faces in images, and provide useful information or suggestions.

  • Natural language processing: Deep learning algorithms enable virtual assistants to process natural language, such as understanding the meaning, context, and intent of user queries, and generating appropriate responses. For example, Alexa uses deep learning to parse user requests and provide relevant answers or actions.

  • Personalization: Deep learning algorithms enable virtual assistants to learn from user preferences, behavior, and feedback, and provide customized user experience. For example, Cortana uses deep learning to adapt to user needs and provide personalized recommendations and reminders.

Self-Driving Cars

Self-driving cars use deep learning.

In many respects, self-driving cars act as virtual assistants (see above), but require additional capabilities to deal with the complexities of driving itself:

  • Computer vision: Deep learning algorithms enable self-driving cars to analyze images and videos, and provide relevant information or actions based on them. For example, Waymo uses deep learning to identify objects, landmarks, text, and faces in images, and provide useful information or suggestions.

  • End-to-end learning: Deep learning algorithms enable self-driving cars to learn directly from raw sensor data, such as camera images, without relying on hand-crafted features or intermediate representations. For example, Nvidia uses deep learning to map the raw pixels from a front-facing camera to the steering commands for a self-driving car.

The Future of Deep Learning

The future of deep learning is a topic that has attracted a lot of interest and speculation from researchers, developers, and the general public. However, it has some limitations and challenges that must be overcome to achieve its full potential. Some of these are:

  • Data dependency: Deep learning requires large amounts of data to train, which can be costly, time-consuming, and prone to bias. It also struggles to generalize to new situations or domains that differ from the training data.

  • Interpretability: Deep learning models are often seen as black boxes that are difficult to understand and explain. This can raise ethical and social issues, such as trust, accountability, and transparency.

  • Scalability: Deep learning models are often computationally intensive and require specialized hardware, such as GPUs and cloud computing. This can limit their accessibility and applicability for some users and scenarios.

  • Robustness: Deep learning models are vulnerable to adversarial attacks, malicious inputs that can fool or manipulate the models. They can also suffer from catastrophic forgetting, the loss of previously learned knowledge when learning new tasks.

To address these challenges and advance the field of deep learning, many researchers and experts have proposed some possible directions and solutions for the future of deep learning. Some of these are:

  • Transfer learning: Transfer learning is a technique that allows a model to leverage the knowledge learned from one task or domain to another task or domain. This can reduce the need for large amounts of data and improve the generalization and adaptation of the model.

  • Self-supervised learning: Self-supervised learning is a type of unsupervised learning that uses the data itself to generate labels or objectives for the model. This can enable the model to learn from unlabeled data and discover useful representations and structures from the data.

  • Generative adversarial networks (GANs): GANs are a type of neural network that consists of two competing models: a generator that tries to create realistic outputs from random inputs, and a discriminator that tries to distinguish between real and fake outputs. This can enable the model to generate novel and diverse outputs, such as images, text, or music.

  • Transformers: Transformers are a neural network that uses attention mechanisms to capture the dependencies and relationships between different parts of the input or output. This can enable the model to process sequential data, such as text or speech, more efficiently and effectively.

  • Explainable AI (XAI): XAI is a field of AI that aims to make AI models more interpretable and transparent by explaining their decisions or behaviors. This can increase the trust and confidence of users and stakeholders in AI applications.

These are some of the possible future directions and opportunities for deep learning. However, there is still much more to explore and discover in this exciting and evolving field. As one of the pioneers of deep learning, Geoffrey Hinton said: "We’re still very far from understanding how brains work or how intelligence works or what consciousness is." Therefore, the future of deep learning is likely to be full of surprises and innovations that will shape our society and culture in profound ways.

Learn More about Deep Learning

Questions about Deep Learning

What is deep learning AI?

Deep learning AI is a type of artificial intelligence that uses artificial neural networks to learn from data and perform tasks that require human-like intelligence, such as image recognition, natural language processing, and speech recognition.

What is deep learning?

Deep learning is a subset of machine learning that uses neural networks with multiple layers to analyze complex patterns and relationships in data. It is inspired by the structure and function of the human brain and has been successful in a variety of tasks, such as computer vision, natural language processing, and speech recognition.

How does deep learning work?

In deep learning, a computer model learns to perform classification tasks directly from images, text, or sound. Deep learning models can achieve state-of-the-art accuracy, sometimes exceeding human-level performance. Models are trained by using a large set of labeled data and neural network architectures that contain many layers.

How is deep learning different from machine learning?

Deep learning differs from machine learning in the type of data that it works with and the methods that it uses to learn. Machine learning algorithms leverage structured, labeled data to make predictions, while deep learning algorithms can ingest and process unstructured data, such as text and images, and automate feature extraction. Machine learning algorithms typically require more human intervention to correct and learn, while deep learning algorithms can learn independently from the environment and past mistakes.

Does ChatGPT use deep learning?

Yes, ChatGPT uses deep learning to generate human-like text. It uses the transformer architecture, a type of neural network that has been successful in various natural language processing tasks and is trained on a massive corpus of text data from the internet.

What is deep learning used for?

Deep learning is used for many applications that require human-like intelligence, such as image recognition, natural language processing, speech recognition, sentiment analysis, recommendation systems, self-driving cars, fraud detection, medical diagnosis, and many more.

What does batch size mean in deep learning?

Batch size is a hyperparameter that defines the number of samples used in one iteration of training a deep learning model. The choice of batch size can have a significant impact on the performance of the model, such as its accuracy, speed, memory usage, and generalization ability.

What is transfer learning in deep learning?

Transfer learning is a technique in which a model trained on one task is used as a starting point for training a model on a different but related task. It applies the weights of the learned features to the new model. The pre-trained model serves as transferred knowledge to be applied in another domain.

Where is deep learning used?

Deep learning is used in various domains and industries that require intelligent solutions, such as healthcare, finance, retail, transportation, education, entertainment, security, agriculture, manufacturing, and many more.

What is the difference between deep learning and neural networks?

Deep learning can handle more complex and unstructured data, such as images, text, and speech, and automate feature extraction. Neural networks are algorithms that mimic the way the biological neurons in the human brain work. Neural networks are simpler and more limited in their capabilities, while deep learning models are more complex and can handle more complex data sets.

Literature on Deep Learning

Here’s the entire UX literature on Deep Learning by the Interaction Design Foundation, collated in one place:

Learn more about Deep Learning

Take a deep dive into Deep Learning with our course AI for Designers .

In an era where technology is rapidly reshaping the way we interact with the world, understanding the intricacies of AI is not just a skill, but a necessity for designers. The AI for Designers course delves into the heart of this game-changing field, empowering you to navigate the complexities of designing in the age of AI. Why is this knowledge vital? AI is not just a tool; it's a paradigm shift, revolutionizing the design landscape. As a designer, make sure that you not only keep pace with the ever-evolving tech landscape but also lead the way in creating user experiences that are intuitive, intelligent, and ethical.

AI for Designers is taught by Ioana Teleanu, a seasoned AI Product Designer and Design Educator who has established a community of over 250,000 UX enthusiasts through her social channel UX Goodies. She imparts her extensive expertise to this course from her experience at renowned companies like UiPath and ING Bank, and now works on pioneering AI projects at Miro.

In this course, you’ll explore how to work with AI in harmony and incorporate it into your design process to elevate your career to new heights. Welcome to a course that doesn’t just teach design; it shapes the future of design innovation.

In lesson 1, you’ll explore AI's significance, understand key terms like Machine Learning, Deep Learning, and Generative AI, discover AI's impact on design, and master the art of creating effective text prompts for design.

In lesson 2, you’ll learn how to enhance your design workflow using AI tools for UX research, including market analysis, persona interviews, and data processing. You’ll dive into problem-solving with AI, mastering problem definition and production ideation.

In lesson 3, you’ll discover how to incorporate AI tools for prototyping, wireframing, visual design, and UX writing into your design process. You’ll learn how AI can assist to evaluate your designs and automate tasks, and ensure your product is launch-ready.

In lesson 4, you’ll explore the designer's role in AI-driven solutions, how to address challenges, analyze concerns, and deliver ethical solutions for real-world design applications.

Throughout the course, you'll receive practical tips for real-life projects. In the Build Your Portfolio exercises, you’ll practise how to  integrate AI tools into your workflow and design for AI products, enabling you to create a compelling portfolio case study to attract potential employers or collaborators.

All open-source articles on Deep Learning

Please check the value and try again.

Open Access - Link to us!

We believe in Open Access and the democratization of knowledge. Unfortunately, world class educational materials such as this page are normally hidden behind paywalls or in expensive textbooks.

If you want this to change, , link to us, or join us to help us democratize design knowledge!