Encoder

Gold definitionUpdated Apr 2, 2026

Definition

An encoder is a neural network component that transforms an input sequence into a dense, context-rich numerical representation. In large language models, encoder-only architectures excel at understanding tasks, providing contextual embeddings for various downstream applications.

At a glance

Executive summary

An encoder is a part of an AI model that takes raw information, like text, and turns it into a rich numerical code that the computer can understand. This process helps the AI grasp the meaning and context of the input, making it useful for tasks like understanding sentiment or classifying text.

TL;DR

An encoder converts raw data into a meaningful numerical representation, helping AI models understand and process information.

Key points

Transforms input sequences into contextualized numerical representations using self-attention
Solves the problem of converting raw data into a machine-understandable, semantically rich format
Used by NLP researchers and ML engineers for tasks like text classification and information retrieval
Unlike decoders, encoders focus on understanding input rather than generating output
Research trend towards more powerful pre-trained encoders (e.g., BERT, RoBERTa) for transfer learning

Use cases

Sentiment analysis: Classifying customer reviews as positive, negative, or neutral.
Named Entity Recognition (NER): Identifying and categorizing key entities (people, organizations, locations) in text.
Information Retrieval: Matching user queries to relevant documents or passages in a database.
Text Summarization (extractive): Identifying the most important sentences in a document to form a summary.
Question Answering: Understanding a question and a given context to extract the correct answer.

Also known as

Transformer encoder, BERT encoder, RoBERTa encoder, text encoder, sequence encoder