Which of the following best describes a key property of Transformer architecture used in LLMs?

Question

BlackTom AI · Accepted Answer

Transformer architecture uses self-attention to capture relationships between tokens, allowing each token to directly attend to every other token in the sequence.

Relevant knowledge points:
- Self-attention computes pairwise interactions between tokens using queries (Q), keys (K), and values (V). The core formula is Attention(Q,K,V) = softmax(QK^T / sqrt(d_k)) V.  
- Positional encodings provide token order information because self-attention itself is order-agnostic.  
- Transformers process tokens in parallel during training (not strictly sequentially), which yields efficiency and better modeling of long-range dependencies compared with recurrent models.

Step-by-step reasoning:
1. Represent each token as an embedding; produce Q, K, V vectors by linear projections of embeddings.  
2. Compute raw scores by multiplying Q and K^T for all token pairs; this yields a matrix of similarity scores where each entry indicates how much one token should attend to another.  
3. Scale by 1/sqrt(d_k) to stabilize gradients, then apply softmax across each row to get attention weights that sum to 1.  
4. Multiply the attention weights by V to produce context-aware token representations combining information from related tokens.  
5. Multi-head attention repeats this in parallel subspaces to capture diverse relations.

Why the correct answer is correct:
- The described mechanism is exactly “self-attention” and is the defining, distinguishing property of Transformers. It enables modeling of long-range and context-dependent relationships without recurrence.

Why other options are incorrect:
- “Convolution layers for sequence modeling”: Convolutions (CNNs) capture local patterns and were used in some sequence models, but they do not form the core of Transformer design.  
- “Relies heavily on recurrence”: Recurrence is the hallmark of RNNs/LSTMs, not Transformers.  
- “Processes input tokens sequentially”: Transformers allow parallel processing of tokens during training; only autoregressive decoding is sequential.

Conclusion: The correct choice is “It uses self-attention to capture relationships between tokens,” which matches the provided answer.

Which of the following best describes a key property of Transformer architecture used in LLMs?单项选择题

类似问题

The transformer architecture only consists of a single encoder component and single decoder component.

What is the main architectural innovation in ChatGPT that allows it to handle complex language tasks?

A team of data scientists is experiencing poor performance with their Transformer model. After inspecting their implementation, you find several suspicious design choices. Which one would you identify as the MOST problematic based on the architectural principles discussed in the lecture?

What operations are part of a standard Transformer block? (Select all that apply.)

Question at position 8 An excess of growth hormone (GH) before puberty can result in the development ofgigantism.Graves' disease.acromegaly.pituitary dwarfism.Clear my selection

Question at position 6 The most abundant supporting cells in the central nervous system are called Question Blank 1 of 1[input].

Question at position 5 In the eye, the lens functions tofocus light on the retina at the back of the eye.protect the surface of the eye.increase vision in low-light conditions.move the eye side to side.Clear my selection

更多留学生实用工具

智能学习助手

风格化写作助手

论文查重助手

文献引用助手

课堂转译助手

课堂笔记助手

Quiz搜索助手

学校历年真题

智能刷题助手

智能匹配练习

希望你的学习变得更简单