If you train a seq-to-seq model for translating English to French, how can the probability distribution for the second predicted French word be written if the input English words are and the output French words are , and is the second output of the network?

Hint: Refer to Lecture 17 Slides 57-65 (general concepts, not really the specific content)

Question

If you train a seq-to-seq model for translating English to French, how can the probability distribution for the second predicted French word be written if the input English words are  and the output French words are , and  is the second output of the network?

Hint: Refer to Lecture 17 Slides 57-65 (general concepts, not really the specific content)

BlackTom AI · Accepted Answer

The question asks how to write the probability distribution for the second predicted French word in a seq-to-seq translation setup, given the input English sequence is (e1, e2, e3) and the output so far is (f1) on the way to predicting f2.

First, consider what the model conditions on when predicting the next target token in a standard encoder–decoder (seq-to-seq) with teacher forcing during training. The second French word f2 is predicted based on the entire source sentence embeddings produced by the encoder from (e1, e2, e3) and the already generated target prefix, which in this case is the single token f1. Thus the probability distribution over possible f2 tokens is conditioned on the input sequence and the previous target token(s).

One can express this formally as the conditional probability of f2 given the source words e1, e2, e3 and the previously produced target token f1. In mathematical notation, this is written as p(f2 | e1, e2, e3, f1).

This encapsulates the standard factorization used in seq-to-seq models, where the decoder uses the encoder’s context (often via attention) and the previously generated target tokens to produce the next token distribution over the target vocabulary.

Now, regarding the provided answer: the expression y(f2) = p(f2 | e1, e2, e3, f1) indeed matches the conventional formulation for the probability distribution of the second output word conditioned on the full source and the first output word. It captures the dependency on all three input words, as well as the first predicted target word, which is exactly what influences the second token prediction in this setup.

If you consider alternative phrasings, you might see the same idea written as p(f2 | f1, e1, e2, e3) or p(f2 | f1, encoder_cartography, decoder_hidden_state_at_step1), but the essential meaning remains the same: the second target word depends on the entire input sequence and the previously generated target symbol(s).

In contrast, any statement that ignores e3 or treats f1 as the only conditioning factor would misrepresent the model’s true conditioning and would be incorrect in the typical seq-to-seq formulation.

To summarize concisely: the correct conditional distribution for the second French word in this setup is p(f2 | e1, e2, e3, f1), reflecting dependence on all source tokens and the previously generated target token.

类似问题

What is an example of a many-to-many sequence modeling task?

Which of the following is true when performing machine translation using a sequence to sequence model Hint Refer to Lecture 17

Is the following statement true or false? In the products of methylation-hydrolysis, every -OH group corresponds to the position of a glycosidic bond in the starting polysaccharide.

Which of the follwoing structures represents amylopectin?

Which of the following is true about an aldopentose?I. It is a monosaccharide.II. It contains a CHO groupIII. It is a disaccharide.IV. It is an oligosaccharide.

For stock A, we have 𝛽 𝑖 = 0.70. Suppose the expected market risk premium next year is 9% and the risk-free rate is 3%. What is the expected return of this stock based on the CAPM? (Please answer in % and round to 2 decimal places. If the answer is 8.057%, then in the box, write 8.06)

更多留学生实用工具

智能学习助手

风格化写作助手

论文查重助手

文献引用助手

课堂转译助手

课堂笔记助手

Quiz搜索助手

学校历年真题

智能刷题助手

智能匹配练习

希望你的学习变得更简单