Samantha, a pharmacy student, was assisting in a research project on medication adherence among elderly patients. To summarise findings quickly, she used GenAI to analyse interview transcripts and generate key insights. The GenAI provided a structured summary, highlighting trends in patient concerns, medication side effects, and adherence behaviours. However, when Samantha reviewed the output, she noticed that certain patient concerns were underrepresented, while other points seemed overemphasised, even though they weren’t common in the original data. Later, Samantha also realised that she had entered identifiable patient responses into the AI tool, unaware that the system stores and processes user inputs to refine its model. What important considerations did Samantha overlook in this experience?

Question

BlackTom AI · Accepted Answer

AI summarisation can introduce sampling and emphasis biases, and many GenAI services may log or retain user inputs — therefore options b and e are the correct considerations Samantha missed.

1) Relevant knowledge points and concepts
- Model bias and summarisation limitations: generative models detect and amplify patterns but can overemphasise unusual phrases or inferred themes, producing summaries that misrepresent frequency or importance.
- Hallucination and omission: models sometimes invent or omit details not supported by the original data.
- Data privacy, confidentiality and input retention: many commercial GenAI tools process and may store user inputs for model improvement; submitting identifiable patient data can violate confidentiality and ethical rules.
- Research ethics and human oversight: automated outputs require verification and de-identification before use in research.

2) Step-by-step reasoning (intermediate logical steps)
Step 1: Samantha fed interview transcripts into a GenAI and accepted the structured summary.
Step 2: She observed underrepresentation of some concerns and overemphasis of uncommon points — this indicates the tool biased the summary or mis-weighted signals (matches concept of summarisation bias/hallucination).
Step 3: She later realised she had entered identifiable patient responses and learned the tool stores/processes inputs — this raises privacy/confidentiality and ethical data-use issues.
Step 4: Match these observations to options: biased summarisation → option b; input storage/privacy risk → option e.

3) Why the correct answers are correct
- b is correct because generative summarisation can prioritise salient linguistic patterns, not true prevalence; human review and quantitative validation (e.g., coding counts) are required to ensure representativeness.
- e is correct because many GenAI providers retain inputs or use them to train models unless explicitly disabled; submitting identifiable health data can breach confidentiality, consent, and regulatory requirements (e.g., data protection laws, research ethics).

4) Why the other options are incorrect
- a is incorrect: AI summaries are not always reliable and do require human verification.
- c is incorrect: GenAI does not automatically anonymise all inputs; anonymisation must be performed by the user or controlled by policy.
- d is a legitimate general concern but was not illustrated by Samantha’s experience (she noted misrepresentation and privacy risks, not attribution errors), so it is not one of the overlooked items in this scenario.

Conclusion: The correct overlooked considerations are b and e, matching the provided answer.

类似问题

In the context of privacy and intellectual property, which of the following actions is considered appropriate?

In the context of privacy and intellectual property, which of the following actions is considered appropriate?在隐私和知识产权的背景下，以下哪一项行为被认为是合适的？

From the seminar on "Delivering AI and Digital Transformation for Social Good" , please list and comment on 2 key takeaways请从“利用人工智能和数字化转型造福社会”研讨会中列举并评论两个主要收获。

From the seminar on "Delivering AI and Digital Transformation for Social Good" , please list and comment on 2 key takeaways

更多留学生实用工具

智能学习助手

风格化写作助手

论文查重助手

文献引用助手

课堂转译助手

课堂笔记助手

Quiz搜索助手

学校历年真题

智能刷题助手

智能匹配练习

希望你的学习变得更简单