The Crutch or the Ceiling? How Different Generations of LLMs Shape EFL Student Writings

Dimensions overall score 3.0

• Abstract. The rapid evolution of Large Language Models (LLMs) has made them powerful tools for enhancing student writing. This study explores the extent and limitations of LLMs in assisting secondary-level English as a Foreign Language (EFL) students with their writing tasksclaim 82%
  Evidencepartial

  Abstract. The rapid evolution of Large Language Models (LLMs) has made them powerful tools for enhancing student writing

  Implicationmissing

  Implication not extracted yet.

  Verificationpartial

  partial

• EarlyGen-LLM Adv-LLM Fig. 3. Readability Test : Automated Readability Index (ARI), Coleman-Liau Index, Flesch-Kincaid Grade Level, Dale-Chall Readability score, Gunning-fog index, Linsear Write, text standard (The estimated school grade level based on Dale-Chall Readability Score) , SMOG As shown in Figure 3claim 82%
  Evidencepartial

  EarlyGen-LLM Adv-LLM Fig. 3. Readability Test : Automated Readability Index (ARI), Coleman-Liau Index, Flesch-Kincaid Grade Level, Dale-Chall Readability score, Gunning-fog index, Linsear Write

  Implicationmissing

  Implication not extracted yet.

  Verificationpartial

  partial

• This can be interpreted as greater reliance on advanced LLMs does not provide a clear advantage, regardless of the student’s writing competencyclaim 82%
  Evidencepartial

  This can be interpreted as greater reliance on advanced LLMs does not provide a clear advantage, regardless of the student’s writing competency

  Implicationmissing

  Implication not extracted yet.

  Verificationpartial

  partial

• (Group 3), as shown in Figure 5a. A notable observation is that the performance gap in lexical diversity between LLM tiers narrows as total CLO scores decreaseclaim 82%
  Evidencepartial

  (Group 3), as shown in Figure 5a. A notable observation is that the performance gap in lexical diversity between LLM tiers narrows as total CLO scores decrease

  Implicationmissing

  Implication not extracted yet.

  Verificationpartial

  partial

• proportion of LLM-generated text is negatively correlated with total CLO scores, as well as with each individual 14•Hengky Susanto, et al. -0.15 -0.1 -0.05 0 0.05 0.1 0.15 Avg. Content Avg. Language Avg. Organization CLO score % AI words % human words Fig. 7claim 82%
  Evidencepartial

  proportion of LLM-generated text is negatively correlated with total CLO scores, as well as with each individual 14•Hengky Susanto, et al. -0.15 -0.1 -0.05 0 0.05 0.1 0.15 Avg. Content Avg. Language Avg

  Implicationmissing

  Implication not extracted yet.

  Verificationpartial

  partial

• [34]. In contrast, student compositions are holistically judged on criteria such as idea development, higher-order logic, coherence, argumentation, emotional impact, and nuanced meaning [ 3, 33]. Therefore, the evaluation criteria used for LLMs differ from those used to evaluate compositions written by EFL studentsclaim 82%
  Evidencepartial

  [34]. In contrast, student compositions are holistically judged on criteria such as idea development, higher-order logic, coherence, argumentation, emotional impact, and nuanced meaning [ 3, 33]. Therefore

  Implicationmissing

  Implication not extracted yet.

  Verificationpartial

  partial

Competitive landscape

This study analyzes how different generations of LLMs impact EFL student writing, suggesting pedagogical shifts to ensure genuine learning rather than superficial fluency.