Using latent semantic analysis to grade brief summaries: A study exploring texts at different academic levels

Olmos, Ricardo; León, José A.; Jorge-Botana, Guillermo; Escudero, Inmaculada
September 2013
Literary & Linguistic Computing;Sep2013, Vol. 28 Issue 3, p388
Academic Journal
In this study, we propose an integrated method to automatically evaluate very brief summaries (around 50 words) using the computational tool latent semantic analysis (LSA). The method proposed is based on a regression equation calculated with a corpus of a 100 summaries (the training sample) and is validated on a different sample of summaries (validation sample). The equation incorporates two parameters extracted from LSA: (1) the semantic similarity of the summary, measured using the summary–expert summaries method and (2) the vector length. The study is based on a sample of 786 summaries by students at four academic levels. All of these students summarized either an expository or a narrative text; their summaries were then evaluated by four graders on a scale of 0–10. The results support three ideas. First, that incorporating both parameters into the method is more successful than the traditional cosine measure. The reliability of LSA for evaluating summaries rises >0.80 level for the expository text. Second, that LSA shows practically the same level of sensitivity as the human graders to the quality of the summaries at different academic levels. Third, that the method overcomes a serious limitation of LSA: its difficulties evaluating very brief texts.


Related Articles

  • What gets tested gets taught. Hinckley, June // Teaching Music;Oct98, Vol. 6 Issue 2, p6 

    Opinion. Focuses on the performance assessment of students, highlighting the assessment method, multiple-choice tests. Details on this teaching method; Indication that teachers depend on evaluation of student performance and production to determine competence; Criticism of teachers as being...

  • Effect of tuned parameters on an LSA multiple choice questions answering model. LIFCHITZ, ALAIN; JHEAN-LAROSE, SANDRA; DENHiÈRE, GUY // Behavior Research Methods;Nov2009, Vol. 41 Issue 4, p1201 

    This article presents the current state of a work in progress, whose objective is to better understand the effects of factors that significantly influence the performance of latent semantic analysis (LSA). A difficult task, which consisted of answering (French) biology multiple choice questions,...

  • Teaching expository text structures through information trade book retellings. Moss, Barbara // Reading Teacher;May2004, Vol. 57 Issue 8, p710 

    Discusses how teachers can use information trade book retellings to improve student comprehension of expository text structures. Factors that contribute to the increase of teachers interested in developing student understanding of expository text; Common expository text structures that are...

  • Inviting Students to Talk About Expository Texts: A Comparison of Two Discourse Environments and Their Effects on Comprehension. Kucan, Linda; Beck, Isabel L. // Reading Research & Instruction;Spring2003, Vol. 42 Issue 3, p1 

    The purpose of this study was to investigate three questions related to talk and text comprehension. First, does the context in which students talk about text during reading affect their comprehension? Second, does talking about texts with peers influence the quality of students' talk? Third, do...

  • READING COMPREHENSION.  // Scholastic News -- Edition 3;11/14/2005, Vol. 62 Issue 8, p4 

    Presents a multiple-choice test on reading comprehension.

  • Who Killed King Tut?  // Know Your World Extra;2003-2003 Skills Extenders, p14 

    Presents a multiple-choice quiz for children about reading comprehension and King Tut of ancient Egypt.

  • Show What You Know.  // Scholastic Action;11/13/2006, Vol. 30 Issue 5/6, p10 

    A multiple-choice quiz base on the play "Sitting Bull: A Native American Legend," which is aimed at testing the reading comprehension of students is presented.

  • Show What You Know.  // Scholastic Action;2/18/2008, Vol. 31 Issue 10, p15 

    A multiple choice quiz on reading comprehension is presented.

  • EDITION 4 READING SKILLS PRACTICE TEST.  // Scholastic News -- Edition 4;10/23/2006, Vol. 69 Issue 6, p4A 

    A multiple-choice examination to develop the reading skills of school children is presented.


Read the Article


Sorry, but this item is not currently available from your library.

Try another library?
Sign out of this library

Other Topics