How Does Smart Software Interpret a School Test
January 29, 2025
A blog post from an authentic dinobaby. He’s old; he’s in the sticks; and he is deeply skeptical.
I spotted an article titled “‘Is This Question Easy or Difficult to You?’: This LSAT Reading Comprehension Question Is Breaking Brains.” Click bait? Absolutely.
Here’s the text to figure out:
Physical education should teach people to pursue healthy, active lifestyles as they grow older. But the focus on competitive sports in most schools causes most of the less competitive students to turn away from sports. Having learned to think of themselves as unathletic, they do not exercise enough to stay healthy.
Imagine you are sitting in a hot, crowded examination room. No one wants to be there. You have to choose one of the following solutions.
(a) Physical education should include noncompetitive activities.
[b] Competition causes most students to turn away from sports.
[c] People who are talented at competitive physical endeavors exercise regularly.
[d] The mental aspects of exercise are as important as the physical ones.
[e] Children should be taught the dangers of a sedentary lifestyle.
Okay, what did you select?
Well, the “correct” answer is [a], Physical education should include noncompetitive activities.
Now how did some of the LLMs or smart software do?
ChatGPT o1 settled on [a].
Claude Sonnet 3.5 spit out a page of text but did conclude that the correct answer as [a].
Gemini 1.5 Pro concluded that [a] was correct.
Llama 3.2 90B output two sentences and the correct answer [a]
Will students use large language models for school work, tests, and real life?
Yep. Will students question or doubt the outputs? Nope.
Are the LLMs “good enough”?
Yep.
Stephen E Arnold, January 29, 2025
Comments
Got something to say?
-
- Subscribe to Beyond Search
Feature archive
News archive
-
