Column – HAIIA

ことわざの意味分類

言語を扱うAIの能力判定方法として「ことわざの意味分類」が一番手軽で難易度も適切だと思っていた。「弘法も筆の誤り」「猿も木から落ちる」を同じグループにする。「犬も歩けば棒にあたる」は若干違う。”歩く”は犬の得意技じゃないから。

「意味の意味」を考えるのは難しいけど、ことわざには表の意味（字面）と奥の意味があって、その両方を皆が同意する。だから、答えに曖昧さが出ない。

今年（2025年）の最初の頃にCopilotで試した時も、上手く行かなかった。ところが8月にGPT5に聞いたら、あっさりと分類するんだもん。びびった。ついつい「物語を読んで、その内容に似ていることわざを見つける」という設問も追加で出したら、こちらも見事に回答。

ホント、ここまでAIが進歩するとは思ってもいなかったので、この時点で「GPT5はすげーー」と思ってました。だから、今までの自分自身のAI研究の内容についても、色々と話してみたんだけど、ちゃんと理解した上で適切な質問を投げてくる。

当時はGPTを無償で使っていたので、全部をしゃべることは出来なかったけど、ついつい色々と書いてしまった。そういう意味で、GPTと私のスタートラインは「ことわざの意味分類」の回答から。

参考までに当時のチャットのログはこちら。（長いです。。）

Classifying the Meanings of Proverbs

I have long thought that classifying the meanings of proverbs is one of the simplest yet appropriately challenging ways to evaluate the language-handling ability of AI systems.
For example, “Even Kōbō makes mistakes with a brush” and “Even monkeys fall from trees” clearly belong to the same group.
“A dog that walks will get hit by a stick,” however, feels slightly different—after all, walking is not exactly a dog’s area of expertise.

Thinking about the “meaning of meaning” is difficult, but proverbs make this task tractable. Each proverb has both a surface meaning (the literal wording) and a deeper meaning, and people generally agree on both. Because of this shared understanding, there is little ambiguity in either the question or the answer.

When I first tried this kind of test using AI tools like Copilot at the beginning of this year (2025), it did not go well. But when I asked GPT-5 in August, it classified the proverbs effortlessly. That genuinely surprised me.
Encouraged, I added another question: “Read a short story and find a proverb that best matches it.” Once again, the response was spot on.

Honestly, I had not expected AI to advance this far. At that point, I was simply thinking, “GPT-5 is incredible.” So I started talking about my own AI research as well. It understood what I was saying and responded with thoughtful, well-targeted questions.

At the time, I was using GPT in the free tier, so I could not explain everything—but I still ended up talking more than I planned. In that sense, the starting line between GPT and me was proverb meaning classification.

For reference, here is the chat log from that time. (It’s long…japanese only)