Back Rohan Paul

Rohan Paul· @rohanpaul_ai · X· 2026-11-11 09:39 · Original publication time AI score54

Study finds few examples can suffice for natural language inference with small open modelsMachine translation

AI introduction

A study tested five small open language models on three natural language inference datasets, asking them to judge whether one sentence follows from, contradicts, or is unrelated to another. On MNLI and SciTail, Gemma 2, Llama 3.1 and Mistral scored about the same with one example or 15. Mistral 7B exceeded the 63.4% MNLI accuracy reported for a published method using the same model.

Original post · Original

The article text is unavailable in this language; an existing version is shown.

Rohan Paul@rohanpaul_ai

This paper finds that a short, direct prompt with a few examples (In-Context Learning) lets small open LLMs match more complex prompting methods on natural language inference.

The task, called NLI, asks whether one sentence follows from another, contradicts it, or is unrelated. Earlier LLM work on it leaned on complex prompting and sampling tricks, assuming more examples help.

They tested 5 small open models, including Gemma 2 9B, Mistral 7B and Llama 3.1 8B, on 3 NLI datasets. Each got 1 short, direct prompt with 0 to 15 examples and no reasoning steps.

Mistral 7B beat the 63.4% MNLI accuracy that a published in-context sampling method got with the same model. Gemma 2, Llama 3.1 and Mistral scored about the same with 1 example or 15 on MNLI and SciTail.

09:39 · 2026-11-11 · UTC+8

Found an error? Send a correction