AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators

[parthp]

The paper proposes AnnoLLM, an annotation system powered by LLMs, which adopts a two-step approach, explain-then-annotate. Concretely, the authors first prompt LLMs to provide explanations for why the specific ground truth answer/label was assigned for a given example. Then, theu construct the few-shot chainof-thought prompt with the self-generated explanation and employ it to annotate the unlabeled data with LLMs. The experiment conducted on three tasks, including user input and keyword relevance assessment, BoolQ, and WiC, demonstrate that AnnoLLM surpasses or performs on par with crowdsourced annotators.