LLM Classification Is Feature Engineering

LLMs-as-classifiers, prompts applied to a context and returning a label, suck to work with. This is especially painful because they often perform pretty decently.

But let’s consider some of the things we’d want in a classifier and see how an LLM-as-classifier stacks up: