Coding with the machines: machine-assisted coding of rare event data
While machine coding of data has dramatically advanced in recent years, the literature raises significant concerns about validation of LLM classification showing, for example, that reliability varies greatly by prompt and temperature tuning, across subject areas and tasks—especially in “zero-shot” applications.
Henry David Overos, Roman Hlatky, Ojashwi Pathak, Harriet Goers, Jordan Gouws-Dewar, Katy Smith, Keith Padraic Chew, Jóhanna K Birnir, Amy H Liu
Published in PNAS Nexus | 4/30/2024