AI, QC, and ISO

Most research firms are making extensive use of AI for various functions, from qualitative analysis and storytelling to statistical programming, and there is already active exploration of its role in querying and interpreting quantitative databases.  Understandably, the insights industry is so bent on discussing what extraordinary things AI seems able to do that it is paying little more than lip service to what mischief it can do.  There is ample evidence in the press – often comic, always galling – that AI can flub even the tasks it is known to perform very well, by presenting us with its hallucinations, repetitions, misrepresentations, and even its own quirky biases or the biases of their trainers.

In an industry where basic data quality – that is to say, accuracy – has always been our bedrock, we need to take active steps now to ensure we keep our footing solid by doing more than just exhorting care and oversight.  AI’s credibility is now such that informal checks will not be adequate.  Even when errors are found through QC, those errors may be taken, ironically, as evidence that AI is capable of seeing things we humans don’t.  It’s important to develop systematic processes that minimize the risk of replacing human error with machine error.