Blog

Getting a response from GenAI is quick and straightforward. But what about the confidence level for that response? In certain applications, especially in the financial domain, confidence scores are required. In a document-parsing task related to financial automation, we tested three approaches to address confidence level: calibrator models, logarithmic probabilities (logprobs), and majority voting. Majority voting turned out to be the most performant technique. Although the idea is relatively simple, the devil is in the details.