Post-calibration of large language model confidence scoring via combined … — Snowflake Inc. (US12032919B1)

ABSTRACT

Abstract

Examples provide a large language model confidence scoring post-calibration based on a combination of temperature scaling, softmax denominator top-k probabilities selection, and polynomial regression. A secure machine learning system receives results generated by a machine learning (ML) model, the results including at least one confidence score. The secure ML system identifies at least one challenge in accuracy of the results generated by the ML model configured to perform document processing and understanding. The secure machine learning system implements confidence scoring recalibration to address at least one challenge, the confidence scoring recalibration including functionality to assess reliability of the results generated by the ML model, and applies post-processing calibration to the at least one confidence score generated by the confidence scoring recalibration to enhance performance of the ML model, the post-processing calibration including adjusting the at least one confidence score generated by the confidence scoring recalibration.

Description

TECHNICAL FIELD

The present disclosure generally relates to special-purpose machines that manage data platforms and databases incorporating Large Language Models (LLMs) and, more specifically, to data platforms that provide LLM confidence scoring post-calibration based on a combination of techniques.

BACKGROUND

Data platforms may be provided through a cloud data platform, which allows organizations, customers, and users to store, manage, analyze, and retrieve data from the cloud. With respect to a type of data processing, a cloud data platform can implement online transactional processing, online analytical processing, a combination of the two, and/or other types of data processing, including machine-learning (ML) models used for prediction analysis and document understanding using large language models (LLMs).

Users employ various types of electronic documents (also referred to herein as “documents”) including, for example, scanned handwritten documents, scanned forms, large documents (e.g., reports), word processing documents (e.g., DOCX documents), postscript documents (e.g., PDF documents), and the like. Additionally, documents can sometimes be in the form of images (e.g., pictures of pages). A given user may want to process these and other document types and use a document process, such as an ML model-based process, to extract data points from documents to allow for document understanding. For instance, a user can setup a document processing pipeline that is configured to ingest (e.g., continuously running) multiple documents of various types (e.g., images, writing, pictures, drawings, etc.), and process each of the documents using one or more ML models to extract data points of interest to the given user.

BRIEF DESCRIPTION OF THE DRAWINGS

Various ones of the appended drawings merely illustrate example embodiments of the present disclosure and should not be considered as limiting its scope.

FIG. 1 illustrates an example computing environment in which a cloud data platform, in communication with a cloud storage platform, can implement a large language model (LLM), according to some example embodiments.

FIG. 2 is a block diagram illustrating components of a compute service manager, according to some example embodiments.

FIG. 3 is a block diagram illustrating components of an execution platform, according to some example embodiments.

FIG. 4 is a computing environment conceptually illustrating an example software architecture executing a user defined function (UDF) by a process running on a given execution node of the execution platform, according to some example embodiments.

FIG. 5 illustrates components of a Post-hoc Calibration (PHC) Solution Set, according to some example embodiments.

FIG. 6 illustrates baseline solutions, according to some example embodiments.

FIG. 7 illustrates an example of a confidence scoring solution as a cascade of five processing steps including the three computationally inexpensive, but effective, techniques constituting PHC Solution Set (with top-k truncated softmax denomination turned on), according to some example embodiments.

FIG. 8 illustrates charts providing accuracy as a function of confidence showing a degree to which an LLM confidence scoring post-calibration problem is solved in a zero-shot prediction and a fine-tuned-model prediction use case examples, according to some example embodiments.

FIG. 9 is a set of charts illustrating bucketized distributions of true and false predictions for various examples of confidence scores, according to some example embodiments.

FIG. 10 illustrates a chart displaying accuracy as a function of confidence using conventional T5 large language models with post-hoc calibration for two different per-token probabilities aggregation functions, according to some example embodiments.

FIG. 11 is an illustration of a series of charts displaying results of zero-shot experiments with text-image-layout transformers (TILT), according to some example embodiments.

FIG. 12 shows an example flow diagram of a method for providing an LLM confidence scoring post-calibration based on a combination of temperature scaling, softmax denominator top-k probabilities selection, and polynomial regression, according to some example embodiments.

FIG. 13 illustrates a machine-learning pipeline, according to some example embodiments.

FIG. 14 illustrates training and use of a machine-learning program, according to some example embodiments.

FIG. 15 illustrates an example diagrammatic representation of a machine in the form of a computer system within which a set of instructions may be executed for causing the machine to perform any one or more of the methodologies discussed herein, according to some example embodiments.

DETAILED DESCRIPTION

The description that follows includes systems, methods, techniques, instruction sequences, and computing machine program products that embody illustrative embodiments of the disclosure. In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide an understanding of various embodiments of the inventive subject matter. It will be evident, however, to those skilled in the art, that embodiments of the inventive subject matter may be practiced without these specific details. In general, well-known instruction instances, protocols, structures, and techniques are not necessarily shown in detail. For the purposes of this description, the phrase “cloud data platform” may be referred to as and used interchangeably with the phrases “a network-based database system,” “a database system,” or merely “a platform.”

Generally, machine learning (ML) models are not guaranteed to generate accurate and/or quality results, and, in the context of document processing or understanding, this can result in inaccurate data extraction, incorrect understanding, improper calibration, and more. To mitigate and address this issue, users who desire a high-level of accuracy during document data extraction can use varying confidence scoring models. Unfortunately, when processing large amounts of documents (e.g., large language models, generative AI, etc.), during continuous document processing, during document understanding, or the like, improving confidence scores is constrained by a trade-off that may consequently exemplify in a decrease of calibration quality or a decrease of a selective prediction accuracy.

Document understanding in machine learning can refer to the process of extracting meaningful information from documents. Document understanding involves extracting and interpreting information from text documents. This can include several tasks such as: text classification (categorizing documents into predefined classes), named entity recognition (NER) (identifying and classifying named entities in the text), sentiment analysis (determining the sentiment expressed in the text), topic modeling (identifying the main topics that the document discusses), information extraction (extracting structured information from unstructured text), text summarization (creating a short summary that captures the main points of the document), question answering (providing answers to questions based on the information in the document), or the like. Document understanding is crucial in many applications, such as search engines, chatbots, content recommendation, generative Artificial Intelligence (AI), ML models, and more. Currently the most advanced document understanding requires using specialized Large Language Models (LLMs) for visually-aware question answering.

In document understanding, a confidence score can be generated (e.g., computed) at or after the point of making predictions or classifications based on the information extracted from the documents. For example, after processing and understanding a document, an ML model can make predictions or classifications based on the extracted information. For each prediction or classification, the model can also provide a confidence score. This score indicates how certain the model is about its prediction or classification. For example, in a sentiment analysis task, a model might predict that a given text document expresses a positive sentiment with a confidence score of 0.85. This means the model is 85% sure that the sentiment of the text is positive. These confidence scores can be particularly useful in practice. They can help in decision-making processes, in evaluating and comparing different models, and in understanding the model's behavior. They can also be used in post-processing steps, such as calibration, to make the model's predictive quality more reliable. According to some example embodiments, an important application of confidence scores is selective prediction in which the model selects the subset of the most confident predictions, including having confidence scores above a specified threshold value. In many document understanding use cases the least confident predictions are forwarded to human annotators. Therefore, it is important to be able to estimate the model predictions accuracy that the customer/user may expect when they set up the confidence score threshold. Low confidence scoring calibration error corresponds to good alignment of the value of the confidence score threshold and the accuracy of the predictions.

Confidence scoring can refer to the probability that a given prediction or classification is correct. For example, confidence scoring can be a measure of the certainty or probability that a given prediction or classification made by a model is correct. Confidence scoring can include a measure of certainty associated with the output of a model (e.g., a machine learning model, a large language model, etc.). For example, in a binary classification problem, a model can predict that a given document expresses a positive sentiment and assign a confidence score to this prediction. If the confidence score is 0.9, it means the model is 90% sure that the document expresses a positive sentiment. Confidence scores can be useful in many ways. They can help in decision-making processes where not only the prediction but also the certainty of the prediction is important. Confidence scores can also be used to rank predictions, prioritize resources, or in post-processing steps like calibration.

Post-processing of confidence scores for their recalibration can refer to the process of adjusting the confidence scores—originally the output probabilities of a predictive model—after they have been generated. One goal is to ensure that the scores accurately reflect the true likelihood of the outcomes. For example, a well-calibrated binary classifier can output probabilities that can be interpreted as the actual chance of a given instance belonging to the positive class. Using the sentiment analysis example above, if the model predicts an instance with a 0.7 probability, then instances with that prediction should actually be positive 70% of the time (e.g., the document expresses a positive sentiment). Calibration can be important in many applications, such as where reliable probability estimates are needed, such as document understanding, risk modeling, medical diagnosis, and the like.

Confidence scoring post-calibration can refer to the confidence scores assigned to predictions after they have been generated by the model. For example, in machine learning, a model makes predictions and assigns a confidence score to each prediction (e.g., a measure of the model's certainty about that prediction). However, these confidence scores may not always accurately reflect the true probabilities of the predictions being true. Post-calibration is a process that can adjust these confidence scores to make them more reliable (e.g., dependable) and interpretable in terms of accuracy of the corresponding predictions. After calibration, the confidence scores should better represent the actual likelihood of each prediction being correct. For example, if a calibrated model gives a confidence score of 0.8 to a prediction of the document expressing a positive sentiment, it means that out of every 100 predictions with this confidence score, it is expected that about 80 of the predictions are correctly identified as positive. This makes the confidence scores more meaningful (e.g., significant, profound, expressive, etc.) and useful (e.g., convenient, beneficial, etc.) in one or more decision-making processes. Confidence scoring post-calibration can be a crucial step in refining the outputs of a machine learning model by helping to improve the reliability and interpretability of the model's predictions.

In many machine learning applications, models provide not only a prediction but also a confidence score or probability associated with each prediction. These scores can be used to rank the predictions from most to least confident. Selective prediction accuracy refers to the accuracy of a model's predictions when considering only a subset of the predictions where the model is most confident. Selective prediction accuracy can then be calculated by considering only the top predictions up to a certain confidence level or percentage of the total. For example, one might calculate the accuracy of the top 10% most confident predictions, or all predictions with a confidence score above a certain threshold. The selective prediction accuracy measurement can be useful in scenarios where it is more important to be correct (e.g., more accurate) on a smaller number of high-confidence predictions than to make predictions on all instances. For example, in document understanding, it might be preferable to make fewer predictions but have a high accuracy on those predictions, rather than making predictions for all documents but with a lower overall accuracy.

Current technologies fail to consider confidence scores post-processing as aimed at more than just confidence score recalibration: they neglect the perspective of selective predictive accuracy and the need for evaluation of confidence scores from both the perspectives: calibration and selective prediction accuracy. They are also not capable of improving the quality of confidence scoring for LLMs in a computationally inexpensive manner.

Example embodiments of the present disclosure improve upon existing techniques and overcome current technical challenges by providing for systems, methods, devices, instructions, and the like for generating a machine-learning (ML) or artificial intelligence (AI) model for improving both calibration and selective prediction accuracy based on a combination of temperature scaling (sometimes in the form of intentional miscalibration), softmax denominator top-k probabilities selection, and polynomial regression, referred to herein as a “Post-Hoc Calibration (PHC) Solution Set.” The PHC Solution Set incorporates post-calibration functions that constitute a sequence of data processing functions and/or data aggregation functions of different input and output data types for a variety of purposes (e.g., use cases). The PHC Solution Set incorporates the temperature scaling function to process a sequence of per-token logits vectors into another sequence of per-token logits vectors.

Temperature scaling is a method used to calibrate the confidence scores (or probabilities) output by an ML model, such as a neural network. The goal of temperature scaling is to adjust these scores so that the scores better reflect the true probabilities

TECHNICAL FIELD

The present disclosure generally relates to special-purpose machines that manage data platforms and databases incorporating Large Language Models (LLMs) and, more specifically, to data platforms that provide LLM confidence scoring post-calibration based on a combination of techniques.

BACKGROUND

Data platforms may be provided through a cloud data platform, which allows organizations, customers, and users to store, manage, analyze, and retrieve data from the cloud. With respect to a type of data processing, a cloud data platform can implement online transactional processing, online analytical processing, a combination of the two, and/or other types of data processing, including machine-learning (ML) models used for prediction analysis and document understanding using large language models (LLMs).

Users employ various types of electronic documents (also referred to herein as “documents”) including, for example, scanned handwritten documents, scanned forms, large documents (e.g., reports), word processing documents (e.g., DOCX documents), postscript documents (e.g., PDF documents), and the like. Additionally, documents can sometimes be in the form of images (e.g., pictures of pages). A given user may want to process these and other document types and use a document process, such as an ML model-based process, to extract data points from documents to allow for document understanding. For instance, a user can setup a document processing pipeline that is configured to ingest (e.g., continuously running) multiple documents of various types (e.g., images, writing, pictures, drawings, etc.), and process each of the documents using one or more ML models to extract data points of interest to the given user.

BRIEF DESCRIPTION OF THE DRAWINGS

Various ones of the appended drawings merely illustrate example embodiments of the present disclosure and should not be considered as limiting its scope.

FIG. 1 illustrates an example computing environment in which a cloud data platform, in communication with a cloud storage platform, can implement a large language model (LLM), according to some example embodiments.

FIG. 2 is a block diagram illustrating components of a compute service manager, according to some example embodiments.

FIG. 3 is a block diagram illustrating components of an execution platform, according to some example embodiments.

FIG. 4 is a computing environment conceptually illustrating an example software architecture executing a user defined function (UDF) by a process running on a given execution node of the execution platform, according to some example embodiments.

FIG. 5 illustrates components of a Post-hoc Calibration (PHC) Solution Set, according to some example embodiments.

FIG. 6 illustrates baseline solutions, according to some example embodiments.

FIG. 7 illustrates an example of a confidence scoring solution as a cascade of five processing steps including the three computationally inexpensive, but effective, techniques constituting PHC Solution Set (with top-k truncated softmax denomination turned on), according to some example embodiments.

FIG. 8 illustrates charts providing accuracy as a function of confidence showing a degree to which an LLM confidence scoring post-calibration problem is solved in a zero-shot prediction and a fine-tuned-model prediction use case examples, according to some example embodiments.

FIG. 9 is a set of charts illustrating bucketized distributions of true and false predictions for various examples of confidence scores, according to some example embodiments.

FIG. 10 illustrates a chart displaying accuracy as a function of confidence using conventional T5 large language models with post-hoc calibration for two different per-token probabilities aggregation functions, according to some example embodiments.

FIG. 11 is an illustration of a series of charts displaying results of zero-shot experiments with text-image-layout transformers (TILT), according to some example embodiments.

FIG. 12 shows an example flow diagram of a method for providing an LLM confidence scoring post-calibration based on a combination of temperature scaling, softmax denominator top-k probabilities selection, and polynomial regression, according to some example embodiments.

FIG. 13 illustrates a machine-learning pipeline, according to some example embodiments.

FIG. 14 illustrates training and use of a machine-learning program, according to some example embodiments.

FIG. 15 illustrates an example diagrammatic representation of a machine in the form of a computer system within which a set of instructions may be executed for causing the machine to perform any one or more of the methodologies discussed herein, according to some example embodiments.

DETAILED DESCRIPTION

The description that follows includes systems, methods, techniques, instruction sequences, and computing machine program products that embody illustrative embodiments of the disclosure. In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide an understanding of various embodiments of the inventive subject matter. It will be evident, however, to those skilled in the art, that embodiments of the inventive subject matter may be practiced without these specific details. In general, well-known instruction instances, protocols, structures, and techniques are not necessarily shown in detail. For the purposes of this description, the phrase “cloud data platform” may be referred to as and used interchangeably with the phrases “a network-based database system,” “a database system,” or merely “a platform.”

Generally, machine learning (ML) models are not guaranteed to generate accurate and/or quality results, and, in the context of document processing or understanding, this can result in inaccurate data extraction, incorrect understanding, improper calibration, and more. To mitigate and address this issue, users who desire a high-level of accuracy during document data extraction can use varying confidence scoring models. Unfortunately, when processing large amounts of documents (e.g., large language models, generative AI, etc.), during continuous document processing, during document understanding, or the like, improving confidence scores is constrained by a trade-off that may consequently exemplify in a decrease of calibration quality or a decrease of a selective prediction accuracy.

Document understanding in machine learning can refer to the process of extracting meaningful information from documents. Document understanding involves extracting and interpreting information from text documents. This can include several tasks such as: text classification (categorizing documents into predefined classes), named entity recognition (NER) (identifying and classifying named entities in the text), sentiment analysis (determining the sentiment expressed in the text), topic modeling (identifying the main topics that the document discusses), information extraction (extracting structured information from unstructured text), text summarization (creating a short summary that captures the main points of the document), question answering (providing answers to questions based on the information in the document), or the like. Document understanding is crucial in many applications, such as search engines, chatbots, content recommendation, generative Artificial Intelligence (AI), ML models, and more. Currently the most advanced document understanding requires using specialized Large Language Models (LLMs) for visually-aware question answering.

In document understanding, a confidence score can be generated (e.g., computed) at or after the point of making predictions or classifications based on the information extracted from the documents. For example, after processing and understanding a document, an ML model can make predictions or classifications based on the extracted information. For each prediction or classification, the model can also provide a confidence score. This score indicates how certain the model is about its prediction or classification. For example, in a sentiment analysis task, a model might predict that a given text document expresses a positive sentiment with a confidence score of 0.85. This means the model is 85% sure that the sentiment of the text is positive. These confidence scores can be particularly useful in practice. They can help in decision-making processes, in evaluating and comparing different models, and in understanding the model's behavior. They can also be used in post-processing steps, such as calibration, to make the model's predictive quality more reliable. According to some example embodiments, an important application of confidence scores is selective prediction in which the model selects the subset of the most confident predictions, including having confidence scores above a specified threshold value. In many document understanding use cases the least confident predictions are forwarded to human annotators. Therefore, it is important to be able to estimate the model predictions accuracy that the customer/user may expect when they set up the confidence score threshold. Low confidence scoring calibration error corresponds to good alignment of the value of the confidence score threshold and the accuracy of the predictions.

Confidence scoring can refer to the probability that a given prediction or classification is correct. For example, confidence scoring can be a measure of the certainty or probability that a given prediction or classification made by a model is correct. Confidence scoring can include a measure of certainty associated with the output of a model (e.g., a machine learning model, a large language model, etc.). For example, in a binary classification problem, a model can predict that a given document expresses a positive sentiment and assign a confidence score to this prediction. If the confidence score is 0.9, it means the model is 90% sure that the document expresses a positive sentiment. Confidence scores can be useful in many ways. They can help in decision-making processes where not only the prediction but also the certainty of the prediction is important. Confidence scores can also be used to rank predictions, prioritize resources, or in post-processing steps like calibration.

Post-processing of confidence scores for their recalibration can refer to the process of adjusting the confidence scores—originally the output probabilities of a predictive model—after they have been generated. One goal is to ensure that the scores accurately reflect the true likelihood of the outcomes. For example, a well-calibrated binary classifier can output probabilities that can be interpreted as the actual chance of a given instance belonging to the positive class. Using the sentiment analysis example above, if the model predicts an instance with a 0.7 probability, then instances with that prediction should actually be positive 70% of the time (e.g., the document expresses a positive sentiment). Calibration can be important in many applications, such as where reliable probability estimates are needed, such as document understanding, risk modeling, medical diagnosis, and the like.

Confidence scoring post-calibration can refer to the confidence scores assigned to predictions after they have been generated by the model. For example, in machine learning, a model makes predictions and assigns a confidence score to each prediction (e.g., a measure of the model's certainty about that prediction). However, these confidence scores may not always accurately reflect the true probabilities of the predictions being true. Post-calibration is a process that can adjust these confidence scores to make them more reliable (e.g., dependable) and interpretable in terms of accuracy of the corresponding predictions. After calibration, the confidence scores should better represent the actual likelihood of each prediction being correct. For example, if a calibrated model gives a confidence score of 0.8 to a prediction of the document expressing a positive sentiment, it means that out of every 100 predictions with this confidence score, it is expected that about 80 of the predictions are correctly identified as positive. This makes the confidence scores more meaningful (e.g., significant, profound, expressive, etc.) and useful (e.g., convenient, beneficial, etc.) in one or more decision-making processes. Confidence scoring post-calibration can be a crucial step in refining the outputs of a machine learning model by helping to improve the reliability and interpretability of the model's predictions.

In many machine learning applications, models provide not only a prediction but also a confidence score or probability associated with each prediction. These scores can be used to rank the predictions from most to least confident. Selective prediction accuracy refers to the accuracy of a model's predictions when considering only a subset of the predictions where the model is most confident. Selective prediction accuracy can then be calculated by considering only the top predictions up to a certain confidence level or percentage of the total. For example, one might calculate the accuracy of the top 10% most confident predictions, or all predictions with a confidence score above a certain threshold. The selective prediction accuracy measurement can be useful in scenarios where it is more important to be correct (e.g., more accurate) on a smaller number of high-confidence predictions than to make predictions on all instances. For example, in document understanding, it might be preferable to make fewer predictions but have a high accuracy on those predictions, rather than making predictions for all documents but with a lower overall accuracy.

Current technologies fail to consider confidence scores post-processing as aimed at more than just confidence score recalibration: they neglect the perspective of selective predictive accuracy and the need for evaluation of confidence scores from both the perspectives: calibration and selective prediction accuracy. They are also not capable of improving the quality of confidence scoring for LLMs in a computationally inexpensive manner.

Example embodiments of the present disclosure improve upon existing techniques and overcome current technical challenges by providing for systems, methods, devices, instructions, and the like for generating a machine-learning (ML) or artificial intelligence (AI) model for improving both calibration and selective prediction accuracy based on a combination of temperature scaling (sometimes in the form of intentional miscalibration), softmax denominator top-k probabilities selection, and polynomial regression, referred to herein as a “Post-Hoc Calibration (PHC) Solution Set.” The PHC Solution Set incorporates post-calibration functions that constitute a sequence of data processing functions and/or data aggregation functions of different input and output data types for a variety of purposes (e.g., use cases). The PHC Solution Set incorporates the temperature scaling function to process a sequence of per-token logits vectors into another sequence of per-token logits vectors.

Temperature scaling is a method used to calibrate the confidence scores (or probabilities) output by an ML model, such as a neural network. The goal of temperature scaling is to adjust these scores so that the scores better reflect the true probabilities of each prediction. In temperature scaling, a single parameter, often referred to as a “temperature,” is learned on a validation set (e.g., validation set is a subset of the dataset that is used during the training of the ML model). This parameter can be used to adjust the logits at the output of the core of the model. The softmax function is commonly used in ML to convert a vector of real numbers into a probability distribution. The temperature parameter adjusts the “sharpness” of this distribution. For example, a high temperature makes the distribution more uniform (e.g., less confident), while a low temperature makes it more concentrated on a few outcomes (e.g., more confident). By learning the optimal temperature on a validation set, the model's confidence scores can be adjusted to better match the true probabilities. This can be particularly useful in scenarios where well-calibrated probabilities are important and cannot be recalibrated at some post-processing step, such as in document understanding, risk modeling, medical diagnosis, or the like.

The PHC Solution Set further incorporates a top-k softmax denomination function to process a sequence of per-token logits vectors to be used to compute the softmax function denominator (e.g., at the next step of softmax function application proving per-token probabilities) into another sequence (e.g., typically modified) of per-token logits vectors to be used to compute the softmax function denominator. The term “top-k softmax” refers to a variation of the softmax function used in machine learning, particularly in classification tasks. While the softmax function is used to convert a vector of real numbers into a probability distribution, the “top-k” component refers to considering only the top k elements according to their probabilities. In other words, the k largest probabilities are selected from the output of the softmax function. These selected probabilities are then renormalized to sum to 1, forming a new probability distribution. Last, the PHC Solution Set combines the temperature scaling function and a top-k softmax denomination function with a regression-based post-hoc calibration function to process a single number, being the result of some aggregation of per-token probabilities (e.g., such as aggregation based on geometric mean, mean, or the like) into a preliminary (e.g., not re-calibrated; sometimes even intentionally miscalibrated) confidence score, and finally, into a single number being the final, re-calibrated confidence score. A regression-based post-hoc calibration function is a method used to adjust the output of an ML model after it has been generated (in the process of the output prediction generation). One goal of regression-based post-hoc calibration is to make the model's confidence scores more accurate and reliable. In the context of machine learning, “post-hoc” refers to adjustments or analyses that are performed after the model has been trained. “Calibration” refers to the process of adjusting the model's predictions to better align with reality. A regression-based post-hoc calibration function works by fitting a regression model to the model's predictions and the true outcomes (e.g., reality). This regression model learns the relationship between the model's predictions and the true outcomes and can then be used to adjust the model's future confidence scores. For example, if a model consistently overestimates the probability of a certain outcome, a regression-based post-hoc calibration function can learn this and adjust the model's future predictions downward. This method can be particularly useful in scenarios where the model's predictions are used to make important decisions, and it is crucial that these predictions are as reliable as possible.

A regression-based post-hoc calibration function may map a single number to a single number. In such a case, the PHC Solution Set regressor is one-dimensional. In consequence, such a computationally inexpensive (e.g., cheap) way of confidence score recalibration may be easily enforced (e.g., ensured) to be monotonic (a function is said to be monotonic if it either entirely increases or decreases). In the special case of polynomial regression, this monotonicity can be verified with full certainty at a low computational cost by means of a standard differential calculus (e.g., polynomial derivative roots inspection) procedure. The use of polynomial regression as a basis for regression-based post-hoc calibration is novel.

Traditional approaches address this problem through prior solutions—including existing state-of-the-art regression-based LLM post-hoc calibration methods based on gradient-boosted tree regression (often referred to as “feature-based forecasters”). However, existing post-hoc calibration methods that are based on gradient-boosted tree regression or any other regression not being one-dimensional, cannot guarantee the unique properties that may be guaranteed when low-degree one-dimensional polynomial regression is used as presented in example embodiments of the present disclosure related to important unique properties of one-dimensional, low-degree polynomial regression used for regression-based post-hoc calibration.

Moreover, as far as inference computation cost is concerned, the existing regression-based LLM post-hoc calibration methods are more expensive (e.g., methods based on gradient-boosted tree regression and methods based on multiple answers and prediction sampling, like Self-Contrastive (SelfCon) Learning). Furthermore, temperature scaling and regression-based post-hoc calibration are the only computationally efficient and functionally effective (e.g., capable of introducing a significant improvement of selective prediction quality and/or calibration quality) methods proposed and advocated by more than just one independent group of researchers. Nevertheless, so far, no combination of temperature scaling and regression based PHC methods has been proposed that enables improving selective prediction quality and calibration quality at the same time.

The problems related to confidence score calibration are challenging for even the leading artificial intelligence/machine learning (AI/ML) market players. In other words, all the existing solutions can provide selective prediction quality improvement at the “cost” of calibration degradation or calibration improvement at the “cost” of selective prediction quality degradation. Moreover, existing methods and approaches to confidence scoring and prediction in LLM output fail to successfully calibrate confidence scores, leading even the strongest existing LLMs to provide output that is prone to LLM hallucinations (e.g., instances where the model generates or predicts information that is not present in the input data). For example, certain LLMs either are not concerned about using the estimated probability of being correct or cannot estimate this probability well enough, causing LLM hallucinations. Such low quality of confidence scores hurts document understanding performance, especially in the challenging zero-shot scenario example (a zero-shot scenario refers to a situation where an ML model is asked to make predictions on a type of data that it has not seen during training). No existing solutions enable improving accuracy for some fraction of the most confident LLM answers (e.g., improving accuracy in a selective prediction application scenario) without compromising calibration, or improving calibration (e.g., reducing calibration error) without compromising accuracy for some fraction of the most confident LLM answers.

Existing methods fail to overcome the technical challenges related to determining if an LLM has produced a prediction that is confident enough. The combination of functions described in the present disclosure have not been considered in prior existing solutions because many data scientists consider that confidence scores are solely about calibrating those scores. However, there have been no attempts at multi-faceted solutions of confidence scoring including a first facet that considers selective prediction performance that is dependent only on different ways of sorting confidence scores (e.g., least confident to most confident, and vice versa). In addition, the second facet includes a separate (e.g., distinct) task of predicting (e.g., guessing) what is the probability of the given prediction being correct.

Example embodiments of the present disclosure overcome the technical difficulties of prior solutions by providing a unique combination (e.g., a sequence) of two complementary techniques referred to herein as “temperature scaling” and “regression-based post-hoc calibration.” The temperature scaling technique is used in example embodiments in a different way than it is used in prior existing methods. Specifically, temperature scaling is used herein to improve selective prediction accuracy rather than to improve calibration; for example, sometimes even deliberately radically increasing calibration error, which goes against and performs in clear contrast to all the existing solutions. The regression-based post-hoc calibration technique is used in example embodiments to compensate for calibration error either not mitigated at the temperature scaling step or deliberately increased at the temperature scaling step (e.g., in clear contrast to existing solutions). In addition, top-k softmax denomination plays a secondary role as an optional element of the solution to provide an impact on the outcome.

For example, using the confidence scores presented in example embodiments of the present disclosure enable better self-awareness of large language models using an innovative solution that make each of the models to significantly know when it is right (e.g., to increase its selective prediction accuracy) and what is the probability that it is right (e.g., to improve its calibration). The instant solution can be applied to any generative LLM, such as document understanding LLMs, AutoSQL LLMs, and the like. The instant solution is an innovative combination of solutions previously only applied individually, as well as currently being applied in different ways.

Example embodiments of the present disclosure ensure uniquely low computational cost of the new solution as a result of combining two computationally inexpensive (e.g., cheap) methods referred to herein as “temperature scaling” and “regression-based post-hoc calibration.” For example, computationally inexpensive methods can include algorithms or processes that require relatively low computational resources to execute, where resources can include processing power, memory, time, or combinations thereof. Generally, if a process is computationally inexpensive, it can be executed quickly and without significantly impacting the performance of computer systems and/or the cloud data platform. Example embodiments provide for improvements over existing techniques by combining multiple computationally inexpensive processes to provide for post-calibration confidence scoring in large language models. This is particularly important in the fields of machine learning and artificial intelligence, specifically in regard to large language models, where complex algorithms used individually can require significant computational power, which slow down the system or make the process impractical for certain applications, like LLMs.

Example embodiments of the present disclosure include multiple advantages previously undiscovered, including for example, one key property of the PHC Solution Set being the ability to improve confidence scores, while this improvement is seen from two perspectives that so far were at least to some extent, in a trade-off relation. In clear contrast to the existing solutions, the example embodiments provided herein can improve both selective prediction quality and calibration as a result of join application of the three techniques that constitute the PHC Solution Set. In general, the problem addressed by the new solution (e.g., PHC Solution Set) is the improvement of the quality of confidence scoring for Large Language Models (LLMs) and additional types of deep learning models or machine learning models in general. The PHC Solution Set provided by the instant disclosure can be applied to any LLM (e.g., AutoSQL, Document Understanding, etc.).

Example embodiments of the present disclosure are directed to systems, methods, machine-readable mediums, devices, instructions, and the like for improved confidence scoring models in document understanding systems. More specifically, examples of the present disclosure provide for large language model (LLM) confidence scoring post-calibration employing a combination of temperature scaling, softmax denominator top-k probabilities selection, and polynomial regression to be used, for example, in document understanding, AutoSQL, and the like. To resolve existing technical problems, example embodiments of a cloud data platform can employ and enable fine-tuned models to provide a rich user experience to use the fine-tuned models to extract information from structured and unstructured documents. Example embodiments of the present disclosure include a cloud data platform operatively integrated with an LLM service to create and provide a fine-tuned machine-learning model from a zero-shot base model to extract information from data files. Example embodiments overcome existing technical problems by providing an effective reduction of calibration error to generate a best outcome, while simultaneously improving selective prediction accuracy.

Example embodiments of an LLM as used herein are understood in the most widespread way (e.g., as any transformer-architecture model such as text-image-layout transformer (TILT) language models being the core of the cloud data platform document understanding, Text-to-Text Transfer Transformer (T5), Generative Pre-Trained Transformers (GPT), and other generative artificial intelligence (AI) applications). Using the LLM described herein, raw predictions can include a sequence of per-token logits transformed (e.g., using a softmax-based function or the like) to the corresponding sequence of per-token probabilities, being the input data for an aggregation function (e.g., geometric mean, etc.), post-calibration function, or some combination thereof. Finally, providing a single confidence score value for the whole sequence of generated (e.g., predicted) model output tokens. Confidence scoring post-calibration is understood as a function, or set or sequence of functions, individually or jointly processing a sequence of per-token logits into the final single-number confidence score (e.g., attributed to the generated LLM prediction and/or answer).

In the context of large language models, and as used herein, a sequence of per-token logits includes a series of numerical values, in the form of algebraic vectors, which represent a model's prediction(s) for each token in a sequence representing the predicted text. Logits include the raw, unnormalized output of a neural network prior to the application of a softmax function that is used to convert the logits into probabilities. For example, the logits can provide a measurement of a model's confidence score (e.g., a belief in each possible token at each position in a sequence) and, after conversion to probabilities, are used in additional machine-learning or application tasks, such as translation, classification, generation, and the like used to convert an ML model output into useful (e.g., understandable, expressive, actually appearing in a scanned document, etc.) text.

In some example embodiments, individual elements of information that are targeted for extraction from an electronic document can be referred to as data points, and the extracted values for those data points can be referred to as data point values. For some embodiments, data points extracted from electronic documents are stored into a table (e.g., of a cloud data platform), such as a database table, where extracted values for the data points can be stored in columns of the table according to data point-to-column mapping. As used herein, an electronic document can comprise a file. Use of various embodiments can, for example, provide for a new confidence score solution that improves both calibration and selective prediction accuracy of electronic documents. In some examples, where certain electronic documents, when processed by an ML model for extraction of one or more data points, result in the ML model providing (e.g., indicating) confidence scores for one or more data point values extracted.

As used herein, a provider is an organization, company, or account (e.g., provider user) that owns and hosts a database or a set of data within the cloud data platform, the provider can be responsible for making the data available to other accounts or consumers for sharing and analysis such as sharing specific databases, schemas, relations, or the like with other accounts. As used herein, a consumer is an organization, company, or account (e.g., consumer user) that accesses and consumes data shared by the provider, where consumers can access and query the shared data without the need for data replication or data movement. Consumers can further combine the shared data with their own data within the cloud data platform to perform various analytical operations on the data.

To summarize the advantages over prior existing solutions, the example embodiments presented in the present disclosure provide a new solution that improves selective prediction accuracy by a couple of tens of percentage points, depending on the example embodiment, while simultaneously reducing calibration error by up to a factor of four of a generative large language model. In contrast to existing solutions that can improve only the selective prediction accuracy or only the calibration, or only improve calibration while decreasing selective prediction accuracy.

FIG. 1 illustrates an example computing environment 100 in which a cloud data platform 102 can implement a Post-Hoc Calibration (PHC) Solution Set based on a combination of temperature scaling, softmax denominator top-k probabilities selection, and polynomial regression, according to some example embodiments. To avoid obscuring the inventive subject matter with unnecessary detail, various functional components that are not germane to conveying an understanding of the inventive subject matter have been omitted from FIG. 1 . However, a skilled artisan will readily recognize that various additional functional components may be included as part of the computing environment 100 to facilitate additional functionality that is not specifically described herein. In other embodiments, the computing environment may comprise another type of network-based database system or a cloud data platform.

As shown, the computing environment 100 comprises the cloud data platform 102 in communication with a cloud storage platform 104 (e.g., AWS*, Microsoft Azure Blob Storage*, or Google Cloud Storage). The cloud data platform 102 is a network-based system used for reporting and analysis of integrated data from one or more disparate sources including one or more storage locations within the cloud storage platform 104 . The cloud storage platform 104 comprises a plurality of computing machines and provides on-demand computer system resources such as data storage and computing power to the cloud data platform 102 .

The cloud data platform 102 comprises a compute service manager 108 , an execution platform 110 , and one or more metadata databases 112 . The cloud data platform 102 hosts and provides data reporting and analysis services to multiple client accounts.

The compute service manager 108 coordinates and manages operations of the cloud data platform 102 . The compute service manager 108 also performs query optimization and compilation as well as managing clusters of computing services that provide compute resources (also referred to as “virtual warehouses”). The compute service manager 108 can support any number of client accounts, such as end-users providing data storage and retrieval requests, system administrators managing the systems and methods described herein, and other components/devices that interact with compute service manager 108 .

The compute service manager 108 is also in communication with a client device 114 . The client device 114 corresponds to a user of one of the multiple client accounts supported by the cloud data platform 102 . A user may utilize the client device 114 to submit data storage, retrieval, and analysis requests to the compute service manager 108 .

The compute service manager 108 is also coupled to one or more metadata databases 112 that store metadata pertaining to various functions and aspects associated with the cloud data platform 102 and its users. For example, metadata database(s) 112 may include a summary of data stored in remote data storage systems as well as data available from a local cache. Additionally, metadata database(s) 112 may include information regarding how data is partitioned and organized in remote data storage systems (e.g., the cloud storage platform 104 ) and local caches. As discussed herein, a “micro-partition” is a batch storage unit, and each micro-partition has contiguous units of storage. By way of example, each micro-partition may contain between 50 MB and 500 MB of uncompressed data (note that the actual size in storage may be smaller because data may be stored compressed). Groups of rows in tables may be mapped into individual micro-partitions organized in a columnar fashion. This size and structure allow for extremely granular selection of the micro-partitions to be scanned, which can comprise millions, or even hundreds of millions, of micro-partitions. This granular selection process for micro-partitions to be scanned is referred to herein as “pruning.” Pruning involves using metadata to determine which portions of a table, including which micro-partitions or micro-partition groupings in the table, are not pertinent to a query, avoiding those non-pertinent micro-partitions when responding to the query, and scanning only the pertinent micro-partitions to respond to the query. Metadata may be automatically gathered on all rows stored in a micro-partition, including the range of values for each of the columns in the micro-partition; the number of distinct values; and/or additional properties used for both optimization and efficient query processing. In one embodiment, micro-partitioning may be automatically performed on all tables. For example, tables may be transparently partitioned using the ordering that occurs when the data is inserted/loaded. However, it should be appreciated that this disclosure of the micro-partition is exemplary only and should be considered non-limiting. It should be appreciated that the micro-partition may include other database storage devices without departing from the scope of the disclosure. Information stored by a metadata database 112 (e.g., key-value pair data store) allows systems and services to determine whether a piece of data (e.g., a given partition) needs to be accessed without loading or accessing the actual data from a storage device.

The compute service manager 108 is further coupled to the execution platform 110 , which provides multiple computing resources that execute various data storage and data retrieval tasks. The execution platform 110 is coupled to cloud storage platform 104 . The cloud storage platform 104 comprises multiple data storage devices 120 - 1 to 120 -N. In some embodiments, the data storage devices 120 - 1 to 120 -N are cloud-based storage devices located in one or more geographic locations. For example, the data storage devices 120 - 1 to 120 -N may be part of a public cloud infrastructure or a private cloud infrastructure. The data storage devices 120 - 1 to 120 -N may be hard disk drives (HDDs), solid state drives (SSDs), storage clusters, Amazon S3™ storage systems, or any other data storage technology. Additionally, the cloud storage platform 104 may include distributed file systems (such as Hadoop Distributed File Systems (HDFS)), object storage systems, and the like.

In some embodiments, the cloud data platform 102 includes a machine-learning (ML) generator 132 . The ML generator 132 comprises suitable circuitry, interfaces, logic, and/or code and is configured to provide generation of ML models for extracting information from one or more documents (e.g., electronic documents) according to various embodiments. In some embodiments, the ML generator 132 can include one or more system functions that can be used to implement a method of generating an ML model as described herein. In some examples, the ML generator 132 can be operatively interconnected to the compute service manager 108 , within the compute service manager 108 (as depicted in FIG. 2 ), connected to the execution platform 110 , connected to the meta database(s) 112 , or otherwise connected within or operatively to the cloud data platform 102 via additional external connections.

The execution platform 110 comprises a plurality of compute nodes. A set of processes on a compute node executes a query plan compiled by the compute service manager 108 . The set of processes can include: a first process to execute the query plan; a second process to monitor and delete cache files using a least recently used (LRU) policy and implement an out of memory (OOM) error mitigation process; a third process that extracts health information from process logs and status to send back to the compute service manager 108 ; a fourth process to establish communication with the compute service manager 108 after a system boot; and a fifth process to handle all communication with a compute cluster for a given job provided by the compute service manager 108 and to communicate information back to the compute service manager 108 and other compute nodes of the execution platform 110 .

In some embodiments, communication links between elements of the computing environment 100 are implemented via one or more data communication networks. These data communication networks may utilize any communication protocol and any type of communication medium. In some embodiments, the data communication networks are a combination of two or more data communication networks (or sub-networks) coupled to one another. In alternate embodiments, these communication links are implemented using any type of communication medium and any communication protocol.

The compute service manager 108 , metadata database(s) 112 , execution platform 110 , and cloud storage platform 104 are shown in FIG. 1 as individual discrete components. However, each of the compute service managers 108 , metadata databases 112 , execution platforms 110 , and cloud storage platforms 104 may be implemented as a distributed system (e.g., distributed across multiple systems/platforms at multiple geographic locations). Additionally, each of the compute service managers 108 , metadata databases 112 , execution platforms 110 , and cloud storage platforms 104 can be scaled up or down (independently of one another) depending on changes to the requests received and the changing needs of the cloud data platform 102 . Thus, in the described embodiments, the cloud data platform 102 is dynamic and supports regular changes to meet the current data processing needs.

During typical operation, the cloud data platform 102 processes multiple jobs determined by the compute service manager 108 . These jobs are scheduled and managed by the compute service manager 108 to determine when and how to execute the job. For example, the compute service manager 108 may divide the job into multiple discrete tasks and may determine what data is needed to execute each of the multiple discrete tasks. The compute service manager 108 may assign each of the multiple discrete tasks to one or more nodes of the execution platform 110 to process the task. The compute service manager 108 may determine what data is needed to process a task and further determine which nodes within the execution platform 110 are best suited to process the task. Some nodes may

CLAIMS

Claims ( 30 )

What is claimed is:

1. A method comprising:

receiving, by at least one hardware processor, results generated by a machine learning (ML) model, the results including at least one confidence score;

enabling high selective prediction accuracy of the results generated by the ML model configured to perform document processing and understanding of an electronic text document;

implementing confidence scoring recalibration to address at least one challenge, the confidence scoring recalibration including functionality to assess reliability of the results generated by the ML model;

applying post-processing calibration to the at least one confidence score generated by the confidence scoring recalibration to enhance performance of the ML model, the post-processing calibration including adjusting the at least one confidence score generated by the confidence scoring recalibration;

based on the adjusting of the at least one confidence score, extracting individual elements of information from the electronic text document, the extracted individual elements of the information including one or more extracted values; and

storing the one or more extracted values in a database including an adjusted confidence score associated with each of the one or more extracted values and the results generated by a ML model.

2. The method of claim 1 , comprising:

utilizing selective prediction accuracy to improve the performance of the ML model, the selective prediction accuracy including the functionality to predict an accuracy of the results generated by the ML model; and

improving both calibration quality and the selective prediction accuracy of the ML model performing the document processing and the understanding.

3. The method of claim 1 , comprising:

identifying a quality of confidence score.

4. The method of claim 3 , comprising:

recalibrating the confidence score based on the quality of the confidence score; and

applying a combination of temperature scaling, softmax denominator top-k probability selection, and polynomial regression to the confidence score.

5. The method of claim 1 , wherein the ML model is a pre-trained large language model.

6. The method of claim 5 , comprising:

generating one or more outputs from the pre-trained large language model; and

applying the pre-trained large language model to make predictions on data not used for pre-training.

7. The method of claim 6 , comprising:

processing, by one or more hardware processor, a first set of electronic documents, using the pre-trained large language model, to extract a set of values for a set of data points for each of the first set of electronic documents; and

recalibrating the at least one confidence score based on polynomial regression.

8. The method of claim 7 , comprising:

processing, by the ML model, a second set of electronic documents, including extracting one or more data points for each of the first set of electronic documents;

providing a new confidence score solution including improving both calibration quality and selective prediction accuracy of the second set of electronic documents based on a combination of temperature scaling, top-k softmax denomination and recalibrating of the at least one confidence score of the first set of electronic documents; and

indicating, by the ML model, confidence scores for one or more data point values extracted from the second set of electronic documents, when processed by the ML model for extraction.

9. The method of claim 8 , comprising:

a sequence of per-token logits including a series of numerical values that represent the predictions for each token in a sequence of electronic documents based on the ML model.

10. The method of claim 9 , wherein logits include raw, unnormalized output of a neural network prior to application of a softmax function, and comprising:

using the softmax function to convert the logits into probabilities, the logits providing a preliminary measurement of the at least one confidence score for the ML model.

11. A system comprising:

one or more hardware processors of a machine; and

at least one memory storing instructions that, when executed by the one or more hardware processors, cause the system to perform operations comprising:

receiving, by at least one hardware processor, results generated by a machine learning (ML) model, the results including at least one confidence score;

enabling high selective prediction accuracy of the results generated by the ML model configured to perform document processing and understanding of an electronic text document;

implementing confidence scoring recalibration to address at least one challenge, the confidence scoring recalibration including functionality to assess reliability of the results generated by the ML model;

applying post-processing calibration to the at least one confidence score generated by the confidence scoring recalibration to enhance performance of the ML model, the post-processing calibration including adjusting the at least one confidence score generated by the confidence scoring recalibration;

based on the adjusting of the at least one confidence score, extracting individual elements of information from the electronic text document, the extracted individual elements of the information including one or more extracted values; and

storing the one or more extracted values in a database including an adjusted confidence score associated with each of the one or more extracted values and the results generated by a ML model.

12. The system of claim 11 , the operations comprising:

utilizing selective prediction accuracy to improve the performance of the ML model, the selective prediction accuracy including the functionality to predict an accuracy of the results generated by the ML model; and

improving both calibration quality and the selective prediction accuracy of the ML model performing the document processing and the understanding.

13. The system of claim 11 , the operations comprising:

identifying a quality of the at least one confidence score.

14. The system of claim 13 , the operations comprising:

recalibrating the at least one confidence score based on the quality of the at least one confidence score; and

applying a combination of temperature scaling, softmax denominator top-k probability selection, and polynomial regression to the at least one confidence score.

15. The system of claim 11 , wherein the ML model is a pre-trained large language model.

16. The system of claim 15 , the operations comprising:

generating one or more outputs from the pre-trained large language model; and

applying the pre-trained large language model to make predictions on data not used for pre-training.

17. The system of claim 16 , wherein the operations comprise:

processing, by one or more hardware processor, a first set of electronic documents, using the pre-trained large language model, to extract a set of values for a set of data points for each of the first set of electronic documents; and

recalibrating the at least one confidence score based on polynomial regression.

18. The system of claim 17 , wherein the operations comprise:

processing, by the ML model, a second set of electronic documents, including extracting one or more data points for each of the first set of electronic documents;

providing a new confidence score solution including improving both calibration quality and selective prediction accuracy of the second set of electronic documents based on a combination of temperature scaling, top-k softmax denomination and recalibrating of the at least one confidence score of the first set of electronic documents; and

indicating, by the ML model, confidence scores for one or more data point values extracted from the second set of electronic documents, when processed by the ML model for extraction.

19. The system of claim 18 , wherein the operations comprise:

a sequence of per-token logits including a series of numerical values that represent the predictions for each token in a sequence of electronic documents based on the ML model.

20. The system of claim 19 , wherein logits include raw, unnormalized output of a neural network prior to application of a softmax function, and comprising:

using the softmax function to convert the logits into probabilities, the logits providing a preliminary measurement of the at least one confidence score for the ML model.

21. A machine-storage medium embodying instructions that, when executed by a machine, cause the machine to perform operations comprising:

receiving, by at least one hardware processor, results generated by a machine learning (ML) model, the results including at least one confidence score;

enabling high selective prediction accuracy of the results generated by the ML model configured to perform document processing and understanding of an electronic text document;

implementing confidence scoring recalibration to address at least one challenge, the confidence scoring recalibration including functionality to assess reliability of the results generated by the ML model;

applying post-processing calibration to the at least one confidence score generated by the confidence scoring recalibration to enhance performance of the ML model, the post-processing calibration including adjusting the at least one confidence score generated by the confidence scoring recalibration;

based on the adjusting of the at least one confidence score, extracting individual elements of information from the electronic text document, the extracted individual elements of the information including one or more extracted values; and

storing the one or more extracted values in a database including an adjusted confidence score associated with each of the one or more extracted values and the results generated by a ML model.

22. The machine-storage medium of claim 21 , wherein the operations comprise:

utilizing selective prediction accuracy to improve the performance of the ML model, the selective prediction accuracy including the functionality to predict an accuracy of the results generated by the ML model; and

improving both calibration quality and the selective prediction accuracy of the ML model performing the document processing and the understanding.

23. The machine-storage medium of claim 21 , wherein the operations comprise:

identifying a quality of confidence score.

24. The machine-storage medium of claim 23 , wherein the operations comprise:

recalibrating the confidence score based on the quality of the confidence score; and

applying a combination of temperature scaling, softmax denominator top-k probability selection, and polynomial regression to the confidence score.

25. The machine-storage medium of claim 21 , wherein the ML model is a pre-trained large language model.

26. The machine-storage medium of claim 25 , wherein the operations comprise:

generating one or more outputs from the pre-trained large language model; and

applying the pre-trained large language model to make predictions on data not used for pre-training.

27. The machine-storage medium of claim 26 , wherein the operations comprise:

processing, by one or more hardware processor, a first set of electronic documents, using the pre-trained large language model, to extract a set of values for a set of data points for each of the first set of electronic documents; and

recalibrating the at least one confidence score based on polynomial regression.

28. The machine-storage medium of claim 27 , wherein the operations comprise:

processing, by the ML model, a second set of electronic documents, including extracting one or more data points for each of the first set of electronic documents;

providing a new confidence score solution including improving both calibration quality and selective prediction accuracy of the second set of electronic documents based on a combination of temperature scaling, top-k softmax denomination and recalibrating of the at least one confidence score of the first set of electronic documents; and

indicating, by the ML model, confidence scores for one or more data point values extracted from the second set of electronic documents, when processed by the ML model for extraction.

29. The machine-storage medium of claim 28 , wherein the operations comprise:

a sequence of per-token logits including a series of numerical values that represent the predictions for each token in a sequence of electronic documents based on the ML model.

30. The machine-storage medium of claim 29 , wherein logits include raw, unnormalized output of a neural network prior to application of a softmax function, and comprising:

using the softmax function to convert the logits into probabilities, the logits providing a preliminary measurement of the at least one confidence score for the ML model.

US18/450,700

2023-08-16

2023-08-16

Post-calibration of large language model confidence scoring via combined techniques

Active

US12032919B1

( en )

Priority Applications (2)

Application Number

Priority Date

Filing Date

Title

US18/450,700

US12032919B1

( en )

2023-08-16

2023-08-16

Post-calibration of large language model confidence scoring via combined techniques

US18/677,561

US12236201B1

( en )

2023-08-16

2024-05-29

Enhanced machine learning model accuracy through post-hoc confidence score calibration

Applications Claiming Priority (1)

Application Number

Priority Date

Filing Date

Title

US18/450,700

US12032919B1

( en )

2023-08-16

2023-08-16

Post-calibration of large language model confidence scoring via combined techniques

Related Child Applications (1)

Application Number

Title

Priority Date

Filing Date

US18/677,561

Continuation

US12236201B1

( en )

2023-08-16

2024-05-29

Enhanced machine learning model accuracy through post-hoc confidence score calibration

Publications (1)

Publication Number

Publication Date

US12032919B1

true

US12032919B1 ( en )

2024-07-09

Family

ID=91760693

Family Applications (2)

Application Number

Title

Priority Date

Filing Date

US18/450,700

Active

US12032919B1

( en )

2023-08-16

2023-08-16

Post-calibration of large language model confidence scoring via combined techniques

US18/677,561

Active

US12236201B1

( en )

2023-08-16

2024-05-29

Enhanced machine learning model accuracy through post-hoc confidence score calibration

Family Applications After (1)

Application Number

Title

Priority Date

Filing Date

US18/677,561

Active

US12236201B1

( en )

2023-08-16

2024-05-29

Enhanced machine learning model accuracy through post-hoc confidence score calibration

Country Status (1)

Country

Link

US

( 2 )

US12032919B1

( en )

Cited By (10)

* Cited by examiner, † Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

CN119299130A

( en )

*

2024-09-09

2025-01-10

四川大学

Method and system for automatically generating network threat traffic detection rules based on LLM

CN119402935A

( en )

*

2025-01-03

2025-02-07

深圳市三德大康电子有限公司

Bluetooth headset automatic pairing method and device for multi-terminal switching

US12236201B1

( en )

2023-08-16

2025-02-25

Snowflake Inc.

Enhanced machine learning model accuracy through post-hoc confidence score calibration

CN119587047A

( en )

*

2025-01-16

2025-03-11

宁夏祥瑞智能科技有限公司

A lock-free loop-based real-time streaming system for electroencephalography

US12367223B1

( en )

*

2024-11-18

2025-07-22

Distributed Analytics Solutions, Ltd.

Calibrated large language models for multi-label classifications

CN121384852A

( en )

*

2025-12-23

2026-01-23

大连计量检验检测研究院有限公司

A household gas detection device

CN121524692A

( en )

*

2025-11-21

2026-02-13

东软集团股份有限公司

A method, apparatus, processor, and device for optimizing entity clustering models.

CN121636662A

( en )

*

2025-12-03

2026-03-10

国电南瑞科技股份有限公司

A knowledge graph query method and apparatus based on the fusion of results from multiple language models

US12579364B1

( en )

*

2024-12-11

2026-03-17

Theodora Lab AI LLC

Machine learning techniques for detecting, measuring, and mitigating bias within textual content

US20260170265A1

( en )

*

2024-12-16

2026-06-18

Paypal, Inc.

Large Language Model (LLM) Token Truncation

Families Citing this family (2)

* Cited by examiner, † Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US12314301B2

( en )

*

2023-08-24

2025-05-27

Microsoft Technology Licensing, Llc.

Code search for examples to augment model prompt

WO2026059957A1

( en )

*

2024-09-11

2026-03-19

Corelogic Solutions, Llc

Natural language model based real estate trend predictions

Citations (2)

* Cited by examiner, † Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US20120109652A1

( en )

*

2010-10-27

2012-05-03

Microsoft Corporation

Leveraging Interaction Context to Improve Recognition Confidence Scores

US20220351088A1

( en )

*

2021-04-30

2022-11-03

Intuit Inc.

Machine learning model-agnostic confidence calibration system and method

Family Cites Families (1)

* Cited by examiner, † Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US12032919B1

( en )

2023-08-16

2024-07-09

Snowflake Inc.

Post-calibration of large language model confidence scoring via combined techniques

2023

2023-08-16

US

US18/450,700

patent/US12032919B1/en

active

Active

2024

2024-05-29

US

US18/677,561

patent/US12236201B1/en

active

Active

Patent Citations (2)

* Cited by examiner, † Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US20120109652A1

( en )

*

2010-10-27

2012-05-03

Microsoft Corporation

Leveraging Interaction Context to Improve Recognition Confidence Scores

US20220351088A1

( en )

*

2021-04-30

2022-11-03

Intuit Inc.

Machine learning model-agnostic confidence calibration system and method

Non-Patent Citations (7)

* Cited by examiner, † Cited by third party

Title

Desai, Shrey, " Calibration of Pre-trained Transformers ", arXiv:2003.07892v3 [cs.CL], (Oct. 15, 2020), 8 pages.

Jiang, Zhengbao, " How Can We Know What Language Models Know ", arXiv:1911.12543v2 [cs. CL], (May 3, 2020), 16 pages.

Kadavath, Saurav, " Language Models (Mostly) Know What They Know ", arXiv:2207.05221v4 [cs. CL], (Nov. 21, 2022), 43 pages.

Openai, " GPT-4 ", [Online]. Retrieved from the Internet: https: openai.com research gpt-4, (Accessed online Jan. 25, 2024), 31 pages.

Saha, Amrita, " DuoRC: Towards Complex Language Understanding with Paraphrased Reading Comprehension ", arXiv:1804.07927v4 [cs.CL], (Oct. 10, 2018), 16 pages.

Si, Chenglei, " Prompting GPT-3 to be Reliable ", arXiv:2210.09150v2 [cs.CL] Feb. 2, 2015, (Feb. 15, 2023), 24 pages.

Ye, Xi, " Can Explanations Be Useful for Calibrating Black Box Models ", arXiv:2110.07586v2 [cs. CL], (Mar. 15, 2022), 14 pages.

Cited By (11)

* Cited by examiner, † Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US12236201B1

( en )

2023-08-16

2025-02-25

Snowflake Inc.

Enhanced machine learning model accuracy through post-hoc confidence score calibration

CN119299130A

( en )

*

2024-09-09

2025-01-10

四川大学

Method and system for automatically generating network threat traffic detection rules based on LLM

US12367223B1

( en )

*

2024-11-18

2025-07-22

Distributed Analytics Solutions, Ltd.

Calibrated large language models for multi-label classifications

US12499134B1

( en )

2024-11-18

2025-12-16

Distributed Analytics Solutions, Ltd.

Reliable outputs from large language models for multi-label classification tasks

US12579364B1

( en )

*

2024-12-11

2026-03-17

Theodora Lab AI LLC

Machine learning techniques for detecting, measuring, and mitigating bias within textual content

US20260170265A1

( en )

*

2024-12-16

2026-06-18

Paypal, Inc.

Large Language Model (LLM) Token Truncation

CN119402935A

( en )

*

2025-01-03

2025-02-07

深圳市三德大康电子有限公司

Bluetooth headset automatic pairing method and device for multi-terminal switching

CN119587047A

( en )

*

2025-01-16

2025-03-11

宁夏祥瑞智能科技有限公司

A lock-free loop-based real-time streaming system for electroencephalography

CN121524692A

( en )

*

2025-11-21

2026-02-13

东软集团股份有限公司

A method, apparatus, processor, and device for optimizing entity clustering models.

CN121636662A

( en )

*

2025-12-03

2026-03-10

国电南瑞科技股份有限公司

A knowledge graph query method and apparatus based on the fusion of results from multiple language models

CN121384852A

( en )

*

2025-12-23

2026-01-23

大连计量检验检测研究院有限公司

A household gas detection device

Also Published As

Publication number

Publication date

US12236201B1

( en )

2025-02-25

US20250061287A1

( en )

2025-02-20

Similar Documents

Publication

Publication Date

Title

US12236201B1

( en )

2025-02-25

Enhanced machine learning model accuracy through post-hoc confidence score calibration

Karthik et al.

2022

Credit card fraud detection by modelling behaviour pattern using hybrid ensemble model

US11487941B2

( en )

2022-11-01

Techniques for determining categorized text

CA2915383C

( en )

2023-05-02

Automated system for generative multimodel multiclass classification and similarity analysis using machine learning

Gan et al.

2022

A convolutional neural network intrusion detection method based on data imbalance: B. Gan et al.

US11790036B2

( en )

2023-10-17

Bias mitigating machine learning training system

US20250335438A1

( en )

2025-10-30

Information retrieval through query history insight

US20230297554A1

( en )

2023-09-21

Encoder-decoder transformer for table generation

US12314418B2

( en )

2025-05-27

Differentially private security system using gaussian noise and dynamic step size

US20250045634A1

( en )

2025-02-06

Data table row to vector embedding

US11531845B1

( en )

2022-12-20

Bias mitigating machine learning training system

JP2025531153A

( en )

2025-09-19

Automatic Query Selectivity Prediction Using Query Graphs

US12572745B1

( en )

2026-03-10

Suffix-based speculative token decoding for artificial intelligence model

US11436444B1

( en )

2022-09-06

Bias mitigating machine learning training system

Guimaraes et al.

2022

A threat monitoring system for intelligent data analytics of network traffic

US12499222B2

( en )

2025-12-16

Systems and methods for machine interpretation of security data via dynamic constraint specification matrix

US20260044539A1

( en )

2026-02-12

Automated completion of questionnaires based on unstructured database objects

US20260072958A1

( en )

2026-03-12

Ai data connectivity for unstructured data repositories

Han et al.

2025

A review of meta-heuristic high utility patterns mining methods

Raeisi et al.

2025

An entropy-based multi-objective feature selection method for network intrusion detection

US20260133964A1

( en )

2026-05-14

Llm-generated text-to-sql verification via directional graphs

WO2026005846A1

( en )

2026-01-02

Iterative data processing optimization engine in a data intelligence system

US20220027400A1

( en )

2022-01-27

Techniques for information ranking and retrieval

Qasim et al.

2026

A Comparative Evaluation of Machine Learning and Deep Learning Models for Healthcare Ransomware Prediction: Architecture Alignment, Feature Importance, and Deployment Strategy

US20260017367A1

( en )

2026-01-15

Systems and methods for artificial intelligence-based cybersecurity threat intelligence

Legal Events

Date

Code

Title

Description

2023-08-16

FEPP

Fee payment procedure

Free format text : ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

2024-06-19

STCF

Information on status: patent grant

Free format text : PATENTED CASE

Related documents

• TRACE: Single-Pass Decoding-Trace Risk Localization for Generation Calibration#1129977
• ProbPlug: A Plugin Uncertainty Network for Reliable Confidence in LLM Binary Classification#997035
• How to Estimate Whether You Have Found Several Needles in a Haystack: Measuring Calibration in Multi-Label Text Classification#1062359
• A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection#129249
• Post-Training Language Models for Crosslingual Consistency#987605