ABSTRACT
Abstract
One embodiment of a method for correcting a response generated by a machine learning model includes receiving the response from the machine learning model, where the response is generated by the machine learning model based on a request and a context, determining a plurality of portions of the context that are similar to one or more portions of the response, for each portion of the context included in the plurality of portions of the context, determining whether the portion of the context supports at least one portion of the response, and performing one or more operations to generate a corrected response based on the response and whether each portion of the context included in the plurality of portions of the context supports at least one portion of the response.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
This application claims priority benefit of the U.S. Provisional Patent Application titled, âTECHNIQUES FOR GENERATING AND CORRECTING LANGUAGE MODEL OUTPUTS,â filed on Mar. 31, 2023, and having Ser. No. 63/493,693. The subject matter of this related application is hereby incorporated herein by reference.
BACKGROUND
Field of the Various Embodiments
The embodiments of the present disclosure relate generally to the fields of computer science, machine learning, and artificial intelligence, and more specifically, to techniques for generating and correcting language model outputs.
Description of the Related Art
Language models have become increasingly capable of performing various natural language processing tasks, such as question answering, sentiment analysis, and entity recognition. Large language models (LLMs) are one type of language model. Conventionally, an LLM is implemented as a neural network that includes a large number (e.g., billions) of parameters and is trained on a large quantity of text data. Once trained, an LLM can oftentimes perform a wide variety of natural language processing tasks, as opposed to only a specific task.
One drawback of conventional language models, and conventional LLMs in particular, is that these models sometimes generate outputs that are incorrect and/or incomplete. Incorrect outputs by LLMs are also referred to as âhallucinations.â For example, when prompted to respond to a request using information from text data that is provided as context, a conventional LLM can sometimes generate a response that contradicts the information in the context or does not include all of the relevant information in the context. As another example, a conventional LLM can sometimes not fully understand knowledge and terminology that is specific to a domain, and the LLM can respond to a request incorrectly as a result.
As the foregoing illustrates, what is needed in the art are more effective techniques for generating outputs using language models.
SUMMARY
One embodiment of the present disclosure sets forth a computer-implemented method for correcting a response generated by a first machine learning model. The method includes receiving the response from the first machine learning model, where the response is generated by the first machine learning model based on a request and a context. The method further includes determining a plurality of portions of the context that are similar to one or more portions of the response. The method also includes, for each portion of the context included in the plurality of portions of the context, determining whether the portion of the context supports at least one portion of the response. In addition, the method includes performing one or more operations to generate a corrected response based on the response and whether each portion of the context included in the plurality of portions of the context supports at least one portion of the response.
Other embodiments of the present disclosure include, without limitation, one or more computer-readable media including instructions for performing one or more aspects of the disclosed techniques as well as one or more computing systems for performing one or more aspects of the disclosed techniques.
One technical advantage of the disclosed techniques relative to the prior art is that the disclosed techniques can identify that a response generated by a language model is hallucinatory by being false and/or incorrect when compared to portions of a context used to generate the response. A hallucinatory response can be modified to be more accurate and/or to include more supporting information from the context. In addition, the disclosed techniques can identify a response as being incomplete when not all relevant portions of a context are included in the response. An incomplete response can be modified to include additional portions of the context that are relevant to a user request. These technical advantages provide one or more technological improvements over prior art approaches.
BRIEF DESCRIPTION OF THE DRAWINGS
So that the manner in which the above recited features of the various embodiments can be understood in detail, a more particular description of the inventive concepts, briefly summarized above, can be found by reference to various embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of the inventive concepts and are therefore not to be considered limiting of scope in any way, and that there are other equally effective embodiments.
FIG. 1 illustrates a computing device configured to implement one or more aspects of various embodiments;
FIG. 2 is a more detailed illustration of the natural language application of FIG. 1 , according to various embodiments;
FIG. 3 is a more detailed illustration of the natural language application of FIG. 1 , according to various other embodiments;
FIG. 4 is a flow diagram of method steps for generating a response to a user request using a language model, according to various embodiments;
FIG. 5 is a flow diagram of method steps for detecting and correcting an incorrect and/or incomplete response by a language model, according to various other embodiments; and
FIG. 6 is a flow diagram of method steps for detecting and correcting an incorrect and/or incomplete response by a language model, according to various other embodiments.
DETAILED DESCRIPTION
In the following description, numerous specific details are set forth to provide a more thorough understanding of the various embodiments. However, it will be apparent to one skilled in the art that the inventive concepts can be practiced without one or more of these specific details.
System Overview
FIG. 1 illustrates a computing device 100 configured to implement one or more aspects of various embodiments. In some embodiments, computing device 100 can be a desktop computer, a laptop computer, a smart phone, a personal digital assistant (PDA), tablet computer, server machine, or any other type of computing device configured to receive input, process data, and optionally display images, and is suitable for practicing one or more embodiments of the present disclosure. As shown, computing device 100 is configured to run a natural language application 120 and a database management system 118 that resides in a memory 116 of computing device 100 .
It is noted that computing device 100 described herein is illustrative and that any other technically feasible configurations fall within the scope of the present disclosure. For example, multiple instances of natural language application 120 and/or database management system 118 could execute on a set of nodes in a data center, cluster, or cloud computing environment to implement the functionality of computing device 100 . In another example, natural language application 120 and/or database management system 118 could be implemented using any number of hardware and/or software components or layers.
As shown, computing device 100 includes, without limitation, an interconnect (bus) 112 that connects one or more processors 102 , an input/output (I/O) device interface 104 coupled to one or more input/output (I/O) devices 108 , memory 116 , a storage 114 that stores a database 115 , and a network interface 106 . Processor(s) 102 may be any suitable processor(s) implemented as a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), an artificial intelligence (AI) accelerator, any other type of processing unit, or a combination of different processing units, such as a CPU configured to operate in conjunction with a GPU. In general, processor(s) 102 can be any technically feasible hardware unit capable of processing data and/or executing software applications. Further, in the context of this disclosure, the computing elements shown in computing device 100 may correspond to a physical computing system (e.g., a system in a data center) or may be a virtual computing instance executing within a computing cloud.
In some embodiments, I/ O devices 108 include devices capable of receiving input, such as a keyboard, a mouse, a touchpad, and/or a microphone, as well as devices capable of providing output, such as a display device and/or speaker. Additionally, I/ O devices 108 can include devices capable of both receiving input and providing output, such as a touchscreen, a universal serial bus (USB) port, and so forth. The I/ O devices 108 can be configured to receive various types of input from an end-user (e.g., a designer) of computing device 100 , and to also provide various types of output to the end-user of computing device 100 , such as displayed digital images or digital videos or text. In some embodiments, one or more of I/ O devices 108 are configured to couple computing device 100 to a network 110 .
In some embodiments, network 110 is any technically feasible type of communications network that allows data to be exchanged between computing device 100 and external entities or devices, such as a web server or another networked computing device. For example, network 110 could include a wide area network (WAN), a local area network (LAN), a wireless (WiFi) network, and/or the Internet, among others.
In some embodiments, storage 114 includes non-volatile storage for applications and data, and may include fixed or removable disk drives, flash memory devices, and CD-ROM, DVD-ROM, Blu-Ray, HD-DVD, or other magnetic, optical, or solid-state storage devices. Natural language application 120 and database management system 118 can be stored in storage 114 and loaded into memory 116 when executed.
In some embodiments, memory 116 includes a random-access memory (RAM) module, a flash memory u
CROSS-REFERENCE TO RELATED APPLICATIONS
This application claims priority benefit of the U.S. Provisional Patent Application titled, âTECHNIQUES FOR GENERATING AND CORRECTING LANGUAGE MODEL OUTPUTS,â filed on Mar. 31, 2023, and having Ser. No. 63/493,693. The subject matter of this related application is hereby incorporated herein by reference.
BACKGROUND
Field of the Various Embodiments
The embodiments of the present disclosure relate generally to the fields of computer science, machine learning, and artificial intelligence, and more specifically, to techniques for generating and correcting language model outputs.
Description of the Related Art
Language models have become increasingly capable of performing various natural language processing tasks, such as question answering, sentiment analysis, and entity recognition. Large language models (LLMs) are one type of language model. Conventionally, an LLM is implemented as a neural network that includes a large number (e.g., billions) of parameters and is trained on a large quantity of text data. Once trained, an LLM can oftentimes perform a wide variety of natural language processing tasks, as opposed to only a specific task.
One drawback of conventional language models, and conventional LLMs in particular, is that these models sometimes generate outputs that are incorrect and/or incomplete. Incorrect outputs by LLMs are also referred to as âhallucinations.â For example, when prompted to respond to a request using information from text data that is provided as context, a conventional LLM can sometimes generate a response that contradicts the information in the context or does not include all of the relevant information in the context. As another example, a conventional LLM can sometimes not fully understand knowledge and terminology that is specific to a domain, and the LLM can respond to a request incorrectly as a result.
As the foregoing illustrates, what is needed in the art are more effective techniques for generating outputs using language models.
SUMMARY
One embodiment of the present disclosure sets forth a computer-implemented method for correcting a response generated by a first machine learning model. The method includes receiving the response from the first machine learning model, where the response is generated by the first machine learning model based on a request and a context. The method further includes determining a plurality of portions of the context that are similar to one or more portions of the response. The method also includes, for each portion of the context included in the plurality of portions of the context, determining whether the portion of the context supports at least one portion of the response. In addition, the method includes performing one or more operations to generate a corrected response based on the response and whether each portion of the context included in the plurality of portions of the context supports at least one portion of the response.
Other embodiments of the present disclosure include, without limitation, one or more computer-readable media including instructions for performing one or more aspects of the disclosed techniques as well as one or more computing systems for performing one or more aspects of the disclosed techniques.
One technical advantage of the disclosed techniques relative to the prior art is that the disclosed techniques can identify that a response generated by a language model is hallucinatory by being false and/or incorrect when compared to portions of a context used to generate the response. A hallucinatory response can be modified to be more accurate and/or to include more supporting information from the context. In addition, the disclosed techniques can identify a response as being incomplete when not all relevant portions of a context are included in the response. An incomplete response can be modified to include additional portions of the context that are relevant to a user request. These technical advantages provide one or more technological improvements over prior art approaches.
BRIEF DESCRIPTION OF THE DRAWINGS
So that the manner in which the above recited features of the various embodiments can be understood in detail, a more particular description of the inventive concepts, briefly summarized above, can be found by reference to various embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of the inventive concepts and are therefore not to be considered limiting of scope in any way, and that there are other equally effective embodiments.
FIG. 1 illustrates a computing device configured to implement one or more aspects of various embodiments;
FIG. 2 is a more detailed illustration of the natural language application of FIG. 1 , according to various embodiments;
FIG. 3 is a more detailed illustration of the natural language application of FIG. 1 , according to various other embodiments;
FIG. 4 is a flow diagram of method steps for generating a response to a user request using a language model, according to various embodiments;
FIG. 5 is a flow diagram of method steps for detecting and correcting an incorrect and/or incomplete response by a language model, according to various other embodiments; and
FIG. 6 is a flow diagram of method steps for detecting and correcting an incorrect and/or incomplete response by a language model, according to various other embodiments.
DETAILED DESCRIPTION
In the following description, numerous specific details are set forth to provide a more thorough understanding of the various embodiments. However, it will be apparent to one skilled in the art that the inventive concepts can be practiced without one or more of these specific details.
System Overview
FIG. 1 illustrates a computing device 100 configured to implement one or more aspects of various embodiments. In some embodiments, computing device 100 can be a desktop computer, a laptop computer, a smart phone, a personal digital assistant (PDA), tablet computer, server machine, or any other type of computing device configured to receive input, process data, and optionally display images, and is suitable for practicing one or more embodiments of the present disclosure. As shown, computing device 100 is configured to run a natural language application 120 and a database management system 118 that resides in a memory 116 of computing device 100 .
It is noted that computing device 100 described herein is illustrative and that any other technically feasible configurations fall within the scope of the present disclosure. For example, multiple instances of natural language application 120 and/or database management system 118 could execute on a set of nodes in a data center, cluster, or cloud computing environment to implement the functionality of computing device 100 . In another example, natural language application 120 and/or database management system 118 could be implemented using any number of hardware and/or software components or layers.
As shown, computing device 100 includes, without limitation, an interconnect (bus) 112 that connects one or more processors 102 , an input/output (I/O) device interface 104 coupled to one or more input/output (I/O) devices 108 , memory 116 , a storage 114 that stores a database 115 , and a network interface 106 . Processor(s) 102 may be any suitable processor(s) implemented as a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), an artificial intelligence (AI) accelerator, any other type of processing unit, or a combination of different processing units, such as a CPU configured to operate in conjunction with a GPU. In general, processor(s) 102 can be any technically feasible hardware unit capable of processing data and/or executing software applications. Further, in the context of this disclosure, the computing elements shown in computing device 100 may correspond to a physical computing system (e.g., a system in a data center) or may be a virtual computing instance executing within a computing cloud.
In some embodiments, I/ O devices 108 include devices capable of receiving input, such as a keyboard, a mouse, a touchpad, and/or a microphone, as well as devices capable of providing output, such as a display device and/or speaker. Additionally, I/ O devices 108 can include devices capable of both receiving input and providing output, such as a touchscreen, a universal serial bus (USB) port, and so forth. The I/ O devices 108 can be configured to receive various types of input from an end-user (e.g., a designer) of computing device 100 , and to also provide various types of output to the end-user of computing device 100 , such as displayed digital images or digital videos or text. In some embodiments, one or more of I/ O devices 108 are configured to couple computing device 100 to a network 110 .
In some embodiments, network 110 is any technically feasible type of communications network that allows data to be exchanged between computing device 100 and external entities or devices, such as a web server or another networked computing device. For example, network 110 could include a wide area network (WAN), a local area network (LAN), a wireless (WiFi) network, and/or the Internet, among others.
In some embodiments, storage 114 includes non-volatile storage for applications and data, and may include fixed or removable disk drives, flash memory devices, and CD-ROM, DVD-ROM, Blu-Ray, HD-DVD, or other magnetic, optical, or solid-state storage devices. Natural language application 120 and database management system 118 can be stored in storage 114 and loaded into memory 116 when executed.
In some embodiments, memory 116 includes a random-access memory (RAM) module, a flash memory unit, or any other type of memory unit or combination thereof. Processor(s) 102 , I/ O device interface 104 , and network interface 106 are configured to read data from and write data to memory 116 . Memory 116 includes various software programs that can be executed by processor(s) 102 and application data associated with said software programs, including natural language application 120 and database management system 118 .
Database management system 118 is configured to store, retrieve, define, and manage data in database 115 . In some embodiments, database management system 118 serves as an interface between database 115 and application programs, such as natural language application 120 . In some embodiments, natural language application 120 is implemented to respond to user requests using a language model and one or more source documents. In such cases, natural language application 120 can identify relevant context information from the source documents, prompt the language model to generate a response using the context information, and progressively correct the response if necessary, as discussed in greater detail below in conjunction with FIGS. 2 - 6 .
Correcting Incorrect and Incomplete Responses of Language Models
FIG. 2 is a more detailed illustration of natural language application 120 of FIG. 1 , according to various embodiments. As shown, in some embodiments, natural language application 120 includes, without limitation, a request processing module 202 , a language model 204 , a prompt generation module 206 , a name entity recognition (NER) model 208 , a coreference resolution module 210 , an embedding module 212 , and a textual entailment model 214 .
Language model 204 is a machine learning model trained to perform one or more natural language processing tasks, such as question answering, summarization, and/or query generation. Any technically feasible language model 204 can be used in some embodiments, such as GPT (Generative Pre-trained Transformer), BERT (Bidirectional Encoder Representations from Transformers), Transformer, or the like. In some embodiments, language model 204 can be a large language model (LLM). Although language model 204 is shown as being included in natural language application 120 , in some embodiments language model 204 can be external to natural language application 120 . For example, in some embodiments, language model 204 can be executed in a cloud computing environment and accessible to natural language application 120 via an application programming interface (API). In the context of language models, prompts are used to guide the language model to generate responses and/or complete tasks. In some embodiments, prompt generation module 206 can, without limitation, generate a prompt that specifies a task to be performed, provides necessary context or background information, and/or controls the format and structure of output by language model 204 . Although one language model 204 is shown for simplicity, in some embodiments, multiple language models can be used, such as different language models that are fine tuned for different tasks. For example, one or more of the prompts described herein can be used to prompt different language models. Prompt generation module 206 communicates with request processing module 202 to receive the task and context required to generate the prompt and inputs the generated prompt into language model 204 or returns the prompt to request processing module 202 for input into language model 204 . After receiving the prompt as input, language model 204 outputs a response (e.g., an answer to a question).
As described, conventional language models, and LLMs in particular, sometimes generate outputs that are incorrect (i.e., âhallucinationsâ) and/or incomplete. In particular, when prompted to generate a response to a user request, such as an answer to a user question, using information from text data that is provided as context, a conventional language model can sometimes generate a response that contradicts the information in the context or does not include all of the relevant information in the context. To address this issue, in response to receiving a request, request processing module 202 identifies text data that is relevant to the request, prompts a language model 204 to generate a response to the request using information from the text data that is provided as context, and corrects the response generated by language model 204 , if necessary.
In response to receiving a user request, request processing module 202 can identify text data that is relevant to the request in any technically feasible manner, such as using a keyword-based search to pull relevant documents from a database (not shown). After identifying relevant text data, request processing module 202 instructs prompt generation module 206 to generate a prompt that includes (1) text asking language model 204 to respond to the request, (2) the text data appended as context, and (3) text instructing language model 204 to only look for the response within the context. Prompt generation module 206 (or request processing module 202 ) then prompts language model 204 to generate a response using the generated prompt.
Oftentimes, responses generated by language model 204 can include inherent ambiguities. For example, pronouns such as âheâ, âsheâ, âitâ can refer to multiple different nouns, and an entity can be described in various ways across a text (e.g., âpresident of the United Statesâ, âheâ). In some embodiments, to address such ambiguities, request processing module 202 can (1) identify entities (e.g., companies, persons, locations, etc.) within a response generated by language model 204 using an NER model 208 , and (2) ask coreference resolution module 210 to perform coreference resolution, which resolves ambiguous references within the response by understanding the relationships between entities in the response. NER model 208 can identify and classify named entities in a text into predefined categories. Named entities are real-world objects, such as persons, organizations, locations, etc. Request processing module 202 provides the response generated by language model 204 to NER model 208 in order to find named entities that can be used by coreference resolution module 210 to resolve ambiguous references in the response. Given the response and name entities as inputs, coreference resolution module 210 generates a resolved response. In some embodiments, NER model 208 and coreference resolution module 210 are used to iteratively process different portions of the response.
In some embodiments, request processing module 202 can use embedding module 212 to generate an embedding that is a compressed representation that captures semantic properties of the response. Embedding module 212 generates a compressed representation of text in the response in which words, phrases, and the entire text are mapped to the embedding, which can be a vector of numbers. Using the embedding, request processing module 202 can perform an embedding search for each portion of the response to identify the top N most similar portions of context, for which embeddings were previously generated, to that portion of the response. For example, in some embodiments, the portions of the response can be sentences from the response, and the portions of the context can be sentences from the context. Any technically feasible similarity metric can be used to identify the top N most similar portions of the context, for example, a Cosine similarity with a value higher than a predefined threshold can be used to identify the N most similar portions of the context whose similarity values are above the threshold. Although described herein primarily with respect to the top N most similar portions of the context for simplicity, it should be noted that if less than N portions of the contexts are within the threshold of similarity, then less than N most similar portions of the context can be used. In some embodiments, embeddings of portions of the context can also be stored in database 115 for improved retrieval time.
In some other embodiments, rather than performing the embedding searches, described above, request processing module 202 can prompt language model 204 to identify the top N most similar portions (e.g., sentences) of the context to each portion (e.g., sentence) of the response.
In some embodiments, to correct a response generated by language model 204 , request processing module 202 (1) instructs prompt generation module 206 to generate a text prompt that is input into language model 204 to augment each portion of the response, (2) computes an entailment score for each portion of the response and each augmented portion of the response that indicates whether the portion of the response or the augmented portion of the response aligns or conflicts with each of the N most similar portions of the context, (3) computes a hallucination score using the computed entailment scores, and (4) modifies the response based on the entailment scores that are computed for portions of response and augmented portions of the response. Textual entailment refers to a relationship between two text fragments indicating whether the truth of one text fragment can be inferred from the other text fragment. Two text fragments can support (i.e., entail) each other, contradict each other, or be neutral. A value can be assigned to indicate the extent of the relationship between two text fragments, which is referred to herein as the entailment score. Textual entailment model 214 is trained to compute an entailment score between a portion of the response or an augmented portion of the response and a portion of the context. For example, if textual entail model 214 generates, for a negated portion in the response and a portion of context, a higher entailment score than textual entail model 214 generates for the portion of response and the portion of context, then the response of the language model 204 conflicts with the portion of context, and vice versa if the original portion of the response is associated with a higher entailment score than the negated portion of the response.
For example, in some embodiments, the augmentation can include negating each portion of the response, such as negating sentences from the response. In some embodiments, request processing module 202 can ask prompt generation module 206 to generate prompts that are input into language model 204 (or another machine learning model that is fine tuned for generating negations) to generate negations of the portions of the response. For example, language model 204 could add the word not and/or any other phrase that generates the opposite meaning of the portions of the response.
If request processing module 202 determines multiple portions of context are relevant to a portion of the response, then the entailment score for each portion of context and augmented portion of response is calculated separately. In addition, one or more portions of context that request processing module 202 uses to determine that a particular portion of the response is hallucinatory can be used to correct the particular portion of the response by, for example, prompting language model 204 to correct the particular portion of the response using the one or more portions of context.
In some embodiments, request processing module 202 also checks whether each portion of the context that is relevant to the user request is within the response to identify any relevant portions of context that are not within the response, indicating that the response is incomplete. In such cases, request processing module 202 can prompt language model 204 to add the portion(s) of context that are relevant to the user request to the response.
The request processing module 202 can modify the response of language model 204 in any suitable manner based on the computed entailment scores for each portion of the context and each augmented portion, and/or based on whether each portion of the context is within the response. For example, in some embodiments, request processing module 202 can compute a total hallucination score by summing the number of portions of the response that are potentially hallucinatory, as indicated by the entailment score for the negation of those portions of the response with respect to any relevant portion of the context being higher than the entailment score for those portion of the response, and dividing by the total number of portions of the response. If the hallucination score is above a threshold, request processing module 202 can prompt language model 204 to modify the potentially hallucinatory portions of the response, thereby generating a new response. The new response can then be processed in the same manner, until a response is generated for which the hallucination score is not above the threshold.
FIG. 3 is a more detailed illustration of natural language application 120 of FIG. 1 , according to various other embodiments. As shown, natural language application 120 includes, without limitation, a request processing module 302 , a language model 304 , and a prompt generation module 306 . Language model 304 and prompt generation module 306 are similar to language model 204 and prompt generation module 206 , respectively, described above in conjunction with FIG. 3 . Although language model 304 is shown as being included in natural language application 120 , in some embodiments language model 304 can be external to natural language application 120 . For example, in some embodiments, language model 304 can be executed in a cloud computing environment and accessible to natural language application 120 via an API. Although one language model 304 is shown for simplicity, in some embodiments, multiple language models can be used, such as different language models that are fine tuned for different tasks.
Similar to request processing module 202 , in response to receiving a user request, request processing module 302 can identify text data that is relevant to the request in any technically feasible manner, such as using a keyword-based search to pull relevant documents from a database (not shown). After identifying relevant text data, request processing module 302 instructs prompt generation module 306 to generate a prompt that includes (1) text asking language model 304 to respond to the request, (2) the text data appended as context, and (3) text instructing language model 304 to only look for the response within the context. Prompt generation module 306 (or request processing module 302 ) then prompts language model 304 to generate a response using the generated prompt.
In some embodiments, for each of a number (M) of portions of the context that are relevant to the response (e.g., that are most similar as determined using an embedding search or by prompting language model 304 to identify the most similar portions of the context), request processing module 302 instructs prompt generation module 306 to generate text prompts to prompt language model 304 multiple (N) (e.g., five) times whether the response agrees or disagrees with that portion of context. In some embodiments, the relevant portions of the context can be chunks from the context of a given length (e.g., a number of tokens). In some embodiments, the prompts can be generated with different parameters, such as different temperature settings for the language model 304 . In some embodiments, the multiple prompts of language model 304 can occur in parallel. The language model 304 then responds the number of times whether the response agrees or disagrees with each portion of the context.
Thereafter, request processing module 302 computes a hallucination score for each relevant portion of the context based on a sum of the number of times that language model 304 determined that the response is hallucinatory with respect to that portion of context. Request processing module 302 then checks whether the hallucination score is above a threshold. For example, if three times out of five times language model 302 determines that a portion of context disagrees with the response, the hallucination score would be sixty percent, which could indicate that the response is hallucinatory with respect to the portion of context if the threshold is fifty percent.
The request processing module 302 can modify the response of language model 304 in any suitable manner based on the computed hallucination score. In some embodiments, request processing module 302 can instruct prompt generation module 306 to generate yet another prompt which includes each portion of the context along with an indication of whether the portion is determined to be hallucinatory or otherwise notable. The prompt further instructs language model 304 to rewrite the response for each hallucinatory portion of the context.
After the response has been rewritten to better agree with each of the portions of the context that the response did not previously agree with, request processing module 302 instructs prompt generation module 306 to generate a prompt to summarize all of the rewritten responses while ensuring that information in the rewritten responses is preserved in the summarized response. Request processing module 302 then prompts language model 304 to check whether the summarized response is hallucinatory with respect to any portions of the context using the same process, described above, and continues to modify the summarized response, if necessary, until a summarized response is generated that is not hallucinatory with respect to any portions of the context.
FIG. 4 is a flow diagram of method steps for generating a response to a user req
CLAIMS
Claims ( 20 )
What is claimed is:
1 . A computer-implemented method for correcting a response generated by a first machine learning model, the method comprising:
receiving the response from the first machine learning model, wherein the response is generated by the first machine learning model based on a request and a context; determining a plurality of portions of the context that are similar to one or more portions of the response; for each portion of the context included in the plurality of portions of the context, determining whether the portion of the context supports at least one portion of the response; and performing one or more operations to generate a corrected response based on the response and whether each portion of the context included in the plurality of portions of the context supports at least one portion of the response.
2 . The computer-implemented method of claim 1 , wherein, for each portion of the context included in the plurality of portions of the context, determining whether the portion of the context supports at least one portion of the response comprises:
prompting a second machine learning model a plurality of times to generate a plurality of determinations of whether the portion of the context supports the at least one portion of the response; and determining whether the portion of the context supports the at least one portion of the response based on the plurality of determinations.
3 . The computer-implemented method of claim 2 , wherein performing one or more operations to generate the corrected response comprises:
for each portion of the context included in the plurality of portions of the context that does not support at least one portion of the response, prompting the second machine learning model to generate an intermediate corrected response based on the portion of the context; and prompting the second machine learning model to generate the corrected response based on the intermediate corrected response generated for each portion of the context included in the plurality of portions of the context that does not support at least one portion of the response.
4 . The computer-implemented method of claim 2 , wherein the first machine learning model is the second machine learning model.
5 . The computer-implemented method of claim 1 , wherein, for each portion of the context included in the plurality of portions of the context, determining whether the portion of the context supports at least one portion of the response comprises:
performing one or more operations to compute a first entailment of the at least one portion of the response by the portion of the context; performing one or more operations to compute a second entailment of a negation of the at least one portion of the response by the portion of the context; and determining whether the portion of the context supports the at least one portion of the response based on the first entailment and the second entailment.
6 . The computer-implemented method of claim 1 , further comprising performing one or more coreference resolution operations on the one or more portions of the response.
7 . The computer-implemented method of claim 1 , wherein the plurality of portions of the context are determined to be similar to the one or more portions of the response based on semantic similarity.
8 . The computer-implemented method of claim 1 , wherein each portion of context included in the plurality of portions of the context is a sentence, and each portion of the response included in the one or more portions of the response is a sentence.
9 . The computer-implemented method of claim 1 , wherein each portion of context included in the plurality of portions of the context includes text of a predefined length, and the one or more portions of the response includes an entirety of the response.
10 . The computer-implemented method of claim 1 , wherein the first machine learning model comprises a large language model (LLM).
11 . One or more non-transitory computer-readable media storing instructions that, when executed by at least one processor, cause the at least one processor to perform the steps of:
receiving a response from a trained first machine learning model, wherein the response is generated by the first machine learning model based on a request and a context; determining a plurality of portions of the context that are similar to one or more portions of the response; for each portion of the context included in the plurality of portions of the context, determining whether the portion of the context supports at least one portion of the response; and performing one or more operations to generate a corrected response based on the response and whether each portion of the context included in the plurality of portions of the context supports at least one portion of the response.
12 . The one or more non-transitory computer-readable media of claim 11 , wherein, for each portion of the context included in the plurality of portions of the context, determining whether the portion of the context supports at least one portion of the response comprises:
prompting a second machine learning model a plurality of times to generate a plurality of determinations of whether the portion of the context supports the at least one portion of the response; and determining whether the portion of the context supports the at least one portion of the response based on the plurality of determinations.
13 . The one or more non-transitory computer-readable media of claim 12 , wherein performing one or more operations to generate the corrected response comprises:
for each portion of the context included in the plurality of portions of the context that does not support at least one portion of the response, prompting the second machine learning model to generate an intermediate corrected response based on the portion of the context; and prompting the second machine learning model to generate the corrected response based on the intermediate corrected response generated for each portion of the context included in the plurality of portions of the context that does not support at least one portion of the response.
14 . The one or more non-transitory computer-readable media of claim 12 , wherein the first machine learning model is the second machine learning model.
15 . The one or more non-transitory computer-readable media of claim 11 , wherein, for each portion of the context included in the plurality of portions of the context, determining whether the portion of the context supports at least one portion of the response comprises:
performing one or more operations to compute a first entailment of the at least one portion of the response by the portion of the context; performing one or more operations to compute a second entailment of a negation of the at least one portion of the response by the portion of the context; and determining whether the portion of the context supports the at least one portion of the response based on the first entailment and the second entailment.
16 . The one or more non-transitory computer-readable media of claim 11 , wherein the instructions, when executed by the at least one processor, further cause the at least one processor to perform the step of performing one or more coreference resolution operations on the one or more portions of the response.
17 . The one or more non-transitory computer-readable media of claim 11 , wherein the instructions, when executed by the at least one processor, further cause the at least one processor to perform the steps of:
determining the context based on the request; and prompting the first machine learning model to generate the response based on the request and the context.
18 . The one or more non-transitory computer-readable media of claim 11 , wherein performing one or more operations to generate the corrected response comprises:
computing a score based on whether each portion of the context included in the plurality of portions of the context supports at least one portion of the response; and appending the score to the response.
19 . The one or more non-transitory computer-readable media of claim 11 , wherein the instructions, when executed by the at least one processor, further cause the at least one processor to perform the steps of:
searching a database based on the request to determine the context, wherein the context includes at least one portion of one or more documents stored in the database; and prompting the first machine learning model to generate the response based on the request and the context.
20 . A system, comprising:
one or more memories storing instructions; and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to:
receive a response from a trained machine learning model, wherein the response is generated by the machine learning model based on a request and a context,
determine a plurality of portions of the context that are similar to one or more portions of the response,
for each portion of the context included in the plurality of portions of the context, determine whether the portion of the context supports at least one portion of the response, and
perform one or more operations to generate a corrected response based on the response and whether each portion of the context included in the plurality of portions of the context supports at least one portion of the response.
US18/621,593
2023-03-31
2024-03-29
Techniques for generating and correcting language model outputs
Pending
US20240330661A1
( en )
Priority Applications (1)
Application Number
Priority Date
Filing Date
Title
US18/621,593
US20240330661A1
( en )
2023-03-31
2024-03-29
Techniques for generating and correcting language model outputs
Applications Claiming Priority (2)
Application Number
Priority Date
Filing Date
Title
US202363493693P
2023-03-31
2023-03-31
US18/621,593
US20240330661A1
( en )
2023-03-31
2024-03-29
Techniques for generating and correcting language model outputs
Publications (1)
Publication Number
Publication Date
US20240330661A1
true
US20240330661A1 ( en )
2024-10-03
Family
ID=92897754
Family Applications (2)
Application Number
Title
Priority Date
Filing Date
US18/621,619
Pending
US20240330279A1
( en )
2023-03-31
2024-03-29
Techniques for generating and correcting database queries using language models
US18/621,593
Pending
US20240330661A1
( en )
2023-03-31
2024-03-29
Techniques for generating and correcting language model outputs
Family Applications Before (1)
Application Number
Title
Priority Date
Filing Date
US18/621,619
Pending
US20240330279A1
( en )
2023-03-31
2024-03-29
Techniques for generating and correcting database queries using language models
Country Status (1)
Country
Link
US
( 2 )
US20240330279A1
( en )
Cited By (5)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20240419912A1
( en )
*
2023-06-13
2024-12-19
Microsoft Technology Licensing, Llc
Detecting hallucination in a language model
US20250103818A1
( en )
*
2023-09-21
2025-03-27
Bitvore Corp.
Hallucination detection as a metric for determining accuracy of results for large language models in machine learning
US20250265253A1
( en )
*
2024-02-15
2025-08-21
Cisco Technology, Inc.
Automatic retrieval augmented generation with expanding context
US20250315618A1
( en )
*
2024-04-08
2025-10-09
Oracle International Corporation
Grounding automatically-generated responses produced by a q&a system
US12613789B2
( en )
*
2023-05-10
2026-04-28
Fivetran Inc.
Artificial intelligence based generation of data connectors
Families Citing this family (12)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US12411841B2
( en )
*
2023-04-27
2025-09-09
Shopify Inc.
Systems and methods for automatically generating source code
US12554709B2
( en )
*
2023-04-28
2026-02-17
Google Llc
In-context text-to-SQL with reduced labeled data
WO2024228909A1
( en )
*
2023-05-03
2024-11-07
Microsoft Technology Licensing, Llc
Collaborative development of machine learning models on specific concepts
US20250036974A1
( en )
*
2023-07-25
2025-01-30
Sap Se
Deployment of machine learning models using large language models and few-shot learning
US12579292B2
( en )
*
2023-11-01
2026-03-17
The Toronto-Dominion Bank
Systems and methods for securing a data stream with attribute-based access control
US12536160B2
( en )
*
2024-01-02
2026-01-27
Wai'i, Inc.
Using a compiler to modify prompts for machine learning models used to generate database queries
US20250272577A1
( en )
*
2024-02-22
2025-08-28
Chemtreat, Inc.
Iterative prompt trainer and report generator
US20260003879A1
( en )
*
2024-06-28
2026-01-01
Goldman Sachs & Co. LLC
Self-healing generative ai/ml pipeline for generating complex data queries leveraging semantic data model
US12436979B1
( en )
*
2024-07-12
2025-10-07
The Toronto-Dominion Bank
Computing systems and methods for query expansion for use in information retrieval
US20260038020A1
( en )
*
2024-07-30
2026-02-05
Microsoft Technology Licensing, Llc
Personalized context-aware digital content recommendations
CN119048029B
( en )
*
2024-10-31
2025-01-28
ä¸å½çµåç³»ç»å·¥ç¨ç¬¬ä¸å»ºè®¾æéå ¬å¸
Cloud metering coding whole-process material management system and method
CN119988420A
( en )
*
2025-01-16
2025-05-13
åæ大å¦
A method combining fine-tuning and prompt for natural language to SQL tasks
Family Cites Families (3)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20240273345A1
( en )
*
2023-02-13
2024-08-15
Jasper AI, Inc.
Automated generative ai module fitting at scale
US12158906B2
( en )
*
2023-03-23
2024-12-03
Shopify Inc.
Systems and methods for generating query responses
US12411841B2
( en )
*
2023-04-27
2025-09-09
Shopify Inc.
Systems and methods for automatically generating source code
2024
2024-03-29
US
US18/621,619
patent/US20240330279A1/en
active
Pending
2024-03-29
US
US18/621,593
patent/US20240330661A1/en
active
Pending
Cited By (5)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US12613789B2
( en )
*
2023-05-10
2026-04-28
Fivetran Inc.
Artificial intelligence based generation of data connectors
US20240419912A1
( en )
*
2023-06-13
2024-12-19
Microsoft Technology Licensing, Llc
Detecting hallucination in a language model
US20250103818A1
( en )
*
2023-09-21
2025-03-27
Bitvore Corp.
Hallucination detection as a metric for determining accuracy of results for large language models in machine learning
US20250265253A1
( en )
*
2024-02-15
2025-08-21
Cisco Technology, Inc.
Automatic retrieval augmented generation with expanding context
US20250315618A1
( en )
*
2024-04-08
2025-10-09
Oracle International Corporation
Grounding automatically-generated responses produced by a q&a system
Also Published As
Publication number
Publication date
US20240330279A1
( en )
2024-10-03
Similar Documents
Publication
Publication Date
Title
US20240330279A1
( en )
2024-10-03
Techniques for generating and correcting database queries using language models
US11222167B2
( en )
2022-01-11
Generating structured text summaries of digital documents using interactive collaboration
US10699080B2
( en )
2020-06-30
Capturing rich response relationships with small-data neural networks
CN111611807B
( en )
2022-12-09
A neural network-based keyword extraction method, device and electronic equipment
US11645314B2
( en )
2023-05-09
Interactive information retrieval using knowledge graphs
US10025819B2
( en )
2018-07-17
Generating a query statement based on unstructured input
JP5936698B2
( en )
2016-06-22
Word semantic relation extraction device
US10372819B2
( en )
2019-08-06
Determining off-topic questions in a question answering system using probabilistic language models
US9805718B2
( en )
2017-10-31
Clarifying natural language input using targeted questions
US9613133B2
( en )
2017-04-04
Context based passage retrieval and scoring in a question answering system
US10430405B2
( en )
2019-10-01
Apply corrections to an ingested corpus
CN102915299A
( en )
2013-02-06
Word segmentation method and device
US20240370779A1
( en )
2024-11-07
Systems and methods for using contrastive pre-training to generate text and code embeddings
US11966704B1
( en )
2024-04-23
Techniques for verifying a veracity of machine learning outputs
CN106469192A
( en )
2017-03-01
Method and device for determining text relevance
JP7369228B2
( en )
2023-10-25
Method, device, electronic device, and storage medium for generating images of user interest
US11443114B1
( en )
2022-09-13
Computing system for entity disambiguation and not-in-list entity detection in a knowledge graph
CN112183114B
( en )
2024-05-14
Model training and semantic integrity recognition method and device
US9633009B2
( en )
2017-04-25
Knowledge-rich automatic term disambiguation
US20250181620A1
( en )
2025-06-05
Fine-grained attribution for document question answering
WO2016191913A1
( en )
2016-12-08
Systems and methods for providing a comment-centered news reader
CN116245139B
( en )
2023-07-07
Training method and device for graph neural network model, event detection method and device
CN112148862A
( en )
2020-12-29
Question intention identification method and device, storage medium and electronic equipment
CN115238077A
( en )
2022-10-25
Artificial intelligence-based text analysis method, device, equipment and storage medium
WO2026016150A1
( en )
2026-01-22
Simultaneous translation from a source language to a target language
Legal Events
Date
Code
Title
Description
2024-03-29
AS
Assignment
Owner name : VIANAI SYSTEMS, INC., CALIFORNIA
Free format text : ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:RAHMAN, WALID;BERGSTEIN, MAXWELL K.;TRUONG, TU;AND OTHERS;SIGNING DATES FROM 20240328 TO 20240329;REEL/FRAME:066954/0233
2024-05-02
STPP
Information on status: patent application and granting procedure in general
Free format text : DOCKETED NEW CASE - READY FOR EXAMINATION