ABSTRACT
Abstract
Techniques are disclosed for fine-tuning a pre-trained machine learning model to be used by a digital assistant for supporting a user's interactions. In one aspect, a method includes accessing a set of training examples, generating a set of synthesized training examples using an iterative process including accessing a dialog script and corresponding prompt template and response template for a predefined scenario, generating one or more prompts based on the dialog script and corresponding prompt template, generating one or more responses associated with each of the one or more prompts based on the dialog script and the response template, and linking each of the responses with the associated prompts to generate one or more synthesized training examples in the set of synthesized training examples. The pre-trained machine learning model is then fine-tuned using the set of training examples and the set of synthesized training examples.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
The present application claims priority and benefit from U.S. Provisional Application No. 63/583,225, filed Sep. 15, 2023, and U.S. Provisional Application No. 63/583,028, filed Sep. 15, 2023, the entire contents of which are incorporated herein by reference for all purposes.
FIELD
The present disclosure relates generally to digital assistants, and more particularly, to techniques for multi-task finetuning (routing and slot-filling) performed by a Large Language Model (LLM) in a digital assistant input pipeline and response generation.
BACKGROUND
Artificial intelligence (AI) has diverse applications, with a notable evolution in the realm of digital assistants or chatbots. Originally, many users sought instant reactions through instant messaging or chat platforms. Organizations, recognizing the potential for engagement, utilized these platforms to interact with entities, such as end users, in real-time conversations.
However, maintaining a live communication channel with entities through human service personnel proved to be costly for organizations. In response to this challenge, digital assistants or chatbots, also known as bots, emerged as a solution to simulate conversations with entities, particularly over the Internet. The bots enabled entities to engage with users through messaging apps they already used or other applications with messaging capabilities.
Initially, traditional chatbots relied on predefined skill or intent models, which required entities to communicate within a fixed set of keywords or commands. Unfortunately, this approach limited an ability of the bot to engage intelligently and contextually in live conversations, hindering its capacity for natural communication. Entities were constrained by having to use specific commands that the bot could understand, often leading to difficulties in conveying intention effectively.
The landscape has since transformed with the integration of Large Language Models (LLMs) into digital assistants or chatbots. LLMs are deep learning algorithms that can perform a variety of natural language processing (NLP) tasks. They use a neural network architecture called a transformer, which can learn from the patterns and structures of natural language and conduct more nuanced and contextually aware conversations for various domains and purposes. This evolution marks a significant shift from rigid keyword-based interactions to a more adaptive and intuitive communication experience compared to traditional chatbots, enhancing the overall capabilities of digital assistants or chatbots in understanding and responding to user queries.
BRIEF SUMMARY
In various embodiments, a computer-implemented method can be used for fine-tuning a pre-trained machine learning model to be used by a digital assistant for supporting a user's interactions. The method can include accessing a set of training examples, wherein each training example of the set of training examples includes a dialog script between a user and a digital assistant, generating a set of synthesized training examples using an iterative process that is performed for each of one or more predefined scenarios, wherein the iterative process includes: (i) accessing a dialog script and corresponding prompt template and response template for a predefined scenario, wherein the prompt template includes prompt placeholders associated with candidate actions, context, and an utterance, and wherein the response template includes response placeholders associated with executable actions; (ii) generating one or more prompts based on the dialog script and corresponding prompt template for the predefined scenario, wherein generating the one or more prompts includes inserting prompt values into the prompt placeholders associated with the candidate actions, the context, and the utterance based on the dialog script for the predefined scenario; (iii) generating one or more responses associated with each of the one or more prompts based on the dialog script and the response template for the predefined scenario, wherein generating the one or more responses includes inserting response values into the response placeholders associated with the executable actions based on the dialog script for the predefined scenario and the associated one or more prompts; and (iv) linking each of the one or more responses with each of the associated one or more prompts to generate one or more synthesized training examples in the set of synthesized training examples. The pre-trained machine learning model is then fine-tuned using the set of training examples and the set of synthesized training examples. The pre-trained machine learning model is configured to learn tasks of action routing and slot-filling for generating an execution plan, wherein the action routing includes identifying one or more of the executable actions from one or more of the candidate actions that are relevant for responding to the utterance based on the context, and slot-filling includes inserting values into argument slots associated with the one or more executable actions based on the context.
In some embodiments, generating the one or more prompts and the one or more responses further comprises selecting, using a random or predefined data split scheme, the prompt values for the prompt placeholders and the response values for the response placeholders based on the dialog script for the predefined scenario, and wherein the random or predefined data split scheme causes the prompt values and the response values to be selected in such a manner that variation within the one or more prompts and the one or more responses is realized in a number of the candidate actions and/or executable actions, type of the candidate actions and/or executable actions, number of tasks within the context, type of tasks within the context, number of argument slots to be filled within the context and/or executable actions, type of argument slots to be filled within the context and/or executable actions, or any combination thereof when the prompt values and the response values are inserted into the prompt placeholders and the response placeholders, respectively.
In some embodiments, the dialog script for the predefined scenario comprises an in-order dialog flow between a user and a digital assistant, an out of order dialog flow between a user and a digital assistant, or at least a portion of a dialog flow between a user and a digital assistant does not logically flow from another portion of the dialog flow.
In some embodiments, the prompt placeholders associated with the candidate actions include one or more argument slots to be filled by the digital assistant, and the response placeholders associated with the executable actions include the one or more argument slots filled with one or more response values.
In some embodiments, the prompt placeholders associated with the context include at least a portion of an execution plan, the execution plan comprises an action including at least one argument slot having missing values, the utterance comprises information for filling in the missing values, the response placeholders associated with the executable actions include the action including the at least one argument slot, and the at least one argument slot is filled in with one or more response values derived from the information in the utterance.
In some embodiments, the fine-tuning includes generating batches of examples selected from the set of training examples and the set of synthesized training examples; and performing an iterative training loop process that includes: inputting examples from the batches into the pre-trained machine learning model; for each batch, computing a loss for the task of action routing; for each batch, computing a loss for the task of slot-filling; and optimizing model parameters based on a combined loss function that takes into account the loss for the task of action routing and the loss for the task of slot-filling.
In some embodiments, the one or more prompts and the one or more responses are generated using a generative artificial intelligence model.
Some embodiments include a system including one or more processors and one or more computer-readable media storing instructions which, when executed by the one or more processors, cause the system to perform part or all of the operations and/or methods disclosed herein.
Some embodiments include one or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, cause a system to perform part or all of the operations and/or methods disclosed herein.
The techniques described above and below may be implemented in a number of ways and in a number of contexts. Several example implementations and contexts are provided with reference to the following figures, as described below in more detail. However, the following implementations and contexts are but a few of many.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a simplified block diagram of a distributed environment incorporating a chatbot system in accordance with various embodiments.
FIG. 2 is an exemplary architecture for an LLM-based digital assistant in accordance with various embodiments.
FIG. 3 is a block diagram of an exemplary machine learning pipeline comprising several subsystems that work together to train, validate, and implement one or more machine learning models to be used in or by a digital assistant in accordance with various embodiments.
FIG. 4 is a block diagram of an exemplary system for fine-tuning large language models in accordance with various embodiments.
FIG. 5 is a flowchart of a process for fine-tuning large language models in accordance with various embodiments.
FIG. 6 is a block diagram illustrating one pattern for implementing a cloud infrastructure as a service system, according to at least one embodiment.
FIG. 7 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system, according to at least one embodiment.
FIG. 8 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system, according to at least one embodiment.
FIG. 9 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system, according to at least one embodiment.
FIG. 10 is a block diagram illustrating an example computer system, according to at least one embodiment.
DETAILED DESCRIPTION
In the following description, for the purposes of explanation, specific details are set forth in order to provide a thorough understanding of certain embodiments. However, it will be apparent that various embodiments may be practiced without these specific details. The figures and description are not intended to be restrictive. The word âexemplaryâ is used herein to mean âserving as an example, instance, or illustration.â Any embodiment or design described herein as âexemplaryâ is not necessarily to be construed as preferred or advantageous over other embodiments or designs.
Introduction
Artificial intelligence techniques have broad applicability. For example, a digital assistant is an artificial intelligence driven interface that helps users accomplish a variety of tasks using natural language conversations. Conventionally, for each digital assistant, a customer may assemble one or more skills that are focused on specific types of tasks, such as tracking inventory, submitting timecards, and creating expense reports. When an end user engages with the digital assistant, the digital assistant evaluates the end user input for the intent of the user and routes the conversation to and from the appropriate skill based on the user's perceived intent. However, there are some disadvantages of traditional intent-based skills including a limited understanding of natural language, inability to handle unknown inputs, limited ability to hold natural conversations off script, and challenges integrating external knowledge.
The advent of large language models (LLMs) like Generative Pretrained Transformer 4 (GPT-4) has propelled the field of digital assistant design to unprecedented levels of sophistication and overcome these disadvantages and others of traditional intent-based skills. An LLM is a neural network that employs a transformer architecture, specifically crafted for processing and generating sequential data, such as text or words in conversations. LLMs undergo training with extensive textual data, gradually honing their ability to generate text that closely mimics human-written or spoken language. While LLMs excel at generalizing to novel scenarios and domains, it is important to note that their output is not guaranteed to be entirely accurate and is some instances they are prone to hallucinations.
Hallucinations refer to instances where the AI generates information that is incorrect, misleading, or fabricated, despite being presented in a confident and plausible manner. These hallucinations pose significant challenges, particularly in various enterprise contexts, as they can lead to the dissemination of inaccurate information, misinterpretation of information, or reliance on non-existent information, ultimately affecting the quality and reliability of responses and decisions. Consequently, it is important to ensure that an LLM adheres to the natural language configuration commands with fidelity without mistake. Additionally, addressing hallucinations requires ongoing refinement of the model, rigorous validation protocols, and continuous monitoring to ensure that the AI's outputs remain accurate and trustworthy. This adds complexity to the deployment and maintenance of LLMs in various enterprise settings where precision and reliability are paramount.
To address these challenges and others, techniques are disclosed herein for fine-tuning LLMs (e.g., further training LLMs on specific tasks) to enhance their understanding of assets that will be used when performing as an agent (including application programming interfaces (APIs)) and to undergo instruction fine-tuning to improve the LLMs' ability to conform to natural language commands. From a capability point of view, this enables an end user to configure various assets such as APIs, Knowledge Documents, and databases (DBs) for a digital assistant and in
CROSS-REFERENCE TO RELATED APPLICATIONS
The present application claims priority and benefit from U.S. Provisional Application No. 63/583,225, filed Sep. 15, 2023, and U.S. Provisional Application No. 63/583,028, filed Sep. 15, 2023, the entire contents of which are incorporated herein by reference for all purposes.
FIELD
The present disclosure relates generally to digital assistants, and more particularly, to techniques for multi-task finetuning (routing and slot-filling) performed by a Large Language Model (LLM) in a digital assistant input pipeline and response generation.
BACKGROUND
Artificial intelligence (AI) has diverse applications, with a notable evolution in the realm of digital assistants or chatbots. Originally, many users sought instant reactions through instant messaging or chat platforms. Organizations, recognizing the potential for engagement, utilized these platforms to interact with entities, such as end users, in real-time conversations.
However, maintaining a live communication channel with entities through human service personnel proved to be costly for organizations. In response to this challenge, digital assistants or chatbots, also known as bots, emerged as a solution to simulate conversations with entities, particularly over the Internet. The bots enabled entities to engage with users through messaging apps they already used or other applications with messaging capabilities.
Initially, traditional chatbots relied on predefined skill or intent models, which required entities to communicate within a fixed set of keywords or commands. Unfortunately, this approach limited an ability of the bot to engage intelligently and contextually in live conversations, hindering its capacity for natural communication. Entities were constrained by having to use specific commands that the bot could understand, often leading to difficulties in conveying intention effectively.
The landscape has since transformed with the integration of Large Language Models (LLMs) into digital assistants or chatbots. LLMs are deep learning algorithms that can perform a variety of natural language processing (NLP) tasks. They use a neural network architecture called a transformer, which can learn from the patterns and structures of natural language and conduct more nuanced and contextually aware conversations for various domains and purposes. This evolution marks a significant shift from rigid keyword-based interactions to a more adaptive and intuitive communication experience compared to traditional chatbots, enhancing the overall capabilities of digital assistants or chatbots in understanding and responding to user queries.
BRIEF SUMMARY
In various embodiments, a computer-implemented method can be used for fine-tuning a pre-trained machine learning model to be used by a digital assistant for supporting a user's interactions. The method can include accessing a set of training examples, wherein each training example of the set of training examples includes a dialog script between a user and a digital assistant, generating a set of synthesized training examples using an iterative process that is performed for each of one or more predefined scenarios, wherein the iterative process includes: (i) accessing a dialog script and corresponding prompt template and response template for a predefined scenario, wherein the prompt template includes prompt placeholders associated with candidate actions, context, and an utterance, and wherein the response template includes response placeholders associated with executable actions; (ii) generating one or more prompts based on the dialog script and corresponding prompt template for the predefined scenario, wherein generating the one or more prompts includes inserting prompt values into the prompt placeholders associated with the candidate actions, the context, and the utterance based on the dialog script for the predefined scenario; (iii) generating one or more responses associated with each of the one or more prompts based on the dialog script and the response template for the predefined scenario, wherein generating the one or more responses includes inserting response values into the response placeholders associated with the executable actions based on the dialog script for the predefined scenario and the associated one or more prompts; and (iv) linking each of the one or more responses with each of the associated one or more prompts to generate one or more synthesized training examples in the set of synthesized training examples. The pre-trained machine learning model is then fine-tuned using the set of training examples and the set of synthesized training examples. The pre-trained machine learning model is configured to learn tasks of action routing and slot-filling for generating an execution plan, wherein the action routing includes identifying one or more of the executable actions from one or more of the candidate actions that are relevant for responding to the utterance based on the context, and slot-filling includes inserting values into argument slots associated with the one or more executable actions based on the context.
In some embodiments, generating the one or more prompts and the one or more responses further comprises selecting, using a random or predefined data split scheme, the prompt values for the prompt placeholders and the response values for the response placeholders based on the dialog script for the predefined scenario, and wherein the random or predefined data split scheme causes the prompt values and the response values to be selected in such a manner that variation within the one or more prompts and the one or more responses is realized in a number of the candidate actions and/or executable actions, type of the candidate actions and/or executable actions, number of tasks within the context, type of tasks within the context, number of argument slots to be filled within the context and/or executable actions, type of argument slots to be filled within the context and/or executable actions, or any combination thereof when the prompt values and the response values are inserted into the prompt placeholders and the response placeholders, respectively.
In some embodiments, the dialog script for the predefined scenario comprises an in-order dialog flow between a user and a digital assistant, an out of order dialog flow between a user and a digital assistant, or at least a portion of a dialog flow between a user and a digital assistant does not logically flow from another portion of the dialog flow.
In some embodiments, the prompt placeholders associated with the candidate actions include one or more argument slots to be filled by the digital assistant, and the response placeholders associated with the executable actions include the one or more argument slots filled with one or more response values.
In some embodiments, the prompt placeholders associated with the context include at least a portion of an execution plan, the execution plan comprises an action including at least one argument slot having missing values, the utterance comprises information for filling in the missing values, the response placeholders associated with the executable actions include the action including the at least one argument slot, and the at least one argument slot is filled in with one or more response values derived from the information in the utterance.
In some embodiments, the fine-tuning includes generating batches of examples selected from the set of training examples and the set of synthesized training examples; and performing an iterative training loop process that includes: inputting examples from the batches into the pre-trained machine learning model; for each batch, computing a loss for the task of action routing; for each batch, computing a loss for the task of slot-filling; and optimizing model parameters based on a combined loss function that takes into account the loss for the task of action routing and the loss for the task of slot-filling.
In some embodiments, the one or more prompts and the one or more responses are generated using a generative artificial intelligence model.
Some embodiments include a system including one or more processors and one or more computer-readable media storing instructions which, when executed by the one or more processors, cause the system to perform part or all of the operations and/or methods disclosed herein.
Some embodiments include one or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, cause a system to perform part or all of the operations and/or methods disclosed herein.
The techniques described above and below may be implemented in a number of ways and in a number of contexts. Several example implementations and contexts are provided with reference to the following figures, as described below in more detail. However, the following implementations and contexts are but a few of many.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a simplified block diagram of a distributed environment incorporating a chatbot system in accordance with various embodiments.
FIG. 2 is an exemplary architecture for an LLM-based digital assistant in accordance with various embodiments.
FIG. 3 is a block diagram of an exemplary machine learning pipeline comprising several subsystems that work together to train, validate, and implement one or more machine learning models to be used in or by a digital assistant in accordance with various embodiments.
FIG. 4 is a block diagram of an exemplary system for fine-tuning large language models in accordance with various embodiments.
FIG. 5 is a flowchart of a process for fine-tuning large language models in accordance with various embodiments.
FIG. 6 is a block diagram illustrating one pattern for implementing a cloud infrastructure as a service system, according to at least one embodiment.
FIG. 7 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system, according to at least one embodiment.
FIG. 8 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system, according to at least one embodiment.
FIG. 9 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system, according to at least one embodiment.
FIG. 10 is a block diagram illustrating an example computer system, according to at least one embodiment.
DETAILED DESCRIPTION
In the following description, for the purposes of explanation, specific details are set forth in order to provide a thorough understanding of certain embodiments. However, it will be apparent that various embodiments may be practiced without these specific details. The figures and description are not intended to be restrictive. The word âexemplaryâ is used herein to mean âserving as an example, instance, or illustration.â Any embodiment or design described herein as âexemplaryâ is not necessarily to be construed as preferred or advantageous over other embodiments or designs.
Introduction
Artificial intelligence techniques have broad applicability. For example, a digital assistant is an artificial intelligence driven interface that helps users accomplish a variety of tasks using natural language conversations. Conventionally, for each digital assistant, a customer may assemble one or more skills that are focused on specific types of tasks, such as tracking inventory, submitting timecards, and creating expense reports. When an end user engages with the digital assistant, the digital assistant evaluates the end user input for the intent of the user and routes the conversation to and from the appropriate skill based on the user's perceived intent. However, there are some disadvantages of traditional intent-based skills including a limited understanding of natural language, inability to handle unknown inputs, limited ability to hold natural conversations off script, and challenges integrating external knowledge.
The advent of large language models (LLMs) like Generative Pretrained Transformer 4 (GPT-4) has propelled the field of digital assistant design to unprecedented levels of sophistication and overcome these disadvantages and others of traditional intent-based skills. An LLM is a neural network that employs a transformer architecture, specifically crafted for processing and generating sequential data, such as text or words in conversations. LLMs undergo training with extensive textual data, gradually honing their ability to generate text that closely mimics human-written or spoken language. While LLMs excel at generalizing to novel scenarios and domains, it is important to note that their output is not guaranteed to be entirely accurate and is some instances they are prone to hallucinations.
Hallucinations refer to instances where the AI generates information that is incorrect, misleading, or fabricated, despite being presented in a confident and plausible manner. These hallucinations pose significant challenges, particularly in various enterprise contexts, as they can lead to the dissemination of inaccurate information, misinterpretation of information, or reliance on non-existent information, ultimately affecting the quality and reliability of responses and decisions. Consequently, it is important to ensure that an LLM adheres to the natural language configuration commands with fidelity without mistake. Additionally, addressing hallucinations requires ongoing refinement of the model, rigorous validation protocols, and continuous monitoring to ensure that the AI's outputs remain accurate and trustworthy. This adds complexity to the deployment and maintenance of LLMs in various enterprise settings where precision and reliability are paramount.
To address these challenges and others, techniques are disclosed herein for fine-tuning LLMs (e.g., further training LLMs on specific tasks) to enhance their understanding of assets that will be used when performing as an agent (including application programming interfaces (APIs)) and to undergo instruction fine-tuning to improve the LLMs' ability to conform to natural language commands. From a capability point of view, this enables an end user to configure various assets such as APIs, Knowledge Documents, and databases (DBs) for a digital assistant and interact with these assets using natural language. Additionally, through fine-tuning, the LLMs are taught to adhere to user guidelines and output formats, such as JavaScript Object Notation (JSON), which can be validated, thus significantly reducing the risk of hallucination.
These techniques address the technical challenges above because by tailoring the LLMs to particular domains or tasks, such as action matching (routing) and slot-filling, the LLMs can learn to recognize and prioritize relevant information, reducing the likelihood of generating erroneous or fabricated content. Fine-tuning involves training the LLMs on specialized datasets that are representative of the specific tasks it will perform, ensuring it understands the context, terminology, and nuances unique to those areas. This process helps in refining the model's parameters and improving its ability to discern between accurate and inaccurate information. Moreover, the fine-tuning allows for the incorporation of domain or task-specific knowledge and validation protocols that can further enhance the model's performance and reliability, ultimately leading to more accurate and trustworthy outputs. In the context of routing and slot-filling, this means more accurate selection and execution of assets such as APIs, Knowledge Documents, and DBs and improved interaction with these assets using natural language, thereby addressing the technical challenges posed by hallucinations. Experiments were run comparing the fine-tuned models versus conventional (non-fine-tuned) models and performance improvement (measured via accuracy in predictions) was demonstrated at between 5% and 25% for the routing task and between 25% and 50% for the slot-filling task.
In various embodiments, a computer-implemented method can be used for fine-tuning a pre-trained machine learning model to be used by a digital assistant for supporting a user's interactions. The method can include accessing a set of training examples, wherein each training example of the set of training examples includes a dialog script between a user and a digital assistant, generating a set of synthesized training examples using an iterative process that is performed for each of one or more predefined scenarios, wherein the iterative process includes: (i) accessing a dialog script and corresponding prompt template and response template for a predefined scenario, wherein the prompt template includes prompt placeholders associated with candidate actions, context, and an utterance, and wherein the response template includes response placeholders associated with executable actions; (ii) generating one or more prompts based on the dialog script and corresponding prompt template for the predefined scenario, wherein generating the one or more prompts includes inserting prompt values into the prompt placeholders associated with the candidate actions, the context, and the utterance based on the dialog script for the predefined scenario; (iii) generating one or more responses associated with each of the one or more prompts based on the dialog script and the response template for the predefined scenario, wherein generating the one or more responses includes inserting response values into the response placeholders associated with the executable actions based on the dialog script for the predefined scenario and the associated one or more prompts; and (iv) linking each of the one or more responses with each of the associated one or more prompts to generate one or more synthesized training examples in the set of synthesized training examples. The pre-trained machine learning model is then fine-tuned using the set of training examples and the set of synthesized training examples. The pre-trained machine learning model is configured to learn tasks of action routing and slot-filling for generating an execution plan, wherein the action routing includes identifying one or more of the executable actions from one or more of the candidate actions that are relevant for responding to the utterance based on the context, and slot-filling includes inserting values into argument slots associated with the one or more executable actions based on the context.
As used herein, when an action is âbased onâ something, this means the action is based at least in part on at least a part of the something. As used herein, the terms âsimilarlyâ, âsubstantially,â âapproximatelyâ and âaboutâ are defined as being largely but not necessarily wholly what is specified (and include wholly what is specified) as understood by one of ordinary skill in the art. In any disclosed embodiment, the term âsimilarlyâ, âsubstantially,â âapproximately,â or âaboutâ may be substituted with âwithin [a percentage] ofâ what is specified, where the percentage includes 0.1, 1, 5, and 10 percent.
Digital Assistant and Knowledge Dialog
A bot (also referred to as an agent, chatbot, chatterbot, or talkbot), implemented as part of or as a digital assistant, is a computer program that can perform conversations with end users. The bot can generally respond to natural-language messages (e.g., questions or comments) through a messaging application that uses natural-language messages. Enterprises may use one or more bot systems to communicate with end users through a messaging application. The messaging application, which may be referred to as a channel, may be an end user preferred messaging application that the end user has already installed and familiar with. Thus, the end user does not need to download and install new applications in order to chat with the bot system. The messaging application may include, for example, over-the-top (OTT) messaging channels (such as Facebook Messenger, Facebook WhatsApp, WeChat, Line, Kik, Telegram, Talk, Skype, Slack, or SMS), virtual private assistants (such as Amazon Dot, Echo, or Show, Google Home, Apple HomePod, etc.), mobile, web, and cloud application extensions or plugins that extend native or hybrid/responsive mobile, web, or cloud applications with chat capabilities, or voice based input (such as devices or apps with interfaces that use Siri, Cortana, Google Voice, or other speech input for interaction).
In some examples, a bot system may be associated with a Uniform Resource Identifier (URI). The URI may identify the bot system using a string of characters. The URI may be used as a webhook for one or more messaging application systems. The URI may include, for example, a Uniform Resource Locator (URL) or a Uniform Resource Name (URN). The bot system may be designed to receive a message (e.g., a hypertext transfer protocol (HTTP) post call message) from a messaging application system. The HTTP post call message may be directed to the URI from the messaging application system. In some embodiments, the message may be different from a HTTP post call message. For example, the bot system may receive a message from a Short Message Service (SMS). While discussion herein may refer to communications that the bot system receives as a message, it should be understood that the message may be an HTTP post call message, a SMS message, or any other type of communication between two systems.
End users may interact with the bot system through a conversational interaction (sometimes referred to as a conversational user interface (UI)), just as interactions between people. In some cases, the interaction may include the end user saying âHelloâ to the bot and the bot responding with a âHiâ and asking the end user how it can help. In some cases, the interaction may also be a transactional interaction with, for example, a banking bot, such as transferring money from one account to another; an informational interaction with, for example, a HR bot, such as checking for vacation balance; or an interaction with, for example, a retail bot, such as discussing returning purchased goods or seeking technical support.
In some embodiments, the bot system may intelligently handle end user interactions without interaction with an administrator or developer of the bot system. For example, an end user may send one or more messages to the bot system in order to achieve a desired goal. A message may include certain content, such as text, emojis, audio, image, video, or other method of conveying a message. In some embodiments, the bot system may convert the content into a standardized form (e.g., a representational state transfer (REST) or API call against enterprise services with the proper parameters) and generate a natural language response. The bot system may also prompt the end user for additional input parameters or request other additional information. In some embodiments, the bot system may also initiate communication with the end user, rather than passively responding to end user utterances. Described herein are various techniques for identifying an explicit invocation of a bot system and determining an input for the bot system being invoked. In certain embodiments, explicit invocation analysis is performed by a master bot based on detecting an invocation name in an utterance. In response to detection of the invocation name, the utterance may be refined or preprocessed for input to a bot that is identified to be associated with the invocation name and/or communication.
FIG. 1 is a simplified block diagram of an environment 100 incorporating a digital assistant system (also described herein as simply a digital assistant or in more specific terms with reference to implementation of agents as an agent assistant) according to certain embodiments. Environment 100 includes a digital assistant builder platform (DABP) 105 that enables users 110 to create and deploy digital assistant systems 115 . For purposes of this disclosure, a digital assistant is an entity that helps users of the digital assistant accomplish various tasks through natural language conversations. The DABP and digital assistant can be implemented using software only (e.g., the digital assistant is a digital entity implemented using programs, code, or instructions executable by one or more processors), using hardware, or using a combination of hardware and software. In some instances, the environment 100 is part of an Infrastructure as a Service (IaaS) cloud service (as described below in detail) and the DABP and digital assistant can be implemented as part of the IaaS by leveraging the scalable computing resources and storage capabilities provided by the IaaS provider to process and manage large volumes of data and complex computations. This setup allows the DABP and digital assistant to deliver real-time, responsive interactions while ensuring high availability, security, and performance scalability to meet varying demand levels. A digital assistant can be embodied or implemented in various physical systems or devices, such as in a computer, a mobile phone, a watch, an appliance, a vehicle, and the like. A digital assistant is also sometimes referred to as a chatbot system. Accordingly, for purposes of this disclosure, the terms digital assistant and chatbot system are interchangeable.
DABP 105 can be used to create one or more digital assistant systems (or DAs). For example, as illustrated in FIG. 1 , user 110 representing a particular enterprise can use DABP 105 to create and deploy a digital assistant 115 A for users of the particular enterprise (e.g., user 125 ). For example, DABP 105 can be used by a bank to create one or more digital assistants for use by the bank's customers, for example to change a 401k contribution, etc. The same DABP 105 platform can be used by multiple enterprises to create digital assistants. As another example, an owner of a restaurant, such as a pizza shop, may use DABP 105 to create and deploy digital assistant 115 B that enables customers of the restaurant to order food (e.g., order pizza).
To create one or more digital assistant systems 115 , the DABP 105 is equipped with a suite of tools 120 , enabling the acquisition of LLMs, agent creation, asset identification, and deployment of digital assistant systems within a service architecture for users via a computing platform such as a cloud computing platform described in detail with respect to FIGS. 6 - 10 . In some instances, the tools 120 can be utilized to access pre-trained and/or fine-tuned LLMs from data repositories or computing systems. The pre-trained LLMs serve as foundational elements, possessing extensive language understanding derived from vast datasets. This capability enables the models to generate coherent responses across various topics, facilitating transfer learning. Pre-trained models offer cost-effectiveness and flexibility, which allows for scalable improvements and continuous pre-training with new data, often establishing benchmarks in Natural Language Processing (NLP) tasks. Conversely, fine-tuned models are specifically trained for tasks or industries (e.g., plan creation utilizing the LLM's in-context learning capability, knowledge or information retrieval on behalf of an agent, response generation for human-like conversation, etc.), enhancing their performance on specific applications and enabling efficient learning from smaller, specialized datasets. Fine-tuning provides advantages such as task specialization, data efficiency, quicker training times, model customization, and resource efficiency. In some embodiments, fine-tuning may be particularly advantageous for niche applications and ongoing enhancement.
In other instances, the tools 120 can be utilized to pre-train and/or fine-tune the LLMs. The tools 120 , or any subset thereof, may be standalone or part of a machine-learning operationalization framework, inclusive of hardware components like processors (e.g., CPU, GPU, TPU, FPGA, or any combination), memory, and storage. This framework operates software or computer program instructions (e.g., TensorFlow, PyTorch, Keras, etc.) to execute arithmetic, logic, input/output commands for training, validating, and deploying machine-learning models in a production environment. In certain instances, the tools 120 implement the training, validating, and deploying of the models using a cloud platform such as Oracle Cloud Infrastructure (OCI). Leveraging a cloud platform can make machine-learning more accessible, flexible, and cost-effective, which can facilitate faster model development and deployment for developers.
The tools 120 further include a prompt-based agent composition unit for creating agents and their associated actions that an end-user can end up invoking. An agent is a container of agent actions and can be part of one or more digital assistants. Each digital assistant may contain one or more agents through a digital assistant relation, which is the intersection entity that links an agent to a digital assistant. The agent and digital assistant are implemented as bot subtypes and may be persisted into an existing BOTS table. This has advantages in terms of reuse of design-time code (e.g., Java code) and UI artifacts.
An agent action is of a specific action type (e.g., knowledge, service or API, LLM, etc.) and contains a description and schema (e.g., JSON schema) which defines the action parameters. The action description and parameters schema are indexed by semantic index and sent to a planner LLM to select the appropriate action(s) to execute. The action parameters are key-value pairs that are input for the action execution. They are derived from the properties in the schema but may also include additional UI/dialog properties that are used for slot-filling dialogs. The actions can be part of one or more classes. For example, some actions may be part of an application event subscription class, which defines an agent action that should be executed when an application event is received. The application event can be received in the form of an update application context command message. An application event property mapping class (part of the application event subscription class) specifically maps the application event payload properties to corresponding agent action parameters. An action can optionally be part of an action group. An action group may be used when importing a plugin manifest, or when importing an external API specification (API spec) such as an Open API spec. An action group is particularly useful when re-importing a plugin or open API spec, so new actions can be added, existing actions can be updated, or actions that are no longer present in the new manifest or Open API spec can be removed. At runtime, an action group may only be used to limit the application context groups that are sent to the LLM as conversation context by looking up the action group name which corresponds to a context group context.
The agents (e.g., 401k Change Contribution Agent) may be primarily defined as a compilation of agent artifacts using natural language within the prompt-based agent composition unit. Users 110 can create functional agents quickly by providing agent artifact information, parameters, and configurations and by pointing to assets. The assets can be or include resources, such as APIs for interfacing with applications, files and/or documents for retrieving knowledge, data stores for interacting with data, and the like, available to the agents for the execution of actions. The assets are imported, and then the users 110 can use natural language again to provide additional API customizations for dialog and routing/reasoning. Most of what an agent does may involve executing actions. An action can be an explicit action that's authored using natural language (similar to creating agent artifactsâe.g., âWhat is the impact of XYZ on my 401k Contribution limit?â action in the below â401k Contribution Agentâ figure) or an implicit action that is created when an asset is imported (automatically imported upon pointing to a given asset based on metadata and/or specifications associated with the assetâe.g., actions created for Change Contribution and Get Contribution API in the below â401k Contribution Agentâ figure). The design time user (e.g., a user 110 of DABP 105 ) can easily create explicit actions. For example, the user 110 can choose the âRich Textâ action type (see Table 1 for a list of exemplary action types) and creates the name artifact âWhat is the impact of XYZ on my 401k Contribution limit?â when the user 110 learns that a new FAQ needs to be added, as it's not currently in the knowledge documents (assets) the agent references (thus was not implicitly added as an action).
TABLE 1
Action Type
Description
1
Prompt
The action is implemented using a prompt to an LLM.
2
Rich Text
The action is implemented using rich text. The most
common use case is FAQs.
3
Flow
The action is implemented using Visual Flow Designer
flow. May be used for complex cases where the
developer is not able to use the out-of-the-box
dialogue and dialog customizations.
There are various ways in which the agents and assets can be associated or added to a digital assistant 115 . In some instances, the agents can be developed by an enterprise and then added to a digital assistant using DABP 105 . In other instances, the agents can be developed and created using DABP 105 and then added to a digital assistant created using DABP 105 . In yet other instances, DABP 105 provides an online digital store (referred to as an âagent storeâ) that offers various pre-created agents directed to a wide range of tasks and actions. The agents offered through the agent store may also expose various cloud services. In order to add the agents to a digital assistant being generated using DABP 105 , a user 110 of DABP 105 can access assets via tools 120 , select specific assets for an agent, initiate a few mock chat conversations with the agent, and indicate that the agent is to be added to the digital assistant created using DABP 105 .
Once deployed in a production environment, such as the architecture described with respect to FIG. 2 , a digital assistant, such as a digital assistant 115 A built using DABP 105 , can be used to perform various tasks via natural language-based conversations between the digital assistant 115 A and its users 125 . As described above, the digital assistant 115 A illustrated in FIG. 1 , can be made available or accessible to its users 125 through a variety of different channels, such as but not limited to, via certain applications, via social media platforms, via various messaging services and applications, and other applications or channels. A single digital assistant can have several channels configured for it so that it can be run on and be accessed by different services simultaneously.
As part of a conversation, a user 125 may provide one or more user inputs 130 to digital assistant 115 A and get responses 135 back from digital assistant 115 A via a user interface element such as a chat window. A conversation can include one or more of user inputs 130 and responses 135 . Via these conversations, a user 125 can request one or more tasks to be performed by the digital assistant 115 A and, in response, the digital assistant 115 A is configured to perform the user-requested tasks and respond with appropriate responses to the user 125 using one or more LLMs 140 . Conversations shown in the chat window can be organized by thread. For example, in some applications, a conversation related to one page of an application should not be mixed with a conversation related to another page of the application. The application and/or the plugins for the application define the thread boundaries (e.g., a set of (nested) plugins can run within their own thread). Effectively, the chat window will only show the history of messages that belong to the same thread. Setting and changing the thread can be performed via the application and/or the plugins using an update application context command message. Additionally or alternatively, the thread can be changed via an execution plan orchestrator when a user query is matched to a plugin semantic action and the plugin runs in a thread different than the current thread. In this case, the planner changes threads, so that any messages sent in response to the action being executed are shown in the correct new thread. Per agent dialog thread, the following information can be maintained by the digital assistant: the application context, the LLM conversation history, the conversation history with the user, and the agent execution context which holds information about the (stacked) execution plan(s) related to this thread.
User inputs 130 are generally in a natural language form and are referred to as utterances, which may also be referred to as prompts, queries, requests, and the like. The user inputs 130 can be in text form, such as when a user types in a sentence, a question, a text fragment, or even a single word and provides it as input to digital assistant 115 A. In some embodiments, a user input 130 can be in audio input or speech form, such as when a user says or speaks something that is provided as input to digital assistant 115 A. The user inputs 130 are typically in a language spoken by the user 125 . For example, the user inputs 130 may be in English, or some other language. When a user input 130 is in speech form, the speech input is converted to text form user input 130 in that particular language and the text utterances are then processed by digital assistant 115 A. Various speech-to-text processing techniques may be used to convert a speech or audio input to a text utterance, which is then processed by digital assistant 115 A. In some embodiments, the speech-to-text conversion may be done by digital assistant 115 A itself. For purposes of this disclosure, it is assumed that the user inputs 130 are text utterances that have been provided directly by a user 125 of digital assistant 115 A or are the results of conversion of input speech utterances to text form. This however is not intended to be limiting or restrictive in any manner.
The user inputs 130 can be used by the digital assistant 115 A to determine a list of candidate agents 145 A-N. The list of candidate agents (e.g., 145 A-N) includes agents configured to perform one or more actions that could potentially facilitate a response 135 to the user input 130 . The list may be determined by running a search, such as a semantic search, on a context and memory store that has one or more indices comprising metadata for all agents 145 available to the digital assistant 115 A. Metadata for the candidate agents 145 A-N in the list of candidate agents is then combined with the user input to construct an input prompt for the one or more LLMs 140 .
<figure-callout id="115A" label="Digital assistant" filenames="US20250094821A1-202503
CLAIMS
Claims ( 20 )
What is claimed is:
1 . A computer-implemented method comprising:
accessing a set of training examples, wherein each training example of the set of training examples comprises a dialog script between a user and a digital assistant; generating a set of synthesized training examples using an iterative process that is performed for each of one or more predefined scenarios, wherein the iterative process comprises;
accessing a dialog script and corresponding prompt template and response template for a predefined scenario, wherein the prompt template comprises prompt placeholders associated with candidate actions, context, and an utterance, and wherein the response template comprises response placeholders associated with executable actions;
generating one or more prompts based on the dialog script and corresponding prompt template for the predefined scenario, wherein generating the one or more prompts comprises inserting prompt values into the prompt placeholders associated with the candidate actions, the context, and the utterance based on the dialog script for the predefined scenario;
generating one or more responses associated with each of the one or more prompts based on the dialog script and the response template for the predefined scenario, wherein generating the one or more responses comprises inserting response values into the response placeholders associated with the executable actions based on the dialog script for the predefined scenario and the associated one or more prompts; and
linking each of the one or more responses with each of the associated one or more prompts to generate one or more synthesized training examples in the set of synthesized training examples; and
fine-tuning, using the set of training examples and the set of synthesized training examples, a pre-trained machine learning model to learn tasks of action routing and slot-filling for generating an execution plan, wherein the action routing comprises identifying one or more of the executable actions from one or more of the candidate actions that are relevant for responding to the utterance based on the context, and slot-filling comprises inserting values into argument slots associated with the one or more executable actions based on the context.
2 . The computer-implemented method of claim 1 , wherein generating the one or more prompts and the one or more responses further comprises selecting, using a random or predefined data split scheme, the prompt values for the prompt placeholders and the response values for the response placeholders based on the dialog script for the predefined scenario, and wherein the random or predefined data split scheme causes the prompt values and the response values to be selected in such a manner that variation within the one or more prompts and the one or more responses is realized in a number of the candidate actions and/or executable actions, type of the candidate actions and/or executable actions, number of tasks within the context, type of tasks within the context, number of argument slots to be filled within the context and/or executable actions, type of argument slots to be filled within the context and/or executable actions, or any combination thereof when the prompt values and the response values are inserted into the prompt placeholders and the response placeholders, respectively.
3 . The computer-implemented method of claim 2 , wherein the dialog script for the predefined scenario comprises an in-order dialog flow between a user and a digital assistant, an out of order dialog flow between a user and a digital assistant, or at least a portion of a dialog flow between a user and a digital assistant does not logically flow from another portion of the dialog flow.
4 . The computer-implemented method of claim 3 , wherein the prompt placeholders associated with the candidate actions include one or more argument slots to be filled by the digital assistant, and the response placeholders associated with the executable actions include the one or more argument slots filled with one or more response values.
5 . The computer-implemented method of claim 3 , wherein the prompt placeholders associated with the context include at least a portion of an execution plan, the execution plan comprises an action including at least one argument slot having missing values, the utterance comprises information for filling in the missing values, the response placeholders associated with the executable actions include the action including the at least one argument slot, and the at least one argument slot is filled in with one or more response values derived from the information in the utterance.
6 . The computer-implemented method of claim 1 , wherein the fine-tuning comprises:
generating batches of examples selected from the set of training examples and the set of synthesized training examples; and performing an iterative training loop process that comprises:
inputting examples from the batches into the pre-trained machine learning model;
for each batch, computing a loss for the task of action routing;
for each batch, computing a loss for the task of slot-filling; and
optimizing model parameters based on a combined loss function that takes into account the loss for the task of action routing and the loss for the task of slot-filling.
7 . The computer-implemented method of claim 1 , wherein the one or more prompts and the one or more responses are generated using a generative artificial intelligence model.
8 . A system comprising:
one or more processors; and one or more computer-readable media storing instructions which, when executed by the one or more processors, cause the system to perform operations comprising:
accessing a set of training examples, wherein each training example of the set of training examples comprises a dialog script between a user and a digital assistant;
generating a set of synthesized training examples using an iterative process that is performed for each of one or more predefined scenarios, wherein the iterative process comprises;
accessing a dialog script and corresponding prompt template and response template for a predefined scenario, wherein the prompt template comprises prompt placeholders associated with candidate actions, context, and an utterance, and wherein the response template comprises response placeholders associated with executable actions;
generating one or more prompts based on the dialog script and corresponding prompt template for the predefined scenario, wherein generating the one or more prompts comprises inserting prompt values into the prompt placeholders associated with the candidate actions, the context, and the utterance based on the dialog script for the predefined scenario;
generating one or more responses associated with each of the one or more prompts based on the dialog script and the response template for the predefined scenario, wherein generating the one or more responses comprises inserting response values into the response placeholders associated with the executable actions based on the dialog script for the predefined scenario and the associated one or more prompts; and
linking each of the one or more responses with each of the associated one or more prompts to generate one or more synthesized training examples in the set of synthesized training examples; and
fine-tuning, using the set of training examples and the set of synthesized training examples, a pre-trained machine learning model to learn tasks of action routing and slot-filling for generating an execution plan, wherein the action routing comprises identifying one or more of the executable actions from one or more of the candidate actions that are relevant for responding to the utterance based on the context, and slot-filling comprises inserting values into argument slots associated with the one or more executable actions based on the context.
9 . The system of claim 8 , wherein generating the one or more prompts and the one or more responses further comprises selecting, using a random or predefined data split scheme, the prompt values for the prompt placeholders and the response values for the response placeholders based on the dialog script for the predefined scenario, and wherein the random or predefined data split scheme causes the prompt values and the response values to be selected in such a manner that variation within the one or more prompts and the one or more responses is realized in a number of the candidate actions and/or executable actions, type of the candidate actions and/or executable actions, number of tasks within the context, type of tasks within the context, number of argument slots to be filled within the context and/or executable actions, type of argument slots to be filled within the context and/or executable actions, or any combination thereof when the prompt values and the response values are inserted into the prompt placeholders and the response placeholders, respectively.
10 . The system of claim 9 , wherein the dialog script for the predefined scenario comprises an in-order dialog flow between a user and a digital assistant, an out of order dialog flow between a user and a digital assistant, or at least a portion of a dialog flow between a user and a digital assistant does not logically flow from another portion of the dialog flow.
11 . The system of claim 10 , wherein the prompt placeholders associated with the candidate actions include one or more argument slots to be filled by the digital assistant, and the response placeholders associated with the executable actions include the one or more argument slots filled with one or more response values.
12 . The system of claim 10 , wherein the prompt placeholders associated with the context include at least a portion of an execution plan, the execution plan comprises an action including at least one argument slot having missing values, the utterance comprises information for filling in the missing values, the response placeholders associated with the executable actions include the action including the at least one argument slot, and the at least one argument slot is filled in with one or more response values derived from the information in the utterance.
13 . The system of claim 8 , wherein the fine-tuning comprises:
generating batches of examples selected from the set of training examples and the set of synthesized training examples; and performing an iterative training loop process that comprises:
inputting examples from the batches into the pre-trained machine learning model;
for each batch, computing a loss for the task of action routing;
for each batch, computing a loss for the task of slot-filling; and
optimizing model parameters based on a combined loss function that takes into account the loss for the task of action routing and the loss for the task of slot-filling.
14 . The system of claim 8 , wherein the one or more prompts and the one or more responses are generated using a generative artificial intelligence model.
15 . One or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:
accessing a set of training examples, wherein each training example of the set of training examples comprises a dialog script between a user and a digital assistant; generating a set of synthesized training examples using an iterative process that is performed for each of one or more predefined scenarios, wherein the iterative process comprises;
accessing a dialog script and corresponding prompt template and response template for a predefined scenario, wherein the prompt template comprises prompt placeholders associated with candidate actions, context, and an utterance, and wherein the response template comprises response placeholders associated with executable actions;
generating one or more prompts based on the dialog script and corresponding prompt template for the predefined scenario, wherein generating the one or more prompts comprises inserting prompt values into the prompt placeholders associated with the candidate actions, the context, and the utterance based on the dialog script for the predefined scenario;
generating one or more responses associated with each of the one or more prompts based on the dialog script and the response template for the predefined scenario, wherein generating the one or more responses comprises inserting response values into the response placeholders associated with the executable actions based on the dialog script for the predefined scenario and the associated one or more prompts; and
linking each of the one or more responses with each of the associated one or more prompts to generate one or more synthesized training examples in the set of synthesized training examples; and
fine-tuning, using the set of training examples and the set of synthesized training examples, a pre-trained machine learning model to learn tasks of action routing and slot-filling for generating an execution plan, wherein the action routing comprises identifying one or more of the executable actions from one or more of the candidate actions that are relevant for responding to the utterance based on the context, and slot-filling comprises inserting values into argument slots associated with the one or more executable actions based on the context.
16 . The one or more non-transitory computer-readable media of claim 15 , wherein generating the one or more prompts and the one or more responses further comprises selecting, using a random or predefined data split scheme, the prompt values for the prompt placeholders and the response values for the response placeholders based on the dialog script for the predefined scenario, and wherein the random or predefined data split scheme causes the prompt values and the response values to be selected in such a manner that variation within the one or more prompts and the one or more responses is realized in a number of the candidate actions and/or executable actions, type of the candidate actions and/or executable actions, number of tasks within the context, type of tasks within the context, number of argument slots to be filled within the context and/or executable actions, type of argument slots to be filled within the context and/or executable actions, or any combination thereof when the prompt values and the response values are inserted into the prompt placeholders and the response placeholders, respectively.
17 . The one or more non-transitory computer-readable media of claim 16 , wherein the dialog script for the predefined scenario comprises an in-order dialog flow between a user and a digital assistant, an out of order dialog flow between a user and a digital assistant, or at least a portion of a dialog flow between a user and a digital assistant does not logically flow from another portion of the dialog flow.
18 . The one or more non-transitory computer-readable media of claim 17 , wherein the prompt placeholders associated with the candidate actions include one or more argument slots to be filled by the digital assistant, and the response placeholders associated with the executable actions include the one or more argument slots filled with one or more response values.
19 . The one or more non-transitory computer-readable media of claim 17 , wherein the prompt placeholders associated with the context include at least a portion of an execution plan, the execution plan comprises an action including at least one argument slot having missing values, the utterance comprises information for filling in the missing values, the response placeholders associated with the executable actions include the action including the at least one argument slot, and the at least one argument slot is filled in with one or more response values derived from the information in the utterance.
20 . The one or more non-transitory computer-readable media of claim 15 , wherein the fine-tuning comprises:
generating batches of examples selected from the set of training examples and the set of synthesized training examples; and performing an iterative training loop process that comprises:
inputting examples from the batches into the pre-trained machine learning model;
for each batch, computing a loss for the task of action routing;
for each batch, computing a loss for the task of slot-filling; and
optimizing model parameters based on a combined loss function that takes into account the loss for the task of action routing and the loss for the task of slot-filling.
US18/885,496
2023-09-15
2024-09-13
Multi-task fine-tuning for planning performed by large language model
Pending
US20250094821A1
( en )
Priority Applications (1)
Application Number
Priority Date
Filing Date
Title
US18/885,496
US20250094821A1
( en )
2023-09-15
2024-09-13
Multi-task fine-tuning for planning performed by large language model
Applications Claiming Priority (3)
Application Number
Priority Date
Filing Date
Title
US202363583028P
2023-09-15
2023-09-15
US202363583225P
2023-09-15
2023-09-15
US18/885,496
US20250094821A1
( en )
2023-09-15
2024-09-13
Multi-task fine-tuning for planning performed by large language model
Publications (1)
Publication Number
Publication Date
US20250094821A1
true
US20250094821A1 ( en )
2025-03-20
Family
ID=94975446
Family Applications (1)
Application Number
Title
Priority Date
Filing Date
US18/885,496
Pending
US20250094821A1
( en )
2023-09-15
2024-09-13
Multi-task fine-tuning for planning performed by large language model
Country Status (1)
Country
Link
US
( 1 )
US20250094821A1
( en )
Cited By (14)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20230055263A1
( en )
*
2021-08-21
2023-02-23
Dynam.Ai, Inc.
Stratification in non-classified heterogeneous object labels
US20250094730A1
( en )
*
2023-09-19
2025-03-20
Kabushiki Kaisha Toshiba
Information processing apparatus, information processing method, and non-transitory computer readable medium
US20250190700A1
( en )
*
2023-12-07
2025-06-12
Microsoft Technology Licensing, Llc
Leveraging generative language models for interactive constraint satisfaction
CN120297322A
( en )
*
2025-06-09
2025-07-11
é¿éäºé£å¤©(æå·)äºè®¡ç®ææ¯æéå ¬å¸
Task processing method, task platform, computing device and computer readable storage medium
US20250307851A1
( en )
*
2024-03-29
2025-10-02
Dell Products L.P.
A method of managing contracts with suppliers of products
US20250307752A1
( en )
*
2024-03-29
2025-10-02
Dell Products L.P.
Question generation to facilitate interpretation of inference model outputs
CN120743796A
( en )
*
2025-09-05
2025-10-03
æææ¯ç½ç»ç§æï¼ä¸æµ·ï¼æéå ¬å¸
Test data preservation method and device based on large model, storage medium and equipment
CN120808112A
( en )
*
2025-07-16
2025-10-17
广ä¸å·¥ä¸å¤§å¦
Road disease intelligent identification and evaluation method based on multi-mode large model and instance segmentation algorithm
US12499387B2
( en )
2024-03-08
2025-12-16
Seekr Technologies Inc.
Question answer (QA) pair generation using agentic workflow system and method that aligns artificial intelligence model with domain-specific principles
CN121235853A
( en )
*
2025-12-02
2025-12-30
å½ç½å±±è¥¿ççµåæéå ¬å¸è¥éæå¡ä¸å¿
Short-term forecasting method, device, and storage medium for different types of user electricity consumption based on LLM decomposition
US20260050752A1
( en )
*
2024-08-15
2026-02-19
Nbcuniversal Media, Llc
System and method for multiple language subtitle synchronization
US12567270B1
( en )
*
2025-08-28
2026-03-03
Squint, Inc.
On-demand state verification using digital image processing and artificial intelligence
US12596739B2
( en )
*
2024-09-10
2026-04-07
Salesforce, Inc.
Post-work summary generation, refinement, and update for field service calls
US20260105277A1
( en )
*
2024-10-16
2026-04-16
Turing Dream SL
Self-learning massive agentic system, apparatus, and method
2024
2024-09-13
US
US18/885,496
patent/US20250094821A1/en
active
Pending
Cited By (15)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20230055263A1
( en )
*
2021-08-21
2023-02-23
Dynam.Ai, Inc.
Stratification in non-classified heterogeneous object labels
US20250094730A1
( en )
*
2023-09-19
2025-03-20
Kabushiki Kaisha Toshiba
Information processing apparatus, information processing method, and non-transitory computer readable medium
US20250190700A1
( en )
*
2023-12-07
2025-06-12
Microsoft Technology Licensing, Llc
Leveraging generative language models for interactive constraint satisfaction
US12579475B2
( en )
*
2024-03-08
2026-03-17
Seekr Technologies Inc.
Artificial intelligence model generated using agentic workflow system and method for artificial intelligence model aligned with domain-specific principles
US12499387B2
( en )
2024-03-08
2025-12-16
Seekr Technologies Inc.
Question answer (QA) pair generation using agentic workflow system and method that aligns artificial intelligence model with domain-specific principles
US20250307851A1
( en )
*
2024-03-29
2025-10-02
Dell Products L.P.
A method of managing contracts with suppliers of products
US20250307752A1
( en )
*
2024-03-29
2025-10-02
Dell Products L.P.
Question generation to facilitate interpretation of inference model outputs
US20260050752A1
( en )
*
2024-08-15
2026-02-19
Nbcuniversal Media, Llc
System and method for multiple language subtitle synchronization
US12596739B2
( en )
*
2024-09-10
2026-04-07
Salesforce, Inc.
Post-work summary generation, refinement, and update for field service calls
US20260105277A1
( en )
*
2024-10-16
2026-04-16
Turing Dream SL
Self-learning massive agentic system, apparatus, and method
CN120297322A
( en )
*
2025-06-09
2025-07-11
é¿éäºé£å¤©(æå·)äºè®¡ç®ææ¯æéå ¬å¸
Task processing method, task platform, computing device and computer readable storage medium
CN120808112A
( en )
*
2025-07-16
2025-10-17
广ä¸å·¥ä¸å¤§å¦
Road disease intelligent identification and evaluation method based on multi-mode large model and instance segmentation algorithm
US12567270B1
( en )
*
2025-08-28
2026-03-03
Squint, Inc.
On-demand state verification using digital image processing and artificial intelligence
CN120743796A
( en )
*
2025-09-05
2025-10-03
æææ¯ç½ç»ç§æï¼ä¸æµ·ï¼æéå ¬å¸
Test data preservation method and device based on large model, storage medium and equipment
CN121235853A
( en )
*
2025-12-02
2025-12-30
å½ç½å±±è¥¿ççµåæéå ¬å¸è¥éæå¡ä¸å¿
Short-term forecasting method, device, and storage medium for different types of user electricity consumption based on LLM decomposition
Similar Documents
Publication
Publication Date
Title
US12430330B2
( en )
2025-09-30
Calibrating confidence scores of a machine learning model trained as a natural language interface
US12299402B2
( en )
2025-05-13
Techniques for out-of-domain (OOD) detection
CN114424185B
( en )
2026-01-02
Stop word data augmentation for natural language processing
US11804219B2
( en )
2023-10-31
Entity level data augmentation in chatbots for robust named entity recognition
CN115398436B
( en )
2025-08-05
Noisy Data Augmentation for Natural Language Processing
JP7843760B2
( en )
2026-04-14
Distance-based logit values for natural language processing
CN116547676B
( en )
2025-10-10
Augmented logit for natural language processing
US12423530B2
( en )
2025-09-23
Training data generation to facilitate fine-tuning embedding models
US12530545B2
( en )
2026-01-20
Data augmentation and batch balancing for training multi-lingual model
JP2023551860A
( en )
2023-12-13
Out-of-domain data augmentation for natural language processing
US20240062108A1
( en )
2024-02-22
Techniques for training and deploying a named entity recognition model
JP2023551322A
( en )
2023-12-07
Keyword data expansion tool for natural language processing
US12541672B2
( en )
2026-02-03
Addressing catastrophic forgetting and over-generalization while training a natural language to a meaning representation language system
CN116490879A
( en )
2023-07-25
Method and system for overprediction in neural networks
US20250094737A1
( en )
2025-03-20
Managing date-time intervals in transforming natural language to a logical form
US12554934B2
( en )
2026-02-17
Multi-task model with context masking
WO2026030330A1
( en )
2026-02-05
Execution and semantic error correction capabilities for natural language to logical form model
US12499385B2
( en )
2025-12-16
Adaptive training data augmentation to facilitate training named entity recognition models
US20260073154A1
( en )
2026-03-12
Plan bank for api chains in routing engine for generative model-based digital assistant
US20260065171A1
( en )
2026-03-05
Adaptive training data augmentation to facilitate training named entity recognition models
US12632786B2
( en )
2026-05-19
Named entity bias detection and mitigation techniques for sentence sentiment analysis
US20260134004A1
( en )
2026-05-14
Automatic Prompt Trainer for Applications Using Large Language Models (LLMs)
Legal Events
Date
Code
Title
Description
2024-10-23
STPP
Information on status: patent application and granting procedure in general
Free format text : DOCKETED NEW CASE - READY FOR EXAMINATION
2024-10-30
AS
Assignment
Owner name : ORACLE INTERNATIONAL CORPORATION, CALIFORNIA
Free format text : ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HETTIGE, BHAGYA GAYATHRI;ABOBAKR, AHMED ATAALLAH ATAALLAH;SRIDHARAN, VANSHIKA;AND OTHERS;SIGNING DATES FROM 20240909 TO 20240916;REEL/FRAME:069075/0361