Large language model training for test case generation — The Toronto-Dominion Bank (US20250077397A1)
Analyze & explore related research in Conceptio
Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.
ABSTRACT
Abstract
An example operation may include one or more of generating a large language model via a user interface, executing the large language model on a repository of software test cases and requirements of the software test cases to train the large language model to understand connections between test case components and test case requirements, receiving a description of features of a software program to be tested, and in response to receiving the description of the features, generating a software test case based on execution of the large language model on the received descriptions, and displaying the software test case via the user interface.
Description
BACKGROUND
Software testing is a process of checking whether an actual software program matches the expected requirements. During testing, a developer may use a testing tool, such as a software application, to evaluate one or more points of interest within the software program being tested. For example, the testing tool can identify gaps, missing requirements, etc., within the source code based on the execution of the software program in contrast to actual requirements, and the developer can submit fixes to repair or correct the defects and bugs. The testing may be repeated until the developer reaches a quality threshold that is acceptable. It has been shown that better-quality software tests shorten testing time. However, such tests require significant manual contribution of a developer(s).
SUMMARY
One example embodiment provides an apparatus that may include a memory, a network interface configured to receive a request to test a software program stored in the memory from a user device, the request comprising a description of requirements of the software program, and a processor coupled to the memory and the network interface, the processor configured to generate a plurality of testing elements based on execution of a generative artificial intelligence (GenAI) model on the description of the requirements of the software program and a repository of test cases, generate a test case for testing the software program where the test case comprises the plurality of testing elements generated by the GenAI model, and store the test case within the memory.
Another example embodiment provides a method that includes one or more of receiving a request to test a software program from a user device, the request comprising a description of requirements of the software program, generating a plurality of testing elements based on execution of a generative artificial intelligence (GenAI) model on the description of the requirements of the software program and a repository of test cases, generating a test case for testing the software program where the test case comprises the plurality of testing elements generated by the GenAI model, and storing the test case within a storage device.
A further example embodiment provides a computer-readable medium comprising instructions, that when read by a processor, cause the processor to perform one or more of receiving a request to test a software program from a user device, the request comprising a description of requirements of the software program, generating a plurality of testing elements based on execution of a generative artificial intelligence (GenAI) model on the description of the requirements of the software program and a repository of test cases, generating a test case for testing the software program where the test case comprises the plurality of testing elements generated by the GenAI model, and storing the test case within a storage device.
A further example embodiment provides an apparatus that may include a memory, and a processor coupled to the memory, the processor configured to receive a description of a plurality of testing elements for testing a software program and storing the software test within the memory, generate an automation script for automating execution of the software test based on execution of a generative artificial intelligence model (GenAI) model on the plurality of testing elements and a repository of automation scripts stored in the memory, attach the automation script to the software test within the memory, and in response to a request to execute the software test, execute the plurality of testing elements within the software test based on the attached automation script.
A further example embodiment provides a method that includes one or more of receiving a description of a plurality of testing elements for testing a software program and storing the software test within a storage device, generating an automation script for automating execution of the software test based on execution of a generative artificial intelligence model (GenAI) model on the plurality of testing elements and a repository of automation scripts, attaching the automation script to the software test within the storage device, and in response to a request to execute the software test, executing the plurality of testing elements within the software test based on the attached automation script.
A further example embodiment provides a computer-readable medium comprising instructions, that when read by a processor, cause the processor to perform one or more of receiving a description of a plurality of testing elements for testing a software program and storing the software test within a storage device, generating an automation script for automating execution of the software test based on execution of a generative artificial intelligence model (GenAI) model on the plurality of testing elements and a repository of automation scripts, attaching the automation script to the software test within the storage device, and in response to a request to execute the software test, executing the plurality of testing elements within the software test based on the attached automation script.
A further example embodiment provides an apparatus that may include a memory, a display, and a processor coupled to the memory and the display, the processor configured to: generate a large language model via a user interface and store the large language model in the memory, execute the large language model on a repository of software test cases and requirements of the software test cases to train the large language model to understand connections between test case components and test case requirements and store the trained large language model in the memory, receive a description of features of a software program to be tested, and in response to receiving the description of the features, generate a software test case based on execution of the large language model on the received descriptions, and display on the display the software test case via a user interface.
A further example embodiment provides a method that includes one or more of generating a large language model via a user interface, executing the large language model on a repository of software test cases and requirements of the software test cases to train the large language model to understand connections between test case components and test case requirements, receiving a description of features of a software program to be tested; and in response to receiving the description of the features, generating a software test case based on execution of the large language model on the received descriptions, and displaying the software test case via the user interface.
A further example embodiment provides a computer-readable medium comprising instructions, that when read by a processor, cause the processor to perform one or more of generating a large language model via a user interface, executing the large language model on a repository of software test cases and requirements of the software test cases to train the large language model to understand connections between test case components and test case requirements, receiving a description of features of a software program to be tested; and in response to receiving the description of the features, generating a software test case based on execution of the large language model on the received descriptions, and displaying the software test case via the user interface.
A further example embodiment provides an apparatus that may include a memory configured to store a log file, a display, and a processor coupled to the memory and the display, the processor configured to execute tests on a software application stored in the memory and running on the apparatus via a test environment of a test platform and log results of the tests in the log file, identify a new feature to be added to the software application based on execution of a machine learning model on logged results of the tests stored in the log file, generate source code for the new feature to be added to the software application based on execution of a generative artificial intelligence (GenAI) model on the new feature and a repository of source code stored in the memory, and display the generated source code on the display via a user interface of the software application.
A further example embodiment provides a method that includes one or more of executing tests on a software application via a test environment of a test platform and logging results of the tests in a log file, identifying a new feature to be added to the software application based on execution of a machine learning model on logged results of the tests stored in the log file, generating source code for the new feature to be added to the software application based on execution of a generative artificial intelligence (GenAI) model on the new feature and a repository of source code, and displaying the generated source code via a user interface of the software application.
A further example embodiment provides a computer-readable medium comprising instructions, that when read by a processor, cause the processor to perform one or more of executing tests on a software application via a test environment of a test platform and logging results of the tests in a log file, identifying a new feature to be added to the software application based on execution of a machine learning model on logged results of the tests stored in the log file, generating source code for the new feature to be added to the software application based on execution of a generative artificial intelligence (GenAI) model on the new feature and a repository of source code, and displaying the generated source code via a user interface of the software application.
A further example embodiment provides an apparatus that may include a memory comprising a repository of source code, and a processor configured to monitor communications that occur with user devices over a shared computer network, detect a security threat from the monitored communications, generate a software program to simulate the security threat over the shared computer network based on execution of a generative artificial intelligence (GenAI) model on the repository of source code and a description of the security threat, install the source code for simulating the security threat on a system associated with the computer network, and execute the source code for simulating the security threat via the system.
A further example embodiment provides a method that includes one or more of monitoring communications that occur with user devices over a shared computer network, detecting a security threat from the monitored communications, generating a software program to simulate the security threat over the shared computer network based on execution of a generative artificial intelligence (GenAI) model on a description of the security threat and a repository of source code, installing the source code for simulating the security threat on a system associated with the computer network, and executing the source code for simulating the security threat via the system.
A further example embodiment provides a computer-readable medium comprising instructions, that when read by a processor, cause the processor to perform one or more of monitoring communications that occur with user devices over a shared computer network, detecting a security threat from the monitored communications, generating a software program to simulate the security threat over the shared computer network based on execution of a generative artificial intelligence (GenAI) model on a description of the security threat and a repository of source code, installing the source code for simulating the security threat on a system associated with the computer network, and executing the source code for simulating the security threat via the system.
A further example embodiment provides an apparatus that may include a memory, a display, and a processor coupled to the memory and the display, the processor configured to receive, via a user interface, a request to test a software program that is stored in the memory, read source code of the software program from the memory and identify a vulnerability in the source code based on the reading, generate repair code for fixing the identified vulnerability based on execution of a generative artificial intelligence (GenAI) model on the source code and a repository of repair code stored in the memory which is used to repair previous vulnerabilities, and display on the display information about the repair code via the user interface.
A further example embodiment provides a method that includes one or more of receiving, via a user interface, a request to test a software program, reading source code of the software program and identifying a vulnerability in the source code based on the reading, generating repair code for fixing the identified vulnerability based on execution of a generative artificial intelligence (GenAI) model on the source code and a repository of repair code used to repair previous vulnerabilities, and displaying information about the repair code via the user interface.
A further example embodiment provides a computer-readable medium comprising instructions, that when read by a processor, cause the processor to perform one or more of receiving, via a user interface, a request to test a software program, reading source code of the software program and identifying a vulnerability in the source code based on the reading, generating repair code for fixing the identified vulnerability based on execution of a generative artificial intelligence (GenAI) model on the source code and a repository of repair code used to repair previous vulnerabilities, and displaying information about the repair code via the user interface.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 A is a diagram illustrating a generative artificial intelligence (GenAI) computing environment for generating test cases according to example embodiments.
FIG. 1 B is a diagram illustrating an example of a software test case generated by GenAI according to example embodiments.
FIG. 2 is a diagram illustrating a process of executing a predictive model on input content according to example embodiments.
FIGS. 3 A- 3 C are diagrams illustrating processes for training an artificial intelligence (AI) model according to example embodiments.
FIG. 4 is a diagram illustrating a process of prompting a GenAI model to generate an architecture diagram according to example embodiments.
FIGS. 5 A- 5 D are diagrams illustrating a process of generating a software test according to example embodiments.
FIGS. 6 A- 6 C are diagrams illustrating a process of generating an automation test script according to example embodiments.
FIGS. 7 A- 7 C are diagrams illustrating a process of identifying and testing a software vulnerability according to example embodiments.
FIGS. 8 A- 8 C are diagrams illustrating a process of generating new features for a software application based on software testing according to example embodiments.
<div id="p-0030" num="0029" cl
BACKGROUND
Software testing is a process of checking whether an actual software program matches the expected requirements. During testing, a developer may use a testing tool, such as a software application, to evaluate one or more points of interest within the software program being tested. For example, the testing tool can identify gaps, missing requirements, etc., within the source code based on the execution of the software program in contrast to actual requirements, and the developer can submit fixes to repair or correct the defects and bugs. The testing may be repeated until the developer reaches a quality threshold that is acceptable. It has been shown that better-quality software tests shorten testing time. However, such tests require significant manual contribution of a developer(s).
SUMMARY
One example embodiment provides an apparatus that may include a memory, a network interface configured to receive a request to test a software program stored in the memory from a user device, the request comprising a description of requirements of the software program, and a processor coupled to the memory and the network interface, the processor configured to generate a plurality of testing elements based on execution of a generative artificial intelligence (GenAI) model on the description of the requirements of the software program and a repository of test cases, generate a test case for testing the software program where the test case comprises the plurality of testing elements generated by the GenAI model, and store the test case within the memory.
Another example embodiment provides a method that includes one or more of receiving a request to test a software program from a user device, the request comprising a description of requirements of the software program, generating a plurality of testing elements based on execution of a generative artificial intelligence (GenAI) model on the description of the requirements of the software program and a repository of test cases, generating a test case for testing the software program where the test case comprises the plurality of testing elements generated by the GenAI model, and storing the test case within a storage device.
A further example embodiment provides a computer-readable medium comprising instructions, that when read by a processor, cause the processor to perform one or more of receiving a request to test a software program from a user device, the request comprising a description of requirements of the software program, generating a plurality of testing elements based on execution of a generative artificial intelligence (GenAI) model on the description of the requirements of the software program and a repository of test cases, generating a test case for testing the software program where the test case comprises the plurality of testing elements generated by the GenAI model, and storing the test case within a storage device.
A further example embodiment provides an apparatus that may include a memory, and a processor coupled to the memory, the processor configured to receive a description of a plurality of testing elements for testing a software program and storing the software test within the memory, generate an automation script for automating execution of the software test based on execution of a generative artificial intelligence model (GenAI) model on the plurality of testing elements and a repository of automation scripts stored in the memory, attach the automation script to the software test within the memory, and in response to a request to execute the software test, execute the plurality of testing elements within the software test based on the attached automation script.
A further example embodiment provides a method that includes one or more of receiving a description of a plurality of testing elements for testing a software program and storing the software test within a storage device, generating an automation script for automating execution of the software test based on execution of a generative artificial intelligence model (GenAI) model on the plurality of testing elements and a repository of automation scripts, attaching the automation script to the software test within the storage device, and in response to a request to execute the software test, executing the plurality of testing elements within the software test based on the attached automation script.
A further example embodiment provides a computer-readable medium comprising instructions, that when read by a processor, cause the processor to perform one or more of receiving a description of a plurality of testing elements for testing a software program and storing the software test within a storage device, generating an automation script for automating execution of the software test based on execution of a generative artificial intelligence model (GenAI) model on the plurality of testing elements and a repository of automation scripts, attaching the automation script to the software test within the storage device, and in response to a request to execute the software test, executing the plurality of testing elements within the software test based on the attached automation script.
A further example embodiment provides an apparatus that may include a memory, a display, and a processor coupled to the memory and the display, the processor configured to: generate a large language model via a user interface and store the large language model in the memory, execute the large language model on a repository of software test cases and requirements of the software test cases to train the large language model to understand connections between test case components and test case requirements and store the trained large language model in the memory, receive a description of features of a software program to be tested, and in response to receiving the description of the features, generate a software test case based on execution of the large language model on the received descriptions, and display on the display the software test case via a user interface.
A further example embodiment provides a method that includes one or more of generating a large language model via a user interface, executing the large language model on a repository of software test cases and requirements of the software test cases to train the large language model to understand connections between test case components and test case requirements, receiving a description of features of a software program to be tested; and in response to receiving the description of the features, generating a software test case based on execution of the large language model on the received descriptions, and displaying the software test case via the user interface.
A further example embodiment provides a computer-readable medium comprising instructions, that when read by a processor, cause the processor to perform one or more of generating a large language model via a user interface, executing the large language model on a repository of software test cases and requirements of the software test cases to train the large language model to understand connections between test case components and test case requirements, receiving a description of features of a software program to be tested; and in response to receiving the description of the features, generating a software test case based on execution of the large language model on the received descriptions, and displaying the software test case via the user interface.
A further example embodiment provides an apparatus that may include a memory configured to store a log file, a display, and a processor coupled to the memory and the display, the processor configured to execute tests on a software application stored in the memory and running on the apparatus via a test environment of a test platform and log results of the tests in the log file, identify a new feature to be added to the software application based on execution of a machine learning model on logged results of the tests stored in the log file, generate source code for the new feature to be added to the software application based on execution of a generative artificial intelligence (GenAI) model on the new feature and a repository of source code stored in the memory, and display the generated source code on the display via a user interface of the software application.
A further example embodiment provides a method that includes one or more of executing tests on a software application via a test environment of a test platform and logging results of the tests in a log file, identifying a new feature to be added to the software application based on execution of a machine learning model on logged results of the tests stored in the log file, generating source code for the new feature to be added to the software application based on execution of a generative artificial intelligence (GenAI) model on the new feature and a repository of source code, and displaying the generated source code via a user interface of the software application.
A further example embodiment provides a computer-readable medium comprising instructions, that when read by a processor, cause the processor to perform one or more of executing tests on a software application via a test environment of a test platform and logging results of the tests in a log file, identifying a new feature to be added to the software application based on execution of a machine learning model on logged results of the tests stored in the log file, generating source code for the new feature to be added to the software application based on execution of a generative artificial intelligence (GenAI) model on the new feature and a repository of source code, and displaying the generated source code via a user interface of the software application.
A further example embodiment provides an apparatus that may include a memory comprising a repository of source code, and a processor configured to monitor communications that occur with user devices over a shared computer network, detect a security threat from the monitored communications, generate a software program to simulate the security threat over the shared computer network based on execution of a generative artificial intelligence (GenAI) model on the repository of source code and a description of the security threat, install the source code for simulating the security threat on a system associated with the computer network, and execute the source code for simulating the security threat via the system.
A further example embodiment provides a method that includes one or more of monitoring communications that occur with user devices over a shared computer network, detecting a security threat from the monitored communications, generating a software program to simulate the security threat over the shared computer network based on execution of a generative artificial intelligence (GenAI) model on a description of the security threat and a repository of source code, installing the source code for simulating the security threat on a system associated with the computer network, and executing the source code for simulating the security threat via the system.
A further example embodiment provides a computer-readable medium comprising instructions, that when read by a processor, cause the processor to perform one or more of monitoring communications that occur with user devices over a shared computer network, detecting a security threat from the monitored communications, generating a software program to simulate the security threat over the shared computer network based on execution of a generative artificial intelligence (GenAI) model on a description of the security threat and a repository of source code, installing the source code for simulating the security threat on a system associated with the computer network, and executing the source code for simulating the security threat via the system.
A further example embodiment provides an apparatus that may include a memory, a display, and a processor coupled to the memory and the display, the processor configured to receive, via a user interface, a request to test a software program that is stored in the memory, read source code of the software program from the memory and identify a vulnerability in the source code based on the reading, generate repair code for fixing the identified vulnerability based on execution of a generative artificial intelligence (GenAI) model on the source code and a repository of repair code stored in the memory which is used to repair previous vulnerabilities, and display on the display information about the repair code via the user interface.
A further example embodiment provides a method that includes one or more of receiving, via a user interface, a request to test a software program, reading source code of the software program and identifying a vulnerability in the source code based on the reading, generating repair code for fixing the identified vulnerability based on execution of a generative artificial intelligence (GenAI) model on the source code and a repository of repair code used to repair previous vulnerabilities, and displaying information about the repair code via the user interface.
A further example embodiment provides a computer-readable medium comprising instructions, that when read by a processor, cause the processor to perform one or more of receiving, via a user interface, a request to test a software program, reading source code of the software program and identifying a vulnerability in the source code based on the reading, generating repair code for fixing the identified vulnerability based on execution of a generative artificial intelligence (GenAI) model on the source code and a repository of repair code used to repair previous vulnerabilities, and displaying information about the repair code via the user interface.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 A is a diagram illustrating a generative artificial intelligence (GenAI) computing environment for generating test cases according to example embodiments.
FIG. 1 B is a diagram illustrating an example of a software test case generated by GenAI according to example embodiments.
FIG. 2 is a diagram illustrating a process of executing a predictive model on input content according to example embodiments.
FIGS. 3 A- 3 C are diagrams illustrating processes for training an artificial intelligence (AI) model according to example embodiments.
FIG. 4 is a diagram illustrating a process of prompting a GenAI model to generate an architecture diagram according to example embodiments.
FIGS. 5 A- 5 D are diagrams illustrating a process of generating a software test according to example embodiments.
FIGS. 6 A- 6 C are diagrams illustrating a process of generating an automation test script according to example embodiments.
FIGS. 7 A- 7 C are diagrams illustrating a process of identifying and testing a software vulnerability according to example embodiments.
FIGS. 8 A- 8 C are diagrams illustrating a process of generating new features for a software application based on software testing according to example embodiments.
FIGS. 9 A- 9 B are diagrams illustrating a process of detecting a security threat and simulating the security threat on a computer network according to example embodiments.
FIG. 10 A is a diagram illustrating a method of generating a software test case using GenAI according to example embodiments.
FIG. 10 B is a diagram illustrating a method of generating an automation script for executing a software test according to example embodiments.
FIG. 10 C is a diagram illustrating a method of training a large language model to understand software test cases and automation scripts according to example embodiments.
FIG. 10 D is a diagram illustrating a method of identifying a vulnerability in a software program and generating a fix for the vulnerability according to example embodiments.
FIG. 10 E is a diagram illustrating a method of identifying a new feature and generating source code for the new feature according to example embodiments.
FIG. 10 F is a diagram illustrating a method of detecting a security threat to a network and simulating intrusion of the security threat according to example embodiments.
FIG. 11 is a diagram illustrating a computing system that may be used in any of the example embodiments described herein.
DETAILED DESCRIPTION
It is to be understood that although this disclosure includes a detailed description of cloud computing, implementation of the teachings recited herein is not limited to a cloud computing environment. Instead, embodiments of the instant solution can be implemented in conjunction with any other computing environment now known or later developed.
The example embodiments are directed to a platform that can generate/design software tests, automation scripts for testing software, source code, software patches, and the like. In some embodiments, a generative artificial intelligence (GenAI) model may be trained to understand software tests, scripts, source code, and the like based on a large corpus of software tests, best practice documentation, software architecture documentation, and the like. The GenAI model may learn a correlation between text (e.g., words) and software test components, script components, and the like through the training process. Furthermore, the GenAI model may also generate new software tests based on descriptions thereof, such as a description of the requirements of the software test.
The example embodiments enable using GenAI models to enhance the test design process. This innovative approach facilitates requirement elaboration, test scenario design, and automated test script creation. By harnessing the power of generative AI, the example embodiments provide a system that can significantly reduce human intervention and errors while speeding up the test design phase.
Traditionally, a quality assurance engineer is responsible for maintaining the infrastructure needed for test execution, often resulting in a monolithic application. The platform described herein revolutionizes this process by providing a web service/cloud-based service that hosts multiple testing frameworks/programming languages, allowing tests to be executed on-demand (in any of the frameworks/programming languages) without requiring QA engineers to manage the infrastructure. This approach improves test development efficiency and enables seamless AI-powered test design process integration.
By leveraging the power of Generative AI and automation testing as a service, the example embodiments provide a platform that can significantly streamline the QA process, allowing for faster test design and execution. In addition, the system described herein can reduce human intervention within the test design process, thereby reducing the chances for human errors and inconsistencies, leading to more reliable and accurate test results. The platform also offers a scalable solution that can adapt to the ever-changing software development requirements, ensuring that the QA process remains efficient and effective. In addition, by eliminating the need for QA engineers to manage the infrastructure and reducing errors in the test design process, the system offers considerable cost savings for organizations.
According to various embodiments, the GenAI model may be a large language model (LLM), such as a multimodal large language model. As another example, the GenAI model may be a transformer neural network (âtransformerâ) or the like. The GenAI model can understand connections between text and components (e.g., boxes, lines, arrows, software architecture, etc.) within software architecture drawings. For example, the GenAI model may include libraries and deep learning frameworks that enable the GenAI model to create software tests, activation scripts, source code, etc., based on text inputs.
By creating software tests from natural language descriptions, the GenAI model can relieve a user from generating such tests manually. Furthermore, the GenAI model described herein can learn activation scripts for executing the software tests âautomaticallyâ on the software program being tested.
Here, the GenAI model may build a software test and/or an automation script for executing the software test.
Furthermore, the GenAI model described herein can be integrated within a larger artificial intelligence system that includes machine learning and artificial intelligence models, other GenAI models, and the like. This system can perform additional functionality through the testing process/testing environment. For example, new product features can be identified based on the software program attributes being tested. As another example, security vulnerabilities can be detected from the testing, and repairs can be generated and even tested using a combination of models.
FIG. 1 A illustrates a GenAI computing environment 100 that includes a system for generating test cases according to example embodiments. In the example of FIG. 1 A , the system includes a test execution service 130 that hosts a software testing environment and can execute software tests stored in a test repository 132 . In the example embodiments, the test execution service 130 can execute a test in any of multiple different frameworks/different programming languages, including PYTHON®, JAVA®, C++, and the like.
According to various embodiments, the test execution service 130 is coupled to a framework connector 140 (software program), which manages the execution of the software tests via a plurality of
different frameworks
141 , 142 , 143 , and 144 . Each framework may include a collection of tools and software libraries necessary for the framework. Here, the framework connector can connect the test execution service 130 to the proper testing framework (from among the plurality of
different frameworks
141 , 142 , 143 , and 144 ) based on a user request or in an automated way. Each framework may create test scripts by developing required code or running commands in the test environment of the respective framework.
In addition, the system includes a GenAI model 120 trained to generate software test cases from input data. As another example, the GenAI model 120 may generate automation scripts for executing software test cases in an automated manner. As another example, the GenAI model 120 may generate source code such as code that can be used to repair a vulnerability in a software program, code that can be used to simulate a security threat such as a virus, cyber-attack, etc., and the like. The GenAI model 120 may be hosted by a host platform that also hosts the test execution service 130 . Further, the GenAI model 120 may communicate with the test execution service 130 directly via API calls or the like. In this example, the GenAI model 120 may include a data store 122 with a repository of software tests, automation scripts, source code, vulnerability code, software patches, and the like.
The data store 122 may be accessed during the execution of the GenAI model 120 .
In addition, the system includes a user interface 110 that is output by the test execution service 130 . For example, the user interface 110 may be included in a front-end of the software application shown on a user device 102 screen that accesses the host platform and the test execution service via a computer network such as the Internet. The user interface 110 may include controls that can be used to build software tests, perform software testing, develop automation scripts, and the like.
FIG. 1 B illustrates an example of a software test case 150 generated by the GenAI model 120 in FIG. 1 A , according to example embodiments. Referring to FIG. 1 B , the software test case 150 includes a diagram with a plurality of boxes/cells identifying the actions to be taken. For example, a first column 152 in the software test case includes a description of the overall purpose of the test case. In this case, the test case is a test of four separate text boxes on a graphical user interface of a software application that receives four input characters, each corresponding to a 16-digit payment card number.
The test case also includes additional components, including four input activities to be performed during testing, as indicated in the four cells in column 154 . The four input activities describe what will be performed during the testing input. In addition, the four input activities in the column 154 are paired with four expected results in column 156 . In this example, the software test also includes a column 158 for actual results observed during the testing process and a column 160 indicating each input activity's pass/fail status. Here, the four cells in column 158 are to be filled in by the test, and the four cells in column 160 are to be selected after the rest have been performed.
FIG. 2 illustrates a process 200 of executing a model 224 on input content according to example embodiments. For example, the model 224 may be the GenAI model 120 described concerning FIG. 1 A , however, embodiments are not limited to it.
Referring to FIG. 2 , a software application 210 may request execution of the model 224 by submitting a request to the host platform 220 . In response, an AI engine 222 may receive the request and trigger the model 224 to execute within a runtime environment of the host platform 220 .
In FIG. 2 , the AI engine 222 may control access to models stored within the model repository 223 . For example, the models may include GenAI models, AI models, machine learning models, neural networks, and/or the like. The software application 210 may trigger execution of the model 224 from the model repository 223 via submission of a call to an API 221 (application programming interface) of the AI engine 222 . The request may include an identifier of the model 224 , such as a unique ID assigned by the host platform 220 , a payload of data (e.g., to be input to the model during execution), and the like. The AI engine 222 may retrieve the model 224 from the model repository 223 in response, and deploy the model 224 within a live runtime environment. After the model is deployed, the AI engine 222 may execute the running instance of the model 224 on the payload of data, and return a result of the execution to the software application 210 .
In some embodiments, the data payload may be a format that cannot be input to the model 224 nor read by a computer processor. For example, the data payload may be text, image, audio, etc. In response, the AI engine 222 may convert the data payload into a format readable by the model 224 , such as a vector or other encoding.
The vector may then be input to the model 224 .
In some embodiments, the software application 210 may display a user interface that enables a user to provide feedback from the output provided by the model 224 . For example, a user may input a confirmation that the test case generated by a GenAI model is correct or includes incorrect content. This information may be added to the results of execution and stored within a log 225 . The log 225 may include an identifier of the input, an identifier of the output, an identifier of the model used, and feedback from the recipient. This information may be used to subsequently retrain the model.
FIG. 3 A illustrates a process 300 A of training a GenAI model 322 according to example embodiments. However, it should be appreciated that the process 300 A shown in FIG. 3 A also applies to other models, such as machine learning models, AI models, and the like. Referring to FIG. 3 A , a host platform 320 , may host an IDE 310 (integrated development environment) where GenAI models, machine learning models, AI models, and the like may be developed, trained, retrained, and the like. In this example, the IDE 310 may include a software application with a user interface accessible by a user device over a network or through a local connection.
For example, the IDE 310 may be embodied as a web application that can be accessed at a network address, URL, etc., by a device. As another example, the IDE 310 may be locally or remotely installed on a computing device used by a user.
The IDE 310 may be used to design a model (via a user interface of the IDE), such as a generative artificial intelligence model that can receive text as input and generate custom imagery, etc. The model can then be executed/trained based on the training data established via the user interface. For example, the user interface may be used to build a new model. The training data for training such a new model may be provided from a training data store such as a database 324 , which includes training samples from the web, customers, and the like. As another example, the training data may be pulled from one or more external data stores 330 such as publicly available sites, etc.
The GenAI model 322 may be executed on training data via an AI engine 321 of the host platform 320 during training. The training data may include a large corpus of text from software tests, best practice documentation, compliance documentation, etc. In the example embodiments, the training data may include software tests, source code, activation scripts, best practice documentation, compliance documentation, etc. The GenAI model 322 may learn mappings/connections between requirements associated with a software test and the components included within the software test case and can thus create software test cases from a description of the requirements of the software test case. When fully trained, the model may be stored within the model repository 323 via the IDE 310 or the like.
As another example, the IDE 310 may be used to retrain the GenAI model 322 after the model has already been deployed. Here, the training process may use executional results that have already been generated/output by the GenAI model 322 in a live environment (including any customer feedback, etc.) to retrain the GenAI model 322 . For example, predicted outputs/images custom generated by the GenAI model 322 and the user feedback of the images may be used to retrain the model to enhance the images generated for all users. The responses may indicate whether the generated software architecture diagram is correct and, if not, what aspects of the diagram are incorrect. This data may be captured and stored within a runtime log 325 or other datastore within the live environment and can be subsequently used to retrain the GenAI model 322 .
FIG. 3 B illustrates a process 300 B of executing a training process for training/retraining the GenAI model 322 via an AI engine 321 . In this example, an executable script 326 is developed and configured to read data from a database 324 and input the data to the GenAI model 322 while the GenAI model is running/executing via the AI engine 321 .
For example, the executable script 326 may use identifiers of data locations (e.g., table IDs, row IDs, column IDs, topic IDs, object IDs, etc.) to identify locations of the training data within the database 324 and query an API 328 of the database 324 . In response, the database 324 may receive the query, load the requested data, and return it to the AI engine 321 , which is input to the GenAI model 322 . The process may be managed via a user interface of the IDE 310 , which enables a human-in-the-loop during the training process (supervised learning). However, it should also be appreciated that the system is capable of unsupervised learning.
The executable script 326 may iteratively retrieve additional training data sets from the database 324 and iteratively input the additional training data sets into the GenAI model 322 during the execution of the model to continue to train the model. The script may continue until instructions within the script tell the script to terminate, which may be based on a number of iterations (training loops), total time elapsed during the training process, etc.
FIG. 3 C illustrates a process 300 C of designing a new AI model via a user interface 340 according to example embodiments. For example, the user interface 340 may be output as part of the software application that interacts with the IDE 310 shown in FIG. 3 A , however, embodiments are not limited to it. Referring to FIG. 3 C , a user can use an input mechanism to make selections from a menu 342 shown on the left-hand side of the user interface 340 to add pieces to the model such as data components, model components, analysis components, etc., within a workspace 344 of the user interface 340 .
In the example of FIG. 3 C , the menu 342 includes a plurality of graphical user interface (GUI) menu options, which can be selected to drill down into additional components that can be added to the model design shown in the workspace 344 .
Here, the GUI menu options include options for adding features such as neural networks, machine learning models, AI models, data sources, conversion processes (e.g., vectorization, encoding, etc.), analytics, etc. The user can continue to add f
CLAIMS
Claims ( 20 )
What is claimed is:
1 . An apparatus comprising:
a memory; a display; and a processor coupled to the memory and the display, the processor configured to: generate a large language model via a user interface and store the large language model in the memory, execute the large language model on a repository of software test cases and requirements of the software test cases to train the large language model to understand connections between test case components and test case requirements and store the trained large language model in the memory, receive a description of features of a software program to be tested, and in response to receiving the description of the features, generate a software test case based on execution of the large language model on the received descriptions, and display on the display the software test case via the user interface.
2 . The apparatus of claim 1 , wherein the processor is further configured to receive user feedback about the software test case via the user interface, and execute the large language model on the software test case and the feedback to further train the large language model.
3 . The apparatus of claim 1 , wherein the processor is further configured to receive a call via an application programming interface (API) with an identifier of a programming language, and in response, generate the software test case for the software program within the identified programming language.
4 . The apparatus of claim 1 , wherein the processor is further configured to execute the large language model on best practices documents of software test cases to further train the large language model to understand best practices.
5 . The apparatus of claim 1 , wherein the processor is further configured to generate and output a prompt on the user interface during the training, and receive a response to the prompt via the user interface.
6 . The apparatus of claim 5 , wherein the processor is further configured to train the large language model based on the prompt output on the user interface and the response to the prompt received via the user interface.
7 . The apparatus of claim 1 , wherein the processor is configured to generate a plurality of test components of the software test including a test specification, a test execution, a test recording, and a test verification based on execution of a GenAI model.
8 . The apparatus of claim 7 , wherein the processor is further configured to generate a respective requirement for each component within the software test based on execution of the large language model on the received descriptions.
9 . A method comprising:
generating a large language model via a user interface; executing the large language model on a repository of software test cases and requirements of the software test cases to train the large language model to understand connections between test case components and test case requirements; receiving a description of features of a software program to be tested; and in response to receiving the description of the features, generating a software test case based on execution of the large language model on the received descriptions, and displaying the software test case via the user interface.
10 . The method of claim 9 , wherein the method further comprises receiving user feedback about the software test case via the user interface, and executing the large language model on the software test case and the feedback to further train the large language model.
11 . The method of claim 9 , wherein the method further comprises receiving a call via an application programming interface (API) with an identifier of a programming language, and in response, generating the software test case for the software program within the identified programming language.
12 . The method of claim 9 , wherein the executing further comprises executing the large language model on best practices documents of software test cases to further train the large language model to understand best practices.
13 . The method of claim 9 , wherein the method further comprises generating and outputting a prompt on the user interface during the training, and receiving a response to the prompt via the user interface.
14 . The method of claim 13 , wherein the executing further comprises training the large language model based on the prompt output on the user interface and the response to the prompt received via the user interface.
15 . The method of claim 9 , wherein the generating comprises generating a plurality of test components of the software test including a test specification, a test execution, a test recording, and a test verification based on execution of a GenAI model.
16 . The method of claim 15 , wherein the generating further comprises generating a respective requirement for each component within the software test based on execution of the large language model on the received descriptions.
17 . A computer-readable medium comprising instructions stored therein which when executed by a processor cause a computer to perform:
generating a large language model via a user interface; executing the large language model on a repository of software test cases and requirements of the software test cases to train the large language model to understand connections between test case components and test case requirements; receiving a description of features of a software program to be tested; and in response to receiving the description of the features, generating a software test case based on execution of the large language model on the received descriptions, and displaying the software test case via the user interface.
18 . The computer-readable medium of claim 17 , wherein the computer is further configured to perform receiving user feedback about the software test case via the user interface, and executing the large language model on the software test case and the feedback to further train the large language model.
19 . The computer-readable medium of claim 17 , wherein the computer is further configured to perform receiving a call via an application programming interface (API) with an identifier of a programming language, and in response, generating the software test case for the software program within the identified programming language.
20 . The computer-readable medium of claim 17 , wherein the executing further comprises executing the large language model on best practices documents of software test cases to further train the large language model to understand best practices.
US18/462,363
2023-09-06
2023-09-06
Large language model training for test case generation
Pending
US20250077397A1
( en )
Priority Applications (1)
Application Number
Priority Date
Filing Date
Title
US18/462,363
US20250077397A1
( en )
2023-09-06
2023-09-06
Large language model training for test case generation
Applications Claiming Priority (1)
Application Number
Priority Date
Filing Date
Title
US18/462,363
US20250077397A1
( en )
2023-09-06
2023-09-06
Large language model training for test case generation
Publications (1)
Publication Number
Publication Date
US20250077397A1
true
US20250077397A1 ( en )
2025-03-06
Family
ID=94774071
Family Applications (1)
Application Number
Title
Priority Date
Filing Date
US18/462,363
Pending
US20250077397A1
( en )
2023-09-06
2023-09-06
Large language model training for test case generation
Country Status (1)
Country
Link
US
( 1 )
US20250077397A1
( en )
Cited By (27)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20250117595A1
( en )
*
2023-10-05
2025-04-10
The Toronto-Dominion Bank
Dynamic prompting based on conversation context
US20250321874A1
( en )
*
2024-04-12
2025-10-16
L&T Technology Services Limited
Method and system of determining test procedures using large language models
US12499241B2
( en )
2023-09-06
2025-12-16
The Toronto-Dominion Bank
Correcting security vulnerabilities with generative artificial intelligence
US12517812B2
( en )
2023-09-06
2026-01-06
The Toronto-Dominion Bank
Security testing based on generative artificial intelligence
US12536264B2
( en )
2024-07-19
2026-01-27
The Toronto-Dominion Bank
Parallel artificial intelligence driven identity checking with biometric prompting
US20260030145A1
( en )
*
2024-07-29
2026-01-29
Sap Se
Generative ai application artifact management enabling safeguarding by key users
US12541894B2
( en )
2023-08-30
2026-02-03
The Toronto-Dominion Bank
Image modification based on goal progression
US12572451B1
( en )
2023-09-29
2026-03-10
Amazon Technologies, Inc.
Training checkpoint validation for machine learning models
US12585435B2
( en )
2023-08-30
2026-03-24
The Toronto-Dominion Bank
Real-time visualization of complex software architecture
US12592301B2
( en )
2023-08-30
2026-03-31
The Toronto-Dominion Bank
Prompt engineering and generative AI for goal-based imagery
US12614026B1
( en )
*
2023-09-29
2026-04-28
Amazon Technologies, Inc.
Reference and training data validation for machine learning models
US12625680B2
( en )
2023-08-30
2026-05-12
The Toronto-Dominion Bank
Creating a model of software architecture
US20260140852A1
( en )
*
2024-11-19
2026-05-21
International Business Machines Corporation
Large Language Model-Based Test-Case Refinement For Application Programming Interfaces
US12641178B2
( en )
2023-10-04
2026-05-26
The Toronto-Dominion Bank
Real-time dynamic visualization of content
US12646051B2
( en )
2023-09-21
2026-06-02
The Toronto-Dominion Bank
Dynamically evolving image based on pursuit of goals
US12645689B2
( en )
2024-06-14
2026-06-02
The Toronto-Dominion Bank
Weighted retrieval for in-context tabular model
US12645429B2
( en )
2023-08-30
2026-06-02
The Toronto-Dominion Bank
Service design coordination among developers
US12645838B2
( en )
2024-07-18
2026-06-02
The Toronto-Dominion Bank
Limiting activity based on a profile
US12650836B2
( en )
2023-08-31
2026-06-09
The Toronto-Dominion Bank
Gap identification and solution recommendation for complex software architecture
US12657566B2
( en )
2023-10-05
2026-06-16
The Toronto-Dominion Bank
GenAI letter of confirmation of benefits
US12670334B2
( en )
2024-03-25
2026-06-30
The Toronto-Dominion Bank
Chatbot with dynamic conversational tone
US12670640B2
( en )
2023-09-21
2026-06-30
The Toronto-Dominion Bank
Partial exchange of NFT via NFT interaction
US12682179B2
( en )
2024-03-28
2026-07-14
The Toronto-Dominion Bank
Response determination based on contextual attributes and previous conversation content
US12693842B2
( en )
2024-01-12
2026-07-28
The Toronto-Dominion Bank
Feature activation based on alternative feature behavior
US12699556B2
( en )
2023-08-30
2026-08-04
The Toronto-Dominion Bank
Software library usage optimization
US12705398B2
( en )
2024-03-25
2026-08-11
The Toronto-Dominion Bank
Youth application with parental guidance role
US12711683B2
( en )
2024-01-12
2026-08-18
The Toronto-Dominion Bank
Dynamic animation based on waiting period
Citations (11)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20190196949A1
( en )
*
2017-12-27
2019-06-27
Accenture Global Solutions Limited
Test scenario and knowledge graph extractor
US20190213116A1
( en )
*
2018-01-10
2019-07-11
Accenture Global Solutions Limited
Generation of automated testing scripts by converting manual test cases
US20190213115A1
( en )
*
2018-01-08
2019-07-11
Accenture Global Solutions Limited
Utilizing artificial intelligence to test cloud applications
US10949337B1
( en )
*
2019-10-29
2021-03-16
Accenture Global Solutions Limited
Utilizing neural network and artificial intelligence models to select and execute test cases in a software development platform
US20230251960A1
( en )
*
2022-02-04
2023-08-10
Optum, Inc.
Machine learning techniques for automated software testing configuration management
US20240241817A1
( en )
*
2021-05-15
2024-07-18
Telefonaktiebolaget Lm Ericsson (Publ)
Artificial intelligence based cognitive test script generation
US20240256423A1
( en )
*
2023-01-26
2024-08-01
Microsoft Technology Licensing, Llc
Program improvement using large language models
US20240265281A1
( en )
*
2023-02-08
2024-08-08
Continuity AI, LLC
System for providing software related answer based on a trained model
US20240330167A1
( en )
*
2023-03-28
2024-10-03
Tuyen Mathew Duong
Intelligent Process For Iterative Software Testing and Development
US20250045185A1
( en )
*
2023-07-31
2025-02-06
International Business Machines Corporation
Large language models for creating a multi-lingual, low-resource code translation dataset
US20250080563A1
( en )
*
2022-10-31
2025-03-06
Rakuten Symphony, Inc.
System and method for simulation of cyberattacks with execution confirmation
2023
2023-09-06
US
US18/462,363
patent/US20250077397A1/en
active
Pending
Patent Citations (12)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20190196949A1
( en )
*
2017-12-27
2019-06-27
Accenture Global Solutions Limited
Test scenario and knowledge graph extractor
US20190370160A1
( en )
*
2017-12-27
2019-12-05
Accenture Global Solutions Limited
Touchless testing platform
US20190213115A1
( en )
*
2018-01-08
2019-07-11
Accenture Global Solutions Limited
Utilizing artificial intelligence to test cloud applications
US20190213116A1
( en )
*
2018-01-10
2019-07-11
Accenture Global Solutions Limited
Generation of automated testing scripts by converting manual test cases
US10949337B1
( en )
*
2019-10-29
2021-03-16
Accenture Global Solutions Limited
Utilizing neural network and artificial intelligence models to select and execute test cases in a software development platform
US20240241817A1
( en )
*
2021-05-15
2024-07-18
Telefonaktiebolaget Lm Ericsson (Publ)
Artificial intelligence based cognitive test script generation
US20230251960A1
( en )
*
2022-02-04
2023-08-10
Optum, Inc.
Machine learning techniques for automated software testing configuration management
US20250080563A1
( en )
*
2022-10-31
2025-03-06
Rakuten Symphony, Inc.
System and method for simulation of cyberattacks with execution confirmation
US20240256423A1
( en )
*
2023-01-26
2024-08-01
Microsoft Technology Licensing, Llc
Program improvement using large language models
US20240265281A1
( en )
*
2023-02-08
2024-08-08
Continuity AI, LLC
System for providing software related answer based on a trained model
US20240330167A1
( en )
*
2023-03-28
2024-10-03
Tuyen Mathew Duong
Intelligent Process For Iterative Software Testing and Development
US20250045185A1
( en )
*
2023-07-31
2025-02-06
International Business Machines Corporation
Large language models for creating a multi-lingual, low-resource code translation dataset
Cited By (28)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US12592301B2
( en )
2023-08-30
2026-03-31
The Toronto-Dominion Bank
Prompt engineering and generative AI for goal-based imagery
US12699556B2
( en )
2023-08-30
2026-08-04
The Toronto-Dominion Bank
Software library usage optimization
US12645429B2
( en )
2023-08-30
2026-06-02
The Toronto-Dominion Bank
Service design coordination among developers
US12541894B2
( en )
2023-08-30
2026-02-03
The Toronto-Dominion Bank
Image modification based on goal progression
US12625680B2
( en )
2023-08-30
2026-05-12
The Toronto-Dominion Bank
Creating a model of software architecture
US12585435B2
( en )
2023-08-30
2026-03-24
The Toronto-Dominion Bank
Real-time visualization of complex software architecture
US12650836B2
( en )
2023-08-31
2026-06-09
The Toronto-Dominion Bank
Gap identification and solution recommendation for complex software architecture
US12499241B2
( en )
2023-09-06
2025-12-16
The Toronto-Dominion Bank
Correcting security vulnerabilities with generative artificial intelligence
US12517812B2
( en )
2023-09-06
2026-01-06
The Toronto-Dominion Bank
Security testing based on generative artificial intelligence
US12670640B2
( en )
2023-09-21
2026-06-30
The Toronto-Dominion Bank
Partial exchange of NFT via NFT interaction
US12646051B2
( en )
2023-09-21
2026-06-02
The Toronto-Dominion Bank
Dynamically evolving image based on pursuit of goals
US12572451B1
( en )
2023-09-29
2026-03-10
Amazon Technologies, Inc.
Training checkpoint validation for machine learning models
US12614026B1
( en )
*
2023-09-29
2026-04-28
Amazon Technologies, Inc.
Reference and training data validation for machine learning models
US12641178B2
( en )
2023-10-04
2026-05-26
The Toronto-Dominion Bank
Real-time dynamic visualization of content
US12657566B2
( en )
2023-10-05
2026-06-16
The Toronto-Dominion Bank
GenAI letter of confirmation of benefits
US20250117595A1
( en )
*
2023-10-05
2025-04-10
The Toronto-Dominion Bank
Dynamic prompting based on conversation context
US12711683B2
( en )
2024-01-12
2026-08-18
The Toronto-Dominion Bank
Dynamic animation based on waiting period
US12693842B2
( en )
2024-01-12
2026-07-28
The Toronto-Dominion Bank
Feature activation based on alternative feature behavior
US12705398B2
( en )
2024-03-25
2026-08-11
The Toronto-Dominion Bank
Youth application with parental guidance role
US12670334B2
( en )
2024-03-25
2026-06-30
The Toronto-Dominion Bank
Chatbot with dynamic conversational tone
US12682179B2
( en )
2024-03-28
2026-07-14
The Toronto-Dominion Bank
Response determination based on contextual attributes and previous conversation content
US20250321874A1
( en )
*
2024-04-12
2025-10-16
L&T Technology Services Limited
Method and system of determining test procedures using large language models
US12645689B2
( en )
2024-06-14
2026-06-02
The Toronto-Dominion Bank
Weighted retrieval for in-context tabular model
US12645838B2
( en )
2024-07-18
2026-06-02
The Toronto-Dominion Bank
Limiting activity based on a profile
US12536264B2
( en )
2024-07-19
2026-01-27
The Toronto-Dominion Bank
Parallel artificial intelligence driven identity checking with biometric prompting
US20260030145A1
( en )
*
2024-07-29
2026-01-29
Sap Se
Generative ai application artifact management enabling safeguarding by key users
US12675390B2
( en )
*
2024-11-19
2026-07-07
International Business Machines Corporation
Large language model-based test-case refinement for application programming interfaces
US20260140852A1
( en )
*
2024-11-19
2026-05-21
International Business Machines Corporation
Large Language Model-Based Test-Case Refinement For Application Programming Interfaces
Similar Documents
Publication
Publication Date
Title
US12517812B2
( en )
2026-01-06
Security testing based on generative artificial intelligence
US12499241B2
( en )
2025-12-16
Correcting security vulnerabilities with generative artificial intelligence
US20250077396A1
( en )
2025-03-06
Test case generation using generating artificial intelligence
US20250077682A1
( en )
2025-03-06
Dynamic generation of features based on software testing
US20250077399A1
( en )
2025-03-06
Generation of automation test script using generative artificial intelligence
US20250077204A1
( en )
2025-03-06
Software library usage optimization
US20250086451A1
( en )
2025-03-13
Continuously evolving user training manual
US20250085936A1
( en )
2025-03-13
System architecture with generative ai
US20250086096A1
( en )
2025-03-13
Automated generation of test data
US20250086441A1
( en )
2025-03-13
Answering software-related questions with generative ai
US12399687B2
( en )
2025-08-26
Generating software architecture from conversation
US12585435B2
( en )
2026-03-24
Real-time visualization of complex software architecture
US20250077227A1
( en )
2025-03-06
Gap identification and solution recommendation for complex software architecture
US20250077556A1
( en )
2025-03-06
Leveraging an architecture blueprint to determine information
US12625680B2
( en )
2026-05-12
Creating a model of software architecture
CA3215254A1
( en )
2025-06-17
Service design coordination among developers
US10642721B2
( en )
2020-05-05
Generation of automated testing scripts by converting manual test cases
US7643982B2
( en )
2010-01-05
Debugging prototyped system solutions in solution builder wizard environment
US11687441B2
( en )
2023-06-27
Intelligent dynamic web service testing apparatus in a continuous integration and delivery environment
US20240012387A1
( en )
2024-01-11
Live streaming and recording of remotely executed robotic process automation workflows
US12056042B2
( en )
2024-08-06
System and method for serverless application testing
US20210232495A1
( en )
2021-07-29
System and method for executing a test case
CA3211211A1
( en )
2025-06-18
Test case generation using generating artificial intelligence
CA3211253A1
( en )
2025-06-18
Large language model training for test case generation
CA3211238A1
( en )
2025-06-18
Generation of automation test script using generative artificial intelligence
Legal Events
Date
Code
Title
Description
2023-09-29
STPP
Information on status: patent application and granting procedure in general
Free format text : DOCKETED NEW CASE - READY FOR EXAMINATION
2023-09-30
AS
Assignment
Owner name : THE TORONTO-DOMINION BANK, CANADA
Free format text : ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SEN, GOPENDU;REEL/FRAME:065082/0982
Effective date : 20230921
2025-06-13
STPP
Information on status: patent application and granting procedure in general
Free format text : NON FINAL ACTION MAILED
2025-09-18
STPP
Information on status: patent application and granting procedure in general
Free format text : RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER
2025-11-14
STPP
Information on status: patent application and granting procedure in general
Free format text : FINAL REJECTION COUNTED, NOT YET MAILED
2025-11-17
STPP
Information on status: patent application and granting procedure in general
Free format text : FINAL REJECTION MAILED
2025-11-18
STPP
Information on status: patent application and granting procedure in general
Free format text : FINAL REJECTION MAILED
2026-02-04
STPP
Information on status: patent application and granting procedure in general
Free format text : DOCKETED NEW CASE - READY FOR EXAMINATION
2026-03-31
STPP
Information on status: patent application and granting procedure in general
Free format text : NON FINAL ACTION COUNTED, NOT YET MAILED
2026-04-02
STPP
Information on status: patent application and granting procedure in general
Free format text : NON FINAL ACTION MAILED
Related documents
- Multi-SWT-Bench: A Multilingual Benchmark for Reproduction Test Generation#1129010
- Multi-SWT-Bench: A Multilingual Benchmark for Reproduction Test Generation#1108830
- Call-Chain-Aware LLM-Based Test Generation for Java Projects#1326254
- When Who You Are Can Change the Code You Get: A Study of Persona-Induced Bias in LLM Code Generation#1036786
- Reproduction Test Generation for Java SWE Issues#168415
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive