Conceptio › Archive › Google Patents

Google Patentsopen access

… the automated analysis or use of data, including use of a large language model — Unlikely Artificial Intelligence Limited (US12067362B2)

Unlikely Artificial Intelligence Limited · Google Patents

Google Patents · Patents · License: Open Access

pedoe unlikelyartificialintelligencelimited williamtunstall

patent, google patents, intellectual property, US12067362B2, Unlikely Artificial Intelligence Limited, William Tunstall-Pedoe, en, 2024

ABSTRACT

Abstract

Methods are provided, such as a method of interacting with a large language model (LLM), including the step of a processing system using a structured, machine-readable representation of data that conforms to a machine-readable language, such as a universal language, to provide new context data for the LLM, in order to improve the output, such as continuation text output, generated by the LLM in response to a prompt; and such as a method of interacting with a LLM, including the step of providing continuation data generated by the LLM to a processing system that uses a structured, machine-readable representation of data that conforms to a machine-readable language, such as a universal language, in which the processing system is configured to analyse the continuation output generated by the LLM in response to a prompt to enable an improved version of that continuation output to be provided to a user. Related computer systems are provided.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a continuation of International Application No. PCT/GB2023/050405, filed on Feb. 22, 2023, which claims priority to GB Application No. GB2202347.7, filed on Feb. 22, 2022; GB Application No. GB2219268.6, filed on Dec. 20, 2022; GB Application No. GB2300624.0, filed on Jan. 16, 2023; and GB Application No. GB2302085.2, filed on Feb. 14, 2023, and is a continuation-in-part of U.S. application Ser. No. 18/001,368, filed on Dec. 9, 2022, the entire contents of each of which being fully incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The field of the invention relates to computer implemented methods for the automated analysis or use of data, including use of a large language model (LLM), and to related computer implemented methods and systems.

2. Technical Background

Natural language (NL) is language evolved for humans such as the English language. Although significant advances have been made in computers' ability to process natural language, computers are still not able to deeply understand the meaning of natural language and use that meaning internally.

For this reason most computer applications typically use structured data to store information that they need for processingâe.g. a relational database: designing the schema, populating the database and writing code to process the fields in the database.

Use of structured data can work well if the application has limited requirements for the type of data required. However, some applications naturally require an extremely broad, heterogeneous collection of data to work well. This means that the schema required would have to be enormous, making building and coding for such an application impractical. We refer to such applications herein as HUB applications (Heterogeneous and Unreasonably Broad).

Examples of HUB applications include an application for managing a person's general health data where there are thousands of tests, thousands of medical conditions and thousands of symptoms. Another related application could be a nutrition tracking application where there are many thousands of substances and foods that can be ingested, each with different metabolic effects on the body.

Another example is an application to match the resume of potential candidates with a job specification: in principle such an application would need structured data to represent every skill that might be of value to any role, every type of experience, every type of previous job.

Accounting is another application where vast heterogeneous data would be valuable: the perfect accounting application would represent every type of contract, every type of service.

In practice some of these applications, where they exist, work with a limited schema that doesn't cover the full range of their ideal properties. Health applications for example, typically work like this ignoring many types of data that they do not cover and instead end up being narrowâlimiting the application to only certain verticals within health.

Applications may also use natural language or augment a limited schema with natural languageâsuch as with current resume matching applications which might represent a few key skills in a structured form but rely largely on keyword searching or statistical natural language processing (NLP) techniques on written resumes otherwise.

In the case of accounting, transactions are represented with limited structured dataâdebits and credits on virtual ledgers with natural language names. The meaning of the natural language names and thus what these transactions represent is generally opaque to the application. Virtual ledgers often group different types of transaction together but fail to represent semantic differences which may be important.

There is no exact threshold for when an application becomes a HUB application but the difficulty of building an application with a hand created schema grows more than linearly with the number of tables as managing these tables as well as the code that maintains them becomes increasingly difficult to do.

These issues could be addressed if there existed a language or way of representing data that computers could fully process and understand but that also had an extremely broad scope.

In conventional Artificial Intelligence (AI), statistical Machine Learning (ML)âparticularly Deep Learning (DL)âhas been widely used. This has provided significant progress with many problems. Despite the progress, the results cannot be explained in a way that makes sense to human users as the solutions are the result of a calculation that may involve billions of weights. It can also be argued that the system lacks ârealâ understanding of the data or at least that the understanding is very different from the way a human user would understand it. The statistical nature of the technology also means that it is inherently unreliable with weights that are initiated randomly and then improved through a process of training where they are exposed to data. For complex problems, this improvement (learning) generally results in a model that works only a percentage of the time and is unable to reliably know when the result it produces is accurate.

Much processing of natural language is now done with DL. One example of DL applied to language is large language models (LLMs). LLMs are usually very large deep learning models potentially containing hundreds of billions of parameters and trained on tens of terabytes of text. Although they can generate explanations, those explanations are not derived from what has happened within the model. The text that LLMs produce is frequently incorrect and often describes things that are not true.

3. Discussion of Related Art

The Wikipedia page about Cyc, of 18 Jul. 2019, states that Cyc is the world's longest-lived artificial intelligence project, attempting to assemble a comprehensive ontology and knowledge base that spans the basic concepts and ârules of thumbâ about how the world works (think common sense knowledge but focusing more on things that rarely get written down or said, in contrast with facts one might find somewhere on the internet or retrieve via a search engine or Wikipedia), with the goal of enabling AI applications to perform human-like reasoning and be less âbrittleâ when confronted with novel situations that were not preconceived.

The Wikipedia page about Cyc, of 18 Jul. 2019, states that the objective of the Cyc project was to codify, in machine-usable form, the millions of pieces of knowledge that compose human common sense; this entailed, along the way, (1) developing an adequately expressive representation language, CycL, (2) developing an ontology spanning all human concepts down to some appropriate level of detail, (3) developing a knowledge base on that ontological framework, comprising all human knowledge about those concepts down to some appropriate level of detail, and (4) developing an inference engine exponentially faster than those used in then-conventional expert systems, to be able to infer the same types and depth of conclusions that humans are capable of, given their knowledge of the world.

The Wikipedia page about Cyc, of 18 Jul. 2019, states that most of Cyc's knowledge, outside math and games, is only true by default; for example, Cyc knows that as a default parents love their children, when you're made happy you smile, taking your first step is a big accomplishment, when someone you love has a big accomplishment that makes you happy, and only adults have children; when asked whether a picture captioned âSomeone watching his daughter take her first stepâ contains a smiling adult person, Cyc can logically infer that the answer is Yes, and âshow its workâ by presenting the step by step logical argument using those five pieces of knowledge from its knowledge base: these are formulated in the language CycL, which is based on predicate calculus and has a syntax similar to that of the Lisp programming language.

The Wikipedia page about Cyc, of 18 Jul. 2019, states that the Cyc project has been described as âone of the most controversial endeavors of the artificial intelligence historyâ; Catherine Havasi, CEO of Luminoso, says that Cyc is the predecessor project to IBM's Watson; machine-learning scientist Pedro Domingos refers to the project as a âcatastrophic failureâ for several reasons, including the unending amount of data required to produce any viable results and the inability for Cyc to evolve on its own; Robin Hanson, a professor of economics at George Mason University, gives a more balanced analysis: âOf course the CYC project is open to criticism on its many particular choices. People have complained about its logic-like and language-like representations, about its selection of prototypical cases to build from (e.g., encyclopedia articles), about its focus on answering over acting, about how often it rebuilds vs. maintaining legacy systems, and about being private vs. publishing everything. But any large project like this would produce such disputes, and it is not obvious any of its choices have been seriously wrong. They had to start somewhere, and in my opinion they have now collected a knowledge base with a truly spectacular size, scope, and integration. Other architectures may well work better, but if knowing lots is anywhere near as important as Lenat thinks, I'd expect serious AI attempts to import CYC's knowledge, translating it into a new representation. No other source has anywhere near CYC's size, scope, and integration.â

The True Knowledge system provided open-domain question answering using structured knowledge and inference. In the True Knowledge system, knowledge in the knowledge base was represented in a single unified format: named relations between pairs of named entities referred to as âfacts.â Facts, and the relations themselves, were first-class entities so facts about facts and facts about the properties of relations were fully supported (Tunstall-Pedoe, W. (2010). True Knowledge: Open-Domain Question Answering Using Structured Knowledge and Inference. AI Magazine, 31(3), 80-92. https://ojs.aaai.org/aimagazine/index.php/aimagazine/article/view/2298).

EP2527990A2 and EP2527990B1 disclose a method of using large language models in machine translation in which a translation model is partitioned into a plurality of language model partitions stored on a plurality of different language model servers. Segments of text are distributed to the servers for translation according to server workload. The language model information is usually collected by computing the frequency of occurrence of sequences of words in a large training corpus of documents.

SUMMARY OF THE INVENTION

According to a first aspect of the invention, there is provided a method of interacting with a LLM, including the step of a processing system using a structured, machine-readable representation of data that conforms to a machine-readable language, such as a universal language, to provide new context data for the LLM, in order to improve the output, such as continuation text output, generated by the LLM in response to a prompt.

According to a second aspect of the invention, there is provided a method of interacting with a LLM, including the step of providing continuation data generated by the LLM to a processing system that uses a structured, machine-readable representation of data that conforms to a machine-readable language, such as a universal language, in which the processing system is configured to analyse the continuation output (e.g. text output) generated by the LLM in response to a prompt to enable an improved version of that continuation output to be provided to a user. An example is shown in FIG. 10 . An advantage is that improved continuation output to that generated by the LLM may be provided.

According to a third aspect of the invention, there is provided a method of interacting with a LLM, including the step of generating continuation data by an LLM, by the LLM using the output of a processing system that uses a structured, machine-readable representation of data that conforms to a machine-readable language, such as a universal language, in which the LLM is configured to use the output of the processing system as a prompt and to generate a continuation output (e.g. text output) that is an improved version of the output from the processing system. An advantage is that an improved version of the output from the processing system may be generated by the LLM.

According to a fourth aspect of the invention, there is provided a method of interacting with a LLM, including the step of training the LLM on the output from a processing system using a structured, machine-readable representation of data that conforms to a machine-readable language, such as a universal language. An advantage is that an improved LLM may be provided.

According to a fifth aspect of the invention, there is provided a method for interacting with one or more users in natural language comprising the steps of:

providing a structured representation of data and a system operable to process language with reference to the structured representation of data to provide additional useful text relevant to the language; providing an LLM operable to provide a continuation of text; receiving a language input from one or more users; processing the language input with the system to produce additional useful text; providing the LLM with the language input plus the additional useful text; providing the one or more users with language derived from a continuation generated by the LLM. An advantage is that additional useful text to that provided by a system operable to process language with reference to the structured representation of data may be provided by a continuation generated by the LLM.

According to a sixth aspect of the invention, there is provided a method of validating natural language for factual accuracy comprising the steps of:

Extracting from the natural language a collection of one or more factual assertions asserted within the natural language; Checking the one or more factual assertions for factual accuracy; Outputting, or making available, the results of the checking.

An advantage is validating natural language for factual accuracy.

According to a seventh aspect of the invention, there is provided a method of learning knowledge in structured form comprising the steps of:

Providing an LLM trained on a large corpus of natural language documents; Providing the LLM with a prompt that will elicit desired knowledge; Eliciting a continuation of the prompt from the LLM; Translating the continuation into the structured form.

An advantage is learning knowledge in structured form.

According to an eighth aspect of the invention, there is provided a method for ensuring that an LLM generates original text comprising the steps of:

Providing a database of previous text that it should not generate; Performing a beam search; Checking potential continuations against the database; When a potential continuation matches non-original text in the database, adjusting the continuation to no longer match non-origin

CROSS-REFERENCE TO RELATED APPLICATIONS

BACKGROUND OF THE INVENTION

1. Field of the Invention

2. Technical Background

Accounting is another application where vast heterogeneous data would be valuable: the perfect accounting application would represent every type of contract, every type of service.

These issues could be addressed if there existed a language or way of representing data that computers could fully process and understand but that also had an extremely broad scope.

3. Discussion of Related Art

SUMMARY OF THE INVENTION

According to a fifth aspect of the invention, there is provided a method for interacting with one or more users in natural language comprising the steps of:

According to a sixth aspect of the invention, there is provided a method of validating natural language for factual accuracy comprising the steps of:

An advantage is validating natural language for factual accuracy.

According to a seventh aspect of the invention, there is provided a method of learning knowledge in structured form comprising the steps of:

An advantage is learning knowledge in structured form.

According to an eighth aspect of the invention, there is provided a method for ensuring that an LLM generates original text comprising the steps of:

An advantage is avoiding copyright infringement.

According to a ninth aspect of the invention, there is provided a method of avoiding hallucination in an LLM comprising the steps of:

Analysing different continuations for a given prompt; Comparing the continuations to identify parts of the response that do not vary and parts of the response that do vary; Identifying the parts that vary as more likely to be hallucination; Replacing or not showing the parts identified as hallucination to the user.

An advantage is avoiding hallucination in an LLM.

According to a tenth aspect of the invention, there is provided a method for adding citations to text generated by an LLM comprising the steps of:

Providing a database of text used to train the LLM which includes sources associated with each section of text stored; checking sections of the continuation generated by the LLM against the database; retrieving sources where the sections of the continuation match text contained within the database. An advantage is that citations are added to text generated by an LLM.

According to an eleventh aspect of the invention, there is provided a computer-implemented method including the steps of

(i) Receiving output from a natural language processing computer process, the output including an answer to a question; (ii) Inputting the received output to a large language model (LLM); (iii) The large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM. (iv) Storing the output based on the input to the LLM.

An advantage is that an improved answer to the question may be provided by the LLM output.

According to a twelfth aspect of the invention, there is provided a computer-implemented method of training a large language model (LLM), including the steps of

(i) receiving output from a natural language processing computer process, the output including an answer to a question; (ii) repeating step (i) at least one thousand times using a set of at least one thousand different questions, and storing the answers to the questions in a training file; (iii) using the training file to train a large language model (LLM); (iv) storing weights characterizing the trained LLM.

An advantage is that a trained LLM with training improved by the use of the training file is provided.

According to a thirteenth aspect of the invention, there is provided a computer-implemented method of generating a training file for a large language model (LLM), including the steps of

According to a fourteenth aspect of the invention, there is provided a computer-implemented method of re-training a large language model (LLM), the LLM having been previously trained using a training file, the method including the steps of

(i) receiving output from a natural language processing computer process, the output including an answer to a question; (ii) repeating step (i) at least one thousand times using a set of at least one thousand different questions, and storing the answers to the questions in a re-training file; (iii) combining the training file and the re-training file, to generate a combined training file; (iv) using the combined training file to re-train the large language model (LLM); (v) storing weights characterizing the re-trained LLM.

An advantage is that an LLM with an improved training is provided.

According to a fifteenth aspect of the invention, there is provided a computer-implemented method of updating the training of a large language model (LLM), the LLM having been previously trained using a training file, the method including the steps of

(i) receiving output from a natural language processing computer process, the output including an answer to a question; (ii) repeating step (i) at least one thousand times using a set of at least one thousand different questions, and storing the answers to the questions in a re-training file; (iii) using the re-training file to update the training of the large language model (LLM); (iv) storing weights characterizing the LLM with the updated training. An advantage is that an LLM with an improved training is provided.

According to a sixteenth aspect of the invention, there is provided a computer-implemented method of improving output using an LLM, including the steps of

(i) receiving a natural language question (e.g. any natural language to which a natural language response is appropriate. It could be for example a command or a request for data or even some kind of social interaction or discussion); (ii) using a computer system including a processor and a memory, the processor configured to use a processing language in which semantic nodes are represented in the processing language, the semantic nodes including semantic links between semantic nodes wherein the semantic links are themselves semantic nodes, in which each semantic node denotes one specific meaning, in which a combination of semantic nodes defines a semantic node, in which expressions in the processing language may be nested, in which the natural language question is represented in the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language; (iii) the processor answering the question using the reasoning steps, the computation units and the semantic nodes, and (iv) inputting the natural language question, and the processor's answer to the question to the LLM; (v) the large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM; (vi) storing the output based on the input to the LLM.

An advantage is that an improved answer to the question may be provided by the LLM output.

According to a seventeenth aspect of the invention, there is provided a computer-implemented method of improving output from an LLM, including the steps of

(i) receiving a first natural language question; (ii) Inputting or providing the received first natural language question to a large language model (LLM); (iii) The large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM; (iv) translating the output into a processing language in which semantic nodes are represented in the processing language, the semantic nodes including semantic links between semantic nodes wherein the semantic links are themselves semantic nodes, in which each semantic node denotes one specific meaning, in which a combination of semantic nodes defines a semantic node, in which expressions in the processing language may be nested; (v) constructing a question which asks if the output is true, in which the question is represented in the processing language; (vi) inputting the question to a computer system including a processor and a memory, the processor configured to use the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language; (vii) the processor answering the question using the reasoning steps, the computation units and the semantic nodes, and (viii) the processor storing an answer to the question in the memory.

An advantage is that the LLM output is checked for accuracy.

According to an eighteenth aspect of the invention, there is provided a computer-implemented method of fact checking output from a large language model (LLM), including the steps of

(i) receiving a text input; (ii) inputting or providing the received text input to a large language model (LLM); (iii) the large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM; (iv) translating the output into a processing language in which semantic nodes are represented in the processing language, the semantic nodes including semantic links between semantic nodes wherein the semantic links are themselves semantic nodes, in which each semantic node denotes one specific meaning, in which a combination of semantic nodes defines a semantic node, in which expressions in the processing language may be nested, wherein translating the output includes extracting the assertions in text generated by the LLM; (v) constructing one or more questions which ask if the extracted assertions are individually true, in which the one or more questions are represented in the processing language; (vi) inputting the one or more questions to a computer system including a processor and a memory, the processor configured to use the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language; (vii) the processor answering the one or more questions using the reasoning steps, the computation units and the semantic nodes, and (viii) the processor storing an individual answer to each of the one or more questions in the memory. An advantage is fact checking output from a large language model.

According to a nineteenth aspect of the invention, there is provided a computer-implemented method of improving output from a computer system including a processor and a memory, the system operable to process the natural language question using a processing language in which semantic nodes are represented in the processing language, the method including using an LLM, the method including the steps of

(i) receiving a natural language question; (ii) using a computer system including a processor and a memory, the system operable to process the natural language question using a processing language in which semantic nodes are represented in the processing language, the semantic nodes including semantic links between semantic nodes wherein the semantic links are themselves semantic nodes, in which each semantic node denotes one specific meaning, wherein the memory is configured to store the representations in the processing language; (iii) the processor answering the question, and (iv) inputting the natural language question, and the processor's answer to the question to the LLM; (v) the large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM; (vi) Storing the output based on the input to the LLM.

An advantage is that output from the computer system including a processor and a memory, the system operable to process the natural language question using a processing language in which semantic nodes are represented in the processing language, is improved using the LLM.

According to a twentieth aspect of the invention, there is provided a computer-implemented method of improving output from an LLM, including the steps of

(i) receiving a first natural language question; (ii) inputting the received first natural language question to a large language model (LLM); (iii) the large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM; (iv) translating the output into a processing language in which semantic nodes are represented in the processing language, the semantic nodes including semantic links between semantic nodes wherein the semantic links are themselves semantic nodes, in which each semantic node denotes one specific meaning; (v) constructing a question which asks if the output is true, in which the question is represented in the processing language; (vi) inputting the question to a computer system including a processor and a memory, the processor configured to use the processing language, wherein the memory is configured to store the representations in the processing language; (vii) the processor answering the question, and (viii) the processor storing an answer to the question in the memory. An advantage is that the LLM output is checked for accuracy.

According to a 21st aspect of the invention, there is provided a computer-implemented method of fact checking output from a large language model (LLM), including the steps of

(i) receiving a text input; (ii) inputting the received text input to a large language model (LLM); (iii) the large language model (LLM) processing the input to the LLM, to generate output based on the input to the LLM; (iv) translating the output into a processing language in which semantic nodes are represented in the processing language, the semantic nodes including semantic links between semantic nodes wherein the semantic links are themselves semantic nodes, in which each semantic node denotes one specific meaning, wherein translating the output includes extracting the assertions in text generated by the LLM; (v) constructing one or more questions which ask if the extracted assertions are individually true, in which the one or more questions are represented in the processing language; (vi) inputting the one or more questions to a computer system including a processor and a memory, the processor configured to use the processing language, wherein the memory is configured to store the representations in the processing language; (vii) the processor answering the one or more questions, and (viii) the processor storing an individual answer to each of the one or more questions in the memory.

An advantage is fact checking output from a large language model.

According to a 22nd aspect of the invention, there is provided a computer implemented method of augmenting search results, including the steps of:

(i) receiving a search query; (ii) an (e.g. internet) search engine processing the search query to generate (e.g. internet) search results; (iii) transforming the search results into a structured, machine-readable representation of data that conforms to a machine-readable language, such as a universal language; (iv) processing the transformed search results using a processing system which uses the structured, machine-readable representation of data that conforms to the machine-readable language, such as a universal language, to produce output; (v) supplying the output of step (iv) as input to a large language model (LLM), and the LLM generating continuation data output in response to the input. An advantage is that search results found by a search engine are augmented.

According to a 23rd aspect of the invention, there is provided a method of fact-checking the output of a large language model (LLM), including the step of providing continuation data generated by the LLM to a processing system that uses a structured, machine-readable representation of data that conforms to a machine-readable language, in which the processing system analyses the continuation output generated by the LLM in response to a prompt, to enable an improved, fact-checked version of that continuation output to be generated and provided to a user. An advantage is fact checking output from a large language model.

Further aspects of the invention are given in the dependent Claims, and in the independent Claims. Aspects of the invention may be combined with each other, or with any of the Concepts described herein, or with any of the Claims.

BRIEF DESCRIPTION OF THE FIGURES

Aspects of the invention will now be described, by way of example(s), with reference to the following Figures, in which:

FIG. 1 shows example screen output for a notification of a job match.

FIG. 2 shows example screen output for a description of a job match.

FIG. 3 shows an example conversation within an app where nutritional data is being communicated with the app.

FIG. 4 shows some example insights that can be derived from a period of time where horizontal health and nutrition data was shown.

FIG. 5 shows an example graph showing daily calories in versus calories out which is an extremely common thing for someone to track if the user is aiming to lose (or gain) weight.

FIG. 6 shows an example of a visualisation capable of being generated from an example of the present invention: it is comparing estimated caffeine in the user's body at the time they went to bed with a calculation of sleep quality.

FIG. 7 shows examples of explanations generated. (a) shows examples of simplified explanations; (b) shows an example of a detailed explanation.

FIG. 8 shows an example of a voice assistant product, referred to herein as âBrianâ and how it fits in with the wider UL platform and other applications built on the UL platform.

FIG. 9 shows an alternative to the example of FIG. 8 .

FIG. 10 shows an example of a method of interacting with a LLM, including the step of providing continuation data generated by the LLM to a processing system that uses a structured, machine-readable representation of data that conforms to a machine-readable language, such as a universal language, in which the processing system is configured to analyse the continuation output (e.g. text output) generated by the LLM in response to a prompt to enable an improved version of that continuation output to be provided to a user.

FIG. 11 shows an example of an intermediate representation semantic graph.

DETAILED DESCRIPTION

This disclosure includes a computer implemented method for the automated analysis or use of data, which comprises the steps of:

(a) storing in a memory a structured, machine-readable representation of data that conforms to a machine-readable language (âmachine representationâ); the machine representation including representations of user speech or text input to a human/machine interface; (b) automatically processing the machine representations to analyse the user speech or text input.

This disclosure includes a computer-based system configured to analyse data, in which the system is configured to:

(a) store in a memory a structured, machine-readable representation of data that conforms to a machine-readable language; the structured, machine-readable representation of data including representations of user speech or text input to a human/machine interface; (b) automatically process the structured representations to analyse the user speech or text input to a human/machine interface.

These aspects of the disclosure may be implemented in a voice assistant or chatbot; the technical advantage achieved is to enable more extensive and faster scaling of the voice assistant or chatbot's capabilities; the invention enables voice assistants and chatbot to be able to answer a broader range of questions, and to do so more accurately; this makes it easier for the voice assistant or chatbot to work with large numbers of different natural languages.

One point of interpretation: the conjunction âorâ should not be narrowly construed to indicate mutual exclusivity but to cover inclusivity too. Hence, the phrase âuser speech or text inputâ means âuser speech on its own, or user text on its own, or both user speech and also user textâ. When the conjunction âorâ is meant exclusively, the phrase âeither . . . orâ is used.

The machine-readable language is extremely expressive yet also extremely simple; simplicity requires less computer processing and thus delivers faster performance. Further details are in the appended Concepts and Claims.

According to a further aspect of the disclosure, there is provided a computer system including a processor and a memory, the processor configured to answer a question, the processor configured to use a processing language in which semantic nodes are represented in the processing language, the semantic nodes including semantic links between semantic nodes wherein the semantic links are themselves semantic nodes, in which each semantic node denotes one specific meaning, in which a combination of semantic nodes defines a semantic node, in which expressions in the processing language may be nested, in which the question is represented in the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language, and wherein the processor is configured to answer the question using the reasoning steps, the computation units and the semantic nodes, and to store an answer to the question in the memory.

An advantage is that because semantic links between semantic nodes are themselves semantic nodes, semantic links and semantic nodes do not need to be processed in a distinctly different way, which simplifies processing, which speeds up response times, which is a technical effect.

An advantage is that because semantic nodes are very widely used in the processing language, processing of the processing language is speeded up, which speeds up response times, which is a technical effect.

The technical effect operates at the level of the architecture of the computer system; that is to say the effect is produced irrespective of the data being processed.

The technical effect results in the computer system being made to operate in a new way, because the computer finds answers to questions faster than using prior art approaches, because semantic nodes are very widely used in the processing language, which means that processing of the processing language is speeded up.

The processing language contributes to the technical character because it produces a technical effect: a processor processing the processing language has the effect that the computer finds answers to questions faster than using prior art approaches, because semantic nodes are very widely used in the processing language, which means that processing of the processing language is speeded up.

The computer system may be configured to output the answer to the question.

The computer system may be configured to output the answer to the question to a display device.

The computer system may be one wherein expressions in the processing language may be nested with no limit inherent to the processing language.

The computer system may be one wherein the semantic nodes each includes a unique identifier.

The computer system may be one wherein the computation units are semantic nodes.

The computer system may be one wherein the question is represented in the processing language with a passage comprising a semantic node that identifies the passage as a question, a list of zero, one or more semantic nodes representing unknown entities being asked about and at least one further passage which represents the semantics of the question in the context of the zero, one or more unknown entities.

The computer system may be one wherein the processing language is universal language.

The computer system may be one wherein the processing language is not a natural language.

The computer system may be one wherein the question relates to search and analysis of documents or web pages, wherein the semantic nodes include representations of at least parts of the documents or the web pages stored in a document store.

The computer system may be one wherein the question relates to a location-based search, using mapping data represented as semantic nodes in the processing language.

The computer system may be one wherein the question relates to a search for defined advertisements or news, wherein the semantic nodes include representations of advertisements, news articles or other information items.

The computer system may be one wherein the question relates to a request for a summary of a news topic, wherein the semantic nodes include representations of news from multiple sources, e.g. to provide a summary or aggregation of the news.

The computer system may be one wherein the question relates to a request for a compatibility match between persons, wherein the semantic nodes include representations of personal information defining one or more attributes of a person, for a plurality of people.

The computer system may be one wherein the question relates to compliance with requirements preventing abusive or illegal social media postings, wherein the semantic nodes include representations of social media postings.

The computer system may be one wherein the question relates to analysing customer reviews, wherein the semantic nodes include representations of customer reviews.

The computer system may be one wherein the question relates to a user's product request, wherein the semantic nodes include representations of product descriptions and user product requests.

The computer system may be one wherein the question relates to a job search, wherein the semantic nodes include representations of job descriptions and job applicants' skills and experience, to determine which job applicants match a job description, or to determine which job descriptions match a job applicant's skills and experience.

The computer system may be one wherein the question relates to health of an individual, wherein the semantic nodes include health data relating to the individual, and health data relating to human beings.

The computer system may be one wherein the question relates to nutrition, wherein the semantic nodes include nutritional data for foods and drinks.

The computer system may be one wherein the question relates to accounting or finance, wherein the semantic nodes include representations of financial or accounting information.

The computer system may be one wherein the question is received by a voice assistant or chatbot, wherein the semantic nodes include representations of user speech input to a human/machine interface and include representations of the human/machine interface itself.

According to a further aspect of the disclosure, there is provided a computer-implemented method, the method using a computer system including a processor and a memory, the processor configured to use a processing language in which semantic nodes are represented in the processing language, the semantic nodes including semantic links between semantic nodes wherein the semantic links are themselves semantic nodes, in which each semantic node denotes one specific meaning, in which a combination of semantic nodes defines a semantic node, in which expressions in the processing language may be nested, in which the question is represented in the processing language, in which reasoning steps are represented in the processing language to represent semantics of the reasoning steps, in which computation units are represented in the processing language, wherein the memory is configured to store the representations in the processing language, the method including the steps of:

(i) the processor answering the question using the reasoning steps, the computation units and the semantic nodes, and (ii) the processor storing an answer to the question in the memory.

Advantages include the advantages of the previous aspect of the disclosure.

The method may be one wherein the question is represented in the processing language with a passage comprising a semantic node that identifies the passage as a question, a list of zero, one or more semantic nodes representing unknown entities being asked about and at least one further passage which represents the semantics of the question in the context of the zero, one or more unknown entities.

The method may be one wherein the unknowns in the question are identified and the passage making up the body of the question is selected for further analysis; processing begins on a list of passages from the body of the question and the selected unknowns; a first passage in the list of passages is selected for processing; processing a single passage comprises three methods: using statically stored processing language passages, utilising computation units and utilising processing language generated from reasoning:

in which the first method is to lookup in the passage store if there are any passages that can be directly mapped with the passage being processed; if the passage is exactly the same structure as a passage in the passage store, with all nodes matching other than the unknowns, then the values the unknowns match against are valid results; the second method is to check if any results can be found by executing computation units; it is checked if this passage matches against any passages in a computation unit description; all non-unknown nodes in the passage being processed must match the same nodes in the corresponding position in the computation description or align with a computation input unknown; the unknowns being processed must align to output unknowns in the description; the computation unit is then called to get valid output values for the processed passage's unknowns; the third method is to see if this passage can be proved by applying any reasoning steps; reasoning steps are searched for where a passage in the second half of the reasoning passage can be unified with the passage being processed; all nodes and structure must be equal between the two passages, other than unknowns in the focus passage or the reasoning passage; if a reasoning passage like this is found it means that this reasoning step could be used to prove the passage being processed; a multi-stage process is used to first find any mappings for unknowns in the processed passage when matching with the reasoning passage; secondly, mappings for unknowns used in the reasoning passage are found by mapping with the passage being processed; this mapping can then be applied to the front half of the reasoning passage to generate a list of passages that, if they can be matched with known or generated processing language and mappings found for them, will prove and find valid mappings for the focus passage; solutions for the list of passages can then be found recursively.

The method may use a computer system of any aspect of the previous aspect of the disclosure.

Aspects of the disclosure may be combined.

Examples of the present invention include systems and methods for creating and using a structured representation of data that is aimed to be as expressive as natural language in terms of breadth but which also can be processed and understood by automated systems. The representation referred to herein as UL (for universal language) is a preferred example. Examples of the invention include systems and methods relating to specific HUB and other applications as well as systems and methods which process, store and utilize UL.

A natural language question may be any natural language to which a natural language response is appropriate. It could be for example a command o

CLAIMS

Claims ( 28 )

The invention claimed is:

1. A computer-implemented method of interacting with a large language model (LLM), including the steps of:

(a) the LLM processing first input data to the LLM to generate first output from the LLM based on the first input data to the LLM;

(b) a processing system using a structured, machine-readable representation of data that conforms to a machine-readable language, in which semantic nodes are represented in the machine-readable language, the semantic nodes including semantic links between semantic nodes wherein the semantic links are themselves semantic nodes, in which each semantic node denotes one specific meaning, in which a combination of semantic nodes defines a semantic node, in which expressions in the machine-readable language are nestable, in which the first output from the LLM is represented in the machine-readable language, in which reasoning steps are represented in the machine-readable language to represent semantics of the reasoning steps, in which computation units are represented in the machine-readable language;

(c) providing the first output generated by the LLM to the processing system that uses the structured, machine-readable representation of data that conforms to the machine-readable language; and

(d) the processing system analyzing the first output generated by the LLM using the reasoning steps, the computation units and the semantic nodes, to generate second output which includes a bias-reduced version of the first output, and providing the second output which is an improved version of that output to be provided to a user.

2. The method of claim 1 , in which the processing system translates at least some of the first output into the structured, machine-readable representation of data to analyse the translated first output.

3. The method of claim 1 , in which the processing system translates the first output into the structured, machine-readable representation of data to analyse the translated first output.

4. The method of claim 2 , in which the processing system analyses the translated first output for at least one of factual inaccuracies, contradictions, and, where necessary or beneficial, provides corrected first output for use in generating second output provided by the LLM, through the processing system providing the corrected first output as a prompt to the LLM.

5. The method of claim 2 , in which the processing system analyses the translated first output and, where necessary or beneficial, provides corrected second output for use in the output that is presented to a user.

6. The method of claim 1 , in which the processing system analyses internal, logical self-consistency, including a correct time ordering of events, of the first output and, where necessary or beneficial, provides logically self-consistent output, for use in the second output that is provided to a user.

7. The method of claim 1 , in which the processing system analyses internal, logical self-consistency of the first output and, where necessary or beneficial, provides logically self-consistent output, for use in the second output that is provided to a user.

8. The method of claim 1 , in which the processing system analyses correspondence of the first output generated by the LLM to how people understand the real world or reason in the real world and, where necessary or beneficial, provides reasoned output, corresponding to how people understand the real world or reason in the real world, for use in the second output that is provided to a user.

9. The method of claim 1 , in which the processing system analyses the first output for bias and, where necessary or beneficial, provides reduced bias or bias-free output, for use in the second output that is provided to a user.

10. The method of claim 1 , in which the processing system analyses the first output for absence of relevant dynamic or real-time information and, where necessary or beneficial, provides any missing or useful dynamic or real-time information for use in the second output that is provided to a user.

11. The method of claim 1 , in which the processing system analyses the first output for any reasoned text and, where necessary or beneficial, provides improved reasoned text for use in the second output that is provided to a user.

12. The method of claim 1 , in which the LLM is answering a question and in which the processing system analyses the answer to that question in the first output and where necessary or beneficial, provides an improved answer for use in the second output that is provided to a user.

13. The method of claim 1 , in which a classifier operates to identify when the first in put data is likely to result in first output where accuracy is important, and/or when accuracy is important in the first output, and to then use the processing system to improve the factual accuracy and/or factual scope of the second output.

14. The method of claim 1 , in which the first output from the LLM is a partial continuation, namely an output made before the LLM has stopped generating or whilst the LLM is still generating.

15. The method of claim 1 , in which a classifier operates to identify when the first input data is likely to result in first output where accuracy is important.

16. The method of claim 1 , when used to improve one or more of: accuracy of the second output; factual accuracy and/or factual scope of the second output; internal, logical self-consistency of the second output; correspondence of the second output to how people understand the real world or reason in the real world; bias removal in the second output; inclusion of dynamic or real-time information in the second output.

17. The method of claim 1 , when used to improve one or more parameters of the second output: level of formality, level of brevity, suitability for speaking via a text to speech system, other style language, level of certainty.

18. The method of claim 1 , in which the LLM is a generative AI based system.

19. The method of claim 1 , in which the LLM is an autoregressive language model.

20. The method of claim 1 , in which the LLM is an autoregressive language model, which is a Generative Pre-trained Transformer.

21. The method of claim 1 , when used for: generation of program code.

22. The method of claim 1 , when used for: the solution of any problem that can be described in natural language.

23. The method of claim 1 , when used for any of the following: generation of poetry, lyrics, creative writing, generation of forms of writing, generation of essays, generation of summaries of knowledge, generation of summaries of longer texts, generation of scientific papers; question answering; internet search.

24. The method of claim 1 , including identifying first output content that breaches predefined policies by translating the first output into the machine-readable language before the first output is displayed to the user and then checking the machine-readable language representation against policies.

25. The method of claim 1 , in which the machine-readable language is a universal language.

26. The method of claim 1 , in which the first output is text output.

27. The method of claim 16 , in which the logical self-consistency is improved, in which the logical self-consistency is a correct time ordering of events.

28. The method of claim 17 , in which the level of brevity is improved, in which the second output is briefer when it includes language to be spoken.

US18/301,573

2021-08-24

2023-04-17

Computer implemented methods for the automated analysis or use of data, including use of a large language model

Active

US12067362B2

( en )

Priority Applications (1)

Application Number

Priority Date

Filing Date

Title

US18/301,573

US12067362B2

( en )

2021-08-24

2023-04-17

Computer implemented methods for the automated analysis or use of data, including use of a large language model

Applications Claiming Priority (16)

Application Number

Priority Date

Filing Date

Title

PCT/GB2021/052196

WO2022043675A2

( en )

2020-08-24

2021-08-24

A computer implemented method for the automated analysis or use of data

GB2202347

2022-02-22

GB2202347.7

2022-02-22

GBGB2202347.7A

GB202202347D0

( en )

2022-02-22

Unlikely ai 2nd gen

US202218001368A

2022-12-09

GB2219268.6

2022-12-20

GBGB2219268.6A

GB202219268D0

( en )

2022-12-20

Unlikely ai 2nd gen ii

GB2219268

2022-12-20

GBGB2300624.0A

GB202300624D0

( en )

2023-01-16

Unlikely ai 2nd gen iii

GB2300624.0

2023-01-16

GB2300624

2023-01-16

GB2302085.2

2023-02-14

GBGB2302085.2A

GB202302085D0

( en )

2023-02-14

Unlikely AI 2nd gen IV

GB2302085

2023-02-14

PCT/GB2023/050405

WO2023161630A1

( en )

2022-02-22

2023-02-22

Computer implemented methods for the automated analysis or use of data, including use of a large language model

US18/301,573

US12067362B2

( en )

2021-08-24

2023-04-17

Computer implemented methods for the automated analysis or use of data, including use of a large language model

Related Parent Applications (3)

Application Number

Title

Priority Date

Filing Date

PCT/GB2021/052196

Continuation

WO2022043675A2

( en )

2020-08-24

2021-08-24

A computer implemented method for the automated analysis or use of data

US18/001,368

Continuation

US20230343333A1

( en )

2020-08-24

2021-08-24

A computer implemented method for the aut0omated analysis or use of data

PCT/GB2023/050405

Continuation

WO2023161630A1

( en )

2021-08-24

2023-02-22

Computer implemented methods for the automated analysis or use of data, including use of a large language model

Publications (2)

Publication Number

Publication Date

US20230274089A1

US20230274089A1 ( en )

2023-08-31

US12067362B2

true

US12067362B2 ( en )

2024-08-20

Family

ID=87766941

Family Applications (1)

Application Number

Title

Priority Date

Filing Date

US18/301,573

Active

US12067362B2

( en )

2021-08-24

2023-04-17

Computer implemented methods for the automated analysis or use of data, including use of a large language model

Country Status (1)

Country

Link

( 1 )

US12067362B2

( en )

Cited By (5)

* Cited by examiner, â Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US12174906B1

( en )

2023-08-24

2024-12-24

Tiny Fish Inc.

Utilizing a query response to automate a task associated with a webpage

US12236216B1

( en )

2023-08-24

2025-02-25

Tiny Fish Inc.

Generate a script to automate a task associated with a webpage

US20250068942A1

( en )

2023-08-24

2025-02-27

Tiny Fish Inc.

Utilizing large language model responses to train an inference pattern engine

US12511497B1

( en )

2023-11-30

2025-12-30

Amazon Technologies, Inc.

Embedding-based large language model tuning

US12541488B2

( en )

2024-05-23

2026-02-03

AtomBeam Technologies Inc.

System and method for generating thoughts with large language models using codewords

Families Citing this family (40)

* Cited by examiner, â Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US11984113B2

( en )

2020-10-06

2024-05-14

Direct Cursus Technology L.L.C

Method and server for training a neural network to generate a textual output sequence

US12393824B2

( en )

2020-12-15

2025-08-19

Intel Corporation

Methods and apparatus for a knowledge-based deep learning refactoring model with tightly integrated functional nonparametric memory

US11900933B2

( en )

2021-04-30

2024-02-13

Edst, Llc

User-customizable and domain-specific responses for a virtual assistant for multi-dwelling units

US12609046B1

( en )

2022-08-01

2026-04-21

Educational Testing Service

Systems and methods for controlled language generation for language learning items

US12299404B2

( en )

2022-08-22

2025-05-13

Rohirrim, Inc.

Computer-generated content based on text classification, semantic relevance, and activation of deep learning large language models

US12596813B2

( en )

2023-01-19

2026-04-07

Citibank, N.A

Autonomous agent observation and control

US12468899B2

( en )

2023-05-08

2025-11-11

Adobe Inc.

Hallucination prevention for natural language insights

US20240419912A1

( en )

2023-06-13

2024-12-19

Microsoft Technology Licensing, Llc

Detecting hallucination in a language model

US12572855B1

( en )

2023-08-28

2026-03-10

Dmitry Shapiro

Systems and methods for refining machine learning models based on user responses to prompts

US12424210B2

( en )

2023-08-28

2025-09-23

Amazon Technologies, Inc.

Natural language processing

WO2025059289A1

( en )

2023-09-13

2025-03-20

Camb Ai, Inc.

System and method for performing translation of colloquial content

US12093965B1

( en )

2023-09-13

2024-09-17

Dmitry Shapiro

Systems and methods for generating response records for individual users based on user responses to prompts

US20250103668A1

( en )

2023-09-25

2025-03-27

Navan, Inc.

Multi-sectional user interfaces for llm-integrated parameter selection for searches

US11922143B1

( en )

2023-10-05

2024-03-05

Dmitry Shapiro

Systems and methods for providing a user interface that facilitates application development

US12614025B2

( en )

2023-10-05

2026-04-28

Bank Of America Corporation

Authorship source analysis for large language models (LLM) using a distributed ledger

US20250117381A1

( en )

2023-10-10

2025-04-10

Google Llc

Utilizing large language model (llm) in responding to multifaceted queries

US12561536B2

( en )

2023-10-23

2026-02-24

T-Mobile Usa, Inc.

Auto-translation system for voicemail transcriptions

WO2025096850A1

( en )

2023-10-31

2025-05-08

Storyline AI Inc.

Systems, methods, and media for automated creation of analytics-driven audio-visual interactive episodes

US12568019B2

( en )

2023-11-06

2026-03-03

Cisco Technology, Inc.

Intelligent auto-prompt engine for network management

US12339839B2

( en )

2023-11-08

2025-06-24

Data Squared USA Inc.

Accuracy and providing explainability and transparency for query response using machine learning models

US20250174320A1

( en )

2023-11-28

2025-05-29

Ryan Grant Renne

Method And System For Processing Medical Data

US20250200392A1

( en )

2023-12-15

2025-06-19

Amazon Technologies, Inc.

Large language model verification

CN117786077A

( en )

2023-12-26

2024-03-29

æ°å¸æååè¡ä»½æéå¬å¸

Livestock knowledge question-answering method and device based on large language model

US20250245665A1

( en )

2024-01-29

2025-07-31

Actimize Ltd

Fraud risk analysis system incorporating a large language model

US12452126B2

( en )

2024-02-13

2025-10-21

T-Mobile Usa, Inc.

Provisioning flow troubleshooting tool

US20250285043A1

( en )

2024-03-11

2025-09-11

Ford Global Technologies, Llc

Refactoring input strings

US20250291918A1

( en )

2024-03-14

2025-09-18

Palo Alto Networks, Inc.

Pipeline for rewriting and validating malicious code with generative artificial intelligence

JP2025144470A

( en )

2024-03-19

2025-10-02

ï¼¬ï½ï½ï½ã¤ãã¼æ ªå¼ä¼ç¤¾

Information processing device, information processing method, and information processing program

US12271696B1

( en )

2024-05-23

2025-04-08

AtomBeam Technologies Inc.

System and method for training and operating large language models using codewords

WO2025245541A1

( en )

2024-05-24

2025-11-27

Tappa Inc.

System and method for generating structured information through text based analysis

US12135949B1

( en )

2024-06-07

2024-11-05

Citibank, N.A.

Layered measurement, grading and evaluation of pretrained artificial intelligence models

US12154019B1

( en )

2024-06-07

2024-11-26

Citibank, N.A.

System and method for constructing a layered artificial intelligence model

US12493772B1

( en )

2024-06-07

2025-12-09

Citibank, N.A.

Layered multi-prompt engineering for pre-trained large language models

US20260086986A1

( en )

2024-09-23

2026-03-26

Aravo Solutions, Inc.

Utilizing a large language model to generate a computing structure

US12367213B1

( en )

2024-09-23

2025-07-22

Aravo Solutions, Inc.

Generating structured data from unstructured data using metadata from a large language model

US12436966B1

( en )

2024-09-23

2025-10-07

Aravo Solutions, Inc.

Utilizing a large language model to generate a computing structure

US12423265B1

( en )

2024-09-23

2025-09-23

Aravo Solutions, Inc.

Prompting a large language model for vector embeddings and metadata to generate an indexed computing file

US12399889B1

( en )

2024-09-23

2025-08-26

Aravo Solutions, Inc.

Modifying a database using a large language model

US12373649B1

( en )

2025-02-12

2025-07-29

U.S. Bancorp, National Association

Remediating hallucinations in language models

US12619831B1

( en )

2025-04-14

2026-05-05

Charles DeLisi

Systems and methods for generating deliberative committees of age-stratified large language models

Citations (248)

* Cited by examiner, â Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US4974191A

( en )

1987-07-31

1990-11-27

Syntellect Software Inc.

Adaptive natural language computer interface system

US5386556A

( en )

1989-03-06

1995-01-31

International Business Machines Corporation

Natural language analyzing apparatus and method

US6085186A

( en )

1996-09-20

2000-07-04

Netbot, Inc.

Method and system using information written in a wrapper description language to execute query on a network

US20020173971A1

( en )

2001-03-28

2002-11-21

Stirpe Paul Alan

System, method and application of ontology driven inferencing-based personalization systems

US20030130976A1

( en )

1998-05-28

2003-07-10

Lawrence Au

Semantic network methods to disambiguate natural language meaning

US20040054626A1

( en )

2002-09-17

2004-03-18

Adn Technologies

Device for processing local accounts data with different formats, equipment and a method for treating associated data

US20040078756A1

( en )

2002-10-15

2004-04-22

Napper Jonathon Leigh

Method of improving recognition accuracy in form-based data entry systems

US20040117189A1

( en )

1999-11-12

2004-06-17

Bennett Ian M.

Query engine for processing voice based queries including semantic decoding

US20040174976A1

( en )

2003-03-07

2004-09-09

Nortel Networks Limited

User controlled call routing for multiple telephony devices

US7085708B2

( en )

2000-09-23

2006-08-01

Ravenflow, Inc.

Computer system with natural language to machine language translator

US20070043708A1

( en )

2000-11-28

2007-02-22

Semscript Ltd.

Knowledge storage and retrieval system and method

US20070055656A1

( en )

2005-08-01

2007-03-08

Semscript Ltd.

Knowledge repository

US20070094224A1

( en )

1998-05-28

2007-04-26

Lawrence Au

Method and system for determining contextual meaning for network search applications

US7231633B2

( en )

2003-07-15

2007-06-12

Lsi Corporation

Debugging with set verbosity level during read and analysis of executable code and associated comments while logging according to set verbosity level

US20070136222A1

( en )

2005-12-09

2007-06-14

Microsoft Corporation

Question and answer architecture for reasoning and clarifying intentions, goals, and needs from contextual clues and content

US20070197882A1

( en )

2006-02-17

2007-08-23

Medred, Llc

Integrated method and system for diagnosis determination

AU2003266850B2

( en )

2002-10-15

2007-11-29

Silverbrook Research Pty Ltd

Method of improving recognition accuracy in form-based data entry systems

US20080033987A1

( en )

2006-08-02

2008-02-07

Carter Joe K

Primenet data management system

US20080046250A1

( en )

2006-07-26

2008-02-21

International Business Machines Corporation

Performing a safety analysis for user-defined voice commands to ensure that the voice commands do not cause speech recognition ambiguities

US20080065974A1

( en )

2006-09-08

2008-03-13

Tom Campbell

Template-based electronic presence management

US20080097748A1

( en )

2004-11-12

2008-04-24

Haley Systems, Inc.

System for Enterprise Knowledge Management and Automation

US7418443B2

( en )

2005-06-03

2008-08-26

Fuji Xerox Co., Ltd.

Question answering system, data search method, and computer program

US20080319735A1

( en )

2007-06-22

2008-12-25

International Business Machines Corporation

Systems and methods for automatic semantic role labeling of high morphological text for natural language processing applications

US20090024590A1

( en )

2007-03-15

2009-01-22

Sturge Timothy

User contributed knowledge database

US20090106612A1

( en )

2007-10-17

2009-04-23

Yogesh Pandey

Enhancing speed of simulation of an ic design while testing scan circuitry

US20090192968A1

( en )

2007-10-04

2009-07-30

True Knowledge Ltd.

Enhanced knowledge repository

US7636697B1

( en )

2007-01-29

2009-12-22

Ailive Inc.

Method and system for rapid evaluation of logical expressions

US20100054154A1

( en )

2008-07-11

2010-03-04

Lambert Paul A

Ip assignment scheme for dynamic peer-to-peer networks

US20100121839A1

( en )

2007-03-15

2010-05-13

Scott Meyer

Query optimization

US20100174692A1

( en )

2007-03-15

2010-07-08

Scott Meyer

Graph store

US20100205167A1

( en )

2009-02-10

2010-08-12

True Knowledge Ltd.

Local business and product search system and method

US20100228724A1

( en )

2009-03-09

2010-09-09

Jonah Petri

Search capability implementation for a device

US20100235162A1

( en )

2009-03-16

2010-09-16

Xerox Corporation

Method to preserve the place of parentheses and tags in statistical machine translation systems

US20100306054A1

( en )

2009-05-28

2010-12-02

Drake Robert A

Method and apparatus for generating advertisements

US20110093500A1

( en )

2009-01-21

2011-04-21

Google Inc.

Query Optimization

US20110238408A1

( en )

2010-03-26

2011-09-29

Jean-Marie Henri Daniel Larcheveque

Semantic Clustering

US20110301941A1

( en )

2009-03-20

2011-12-08

Syl Research Limited

Natural language processing method and system

US20110307435A1

( en )

2010-05-14

2011-12-15

True Knowledge Ltd

Extracting structured knowledge from unstructured text

US20110320187A1

( en )

2010-06-28

2011-12-29

ExperienceOn Ventures S.L.

Natural Language Question Answering System And Method Based On Deep Semantics

US20120259891A1

( en )

2011-04-11

2012-10-11

David Edoja

Method, system and program for analytics data delivering

US20120259621A1

( en )

2006-10-10

2012-10-11

Konstantin Anisimovich

Translating Texts Between Languages

US8352474B2

( en )

2010-06-16

2013-01-08

Fuji Xerox Co., Ltd.

System and method for retrieving information using a query based index

US20130013580A1

( en )

2011-06-22

2013-01-10

New Jersey Institute Of Technology

Optimized ontology based internet search systems and methods

US20130042000A1

( en )

2010-03-03

2013-02-14

Nec Corporation

Configuration information management server, configuration information management method and configuration management program

US20130125102A1

( en )

2011-11-16

2013-05-16

Fujitsu Limited

Application server

US20130145288A1

( en )

2011-12-05

2013-06-06

Zoosk, Inc., a Delaware corporation

System and Method for Identifying Nearby, Compatible Users

US20130246322A1

( en )

2012-03-15

2013-09-19

Cept Systems Gmbh

Methods, Apparatus and Products for Semantic Processing of Text

US20130332147A1

( en )

2012-06-11

2013-12-12

Nvoq Incorporated

Apparatus and Methods to Update a Language Model in a Speech Recognition System

US8620872B1

( en )

2008-09-10

2013-12-31

Amazon Technologies, Inc.

System for comparing content

US20140032219A1

( en )

2012-07-27

2014-01-30

Solomon Z. Lerner

Parsimonious Protection of Sensitive Data in Enterprise Dialog Systems

US20140046891A1

( en )

2012-01-25

2014-02-13

Sarah Banas

Sapient or Sentient Artificial Intelligence

US20140108313A1

( en )

2012-10-16

2014-04-17

Sap Ag

Ranking in cascading learning system

US20140143533A1

( en )

2012-11-16

2014-05-22

Nuance Communications, Inc.

Securing speech recognition data

US20140150117A1

( en )

2011-05-31

2014-05-29

Rakuten, Inc.

Information processing system, information processing method, information processing device, program and storage medium

US20140156614A1

( en )

2012-12-05

2014-06-05

Kirk KRAPPE

Managing structured data fields within a social media channel

US8818862B2

( en )

2008-05-01

2014-08-26

Primal Fusion, Inc.

System and method for purchasing advertisements associated with words and phrases

US20140258261A1

( en )

2013-03-11

2014-09-11

Xerox Corporation

Language-oriented focused crawling using transliteration based meta-features

US20140359554A1

( en )

2013-06-03

2014-12-04

Sap Ag

Synchronizing real and virtual software development

US8924928B1

( en )

2001-09-19

2014-12-30

Steven G. Belovich

Rule-based, run-time-alterable, client-agnostic client-server system

US20150019462A1

( en )

2013-07-15

2015-01-15

Senscio Systems

Systems and methods for semantic reasoning

US20150066475A1

( en )

2013-08-29

2015-03-05

Mustafa Imad Azzam

Method For Detecting Plagiarism In Arabic

US20150142704A1

( en )

2013-11-20

2015-05-21

Justin London

Adaptive Virtual Intelligent Agent

US20150205942A1

( en )

2012-07-12

2015-07-23

Rowem Inc.

Password Authentication System And Password Authentication Method Using Consecutive Password Authentication

US20150220515A1

( en )

2006-10-10

2015-08-06

Abbyy Infopoisk Llc

Deep model statistics method for machine translation

US20150261744A1

( en )

… the automated analysis or use of data, including use of a large language model — Unlikely Artificial Intelligence Limited (US12067362B2)

Related documents