Conceptio › Archive › Google Patents

Google Patentsopen access

Generating augmented reality experiences utilizing physical objects to … — Meta Platforms Technologies, Llc (US20260037214A1)

Meta Platforms Technologies, Llc · Google Patents

Google Patents · Patents · License: Open Access

patent, google patents, intellectual property, US20260037214A1, Meta Platforms Technologies, Llc, Andrew Lovitt, en, 2026

ABSTRACT

Abstract

This disclosure describes embodiments of methods, non-transitory computer-readable media, and systems for detecting that a physical space includes a physical object corresponding to an analogous virtual object from an augmented reality experience and rendering (or otherwise modifying) the augment reality experience to integrate the physical object as part of the experience. In particular, the disclosed systems can determine that a physical object within a physical environment corresponds to an analogous virtual object of an augmented reality experience. Based on this correspondence, the disclosed systems can modify one or more of the virtual graphics, sound, or other features corresponding to the augmented reality experience to represent the virtual object using the physical object. For example, the disclosed systems can modify acoustic features of a sound for the augmented reality experience to simulate the sound originating from (or being affected by) the physical object.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. application Ser. No. 18/464,995, filed Sep. 11, 2023, entitled âGenerating Augmented Reality Experiences Utilizing Physical Objects To Represent Analogous Virtual Objectsâ, which is a continuation of U.S. application Ser. No. 16/915,684, filed Jun. 29, 2020, now U.S. Pat. No. 11,755,275, issued Sep. 12, 2023, entitled âGenerating Augmented Reality Experiences Utilizing Physical Objects To Represent Analogous Virtual Objectsâ, each of which is hereby incorporated by reference herein.

BACKGROUND

In recent years, augmented reality systems have significantly improved the realism and detail of virtual imagery. For example, existing augmented reality systems can generate colorful and interactive augmented reality experiences that overlay virtual objects over real physical environments. In some cases, an existing augmented reality system can generate an interactive augmented reality experience for a game or simulation, where the experience includes virtual objects positioned at specific locations within a physical space. A user of an augmented-reality-computing device can view and interact with such virtual objects as part of the game or simulation.

Although conventional augmented reality systems can generate engaging and realistic augmented reality experiences, such systems often consume excessive computer processing, memory, or other computing resources to produce the realism and detail of today's augmented reality experiences. To generate a single frame of a virtual experience, for instance, some existing augmented reality systems consume much of the processing power of a Graphics Processing Unit (âGPUâ) to render the frame with high resolutions of 1920 by 1080 pixels (or greater). Because a GPU often processes at speeds slower than a general Central Processing Unit (âCPUâ), some existing augmented reality systems lack the processing power to render realistic virtual objects or entire augmented-reality experiences in real (or near-real) time.

In addition to consuming significant processing power, some existing augmented reality systems inefficiently transfer memory between main memory (e.g., host memory) and GPU dedicated memory (e.g., device memory). For example, because GPUs generally operate at a much lower clock speed than a CPU in existing augmented reality systems, transfers between host memory and device memory often have limited bandwidth and high latency. This performance bottleneck results in poorly optimized GPU-acceleration applications, such as when existing augmented reality systems generate augmented reality experiences.

Such processing speeds and memory transfers become even more difficult when existing augmented reality systems use a head-mounted device, a mobile computing device, or other smaller computing devices to render augmented reality experiences. Because computing devices require such processing and memory to extemporaneously render augmented reality, some augmented reality systems execute programs designed to produce lower resolution and less realistic virtual objects.

Beyond the computing-resource demands of virtual graphics, some existing augmented reality systems consume significant computing resources by incorporating sound into augmented reality experiences. For example, existing augmented reality systems utilize excessive processing and memory in altering sounds to simulate those sounds coming from virtual objects in an augmented reality experience. In comparison to complex sounds produced by physical objects (e.g., the complex sound of a car engine that includes multiple sound components), existing augmented reality systems consume increased computing resources in attempting to simulated complex sounds coming from a virtual object. For example, some existing systems waste significant computing resources in generating multiple audio streams corresponding to the complex sound and then altering each audio stream to simulate origination from a virtual objectâall to complete the illusion that the virtual object is creating the complex sound in the same way that a similar physical object would create the same sound.

As suggested by the computing-resource demands described above, by rigidly rendering virtual object after virtual objectâframe after frameâexisting augmented reality systems can consume loads of processing power and memory for augmented reality experiences in common physical environments. In some cases, augmented reality systems perform the same algorithms and computer processing to map a physical space and render the same virtual objectsâeven when a computing device has previously encountered the physical space and its constituent physical objects. Despite one computing device or another computing device mapping a common physical object or rendering common virtual objects, some conventional augmented reality systems often operate in isolation and do not save previously three-dimensional mappings or share such mappings or other calculations with other computing devices that may share the same physical space or virtual objects.

SUMMARY

This disclosure describes one or more embodiments of methods, non-transitory computer-readable media, and systems that solve the foregoing problems or provide other benefits. For instance, the disclosed systems can detect that a physical space includes a physical object corresponding to an analogous virtual object from an augmented reality experience and present the augment reality experience by anchoring or changing a soundâor modifying graphicsâfor the augmented reality experience to simulate the physical object as part of the experience. In particular, the disclosed systems can determine that a physical object within a physical environment corresponds to an analogous virtual object of an augmented reality experience. Based on this correspondence, the disclosed systems can modify one or more of the virtual graphics, sound, or other features corresponding to the augmented reality experience to represent the virtual object using the physical object. To integrate the physical object into the augmented reality experience, the disclosed systems can modify acoustic features of a sound for the augmented reality experience to simulate the sound originating from (or being affected by) the physical object. Additionally or alternatively, the disclosed systems can modify or omit virtual graphics to depict the physical object as part of the augmented reality experience and extemporaneously modify the augmented reality experience based on user interactions with the physical object or corresponding virtual graphic.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description provides one or more embodiments with additional specificity and detail through the use of the accompanying drawings, as briefly described below.

FIG. 1 illustrates an example environment in which an augmented reality system can operate in accordance with one or more embodiments.

FIG. 2 illustrates an overview of an augmented reality system determining a physical object corresponds to an analogous virtual object for an augmented reality experience and presenting the augmented reality experience by modifying one or more features of the experience to integrate the physical object in accordance with one or more embodiments.

FIG. 3 A illustrates a view of an augmented reality system mapping a physical environment and localizing an augmented-reality-computing device in accordance with one or more embodiments.

FIG. 3 B illustrates a schematic of an augmented reality system leveraging the process by which a user localizes a sound in accordance with one or more embodiments.

FIGS. 4 A- 4 D illustrate an augmented reality system modifying one or both of graphic and acoustic features of an augmented reality experience to integrate a physical object within the experience in accordance with one or more embodiments.

FIGS. 5 A- 5 B illustrate an augmented reality system modifying acoustic features of an augmented reality experience based on physical characteristics of a physical object in accordance with one or more embodiments.

FIGS. 6 A- 6 E illustrate an augmented reality system modifying graphic or interactive features of an augmented reality experience to integrate a physical object in accordance with one or more embodiments.

FIG. 7 illustrates a schematic diagram of an augmented reality system in accordance with one or more embodiments.

FIG. 8 illustrates a flowchart of a series of acts for determining a physical object from a physical environment corresponds to an analogous virtual object for an augmented reality experience and modifying acoustic features of a sound for the augmented reality experience to integrate the physical object into the augmented reality experience in accordance with one or more embodiments.

FIG. 9 illustrates a block diagram of an exemplary computing device in accordance with one or more embodiments.

FIG. 10 is an example network environment of an augmented reality system in accordance with one or more embodiments.

DETAILED DESCRIPTION

This disclosure describes one or more embodiments of an augmented reality system that detects a physical object from a physical environment corresponds to an analogous virtual object from an augmented reality experience and presents the augment reality experience by anchoring or changing a sound for the augmented reality experienceâor modifying or removing graphics representing the analogous virtual object for the augmented reality experienceâto integrate the physical object into the augmented reality experience. For example, the augmented reality system can anchor acoustic features (or other features) of a sound for the augmented reality experience to a physical object corresponding to an analogous virtual object from the augmented reality experience. The augmented reality system can further generate or modify graphical features of virtual objects to simulate the physical object as an interactive part of the augmented reality experience. By anchoring or changing a sound to integrate a physical object into an augmented reality experience without (or instead of) an analogous virtual object, the augmented reality system efficiently renders graphics or generates sound for the augmented reality experienceâthereby reducing the computer processing and other computing resources for conventionally rendering such an experience.

In some embodiments, for example, the augmented reality system captures a data stream corresponding to a physical environment utilizing an augmented-reality-computing device, such as a head-mounted-display device, a smart phone, or a smart tablet. By analyzing the captured data stream, the augmented reality system determines that a physical object in the physical environment corresponds to an analogous virtual object of an augmented reality experience. The augmented reality system can then signal or otherwise trigger the augmented-reality-computing device to present the augmented reality experience without utilizing the analogous virtual object. In some cases, for instance, the augmented-reality-computing device renders an augmented reality scene for display utilizing the physical object instead of the analogous virtual object. While presenting the augmented reality experience, the augmented reality system can modify acoustic features of a sound for the augmented reality experience to simulate either the sound originating from the physical object or an effect on the sound by the physical object. Additionally, or alternatively, the augmented reality system can modify or remove virtual graphics representing (or part of) the analogous virtual object for the augmented reality experience to integrate the physical object into the augmented reality experience.

To further illustrate, the augmented reality system can capture a data stream, corresponding to a physical environment, such as an image data, audio data, or data capture by environmental sensors. The augmented reality system can further map the physical environment relative to the augmented-reality-computing device to identify candidate physical objects within the physical environment. For example, the augmented reality system can map the physical environment to determine spatial relationships between features and objects of the physical environment (e.g., walls, furniture, windows, books, toys) and the augmented-reality-computing device. The augmented reality system can further recognize and analyze the physical objects within the physical environment to determine object types, object classifications, object features, and/or object characteristics.

In one or more embodiments, the augmented reality system further determines physical objects detected within a physical environment are analogous to virtual objects within a corresponding augmented reality experience. The physical object need not be identical to an analogous virtual object but share common visual characteristics. For example, the augmented reality system can analyze virtual objects within (or as part of) the augmented reality experience to determine types, classifications, features, and characteristics of the virtual objects. The physical object may also share functional characteristics with an analogous virtual object. For example, the augmented reality system can analyze virtual objects within the augmented reality experience to determine a function of one or more virtual objects. In some cases, the augmented reality system determines a physical object displays one or more images or produces audio as a function corresponding to an analogous virtual object. To illustrate, the augmented reality system can determine that (i) a function of a physical stereo system is to produce music or other auditory sounds similar to a virtual stereo system or that (ii) a function of a physical television or display screen is to display images similar to a virtual display screen.

In at least one embodiment, the augmented reality system can further identify analogous virtual objects by determining threshold matches between the types, classifications, features, functions, and characteristics of the physical objects and the virtual objects. For instance, the augmented reality system can determine a physical object matches an analogous virtual object based on an object-matching score or other appropriate techniques.

CROSS-REFERENCE TO RELATED APPLICATIONS

BACKGROUND

SUMMARY

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description provides one or more embodiments with additional specificity and detail through the use of the accompanying drawings, as briefly described below.

FIG. 1 illustrates an example environment in which an augmented reality system can operate in accordance with one or more embodiments.

FIG. 3 A illustrates a view of an augmented reality system mapping a physical environment and localizing an augmented-reality-computing device in accordance with one or more embodiments.

FIG. 3 B illustrates a schematic of an augmented reality system leveraging the process by which a user localizes a sound in accordance with one or more embodiments.

FIG. 7 illustrates a schematic diagram of an augmented reality system in accordance with one or more embodiments.

FIG. 9 illustrates a block diagram of an exemplary computing device in accordance with one or more embodiments.

FIG. 10 is an example network environment of an augmented reality system in accordance with one or more embodiments.

DETAILED DESCRIPTION

Upon detecting a physical object corresponds to an analogous virtual object from an augmented reality experience, the augmented reality system can present the augmented reality experience without some or all of the analogous virtual object. For example, the augmented reality system can generate, render, or otherwise present the augmented reality experience without utilizing the analogous virtual object. In some cases, the augmented reality system can render the augmented reality experience utilizing the physical object instead of the analogous virtual object. In some embodiments, the augmented reality system renders a portion of the analogous virtual object as an overlay on the corresponding physical object.

In addition to presenting an augmented reality experience without utilizing the analogous virtual object and instead utilizing a detected physical object, the augmented reality system can further anchor acoustic features of a sound or graphical features for the augmented reality experience to the physical object. For example, the augmented reality system can anchor or change acoustic features of a sound for the augmented reality experience to the physical object to simulate either the sound originating from the physical object or an effect on the sound by the physical object. To illustrate, the augmented reality system can anchor acoustic features of music for an augmented reality experience to a physical speaker identified in a physical environment. In one or more embodiments, the augmented reality system anchors acoustic or graphical features of the augmented reality experience by associating a location of the physical object with the anchored acoustic or graphical feature, such that any display, playback, or presentation associated with that feature within the augmented reality experience appears to originate from (or be affected by) the location and other characteristics of the physical object.

As indicated above, in certain implementations, the augmented reality system modifies acoustic features of a sound for an augmented reality experience to simulate either the sound originating from a physical object or an effect on the sound by the physical object. For example, the augmented reality system can modify acoustic features of the sound based on a distance and angle between the location of the physical object to which the sound is anchored and the augmented-reality-computing device. The augmented reality system can additionally modify acoustic features of the sound based on spectral localization cues that inform how the user understands the location of the sound, as well as on visual characteristics of the anchored physical object that may affect how the sound is heard (e.g., the size of the physical object, the direction that the physical object is pointing). In at least one embodiment, the augmented reality system can simulate sounds to be affected by a physical property of the physical object, such as with a sound that is altered to simulate that the sound originates outside of a window within the physical environment.

To further or otherwise enhance the augmented reality experience, the augmented reality system can modify graphical features of the augmented reality experience corresponding to the physical object for display within the physical environment. For example, the augmented reality system can generate a full or partial overlay for the physical object based on the analogous virtual object. In some cases, the augmented reality system generates a graphical overlay appearing similar to the analogous virtual object to modify the appearance of the physical object to simulate the analogous virtual object. The augmented reality system can further position the graphical overlay at the location of the physical object within the augmented reality experience. By positioning the graphical overlay in this manner, the augmented reality system can partially or completely obscure the underlying physical object, such as by giving a physical book a new virtual cover.

In one or more embodiments, the augmented reality system can also track user motions and interactions with (or in relation to) a physical object within an augmented reality experience. For example, the augmented reality system can track user interactions with a physical input device (e.g., the user typing on a physical keyboard) to generate new virtual objects in the augmented reality experience (e.g., a virtual graphic overlay on a computer screen that includes text corresponding to the tracked typing). In another example, the augmented reality system can track user interactions with a physical input device (e.g., a user pushing buttons on a physical game controller) to affect existing virtual objects in the augmented reality experience (e.g., virtual game characters from a virtual video game).

In at least one embodiment, the augmented reality system can further detect augmented-reality-computing devices in a shared augmented reality experience within a common physical environment. For example, the augmented reality system can detect that two separate augmented-reality-computing devices are generating the same set of augmented reality experiences within a common physical environment. In response, the augmented reality system can integrate the augmented reality experience for both devices in order for those devices to share information. Thus, the users of those augmented-reality-computing devices can cooperatively work through the same augmented reality experience within the common physical environment.

As mentioned above, the augmented reality system provides many technical advantages and benefits over conventional augmented reality systems and methods. For example, the augmented reality system improves the efficiency with which conventional augmented reality systems render and present augmented reality experiences. In comparison to conventional systems, the disclosed augmented reality system more efficiently uses and extends computing resources by selectively rendering or omitting certain virtual objects from an augmented reality experience and integrating an analogous physical object from a physical environment instead of such virtual objects. The disclosed augmented reality system can further extend computing resources by presenting or rendering only portions of a virtual object that differ from an analogous physical objectâthereby avoiding the additional computing resources needed to render a full virtual object. The disclosed augmented reality system can accordingly use a physical object in conjunction with virtual objects to create a more realistic augmented-reality experience. By modifying and generating fewer graphical features for a virtual object of an augmented reality experience based on integrating an analogous physical object, for instance, the augmented reality system saves the computer processing power and transitory memory that would have conventionally been used by existing augmented-reality-display devices to render virtual objects for the same or similar augmented reality experiences. In some cases, the augmented reality system further saves memory storage that would have been utilized in storing (or transferring memory for) three-dimensional models or other information associated with the virtual objects once rendered.

In addition to more efficient virtual renderings, in some cases, the augmented reality system improves the efficiency with which systems generate or modify sounds for augmented reality. For example, the augmented reality system can save computer processing resources by consolidating one or more audio streams of a complex sound that is anchored to a physical object and then modify the consolidated audio streams. As explained further below, in one or more embodiments, the augmented reality system generates these efficiencies in consolidating or modifying audio streams (and other acoustic sound features) by leveraging the way the human auditory system understands and interpolates sound, such that a user of the augmented reality system notices no decrease in sound quality despite sound modifications that save computer processing and memory.

Moreover, the augmented reality system avoids the rigid requirements of augmented reality experiences that are typically imposed on conventional systems. For example, augmented reality experiences are generally non-scalable. In some instances, conventional systems require rendering all virtual objects within an augmented reality scene regardless of the physical environment over which the augmented reality scene or other augmented reality experience is overlaid and the functionality of an augmented-reality-computing device. The augmented reality system overcomes this rigidity by generating augmented reality experiences that are scalable based on the contents of the current physical environment.

For example, an augmented reality experience may include a specific type of virtual speaker corresponding to music for the experience. By anchoring and modifying sound to a physical speaker from a physical environment rather than to the virtual speaker, the augmented reality system can scale down the sound quality or other characteristics for the augmented reality experience. Similarly, the augmented reality system can scale down rendering virtual objects based on the physical objects detected in a physical environment. The augmented reality system can further utilize more or fewer physical objects within a physical environment depending on the processing and memory capabilities of a given augmented-reality-computing device. Thus, the augmented reality system is more flexible than conventional systems because it can adjust an augmented reality experience to include or exclude virtual objects (or modify sounds) based on the physical objects currently available and computing device capabilities.

As illustrated by the foregoing discussion, the present disclosure utilizes a variety of terms to describe features and advantages of the augmented reality system. Additional detail is now provided regarding the meaning of such terms. For example, as used herein, âaugmented realityâ refers to a composite view including computer-generated elements real-world or physical elements from a physical environment. For instance, in one or more embodiments, the augmented reality system generates an augmented reality experience including one or more virtual objects and positions the virtual objects over the user's view within an augmented-reality-computing device. In one or more embodiments, the augmented reality system presents and/or renders an augmented reality experience utilizing a particular physical object instead of an analogous virtual object. Additionally, in at least one embodiment, the augmented reality system presents an augmented reality experience by superimposing a virtual graphic overlay over a portion of a particular physical object or over the entire particular physical object.

As used herein, an âaugmented-reality-computing deviceâ refers to a computing device that generates and presents an augmented reality experience. For example, an augmented-reality-computing device can generate, render, and/or present a display of an augmented reality experience comprising one or more virtual objects and physical objects. Additionally or alternatively, an augmented-reality-computing device can generate and/or present an audio-only augmented reality experience without virtual objects as visual components, but rather generate or present one or more virtual sounds. An augmented-reality-computing device can be a head-mounted-computing device, such as a virtual reality headset, mixed reality headset, augmented reality glasses, smart glasses, and/or a head-embedded computing device. In some cases, other computing devices can also function as augmented-reality-computing devices, such as smart phones and/or smart tables (e.g., with rear-facing cameras). In at least one embodiment, an augmented-reality-computing device also includes audio playback features (e.g., headphones, ear buds) that provide audio associated with an augmented reality experience to the user wearing the device. An augmented-reality-computing device can further include various environmental sensors (e.g., a gyroscope, an accelerometer) to enable movement detection.

As further used herein, an âaugmented reality experienceâ refers to one or more augmented reality graphics, sounds, or other features generated or provided via an augmented-reality-computing device. Such features can be part of a game experience, an educational experience, a business experience, an entertainment experience, or similar. In one or more embodiments, an augmented reality experience includes one or more augmented reality scenes, each including virtual objects and/or sounds associated with each augmented reality scene. Accordingly, as described below, this disclosure's references to augmented reality experience may comprise or constitute an augmented reality scene with one or more virtual objects. By contrast, in some cases, an augmented reality experience includes generating or presenting one or more virtual sounds without rendering or otherwise presenting virtual objects.

As just indicated, an augmented reality experience can include an augmented reality scene. An âaugmented reality sceneâ refers to a composite image or view comprising one or more virtual objects and physical (or real-world) objects. In some cases, an augmented reality scene comprises a three-dimensional image or environment comprising both a virtual object and a physical object with which a user can interact using computer detection or environmental sensors. In one or more embodiments, an augmented reality scene further includes or corresponds to one or more sounds that further inform or enhance the augmented reality scene. For example, a sound for an augmented reality scene can include music, sound effects, human speech, and any other type of sound.

As used herein, a âvirtual objectâ refers to a computer-generated-graphical object rendered as part of an augmented reality scene or other augmented reality experience. For example, a virtual object may include an object generated by a computing device for display within an augmented reality scene or for use within an augmented reality application. Such virtual objects may be, but are not limited to, virtual accessories, animals, books, electronic devices, vehicles, windows, or any other graphical object created by a computer. A virtual object can have features, characteristics, and other qualities (e.g., as defined by a model, a file, a database).

As used herein, an âanalogous virtual objectâ is a virtual object for an augmented reality experience that is determined to be an analog of a corresponding physical object in a physical environment. For example, an analogous virtual object may or may not be identical to a corresponding physical object. In at least one embodiment, an analogous virtual object shares at least one feature and/or characteristic of a corresponding physical object.

In one or more embodiments, the augmented reality system can modify acoustic features based on spectral localization cues. As discussed below, âspectral localization cuesâ refer cues that inform or stimulate how the human brain localizes sound outside of the human head. Spectral localization cues are generally individual to a user and include how the user's head and the intricacies of his or her ears effect the frequencies that eventually reach the user's ear drums. For example, due to the complexities of the human ear (e.g., the shape of the outer ear with its concave and asymmetrical folds), a person may only hear a subset of the spectrum of frequencies within a single sound. A different user may hear a different subset of spectrum of frequencies within the same sound because of physical differences in his or her ears. The way that the person hears and locates sound is further affected by the size and shape of his or her head, which sound must travel around to reach both ears. In at least one embodiment, the augmented reality system utilizes average spectral localization cues (e.g., based on an average ear and head size) to modify the originating location of a sound.

As used herein, a âphysical environmentâ refers to a physical space surrounding or within proximity to a user detected in whole or in party by an augmented-reality-computing device. In some embodiments, a physical environment includes physical objects located in a physical space detected by a camera, microphone, or other sensor of an augmented-reality-computing device. A physical environment can be indoors (e.g., a bedroom, and office, a classroom) or outdoors (e.g., a park, a beach, a playground, a shopping mall). A physical environment can include area indicators (e.g., a floor, walls, a ceiling), which define the area or confines of the physical environment, and physical objects, which reside within the defined area or confines of the physical environment.

As used herein, a âphysical objectâ refers to a real-world article in a physical area. Such physical objects may be, but are not limited to, physical accessories, animals, books, electronic devices, vehicles, windows, or any other tangible or physical object in the real world. In some cases, physical objects may be free-standing or may be positioned on other physical objects (e.g., as a lamp may be positioned on desk). Physical objects can have classifications, types, features, and characteristics, as discussed below.

As used herein, âacoustic featuresâ refers to sound components present in (or detected from) a sound. For example, acoustic features of a sound may include an amplitude for the sound, one or more frequencies that make up the sound, the volume of the sound, timbre of the sound, the reverberation of the sound, or the color or loudness of the sound. In at least one embodiment, acoustic features of a complex sound may include two or more audio streams that represent sub-sounds within the complex sound.

As used herein, a âsound profileâ refers to acoustic instructions associated with a virtual object or other object. For example, a sound profile associated with a virtual object can inform how sounds originating from the virtual object should sound. As such, the virtual object's sound profile may include various acoustic features, such as a sound volume, a level of sound degradation, a level of sound enhancement, and various level specifications (e.g., associated with treble levels, bass levels).

As used herein, a âdata streamâ refers to a sequence of data captured by an augmented-reality-computing device. In one or more embodiments, a data stream can include an image stream captured by a camera or other image-capturing device, an audio stream captured by a microphone or other audio input, or a data stream captured by one or more environmental sensors associated with the augmented-reality-computing device. For example, a data stream may include optical data captured by an optical sensor or laser data captured by a laser scanner. In either case, the data stream may be captured as part of a simultaneous location and mapping (âSLAMâ). As a further example, an environmental data stream from a gyroscope of an augmented-reality-computing device can include a stream of data indicating a real-time tilt and orientation associated with the augmented-reality-computing device. A data stream may be continuous or intermittent or have a starting point or capture and ending point of capture. For example, an augmented-reality-computing device may capture one or more intermittent sequences of data in response to detecting movement (e.g., while the user is moving his or her head), and then return to a passive mode where data sequences are no longer captured. Alternatively, an augmented-reality-computing device can capture one or more data streams continuously.

As noted above, a data stream may include an image stream or an audio stream. As used herein, an âimage streamâ refers to a sequence of images captured by (or received from) a camera or other image-capturing device. In some case, an image stream includes a sequence of still images captured by a camera divide (e.g., a micro-camera associated with an augmented-reality-computing device). In at least one embodiment, an image stream can be provided by a camera in real time or near-real time. Additionally, as used herein, an âaudio streamâ refers to a sequence of data comprising audio information. In some cases, an audio stream includes a sequence of data captured by a microphone that a computing device encodes or transforms into data packets comprising audio information (e.g., acoustic tones and/or frequencies).

FIG. 1 illustrates an example block diagram of an environment 100 for implementing an augmented reality system 102 . As illustrated in FIG. 1 , the augmented reality system 102 includes augmented-reality-computing devices 106 a and 106 b, and server(s) 104 , which are communicatively coupled through a network 110 . As shown in FIG. 1 , the augmented-reality-computing devices 106 a and 106 b include augmented reality applications 108 a and 108 b, respectively. Additionally shown in FIG. 1 , the server(s) 104 includes an augmented reality system 102 . Further shown in FIG. 1 , the augmented-reality-computing devices 106 a and 106 b are associated with users 112 a and 112 b, respectively.

The augmented-reality-computing devices 106 a and 106 b, and the server(s) 104 communicate via the network 110 , which may include one or more networks and may use one or more communication platforms or technologies suitable for transmitting data and/or communication signals. In one or more embodiments, the network 110 includes the Internet or World Wide Web. The network 110 , however, can include various other types of networks that use various communication technologies and protocols, such as a corporate intranet, a virtual private network (âVPNâ), a local area network (âLANâ), a wireless local network (âWLANâ), a cellular network, a wide area network (âWANâ), a metropolitan area network (âMANâ), or a combination of two or more such networks.

Although FIG. 1 illustrates a particular arrangement of the augmented-reality-computing devices 106 a and 106 b, the server(s) 104 , and the network 110 , various additional arrangements are possible. For example, the augmented-reality-computing devices 106 a and 106 b may directly communicate with the augmented reality system 102 , bypassing the network 110 . Further, the environment 100 can include any number of augmented-reality-computing devices communicating with the augmented reality system 102 . Additional details relating to the network 110 are explained below with reference to FIG. 10 .

Although FIG. 1 illustrates the augmented reality system 102 hosted by the server(s) 104 , the functionality of the augmented reality system 102 may reside elsewhere. For example, some or all of the functionality of the augmented reality system 102 may be performed by the augmented reality applications 108 a and 108 b on the augmented-reality-computing devices 106 a and 106 b, respectively. Thus, the augmented-reality-computing devices 106 a and 106 b can generate and display or otherwise present augmented reality experiences in the absence of a network connection to the augmented reality system 102 . Additionally or alternatively, the augmented-reality-computing devices 106 a and 106 b can provide an image stream of a physical environment to the augmented reality system 102 via the network 110 , and then receive and display data for an augmented reality experience generated by the augmented reality system 102 . Additionally or alternatively, the augmented-reality-computing devices 106 a and 106 b may receive data comprising computer-executable rendering instructions from the augmented reality system 102 and generate a rendering of an augmented reality experience based on the rendering instructions.

As suggested above, the augmented-reality-computing devices 106 a and 106 b each include an augmented reality display, a video capturing device (e.g., a digital camera), and an audio playback mechanism (e.g., headphones). For example, in one or more embodiments, the augmented reality display of the augmented-reality-computing devices 106 a and 106 b displays a virtual graphic overlay displayed in connection with the wearer's normal view. In at least one embodiment, the augmented reality display operates as a pair of lenses (e.g., eye glass lenses, contact lenses) positioned over the wearer's eyes. Additionally, in one or more embodiments, the video capturing devices associated with the augmented-reality-computing devices 106 a and 106 b are micro digital video cameras mounted (e.g., to an earpiece, or over the bridge of the wearer's nose) to the augmented-reality-computing devices 106 a and 106 b, respectively. Further, the audio playback mechanism of the augmented-reality-computing devices 106 a and 106 b may include right and left headphones, ear buds, or speakers built into a portion of the augmented-reality-computing devices 106 a and 106 b (e.g., built into the earpieces). Thus, in some embodiments, the augmented-reality-computing devices 106 a and 106 b are similar to eyeglasses with all the component parts built-in. In one or more embodiments, the augmented-reality-computing devices 106 a and 106 b also include at least one processor capable of executing software code.

As mentioned above, in some embodiments, the augmented reality system 102 anchors acoustic or graphical features of an augmented reality experience to a physical object in a physical environment based on the physical object being analogous to a virtual object in the augmented reality experience. More specifically, the augmented reality system 102 can render or otherwise present the augmented reality experience without utilizing the analogous virtual object, but rather utilizing the physical object. In accordance with one or more embodiments, FIG. 2 illustrates an overview of the augmented reality system 102 determining a physical object from a physical environment corresponds to an analogous virtual object for an augmented reality experience and presenting the augmented reality experience by modifying one or more features of the experience to integrate the physical object.

As depicted, FIG. 2 illustrates the augmented reality system 102 capturing a data stream from an augmented-reality-computing device 202 . In one or more embodiments, the augmented reality system 102 can capture an image stream via a camera of the augmented-reality-computing device. The augmented reality system 102 can receive the image steam over a network connection with the augmented-reality-computing device. Additionally or alternatively, the augmented reality system 102 can capture and process the image stream from the camera at the augmented-reality-computing device.

The augmented reality system 102 can further determine a physical object corresponds to an analogous virtual object 204 . More specifically, the augmented reality system 102 can determine that a physical object in the physical environment corresponds to an analogous virtual object in the augmented reality experience. In one or more embodiments, the augmented reality system 102 makes this determination in part by mapping the physical environment to identify the physical objects in the physical environment. For example, the augmented reality system 102 can utilize or implement a SLAM system to extract area indicators (e.g., walls, floor, ceiling) and objects (e.g., windows, furniture, books, dishes, toys, TVs) of the physical environment, determine a location of the augmented-reality-computing device within the physical environment, and calculate distances (e.g., horizontal, vertical, and angular) between the augmented-reality-computing device and the extracted area indicators and objects.

In one or more embodiments, the augmented reality system 102 further inventories the physical objects in the physical environment. For example, the augmented reality system 102 can utilize image analysis, web-lookups, and other techniques to identify and classify the physical environment objects.

For instance, utilizing any of these techniques, the augmented reality system 102 determines (i) that a particular shape or outline in the physical environment is an object and (ii) a category or classification associated with the object based on broad categories or classifications, such as âfurniture,â âbook,â âdÃ©cor.â Based on the broad classification of the object, the augmented reality system 102 can further determine additional features and characteristics of the object, such as the functionality of the object, the physical limitations of the object, and so forth. In at least one embodiment, the augmented reality system 102 can store all this information in association with the identified physical object for later use in generating and presenting an augmented reality experience.

Similarly, the augmented reality system 102 can inventory virtual objects associated with an augmented reality experience. For example, the augmented reality system 102 can access an augmented reality scene of an augmented reality experience to determine one or more virtual objects associated with the augmented reality scene. In one or more embodiments, the augmented reality system 102 can analyze metadata, display instructions, and other information associated with the augmented reality scene to identify virtual objects included in the augmented reality scene. The augmented reality system 102 can further identify a type or classification of the virtual objects based on image analysis, metadata, or other display instructions associated with the augmented reality scene. Based on the identified type or classification, the augmented reality system 102 can further determine features and characteristics of the virtual objects.

The augmented reality system 102 can then determine that one or more physical objects of the physical environment correspond to one or more virtual objects of the augmented reality experience based on the identified characteristics and features of both the virtual objects of the augmented reality experience and the physical objects of the physical environment. For example, the augmented reality system 102 can calculate an object-matching score between each physical object and each virtual object indicating a degree to which one or more features or characteristics of each physical object match one or more features or characteristics of each virtual object.

Briefly, in some cases, the augmented reality system 102 can calculate the object-matching score between a physical object and a virtual object by adding a point or value to the object-matching score for each matching characteristic and/or feature identified between the two objects. In at least one embodiment, the augmented reality system 102 can further weight the point or value based on a relevancy associated with the matching characteristic and/or feature (e.g., as with a characteristic and/or feature indicating appearance or function).

In one or more embodiments, the augmented reality system 102 determines that a particular physical object corresponds to a particular virtual object when the object-matching score between the two objects satisfies an object-matching threshold. If the object-matching score associated with a particular physical object and a particular virtual object satisfies the object-matching threshold, the augmented reality system 102 can determine that the physical object corresponds to the analogous virtual object.

As further shown in FIG. 2 , the augmented reality system 102 can generate or modify various acoustic or graphical features for the augmented reality experience based on the physical object corresponding to the analogous virtual object. For example, in one or more embodiments, the augmented reality system 102 can modify one or more acoustic features of a sound for the augmented reality experience to simulate that the sound originates from the physical object 206 . In at least one embodiment, the augmented reality system 102 can modify the acoustic features of the sound based on horizontal, vertical, and angular distances between the location of the physical object and the augmented-reality-computing device, as well as on other spectral localization cues. The augmented reality system 102 can further modify the acoustic features based on characteristics of the physical object. In some cases, the overall effect of modifying the acoustic features is to simulate, from the perspective of the user of the augmented-reality-computing device, that the sound originates from the physical object, even though the physical object is not actually producing the sound.

In additional or alternative embodiments, the augmented reality system 102 can generate or modify virtual objects based on tracking user interactions with the physical object 208 . For example, the augmented reality system 102 can track user interactions with the physical object as part of a game or other augmented reality experience. To illustrate but one example, the augmented reality system 102 can track user interactions with a physical remote control to change the television channel displayed on a virtual television screen within the augmented reality experience.

In an additional or alternative embodiment, the augmented reality system 102 can generate a virtual graphic overlay associated with the virtual object 210 . For example, the augmented reality system 102 can generate the virtual graphic overlay based on the analogous virtual object to cover or obscure all or a portion of the physical object when the virtual graphic overlay is positioned at the location of the physical object. The augmented reality system 102 can generate the virtual graphic overlay based on visual characteristics of the analogous virtual object, such that the overlay causes the physical object to appear differently to the user of the augmented-reality-computing device. Additionally or alternatively, the augmented reality system 102 can generate the virtual graphical overlay based on a difference between the physical object and the analogous virtual object by rendering only a portion of a virtual object that differs from an analogous physical object. In at least one embodiment, the augmented reality system 102 can update or replace the virtual graphic overlay based on further user interactions with an area of the physical object on which the virtual graphic overlay is superimposed.

As further shown in FIG. 2 , the augmented reality system 102 can further present the augmented reality experience without the analogous virtual object 212 . As suggested above, in some cases, the augmented reality system 102 presents the augmented reality experience using the physical object rather than an analogous virtual object. For example, in response to determining the object-matching score between the two objects is greater than or equal to the object-matching threshold, the augmented reality system 102 can determine that the virtual object is analogous to the physical object and anchor one or more features of the augmented reality scene to the physical object rather than rendering the analogous virtual object. For instance, the augmented reality system 102 can associate the one or more features of the augmented reality experience with the location of the physical object, as determined via SLAM or a similar algorithm. In at lea

CLAIMS

Claims ( 21 )

1 . (canceled)

2 . A method, comprising:

generating an augmented reality (AR) experience within a physical environment via a first augmented-reality computing device associated with a first user; obtaining data indicating a second augmented-reality computing device associated with a second user is within the physical environment; generating a virtual object within the AR experience via the first augmented-reality computing device; causing presentation of the virtual object to the first user; transmitting information regarding the virtual object generated by the first augmented-reality computing device to the second augmented-reality computing device; and causing presentation of the virtual object generated by the first augmented-reality computing device to the second user via the second augmented-reality computing device.

3 . The method of claim 2 , further comprising:

generating a virtual indication within the AR experience identifying that the virtual object is within a field of view of the first augmented-reality computing device and a field of view of the second augmented-reality computing device; obtaining data indicating an interaction between the first user and the virtual object; generating another virtual object within the AR experience based on the interaction; and generating instructions configured to cause the other virtual object to be displayed within the field of view of the second augmented-reality computing device at the second augmented-reality computing device.

4 . The method of claim 3 , further comprising:

obtaining data indicating an interaction with the other virtual object by the second user of the second augmented-reality computing device; and generating a third virtual object within the AR experience based on the interaction with the other virtual object.

5 . The method of claim 4 , wherein:

the third virtual object is displayed within the field of view of the first augmented-reality computing device and not displayed within the field of view of the second augmented-reality computing device.

6 . The method of claim 2 , wherein:

obtaining data indicating the first augmented-reality computing device is within the AR experience includes receiving data via one or more sensors at the first augmented-reality computing device including at least one of an imaging device, connectivity sensors, and proximity sensors.

7 . The method of claim 3 , wherein:

the virtual object is displayed at a first location within the AR experience such that a first portion of the virtual object is configured to be visible via the field of view of the first augmented-reality computing device and a second portion of the virtual object is configured to be visible via the field of view of the second augmented-reality computing device.

8 . The method of claim 3 further comprising:

obtaining data indicating a third augmented-reality computing device is within proximity to the first augmented-reality computing device; and

in accordance with a determination that the third augmented-reality computing device is not within the AR experience, forgo displaying a fourth virtual object within the field of view of the third augmented-reality computing device.

9 . The method of claim 2 , further comprising:

generating an additional virtual object within the AR experience; causing display of the additional virtual object at the display of the first augmented-reality computing device; and forgo generating instructions configured to display the additional virtual object at the second augmented-reality computing device.

10 . A non-transitory computer readable storage medium including instructions that, when executed by a first augmented-reality computing device, cause the first augmented-reality computing device to:

generate an augmented reality (AR) experience within a physical environment via a first augmented-reality computing device associated with a first user; obtain data indicating a second augmented-reality computing device associated with a second user is within the physical environment; generate a virtual object within the AR experience via the first augmented-reality computing device; cause presentation of the virtual object to the first user; transmit information regarding the virtual object generated by the first augmented-reality computing device to the second augmented-reality computing device; and cause presentation of the virtual object generated by the first augmented-reality computing device to the second user via the second augmented-reality computing device.

11 . The non-transitory computer readable storage medium of claim 10 , further including instructions that cause the first augmented-reality computing device to:

generate a virtual indication within the AR experience identifying that the virtual object is within a field of view of the first augmented-reality computing device and a field of view of the second augmented-reality computing device; obtain data indicating an interaction between the first user and the virtual object; generate another virtual object within the AR experience based on the interaction; and generate instructions configured to cause the other virtual object to be displayed within the field of view of the second augmented-reality computing device at the second augmented-reality computing device.

12 . The non-transitory computer readable storage medium of claim 11 , further including instructions that cause the first augmented-reality computing device to:

obtain data indicating an interaction with the other virtual object by the second user of the second augmented-reality computing device; and generate a third virtual object within the AR experience based on the interaction with the other virtual object.

13 . The non-transitory computer readable storage medium of claim 12 , wherein:

14 . The non-transitory computer readable storage medium of claim 10 , wherein:

15 . The non-transitory computer readable storage medium of claim 11 , wherein:

16 . The non-transitory computer readable storage medium of claim 11 , further including instructions that cause the first augmented-reality computing device to:

obtain data indicating a third augmented-reality computing device is within proximity to the first augmented-reality computing device; and in accordance with a determination that the second augmented-reality computing device is not within the AR experience, forgo displaying a fourth virtual object within the field of view of the third augmented-reality computing device.

17 . The non-transitory computer readable storage medium of claim 10 , further including instructions that cause the first augmented-reality computing device to:

generate an additional virtual object within the AR experience; cause display of the additional virtual object at the display of the first augmented-reality computing device; and forgo generating instructions configured to display the additional virtual object at the second augmented-reality computing device.

18 . A first augmented-reality computing device, comprising:

one or more sensors; one or more programs, wherein the one or more programs are stored in memory and configured to be executed by one or more processors, the one or more programs including instructions for:

generating an augmented reality (AR) experience within a physical environment via a first augmented-reality computing device associated with a first user;

obtaining data, via the one or more sensors, indicating a second augmented-reality computing device associated with a second user is within the physical environment;

generating a virtual object within the AR experience via the first augmented-reality computing device;

causing presentation of the virtual object to the first user;

transmitting information regarding the virtual object generated by the first augmented-reality computing device to the second augmented-reality computing device; and

causing presentation of the virtual object generated by the first augmented-reality computing device to the second user via the second augmented-reality computing device.

19 . The first augmented-reality computing device of claim 18 , wherein the one or more programs also include instructions for:

20 . The first augmented-reality computing device of claim 19 , wherein the one or more programs also include instructions for:

21 . The first augmented-reality computing device of claim 20 , wherein:

US19/300,535

2020-06-29

2025-08-14

Generating augmented reality experiences utilizing physical objects to represent analogous virtual objects

Pending

US20260037214A1

( en )

Priority Applications (1)

Application Number

Priority Date

Filing Date

Title

US19/300,535

US20260037214A1

( en )

2020-06-29

2025-08-14

Generating augmented reality experiences utilizing physical objects to represent analogous virtual objects

Applications Claiming Priority (3)

Application Number

Priority Date

Filing Date

Title

US16/915,684

US11755275B2

( en )

2020-06-29

Generating augmented reality experiences utilizing physical objects to represent analogous virtual objects

US18/464,995

US20240036814A1

( en )

2020-06-29

2023-09-11

Generating augmented reality experiences utilizing physical objects to represent analogous virtual objects

US19/300,535

US20260037214A1

( en )

2020-06-29

2025-08-14

Generating augmented reality experiences utilizing physical objects to represent analogous virtual objects

Related Parent Applications (1)

Application Number

Title

Priority Date

Filing Date

US18/464,995

Continuation

US20240036814A1

( en )

2020-06-29

2023-09-11

Generating augmented reality experiences utilizing physical objects to represent analogous virtual objects

Publications (1)

Publication Number

Publication Date

US20260037214A1

true

US20260037214A1 ( en )

2026-02-05

Family

ID=79031927

Family Applications (3)

Application Number

Title

Priority Date

Filing Date

US16/915,684

Active

2040-07-24

US11755275B2

( en )

2020-06-29

Generating augmented reality experiences utilizing physical objects to represent analogous virtual objects

US18/464,995

Abandoned

US20240036814A1

( en )

2020-06-29

2023-09-11

Generating augmented reality experiences utilizing physical objects to represent analogous virtual objects

US19/300,535

Pending

US20260037214A1

( en )

2020-06-29

2025-08-14

Generating augmented reality experiences utilizing physical objects to represent analogous virtual objects

Family Applications Before (2)

Application Number

Title

Priority Date

Filing Date

US16/915,684

Active

2040-07-24

US11755275B2

( en )

2020-06-29

Generating augmented reality experiences utilizing physical objects to represent analogous virtual objects

US18/464,995

Abandoned

US20240036814A1

( en )

2020-06-29

2023-09-11

Generating augmented reality experiences utilizing physical objects to represent analogous virtual objects

Country Status (1)

Country

Link

( 3 )

US11755275B2

( en )

Families Citing this family (23)

* Cited by examiner, â Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US10743131B2

( en )

2016-09-06

2020-08-11

Flying Eye Reality, Inc.

Social media systems and methods and mobile devices therefor

DK180639B1

( en )

2018-06-01

2021-11-04

Apple Inc

DISABILITY OF ATTENTION-ATTENTIVE VIRTUAL ASSISTANT

US11315326B2

( en )

2019-10-15

2022-04-26

At&T Intellectual Property I, L.P.

Extended reality anchor caching based on viewport prediction

US11250634B1

( en )

2020-07-28

2022-02-15

Rovi Guides, Inc.

Systems and methods for automated insertion of supplemental content into a virtual environment using a machine learning model

US12602143B2

( en )

2021-10-13

2026-04-14

Comake, Inc.

Methods to infer content relationships from user actions and system automations

US11582571B2

( en )

2021-05-24

2023-02-14

International Business Machines Corporation

Sound effect simulation by creating virtual reality obstacle

WO2023277897A1

( en )

2021-06-30

2023-01-05

Hewlett-Packard Development Company, L.P.

Replacing synthetic object within xr environment

WO2023034224A1

( en )

2021-08-31

2023-03-09

Chinook Labs Llc

Digital assistant for providing graphical overlays of video events

US11501530B1

( en )

2021-09-08

2022-11-15

International Business Machines Corporation

Spatio-temporal relation based MR content positioning

US12572326B2

( en )

2021-09-16

2026-03-10

Apple Inc.

Digital assistant for moving and copying graphical elements

TW202324172A

( en )

2021-11-10

2023-06-16

ç¾ååå¹³å°æè¡æéå¬å¸

Automatic artificial reality world creation

US12051157B2

( en )

2021-11-18

2024-07-30

Yap Studios, LLC

Meta-binding spatial publishing platform

WO2023105653A1

( en )

2021-12-07

2023-06-15

ãã¯ã»ã«æ ªå¼ä¼ç¤¾

Head-mounted display, head-mounted display system, and display method for head-mounted display

CN119404492A

( en )

2022-05-03

2025-02-07

åä¸ºææ¯æéå¬å¸

Method and apparatus for scalable semantically aware augmented reality (AR) + Internet system

US11922587B2

( en )

2022-05-10

2024-03-05

Snap Inc.

Dynamic augmented reality experience

US11960653B2

( en )

2022-05-10

2024-04-16

Snap Inc.

Controlling augmented reality effects through multi-modal human interaction

GB2621112B

( en )

2022-07-25

2024-12-11

Sony Interactive Entertainment Europe Ltd

Identifying devices in a mixed-reality environment

US12541281B2

( en )

2022-09-07

2026-02-03

Meta Platforms Technologies, Llc

Opportunistic adaptive tangible user interfaces for use in extended reality environments

US11988833B2

( en )

2022-10-04

2024-05-21

Snap Inc.

AR system bending correction

US20240202986A1

( en )

2022-12-20

2024-06-20

Rovi Guides, Inc.

Systems and methods for conceptualizing a virtual or live object

US12125096B2

( en )

2023-01-12

2024-10-22

Flying Eye Reality, Inc.

Augmented reality social media platform, systems, and methods related thereto

WO2024241106A1

( en )

2023-05-19

2024-11-28

Sony Group Corporation

Environmental artificial intelligence soundscape experience

US20250039626A1

( en )

2023-07-26

2025-01-30

Ambit Microsystems (Shanghai) Ltd.

Method and system for simulating spatial sound in augmented reality and computer-readable storage medium

Family Cites Families (18)

* Cited by examiner, â Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US7818077B2

( en )

2004-05-06

2010-10-19

Valve Corporation

Encoding spatial data in a multi-channel sound file for an object in a virtual environment

US9037468B2

( en )

2008-10-27

2015-05-19

Sony Computer Entertainment Inc.

Sound localization for user in motion

US9122053B2

( en )

2010-10-15

2015-09-01

Microsoft Technology Licensing, Llc

Realistic occlusion for a head mounted augmented reality display

US9338420B2

( en )

2013-02-15

2016-05-10

Qualcomm Incorporated

Video analysis assisted generation of multi-channel audio data

US20140328505A1

( en )

2013-05-02

2014-11-06

Microsoft Corporation

Sound field adaptation based upon user tracking

US9971853B2

( en )

2014-05-13

2018-05-15

Atheer, Inc.

Method for replacing 3D objects in 2D environment

US9906885B2

( en )

2016-07-15

2018-02-27

Qualcomm Incorporated

Methods and systems for inserting virtual sounds into an environment

US10606449B2

( en )

2017-03-30

2020-03-31

Amazon Technologies, Inc.

Adjusting audio or graphical resolutions for data discovery

US10154360B2

( en )

2017-05-08

2018-12-11

Microsoft Technology Licensing, Llc

Method and system of improving detection of environmental sounds in an immersive environment

JP7069856B2

( en )

2018-03-08

2022-05-18

Fixing device and image forming device

US10546431B2

( en )

2018-03-29

2020-01-28

Disney Enterprises, Inc.

Systems and methods to augment an appearance of physical object for an augmented reality experience

US10237675B1

( en )

2018-05-22

2019-03-19

Microsoft Technology Licensing, Llc

Spatial delivery of multi-source audio content

US11651555B2

( en )

2018-05-31

2023-05-16

Microsoft Technology Licensing, Llc

Re-creation of virtual environment through a video call

CN112219205B

( en )

2018-06-05

2022-10-25

å¥è·å¬å¸

Matching of content to a spatial 3D environment

US11210855B2

( en )

2018-06-29

2021-12-28

Ssam Sports, Inc.

Analyzing 2D movement in comparison with 3D avatar

US11337023B2

( en )

2019-12-20

2022-05-17

Magic Leap, Inc.

Physics-based audio and haptic synthesis

WO2021163289A1

( en )

2020-02-13

2021-08-19

Magic Leap, Inc.

Cross reality system for large scale environments

US11361735B1

( en )

2020-04-15

2022-06-14

Apple Inc.

Head-mountable device with output for distinguishing virtual and physical objects

2020

2020-06-29

US16/915,684

patent/US11755275B2/en

active

Active

2023

2023-09-11

US18/464,995

patent/US20240036814A1/en

not_active

Abandoned

2025

2025-08-14

US19/300,535

patent/US20260037214A1/en

active

Pending

Also Published As

Publication number

Publication date

US20240036814A1

( en )

2024-02-01

US20210405959A1

( en )

2021-12-30

US11755275B2

( en )

2023-09-12

Generating augmented reality experiences utilizing physical objects to … — Meta Platforms Technologies, Llc (US20260037214A1)

Related documents