Google Patentsopen access

… as message objects to facilitate communications sessions in an augmented … — Magical Technologies, Llc (US11249714B2)

ABSTRACT

Abstract

Systems and methods of shareable virtual objects and virtual objects as message objects to facilitate communications sessions in an augmented reality environment are disclosed. In one aspect, embodiments of the present disclosure include a method, which may be implemented on a system, to facilitate sharing of an object in an augmented reality environment of a real world environment. The method can further include world-locking the object or user-locking the object in accordance with a request of the object to be posted in the augmented reality environment for a given user. In world-locking the object, the object is enabled to be interacted with by the given user if and when the given user is at or in the vicinity of the given location. In user-locking the object, the object is enabled to be interacted with by the given user regardless of where the given user is physically located in the real world environment.

Description

CLAIM OF PRIORITY

This application claims the benefit of:

* U.S. Provisional Application No. 62/557,775, filed Sep. 13, 2017 and entitled “Systems and Methods of Augmented Reality Enabled Applications Including Social Activities or Web Activities and Apparatuses of Tools Therefor,” (8004.US00), the contents of which are incorporated by reference in their entirety;

* U.S. Provisional Application No. 62/575,458, filed Oct. 22, 2017 and entitled “Systems, Methods and Apparatuses of Single directional or Multi-directional Lens/Mirrors or Portals between the Physical World and a Digital World of Augmented Reality (AR) or Virtual Reality (VR) Environment/Objects; Systems and Methods of On-demand Curation of Crowdsourced (near) Real time Imaging/Video Feeds with Associated VR/AR Objects; Systems and Methods of Registry, Directory and/or Index for Augmented Reality and/or Virtual Reality Objects,” (8005.US00), the contents of which are incorporated by reference in their entirety; and

* U.S. Provisional Application No. 62/581,989, filed Nov. 6, 2017 and entitled “Systems, Methods and Apparatuses of: Determining or Inferring Device Location using Digital Markers; Virtual Object Behavior Implementation and Simulation Based on Physical Laws or Physical/Electrical/Material/Mechanical/Optical/Chemical Properties; User or User Customizable 2D or 3D Virtual Objects; Analytics of Virtual Object Impressions in Augmented Reality and Applications; Video objects in VR and/or AR and Interactive Multidimensional Virtual Objects with Media or Other Interactive Content,” (8006.US00), the contents of which are incorporated by reference in their entirety.

* U.S. Provisional Application No. 62/613,595, filed Jan. 4, 2018 and entitled “Systems, methods and apparatuses of: Creating or Provisioning Message Objects Having Digital Enhancements Including Virtual Reality or Augmented Reality Features and Facilitating Action, Manipulation, Access and/or Interaction Thereof,” (8008.US00), the contents of which are incorporated by reference in their entirety.

* U.S. Provisional Application No. 62/621,470, filed Jan. 24, 2018 and entitled “Systems, Methods and Apparatuses to Facilitate Gradual and Instantaneous Change or Adjustment in Levels of Perceptibility of Virtual Objects and Reality Object in a Digital Environment,” (8009.US00), the contents of which are incorporated by reference in their entirety.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is related to U.S. application Ser. No. 16/130,541, also filed on Sep. 13, 2018 and entitled “Systems And Methods Of Virtual Billboarding And Collaboration Facilitation In An Augmented Reality Environment,” the contents of which are incorporated by reference in their entirety.

This application is related to U.S. application Ser. No. 16/160,582, also filed on Sep. 13, 2018 and entitled “Systems And Methods Of Rewards Object Spawning And Augmented Reality Commerce Platform Supporting Multiple Seller Entities”, the contents of which are incorporated by reference in their entirety.

This application is related to PCT Application no. PCT/US18/50952, also filed on Sep. 13, 2018 and entitled “Systems And Methods Of Shareable Virtual Objects and Virtual Objects As Message Objects To Facilitate Communications Sessions In An Augmented Reality Environment”, the contents of which are incorporated by reference in their entirety.

This application is related to PCT Application no. PCT/US2018/44844, filed Aug. 1, 2018 and entitled “Systems, Methods and Apparatuses to Facilitate Trade or Exchange of Virtual Real-Estate Associated with a Physical Space”, the contents of which are incorporated by reference in their entirety.

This application is related to PCT Application no. PCT/US2018/45450, filed Aug. 6, 2018 and entitled “Systems, Methods and Apparatuses for Deployment and Targeting of Context-Aware Virtual Objects and/or Objects and/or Behavior Modeling of Virtual Objects Based on Physical Principles”, the contents of which are incorporated by reference in their entirety.

TECHNICAL FIELD

The disclosed technology relates generally to augmented reality environments and virtual objects that are shareable amongst users.

BACKGROUND

The advent of the World Wide Web and its proliferation in the 90's transformed the way humans conduct business, live lives, consume/communicate information and interact with or relate to others. A new wave of technology is on the cusp of the horizon to revolutionize our already digitally immersed lives.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example block diagram of a host server able to deploy virtual objects for various applications, in accordance with embodiments of the present disclosure.

FIG. 2A depicts an example diagram showing an example of a virtual object to facilitate an augmented reality experience including a communications session and an example of a virtual object which includes a shareable note in an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 2B depicts an example diagram illustrating an example of virtual object posted in an augmented reality environment for a user by another entity, in accordance with embodiments of the present disclosure.

FIG. 2C depicts an example diagram depicting collaboration facilitated through a virtual object in an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 2D depicts an example diagram of a marketplace administered in an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 2E depicts an example diagram showing an example user experience flow for creating virtual objects, managing a collection of virtual objects, sharing and posting virtual objects or responding to virtual objects, in accordance with embodiments of the present disclosure.

FIG. 3A depicts an example functional block diagram of a host server that deploys and administers virtual objects for various disclosed applications, in accordance with embodiments of the present disclosure.

FIG. 3B depicts an example block diagram illustrating the components of the host server that deploys and administers virtual objects for various disclosed applications, in accordance with embodiments of the present disclosure.

FIG. 4A depicts an example functional block diagram of a client device such as a mobile device that enables virtual object manipulation and/or virtual object collection for various disclosed applications, in accordance with embodiments of the present disclosure.

FIG. 4B depicts an example block diagram of the client device, which can be a mobile device that enables virtual object manipulation and/or virtual object collection for various disclosed applications, in accordance with embodiments of the present disclosure.

FIG. 5A graphically depicts diagrammatic examples showing user experience flows in navigating an example user interface for accessing, viewing or interacting with an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 5B graphically depicts example user interfaces for logging in to, signing up for and viewing a user profile in an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 5C graphically depicts example user interfaces for managing friends in an augmented reality environment and an example user interface to manage application settings, in accordance with embodiments of the present disclosure.

FIG. 5D graphically depicts example user interfaces of an augmented reality environment showing a camera view and a map view, in accordance with embodiments of the present disclosure.

FIG. 5E graphically depicts example user interfaces for viewing notifications in an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 5F graphically depicts example user interfaces for placing a virtual object at a physical location and example user interfaces for sharing a virtual object with another user via an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 5G graphically depicts example user interfaces for responding to a message or a virtual object with another virtual object via an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 6A graphically depicts example user interfaces for creating, posting and/or sharing a virtual billboard object having text content, in accordance with embodiments of the present disclosure.

FIG. 6B graphically depicts additional example user interfaces for creating, posting and/or sharing a virtual object s having multimedia content in accordance with embodiments of the present disclosure.

FIG. 6C graphically depicts example user interfaces for creating a virtual object, posting a virtual object and placing a virtual object at a physical location, in accordance with embodiments of the present disclosure.

FIG. 7 graphically depicts example user interfaces for creating a virtual billboard, posting a virtual billboard at a physical location, sharing the virtual billboard and views of examples of virtual billboard objects placed at physical locations, in accordance with embodiments of the present disclosure.

FIG. 8 graphically depicts views of examples of virtual objects associated with a physical location, in accordance with embodiments of the present disclosure.

FIG. 9A depicts a flow chart illustrating an example process to share a virtual object with a recipient user, in accordance with embodiments of the present disclosure.

FIG. 9B depicts a flow chart illustrating an example process to facilitate a communications session in a real world environment via an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 10A-10B depict flow charts illustrating example processes of posting virtual objects that are world locked and user locked, in accordance with embodiments of the present disclosure.

FIG. 11 depicts a flow chart illustrating an example process to facilitate collaboration in an augmented reality environment through a virtual object, in accordance with embodiments of the present disclosure.

FIG. 12A depicts a flow chart illustrating an example process to provide an educational experience via an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 12B depicts a flow chart illustrating an example process to facilitate interaction with a virtual billboard associated with a physical location, in accordance with embodiments of the present disclosure.

FIG. 13A depicts a flow chart illustrating an example process to administer a marketplace having multiple seller entities via an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 13B depicts a flow chart illustrating an example process to spawn a rewards object in an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 14 is a block diagram illustrating an example of a software architecture that may be installed on a machine, in accordance with embodiments of the present disclosure.

FIG. 15 is a block diagram illustrating components of a machine, according to some example embodiments, able to read a set of instructions from a machine-readable medium (e.g., a machine-readable storage medium) and perform any one or more of the methodologies discussed herein.

DETAILED DESCRIPTION

The following description and drawings are illustrative and are not to be construed as limiting. Numerous specific details are described to provide a thorough understanding of the disclosure. However, in certain instances, well-known or conventional details are not described in order to avoid obscuring the description. References to one or an embodiment in the present disclosure can be, but not necessarily are, references to the same embodiment; and, such references mean at least one of the embodiments.

Reference in this specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosure. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Moreover, various features are described which may be exhibited by some embodiments and not by others. Similarly, various requirements are described which may be requirements for some embodiments but not other embodiments.

The terms used in this specification generally have their ordinary meanings in the art, within the context of the disclosure, and in the specific context where each term is used. Certain terms that are used to describe the disclosure are discussed below, or elsewhere in the specification, to provide additional guidance to the practitioner regarding the description of the disclosure. For convenience, certain terms may be highlighted, for example using italics and/or quotation marks. The use of highlighting has no influence on the scope and meaning of a term; the scope and meaning of a term is the same, in the same context, whether or not it is highlighted. It will be appreciated that the same thing can be said in more than one way.

Consequently, alternative language and synonyms may be used for any one or more of the terms discussed herein, nor is any special significance to be placed upon whether or not a term is elaborated or discussed herein. Synonyms for certain terms are provided. A r

CLAIM OF PRIORITY

This application claims the benefit of:

* U.S. Provisional Application No. 62/557,775, filed Sep. 13, 2017 and entitled “Systems and Methods of Augmented Reality Enabled Applications Including Social Activities or Web Activities and Apparatuses of Tools Therefor,” (8004.US00), the contents of which are incorporated by reference in their entirety;

* U.S. Provisional Application No. 62/575,458, filed Oct. 22, 2017 and entitled “Systems, Methods and Apparatuses of Single directional or Multi-directional Lens/Mirrors or Portals between the Physical World and a Digital World of Augmented Reality (AR) or Virtual Reality (VR) Environment/Objects; Systems and Methods of On-demand Curation of Crowdsourced (near) Real time Imaging/Video Feeds with Associated VR/AR Objects; Systems and Methods of Registry, Directory and/or Index for Augmented Reality and/or Virtual Reality Objects,” (8005.US00), the contents of which are incorporated by reference in their entirety; and

* U.S. Provisional Application No. 62/581,989, filed Nov. 6, 2017 and entitled “Systems, Methods and Apparatuses of: Determining or Inferring Device Location using Digital Markers; Virtual Object Behavior Implementation and Simulation Based on Physical Laws or Physical/Electrical/Material/Mechanical/Optical/Chemical Properties; User or User Customizable 2D or 3D Virtual Objects; Analytics of Virtual Object Impressions in Augmented Reality and Applications; Video objects in VR and/or AR and Interactive Multidimensional Virtual Objects with Media or Other Interactive Content,” (8006.US00), the contents of which are incorporated by reference in their entirety.

* U.S. Provisional Application No. 62/613,595, filed Jan. 4, 2018 and entitled “Systems, methods and apparatuses of: Creating or Provisioning Message Objects Having Digital Enhancements Including Virtual Reality or Augmented Reality Features and Facilitating Action, Manipulation, Access and/or Interaction Thereof,” (8008.US00), the contents of which are incorporated by reference in their entirety.

* U.S. Provisional Application No. 62/621,470, filed Jan. 24, 2018 and entitled “Systems, Methods and Apparatuses to Facilitate Gradual and Instantaneous Change or Adjustment in Levels of Perceptibility of Virtual Objects and Reality Object in a Digital Environment,” (8009.US00), the contents of which are incorporated by reference in their entirety.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is related to U.S. application Ser. No. 16/130,541, also filed on Sep. 13, 2018 and entitled “Systems And Methods Of Virtual Billboarding And Collaboration Facilitation In An Augmented Reality Environment,” the contents of which are incorporated by reference in their entirety.

This application is related to U.S. application Ser. No. 16/160,582, also filed on Sep. 13, 2018 and entitled “Systems And Methods Of Rewards Object Spawning And Augmented Reality Commerce Platform Supporting Multiple Seller Entities”, the contents of which are incorporated by reference in their entirety.

This application is related to PCT Application no. PCT/US18/50952, also filed on Sep. 13, 2018 and entitled “Systems And Methods Of Shareable Virtual Objects and Virtual Objects As Message Objects To Facilitate Communications Sessions In An Augmented Reality Environment”, the contents of which are incorporated by reference in their entirety.

This application is related to PCT Application no. PCT/US2018/44844, filed Aug. 1, 2018 and entitled “Systems, Methods and Apparatuses to Facilitate Trade or Exchange of Virtual Real-Estate Associated with a Physical Space”, the contents of which are incorporated by reference in their entirety.

This application is related to PCT Application no. PCT/US2018/45450, filed Aug. 6, 2018 and entitled “Systems, Methods and Apparatuses for Deployment and Targeting of Context-Aware Virtual Objects and/or Objects and/or Behavior Modeling of Virtual Objects Based on Physical Principles”, the contents of which are incorporated by reference in their entirety.

TECHNICAL FIELD

The disclosed technology relates generally to augmented reality environments and virtual objects that are shareable amongst users.

BACKGROUND

The advent of the World Wide Web and its proliferation in the 90's transformed the way humans conduct business, live lives, consume/communicate information and interact with or relate to others. A new wave of technology is on the cusp of the horizon to revolutionize our already digitally immersed lives.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example block diagram of a host server able to deploy virtual objects for various applications, in accordance with embodiments of the present disclosure.

FIG. 2A depicts an example diagram showing an example of a virtual object to facilitate an augmented reality experience including a communications session and an example of a virtual object which includes a shareable note in an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 2B depicts an example diagram illustrating an example of virtual object posted in an augmented reality environment for a user by another entity, in accordance with embodiments of the present disclosure.

FIG. 2C depicts an example diagram depicting collaboration facilitated through a virtual object in an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 2D depicts an example diagram of a marketplace administered in an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 2E depicts an example diagram showing an example user experience flow for creating virtual objects, managing a collection of virtual objects, sharing and posting virtual objects or responding to virtual objects, in accordance with embodiments of the present disclosure.

FIG. 3A depicts an example functional block diagram of a host server that deploys and administers virtual objects for various disclosed applications, in accordance with embodiments of the present disclosure.

FIG. 3B depicts an example block diagram illustrating the components of the host server that deploys and administers virtual objects for various disclosed applications, in accordance with embodiments of the present disclosure.

FIG. 4A depicts an example functional block diagram of a client device such as a mobile device that enables virtual object manipulation and/or virtual object collection for various disclosed applications, in accordance with embodiments of the present disclosure.

FIG. 4B depicts an example block diagram of the client device, which can be a mobile device that enables virtual object manipulation and/or virtual object collection for various disclosed applications, in accordance with embodiments of the present disclosure.

FIG. 5A graphically depicts diagrammatic examples showing user experience flows in navigating an example user interface for accessing, viewing or interacting with an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 5B graphically depicts example user interfaces for logging in to, signing up for and viewing a user profile in an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 5C graphically depicts example user interfaces for managing friends in an augmented reality environment and an example user interface to manage application settings, in accordance with embodiments of the present disclosure.

FIG. 5D graphically depicts example user interfaces of an augmented reality environment showing a camera view and a map view, in accordance with embodiments of the present disclosure.

FIG. 5E graphically depicts example user interfaces for viewing notifications in an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 5F graphically depicts example user interfaces for placing a virtual object at a physical location and example user interfaces for sharing a virtual object with another user via an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 5G graphically depicts example user interfaces for responding to a message or a virtual object with another virtual object via an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 6A graphically depicts example user interfaces for creating, posting and/or sharing a virtual billboard object having text content, in accordance with embodiments of the present disclosure.

FIG. 6B graphically depicts additional example user interfaces for creating, posting and/or sharing a virtual object s having multimedia content in accordance with embodiments of the present disclosure.

FIG. 6C graphically depicts example user interfaces for creating a virtual object, posting a virtual object and placing a virtual object at a physical location, in accordance with embodiments of the present disclosure.

FIG. 7 graphically depicts example user interfaces for creating a virtual billboard, posting a virtual billboard at a physical location, sharing the virtual billboard and views of examples of virtual billboard objects placed at physical locations, in accordance with embodiments of the present disclosure.

FIG. 8 graphically depicts views of examples of virtual objects associated with a physical location, in accordance with embodiments of the present disclosure.

FIG. 9A depicts a flow chart illustrating an example process to share a virtual object with a recipient user, in accordance with embodiments of the present disclosure.

FIG. 9B depicts a flow chart illustrating an example process to facilitate a communications session in a real world environment via an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 10A-10B depict flow charts illustrating example processes of posting virtual objects that are world locked and user locked, in accordance with embodiments of the present disclosure.

FIG. 11 depicts a flow chart illustrating an example process to facilitate collaboration in an augmented reality environment through a virtual object, in accordance with embodiments of the present disclosure.

FIG. 12A depicts a flow chart illustrating an example process to provide an educational experience via an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 12B depicts a flow chart illustrating an example process to facilitate interaction with a virtual billboard associated with a physical location, in accordance with embodiments of the present disclosure.

FIG. 13A depicts a flow chart illustrating an example process to administer a marketplace having multiple seller entities via an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 13B depicts a flow chart illustrating an example process to spawn a rewards object in an augmented reality environment, in accordance with embodiments of the present disclosure.

FIG. 14 is a block diagram illustrating an example of a software architecture that may be installed on a machine, in accordance with embodiments of the present disclosure.

FIG. 15 is a block diagram illustrating components of a machine, according to some example embodiments, able to read a set of instructions from a machine-readable medium (e.g., a machine-readable storage medium) and perform any one or more of the methodologies discussed herein.

DETAILED DESCRIPTION

The following description and drawings are illustrative and are not to be construed as limiting. Numerous specific details are described to provide a thorough understanding of the disclosure. However, in certain instances, well-known or conventional details are not described in order to avoid obscuring the description. References to one or an embodiment in the present disclosure can be, but not necessarily are, references to the same embodiment; and, such references mean at least one of the embodiments.

Reference in this specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosure. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Moreover, various features are described which may be exhibited by some embodiments and not by others. Similarly, various requirements are described which may be requirements for some embodiments but not other embodiments.

The terms used in this specification generally have their ordinary meanings in the art, within the context of the disclosure, and in the specific context where each term is used. Certain terms that are used to describe the disclosure are discussed below, or elsewhere in the specification, to provide additional guidance to the practitioner regarding the description of the disclosure. For convenience, certain terms may be highlighted, for example using italics and/or quotation marks. The use of highlighting has no influence on the scope and meaning of a term; the scope and meaning of a term is the same, in the same context, whether or not it is highlighted. It will be appreciated that the same thing can be said in more than one way.

Consequently, alternative language and synonyms may be used for any one or more of the terms discussed herein, nor is any special significance to be placed upon whether or not a term is elaborated or discussed herein. Synonyms for certain terms are provided. A recital of one or more synonyms does not exclude the use of other synonyms. The use of examples anywhere in this specification including examples of any terms discussed herein is illustrative only, and is not intended to further limit the scope and meaning of the disclosure or of any exemplified term. Likewise, the disclosure is not limited to various embodiments given in this specification.

Without intent to further limit the scope of the disclosure, examples of instruments, apparatus, methods and their related results according to the embodiments of the present disclosure are given below. Note that titles or subtitles may be used in the examples for convenience of a reader, which in no way should limit the scope of the disclosure. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. In the case of conflict, the present document, including definitions will control.

Embodiments of the present disclosure include systems, methods and apparatuses of platforms (e.g., as hosted by the host server 100 as depicted in the example of FIG. 1 ) of shareable virtual objects and virtual objects as message objects to facilitate communications sessions in an augmented reality environment. In general, the object or virtual object is generally digitally rendered or synthesized by a machine (e.g., a machine can be one or more of, client device 102 of FIG. 1 , client device 402 of FIG. 4A or server 100 of FIG. 1 , server 300 of FIG. 3A ) to be presented in the AR environment and have human perceptible properties to be human discernible or detectable.

Further embodiments include, systems and methods of collaboration facilitation in an augmented reality environment. Embodiments of the present disclosure further include providing an educational experience in a real world environment, via an augmented reality platform. Embodiments of the present disclosure further include systems, methods and apparatuses to facilitate interaction with a virtual billboard associated with a physical location in the real world environment.

Further embodiments of the present disclosure further include systems, methods and apparatuses of platforms (e.g., as hosted by the host server 100 as depicted in the example of FIG. 1 ) to spawn a rewards object in an augmented reality platform having value to a user in the real world environment. Yet further embodiments of the present disclosure include an augmented reality commerce platform administer a marketplace which supports multiple seller entities via an augmented reality environment.

One embodiment includes, sending a virtual object (VOB) as a message.

For example, the VOB appears in the recipient's inbox, message stream, or device as a 3D object that can be interacted with (open it, talk to it, touch it or play with it, read it share it, reply to it, file it, publish it, edit it, customize it, tag it or annotate it etc). The recipient's inbox can include a 2D or 3D interface (list, plane, 3D space). The VOB can be set to appear at a fixed distance relative to or near the user, and/or at specific times, and/or within or near specified geolocations or named places, and/or in specified contexts (shopping, working, at home).

In general, VOBs can function like messages that have one or more recipients like email (to, CC, Bcc) and can also be shared with groups or made public, tagged or pinned to the top or a certain location in an interface. VOBs can also carry envelope metadata (e.g. created date, sent date, received date, etc). A thread of related VOB messages is a conversation that uses VOBs as the medium.

Embodiments of the present disclosure further include systems, methods and apparatuses of: creating, rendering, depicting, provisioning, and/or generating message objects with digital enhancements. The enhanced messages can include virtual and/or augmented reality features. The enhanced messages can further be rendered, accessed, transmitted, manipulated, acted on and/or otherwise interacted with via various networks in digital environments by or amongst users, real or digital entities, other simulated/virtual objects or computing systems including any virtual reality (VR), non-virtual reality, augmented reality (AR) and/or mixed reality (mixed AR, VR and/or reality) environments or platforms. For instance, enhanced messages can be shared, transmitted or sent/received via communication channels including legacy SMS, Internet, mobile network via web services, applications (e.g., mobile apps) or dedicated platforms such as VR/AR or mixed VR/AR platforms or environments.

Example embodiments of the present disclosure further include, by way of example:

In one embodiment, a user drafts, writes or composes a message having augmented reality (“AR”) content. The AR content can include one or more virtual objects. A virtual object can include a 2D or 3D graphical rendering, which can include one or more of: text, images, audio, video, or computer graphics animation. The virtual object can appear and be accessed (preview, view, shared, edited, modified), acted on and/or interacted with via an imaging device such as a smartphone camera, wearable device such as an augmented reality (AR) or virtual reality (VR) headset, gaming consoles, any wearable technology, AR glasses, wearable smart watch, wearable computer, heads up display, advanced textiles, smart garments, smart shoes, smart helmets, activity trackers, in car display or in car navigation panel or unit, etc.)

The message can be for instance, supplemented, formatted, optimized or designed with additional text, graphical content, multimedia content, and/or simulated objects or virtual objects. The user can place the enhanced messages (or any of the simulated or virtual objects) in physical locations relative to their device, and/or also relative to other virtual objects or simulated objects in the scene to construct a scene relative to a user view or the user's camera perspective.

In one example, if a user places a virtual object visually in front of their physical position, the virtual or simulated object can be saved to that physical position or near that physical position or within a range of the physical location. The user can also place and save the object for example, at any angle e.g., 10 degrees to the right of their front position

If placed at a particular angle, and/or size and/or swivel the virtual object, it can be saved in that particular relative location and orientation. The user can also turn around and place it behind themselves and then turn forward again, before sending the message, then it can then save that virtual object or simulated object as being behind them.

In one embodiment, the user can further select, identify or specify recipients of for the message. For example, the Recipients can be from existing contacts lists, or can be added as new contacts and/or individuals or named groups or lists of individuals.

Note that a recipient need not be a single person. For instance, a recipient can be an AR enabled chatroom, group or mailing list An AR enabled chatroom, group or mailing list can be associated with a name and/or address. It may also have policies and permissions governing who is the admin, what other roles exist and what their access, preview, view, edit, delete, enhance, sharing, read/write/invite permissions are.

In one embodiment, an AR enabled chatroom or group is or includes a shared social space where AR messages (e.g., enhanced messages) that are sent to the chatroom or group can be updated synchronously and/or asynchronously to all the recipients. This enables a real-time or near-real-time or asynchronous AR experience for the participants. In some instances, posting AR content to the chatroom or group can be equivalent to sending an AR message to the members of the group or chatroom.

According to embodiments of the present disclosure, a user sends a message (e.g., the enhanced message) to recipients and the message can be transmitted to recipients. The recipients can be notified that they have received an AR message or that a system has received a message intended for them. If recipients are members of a group that is the recipient or intended recipient, then a group notification can be sent to individual members of the group. Recipients can be notified with a text message, social invite on Facebook or Twitter or another social network, a message in a chatroom, an email message, or a notification on their phone, or a notification in a particular messaging or other type of mobile, desktop or enterprise/Web app.

In some embodiments, individual recipients can open the AR message to access, preview or view its content. The relevant application can also automatically open to access the message. For example, by clicking on or otherwise selecting the notification, the AR message can be acted on or interacted with. For example, the AR message can be opened and rendered in the appropriate application or reader for display or further action or interaction.

Identifiers for an application or plug-in used to display, present or depict it can be conveyed or included in metadata in the notification, or it can be implicit in the type of notification or channel that the notification is sent through and received in. The AR application that detects or receives and renders the message can depict or display the content of the message appropriately. In particular, virtual objects or simulated objects that were placed into relative positions around the sender can be rendered in the same relative positions around the receiver.

If the sender places a virtual/simulated object, or set of objects, in front of their camera when they composed the message, then those objects can appear in front of the recipient's camera in the same relative positions that the sender intended. If a user puts a virtual object behind themselves, the virtual object can also be behind the receiver when they receive the AR message and the receiver can then turn around and see the virtual object behind them in that case.

In addition, individual recipients can perform social actions on a received AR message, based on the policies and permissions of the application they use to receive it and/or the metadata on the message itself. They can also reply to an AR message with another AR message. They may reply to an AR message with a new AR message, or a non-AR message. With proper permissions and/or upon meeting certain criteria, users or recipients can modify an AR message to be stored, posted publicly/privately and/or sent in reply or forwarded to another user or group of users.

In an event when an AR message is configured to allow modifications, certain (or any) recipients can add modifications such as additional virtual objects, to the AR message, and these modifications can be be added to the original Message and the sender and any/all other recipients of the AR message will also get these updates. Revisions to the original message can be stored so users can roll back to or view any of the previous versions.

In other words, an AR message can be configured to be a collaborative object that can be modified on an ongoing basis by the sender and any/all recipients such that they can collaboratively add to, interact with or act on the content of the message. Modifications to an AR message can be subject to permissions, criterion and/or policies such as moderation approval by the sender or an admin.

In some embodiments, users can forward an AR message to other recipients Forwarding an AR message sends it to other recipients with forwarding metadata in the header of the message. They can comment on the AR message. A comment can be or include by way of example, text, document, message, emoji, emoticon, a gif, audio or video, that appears on an associated comments thread which can be non AR based or AR based.

A comment can also be created and rendered as an AR comment object or part of an AR comments digest object that is associated with the AR message. Users can save, tag, flag, delete, archive, rate, like, mark as spam, apply rules or filters, file into a folder, and perform other actions activities on or interact AR messages that are similar to the activities that can be performed on email, text messages or other digital objects rendered in a digital environment.

AR Billboarding and Messaging

Embodiments of the present disclosure further include AR billboarding and/or messaging. In one embodiment, when people post in the disclosed VR/AR environment, the system enables them to select a person and/or place and/or time to post it to. If they choose just the person it is a message. If just a place it can be a billboard. A person and place is a sticky note or targeted billboard. If they choose just a time then it appears for everyone they can post it to at a time. Place and time or person and time also have effects. The disclosed system can obey, implement or enforce the rules and displays what is permitted under the constraints. Enabling a person to post to a place that they are not in can be implemented. What plane do they post to, at what angle, what altitude in a building, the system can enable them to choose that with an altitude or floor of building selector, or building map. In some instance, it is free to post to place you are in but not free to places you are not in.

People could collaboratively tag places and things in places for us. A person chooses a plane at a place and names it “desk in my room” then that surface is logged with geo cords. It goes on the map as a named target. Other users can select it and post an object to that place. So the object appears at that named location relative to that place for anyone who is there. Billboards could be compound objects like that—e.g., as collages made of stickers. The “try it” sticker on top of the “Lose Weight Feel Great” object. Compound objects can be implemented in the system as sticking things on or next to or near other things to build grouped objects and then define as a grouped object so it moves as one—like in a drawing app.

Algorithmic AR/VR Content Feeds

Embodiments of the present disclosure further include algorithmic AR/VR Content Feeds. A an example embodiment can include: (1) the Twitter near a billboard you designed for the video—or something like it, (2) some interesting things auto-spawning near the user (gems, 50% off coupon, secret VIP event ticket, free gift offer). Some of these could appear and then self-delete after n seconds or minutes, giving you a reason to look at the same place again even though you looked recently.

The system can award points for actions in the environment, like when you find a special virtual object, or when you do things like click on objects, etc. all of this speaks to the addictiveness and reward of the app. With a process that makes it relevant to each user (e.g., user activity stream for each user). So the story is that users get: messages from friends that are highly relevant by nature, relevant content near them about their interests (from public+layers they follow, in an algo-curated public feed), and relevant rewards and sponsored content (rewards from us and targeted ads/offers from sponsors). The content from friends and the relevant content around them+rewards, keeps them looking and coming back.

If the sponsored ad content is at least not irrelevant, and ideally also provides some points or other kinds of rewards (social reward, access to exclusive events, big savings, etc.) then users will not only tolerate them but may even enjoy and want them. If for example a sponsor made some great sponsored 3D content and even made it rewarding to engage with it, and it was relevant to Joe, then Joe would enjoy it rather than find it annoying. The difference between something being “content” or “annoying advertising” is relevancy and quality.

The system can reward users in a number of ways. One is the depicted content itself may be delightful or entertaining or useful. The other is that is the system includes a built in treasure hunt metagame—which spawns rewards that the system provides and that sponsors can pay for, to each user, intelligently. that the process is designed and adapted to keep users playing just like a slot machine.

In one embodiment, the system's reward system is akin to a real world casino. For example, the system provides a unique experience to the users so Joe doesn't see the same content or rewards every time he logs in. An “ad” in the system should be content+reward. If it is just content it has to be rewarding in itself. Otherwise at least add points rewards to it. The uncertainty and luck of discovery aspect—the potential jackpot aspect—these make it fun and addictive.

Statistically, the system can use and implement casino math and/or relevant mathematical adaptations for this—specifically, slot machines dynamically adjust the probability of a player winning based on how they are playing, to keep them betting. The system implements this or a version of this. A “bet” is another minute of attention in the environment.

Like a customized slot machine where jackpots are sponsored targeted ads (that the customer actually wants). But jackpots are both actually—the system can provide points awards to users for actions they take (time spent, interactions, clicks, etc.) and just by luck and based on their karma. So can advertisers—advertisers can insert rewards and the disclosed system runs them in the spawn. There are also other kinds of jackpots beyond just points—for example a coupon has a bar code and gives the user a huge discount at a store but may not dispense any points. Similarly a really great little game or collectors item VOB could also be rewarding to a user that likes that.

There can be several streams of content that users are exposed to in the disclosed VR/AR environment: (1) objects addressed explicitly to them, (2) objects that are shared with users and groups they follow (but are not explicitly addressed to them), (3) objects that are shared with them by the system, and sponsors of the system. The public layer can include of (2)+(3). All other layers can show for example, either (1) or only (2). The system's content and ads only appear in the public layer. A user can choose not to see the public layer, but they cannot choose to see the public layer without sponsored content (ads). The system ensures that public layer is so good and rewarding that users want to see it all the time.

One embodiment of the present disclosure includes some coins and gems and power-up objects—the system associates or assigns points with them, but advertisers can also buy them and run them and add some kind of branding to them so that when users redeem them they see some sponsored content, or have to first go through a sponsored interaction.

The key is lots of quality custom or adapted content to always keep the user engaged: There has to be the optimal ratio. Too much reward is also no longer special. In one embodiment, it is 80/n where n is usually 20% and there is a bell curve of greater and lower reward frequency where the frequency increases a bit while the user has good karma, and then there is another variable for the probability of different sized points rewards as well. For instance, a user who is more engaged can earn better karma and sees more rewards for a while, and there is a dice roll every time a reward spawns where the probability of a higher value reward also can change based on user karma for a while. The more a user uses with the disclosed AR/VR environment and the more they engage, the better they perform. Instead of 80/20 maybe it becomes 70/30 best case, and the user can earn bigger rewards on average during that time. But then it decays unless the user maintain it.

As for the 80 or 70% of the experience that is non-sponsored content, that can be user generated content (UGC) or content from APIs like Yelp and Twitter. Messaging, billboarding/community, publishing are the UGC part. Then we need a healthy amount of API content that is useful and contextually relevant (geo-relevant to start with).

In one example, of the 80% to 70%, about half is allocated or earmarked for UGC, and half could be from APIs. In fact even if there was only API content it could be useful in certain contexts. Like as a tourist or when looking for a place to eat—so that's a mechanism to fill the world with a lot of content: Twitter, Facebook, Insta, Yelp, Wikipedia, Google Maps—about things near you, the place near you. APIs of an object with the Linkedin profile, Insta profile, of each person near you can also be utilized.

Billboarding can be advantageous where there are lots of users. Messaging can be initially useful between friends. The API content can be what populates the world and is the primary excuse for looking through our lens. Adding geo-relevant Twitter, news, media content, and other social media content into the AR view is a good place to start because there is almost always something near you. Note that with the web there is really no page to look at that shows you relevant information from every network to where you are geographically and temporally right now. In a 2D interface you have a list to work with. But in AR, such as the disclosed AR environment, which is 3D, there is so much more room to work with. This touches on the desktop concept, and even personal intelligent assistants—basically the disclosed process includes an intelligent assistant that shows users relevant objects to user's context, through the system's lens.

A user's context includes, for example the user's identity, past, the user's present activity, location, time of day, who else is there. Usually the system will either have no API content or UGC for a user and place. The disclosed system executes processes which understand that users want to see messages from friends, the best content from people, groups and brands they follow, and the best, most popular, or most relevant stuff from public. Users also want to see rewards that give them value that matters to them—including points where redeemable (cryptocurrency, digital currency, fiat currency, etc.) or useful (buy virtual goods or pay to boost posts and get attention in the AR environment platform or system), and they want to see very entertaining and/or relevant content from sponsors.

The innovative process includes optimizing this mix to attain and achieve the most engagement and continued engagement from each individual user; the process includes a/b testing across users and populations of users and when it learns that some item is getting great response for some demographic and context it also can increase the frequency of that content for that audience and context. Or it can move it to the “top level” or outside of containers; basically to the disclosed system provides a great, non-cluttered UX; there always has to be enough to keep it engaging and interesting, and the system ensures that it nests or manages the world of information; the disclosed innovative rewards system helps with ensuring there is at least a chance there is a reward all the time; as an example, it's inovative function is to drive engagement with the UX.

In a further embodiment, the system enables users to attach AR/VR billboards together in the AR/VR environment—to tile them to make larger surfaces. For example attach a billboard to right edge of a billboard. They can then for example make a passageway formed of billboards facing in that you walk through. It could be an experiential thing—an art exhibit, a sequence of messages. Or a room. Or a maze. Or an entire building, etc.

The system enables these structures to be grouped so that the edges connect and precise angles and they remain that way relative to each other. In further embodiments some other shapes billboards are enabled to be designed customized, or selected—for example, a long rectangle that is as wide as two billboards so you can write a long wide message. A taller one. A triangle shape. A hexagon shape. A sphere. A cube. A rectangular block. These enable lots of billboard structures to be built. Users may be able to generate a non standard shape or design any geometric shape they desire. These become structures users can build. Each surface can have color, texture, and optional content (text or image). This enables AR objects to be built like LEGO. The 2d Wall shape one of many building blocks enabled in the disclosed system/platform.

One embodiment also includes billboard shapes that look like standard sign shapes (like a stop sign shape, a one way shape, etc) and flags that flap in the wind gently. These could have polls holding them up. Billboard shapes also include doorway, archway, window and portal objects shapes too. People can build cool things to see and explore. This enables second life kinda of activities on your own layer. For the public/main layer the real estate game would be necessary to build things that others see. Also note that in a room made of billboard wall objects if it has a floor and a ceiling and four walls it can have a light source inside it.

One embodiment further includes—a billboard or set of billboards that are portable in real space and/or digital space, e.g., that a user can take with them. For example a protest—they can carry them with them. Wherever the user is their billboard can be near them or above them. Other users can see that.

In addition, an object or billboard can be placed in a room so that it always appears in the right part of the room in the right orientation to the room—based on the shape of the room. For example the camera may be able to determine the geometry of the room.

In one embodiment, the system enables users to map spaces for us—crowd sourced—to map or determine the geometry of the room. If I am in my living room and I want to place an object precisely and have it always be oriented that way then to escape the limits of GPS I could map the room—in that mode the user would enter a special mode in the AR/VR environment and would walk the boundaries of the room as well as perhaps take photos from each wall of the room—the software agent or module of the AR/VR environment would learn the room. From this it could then keep 3D objects oriented relative to the shape of the room. The system can build up a database of mapped rooms. And this could improve if more users helped with more perspectives or to fill in missing pieces or missing rooms. The system can also award points to users for doing this.

Embodiments of the present disclosure further include physical stickers which can include stickers visible to the human eye, stickers that are highly reflective of IR and/or stickers that are not visible to the human eye. For example, A user could make stickers that appear transparent/almost invisible to the naked eye but that are highly visible to IR from camera phones—for example, using IR range finding device or sensors that can be built into portable devices. In one embodiment, these invisible stickers can be placed on the walls of rooms to enable devices to triangulate more precise locations in the rooms. In one embodiment, billboards can be configured or selected to be stuck to walls and other surfaces. This could also include taking a photo of the place where it appears to help identify it. Billboards can be transparent as well—so users just see the letters in space or on a wall.

One embodiment includes some special objects like lockable boxes and chests. They can be opened if the user has the key. Or the combination. Or certain required items. The system enables any virtual or digital content to be placed on any surface.

One problem addressed by the disclosed platform is if people build in the same or overlapping locations in their private layers the system decides what is show to users by default (e.g., in the public layer at that location). In one embodiment, the system can show a summary object that says “there are two giant castles here” unless one of them outbids the other to rent that real estate. If they rent the real estate then they can be the default and the other object is in a small orb or container that indicates there are other objects or content there.

One embodiment of the AR environment includes, a standardized container shape—an orb—that is recognizable. It appears wherever the system needs to contain and summarize many objects at a place without showing them. The orb can be identifiable in an iconography as an a special thing. It can have a number on it and could also have an icon for the type of things it contains. A Twitter orb would have the Twitter logo and a badge for the number of Tweets it contains. A MYXR Orb (e.g., an AR/VR orb) would have a MYXR logo and a badge for the number of MYXR or AR/VR objects (billboards, blocks, grouped named structures, it includes.). An example interaction would enable a user to activate an Orb and see an inventory of its contents and then choose the ones to pop out into the environment. They could also pop them all out and then fold them back into the Orb. The size of the Orb could reflect the number of objects it contains as well. In order to not be useful and encourage interaction with it, there could be one Orb per location containing sub orbs per type of collection (Twitter, Videos, Photos, etc).

Orbs can generally be a bit out of the way in any location—they could either float at about 7 feet altitude, or they could be in the heads up display (HUD) rather than objects in the scene. If they are in the HUD then they would basically be alerts that would appear and say “200 Tweets here”—if you tap on that alert then it puts a timeline of Tweets in front of the user and they can then see them all. Or if it says “100 billboards here” then it gives the user a list of them and they can choose which obese to see in the space”.

One embodiment also includes balloon objects with messages on them. They float to the ceiling or a specific altitude. A giant hot air balloon even. And a dirigible of any size. A small one could cruise around a mall or office. A giant one could cruise over a park or a city. The system can construct some more building block objects to build structures and shapes with and enable them to be glued together to make structures.

The system deals with multiple objects in the same place from different layers in multiple ways—in the Main or Public view. In one example, the system can summarize them into an Orb, unless there is either <n of them, or they pay to promote. Promoted objects always appear on their own. If multiple or all of them pay they all appear outside the orb but share the location—which could involve them all rotating through the desired location, like a slow orbit. Also note an orb cannot contain only 1 object—so if there is only 1 object left in an orb, the orb goes away and the object becomes first level.

So in one example, a rule can bet, for any location, to avoid clutter if there are >20 virtual objects there, we put them into <20 Orbs (one Orb named for each layer that is represented and that has >2 virtual objects depicted, presented or posted at the location, and for any layers that only have 1 item at the location they go into an orb for Other Layers). An Orb for a layer has the icon or face for that layer on it.

For example, if you posted 5 VOBs to a location that has >20 items, an Orb would be formed for you and would contain your virtual objects; it would have your avatar or face on it, and a badge for “5”. Other Orbs would group other virtual objects at the location. Except any sponsored/promoted items would appear outside their Orbs as well. You would want them to be listed inside their Orbs too so that if someone looks inside they are there as well, but they are already popped out of the Orb. So the listing of Orb contents would show them, but the state would indicate that they are already visible outside the Orb.

When objects are placed in a location on a layer, the system will present them in such a manner so as to prevent them from being right on top of each other, or overly overlapping, unless that was intentional (for example building a VOB where you want an emoticon right on the upper right corner of a billboard). When the system presents or depicts the Main or Public views, the system can move VOBs around slowly at the locations they wanted. So They are all orbiting there. There could be multiple billboards (e.g., 3 billboards) from different locations at a certain location—and in the Main or Public views, the system can present them to prevent them from overlapping in space—by orbiting or some other way. Another option would be that there is a marketplace—whoever pays the most points to promote their item gets the location, and then items where paid less are given locations near or around it. This could be a variant on the real-estate game.

When a user makes an item, there is an optional “Promote this item” field where you can pay some of your points, or buy more points, to promote it. The points you pay to promote it are somehow ticked down over time, and have to be refreshed when they run out, unless the user can assign an ongoing auto-pay budget to keep the points allotment at a certain level.

At a given location the objects can be arranged from the desired location of objects based on their points budgets. A user could look and see where their object is appearing in the scene and add more points to push it to the center position, or closer to the center, of a desired location. The distance between objects can be configured as the different in points budgets. For example, if the central object paid 100, and the next object paid 50, and then next paid 40, there would be a greater distance between object 1 and 2 than between 2 and 3.

Examples of Video Objects in Virtual and/or Augmented Reality

A further embodiment of the present disclosure includes video objects in virtual reality and/or augmented reality. One embodiment includes the ability to render or depict a video on a surface or billboard or virtual screen in a mixed reality environment. Similar to picture billboards. The virtual video screen can behave like a 2D or 3D object in the augmented reality scene. It can be a stationary virtual object at a location and the user can walk around it like a physical object. Or it can turn or orient to the user. Users can interact with it to start, stop, rewind, fast forward, mute sound, adjust volume. It can autoplay when a user is in proximity or it can loop. User may also be able to tune the virtual video screen to a channel or a particular content selection. It may also have a hide or directory or search function or playlist.

User Generated and User-Customizable 3D or 2D Virtual Objects

Further embodiments of the present disclosure include user generated and user-customizable 3d or 2d virtual objects. In one embodiment, the system/platform implements 3d or 2d virtual objects that can be user generated and/or user customized. For example, users can choose a virtual object type from a library of types or templates, and then customize it with optional text, formatting, color, fonts, design elements, shapes, borders,

CLAIMS

Claims ( 20 )

What is claimed is:

1. A method to facilitate an augmented reality experience in an augmented reality environment of a real world environment, the method, comprising:

causing to be perceptible, by a recipient user of the augmented reality environment, a virtual object, such that the recipient user is able to engage in the augmented reality experience;

wherein, the virtual object is shared with the recipient user by another entity that uses the augmented reality environment;

wherein, the virtual object is depicted such that the recipient user engages in the augmented reality experience via the augmented reality environment;

rendering or depicting content associated with the virtual object, responsive to detection of a contextual trigger;

wherein, the contextual trigger occurs when the recipient user is engaging in a given action or activity in a given location in the real world environment or at a given time;

wherein:

the content associated with the virtual object is rendered or depicted during the augmented reality experience includes multiple stages of rendering;

wherein, the multiple stages of rendering are tracked with and identified by a status parameter of the virtual object.

2. The method of claim 1 , wherein:

the contextual trigger occurs at one or more of:

predetermined times or time periods;

certain locations in the real world environment.

3. The method of claim 1 , further comprising:

detecting an interaction trigger with respect to the virtual object, the interaction trigger being detected responsive to engagement with the augmented reality experience in the augmented reality environment;

further rendering or depicting the content associated with the virtual object;

wherein, the content associated with the virtual object is further rendered or depicted, in a subsequent stage of the multiple stages of rendering;

wherein, the subsequent stage of rendering is identified based on one or more of, the status parameter or the interaction trigger;

wherein, a state of the virtual object after the content has cycled through the multiple stages of rendering is further determined by the status parameter of the virtual object, the state including, one or more of:

an expire state, a vanish state, a continue state, a pause state, a morph state, and a change state.

4. The method of claim 1 , wherein:

the content associated with the virtual object is rendered or depicted during the augmented reality experience includes multiple stages of rendering;

wherein, the multiple stages of rendering includes, one or more of:

depiction or presentation of a name, title or subject of the virtual object;

depiction of behaviors or animations of the virtual object;

depiction of a payload portion of the content;

wherein, the payload portion of the content includes, one or more of:

a video, a text based message, a game, an experience, a prize and points.

5. The method of claim 1 , wherein:

the augmented reality experience includes one or more of:

(i) a real time or near real time augmented reality chat session or an augmented reality messaging session;

(ii) asynchronous exchanges of augmented reality messages, augmented reality conversations, augmented reality electronic mail or augmented reality greeting cards; and

(iii) a multiplayer mode gaming experience, wherein, the recipient user and the other entity interact with the virtual object.

6. The method of claim 1 , wherein, the virtual object includes, one or more of:

(i) a greeting card, wherein, the content of the greeting card is at least in part specified or designed by the other entity and designated for the recipient user; wherein, the greeting card includes a template style or template design having 3D features or 3D scenes that are preconfigured; and

(ii) a business card, wherein, the business card is for the other entity and designated for the recipient user;

wherein, the content associated with the virtual object includes one or more of a text message, a conversation, a graphical message, music or other audible content, works of art, animation, games, scenes, limericks, jokes, a 3D scene, and an animated 3D object.

7. A system to facilitate an augmented reality experience in an augmented reality environment of a real world environment, the system, comprising:

a processor,

memory coupled to the processor, the memory having stored thereon instructions, which when executed by the processor, cause the processor to:

cause to be perceptible, by a recipient user of the augmented reality environment, a virtual object, such that the recipient user is able to engage in the augmented reality experience;

render or present content associated with the virtual object, responsive to detection of a contextual trigger;

wherein, the virtual object is shared with the recipient user by another entity that uses the augmented reality environment;

wherein, the content associated with the virtual object is rendered or presented during the augmented reality experience includes multiple stages of rendering;

wherein, the multiple stages of rendering are tracked with and identified by a status parameter of the virtual object.

8. The system of claim 7 ,

wherein, the contextual trigger occurs when the recipient user is engaging in a given action or activity in a given location in the real world environment or at a given time.

9. The system of claim 7 , wherein:

the contextual trigger occurs at one or more of:

predetermined times or time periods;

certain locations in the real world environment.

10. The system of claim 7 , wherein, the memory having further stored thereon instructions, which when executed by the processor, cause the processor to:

detect an interaction trigger with respect to the virtual object, the interaction trigger being detected responsive to engagement with the augmented reality experience in the augmented reality environment;

further render or present the content associated with the virtual object;

wherein, the content associated with the virtual object is further rendered or presented, in a subsequent stage of the multiple stages of rendering.

11. The system of claim 7 ,

wherein, a state of the virtual object after the content has cycled through the multiple stages of rendering is further determined by the status parameter of the virtual object.

12. The system of claim 7 ,

wherein, the multiple stages of rendering includes, one or more of:

presentation of a name, title or subject of the virtual object;

depiction of behaviors or animations of the virtual object.

13. The system of claim 7 , wherein, the multiple stages of rendering includes:

depiction of a payload portion of the content.

14. The system of claim 7 , wherein:

the augmented reality experience includes one or more of:

(i) a real time or near real time augmented reality chat session or an augmented reality messaging session; and

(ii) asynchronous exchanges of augmented reality messages, augmented reality conversations, augmented reality electronic mail or augmented reality greeting cards.

15. The system of claim 7 , wherein:

the virtual object is caused to be perceptible to the recipient user, in response to

determination that the recipient user is an intended recipient, or

determination that the recipient user is in a given location in the real world environment.

16. A method to facilitate an augmented reality experience in an augmented reality environment of a real world environment, the method, comprising:

causing to be perceptible, by a recipient user of the augmented reality environment, a virtual object, such that the recipient user is able to engage in the augmented reality experience;

wherein, the virtual object is shared with the recipient user by another entity that uses the augmented reality environment;

wherein, the virtual object is depicted such that the recipient user engages in the augmented reality experience via the augmented reality environment;

wherein, content associated with the virtual object is rendered or presented during the augmented reality experience includes multiple stages of rendering;

wherein, the multiple stages of rendering are tracked with and identified by a status parameter of the virtual object.

17. The method of claim 16 , further comprising:

rendering or presenting the content associated with the virtual object, responsive to detection of a contextual trigger;

wherein, the contextual trigger occurs when the recipient user is engaging in a given action or activity in a given location in the real world environment.

18. The method of claim 16 , further comprising:

rendering or presenting the content associated with the virtual object, responsive to detection of a contextual trigger;

wherein, the contextual trigger occurs when the recipient user is engaging in a given action or activity at a given time.

19. The method of claim 16 , further comprising, rendering or presenting the content associated with the virtual object, responsive to detection of a contextual trigger;

wherein:

the contextual trigger occurs at one or more of:

predetermined times or time periods;

certain locations in the real world environment.

20. The method of claim 16 , further comprising:

detecting an interaction trigger with respect to the virtual object, the interaction trigger being detected responsive to engagement with the augmented reality experience in the augmented reality environment;

further rendering or presenting the content associated with the virtual object;

wherein, the content associated with the virtual object is further rendered or presented, in a subsequent stage of the multiple stages of rendering;

wherein, the subsequent stage of rendering is identified based on one or more of, the status parameter or the interaction trigger.

US16/130,499

2017-09-13

2018-09-13

Systems and methods of shareable virtual objects and virtual objects as message objects to facilitate communications sessions in an augmented reality environment

Expired - Fee Related

US11249714B2

( en )

Priority Applications (1)

Application Number

Priority Date

Filing Date

Title

US16/130,499

US11249714B2

( en )

2017-09-13

2018-09-13

Systems and methods of shareable virtual objects and virtual objects as message objects to facilitate communications sessions in an augmented reality environment

Applications Claiming Priority (6)

Application Number

Priority Date

Filing Date

Title

US201762557775P

2017-09-13

2017-09-13

US201762575458P

2017-10-22

2017-10-22

US201762581989P

2017-11-06

2017-11-06

US201862613595P

2018-01-04

2018-01-04

US201862621470P

2018-01-24

2018-01-24

US16/130,499

US11249714B2

( en )

2017-09-13

2018-09-13

Systems and methods of shareable virtual objects and virtual objects as message objects to facilitate communications sessions in an augmented reality environment

Publications (2)

Publication Number

Publication Date

US20190107990A1

US20190107990A1 ( en )

2019-04-11

US11249714B2

true

US11249714B2 ( en )

2022-02-15

Family

ID=65723564

Family Applications (3)

Application Number

Title

Priority Date

Filing Date

US16/130,499

Expired - Fee Related

US11249714B2

( en )

2017-09-13

2018-09-13

Systems and methods of shareable virtual objects and virtual objects as message objects to facilitate communications sessions in an augmented reality environment

US16/130,541

Abandoned

US20190107991A1

( en )

2017-09-13

2018-09-13

Systems and methods of virtual billboarding and collaboration facilitation in an augmented reality environment

US16/130,582

Abandoned

US20190108578A1

( en )

2017-09-13

2018-09-13

Systems and methods of rewards object spawning and augmented reality commerce platform supporting multiple seller entities

Family Applications After (2)

Application Number

Title

Priority Date

Filing Date

US16/130,541

Abandoned

US20190107991A1

( en )

2017-09-13

2018-09-13

Systems and methods of virtual billboarding and collaboration facilitation in an augmented reality environment

US16/130,582

Abandoned

US20190108578A1

( en )

2017-09-13

2018-09-13

Systems and methods of rewards object spawning and augmented reality commerce platform supporting multiple seller entities

Country Status (2)

Country

Link

US

( 3 )

US11249714B2

( en )

WO

( 1 )

WO2019055703A2

( en )

Cited By (25)

* Cited by examiner, † Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US20220080311A1

( en )

*

2017-10-27

2022-03-17

Nicholas T. Hariton

Systems and Methods for Rendering a Virtual Content Object in an Augmented Reality Environment

US20220084295A1

( en )

2020-09-16

2022-03-17

Snap Inc.

Context triggered augmented reality

US11354728B2

( en )

*

2019-03-24

2022-06-07

We.R Augmented Reality Cloud Ltd.

System, device, and method of augmented reality based mapping of a venue and navigation within a venue

US11398088B2

( en )

2018-01-30

2022-07-26

Magical Technologies, Llc

Systems, methods and apparatuses to generate a fingerprint of a physical location for placement of virtual objects

US20220321617A1

( en )

*

2021-03-30

2022-10-06

Snap Inc.

Automatically navigating between rooms within a virtual conferencing system

US11467656B2

( en )

2019-03-04

2022-10-11

Magical Technologies, Llc

Virtual object control of a physical device and/or physical device control of a virtual object

US11494991B2

( en )

2017-10-22

2022-11-08

Magical Technologies, Llc

Systems, methods and apparatuses of digital assistants in an augmented reality environment and local determination of virtual object placement and apparatuses of single or multi-directional lens as portals between a physical world and a digital world component of the augmented reality environment

US11611608B1

( en )

*

2019-07-19

2023-03-21

Snap Inc.

On-demand camera sharing over a network

US20230111738A1

( en )

*

2021-10-08

2023-04-13

BlueStack Systems, Inc.

Methods, Systems and Computer Program Products for In-Application Virtual Object Sharing

US20230141621A1

( en )

*

2021-11-09

2023-05-11

Wonder People Co., Ltd.

Method for providing battle royale game which allows players to search for sub items used for upgrading or repairing main items and game server using the same

US11810226B2

( en )

2018-02-09

2023-11-07

Nicholas T. Hariton

Systems and methods for utilizing a living entity as a marker for augmented reality content

US11823312B2

( en )

2017-09-18

2023-11-21

Nicholas T. Hariton

Systems and methods for utilizing a device as a marker for augmented reality content

US20240071002A1

( en )

*

2022-08-30

2024-02-29

Rovi Guides, Inc.

Systems and methods for pinning content items to locations in an augmented reality display based on user preferences

US11983830B2

( en )

2018-04-27

2024-05-14

Nicholas T. Hariton

Systems and methods for generating and facilitating access to a personalized augmented rendering of a user

US12020692B1

( en )

2023-05-17

2024-06-25

Bank Of America Corporation

Secure interactions in a virtual environment using electronic voice

US12019857B2

( en )

*

2021-12-22

2024-06-25

Oshkosh Corporation

Digital twin hydraulic troubleshooting

US12086944B2

( en )

2019-04-30

2024-09-10

Nicholas T. Hariton

Systems and methods for conveying virtual content from external resources and electronic storage in an augmented reality environment at different or same locations

US12154230B2

( en )

2020-09-09

2024-11-26

Snap Inc.

Augmented reality messenger system

US12244605B2

( en )

2022-04-29

2025-03-04

Bank Of America Corporation

System and method for geotagging users for authentication

US12261954B2

( en )

2022-05-23

2025-03-25

Bank Of America Corporation

System and method to validate a rendered object using non-fungible tokens

US12277654B2

( en )

2022-08-30

2025-04-15

Adeia Guides Inc.

Systems and methods for pinning content items to locations in an augmented reality display based on user preferences

US12284146B2

( en )

2020-09-16

2025-04-22

Snap Inc.

Augmented reality auto reactions

US12340710B2

( en )

*

2021-02-04

2025-06-24

Riverfield Inc.

Information processing device and assistance system

US12462490B2

( en )

2022-10-31

2025-11-04

Bank Of America Corporation

Private data-less card device enabled for augmented reality display of data

US12524777B2

( en )

2020-11-30

2026-01-13

Snap Inc.

Reward-based real-time communication session

Families Citing this family (238)

* Cited by examiner, † Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US8314790B1

( en )

*

2011-03-29

2012-11-20

Google Inc.

Layer opacity adjustment for a three-dimensional object

JP6054381B2

( en )

2011-06-07

2016-12-27

ナイキ イノベイト シーブイ

Virtual performance system

US10282914B1

( en )

*

2015-07-17

2019-05-07

Bao Tran

Systems and methods for computer assisted operation

JP2018005091A

( en )

*

2016-07-06

2018-01-11

富士通株式会社

Display control program, display control method and display controller

US10593116B2

( en )

2016-10-24

2020-03-17

Snap Inc.

Augmented reality object manipulation

US10242503B2

( en )

2017-01-09

2019-03-26

Snap Inc.

Surface aware lens

US20180210628A1

( en )

2017-01-23

2018-07-26

Snap Inc.

Three-dimensional interaction system

US10957119B2

( en )

*

2017-03-15

2021-03-23

Facebook, Inc.

Visual editor for designing augmented-reality effects

US10530597B1

( en )

*

2017-04-01

2020-01-07

Smart Power Partners LLC

System for controlling a plurality of power switches configured to apply power to devices

WO2019023659A1

( en )

2017-07-28

2019-01-31

Magical Technologies, Llc

Systems, methods and apparatuses of seamless integration of augmented, alternate, virtual, and/or mixed realities with physical realities for enhancement of web, mobile and/or other digital experiences

US11249714B2

( en )

2017-09-13

2022-02-15

Magical Technologies, Llc

Systems and methods of shareable virtual objects and virtual objects as message objects to facilitate communications sessions in an augmented reality environment

US11861136B1

( en )

*

2017-09-29

2024-01-02

Apple Inc.

Systems, methods, and graphical user interfaces for interacting with virtual reality environments

US11087558B1

( en )

2017-09-29

2021-08-10

Apple Inc.

Managing augmented reality content associated with a physical location

US11232502B2

( en )

*

2017-12-20

2022-01-25

Signify Holding B.V.

Lighting and internet of things design using augmented reality

US10699485B2

( en )

*

2018-01-04

2020-06-30

Universal City Studios Llc

Systems and methods for textual overlay in an amusement park environment

US10904374B2

( en )

2018-01-24

2021-01-26

Magical Technologies, Llc

Systems, methods and apparatuses to facilitate gradual or instantaneous adjustment in levels of perceptibility of virtual objects or reality object in a digital scene

US11014242B2

( en )

*

2018-01-26

2021-05-25

Microsoft Technology Licensing, Llc

Puppeteering in augmented reality

US10438414B2

( en )

2018-01-26

2019-10-08

Microsoft Technology Licensing, Llc

Authoring and presenting 3D presentations in augmented reality

WO2019183276A1

( en )

*

2018-03-20

2019-09-26

Wright Rocky Jerome

Augmented reality and messaging

US10740966B2

( en )

*

2018-05-14

2020-08-11

Microsoft Technology Licensing, Llc

Fake thickness on a two-dimensional object

WO2019237085A1

( en )

2018-06-08

2019-12-12

Vulcan Inc.

Session-based information exchange

US11195336B2

( en )

2018-06-08

2021-12-07

Vulcan Inc.

Framework for augmented reality applications

US11095947B2

( en )

*

2018-06-12

2021-08-17

Number 9, LLC

System for sharing user-generated content

US10996831B2

( en )

2018-06-29

2021-05-04

Vulcan Inc.

Augmented reality cursors

US11010986B2

( en )

2018-08-30

2021-05-18

Apple Inc.

Virtual object kit

CN109272380B

( en )

*

2018-08-30

2023-01-06

腾讯科技（深圳）有限公司

Trading method, device, equipment and storage medium of virtual pet commodity

US11030813B2

( en )

2018-08-30

2021-06-08

Snap Inc.

Video clip object tracking

JP7142315B2

( en )

*

2018-09-27

2022-09-27

パナソニックＩｐマネジメント株式会社

Explanation support device and explanation support method

JP7503542B2

( en )

*

2018-10-05

2024-06-20

マジック リープ， インコーポレイテッド

Rendering location-specific virtual content anywhere

US11810202B1

( en )

2018-10-17

2023-11-07

State Farm Mutual Automobile Insurance Company

Method and system for identifying conditions of features represented in a virtual model

US11024099B1

( en )

2018-10-17

2021-06-01

State Farm Mutual Automobile Insurance Company

Method and system for curating a virtual model for feature identification

US11556995B1

( en )

2018-10-17

2023-01-17

State Farm Mutual Automobile Insurance Company

Predictive analytics for assessing property using external data

JP6485935B1

( en )

*

2018-10-18

2019-03-20

ベルフェイス株式会社

Online communication review system, method, and computer program

US11340709B2

( en )

*

2018-10-18

2022-05-24

Hewlett-Packard Development Company, L.P.

Relative gestures

US11288733B2

( en )

*

2018-11-14

2022-03-29

Mastercard International Incorporated

Interactive 3D image projection systems and methods

US11176737B2

( en )

2018-11-27

2021-11-16

Snap Inc.

Textured mesh building

US10678492B1

( en )

*

2018-11-27

2020-06-09

International Business Machines Corporation

Co-located augmented reality sharing between augmented reality devices

US10505726B1

( en )

2018-12-07

2019-12-10

Nike, Inc.

System and method for providing cryptographically secured digital assets

EP3899865A1

( en )

2018-12-20

2021-10-27

Snap Inc.

Virtual surface modification

US11908345B2

( en )

*

2018-12-27

2024-02-20

Intel Corporation

Engagement level determination and dissemination

US10873724B1

( en )

2019-01-08

2020-12-22

State Farm Mutual Automobile Insurance Company

Virtual environment generation for collaborative building assessment

CN110020909A

( en )

*

2019-01-14

2019-07-16

启云科技股份有限公司

Using the purchase system of virtual reality technology

US10984575B2

( en )

2019-02-06

2021-04-20

Snap Inc.

Body pose estimation

US11511199B2

( en )

*

2019-02-28

2022-11-29

Vsn Vision Inc.

Systems and methods for creating and sharing virtual and augmented experiences

USD978895S1

( en )

*

2019-03-13

2023-02-21

Twitter, Inc.

Display screen with graphical user interface

US10554596B1

( en )

2019-03-28

2020-02-04

Wormhole Labs, Inc.

Context linked messaging system

US11494168B2

( en )

2019-04-03

2022-11-08

HIA Technologies Inc.

Computer system and method for facilitating an interactive conversational session with a digital conversational character in an augmented environment

FI20195293A1

( en )

*

2019-04-11

2020-10-12

Desigence Oy

A client device, a host computer, a system and a method for a virtual object on augmented reality

US11049072B1

( en )

2019-04-26

2021-06-29

State Farm Mutual Automobile Insurance Company

Asynchronous virtual collaboration environments

US11032328B1

( en )

*

2019-04-29

2021-06-08

State Farm Mutual Automobile Insurance Company

Asymmetric collaborative virtual environments

US11467794B1

( en )

2019-04-30

2022-10-11

Splunk Inc.

Displaying device-associated data in an extended reality environment

US11461378B1

( en )

2019-04-30

2022-10-04

Splunk Inc.

Generating dashboards based on user-based filters

US11853366B1

( en )

*

2019-04-30

2023-12-26

Splunk Inc.

Generating extended reality views based on user-based filters

JP7397883B2

( en )

*

2019-05-31

2023-12-13

アップル インコーポレイテッド

Presentation of communication data based on environment

CN110276006A

( en )

*

2019-06-05

2019-09-24

北京字节跳动网络技术有限公司

Methods of exhibiting, device, electronic equipment and the computer readable storage medium of painted eggshell

WO2021021328A2

( en )

*

2019-06-14

2021-02-04

Quantum Interface, Llc

Predictive virtual training systems, apparatuses, interfaces, and methods for implementing same

US10897564B1

( en )

2019-06-17

2021-01-19

Snap Inc.

Shared control of camera device by multiple devices

CA3139465A1

( en )

2019-06-20

2020-12-24

Barrie A. Loberg

Voice communication system within a mixed-reality environment

US11189098B2

( en )

*

2019-06-28

2021-11-30

Snap Inc.

3D object camera customization system

US11775130B2

( en )

*

2019-07-03

2023-10-03

Apple Inc.

Guided retail experience

US11816800B2

( en )

2019-07-03

2023-11-14

Apple Inc.

Guided consumer experience

US11941762B2

( en )

2019-07-15

2024-03-26

Samsung Electronics Co., Ltd.

System and method for augmented reality scenes

US11340857B1

( en )

2019-07-19

2022-05-24

Snap Inc.

Shared control of a virtual object by multiple devices

US11704874B2

( en )

2019-08-07

2023-07-18

Magic Leap, Inc.

Spatial instructions and guides in mixed reality

US12190579B2

( en )

*

2019-08-09

2025-01-07

Sony Group Corporation

Image processing apparatus, image processing method, and program

WO2021029497A1

( en )

*

2019-08-14

2021-02-18

Samsung Electronics Co., Ltd.

Immersive display system and method thereof

US11232646B2

( en )

2019-09-06

2022-01-25

Snap Inc.

Context-based virtual object rendering

US11307647B2

( en )

2019-09-11

2022-04-19

Facebook Technologies, Llc

Artificial reality triggered by physical object

US11132827B2

( en )

2019-09-19

2021-09-28

Facebook Technologies, Llc

Artificial reality system architecture for concurrent application execution and collaborative 3D scene rendering

WO2021055959A1

( en )

*

2019-09-20

2021-03-25

Fabric Global Pbc

Augmented reality public messaging experience

KR102768181B1

( en )

*

2019-10-07

2025-02-17

주식회사 플랫팜

An apparatus for providing message services building an expression item database adoptable to a augmented reality and a method using it

CN114586071B

( en )

2019-10-15

2026-03-03

奇跃公司

Cross-reality system supporting multiple device types

JP7150894B2

( en )

2019-10-15

2022-10-11

ベイジン・センスタイム・テクノロジー・デベロップメント・カンパニー・リミテッド

AR scene image processing method and device, electronic device and storage medium

US11727532B1

( en )

2019-11-03

2023-08-15

Wells Fargo Bank N.A.

Workplace digital billboard experience

US10990251B1

( en )

*

2019-11-08

2021-04-27

Sap Se

Smart augmented reality selector

CN112905243B

( en )

*

2019-11-15

2022-05-13

成都鼎桥通信技术有限公司

Method and device for simultaneously operating double systems

WO2021118962A1

( en )

2019-12-09

2021-06-17

Magic Leap, Inc.

Cross reality system with simplified programming of virtual content

US11816757B1

( en )

*

2019-12-11

2023-11-14

Meta Platforms Technologies, Llc

Device-side capture of data representative of an artificial reality environment

US10715851B1

( en )

*

2019-12-16

2020-07-14

BigScreen, Inc.

Digital rights managed virtual reality content sharing

FR3104731A1

( en )

*

2019-12-17

2021-06-18

Orange

Interaction device, survey method, virtual reality headset, method of reproducing a real point in a virtual space, device and method of virtualizing a real space

WO2021121932A1

( en )

*

2019-12-17

2021-06-24

Rhizomenet Pty. Ltd.

Message distribution service

US11240631B2

( en )

2019-12-17

2022-02-01

Rhizomenet Pty. Ltd.

Message distribution service

CN111010586B

( en )

*

2019-12-19

2021-03-19

腾讯科技（深圳）有限公司

Live broadcast method, device, equipment and storage medium based on artificial intelligence

US11227442B1

( en )

2019-12-19

2022-01-18

Snap Inc.

3D captions with semantic graphical elements

US11263817B1

( en )

2019-12-19

2022-03-01

Snap Inc.

3D captions with face tracking

JP1696319S

( en )

*

2019-12-25

2021-10-04

US11488373B2

( en )

2019-12-

Related documents

• Generating augmented reality experiences utilizing physical objects to … — Meta Platforms Technologies, Llc (US20260037214A1)#653960
• Virtual reality system using multiple force arrays for a solver — DeepMotion, Inc. (US10657696B2)#653961
• Tracking position of device inside-out for virtual reality interactivity — Sony Interactive Entertainment America Llc (US10535153B2)#653962
• Augmented reality system with spatialized audio tied to user manipulated … — Magic Leap, Inc. (US11231770B2)#653963
• Big data-based autonomous flight drone system and autonomous flight method … — Rgblab Co., Ltd. (US20210390867A1)#653964