ABSTRACT
Abstract
The present method comprises: obtaining a texture of one or more 2D application interfaces to be displayed; determining a virtual reality scene to be displayed, and writing the virtual reality scene into a frame buffer of an Android system in a left-right screen splitting mode using an OpenGL function; sketching contents in the frame buffer of the Android system onto left and right screens of the virtual reality device respectively to form a virtual screen in the virtual reality scene; sketching the obtained texture of the one or more 2D application interfaces to be displayed onto the virtual screen in the virtual reality scene of the left and right screens, respectively. The method renders the 2D application interface as images in left and right eyes simultaneously, with a three-dimensional sense, and facilitates the use of a large number of existing Android applications in the virtual reality system.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
This application is a national phase entry of International Application No. PCT/CN2016/074160, filed Feb. 19, 2016 and titled âA Method And Apparatus For Displaying 2D Application Interface In Virtual Reality Device,â which claims priority to and the benefit of Chinese Patent Application No.: 201511026678.8, filed Dec. 31, 2015 and titled âA Method And Apparatus For Displaying 2D Application Interface In Virtual Reality Device.â The contents of the above-identified Applications are relied upon and incorporated herein by reference in their entirety.
FIELD OF THE INVENTION
The invention relates to the field of virtual reality, in particular to a method and apparatus for displaying 2D application interface in virtual reality devices.
BACKGROUND OF THE INVENTION
A virtual reality system is a kind of computer simulation system that can create a virtual world experienced by users. It is a system simulation of interactive 3D dynamic scene and physical activity with multi-source information mixed. The virtual reality system uses a computer to generate an emulated environment and enables users to immerse in the environment. For example, users achieve their interaction with the virtual reality system by wearing virtual reality helmets or other virtual reality devices.
In recent years, because of its advantages of open-source and complete ecological environment, Android system has gradually become the first choice of many virtual reality operating systems. However, the virtual reality device requires to render a left-eye picture and a right-eye picture to produce a dimensional sense. Since the existing Android applications are not developed and designed particularly for the virtual reality device, they typically use 2D application interface and do not meet the demands for virtual reality device. Therefore, huge amounts of Android applications cannot be used in a virtual reality system, which leads to the lack of applications and poor ecological environment of virtual reality systems.
SUMMARY OF THE INVENTION
Pertaining to aforementioned problems, this disclosure provides a method and apparatus for displaying 2D application interface in virtual reality devices that overcomes the aforementioned problems or at least partially addresses the aforementioned problems.
According to one aspect of the disclosure, it provides a method of displaying a 2D application interface in a virtual reality device, including:
obtaining a texture of one or more 2D application interfaces to be displayed;
determining a virtual reality scene to be displayed, and writing the virtual reality scene into a frame buffer of the Android system in a left-right screen splitting mode using an OpenGL function;
sketching contents in the frame buffer of the Android system onto the left and right screens of the virtual reality device respectively to form a virtual screen in the virtual reality scene;
sketching the obtained texture of the one or more 2D application interfaces to be displayed onto the virtual screen in the virtual reality scene of the left and right screens, respectively.
Alternatively, obtaining the texture of the one or more 2D application interfaces to be displayed includes:
applying for the corresponding layers for the one or more 2D application interfaces to be displayed, respectively;
calling a SurfaceFlinger module in the Android system that is responsible for display and synthesis, labelling the synthesis mode as a GLES synthesis mode in the setUp HWComposer( ) function of the SurfaceFlinger module, and synthesizing the layers using the GLES synthesis mode.
Alternatively, synthesizing the layers using the GLES synthesis mode comprises:
determining display relations of the layers;
creating a texture object bound to GL_TEXTURE_2D and a frame buffer object bound to GL_FRAMBUFFER function using the OpenGL function, and correlating the frame buffer object with the texture object; and
according to the display relations of the layers, sketching the 2D application interfaces to be displayed in each of the layers into the texture object.
Alternatively, sketching the obtained texture of the one or more 2D application interfaces to be displayed onto the virtual screen in the virtual reality scene of the left and right screens includes:
obtaining the texture of the one or more 2D application interfaces to be displayed from the frame buffer object related to the texture object and sketching the texture of the one or more 2D application interfaces to be displayed onto the virtual reality screen of the left and right screens respectively using the OpenGL function.
Alternatively, determining the virtual reality scene to be displayed includes: obtaining data of a user's head state by a sensor of the virtual reality device, and determining the virtual reality scene to be displayed based on the data of the user's head state.
According to another aspect of the disclosure, an apparatus for displaying 2D application interface in a virtual reality device is provided. The apparatus includes:
a 2D application interface processing unit configured to obtain a texture of one or more 2D application interfaces to be displayed;
a virtual reality scene processing unit configured to determine a virtual reality scene to be displayed, and write the virtual reality scene into a frame buffer of an Android system in a left-right screen splitting mode using an OpenGL function;
a sketching unit configured to sketch contents in the frame buffer of the Andrew system onto left and right screens of the virtual reality device respectively to form a virtual screen in the virtual reality scene, and to sketch the obtained texture of the one or more 2D application interfaces to be displayed onto the virtual screen in the virtual reality scene of the left and right screens, respectively.
Alternatively, the 2D application interface processing unit includes:
a layer applying module configured to apply for the corresponding layers for the one or more 2D application interfaces to be displayed, respectively;
a synthesis module configured to call a SurfaceFlinger module in the Android system that is responsible for display synthesis, to label the synthesis mode as a GLES synthesis mode in the setUp HWComposer( ) function of the SurfaceFlinger module, and to synthesize the layers using the GLES synthesis mode.
Alternatively, the synthesis module is specifically configured to determine display relations of the layers, to create a texture object bound to GL_TEXTURE_2D and a frame buffer object bound to GL_FRAMBUFFER by the OpenGL function, to correlate the frame buffer object with the texture object, and to sketch the 2D application interfaces to be displayed in each of the layers into the texture object according to the display relations of the layers.
Alternatively, the sketching unit is specifically configured to obtain the texture of the one or more 2D application interfaces to be displayed from the frame buffer object correlated to the texture object, and to sketch the texture onto the virtual screen in the virtual reality scene of the left and right screens respectively using the OpenGL function.
Alternatively, the virtual reality scene processing unit is specifically configured to obtain data of a user's head state by a sensor of the virtual reality device, and to determine the virtual reality scene to be displayed based on the data of the user's head state.
Based on the discussion above, the technical scheme of this disclosure utilizes the following technical approaches to address the problem that the existing 2D application interface cannot be rendered in the virtual reality device of the virtual reality system: first, a texture of one or more 2D application interfaces to be displayed is obtained; and a virtual reality scene to be displayed is then determined; the determined virtual reality scene to be displayed is written into a frame buffer of the Android system in the left-right screen splitting mode using the OpenGL function; and the technique used in the Android system about reading contents of the frame buffer of the Android system for sketching is leveraged to display the virtual reality scene onto left and right screens of the virtual reality device, thus forming a virtual screen in the virtual reality scene; finally, the texture of the one or more 2D application interfaces to be displayed is sketched onto the virtual screen in the virtual reality scene of the left and right screens, respectively, so as to render the 2D application interface as images in left and right eyes simultaneously, with a three-dimensional sense. The method facilitates the use of a large number of existing Android applications in the virtual reality system, and thus is at low cost, simple and has improved the ecological environment of the virtual reality system and fits to practical use.
The above description only presents the outline of technical solution of this disclosure. In order to make the technical methods of this disclosure be understood more clearly, to enable the implementation of the methods in accordance with the specification, and to make the aforementioned and other purposes, features, and advantages of this disclosure more understandable, the following embodiments of the present disclosure are presented.
BRIEF DESCRIPTION OF THE DRAWINGS
By reading the following detailed description of the preferred embodiments below, a variety of other advantages and benefits of this disclosure will become clear to one of ordinary skill in the related art. Appended drawings are used only for illustrating the purpose of the preferred embodiments and are not considered as limitations of this disclosure. Moreover, throughout the drawings, the same reference symbols represent the same parts of the system. The brief description of the appended drawings is illustrated below.
FIG. 1 is a flow chart of a method for displaying the 2D application interface in the virtual reality device according to one embodiment of this disclosure.
FIG. 2 is the structural diagram of an apparatus for displaying the 2D application interface in the virtual reality device according to one embodiment of the disclosure.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
The exemplary embodiments of the invention will be described in more detail according to the appended drawings. Although the drawings show the exemplary embodiments of this disclosure, it should be understood that this disclosure may be implemented in various ways and should not be limited to the embodiments stated here. On the contrary, presenting these embodiments is to facilitate more thoroughly understanding of this disclosure, and to completely convey the scope of this disclosure to one of ordinary skill in the related art.
Referring to FIG. 1 , illustrated is a flow chart of the method of displaying the 2D application interface in the virtual reality device according to one embodiment of the disclosure. As shown in FIG. 1 , the method includes:
Step S 110 , obtaining a texture of one or more 2D application interfaces to be displayed.
In the Android system, the texture is the specific contents of the application interfaces that is loaded into the graphics memory and needs to be shown on the screen of the display device, such as a picture in the pvr format. In a more complex case, for example, a user will run an application A, and while the application A is started an appended advertisement application B will be run, in which the application B is displayed only in the central area of the screen. At this time, a plurality of 2D application interfaces to be displayed are synthesized to acquire a synthesized texture.
Step S 120 , determining a virtual reality scene to be displayed, and writing the virtual reality scene into a frame buffer of the Android system using the OpenGL functioning the left-right screen splitting mode using an OpenGL function.
Step S 130 , sketching the contents in the frame buffer of the Android system onto left and right screens of the virtual reality device respectively to form a virtual screen in the virtual reality scene.
In the Android system, writing data in a specific format into the Framebuffer of the Android system means outputting the displayed contents to the screen. Therefore, the data written in the Framebuffer at step S 120 is not processed between step S 120 and step S 130 , but only displayed on the left and right screens in the virtual reality device to form the virtual screen in the virtual reality scene. In particular, a grid can be established for the left and right screens of the virtual reality system. Therefore, when writing the virtual reality scene into the frame buffer of the Android system in the left-right screen splitting mode using the OpenGL function at step S 120 , related information about the grid for sketching the virtual reality scene is included in order to sketch the virtual reality scene to the grid of the left screen and right screens respectively and to display the result.
Step S 140 , sketching the obtained texture of the one or more 2D application interfaces to be displayed onto the virtual screen in the virtual reality scene of the left and right screens, respectively.
For example, if the virtual reality scene is a living room, the virtual screen of the virtual reality scene may be a rear projection TV in the living room; and if the virtual reality scene is a cinema, the virtual screen of the virtual reality scene may be the movie screen in the cinema and so on.
It can be seen that, in order to address the existing problem that the 2D application interface cannot be rendered on the virtual reality device of the virtual reality system, the method shown in FIG. 1 utilizes the following techniques: first, a texture of one or more 2D application interfaces to be displayed is acquired; and a virtual reality scene to be displayed is then determined; the determined virtual reality scene to be displayed is written into a frame buffer of the Android system in the left-right screen splitting mode by using a OpenGL function; the technique used in the Android system about reading contents of the frame buffer of the Android system for sketching is leveraged to display the virtual reality scene onto left and right screens of the virtual reality device, thus forming a virtual screen in the virtual reality scene; finally, the texture of the one or more 2D application interfaces to be displayed is sketched onto the virtual screen in the virtual reality scene of the left and right screens, respectively, so as to render the 2D application interface as images in left and right eyes simultaneously, with a three-dimensional sense. The method facilitates the use of a large number of existing Android applications in the virtual reality system, and thus is at low cost, simple and has improved the ecological environment of the virtual reality system and fits for practical use.
<div id="p-0047" num="0046" class="desc
CROSS-REFERENCE TO RELATED APPLICATIONS
This application is a national phase entry of International Application No. PCT/CN2016/074160, filed Feb. 19, 2016 and titled âA Method And Apparatus For Displaying 2D Application Interface In Virtual Reality Device,â which claims priority to and the benefit of Chinese Patent Application No.: 201511026678.8, filed Dec. 31, 2015 and titled âA Method And Apparatus For Displaying 2D Application Interface In Virtual Reality Device.â The contents of the above-identified Applications are relied upon and incorporated herein by reference in their entirety.
FIELD OF THE INVENTION
The invention relates to the field of virtual reality, in particular to a method and apparatus for displaying 2D application interface in virtual reality devices.
BACKGROUND OF THE INVENTION
A virtual reality system is a kind of computer simulation system that can create a virtual world experienced by users. It is a system simulation of interactive 3D dynamic scene and physical activity with multi-source information mixed. The virtual reality system uses a computer to generate an emulated environment and enables users to immerse in the environment. For example, users achieve their interaction with the virtual reality system by wearing virtual reality helmets or other virtual reality devices.
In recent years, because of its advantages of open-source and complete ecological environment, Android system has gradually become the first choice of many virtual reality operating systems. However, the virtual reality device requires to render a left-eye picture and a right-eye picture to produce a dimensional sense. Since the existing Android applications are not developed and designed particularly for the virtual reality device, they typically use 2D application interface and do not meet the demands for virtual reality device. Therefore, huge amounts of Android applications cannot be used in a virtual reality system, which leads to the lack of applications and poor ecological environment of virtual reality systems.
SUMMARY OF THE INVENTION
Pertaining to aforementioned problems, this disclosure provides a method and apparatus for displaying 2D application interface in virtual reality devices that overcomes the aforementioned problems or at least partially addresses the aforementioned problems.
According to one aspect of the disclosure, it provides a method of displaying a 2D application interface in a virtual reality device, including:
obtaining a texture of one or more 2D application interfaces to be displayed;
determining a virtual reality scene to be displayed, and writing the virtual reality scene into a frame buffer of the Android system in a left-right screen splitting mode using an OpenGL function;
sketching contents in the frame buffer of the Android system onto the left and right screens of the virtual reality device respectively to form a virtual screen in the virtual reality scene;
sketching the obtained texture of the one or more 2D application interfaces to be displayed onto the virtual screen in the virtual reality scene of the left and right screens, respectively.
Alternatively, obtaining the texture of the one or more 2D application interfaces to be displayed includes:
applying for the corresponding layers for the one or more 2D application interfaces to be displayed, respectively;
calling a SurfaceFlinger module in the Android system that is responsible for display and synthesis, labelling the synthesis mode as a GLES synthesis mode in the setUp HWComposer( ) function of the SurfaceFlinger module, and synthesizing the layers using the GLES synthesis mode.
Alternatively, synthesizing the layers using the GLES synthesis mode comprises:
determining display relations of the layers;
creating a texture object bound to GL_TEXTURE_2D and a frame buffer object bound to GL_FRAMBUFFER function using the OpenGL function, and correlating the frame buffer object with the texture object; and
according to the display relations of the layers, sketching the 2D application interfaces to be displayed in each of the layers into the texture object.
Alternatively, sketching the obtained texture of the one or more 2D application interfaces to be displayed onto the virtual screen in the virtual reality scene of the left and right screens includes:
obtaining the texture of the one or more 2D application interfaces to be displayed from the frame buffer object related to the texture object and sketching the texture of the one or more 2D application interfaces to be displayed onto the virtual reality screen of the left and right screens respectively using the OpenGL function.
Alternatively, determining the virtual reality scene to be displayed includes: obtaining data of a user's head state by a sensor of the virtual reality device, and determining the virtual reality scene to be displayed based on the data of the user's head state.
According to another aspect of the disclosure, an apparatus for displaying 2D application interface in a virtual reality device is provided. The apparatus includes:
a 2D application interface processing unit configured to obtain a texture of one or more 2D application interfaces to be displayed;
a virtual reality scene processing unit configured to determine a virtual reality scene to be displayed, and write the virtual reality scene into a frame buffer of an Android system in a left-right screen splitting mode using an OpenGL function;
a sketching unit configured to sketch contents in the frame buffer of the Andrew system onto left and right screens of the virtual reality device respectively to form a virtual screen in the virtual reality scene, and to sketch the obtained texture of the one or more 2D application interfaces to be displayed onto the virtual screen in the virtual reality scene of the left and right screens, respectively.
Alternatively, the 2D application interface processing unit includes:
a layer applying module configured to apply for the corresponding layers for the one or more 2D application interfaces to be displayed, respectively;
a synthesis module configured to call a SurfaceFlinger module in the Android system that is responsible for display synthesis, to label the synthesis mode as a GLES synthesis mode in the setUp HWComposer( ) function of the SurfaceFlinger module, and to synthesize the layers using the GLES synthesis mode.
Alternatively, the synthesis module is specifically configured to determine display relations of the layers, to create a texture object bound to GL_TEXTURE_2D and a frame buffer object bound to GL_FRAMBUFFER by the OpenGL function, to correlate the frame buffer object with the texture object, and to sketch the 2D application interfaces to be displayed in each of the layers into the texture object according to the display relations of the layers.
Alternatively, the sketching unit is specifically configured to obtain the texture of the one or more 2D application interfaces to be displayed from the frame buffer object correlated to the texture object, and to sketch the texture onto the virtual screen in the virtual reality scene of the left and right screens respectively using the OpenGL function.
Alternatively, the virtual reality scene processing unit is specifically configured to obtain data of a user's head state by a sensor of the virtual reality device, and to determine the virtual reality scene to be displayed based on the data of the user's head state.
Based on the discussion above, the technical scheme of this disclosure utilizes the following technical approaches to address the problem that the existing 2D application interface cannot be rendered in the virtual reality device of the virtual reality system: first, a texture of one or more 2D application interfaces to be displayed is obtained; and a virtual reality scene to be displayed is then determined; the determined virtual reality scene to be displayed is written into a frame buffer of the Android system in the left-right screen splitting mode using the OpenGL function; and the technique used in the Android system about reading contents of the frame buffer of the Android system for sketching is leveraged to display the virtual reality scene onto left and right screens of the virtual reality device, thus forming a virtual screen in the virtual reality scene; finally, the texture of the one or more 2D application interfaces to be displayed is sketched onto the virtual screen in the virtual reality scene of the left and right screens, respectively, so as to render the 2D application interface as images in left and right eyes simultaneously, with a three-dimensional sense. The method facilitates the use of a large number of existing Android applications in the virtual reality system, and thus is at low cost, simple and has improved the ecological environment of the virtual reality system and fits to practical use.
The above description only presents the outline of technical solution of this disclosure. In order to make the technical methods of this disclosure be understood more clearly, to enable the implementation of the methods in accordance with the specification, and to make the aforementioned and other purposes, features, and advantages of this disclosure more understandable, the following embodiments of the present disclosure are presented.
BRIEF DESCRIPTION OF THE DRAWINGS
By reading the following detailed description of the preferred embodiments below, a variety of other advantages and benefits of this disclosure will become clear to one of ordinary skill in the related art. Appended drawings are used only for illustrating the purpose of the preferred embodiments and are not considered as limitations of this disclosure. Moreover, throughout the drawings, the same reference symbols represent the same parts of the system. The brief description of the appended drawings is illustrated below.
FIG. 1 is a flow chart of a method for displaying the 2D application interface in the virtual reality device according to one embodiment of this disclosure.
FIG. 2 is the structural diagram of an apparatus for displaying the 2D application interface in the virtual reality device according to one embodiment of the disclosure.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
The exemplary embodiments of the invention will be described in more detail according to the appended drawings. Although the drawings show the exemplary embodiments of this disclosure, it should be understood that this disclosure may be implemented in various ways and should not be limited to the embodiments stated here. On the contrary, presenting these embodiments is to facilitate more thoroughly understanding of this disclosure, and to completely convey the scope of this disclosure to one of ordinary skill in the related art.
Referring to FIG. 1 , illustrated is a flow chart of the method of displaying the 2D application interface in the virtual reality device according to one embodiment of the disclosure. As shown in FIG. 1 , the method includes:
Step S 110 , obtaining a texture of one or more 2D application interfaces to be displayed.
In the Android system, the texture is the specific contents of the application interfaces that is loaded into the graphics memory and needs to be shown on the screen of the display device, such as a picture in the pvr format. In a more complex case, for example, a user will run an application A, and while the application A is started an appended advertisement application B will be run, in which the application B is displayed only in the central area of the screen. At this time, a plurality of 2D application interfaces to be displayed are synthesized to acquire a synthesized texture.
Step S 120 , determining a virtual reality scene to be displayed, and writing the virtual reality scene into a frame buffer of the Android system using the OpenGL functioning the left-right screen splitting mode using an OpenGL function.
Step S 130 , sketching the contents in the frame buffer of the Android system onto left and right screens of the virtual reality device respectively to form a virtual screen in the virtual reality scene.
In the Android system, writing data in a specific format into the Framebuffer of the Android system means outputting the displayed contents to the screen. Therefore, the data written in the Framebuffer at step S 120 is not processed between step S 120 and step S 130 , but only displayed on the left and right screens in the virtual reality device to form the virtual screen in the virtual reality scene. In particular, a grid can be established for the left and right screens of the virtual reality system. Therefore, when writing the virtual reality scene into the frame buffer of the Android system in the left-right screen splitting mode using the OpenGL function at step S 120 , related information about the grid for sketching the virtual reality scene is included in order to sketch the virtual reality scene to the grid of the left screen and right screens respectively and to display the result.
Step S 140 , sketching the obtained texture of the one or more 2D application interfaces to be displayed onto the virtual screen in the virtual reality scene of the left and right screens, respectively.
For example, if the virtual reality scene is a living room, the virtual screen of the virtual reality scene may be a rear projection TV in the living room; and if the virtual reality scene is a cinema, the virtual screen of the virtual reality scene may be the movie screen in the cinema and so on.
It can be seen that, in order to address the existing problem that the 2D application interface cannot be rendered on the virtual reality device of the virtual reality system, the method shown in FIG. 1 utilizes the following techniques: first, a texture of one or more 2D application interfaces to be displayed is acquired; and a virtual reality scene to be displayed is then determined; the determined virtual reality scene to be displayed is written into a frame buffer of the Android system in the left-right screen splitting mode by using a OpenGL function; the technique used in the Android system about reading contents of the frame buffer of the Android system for sketching is leveraged to display the virtual reality scene onto left and right screens of the virtual reality device, thus forming a virtual screen in the virtual reality scene; finally, the texture of the one or more 2D application interfaces to be displayed is sketched onto the virtual screen in the virtual reality scene of the left and right screens, respectively, so as to render the 2D application interface as images in left and right eyes simultaneously, with a three-dimensional sense. The method facilitates the use of a large number of existing Android applications in the virtual reality system, and thus is at low cost, simple and has improved the ecological environment of the virtual reality system and fits for practical use.
In one embodiment of this disclosure, at step S 110 of the method shown in FIG. 1 , obtaining a texture of one or more 2D application interfaces to be displayed includes: applying for the corresponding layers for the one or more 2D application interfaces to be displayed, respectively; calling a SurfaceFlinger module in the Android system that is responsible for display and synthesis, labelling the synthesis mode as a GLES synthesis mode in the setUp HWComposer( ) function of the SurfaceFlinger module and synthesizing the layers using the GLES synthesis mode.
As mentioned above, in the Android system, when multiple 2D application interfaces are to be displayed on the screen of the display device, the interfaces to be displayed are synthesized. Before calling the SurfaceFlinger module for synthesizing the 2D application interfaces in software mode, a Client category shall be created, and a layer is then applied for from the SurfaceFlinger module. In the Android system, synthesizing the 2D application interfaces may be implemented by calling and Overlay by hardware, or calling the SurfaceFlinger by software, wherein the manner of hardware is simple but not suitable to implement the technical solution of this disclosure. Therefore, the method of calling the SurfaceFlinger is proposed in this embodiment. Likewise, in the process of calling SurfaceFlingere, the following procedures are performed: calling the preComposition( ) function to start the synthesis; calling the rebuilidLayerStacks function to reconstruct a layer stack to be synthesized; and calling the setUp HWComposer( ) function to set a synthesis mode. In the existing techniques, taking into account the synthesis efficiency, SurfaceFlinger often utilizes the HWC synthetic mode to synthesize the layers, which is also not suitable for the technical solution of the embodiment. Therefore, in this embodiment, the synthesis mode is labelled as the GLES synthesis mode.
After the above procedures, SurfaceFlinger further calls the doDebugFlashRegions( ) function for debugging, and then performs the synthesis which is the most important stage. SurfaceFlinger can call the deComposition( ) function for synthesis. During the execution of this function, the dodisplay Composition ( ) function can be called to accomplish the synthesis for each display device so as to achieve displaying on multiple display devices such as a tablet computer and a smart phone. In the execution of the dodisplayComposition( ) function, the doComposeSurfaces ( ) function can also be called to sketch the contents of each of the layers to the Framebuffer of the Android system. However, in the technical scheme of the disclosure, since the virtual reality scene is to be written into the Framebuffer of the Android system, the one or more 2D application interfaces to be displayed cannot be processed in the above-described way.
Therefore, specifically, in the above method according to one embodiment of the disclosure, synthesizing the layers using the GLES synthesis mode includes: determining the display relations of the layers; creating a texture object bound to GL_TEXTURE_2D and a frame buffer object bound to GL_FRAMBUFFER using the OpenGL function, and correlating the frame buffer object with the texture object; and according to the display relationship of the layers, sketching the 2D application interfaces to be displayed in each of the layers into the texture object.
Below are exemplary codes of one specific embodiment for creating a texture object bound to GL_TEXTURE_2D and a frame buffer object bound to GL_FRAMBUFFER, and correlating the frame buffer object with the texture object:
//Generate a texture object and bind the object to GL_TEXTURE_2D and then initialize glGenTextures(1, &mScreenTexture); glBindTexture(GL_TEXTURE_2D, mScreenTexture); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_RGBA, GL_UNSIGNED_BYTE, 0); //Generate a Framebuffer object and bind the object to GL_FRAMEBUFFER glGenFramebuffers(1, &m Framebuffer); glBindFramebuffer(GL_FRAMEBUFFER, mFramebuffer); //Correlate Texture with Framebuffer. glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, mScreenTexture, 0);
This way allows the synthesis results of the ComposeSurfaces ( ) function to be sketched to the created texture object, thereby converting the 2D application interfaces into a texture object.
Those skilled in the art should understand that the example of synthesizing the layers in the GLES synthesis mode provided in this embodiment does not pose restrictions to other embodiments. The advantage of the embodiment is that it sketches the 2D application interfaces to be displayed in each of the layers into the new texture object according to the display relations without using the Frambuffer of the Android system.
The texture object is bound to another new frame buffer object, Frambuffer. Therefore, specifically, in one embodiment of the present disclosure, sketching the obtained texture of the one or more 2D application interfaces to be displayed onto the virtual screen in the virtual reality scene of the left and right screens respectively includes: obtaining the texture of the one or more 2D application interfaces to be displayed from the frame buffer object correlated to the texture object, and sketching the texture onto the virtual screen in the virtual reality scene of the left and right screens respectively using the OpenGL function. In the above embodiment, the OpenGL ES function may be specifically used to complete the steps of sketching the texture onto the virtual screen, creating a texture object bound to the GL_TEXTURE_2D and a frame buffer object bound to the GL_FRAMBUFFER and correlating the frame buffer object with the texture object.
In one embodiment of the disclosure, in the method described above, determining the virtual reality scene to be displayed includes: obtaining the data of the user's head state by a sensor of the virtual reality device, and determining the virtual reality scene to be displayed according to the data of the user's head state.
For example, obtaining the data of users' head state from the sensor of the virtual reality device, e.g., the angle of the head, facing orientation, etc., can be implemented as the following. Specifically, the specific value of the rotation vector can be obtained from such a device as a linear acceleration sensor. The different angles presented when the virtual reality scene is rendered on the screen of the virtual reality device are determined according to these data. In this case, users can feel like being in a virtual scene. It greatly enriches the users' experience and allows the user to have an excellent immersive feeling.
FIG. 2 shows a structural diagram of an apparatus for displaying a 2D application interface in a virtual reality device according to one embodiment of the disclosure. As it shown in FIG. 2 , the apparatus 200 for displaying the 2D application interface in the virtual reality device includes:
a 2D application interface processing unit 210 configured to obtain a texture of one or more 2D application interfaces to be displayed;
a virtual reality scene processing unit 220 configured to determine a virtual reality scene to be displayed and write the virtual reality scene into a frame buffer of an Android system in a left-right screen splitting mode using an OpenGL function;
a sketching unit 230 configured to sketch contents in the frame buffer of the Android system onto left and right screens of the virtual reality device respectively to form a virtual screen in the virtual reality scene, and sketch the obtained texture of the one or more 2D application interfaces to be displayed onto the virtual screen in the virtual reality scene of the left and right screens, respectively.
It can be seen that to address the existing problem of not being able to render a 2D application interface on a virtual reality device in the virtual reality system, the apparatus shown in FIG. 2 takes the following techniques: first, the 2D application interface processing unit 210 obtains the texture of one or more 2D application interfaces to be displayed; the virtual reality scene processing unit 220 further determines the virtual reality scene to be displayed and writes the virtual reality scene to be displayed into the frame buffer of the Android system in the left-right screen splitting mode using the OpenGL function; the sketching unit 230 complete the display of the virtual reality scene on the left and right screens of the virtual reality device using the Android system's technology of reading the contents in the frame buffer for sketching, to form a virtual screen in the virtual reality scene; finally, the texture of the one or more 2D application interfaces to be displayed is sketched onto the virtual screen in the virtual reality scene of the left and right screens respectively so as to render the 2D application interfaces as the images in the left and right eyes simultaneously, with a three-dimensional sense. This method allows a large number of existing Android applications to be used in a virtual reality system, and thus is at low cost, simple and has improved the ecological environment of the virtual reality system and fits to practical use.
In one embodiment of this disclosure, in the apparatus shown in FIG. 2 , the 2D application interface processing unit 210 includes:
a layer applying module 211 configured to apply for corresponding layers for the one or more 2D application interfaces to be displayed respectively;
a synthesis module 212 configured to call a SurfaceFlinger module in the Android system that is responsible for display synthesis, to label the synthesis mode as a GLES synthesis mode in the setUpHWComposer ( ) function of the SurfaceFlinger module and to synthesize the layers using the GLES synthesis mode.
In one embodiment of this disclosure, in the above-described apparatus, the synthesis module 212 is specifically configured to determine the display relations of the layers. The synthesis module 212 creates a texture object bound to GL_TEXTURE_2D and a frame buffer object bound to GL_FRAMBUFFER using the OpenGL function, and correlates the frame buffer object with the texture object. According to the display relations of the layers, the synthesis module 212 sketches the 2D application interfaces to be displayed in each of the layers into the texture object.
In one embodiment of the disclosure, in the above-described apparatus, the sketching unit 230 is specifically configured to obtain the texture of one or more 2D application interfaces to be displayed from the frame buffer object associated with the texture object and use the OpenGL function to sketch the texture onto the virtual screen in the virtual reality scene of the left and right screens respectively.
In one embodiment of the disclosure, in the above-described apparatus, the virtual reality scene processing unit 220 is specifically configured to obtain the data of a user's head state through a sensor of the virtual reality device and determine the virtual reality scene to be displayed according to the data of the user's head state.
It should be noted that the specific implementation of the above-mentioned embodiments of the apparatus of FIG. 2 is the same as the implementation of the previously described corresponding embodiments of the method respectively. Therefore they will not be described again here.
In summary, in view of the existing problem of not being able to render the 2D application interface on a virtual reality device in the virtual reality system, the present disclosure takes the following techniques: first, the texture of the one or more 2D application interfaces to be displayed is obtained; then the virtual reality scene to be displayed is determined; the virtual reality scene is written into the frame buffer of the Android system in the left-right screen splitting mode using the OpenGL function; the virtual reality scene being displayed on the left and right screens of the virtual reality device is realized based on the technology of the Android system about reading the contents in the frame buffer for sketching, thus forming the virtual screen in the virtual reality scene; finally, the texture of one or more 2D application interfaces to be displayed is sketched onto the virtual screen in the virtual reality scene of the left and right screens respectively so as to render the 2D application interfaces as the images in left and right eyes simultaneously, with a three-dimensional sense. This method allows a large number of existing Android applications to be used in a virtual reality system, and therefore is at low cost, simple and has improved the ecological environment of the virtual reality system and fits to practical use.
The above description is only preferred embodiments of the disclosure, not intended to limit the protecting scope of this disclosure. Any modification, equivalent replacement, improvement etc. within the spirit and principles of this disclosure are included within the protecting scope of this disclosure.
Claims ( 10 )
The invention claimed is:
1. A method of displaying a 2D application interface in a virtual reality device, the method comprising:
obtaining a texture of one or more 2D application interfaces to be displayed;
determining a virtual reality scene to be displayed;
establishing a grid for left and right screens of the virtual reality device;
after establishing the grid, writing the virtual reality scene into a frame buffer of an Android system in a left-right screen splitting mode using an OpenGL function;
sketching contents in the frame buffer of the Android system onto the grid of left and right screens of the virtual reality device to form a virtual screen in the virtual reality scene; and
after that the virtual screen is formed by sketching the contents onto the grid, sketching the obtained texture of the one or more 2D application interfaces onto the virtual screen in the virtual reality scene of the left and right screens.
2. The method of claim 1 , wherein obtaining the texture of the one or more 2D application interfaces to be displayed comprises:
applying for corresponding layers for the one or more 2D application interfaces to be displayed calling a SurfaceFlinger module in the Android system that is responsible for display and synthesis, labelling the synthesis mode as a GLES synthesis mode in the setUp HWComposer( ) function of the SurfaceFlinger module, and
synthesizing the layers using the GLES synthesis mode.
3. The method of claim 2 , wherein synthesizing the layers using the GLES synthesis mode comprises:
determining display relations of the layers;
creating a texture object bound to GL_TEXTURE_2D and a frame buffer object bound to GL_FRAMEBUFFER using the OpenGL function;
correlating the frame buffer object with the texture object; and
according to the display relations of the layers, sketching the 2D application interfaces to be displayed in each of the layers into the texture object.
4. The method of claim 3 , wherein sketching the obtained texture of the one or more 2D application interfaces to be displayed onto the virtual screen in the virtual reality scene of the left and right screens comprises:
obtaining the texture of the one or more 2D application interfaces to be displayed from the frame buffer object correlated to the texture object; and
sketching the texture onto the virtual screen in the virtual reality scene of the left and right screens respectively using the OpenGL function.
5. The method of claim 4 , wherein determining the virtual reality scene to be displayed comprises:
obtaining data of a user's head state by a sensor of the virtual reality device; and
determining the virtual reality scene to be displayed based on the data of the user's head state.
6. A virtual reality device comprising a processor configured to:
obtain a texture of one or more 2D application interfaces to be displayed;
determine a virtual reality scene to be displayed;
establishing a grid for left and right screens of the virtual reality device;
after establishing the grid, write the virtual reality scene into a frame buffer of an Android system in a left-right screen splitting mode using an OpenGL function;
sketch contents in the frame buffer of the Android system onto the grid of the left and right screens of the virtual reality device to form a virtual screen in the virtual reality scene; and
after that the virtual screen is formed by sketching the contents onto the grid, sketch the obtained texture of the one or more 2D application interfaces to be displayed onto the virtual screen in the virtual reality scene of the left and right screens.
7. The virtual reality device of claim 6 , wherein the processor is further configured to:
apply for corresponding layers for the one or more 2D application interfaces to be displayed;
call a SurfaceFlinger module in the Android system that is responsible for display and synthesis, to label the synthesis mode as a GLES synthesis mode in the setUp HWComposer ( ) function of the SurfaceFlinger module; and
synthesize the layers using the GLES synthesis mode.
8. The virtual reality device of claim 7 , wherein the processor is further configured to:
determine display relations of the layers to create a texture object bound to GL_TEXTURE_2D and a frame buffer object bound to_GL_FRAMEBUFFER using the OpenGL function, correlate the frame buffer object with the texture object; and
sketch the 2D application interfaces to be displayed in each of the layers into the texture object according to the display relations of the layers.
9. The virtual reality device of claim 8 , wherein the processor is further configured to:
obtain the texture of the one or more 2D application interfaces to be displayed from the frame buffer object correlated to the texture object; and
sketch the texture onto the virtual screen in the virtual reality scene of the left and right screens respectively using the OpenGL function.
10. The virtual reality device of claim 9 , wherein the processor is further configured to:
obtain data of a user's head state by a sensor of the virtual reality device; and
determine the virtual reality scene to be displayed based on the data of the user's head state.
US15/115,092
2015-12-31
2016-02-19
Method and apparatus for displaying 2D application interface in virtual reality device
Active
2037-12-25
US10902663B2
( en )
Applications Claiming Priority (4)
Application Number
Priority Date
Filing Date
Title
CN201511026678.8A
CN105447898B
( en )
2015-12-31
2015-12-31
The method and apparatus of 2D application interface are shown in a kind of virtual reality device
CN201511026678
2015-12-31
CN201511026678.8
2015-12-31
PCT/CN2016/074160
WO2017113488A1
( en )
2015-12-31
2016-02-19
Method and apparatus for displaying 2d application interface in virtual reality device
Publications (2)
Publication Number
Publication Date
US20180018806A1
US20180018806A1 ( en )
2018-01-18
US10902663B2
true
US10902663B2 ( en )
2021-01-26
Family
ID=55558029
Family Applications (1)
Application Number
Title
Priority Date
Filing Date
US15/115,092
Active
2037-12-25
US10902663B2
( en )
2015-12-31
2016-02-19
Method and apparatus for displaying 2D application interface in virtual reality device
Country Status (3)
Country
Link
US
( 1 )
US10902663B2
( en )
CN
( 1 )
CN105447898B
( en )
WO
( 1 )
WO2017113488A1
( en )
Families Citing this family (41)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
CN105657408B
( en )
*
2015-12-31
2018-11-30
å京å°é¸ççç§ææéå ¬å¸
The implementation method and virtual reality device of virtual reality scenario
US20170262049A1
( en )
*
2016-03-11
2017-09-14
Empire Technology Development Llc
Virtual reality display based on orientation offset
CN105913477B
( en )
*
2016-04-18
2019-02-12
å京å°é¸ççç§ææéå ¬å¸
A kind of method and apparatus for realizing virtual reality system solid starting animation
CN105957130B
( en )
*
2016-04-18
2019-08-02
å京å°é¸ççç§ææéå ¬å¸
A kind of method and apparatus for realizing virtual reality system plane starting animation
US10043302B2
( en )
2016-04-18
2018-08-07
Beijing Pico Technology Co., Ltd.
Method and apparatus for realizing boot animation of virtual reality system
CN106098022B
( en )
*
2016-06-07
2019-02-12
å京å°é¸ççç§ææéå ¬å¸
A kind of method and apparatus shortening picture delay
CN106126021A
( en )
*
2016-06-21
2016-11-16
ä¸æµ·ä¹ç¸ç§ææéå ¬å¸
A kind of interface display method and device
CN106201724A
( en )
*
2016-07-14
2016-12-07
ä¹è§æ§è¡ï¼å京ï¼æéå ¬å¸
A kind of Virtual Reality equipment and the processing method of user interface UI event thereof
CN107817894A
( en )
*
2016-09-12
2018-03-20
ä¸å ´é讯è¡ä»½æéå ¬å¸
Display processing method and device
CN106507093A
( en )
*
2016-09-26
2017-03-15
å京å°é¸ççç§ææéå ¬å¸
A kind of display mode switching method of virtual reality device and device
CN106570927A
( en )
*
2016-10-14
2017-04-19
æ å·Tcl移å¨éä¿¡æéå ¬å¸
Method of realizing virtual reality based on Android system, terminal and system
CN106502396B
( en )
*
2016-10-20
2020-10-23
ç½æï¼æå·ï¼ç½ç»æéå ¬å¸
Virtual reality system, interaction method and device based on virtual reality
CN108604385A
( en )
*
2016-11-08
2018-09-28
å为ææ¯æéå ¬å¸
A kind of application interface display methods and device
<span it
CLAIMS
Claims ( 10 )
The invention claimed is:
1. A method of displaying a 2D application interface in a virtual reality device, the method comprising:
obtaining a texture of one or more 2D application interfaces to be displayed;
determining a virtual reality scene to be displayed;
establishing a grid for left and right screens of the virtual reality device;
after establishing the grid, writing the virtual reality scene into a frame buffer of an Android system in a left-right screen splitting mode using an OpenGL function;
sketching contents in the frame buffer of the Android system onto the grid of left and right screens of the virtual reality device to form a virtual screen in the virtual reality scene; and
after that the virtual screen is formed by sketching the contents onto the grid, sketching the obtained texture of the one or more 2D application interfaces onto the virtual screen in the virtual reality scene of the left and right screens.
2. The method of claim 1 , wherein obtaining the texture of the one or more 2D application interfaces to be displayed comprises:
applying for corresponding layers for the one or more 2D application interfaces to be displayed calling a SurfaceFlinger module in the Android system that is responsible for display and synthesis, labelling the synthesis mode as a GLES synthesis mode in the setUp HWComposer( ) function of the SurfaceFlinger module, and
synthesizing the layers using the GLES synthesis mode.
3. The method of claim 2 , wherein synthesizing the layers using the GLES synthesis mode comprises:
determining display relations of the layers;
creating a texture object bound to GL_TEXTURE_2D and a frame buffer object bound to GL_FRAMEBUFFER using the OpenGL function;
correlating the frame buffer object with the texture object; and
according to the display relations of the layers, sketching the 2D application interfaces to be displayed in each of the layers into the texture object.
4. The method of claim 3 , wherein sketching the obtained texture of the one or more 2D application interfaces to be displayed onto the virtual screen in the virtual reality scene of the left and right screens comprises:
obtaining the texture of the one or more 2D application interfaces to be displayed from the frame buffer object correlated to the texture object; and
sketching the texture onto the virtual screen in the virtual reality scene of the left and right screens respectively using the OpenGL function.
5. The method of claim 4 , wherein determining the virtual reality scene to be displayed comprises:
obtaining data of a user's head state by a sensor of the virtual reality device; and
determining the virtual reality scene to be displayed based on the data of the user's head state.
6. A virtual reality device comprising a processor configured to:
obtain a texture of one or more 2D application interfaces to be displayed;
determine a virtual reality scene to be displayed;
establishing a grid for left and right screens of the virtual reality device;
after establishing the grid, write the virtual reality scene into a frame buffer of an Android system in a left-right screen splitting mode using an OpenGL function;
sketch contents in the frame buffer of the Android system onto the grid of the left and right screens of the virtual reality device to form a virtual screen in the virtual reality scene; and
after that the virtual screen is formed by sketching the contents onto the grid, sketch the obtained texture of the one or more 2D application interfaces to be displayed onto the virtual screen in the virtual reality scene of the left and right screens.
7. The virtual reality device of claim 6 , wherein the processor is further configured to:
apply for corresponding layers for the one or more 2D application interfaces to be displayed;
call a SurfaceFlinger module in the Android system that is responsible for display and synthesis, to label the synthesis mode as a GLES synthesis mode in the setUp HWComposer ( ) function of the SurfaceFlinger module; and
synthesize the layers using the GLES synthesis mode.
8. The virtual reality device of claim 7 , wherein the processor is further configured to:
determine display relations of the layers to create a texture object bound to GL_TEXTURE_2D and a frame buffer object bound to_GL_FRAMEBUFFER using the OpenGL function, correlate the frame buffer object with the texture object; and
sketch the 2D application interfaces to be displayed in each of the layers into the texture object according to the display relations of the layers.
9. The virtual reality device of claim 8 , wherein the processor is further configured to:
obtain the texture of the one or more 2D application interfaces to be displayed from the frame buffer object correlated to the texture object; and
sketch the texture onto the virtual screen in the virtual reality scene of the left and right screens respectively using the OpenGL function.
10. The virtual reality device of claim 9 , wherein the processor is further configured to:
obtain data of a user's head state by a sensor of the virtual reality device; and
determine the virtual reality scene to be displayed based on the data of the user's head state.
US15/115,092
2015-12-31
2016-02-19
Method and apparatus for displaying 2D application interface in virtual reality device
Active
2037-12-25
US10902663B2
( en )
Applications Claiming Priority (4)
Application Number
Priority Date
Filing Date
Title
CN201511026678.8A
CN105447898B
( en )
2015-12-31
2015-12-31
The method and apparatus of 2D application interface are shown in a kind of virtual reality device
CN201511026678
2015-12-31
CN201511026678.8
2015-12-31
PCT/CN2016/074160
WO2017113488A1
( en )
2015-12-31
2016-02-19
Method and apparatus for displaying 2d application interface in virtual reality device
Publications (2)
Publication Number
Publication Date
US20180018806A1
US20180018806A1 ( en )
2018-01-18
US10902663B2
true
US10902663B2 ( en )
2021-01-26
Family
ID=55558029
Family Applications (1)
Application Number
Title
Priority Date
Filing Date
US15/115,092
Active
2037-12-25
US10902663B2
( en )
2015-12-31
2016-02-19
Method and apparatus for displaying 2D application interface in virtual reality device
Country Status (3)
Country
Link
US
( 1 )
US10902663B2
( en )
CN
( 1 )
CN105447898B
( en )
WO
( 1 )
WO2017113488A1
( en )
Families Citing this family (41)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
CN105657408B
( en )
*
2015-12-31
2018-11-30
å京å°é¸ççç§ææéå ¬å¸
The implementation method and virtual reality device of virtual reality scenario
US20170262049A1
( en )
*
2016-03-11
2017-09-14
Empire Technology Development Llc
Virtual reality display based on orientation offset
CN105913477B
( en )
*
2016-04-18
2019-02-12
å京å°é¸ççç§ææéå ¬å¸
A kind of method and apparatus for realizing virtual reality system solid starting animation
CN105957130B
( en )
*
2016-04-18
2019-08-02
å京å°é¸ççç§ææéå ¬å¸
A kind of method and apparatus for realizing virtual reality system plane starting animation
US10043302B2
( en )
2016-04-18
2018-08-07
Beijing Pico Technology Co., Ltd.
Method and apparatus for realizing boot animation of virtual reality system
CN106098022B
( en )
*
2016-06-07
2019-02-12
å京å°é¸ççç§ææéå ¬å¸
A kind of method and apparatus shortening picture delay
CN106126021A
( en )
*
2016-06-21
2016-11-16
ä¸æµ·ä¹ç¸ç§ææéå ¬å¸
A kind of interface display method and device
CN106201724A
( en )
*
2016-07-14
2016-12-07
ä¹è§æ§è¡ï¼å京ï¼æéå ¬å¸
A kind of Virtual Reality equipment and the processing method of user interface UI event thereof
CN107817894A
( en )
*
2016-09-12
2018-03-20
ä¸å ´é讯è¡ä»½æéå ¬å¸
Display processing method and device
CN106507093A
( en )
*
2016-09-26
2017-03-15
å京å°é¸ççç§ææéå ¬å¸
A kind of display mode switching method of virtual reality device and device
CN106570927A
( en )
*
2016-10-14
2017-04-19
æ å·Tcl移å¨éä¿¡æéå ¬å¸
Method of realizing virtual reality based on Android system, terminal and system
CN106502396B
( en )
*
2016-10-20
2020-10-23
ç½æï¼æå·ï¼ç½ç»æéå ¬å¸
Virtual reality system, interaction method and device based on virtual reality
CN108604385A
( en )
*
2016-11-08
2018-09-28
å为ææ¯æéå ¬å¸
A kind of application interface display methods and device
CN106598514B
( en )
*
2016-12-01
2020-06-09
æ å·Tcl移å¨éä¿¡æéå ¬å¸
Method and system for switching virtual reality mode in terminal equipment
CN108228121B
( en )
*
2016-12-15
2021-05-07
ä¸ç§å达软件è¡ä»½æéå ¬å¸
Browser split screen method and device and mobile terminal
CN106792093B
( en )
*
2016-12-20
2023-09-19
é£çä¿¡æ¯ææ¯ï¼å¤©æ´¥ï¼æéå ¬å¸
Video split-screen playing method and device and playing terminal
CN106851240A
( en )
*
2016-12-26
2017-06-13
ç½æï¼æå·ï¼ç½ç»æéå ¬å¸
The method and device of image real time transfer
US11294533B2
( en )
2017-01-26
2022-04-05
Huawei Technologies Co., Ltd.
Method and terminal for displaying 2D application in VR device
CN106990838B
( en )
*
2017-03-16
2020-11-13
æ å·Tcl移å¨éä¿¡æéå ¬å¸
Method and system for locking display content in virtual reality mode
CN107132914A
( en )
*
2017-04-10
2017-09-05
éå²æµ·ä¿¡ç§»å¨éä¿¡ææ¯è¡ä»½æéå ¬å¸
Method and terminal for the VR terminal split-screen displays shown
CN107277483B
( en )
*
2017-05-11
2019-05-14
æ·±å³å¸å æçµåè¡ä»½æéå ¬å¸
A kind of virtual reality display methods, device and virtual reality glasses
CN109743892B
( en )
*
2017-07-04
2020-10-13
è ¾è®¯ç§æï¼æ·±å³ï¼æéå ¬å¸
Display method and device for virtual reality content
KR101990373B1
( en )
*
2017-09-29
2019-06-20
í´ë¦í¸ 주ìíì¬
Method and program for providing virtual reality image
US11861136B1
( en )
*
2017-09-29
2024-01-02
Apple Inc.
Systems, methods, and graphical user interfaces for interacting with virtual reality environments
CN107846584B
( en )
*
2017-11-02
2019-05-07
ä¸å½çµåç§æéå¢å ¬å¸ç¬¬äºåå «ç 究æ
Virtual reality adaptive desktop synchronous projection method based on scene management development library
CN108090944A
( en )
*
2017-12-29
2018-05-29
æ·±å³å¤åæ°ææ¯æéè´£ä»»å ¬å¸
A kind of adaptive VR display methods and device
US11694392B2
( en )
*
2018-05-22
2023-07-04
Apple Inc.
Environment synthesis for lighting an object
CN108921050B
( en )
*
2018-06-14
2021-10-15
åä¸ç§æ大å¦
A virtual reality image processing system based on mobile terminal
CN109308742A
( en )
*
2018-08-09
2019-02-05
éåºç±å¥èºæºè½ç§ææéå ¬å¸
A kind of method and apparatus running 2D application in the 3D scene of virtual reality
CN110597577A
( en )
*
2019-05-31
2019-12-20
ç æµ·å ¨å¿ç§æè¡ä»½æéå ¬å¸
Head-mounted visual equipment and split-screen display method and device thereof
US11727272B2
( en )
2019-06-19
2023-08-15
Nvidia Corporation
LIDAR-based detection of traffic signs for navigation of autonomous vehicles
US11532183B1
( en )
2019-06-19
2022-12-20
Checkpoint ID, Inc.
Identification verification system and related methods
CN111459266A
( en )
*
2020-03-02
2020-07-28
éåºç±å¥èºæºè½ç§ææéå ¬å¸
A method and apparatus for operating a 2D application in a 3D scene of virtual reality
CN112230775A
( en )
*
2020-10-21
2021-01-15
æ½åæå°çµåæéå ¬å¸
Popup window processing method and device and computer readable storage medium
CN112200901A
( en )
*
2020-10-30
2021-01-08
å京ç±å¥èºæºè½ç§ææéå ¬å¸
Three-dimensional display method and device of target application and virtual reality equipment
WO2022155150A1
( en )
*
2021-01-13
2022-07-21
Arris Enterprises Llc
Rendering scrolling captions
CN113342220B
( en )
*
2021-05-11
2023-09-12
æå·çµä¼´ç§ææéå ¬å¸
Window rendering method, head-mounted display kit, and computer-readable medium
CN113589927B
( en )
*
2021-07-23
2023-07-28
æå·çµä¼´ç§ææéå ¬å¸
Split screen display method, head-mounted display device and computer readable medium
CN114185508B
( en )
*
2022-02-15
2022-04-26
éºéºè½¯ä»¶æéå ¬å¸
Method for left-right screen splitting on Linux compatible android system
CN115390668B
( en )
*
2022-08-11
2026-04-03
èæ¥æ±½è½¦ç§æ(å®å¾½)æéå ¬å¸
Image display methods, image display devices, media, and computer equipment
CN120653098A
( en )
*
2024-03-13
2025-09-16
京ä¸æ¹ç§æéå¢è¡ä»½æéå ¬å¸
Interactive control method, interactive display method, virtual reality device and storage medium
Citations (8)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
CN101266546A
( en )
2008-05-12
2008-09-17
æ·±å³å为éä¿¡ææ¯æéå ¬å¸
Method for accomplishing operating system three-dimensional display and three-dimensional operating system
US20120038745A1
( en )
*
2010-08-10
2012-02-16
Yang Yu
2D to 3D User Interface Content Data Conversion
WO2012031406A1
( en )
2010-09-10
2012-03-15
éå²æµ·ä¿¡ä¿¡è¯ç§ææéå ¬å¸
Display method and equipment for 3d tv interface
CN102937968A
( en )
2012-10-11
2013-02-20
ä¸æµ·äº¤é大å¦
Double-eye 3D (three-dimensional) realizing method and system based on Canvas
GB2494434A
( en )
*
2011-09-08
2013-03-13
Sharp Kk
Conversion of Graphics for Multi-View 3D Displays
CN105192982A
( en )
2015-09-07
2015-12-30
å京å°é¸ççç§ææéå ¬å¸
Image correcting method and system for adjustable virtual reality headset
US20160188279A1
( en )
*
2014-12-27
2016-06-30
Intel Corporation
Mode-switch protocol and mechanism for hybrid wireless display system with screencasting and native graphics throwing
US20170053443A1
( en )
*
2015-08-21
2017-02-23
Verizon Patent And Licensing Inc.
Gesture-based reorientation and navigation of a virtual reality (vr) interface
2015
2015-12-31
CN
CN201511026678.8A
patent/CN105447898B/en
active
Active
2016
2016-02-19
US
US15/115,092
patent/US10902663B2/en
active
Active
2016-02-19
WO
PCT/CN2016/074160
patent/WO2017113488A1/en
not_active
Ceased
Patent Citations (10)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
CN101266546A
( en )
2008-05-12
2008-09-17
æ·±å³å为éä¿¡ææ¯æéå ¬å¸
Method for accomplishing operating system three-dimensional display and three-dimensional operating system
US20120038745A1
( en )
*
2010-08-10
2012-02-16
Yang Yu
2D to 3D User Interface Content Data Conversion
CN103081002A
( en )
2010-08-10
2013-05-01
ç´¢å°¼å ¬å¸
2D to 3D user interface content data conversion
WO2012031406A1
( en )
2010-09-10
2012-03-15
éå²æµ·ä¿¡ä¿¡è¯ç§ææéå ¬å¸
Display method and equipment for 3d tv interface
GB2494434A
( en )
*
2011-09-08
2013-03-13
Sharp Kk
Conversion of Graphics for Multi-View 3D Displays
CN102937968A
( en )
2012-10-11
2013-02-20
ä¸æµ·äº¤é大å¦
Double-eye 3D (three-dimensional) realizing method and system based on Canvas
US20160188279A1
( en )
*
2014-12-27
2016-06-30
Intel Corporation
Mode-switch protocol and mechanism for hybrid wireless display system with screencasting and native graphics throwing
US20170053443A1
( en )
*
2015-08-21
2017-02-23
Verizon Patent And Licensing Inc.
Gesture-based reorientation and navigation of a virtual reality (vr) interface
CN105192982A
( en )
2015-09-07
2015-12-30
å京å°é¸ççç§ææéå ¬å¸
Image correcting method and system for adjustable virtual reality headset
US20170070729A1
( en )
2015-09-07
2017-03-09
Beijing Pico Technology Co., Ltd.
Image correction method and system for adjustable virtual reality helmet
Non-Patent Citations (5)
* Cited by examiner, â Cited by third party
Title
" CrossApp " http://www.arvrschool.com/read.php?tid=485&fid=84 (Oct. 16, 2015.
" Graphic Architecture " http ://source .android.com/devices/graphics/architecture.html (Nov. 9, 2015).
OpenGL-Frame buffers (https://open.gl/framebuffers) (as appearing on May 17, 2015, and available at https://web.archive.org/web/20150517074715/https://open.gl/framebuffers).
*
OpenGLâFrame buffers (https://open.gl/framebuffers) (as appearing on May 17, 2015, and available at https://web.archive.org/web/20150517074715/https://open.gl/framebuffers).
*
State Intellectual Property Office of the P.R. China, International Search Report in International Application No. PCT/CN2016/074160 (dated Sep. 28, 2016).
Also Published As
Publication number
Publication date
CN105447898A
( en )
2016-03-30
CN105447898B
( en )
2018-12-25
US20180018806A1
( en )
2018-01-18
WO2017113488A1
( en )
2017-07-06
Similar Documents
Publication
Publication Date
Title
US20180018806A1
( en )
2018-01-18
Method and Apparatus for Displaying 2D Application Interface in Virtual Reality Device
US20230039100A1
( en )
2023-02-09
Multi-layer reprojection techniques for augmented reality
CN113436343B
( en )
2024-06-04
Picture generation method and device for virtual concert hall, medium and electronic equipment
US10935788B2
( en )
2021-03-02
Hybrid virtual 3D rendering approach to stereovision
CN109478344A
( en )
2019-03-15
Method and apparatus for synthesizing image
CN105027563A
( en )
2015-11-04
Low-latency image display on multi-monitor devices
CN107852573A
( en )
2018-03-27
The social interaction of mixed reality
US12432329B2
( en )
2025-09-30
Methods, systems, and media for generating and rendering immersive video content
US20150339038A1
( en )
2015-11-26
System and method for capturing occluded graphical user interfaces
CN106569700B
( en )
2020-09-25
Screenshot method and screenshot device
CN105959666A
( en )
2016-09-21
Method and device for sharing 3d image in virtual reality system
US8938093B2
( en )
2015-01-20
Addition of immersive interaction capabilities to otherwise unmodified 3D graphics applications
US20170186243A1
( en )
2017-06-29
Video Image Processing Method and Electronic Device Based on the Virtual Reality
CN105528207A
( en )
2016-04-27
Virtual reality system, and method and apparatus for displaying Android application images therein
WO2021135320A1
( en )
2021-07-08
Video generation method and apparatus, and computer system
CN109308742A
( en )
2019-02-05
A kind of method and apparatus running 2D application in the 3D scene of virtual reality
CN105844581A
( en )
2016-08-10
Image drawing method, apparatus and equipment
WO2017113729A1
( en )
2017-07-06
360-degree image loading method and loading module, and mobile terminal
US10025099B2
( en )
2018-07-17
Adjusted location hologram display
US11962743B2
( en )
2024-04-16
3D display system and 3D display method
CN106375750B
( en )
2018-08-24
A kind of image display method and display device
CN117173309A
( en )
2023-12-05
Image rendering methods, devices, equipment, media and program products
CN113947671A
( en )
2022-01-18
Panoramic 360-degree image segmentation and synthesis method, system and medium
US20260010384A1
( en )
2026-01-08
Application-based screen recording
CN119484794B
( en )
2025-11-18
A VR display method and VR device for 2D applications
Legal Events
Date
Code
Title
Description
2016-07-28
AS
Assignment
Owner name : BEIJING PICO TECHNOLOGY CO., LTD., CHINA
Free format text : ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:LI, LIGANG;REEL/FRAME:039283/0921
Effective date : 20160706
2017-10-03
FEPP
Fee payment procedure
Free format text : ENTITY STATUS SET TO SMALL (ORIGINAL EVENT CODE: SMAL); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY
2018-12-28
STPP
Information on status: patent application and granting procedure in general
Free format text : NON FINAL ACTION MAILED
2019-04-10
STPP
Information on status: patent application and granting procedure in general
Free format text : RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER
2019-05-01
STPP
Information on status: patent application and granting procedure in general
Free format text : NON FINAL ACTION MAILED
2019-08-08
STPP
Information on status: patent application and granting procedure in general
Free format text : RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER
2019-08-14
STPP
Information on status: patent application and granting procedure in general
Free format text : FINAL REJECTION MAILED
2019-10-23
STPP
Information on status: patent application and granting procedure in general
Free format text : RESPONSE AFTER FINAL ACTION FORWARDED TO EXAMINER
2019-11-14
STPP
Information on status: patent application and granting procedure in general
Free format text : ADVISORY ACTION MAILED
2019-12-21
STPP
Information on status: patent application and granting procedure in general
Free format text : DOCKETED NEW CASE - READY FOR EXAMINATION
2020-04-13
STPP
Information on status: patent application and granting procedure in general
Free format text : NON FINAL ACTION MAILED
2020-12-16
STPP
Information on status: patent application and granting procedure in general
Free format text : PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED
2021-01-06
STCF
Information on status: patent grant
Free format text : PATENTED CASE
2024-05-20
MAFP
Maintenance fee payment
Free format text : PAYMENT OF MAINTENANCE FEE, 4TH YR, SMALL ENTITY (ORIGINAL EVENT CODE: M2551); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY
Year of fee payment : 4
2024-07-22
FEPP
Fee payment procedure
Free format text : ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY