Microsoft Technology Licensing, LLCDownload PDFPatent Trials and Appeals BoardDec 20, 20212021000191 (P.T.A.B. Dec. 20, 2021) Copy Citation UNITED STATES PATENT AND TRADEMARK OFFICE UNITED STATES DEPARTMENT OF COMMERCE United States Patent and Trademark Office Address: COMMISSIONER FOR PATENTS P.O. Box 1450 Alexandria, Virginia 22313-1450 www.uspto.gov APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO. 14/723,065 05/27/2015 Tom Salter 355835-US-NP 9090 69316 7590 12/20/2021 MICROSOFT CORPORATION ONE MICROSOFT WAY REDMOND, WA 98052 EXAMINER LIU, GORDON G ART UNIT PAPER NUMBER 2612 NOTIFICATION DATE DELIVERY MODE 12/20/2021 ELECTRONIC Please find below and/or attached an Office communication concerning this application or proceeding. The time period for reply, if any, is set in the attached communication. Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the following e-mail address(es): chriochs@microsoft.com ljohnson@microsoft.com usdocket@microsoft.com PTOL-90A (Rev. 04/07) UNITED STATES PATENT AND TRADEMARK OFFICE ____________ BEFORE THE PATENT TRIAL AND APPEAL BOARD ____________ Ex parte TOM SALTER, BEN SUGDEN, LAURA MASSEY, and BRIAN KEANE ____________ Appeal 2021-000191 Application 14/723,065 Technology Center 2600 ____________ Before JOSEPH L. DIXON, NORMAN H. BEAMER, and MICHAEL T. CYGAN, Administrative Patent Judges. BEAMER, Administrative Patent Judge. DECISION ON APPEAL Appellant1 appeals under 35 U.S.C. § 134(a) from the Examiner’s Non-Final Rejection of claims 1–8 and 10–19. Claims 9 and 20 are cancelled. We have jurisdiction over the pending rejected claims under 35 U.S.C. § 6(b). We affirm. 1 We use the word “Appellant” to refer to “applicant” as defined in 37 C.F.R. § 1.42 (2019). Appellant identifies Microsoft Technology Licensing, LLC as the real party in interest. (Appeal Br. 4.) Appeal 2021-000191 Application 14/723,065 2 THE INVENTION Appellant’s disclosed and claimed invention is directed to a head mounted display (HMD) device operating in a real world physical environment with a visibly rendered projected gaze ray showing what a user is looking at in the environment. (Abstr.) Independent claim 1, reproduced below, is illustrative of the subject matter on appeal: 1. One or more computer readable memories storing computer-executable instructions which, when executed by one or more processors in a local head mounted display (HMD) device located in a physical environment, perform: using data from a sensor package incorporated into the HMD device to dynamically perform head tracking of a user within the physical environment; responsively to the head tracking, determining a field of view of a mixed reality or virtual reality environment that is renderable by the local HMD device, the field of view being variable depending at least in part on a pose of the user’s head in the physical environment; receiving data that is transmitted from a remote HMD device utilized by a remote user, the data including coordinates of an origin and an intercept of a virtual gaze ray that is visibly rendered and is projected from the origin at a view position of the remote HMD device and terminates at the intercept at a point of intersection between the virtual gaze ray and the mixed reality or virtual reality environment, wherein the visibly rendered virtual gaze ray with the coordinates of its origin and intercept enable the user to see where the remote user is looking; and visibly rendering the virtual gaze ray and an avatar representing the remote user on the local HMD device using the received coordinates from the data within the field of view, wherein the rendered virtual gaze ray projects from a location Appeal 2021-000191 Application 14/723,065 3 on the rendered avatar corresponding to the view position of the remote HMD device and extends to the intercept. (Appeal Br. 17 (Claims App.).) REJECTIONS The Examiner rejected claims 1, 3–7, 10–12, and 14–19 under 35 U.S.C. § 103 as being unpatentable over Mount, et al. (US 20130141419 A1, pub. June 6, 2013), Maciocci, et al. (US 20120249741 A1, pub. Oct. 4, 2012), Abe, et al. (US 20150356788 A1, pub. Dec. 10, 2015), Tanguay, et al. (US 20060092178 A1, pub. May 4, 2006), and Liu, et al. (US 20130050258 A1, pub. Feb. 28, 2013). (Non-Final Act. 5–35.) The Examiner rejected claim 2 under 35 U.S.C. § 103 as being unpatentable over Mount, Maciocci, Abe, Tanguay, Liu, and Wilairat, et al. (US 20150261293 A1, pub. Sep. 17, 2015). (Non-Final Act. 35–36.) The Examiner rejected claim 8 under 35 U.S.C. § 103 as being unpatentable over Mount, Maciocci, Abe, Tanguay, Liu, and Gemmell et al., “Gaze Awareness for Videoconferencing: A Software Approach” (2000). (Non-Final Act. 36–38.) The Examiner rejected claim 13 under 35 U.S.C. § 103 as being unpatentable over Mount, Maciocci, Abe, Tanguay, Liu, and Raghoebardayal, et al. (US 20140132715 A1, pub. May 15, 2014). (Non- Final Act. 38–39.) ISSUE ON APPEAL Appellant’s arguments present the following issue:2 2 Rather than reiterate the arguments of Appellant and the positions of the Examiner, we refer to the Appeal Brief (filed Oct. 2, 2018) (Appeal Br.); the Appeal 2021-000191 Application 14/723,065 4 Whether the Examiner erred in finding the combination of Mount, Maciocci, Abe, Tanguay, and Liu would have taught or suggested the limitations of independent claims 1, 12, and 16. (Appeal Br. 6–13.) ANALYSIS We have reviewed the Examiner’s rejection in light of Appellant’s arguments. Arguments Appellant could have made, but chose not to make are deemed to be waived. See 37 C.F.R. § 41.37(c)(1)(iv). We disagree with Appellant’s arguments, and we adopt as our own: (1) the pertinent findings and reasons set forth by the Examiner in the Action from which this appeal is taken (Final Act. 5–39); and (2) the corresponding reasons set forth by the Examiner in the Examiner’s Answer in response to Appellant’s Appeal Brief. (Ans. 3–7.) We concur with the applicable conclusions reached by the Examiner and emphasize the following. In rejecting the independent claims, the Examiner relies on the disclosure in Mount of head-mounted displays that visually augment an observed physical space by superimposing virtual objects on the view, where different users can view the same space from different perspectives, including viewing the same virtual objects but each from their own perspective. (Final Act. 5–6; Mount Figs.1A–C, ¶¶ 16, 25.) The head- mounted displays have the capability of tracking the user’s head movements to determine the field of view in the virtual reality environment. (Final Act. 6; Mount ¶¶ 25, 67.) Non-Final Office Action (mailed May 2, 2018) (Non-Final Act.); and the Examiner’s Answer (mailed Dec. 11, 2018) (Ans.) for the respective details. Appeal 2021-000191 Application 14/723,065 5 The Examiner further relies on the disclosure in Maciocci of visibly rendering a gaze ray (which Maciocci refers to as a “virtual input device”) on a head-mounted display using received coordinates based on a vector pointing from the display to a portion of a virtual object being pointed to by the user. (Final Act. 7–8; Maciocci Fig. 37, ¶¶ 69, 302.) Although Maciocci discloses multiple users with multiple head-mounted displays viewing the same environment, it does not explicitly disclose that one user can view the virtual input device generated by a different user. (Final Act. 8; Maciocci Fig. 2, ¶ 87.) The Examiner finds that this aspect of the claims is obvious in view of Maciocci in combination with Abe, given the disclosure in Abe of multiple users sharing virtual objects viewed from head-mounted displays, including avatars of the users. (Final Act. 8–10; Maciocci ¶ 3; Abe Figs. 2, 15, ¶¶ 104, 177.) In addition, the Examiner relies on the disclosure of Tanguay of a “virtual laser pointer,” in which the projected light of light of a laser pointer on a surface is tracked and reproduced in the virtual reality environment viewed by users via head-mounted displays, which the Examiner finds teaches the claim requirement of transmitting coordinate data in order to see where a remote user is looking. (Final Act. 10–12; Tanguay ¶¶ 33, 101– 104.) For this aspect of the claims the Examiner also relies on the disclosure of Liu of a head-mounted display that tracks a user’s gaze direction and highlights an object in the view. (Final Act. 13–14; Liu Figs. 15A, 16B, ¶¶ 136–137, 164.) Appellant argues that Tanguay does not disclose transmitting the coordinates of a virtual gaze ray, because it is limited to displaying the mark that the laser light makes on a two-dimensional surface. (Appeal Br. 7–8.) Appeal 2021-000191 Application 14/723,065 6 Appellant also argues that Liu is limited to a single user, and therefore cannot satisfy the claim requirement of showing where a remote user is looking. (Appeal Br. 8–9.) In addition, Appellant argues that Abe only shares virtual objects, but does not share virtual gaze rays. (Appeal Br. 9– 10.) In regard to the virtual input device of Maciocci (the asserted visual gaze ray), Appellant argues that it is solely disclosed as limited to the view of the single user that is doing the pointing. (Appeal Br. 11–12.) Also for independent claims 12 and 16, Appellant argues that Maciocci cannot control the visual appearance of the gaze ray, and that the shared virtual objects of Maciocci are not equivalent to gaze rays. (Appeal Br. 13–15.) In the Answer, the Examiner reiterates the finding that the combination of cited references teaches or suggests the requirements of the independent claims: [T]he combination of the prior arts on record, Maciocci, etc. (US 20120249741 Al), ABE, etc. (US 20150356788 Al), Tanguay, etc. (US 20060092178 Al), and Liu, etc. (US 20130050258 Al) teach these claimed limitations in such a way that Maciocci teaches that the virtual gaze ray may be rendered visible on the local HMD (See Maciocci: Fig. 37, and [0302]), and ABE teaches that the local HMD and remote HMD may share virtual/real objects (See ABE: Figs.12-15, and [0177]), while Tanguay teaches how to transmit and receive data between local and remote HMDs (See Tanguay: [0033] and [0104]), and Liu teaches the details of the gaze ray related data like interception coordinates (See Liu: Fig. 15, and [0136]), etc. Thus, the remote gaze ray of Maciocci may be a virtual object shared with (received by) the local HMD and rendered visible in the local HMD, and this combination may be corresponding to the limitations mentioned above. (Ans. 4.) The Examiner also responds that Abe teaches that two head- mounted displays may share assisted reality space, including, by implication, Appeal 2021-000191 Application 14/723,065 7 transmitting data between displays, and that in addition Tanguay explicitly teaches such data transmission. (Ans. 4–5.) The Examiner further emphasizes that Maciocci and Abe teach the rendering and sharing of a virtual gaze ray. (Ans. 5.) Appellant does not reply to the Examiner’s Answer, and we are not persuaded that the Examiner’s findings are incorrect. Appellant’s arguments are unpersuasive as focusing on references individually, whereas the Examiner relies on the combination of references. Non-obviousness cannot be established by attacking references individually where, as here, the ground of unpatentability is based upon the teachings of a combination of references. In re Merck & Co., Inc., 800 F.2d 1091, 1097 (Fed. Cir. 1986). Rather, the test for obviousness is whether the combination of references, taken as a whole, would have suggested the patentee’s invention to a person having ordinary skill in the art. In re Keller, 642 F.2d 413, 425 (CCPA 1981). Therefore, we sustain the rejections of independent claims 1, 12, and 16 over Mount, Maciocci, Abe, Tanguay, and Liu. We also sustain the rejections of claims 2–8, 10, 11, 13–15, and 17–19, which are not argued separately with particularity. (Appeal Br. 13, 15.) Appeal 2021-000191 Application 14/723,065 8 DECISION SUMMARY In summary: Claim(s) Rejected 35 U.S.C. § Reference(s)/Basis Affirmed Reversed 1, 3–7, 10– 12, 14–19 103 Mount, Maciocci, Abe, Tanguay, Liu 1, 3–7, 10– 12, 14–19 2 103 Mount, Maciocci, Abe, Tanguay, Liu, Wilairat 2 8 103 Mount, Maciocci, Abe, Tanguay, Liu, Gemmell 8 13 103 Mount, Maciocci, Abe, Tanguay, Liu, Raghoebardayal 13 Overall Outcome 1–8, 10–19 TIME PERIOD FOR RESPONSE No time period for taking any subsequent action in connection with this appeal may be extended under 37 C.F.R. § 1.136(a)(1)(iv). AFFIRMED Copy with citationCopy as parenthetical citation