Hewlett-Packard Development Company , LP

Patent Trials and Appeals BoardAug 30, 2021

2020004275 (P.T.A.B. Aug. 30, 2021)

UNITED STATES PATENT AND TRADEMARK OFFICE UNITED STATES DEPARTMENT OF COMMERCE United States Patent and Trademark Office Address: COMMISSIONER FOR PATENTS P.O. Box 1450 Alexandria, Virginia 22313-1450 www.uspto.gov APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO. 15/500,557 01/31/2017 Jinman Kang 84592932 1017 22879 7590 08/30/2021 HP Inc. 3390 E. Harmony Road Mail Stop 35 Fort Collins, CO 80528-9544 EXAMINER CHANG, DANIEL ART UNIT PAPER NUMBER 2487 NOTIFICATION DATE DELIVERY MODE 08/30/2021 ELECTRONIC Please find below and/or attached an Office communication concerning this application or proceeding. The time period for reply, if any, is set in the attached communication. Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the following e-mail address(es): ipa.mail@hp.com jessica.pazdan@hp.com yvonne.bailey@hp.com PTOL-90A (Rev. 04/07) UNITED STATES PATENT AND TRADEMARK OFFICE ____________________ BEFORE THE PATENT TRIAL AND APPEAL BOARD ____________________ Ex parte JINMAN KANG, BEN WYNNE, DAVID BRADLEY SHORT, and CHRISTOPHER S. TANNER ____________________ Appeal 2020 -004275 Application 15/500,557 Technology Center 2400 ____________________ Before MARC S. HOFF, JAMES R. HUGHES, and JOYCE CRAIG, Administrative Patent Judges. HOFF, Administrative Patent Judge. DECISION ON APPEAL STATEMENT OF THE CASE Appellant1 appeals under 35 U.S.C. § 134 from a Final Rejection of claims 1−8, 11−14, and 16−22.2 We have jurisdiction under 35 U.S.C. § 6(b). We reverse. Appellant’s invention is a system and method for reconstructing the surface geometry of a three-dimensional object. A turntable is rotated to first 1 We use the word “Appellant” to refer to “applicant” as defined in 37 C.F.R. § 1.42. Appellant identifies the real party in interest as Hewlett-Packard Development Company, L.P. App. Br. 1. 2 Claims 9 and 10 have been cancelled. Clam 15 stands allowed. Appeal 2020-004275 Application 15/500,557 2 and second positions and two-dimensional (2D) and three-dimensional (3D) data sets are obtained using a 2D high-resolution camera and a 3D depth camera. Corresponding features from the 2D data sets are identified and used to identify the same features in the 3D data sets. The 3D corresponding features are used to calculate a 3D homography, which is used to align the 3D data sets. A 3D mesh is generated from the aligned data sets. Abstract. Claim 1 is reproduced below: 1. A system for reconstructing a three-dimensional (3D) object, comprising: a cluster of heterogeneous sensors, including a two- dimensional (2D) high-resolution sensor and a 3D depth sensor; a turntable; and a calibration system including a processor and a storage medium storing calibration instructions executable on the processor to: when the turntable is at a first position, receive a first 2D data set of an object on the turntable acquired using the 2D high- resolution sensor, and a first 3D data set of the object on the turntable acquired using the 3D depth sensor; when the turntable is at a second position different from the first position, receive a second 2D data set of the object on the turntable acquired using the 2D high-resolution sensor, and a second 3D data set of the object on the turntable acquired using the 3D depth sensor; determine corresponding features between the first 2D data set and the second 2D data set to identify a set of high-resolution corresponding coordinate points; map the set of high-resolution corresponding coordinate points onto an image plane of the 3D depth sensor to identify a set of coordinate points; align the first and second 3D data sets using the set of coordinate points; and generate a representation of the 3D object using the aligned first and second 3D data sets. Appeal 2020-004275 Application 15/500,557 3 The prior art relied upon by the Examiner as evidence is: Name Reference Date Wolberg et al. US 2008/0310757 A1 Dec. 18, 2008 Khan et al. US 2009/0301265 A1 Dec. 10, 2009 Nguyen et al. US 2011/0115886 A1 May 19, 2011 Shotton et al. US 2012/0306876 A1 Dec. 6, 2012 Kim et al. US 2015/0257242 A1 Oct. 8, 2015 Chao et al., “Semi-automatic range to range registration: a feature-based method” Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM’05), 225-61 (2005). Claims 1, 4−7, 16, and 203 stand rejected under 35 U.S.C. § 103 as being unpatentable over Kim and Wolberg.4 Claims 2 and 3 stand rejected under 35 U.S.C. § 103 as being unpatentable over Kim, Wolberg, and Nguyen. Claims 8, 18, 19, and 22 stand rejected under 35 U.S.C. § 103 as being unpatentable over Kim, Wolberg, and Khan. Claims 11, 17, and 21 stand rejected under 35 U.S.C. § 103 as being unpatentable over Kim, Wolberg, and Chao. Claims 12 and 13 stand rejected under 35 U.S.C. § 103 as being unpatentable over Kim, Wolberg, and Khan. Claim 14 stands rejected under 35 U.S.C. § 103 as being unpatentable over Kim, Wolberg, Chao, and Shotton. 3 The Examiner’s Statement of Rejection erroneously includes claim 11. Final Act. 6. 4 The Examiner has withdrawn the rejection of claims 11−15 under 35 U.S.C. § 112(a). Ans. 22. Appeal 2020-004275 Application 15/500,557 4 Throughout this decision, we make reference to the Appeal Brief (“App. Br.,” filed Nov. 14, 2019), the Reply Brief (“Reply Br.,” filed May 21, 2020), and the Examiner’s Answer (“Ans.,” mailed Mar. 23, 2020) for their respective details. ISSUE Does the combination of Kim and Wolberg teach or suggest acquiring a first 3D data set using a 3D depth sensor, acquiring a second 3D data set using said 3D depth sensor, and aligning the first and second 3D data sets? ANALYSIS Claims 1, 4−7, 16, and 20 Independent claims 1 and 6 recite, in pertinent part, receiving “a first 3D data set of the object on the turntable acquired using the 3D depth sensor” and “a second 3D data set of the object on the turntable acquired using the 3D depth sensor.” The claims further recite “align[ing] the first and second 3D data sets using the set of coordinate points.” Claim 1 (Claim App. i). The Examiner finds that Kim teaches acquiring the first and second 3D data sets on a turntable using a 3D depth sensor, but does not teach aligning the two 3D data sets. Final Act. 7−8. The Examiner then finds that Wolberg teaches the claimed aligning. Ans. 8. The Examiner finds that Wolberg teaches aligning 3D range model Mrange with a sparse 3D model Msfm. Wolberg ¶ 63. Appellant argues persuasively that the Examiner erred in finding that Wolberg teaches the claimed aligning step. First, we agree with Appellant that the claimed invention requires the acquisition of the first and second 3D data sets with the same 3D depth sensor. Appeal Br. 9. In particular, claim 1 Appeal 2020-004275 Application 15/500,557 5 (as well as claim 6) recites “when the turntable is at a first position, receive . . . a first 3D data set of the object on the turntable acquired using the 3D depth sensor” and “when the turntable is at a second position different from the first position, receive . . . a second 3D data set of the object on the turntable acquired using the 3D depth sensor” (claim 1 (Claim App. i)). Therefore, the claims require that the same 3D depth sensor acquires the two distinct 3D data sets. We also agree with Appellant that, while Wolberg’s 3D range model Mrange is obtained using 3D range scans by laser range scanner 14, Wolberg’s 3D model Msfm is obtained using a sequence of 2D photographs. Id., citing Wolberg ¶ 14, 48. Thus, the 3D data sets aligned by Wolberg not only are not acquired by the same sensor, as the claims require, but one of the data sets (Msfm) derives from a 2D data set rather than a 3D data set. We find that the combination of Kim and Wolberg fails to teach or suggest aligning a first 3D data set and a second 3D data set, in which both 3D data sets were acquired by the same 3D depth sensor. Accordingly, we do not sustain the Examiner’s § 103 rejection of claim 1, 4−7, 16, and 20. Claim 11 Independent claim 11, like independent claims 1 and 6, recites acquiring a first 3D data set of the object on the turntable using the 3D depth camera and acquiring a second 3D data set of the object on the turntable using the 3D depth camera. Claim 11 further recites mapping the first set of high-resolution corresponding points onto an image plane of the 3D depth camera and determining corresponding points between a first set of depth data of the first 3D data set and a second set of depth data of the second 3D Appeal 2020-004275 Application 15/500,557 6 data set, and aligning the first and second sets of depth data using a 3D homography. See claim 11 (Claim App. iii). As discussed supra with respect to claims 1 and 6, we find that Wolberg does not teach aligning a first 3D data set and a second 3D data set acquired by the same sensor or camera, i.e., the recited “3D depth camera” (claim 11, Claim App. iii). We further determine, as a result, that Wolberg does not teach or suggest mapping the high-resolution corresponding points onto an image plane of the 3D depth camera or determining corresponding points between depth data of the first 3D data set and the second 3D data set. The combination of Kim, Wolberg, and Chao therefore fails to teach or suggest all of the elements of the claimed invention. We do not sustain the Examiner’s § 103 rejection of claim 11. Claims 2, 3, 8-10, 12−14, and 16−22 Claims 2 and 3 depend from claim 1. The Examiner does not find that Nguyen supplies any of the teachings we determine to be missing from the combination of Kim and Wolberg. Accordingly, we do not sustain the Examiner’s § 103 rejection of claims 2 and 3 over Kim, Wolberg, and Nguyen. Claim 8 depends from independent claim 6. Claims 18, 19, and 22 depend from independent claim 1. The Examiner does not find that Khan supplies any of the teachings we determine to be missing from the combination of Kim and Wolberg. Accordingly, we do not sustain the Examiner’s § 103 rejection of claims 8, 18, 19, and 22 over Kim, Wolberg, and Nguyen. Claim 17 depends from independent claim 1 and claim 21 depends from independent claim 6. The Examiner does not find that Chao supplies Appeal 2020-004275 Application 15/500,557 7 any of the teachings we determine to be missing from the combination of Kim and Wolberg. Accordingly, we do not sustain the Examiner’s § 103 rejection of claims 17 and 21 over Kim, Wolberg, and Nguyen. Claims 12 and 13 depend from independent claim 11. The Examiner does not find that Khan supplies any of the teachings we determine to be missing from the combination of Kim, Wolberg, and Chao. Accordingly, we do not sustain the Examiner’s § 103 rejection of claims 12 and 13 over Kim, Wolberg, Chao, and Khan. Claim 14 depends from independent claim 11. The Examiner does not find that Shotton supplies any of the teachings we determine to be missing from the combination of Kim, Wolberg, Chao, and Khan. Accordingly, we do not sustain the Examiner’s § 103 rejection of claims 12 and 13 over Kim, Wolberg, Chao, Khan, and Shotton. CONCLUSION The combination of Kim and Wolberg does not teach or suggest acquiring a first 3D data set using a 3D depth sensor, acquiring a second 3D data set using said 3D depth sensor, and aligning the first and second 3D data sets. Appeal 2020-004275 Application 15/500,557 8 DECISION SUMMARY In summary: Claims Rejected 35 U.S.C. § Reference(s)/ Basis Affirmed Reversed 1, 4−7, 16, 20 103 Kim, Wolberg 1, 4−7, 16, 20 2, 3 103 Kim, Wolberg, Nguyen 2, 3 8, 18, 19, 22 103 Kim, Wolberg, Khan 8, 18, 19, 22 11, 17, 21 103 Kim, Wolberg, Chao 11, 17, 21 12, 13 103 Kim, Wolberg, Chao, Khan 12, 13 14 103 Kim, Wolberg, Chao, Khan, Shotton 14 Overall Outcome 1−8, 11−14, 16−22 ORDER The Examiner’s decision to reject claims 1−8, 11−14, and 16−22 is reversed. REVERSED