Alj, Youssef et al.Download PDFPatent Trials and Appeals BoardDec 31, 201914372018 - (D) (P.T.A.B. Dec. 31, 2019) Copy Citation UNITED STATES PATENT AND TRADEMARK OFFICE UNITED STATES DEPARTMENT OF COMMERCE United States Patent and Trademark Office Address: COMMISSIONER FOR PATENTS P.O. Box 1450 Alexandria, Virginia 22313-1450 www.uspto.gov APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO. 14/372,018 07/13/2014 Youssef Alj 2011P00137WOUS 8755 147897 7590 12/31/2019 Tutunjian & Bitetto, P.C. 401 Broadhollow Road Suite 402 Melville, NY 11747 EXAMINER BADER, ROBERT N. ART UNIT PAPER NUMBER 2619 NOTIFICATION DATE DELIVERY MODE 12/31/2019 ELECTRONIC Please find below and/or attached an Office communication concerning this application or proceeding. The time period for reply, if any, is set in the attached communication. Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the following e-mail address(es): docketing@tb-iplaw.com PTOL-90A (Rev. 04/07) UNITED STATES PATENT AND TRADEMARK OFFICE BEFORE THE PATENT TRIAL AND APPEAL BOARD Ex parte YOUSSEF ALJ, GUILLAUM BOISSON, PHILIPPE BORDES, LUCE MORIN, and MURIEL PRESSIGOUT, Appeal 2019-001565 Application 14/372, 018 Technology Center 2600 BEFORE ROBERT E. NAPPI, KRISTEN L. DROESCH, and CATHERINE SHIANG, Administrative Patent Judges. DROESCH, Administrative Patent Judge. DECISION ON APPEAL STATEMENT OF THE CASE Pursuant to 35 U.S.C. § 134(a), Appellant1 appeals from the Examiner’s decision to reject claims 1, 2, 4–9, and 11–15. We have jurisdiction under 35 U.S.C. § 6(b). We AFFIRM. CLAIMED SUBJECT MATTER The claims are directed to a method of texturing a mesh associated with a surface representative of a scene captured in a plurality of images of a 1 We use the word Appellant to refer to “applicant” as defined in 37 C.F.R. § 1.42(a). Appellant identifies the real party in interest as Thomson Licensing DTV. Appeal Br. 4. Appeal 2019-001565 Application 14/372,018 2 multi-view video stream. See Spec. 1:6–11; Abstract. Claim 1, reproduced below, is illustrative of the claimed subject matter: 1. A method for texturing a mesh, the mesh being associated with a surface representative of a scene captured according to a plurality of points of view, an image of the scene being associated with each point of view, the method comprising, for one mesh element of said mesh that is at least partially visible from at least two first images of the plurality of images: a) selecting, by a processor, a first image among said at least two first images; b) projecting, by said processor, said mesh element onto said selected first image and texturing said mesh element with a first texture information obtained from pixels of said selected first image, said pixels corresponding to the projection of said mesh element onto said selected first image; c) for each first image of said at least two first images, comparing, by said processor, the first texture information of said textured mesh element with the texture information associated with the pixels of said each first image that corresponds to the projection of said textured mesh element onto said each first image; d) computing, by said processor, an error information for said first texture information according to a result of said comparing; e) selecting, by said processor, a first image among said at least two first images different from the first image selected at operation a) and reiterating operations b) - e) until each first image has been selected; f) selecting, by said processor, the first texture information minimizing said error information among the plurality of first texture information. Appeal 2019-001565 Application 14/372,018 3 REFERENCES The prior art relied upon by the Examiner is: Name Reference Date Debevec et al. Efficient View-Dependent Image-Based Rendering with Projective Texture-Mapping, Proceedings of the Eurographics Workshop Vienna Austria, pp. 105–116. 1998 Lindstrom et al. Image-Driven Simplification, ACM Transactions on Graphics Vol. 19, No. 3, pp. 204–241 2000 Eckert et al. Shape Refinement for Reconstruction 3D- Objects Using an Analysis-Synthesis Approach, 2001 International Conference ICIP, IEEE Vol. 3, pp. 903–906 2001 Carranza et al. Free-Viewpoint Video of Human Actors, ACM Transactions on Graphics (TOG) – Proceedings of ACM SIGGRAPH 2003, Vol. 22, Issue 3, pp. 569–577. 2003 Bogomjakov et al. Free-Viewpoint Video from Depth Cameras, 2006 Proceedings Vision, Modeling, and Visualization, pp. 89–96. 2006 Iiyama et al. Super-Resolution Texture Mapping from Multiple Viewpoints, IEEE 2010 International Conference on Pattern Recognition, pp. 1820– 23 2010 REJECTIONS Claims 1, 2, 8, 9, and 15 stand rejected under 35 U.S.C. § 103(a) as being unpatentable over Iiyama et al. (“Iiyama”), Eckert et al. (“Eckert”), and Lindstrom et al (“Lindstrom”). Final Act. 2–15. Claims 4, 5, 11, and 12 stand rejected under 35 U.S.C. § 103(a) as being unpatentable over Iiyama, Eckert, Lindstrom, and Debevec et al. (“Debevec”). Final Act. 15–19. Appeal 2019-001565 Application 14/372,018 4 Claims 6 and 13 stand rejected under 35 U.S.C. § 103(a) as being unpatentable over Iiyama, Eckert, Lindstrom, and Carranza e al. (“Carranza”). Final Act. 19–21. Claims 7 and 14 stand rejected under 35 U.S.C. § 103(a) as being unpatentable over Iiyama, Eckert, Lindstrom, and Bogomjakov et al. (“Bogomjakov”). Final Act. 21–23. OPINION Rejections of Claims 1, 2, 8, 9, and 15 Appellant argues claims 1, 2 and 15 together as a group. See Appeal Br. 10–17. Appellant does not present substantive arguments addressing the limitations of claims 8 and 9 apart from arguments addressing the limitations of claim 1. See Appeal Br. 18–19. As a result, we group claims 8 and 9 together with claims 1, 2, and 15. Pursuant to our authority under C.F.R. § 41.37(c)(1)(iv) (2017), we choose claim 1 as representative of the group. Appellant disputes the Examiner’s findings that the combination of Iiyama, Eckert, and Lindstrom teach or suggest the limitations recited in steps c) through f) of claim 1. See Appeal Br. 12. Specifically, Appellant contends that Iiyama defines two cost functions Earea (L) and Etex (L), and argues that cost function Earea (L) does not consider texture information itself but the size of the projection of the patch. See id. at 14 (citing Iiyama p. 1821, right column, last paragraph through p. 1822, left column, first paragraph, equation 3, section III, third paragraph). As to Etex (L), Appellant argues this cost function considers the patch with regard to the neighbor patches, and not the texture information onto which the patch may be projected. See id. (citing Iiyama equation 5). Appellant contends that with Iiyama’s first cost function and second cost function, there is no comparing Appeal 2019-001565 Application 14/372,018 5 of the first texture information of the textured mesh element with the texture information associated with the pixels of each first image that corresponds to the projection of the textured mesh element onto said each first image. See id. Appellant argues that Eckert fails to cure the deficiencies of Iiyama because Eckert teaches a shape refinement process based on the local deformations of a triangular mesh, including moving each triangle of the mesh into seven different trial positions, and texturing the triangles for each trial position by using the texture information of only one reference image, but does not teach or suggest testing of the texture obtained from each image of the plurality. See id. at 14–15. Appellant also argues that Lindstrom is completely silent with respect to testing of the texture obtained from each image of a plurality of images. See id. at 16–17. Appellant’s arguments are not persuasive of error in the Examiner’s rejection because the arguments address the teachings of each of Iiyama, Eckert, and Lindstrom alone. One cannot show non-obviousness by attacking references individually where the rejections are based on combinations of references. In re Merck & Co., 800 F.2d 1091, 1097 (Fed. Cir. 1986); In re Keller, 642 F.2d 413, 426 (CCPA 1981). Contrary to Appellant’s arguments, the Examiner acknowledges that Iiyama does not explicitly teach projecting the texture onto the input images to estimate the error. See Final Act. 7–8 (citing Iiyama, section III, paragraphs 3, 7); Ans. 4. Instead of relying on Iiyama’s teachings, the Examiner relies on Eckert for teaching a photo consistency error metric for measuring the similarity between a synthesized view (i.e., a textured triangle projected into image planes of all other cameras that observe it), and all of the other input images which observe the given element of the mesh, as modified by Lindstrom’s teaching of comparing models using metrics that measure the Appeal 2019-001565 Application 14/372,018 6 difference between a set of images taken of an original object (reference camera images) and a set of images taken of a modified model of the object being optimized (textured mesh elements projected onto reference camera views). See Final Act. 7–8 (citing Eckert, section 3, paragraph 3), id. at 10 (citing Lindstrom, sections 3, 4.2.3, 4.2, 4.2.1, 4.2.2 ); Ans. 4–5. The Examiner determined that it would have been obvious to modify Iiyama’s super-resolution texture mapping system to include Eckert’s photo- consistency error metric in addition to Iiyama’s patch edge artifact error metric Etex(i,j) by replacing Etex(i,j) with Ephotex(i, j) = Etex(i,j) +Ephoto(i) + Ephoto(j), and including every reference camera image, as suggested by Lindstrom, thereby including the selected reference camera image being used to texture the mesh element in the set of reference camera images used for comparison between original reference images and the textured model. See Final Act. 9, 11–12; Ans. 4–5. Appellant further argues that the combination of Iiyama and Eckert would lead one with ordinary skill in the art to select texture information from one single image to project the textured triangle onto the other images and to project the thus obtained textured triangle onto all camera images but the reference image. See Appeal Br. 16. Appellant contends the combination of Iiyama and Eckert would teach away from the concept of texturing the triangle with the texture obtained from each and every camera image of the plurality to test the texture of each camera image. See id. Appellant’s arguments are not persuasive of Examiner error because they do not address the teachings of Iiyama, Eckert, and Lindstrom, as combined by the Examiner. The Examiner does not propose modifying Iiyama in view of Eckert to use only texture information from one image to project onto other images. Instead, as explained by the Examiner, Appeal 2019-001565 Application 14/372,018 7 Lindstrom’s teaching of comparing models using metrics that measure the difference between a set of images taken of an original object and a set of images taken of a modified model of the object being optimized, in combination with Iiyama and Eckert, teaches measuring the photo consistency between all of the original images and all of the corresponding synthesized images of the modified model. See Ans. 5; see also Final Act. 10–12 (similar explanation). For the foregoing reasons, we are not persuaded of error in the Examiner’s rejections of claims 1, 2, 8, 9, and 15. Rejections of Claims 4–7 and 11–14 Claims 4–7 depend from claim 1, and claims 11–14 depend from independent claim 8. For the rejections of these claims, Appellant argues that each of the additional references (i.e., Debevec, Carranza, and Bogomjakov) fail to teach or suggest at least the use of first image(s) of the scene from which the mesh element is only partially visible as is recited in claims 1 and 8. See Appeal Br. 20–22. Appellant’s arguments are not persuasive of error in the Examiner’s rejection because they are misplaced. The Examiner does not rely on Debevec, Carranza, or Bogomjakov for teaching or suggesting “the mesh being associated with a surface representative of a scene captured according to a plurality of points of view, an image of the scene being associated with each point of view . . . for one mesh element of said mesh that is at least partially visible from at least two images of the plurality of images,” as recited in claim 1. The Examiner instead relies on Iiyama for teaching the aforementioned limitations. See Final Act. 2–3. Appeal 2019-001565 Application 14/372,018 8 Accordingly, for this reason, we are not persuaded of error in the Examiner’s rejections of claims 4–7 and 11–14. CONCLUSION We affirm the Examiner’s rejections of claims 1, 2, 4–9, and 11–15 under 35 U.S.C. § 103(a). In summary: Claims Rejected 35 U.S.C. § References Affirmed Reversed 1, 2, 8, 9, 15 103(a) Iiyama, Eckert, Lindstrom 1, 2, 8, 9, 15 4, 5, 11, 12 103(a) Iiyama, Eckert, Lindstrom, Debevec 4, 5, 11, 12 6, 13 103(a) Iiyama, Eckert, Lindstrom, Carranza 6, 13 7, 14 103(a) Iiyama, Eckert, Lindstrom, Bogomjakovic 7, 14 Overall Outcome 1, 2, 4–9, 11–15 TIME PERIOD FOR RESPONSE No time period for taking any subsequent action in connection with this appeal may be extended under 37 C.F.R. § 1.136(a). See 37 C.F.R. § 1.136(a)(1)(iv). AFFIRMED Copy with citationCopy as parenthetical citation