Apple Inc.Download PDFPatent Trials and Appeals BoardDec 7, 20212020002774 (P.T.A.B. Dec. 7, 2021) Copy Citation UNITED STATES PATENT AND TRADEMARK OFFICE UNITED STATES DEPARTMENT OF COMMERCE United States Patent and Trademark Office Address: COMMISSIONER FOR PATENTS P.O. Box 1450 Alexandria, Virginia 22313-1450 www.uspto.gov APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO. 15/455,890 03/10/2017 Jae Hoon Kim 082438.310200 1060 139020 7590 12/07/2021 BakerHostetler / Apple Inc. Washington Square, Suite 1100, 1050 Connecticut Ave, NW Washington, DC 20036-5304 EXAMINER ZENATI, AMAL S ART UNIT PAPER NUMBER 2651 NOTIFICATION DATE DELIVERY MODE 12/07/2021 ELECTRONIC Please find below and/or attached an Office communication concerning this application or proceeding. The time period for reply, if any, is set in the attached communication. Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the following e-mail address(es): eofficemonitor@bakerlaw.com PTOL-90A (Rev. 04/07) UNITED STATES PATENT AND TRADEMARK OFFICE _______________ BEFORE THE PATENT TRIAL AND APPEAL BOARD _______________ Ex parte JAE HOON KIM, CHRIS Y. CHUNG, DAZHONG ZHANG, HANG YUAN, HIS-JUNG WU, XIAOSONG ZHOU, and JIEFU ZHAI _______________ Appeal 2020-002774 Application 15/455,890 Technology Center 2600 _______________ Before JOHN A. JEFFERY, JOHNNY A. KUMAR, and JAMES W. DEJMEK, Administrative Patent Judges. DEJMEK, Administrative Patent Judge. DECISION ON APPEAL Appellant1 appeals under 35 U.S.C. § 134(a) from a Final Rejection of claims 1–34. We have jurisdiction over the pending claims under 35 U.S.C. § 6(b). We reverse. 1 Throughout this Decision, we use the word “Appellant” to refer to “applicant” as defined in 37 C.F.R. § 1.42 (2018). Appellant identifies Apple Inc. as the real party in interest. Appeal Br. 1. Appeal 2020-002774 Application 15/455,890 2 STATEMENT OF THE CASE Introduction Appellant’s disclosed and claimed invention generally relates to “video processing and, more particularly, perspective shifting in a video conferencing session.” Spec. ¶ 1. More specifically, Appellant describes an embodiment in which an image analyzer parses video data into one or more foreground elements and one or more background elements. Spec. ¶ 19, Fig. 2A. To achieve a perspective shift, in response to an input from a recipient device, the image analyzer may shift the identified foreground elements with respect to the identified background elements when the video data is reconstructed prior to transmission to the recipient device. Spec. ¶¶ 19, 28. Claim 1 is illustrative of the subject matter on appeal and is reproduced below with the disputed limitations emphasized in italics: 1. A method comprising: generating, by a first terminal, a video stream; distinguishing, by an analyzer, foreground pixels of a frame of the video stream representing a foreground element from background pixels of the frame representing a background element; responsive to information representing a viewing condition at a second terminal, creating, by the analyzer, a modified video frame by compositing the foreground pixels and the background pixels including a shift in the relative positions of the foreground pixels and the background pixels; and transmitting the modified video frame to the second terminal. Appeal 2020-002774 Application 15/455,890 3 The Examiner’s Rejections 1. Claims 1–19, 27, and 32–34 stand rejected under 35 U.S.C. § 103 as being unpatentable over El-Saban et al. (US 2012/0327172 A1; Dec. 27, 2012) (“El-Saban”) or Lee (US 9,258,520 B2; Feb. 9, 2016) in view of Didow et al. (US 2007/0064120 A1; Mar. 22, 2007) (“Didow”) or Knee et al. (EP 2077525 A1; July 8, 2009) (“Knee”). Final Act. 2–5. 2. Claims 20–26 and 28–31 stand rejected under 35 U.S.C. § 103 as being unpatentable over El-Saban or Lin et al. (US 2015/0029294 A1; Jan. 29, 2015) (Lin”) in view of Didow. Final Act. 6–7. ANALYSIS2 Each of the independent claims (i.e., claims 1, 14, 20, 27, 28, 33, and 34) generally recite, inter alia, distinguishing foreground and background pixels of the same frame of a video stream and, responsive to a viewing condition input at a second (i.e., recipient) device, creating a modified video frame by compositing the foreground and background pixels (from the same frame) in a manner to achieve a shift in the relative position of the foreground pixels and background pixels. The Examiner finds, alternatively, that either El-Saban, Lee, Didow, Knee, or Lin teaches this aspect of the claimed invention. See Final Act. 2–7; Ans. 4–8. We begin our analysis with a brief review of the prior art. Although El-Saban describes various foreground/background separation techniques (see, e.g., El-Saban ¶¶ 8–9, 43–45), El-Saban 2 Throughout this Decision, we have considered the Appeal Brief, filed September 17, 2019 (“Appeal Br.”); the Reply Brief, filed February 27, 2020 (“Reply Br.”); the Examiner’s Answer, mailed January 10, 2020 (“Ans.”); and the Final Office Action, mailed March 22, 2019 (“Final Act.”), from which this Appeal is taken. Appeal 2020-002774 Application 15/455,890 4 describes allowing a user to define a template on the display of a video capture device (e.g., a camera). El-Saban ¶ 66. El-Saban further describes the template is superimposed on the display and the device is moved to position the object within the superimposed template. El-Saban ¶¶ 67–69, Figs. 3A, 3B. Contrary to the Examiner’s findings (see Final Act. 2–6; Ans. 4–6), El-Saban does not create a modified frame by shifting the foreground pixels and background pixels from a single frame relative to each other, but instead combines foreground regions and background regions from different images. Moreover, Lee teaches a more intuitive approach to allow a user to center the subject of a video stream (captured by a first device) on the display of a second device (i.e., a receiving device). Lee, col. 1, ll. 31–39. In particular, Lee teaches presenting a mirror image of the captured image to a user of the first device such that if the subject appears too far left or right from center, the user can intuitively move the subject or the camera in the opposite direction to better align the subject. Lee, col. 6, ll. 21–45. Thus, Lee does not teach creating a modified frame by shifting the foreground pixels relative to the background pixels of the same frame. Didow relates to “a method of creating event still photos or videos using chroma-key technology.” Didow ¶ 2. Didow discloses that in such techniques, a subject stands (or is placed) in front of a “green screen” (or “blue screen”) and a substitute background is inserted into a resulting composite photograph. Didow ¶¶ 5, 25. In other words, rather than creating a modified frame by shifting the foreground pixels relative to the background pixels of the same frame, Didow’s resulting composite image uses the background from a separate image along with the foreground subject of a different image. Appeal 2020-002774 Application 15/455,890 5 Knee generally relates to “automatically reframing a moving image sequence where the framing is adjusted in dependence upon foreground and background segment weights derived for pixels in images in the sequence.” Knee ¶ 4. However, as relied on by the Examiner, Knee does not describe the reframing as shifting the foreground pixels relative to the background pixels of the same frame. Rather, it would appear that the relative positioning of the foreground and background pixels remains constant. Finally, Lin generally relates to “integrating user personas with other display content during video conferencing.” Lin ¶ 1. Lin describes embedding an extracted video persona “into a background feed, which may be, e.g., a background video, a desktop, slides, images, or any application window.” Lin ¶ 3. Similar to Didow, rather than creating a modified frame by shifting the foreground pixels relative to the background pixels of the same frame, Lin’s resulting composite image uses the background from a separate image along with the foreground subject of a different image. Appellant disputes the Examiner’s findings that any of the identified references teach creating a modified frame of a video stream by compositing the foreground pixels and background pixels of the same frame with a relative shift. Appeal Br. 7–12; Reply Br. 2–4.3 In particular, Appellant asserts that rather than describing compositing foreground pixels and background pixels from the same frame to achieve a perspective shift, El- Saban describes capturing a new picture of a new scene to create a desired picture. Appeal Br. 8–9 (citing El-Saban ¶¶ 68–69, Figs. 3A, 3B); Reply Br. 2. Appellant further asserts that Lee merely describes moving an object 3 The Appeal Brief does not include page numbers. For reference herein, we assume the first page of the Appeal Brief is page 1 and the subsequent pages are consecutively numbered. Appeal 2020-002774 Application 15/455,890 6 (e.g., the subject of the video frame) or a camera to center the object in the video frame. Appeal Br. 9–10 (citing Lee, col. 6, ll. 21–45, Figs. 5A, 5B); Reply Br. 2–3. Appellant argues Didow describes a traditional “green screen” technique to combine an image of an object in front of a green background with a second image of a scene by replacing the green background in a first image with the scene captured in a second image. Appeal Br. 10–11 (citing Didow ¶¶ 55–56); Reply Br. 3. Further, Appellant argues that Knee merely describes “reframing an image without any compositing or shifting of foreground relative to background.” Appeal Br. 11–12 (citing Knee ¶ 7); Reply Br. 2. In addition, Appellant argues that the combination of either El-Saban or Lee in view of either Didow or Knee fails to teach compositing foreground and background pixels from the same frame, including a shift in the relative positions of the foreground and background pixels. Appeal Br. 12. Based on our review of the prior art (discussed above), we agree with Appellant that, as identified by the Examiner, the prior art (alone or in combination) fails to teach distinguishing foreground pixels and background pixels in a frame of a video stream and, in response to a request for an altered viewing condition at a second terminal, creating a modified video frame by compositing the foreground and background pixels including a shift in the relative positions of the foreground pixels and background pixels, as set forth in independent claims 1, 14, 20, 27, and 28. For the reasons discussed supra, we are persuaded of Examiner error. Accordingly, we do not sustain the Examiner’s rejection of independent claim 1. For similar reasons, we do not sustain the Examiner’s rejections of independent claims 14, 20, 27, and 28, which recite limitations commensurate in scope to the disputed limitations of independent claim 1. Appeal 2020-002774 Application 15/455,890 7 In addition, we do not sustain the Examiner’s rejections of claims 2–13, 15– 19, 21–26, and 29–32, which depend directly or indirectly therefrom. In addition, the prior art similarly fails to teach identifying a foreground portion and a background portion from a single digital image in a video stream and, in response to a request for an altered viewing condition at a destination terminal, creating a modified video frame by compositing the foreground and background portions including a shift in the relative positions of the foreground portion and background portion, as recited in independent claim 33 and 34. As discussed above, the cited references generally composite a modified image using foreground and background portions taken from different images. Accordingly, we do not sustain the Examiner’s rejection of independent claims 33 and 34. CONCLUSION We reverse the Examiner’s decision rejecting claims 1–34 under 35 U.S.C. § 103. DECISION SUMMARY Claim(s) Rejected 35 U.S.C. § Reference(s)/Basis Affirmed Reversed 1–19, 27, 32–34 103 El-Saban, Lee, Didow, Knee 1–19, 27, 32–34 20–26, 28–31 103 El-Saban, Lin, Didow 20–26, 28–31 Overall Outcome 1–34 REVERSED Copy with citationCopy as parenthetical citation