Michael Tamir et al.Download PDFPatent Trials and Appeals BoardSep 13, 201911909080 - (D) (P.T.A.B. Sep. 13, 2019) Copy Citation UNITED STATES PATENT AND TRADEMARK OFFICE UNITED STATES DEPARTMENT OF COMMERCE United States Patent and Trademark Office Address: COMMISSIONER FOR PATENTS P.O. Box 1450 Alexandria, Virginia 22313-1450 www.uspto.gov APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO. 11/909,080 09/19/2007 Michael Tamir 3621/2 1949 44696 7590 09/13/2019 Dr. Mark M. Friedman Moshe Aviv Tower, 54th floor 7 Jabotinsky St. Ramat Gan, 5252007 ISRAEL EXAMINER MAI, KEVIN S ART UNIT PAPER NUMBER 2456 NOTIFICATION DATE DELIVERY MODE 09/13/2019 ELECTRONIC Please find below and/or attached an Office communication concerning this application or proceeding. The time period for reply, if any, is set in the attached communication. Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the following e-mail address(es): friedpat.uspto@gmail.com patents@friedpat.com rivka_f@friedpat.com PTOL-90A (Rev. 04/07) UNITED STATES PATENT AND TRADEMARK OFFICE ____________________ BEFORE THE PATENT TRIAL AND APPEAL BOARD ____________________ Ex parte MICHAEL TAMIR and GAL OZ ____________________ Appeal 2018-008357 Application 11/909,0801 Technology Center 2400 ____________________ Before MURRIEL E. CRAWFORD, MARC S. HOFF, and CATHERINE SHIANG, Administrative Patent Judges. HOFF, Administrative Patent Judge. DECISION ON APPEAL STATEMENT OF THE CASE Appellants appeal under 35 U.S.C. § 134 from a final rejection of claims 62, 64, 65, 79–81, 84, 87–90, 93, and 95–101.2 We have jurisdiction under 35 U.S.C. § 6(b). We affirm. Appellants’ invention concerns real-time object tracking and motion capture in sports events, specifically tracking of athletes and objects such as balls and cars. Spec. 1. A plurality of television cameras are peripherally deployed at a sports arena such that any given point on the playing field is 1 Appellants state that the real party in interest is Sportview Ltd. Br. 3. 2 Claims 1–61, 63, 66–78, 82, 83, 85, 86, 91, 92, and 94 have been cancelled. Appeal 2018-008357 Application 11/909,080 2 covered by at least one camera. A processing unit performs object segmentation, blob analysis, and 3D object localization and tracking. Spec. 2. Captured player body organ or joints location data can be used to generate a three-dimensional display of the sporting event. Abstract. Claim 62 is exemplary of the claims on appeal: 62. A system for real-time object localization and tracking in a sports event comprising: a. a plurality of fixed cameras operative to capture video of a sports playing field including objects located therein, the plurality of fixed cameras positioned at a single location relative to the sports playing field, with each camera covering a portion of the sports playing field to capture video of objects on the sports playing; b. an image processing unit operative to: receive video frames from each camera including an image, and to detect and segment from the image, a background image defined by the sports playing field of the image, and, a foreground image including, objects which move on the sports playing field, each object represented by pixels of the foreground image, the pixels initially being isolated in the segmented foreground image, with at least a portion of the isolated pixels connected into clusters by pixel connection processes to define a blob, which represents the object, analyze color attributes of each blob to assign the corresponding object an identity group, and, determining the location of each of the objects in a field of view of each of the cameras, the field of view covering at least a portion of the sports playing field, the portion of the sports playing field represented in the background image, and locating each of the objects on the sports playing field; and, c. a central server operative to provide in real-time, localization and tracking information on the objects which have been located on the sports playing field, and provide at least a portion of the field of view of each of the cameras, including localization and tracking information, for display in broadcast media. Appeal 2018-008357 Application 11/909,080 3 The Examiner relies upon the following prior art in rejecting the claims on appeal: Uesaki US 2003/0051256 A1 Mar. 13, 2003 Foote US 7,015,954 B1 Mar. 21, 2006 Aman US 2007/0279494 A1 Dec. 6, 2007 Huxley WO 99/17250 Apr. 8, 1999 Claims 62, 64, 65, 79–81, 84, 87–90, 93, and 95–101 stand rejected under 35 U.S.C. § 102(e) as being anticipated by Aman. Claims 62 and 65 stand rejected under 35 U.S.C. § 102(e) as being anticipated by Foote. Claim 64 stands rejected under 35 U.S.C. § 103(a) as being unpatentable over Foote and Aman. Claims 62, 64, 65, 79–81, 84, 87–90, 93, and 95–101 stand rejected under 35 U.S.C. § 103(a) as being unpatentable over Aman and Uesaki. Claims 62, 64, 65, 79–81, 84, 87–90, 93, and 95–101 stand rejected under 35 U.S.C. § 103(a) as being unpatentable over Aman and Huxley. Claims 62, 64, 65, 79–81, 84, 87–90, 93, and 95–101 stand rejected under 35 U.S.C. § 103(a) as being unpatentable over Aman and Applicant’s Admitted Prior Art (AAPA). Throughout this decision, we make reference to the Appeal Brief (“Br.,” filed February 5, 2018), and the Examiner’s Answer (“Ans.,” mailed June 1, 2018) for their respective details. Appeal 2018-008357 Application 11/909,080 4 ISSUES Appellants’ arguments present us with the following issues: 1. Does Aman disclose connecting isolated pixels into clusters by pixel connection processes to define a blob? 2. Does Aman disclose analyzing the color attributes of each blob to assign the corresponding object an identity group? 3. Does Aman disclose matching player portions detected by different video cameras based on their locations on the playing field and on morphological knowledge of the human body? 4. Does the combination of Aman and Uesaki disclose or suggest analyzing each blob, based on color attributes, to identify the blob as corresponding to an identity group for players? ANALYSIS § 102(E) REJECTION OF CLAIMS 62, 64, 65, 84, 87–90, AND 96–101 OVER AMAN Appellants argue that the Examiner erred in that Aman fails to teach “at least a portion of the isolated pixels connected into clusters by pixel connection processes to define a blob, which represents the object.” Br. 20. Appellants assert that Aman does not teach “connecting pixels, or even groups of pixels, to form a larger unit, such as the recited blobs of claim 62.” Id. We do not agree with Appellants that Aman fails to teach this “connecting pixels” limitation. The Examiner finds, and we agree, that Aman discloses “examining small blocks of adjacent pixels from the subtracted image 10s in order to determine their average grayscale value . . . Appeal 2018-008357 Application 11/909,080 5 . [W]herever there is a large enough change in contrast within the subtracted image 10s, the pixels of the gradient image log are set to white forming in effect a ‘line-drawing’ of the foreground object.” Aman ¶ 320. The Examiner finds that this line-drawing corresponds to the claimed connecting of pixels to create a blob representing the foreground object. Ans. 3. As a result, we agree with the Examiner that Aman ¶ 320 teaches the claimed “at least a portion of the isolated pixels connected into clusters by pixel connection processes to define a blob, which represents the object.” We further agree with the Examiner’s finding that Aman discloses another instance of connecting pixels to create a blob: that the “configuration of pixels forming an extended, narrow straight line can be detected and interpreted as a player’s stick.” Aman ¶ 350. The next limitation of independent claims 62 and 84 concerns (as specifically recited in claim 62) “analyz[ing] color attributes of each blob to assign the corresponding object an identity group.” In general, a reference is not required to teach the steps of a method in the order recited in the claim. In the claims under appeal, however, we determine from context that a blob must be defined before its color attributes may be analyzed. We have reviewed the Examiner’s citations to Aman (Ans. 4–5), and we find that Aman’s various cited teachings, of (a) tracking and identifying individual players and referees (¶ 402); (b) reducing the stream of data to “those areas of the surface 2 where foreground objects 10e (players, referees, equipment, the puck, etc.) are found” (¶ 322); (c) establishing a table of pre-known color tones “for all participant skin complexions as well as home and away uniforms, such that each pixel in the extracted foreground images can be encoded to represent one of these color tones” (¶ 181); (d) identifying a Appeal 2018-008357 Application 11/909,080 6 player’s face region, with reference to color tone table 104a (¶ 468); and (e) creating a minimum bounding box around any foreground areas where a known skin color tone is found (¶ 331), do not correspond to teachings of analyzing the color attributes of the “blobs,” connected in the previous step through pixel connection processes, to assign that corresponding object an identity group, as recited in independent claim 62. The cited teachings also do not correspond to “analyz[ing] each blob, based on color attributes, to identify the blob as corresponding to an identity group for players,” as recited in independent claim 84. Accordingly, we find that Aman does not teach all the limitations of the invention recited in independent claims 62 and 84. We do not sustain the Examiner’s § 102(e) rejection of claims 62, 64, 65, 84, 87–90, and 96–101. § 102(E) REJECTION OF CLAIMS 93, 79–81, AND 95 OVER AMAN Independent claim 93 recites, in pertinent part, “matching player portions detected by different video cameras based on their locations on the playing field and on morphological knowledge of human body.” Appellants assert that Aman does not teach using morphological knowledge of the human body. Br. 24. Appellants’ position is that Aman reads non-visible markings placed on key spots of a player’s uniform, such as shoulders, elbows, waists, knees, and ankles. Id.; Aman ¶ 98. Appellants reason that since these body parts are marked, Aman does not need to have any morphological knowledge of the body. Id. We are not persuaded by Appellants’ argument. The Examiner finds, and we agree, that the Specification discloses that “[a]utomatic foreground blobs segmentation into player’s head, torso, hands and legs is then performed in step 734 by IPPIU 704 using pattern recognition algorithms Appeal 2018-008357 Application 11/909,080 7 that are well known in the art,” citing a publication by J.M. Buades et al. Spec. 14; Ans. 6–7. We agree with the Examiner that the cited section of the Specification constitutes an admission that matching player portions detected by different video cameras based on their locations on the playing field and on morphological knowledge of the human body is well-known in the art. Further, the Examiner cites to several paragraphs of Aman as suggesting that Aman employs morphological knowledge of the human body. For example, Aman teaches that “after the location of player 10’s head 10sH is determined an oval 10sB will be optimally fit around the remaining portion of the foreground object’s gradient outline.” Aman ¶ 350; Ans. 8–9. This concept of an oval is pertinent because “using image analysis the body oval 10sB can be analyzed to determine the orientation of the player’s 10 shoulders.” Id. Aman thus demonstrates the knowledge that the position of a player’s shoulders can be determined from the location of a player’s head, which requires morphological knowledge of the human body. Aman further teaches that “oval 10sB will approach an elliptical shape as the player stands upright”; this information can be used in combination with the fact that “the front of the player’s 10 body, and therefore the ‘front’ of any representative ellipse, is oriented in the direction of the front of the helmet sticker 9a.” Id. The player’s neck is then disclosed to be “assumed to be a fixed distance from the center of the helmet sticker 9a,” another indication that Aman employs morphological knowledge of the human body, i.e., the relationship of the neck to the head. Id. We agree with the Examiner’s finding that Aman teaches “matching player portions detected by different video cameras based on their locations Appeal 2018-008357 Application 11/909,080 8 on the playing field and on morphological knowledge of human body,” as independent claim 93 requires. We find that the Examiner did not err in rejecting claim 93, or dependent claims 79–81 and 95, as being anticipated by Aman. We sustain the Examiner’s § 102(e) rejection of claim 93, as well as claims 79–81 and 95 dependent therefrom. § 102 REJECTION OF CLAIMS 62 AND 65 OVER FOOTE The Examiner finds that Foote discloses “isolated pixels connected into clusters by pixel connection processes to define a blob,” as independent claim 62 requires. Ans. 9. The Examiner cites to columns 2 and 15 of Foote in support of this finding. Final Act. 29. Foote discloses at column 2, lines 54–59 that “[t]he wide field sensor is utilized to locate people or regions of interest . . . . By tracking shape, motion, color, and/or audio cues, the location of people or other items of interest in the room or scene being captured by the cameras can be estimated.” The cited portion of Foote column 15 concerns estimating the location of an acoustic source by finding peaks or corresponding features in acoustic signals from microphones. Neither section discloses the claimed connection of pixels to form blobs. We agree with Appellants that Foote does not teach all the limitations of the invention recited in claims 62 and 65. Br. 25–26. We do not sustain the Examiner’s § 102(e) rejection. § 103(A) REJECTION OVER AMAN AND UESAKI Appellants argue that the Examiner erred in rejecting the claims over the combination of Aman and Uesaki because (a) Aman fails to disclose pixels connected into clusters by pixel connection processes to define a blob, (b) Aman fails to disclose analyzing color attributes of each blob to assign Appeal 2018-008357 Application 11/909,080 9 the corresponding object an identity group, and (c) Uesaki does not teach any “creation of blobs, or operation on the blobs.” Br. 28–29. As discussed supra with respect to the Examiner’s § 102(e) rejection over Aman, we find that Aman teaches “isolated pixels connected into clusters by pixel connection processes to define a blob.” Further, while we find supra that Aman does not teach analyzing color attributes of blobs to assign the corresponding object an identity group, we agree with the Examiner that Aman does teach analyzing color attributes of objects. For example, Aman teaches extraction of a full color image 10fc, and determining whether foreground pixels correspond to non-facial color tones. Aman ¶ 331. Aman teaches using the captured images of a selected player’s uniform, preferably including their jersey number, to compare and pattern match against a pre-known database thereby allowing for participant identification. Aman ¶ 156. Aman teaches a table of pre-known color tones established for all participant skin complexions as well as home and away uniforms, implicitly suggesting that images will be analyzed for regions that correspond to those pre-known color tones. Aman ¶ 181. We agree with the Examiner’s finding that Uesaki teaches the claimed analyzing color attributes of objects. Final Act. 36–37. “Because the color of the uniform can be obtained in advance, it is possible to identify the team with the color information.” Uesaki ¶ 69. “For example, the team can be identified by matching the color information of the object with the color information of the uniform.” Uesaki ¶ 70. We do not agree with Appellants that there would be no motivation to combine Aman and Uesaki, “except as ‘gleaned from applicant’s disclosure.’” Br. 29 (emphasis omitted). Appellants’ contention is based on Appeal 2018-008357 Application 11/909,080 10 the predicate that Uesaki teaches neither “creation of blobs” nor “operation on the blobs.” Id. As discussed supra, however, we disagree with that predicate. We agree with the Examiner’s rationale that it would have been obvious to apply the practice of using colors to identify teams, taught by Uesaki, to the invention of Aman, because both references are directed to identifying and tracking objects, and the modification simply calls for the application of a known technique to similar devices. Final Act. 37. Appellants’ argument that the combination fails to teach the limitations of independent claim 93, because Uesaki fails to teach “matching player portions detected by different video cameras based on their locations on the playing field and on morphological knowledge of human body,” is not persuasive. Br. 29. As discussed supra with respect to the § 102(e) rejection of claim 93 over Aman, we agree with the Examiner that Aman teaches this limitation. Therefore, there is no need to look to Uesaki to remedy any deficiency in the Aman reference. We conclude that it would have been obvious to combine Aman with Uesaki to achieve the claimed invention. We sustain the Examiner’s rejection of claims 62, 64, 65, 79–81, 84, 87–90, 93, and 95–101 over Aman and Uesaki. FURTHER GROUNDS OF REJECTION We sustain supra the rejection of all pending claims over the combination of Aman and Uesaki. As a consequence, we need not reach the question of whether the Examiner erred in rejecting all pending claims over Aman and Huxley, or erred in rejecting all pending claims over Aman in view of admitted prior art, or erred in rejecting claim 64 over Foote and Aman. Appeal 2018-008357 Application 11/909,080 11 CONCLUSIONS 1. Aman discloses connecting isolated pixels into clusters by pixel connection processes to define a blob. 2. Aman does not disclose analyzing the color attributes of each blob to assign the corresponding object an identity group. 3. Aman discloses matching player portions detected by different video cameras based on their locations on the playing field and on morphological knowledge of the human body. 4. The combination of Aman and Uesaki suggests analyzing each blob, based on color attributes, to identify the blob as corresponding to an identity group for players. ORDER The Examiner’s decision to reject claims 62, 64, 65, 79–81, 84, 87– 90, 93, and 95–101 is affirmed. No time period for taking any subsequent action in connection with this appeal may be extended under 37 C.F.R. § 1.136(a). AFFIRMED Copy with citationCopy as parenthetical citation