Michael Tamir et al.

Patent Trials and Appeals Board

Sep 13, 2019

11909080 - (D) (P.T.A.B. Sep. 13, 2019)

UNITED STATES PATENT AND TRADEMARK OFFICE
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O. Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
11/909,080 09/19/2007 Michael Tamir 3621/2 1949
44696 7590 09/13/2019
Dr. Mark M. Friedman
Moshe Aviv Tower, 54th floor
7 Jabotinsky St.
Ramat Gan, 5252007
ISRAEL
EXAMINER
MAI, KEVIN S
ART UNIT PAPER NUMBER
2456
NOTIFICATION DATE DELIVERY MODE
09/13/2019 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
friedpat.uspto@gmail.com
patents@friedpat.com
rivka_f@friedpat.com
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
____________________

BEFORE THE PATENT TRIAL AND APPEAL BOARD
____________________

Ex parte MICHAEL TAMIR and GAL OZ
____________________

Appeal 2018-008357
Application 11/909,0801
Technology Center 2400
____________________

Before MURRIEL E. CRAWFORD, MARC S. HOFF, and
CATHERINE SHIANG, Administrative Patent Judges.

HOFF, Administrative Patent Judge.

DECISION ON APPEAL

STATEMENT OF THE CASE
Appellants appeal under 35 U.S.C. § 134 from a final rejection of
claims 62, 64, 65, 79–81, 84, 87–90, 93, and 95–101.2 We have jurisdiction
under 35 U.S.C. § 6(b).
We affirm.
Appellants’ invention concerns real-time object tracking and motion
capture in sports events, specifically tracking of athletes and objects such as
balls and cars. Spec. 1. A plurality of television cameras are peripherally
deployed at a sports arena such that any given point on the playing field is

1 Appellants state that the real party in interest is Sportview Ltd. Br. 3.
2 Claims 1–61, 63, 66–78, 82, 83, 85, 86, 91, 92, and 94 have been cancelled.
Appeal 2018-008357
Application 11/909,080

2
covered by at least one camera. A processing unit performs object
segmentation, blob analysis, and 3D object localization and tracking. Spec.
2. Captured player body organ or joints location data can be used to generate
a three-dimensional display of the sporting event. Abstract.
Claim 62 is exemplary of the claims on appeal:
62. A system for real-time object localization and tracking in a
sports event comprising:
a. a plurality of fixed cameras operative to capture video
of a sports playing field including objects located therein, the
plurality of fixed cameras positioned at a single location
relative to the sports playing field, with each camera covering a
portion of the sports playing field to capture video of objects on
the sports playing;
b. an image processing unit operative to:
receive video frames from each camera including an
image, and to detect and segment from the image, a background
image defined by the sports playing field of the image, and, a
foreground image including, objects which move on the sports
playing field, each object represented by pixels of the
foreground image, the pixels initially being isolated in the
segmented foreground image, with at least a portion of the
isolated pixels connected into clusters by pixel connection
processes to define a blob, which represents the object,
analyze color attributes of each blob to assign the
corresponding object an identity group, and,
determining the location of each of the objects in a field
of view of each of the cameras, the field of view covering at
least a portion of the sports playing field, the portion of the
sports playing field represented in the background image, and
locating each of the objects on the sports playing field; and,
c. a central server operative to provide in real-time,
localization and tracking information on the objects which have
been located on the sports playing field, and provide at least a
portion of the field of view of each of the cameras, including
localization and tracking information, for display in broadcast
media.
Appeal 2018-008357
Application 11/909,080

3
The Examiner relies upon the following prior art in rejecting the
claims on appeal:
Uesaki US 2003/0051256 A1 Mar. 13, 2003
Foote US 7,015,954 B1 Mar. 21, 2006
Aman US 2007/0279494 A1 Dec. 6, 2007
Huxley WO 99/17250 Apr. 8, 1999

Claims 62, 64, 65, 79–81, 84, 87–90, 93, and 95–101 stand rejected
under 35 U.S.C. § 102(e) as being anticipated by Aman.
Claims 62 and 65 stand rejected under 35 U.S.C. § 102(e) as being
anticipated by Foote.
Claim 64 stands rejected under 35 U.S.C. § 103(a) as being
unpatentable over Foote and Aman.
Claims 62, 64, 65, 79–81, 84, 87–90, 93, and 95–101 stand rejected
under 35 U.S.C. § 103(a) as being unpatentable over Aman and Uesaki.
Claims 62, 64, 65, 79–81, 84, 87–90, 93, and 95–101 stand rejected
under 35 U.S.C. § 103(a) as being unpatentable over Aman and Huxley.
Claims 62, 64, 65, 79–81, 84, 87–90, 93, and 95–101 stand rejected
under 35 U.S.C. § 103(a) as being unpatentable over Aman and Applicant’s
Admitted Prior Art (AAPA).
Throughout this decision, we make reference to the Appeal Brief
(“Br.,” filed February 5, 2018), and the Examiner’s Answer (“Ans.,” mailed
June 1, 2018) for their respective details.
Appeal 2018-008357
Application 11/909,080

4
ISSUES
Appellants’ arguments present us with the following issues:
1. Does Aman disclose connecting isolated pixels into clusters by
pixel connection processes to define a blob?
2. Does Aman disclose analyzing the color attributes of each blob to
assign the corresponding object an identity group?
3. Does Aman disclose matching player portions detected by different
video cameras based on their locations on the playing field and on
morphological knowledge of the human body?
4. Does the combination of Aman and Uesaki disclose or suggest
analyzing each blob, based on color attributes, to identify the blob as
corresponding to an identity group for players?

ANALYSIS
§ 102(E) REJECTION OF CLAIMS 62, 64, 65, 84, 87–90, AND 96–101 OVER
AMAN
Appellants argue that the Examiner erred in that Aman fails to teach
“at least a portion of the isolated pixels connected into clusters by pixel
connection processes to define a blob, which represents the object.” Br. 20.
Appellants assert that Aman does not teach “connecting pixels, or even
groups of pixels, to form a larger unit, such as the recited blobs of claim 62.”
Id.
We do not agree with Appellants that Aman fails to teach this
“connecting pixels” limitation. The Examiner finds, and we agree, that
Aman discloses “examining small blocks of adjacent pixels from the
subtracted image 10s in order to determine their average grayscale value . . .
Appeal 2018-008357
Application 11/909,080

5
. [W]herever there is a large enough change in contrast within the subtracted
image 10s, the pixels of the gradient image log are set to white forming in
effect a ‘line-drawing’ of the foreground object.” Aman ¶ 320. The
Examiner finds that this line-drawing corresponds to the claimed connecting
of pixels to create a blob representing the foreground object. Ans. 3. As a
result, we agree with the Examiner that Aman ¶ 320 teaches the claimed “at
least a portion of the isolated pixels connected into clusters by pixel
connection processes to define a blob, which represents the object.”
We further agree with the Examiner’s finding that Aman discloses
another instance of connecting pixels to create a blob: that the
“configuration of pixels forming an extended, narrow straight line can be
detected and interpreted as a player’s stick.” Aman ¶ 350.
The next limitation of independent claims 62 and 84 concerns (as
specifically recited in claim 62) “analyz[ing] color attributes of each blob to
assign the corresponding object an identity group.” In general, a reference is
not required to teach the steps of a method in the order recited in the claim.
In the claims under appeal, however, we determine from context that a blob
must be defined before its color attributes may be analyzed. We have
reviewed the Examiner’s citations to Aman (Ans. 4–5), and we find that
Aman’s various cited teachings, of (a) tracking and identifying individual
players and referees (¶ 402); (b) reducing the stream of data to “those areas
of the surface 2 where foreground objects 10e (players, referees, equipment,
the puck, etc.) are found” (¶ 322); (c) establishing a table of pre-known color
tones “for all participant skin complexions as well as home and away
uniforms, such that each pixel in the extracted foreground images can be
encoded to represent one of these color tones” (¶ 181); (d) identifying a
Appeal 2018-008357
Application 11/909,080

6
player’s face region, with reference to color tone table 104a (¶ 468); and (e)
creating a minimum bounding box around any foreground areas where a
known skin color tone is found (¶ 331), do not correspond to teachings of
analyzing the color attributes of the “blobs,” connected in the previous step
through pixel connection processes, to assign that corresponding object an
identity group, as recited in independent claim 62. The cited teachings also
do not correspond to “analyz[ing] each blob, based on color attributes, to
identify the blob as corresponding to an identity group for players,” as
recited in independent claim 84.
Accordingly, we find that Aman does not teach all the limitations of
the invention recited in independent claims 62 and 84. We do not sustain the
Examiner’s § 102(e) rejection of claims 62, 64, 65, 84, 87–90, and 96–101.
§ 102(E) REJECTION OF CLAIMS 93, 79–81, AND 95 OVER AMAN
Independent claim 93 recites, in pertinent part, “matching player
portions detected by different video cameras based on their locations on the
playing field and on morphological knowledge of human body.”
Appellants assert that Aman does not teach using morphological
knowledge of the human body. Br. 24. Appellants’ position is that Aman
reads non-visible markings placed on key spots of a player’s uniform, such
as shoulders, elbows, waists, knees, and ankles. Id.; Aman ¶ 98. Appellants
reason that since these body parts are marked, Aman does not need to have
any morphological knowledge of the body. Id.
We are not persuaded by Appellants’ argument. The Examiner finds,
and we agree, that the Specification discloses that “[a]utomatic foreground
blobs segmentation into player’s head, torso, hands and legs is then
performed in step 734 by IPPIU 704 using pattern recognition algorithms
Appeal 2018-008357
Application 11/909,080

7
that are well known in the art,” citing a publication by J.M. Buades et al.
Spec. 14; Ans. 6–7. We agree with the Examiner that the cited section of the
Specification constitutes an admission that matching player portions
detected by different video cameras based on their locations on the playing
field and on morphological knowledge of the human body is well-known in
the art.
Further, the Examiner cites to several paragraphs of Aman as
suggesting that Aman employs morphological knowledge of the human
body. For example, Aman teaches that “after the location of player 10’s head
10sH is determined an oval 10sB will be optimally fit around the remaining
portion of the foreground object’s gradient outline.” Aman ¶ 350; Ans. 8–9.
This concept of an oval is pertinent because “using image analysis the body
oval 10sB can be analyzed to determine the orientation of the player’s 10
shoulders.” Id. Aman thus demonstrates the knowledge that the position of a
player’s shoulders can be determined from the location of a player’s head,
which requires morphological knowledge of the human body. Aman further
teaches that “oval 10sB will approach an elliptical shape as the player stands
upright”; this information can be used in combination with the fact that “the
front of the player’s 10 body, and therefore the ‘front’ of any representative
ellipse, is oriented in the direction of the front of the helmet sticker 9a.” Id.
The player’s neck is then disclosed to be “assumed to be a fixed distance
from the center of the helmet sticker 9a,” another indication that Aman
employs morphological knowledge of the human body, i.e., the relationship
of the neck to the head. Id.
We agree with the Examiner’s finding that Aman teaches “matching
player portions detected by different video cameras based on their locations
Appeal 2018-008357
Application 11/909,080

8
on the playing field and on morphological knowledge of human body,” as
independent claim 93 requires. We find that the Examiner did not err in
rejecting claim 93, or dependent claims 79–81 and 95, as being anticipated
by Aman. We sustain the Examiner’s § 102(e) rejection of claim 93, as well
as claims 79–81 and 95 dependent therefrom.
§ 102 REJECTION OF CLAIMS 62 AND 65 OVER FOOTE
The Examiner finds that Foote discloses “isolated pixels connected
into clusters by pixel connection processes to define a blob,” as independent
claim 62 requires. Ans. 9. The Examiner cites to columns 2 and 15 of Foote
in support of this finding. Final Act. 29. Foote discloses at column 2, lines
54–59 that “[t]he wide field sensor is utilized to locate people or regions of
interest . . . . By tracking shape, motion, color, and/or audio cues, the
location of people or other items of interest in the room or scene being
captured by the cameras can be estimated.” The cited portion of Foote
column 15 concerns estimating the location of an acoustic source by finding
peaks or corresponding features in acoustic signals from microphones.
Neither section discloses the claimed connection of pixels to form blobs.
We agree with Appellants that Foote does not teach all the limitations
of the invention recited in claims 62 and 65. Br. 25–26. We do not sustain
the Examiner’s § 102(e) rejection.
§ 103(A) REJECTION OVER AMAN AND UESAKI
Appellants argue that the Examiner erred in rejecting the claims over
the combination of Aman and Uesaki because (a) Aman fails to disclose
pixels connected into clusters by pixel connection processes to define a blob,
(b) Aman fails to disclose analyzing color attributes of each blob to assign
Appeal 2018-008357
Application 11/909,080

9
the corresponding object an identity group, and (c) Uesaki does not teach
any “creation of blobs, or operation on the blobs.” Br. 28–29.
As discussed supra with respect to the Examiner’s § 102(e) rejection
over Aman, we find that Aman teaches “isolated pixels connected into
clusters by pixel connection processes to define a blob.” Further, while we
find supra that Aman does not teach analyzing color attributes of blobs to
assign the corresponding object an identity group, we agree with the
Examiner that Aman does teach analyzing color attributes of objects. For
example, Aman teaches extraction of a full color image 10fc, and
determining whether foreground pixels correspond to non-facial color tones.
Aman ¶ 331. Aman teaches using the captured images of a selected player’s
uniform, preferably including their jersey number, to compare and pattern
match against a pre-known database thereby allowing for participant
identification. Aman ¶ 156. Aman teaches a table of pre-known color tones
established for all participant skin complexions as well as home and away
uniforms, implicitly suggesting that images will be analyzed for regions that
correspond to those pre-known color tones. Aman ¶ 181.
We agree with the Examiner’s finding that Uesaki teaches the claimed
analyzing color attributes of objects. Final Act. 36–37. “Because the color of
the uniform can be obtained in advance, it is possible to identify the team
with the color information.” Uesaki ¶ 69. “For example, the team can be
identified by matching the color information of the object with the color
information of the uniform.” Uesaki ¶ 70.
We do not agree with Appellants that there would be no motivation to
combine Aman and Uesaki, “except as ‘gleaned from applicant’s
disclosure.’” Br. 29 (emphasis omitted). Appellants’ contention is based on
Appeal 2018-008357
Application 11/909,080

10
the predicate that Uesaki teaches neither “creation of blobs” nor “operation
on the blobs.” Id. As discussed supra, however, we disagree with that
predicate. We agree with the Examiner’s rationale that it would have been
obvious to apply the practice of using colors to identify teams, taught by
Uesaki, to the invention of Aman, because both references are directed to
identifying and tracking objects, and the modification simply calls for the
application of a known technique to similar devices. Final Act. 37.
Appellants’ argument that the combination fails to teach the
limitations of independent claim 93, because Uesaki fails to teach “matching
player portions detected by different video cameras based on their locations
on the playing field and on morphological knowledge of human body,” is
not persuasive. Br. 29. As discussed supra with respect to the § 102(e)
rejection of claim 93 over Aman, we agree with the Examiner that Aman
teaches this limitation. Therefore, there is no need to look to Uesaki to
remedy any deficiency in the Aman reference.
We conclude that it would have been obvious to combine Aman with
Uesaki to achieve the claimed invention. We sustain the Examiner’s
rejection of claims 62, 64, 65, 79–81, 84, 87–90, 93, and 95–101 over Aman
and Uesaki.
FURTHER GROUNDS OF REJECTION
We sustain supra the rejection of all pending claims over the
combination of Aman and Uesaki. As a consequence, we need not reach the
question of whether the Examiner erred in rejecting all pending claims over
Aman and Huxley, or erred in rejecting all pending claims over Aman in
view of admitted prior art, or erred in rejecting claim 64 over Foote and
Aman.
Appeal 2018-008357
Application 11/909,080

11
CONCLUSIONS
1. Aman discloses connecting isolated pixels into clusters by pixel
connection processes to define a blob.
2. Aman does not disclose analyzing the color attributes of each blob
to assign the corresponding object an identity group.
3. Aman discloses matching player portions detected by different
video cameras based on their locations on the playing field and on
morphological knowledge of the human body.
4. The combination of Aman and Uesaki suggests analyzing each
blob, based on color attributes, to identify the blob as corresponding to an
identity group for players.
ORDER
The Examiner’s decision to reject claims 62, 64, 65, 79–81, 84, 87–
90, 93, and 95–101 is affirmed.
No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a).

AFFIRMED