APPLE INC.

Patent Trials and Appeals Board

Aug 10, 2021

2020002294 (P.T.A.B. Aug. 10, 2021)

UNITED STATES PATENT AND TRADEMARK OFFICE
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O. Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
15/750,046 02/02/2018 Eberhard Schmidt 27753-50158US 3346
149399 7590 08/10/2021
F&P, LLP-Arlen Team
2712 Augustine Drive
Suite 240
Santa Clara, CA 95054
EXAMINER
MARANDI, JAMES R
ART UNIT PAPER NUMBER
2421
NOTIFICATION DATE DELIVERY MODE
08/10/2021 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
mail@fernando-ip.com
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
_______________

BEFORE THE PATENT TRIAL AND APPEAL BOARD
_______________

Ex parte EBERHARD SCHMIDT, TOM SENGELAUB, and
CHRISTIAN VILLWOCK
_______________

Appeal 2020-002294
Application 15/750,046
Technology Center 2400
_______________

Before ROBERT E. NAPPI, JOHN A. JEFFERY, and
MATTHEW J. McNEILL, Administrative Patent Judges.

NAPPI, Administrative Patent Judge.

DECISION ON APPEAL
Appellant1 appeals under 35 U.S.C. § 134(a) from the Examiner’s
final rejection of claims 28 through 47. Claims 1 through 27 have been
canceled. We have jurisdiction under 35 U.S.C. § 6(b).
We AFFIRM.

1 We use the word Appellant to refer to “applicant” as defined in 37 C.F.R.
§ 1.42(a). According to Appellant, Apple Inc., is the real party-in-interest.
Appeal Br. 1.
Appeal 2020-002294
Application 15/750,046

2
INVENTION
The invention relates to a method for displaying a stream of images in
a display area of a display device, wherein a point of regard of a user
looking at the display area is determined and a first area within the display
area around the determined point of regard is determined. Spec. 1. The
rendered scene can then be presented as foveated rendering, wherein a point
of regard of a user or an area of interest is determined and a scene is
rendered with high resolution around this point of regard and with lower
resolution elsewhere. Spec. 1. Claim 28 is illustrative of the invention and
is reproduced below.
28. A method comprising:
determining gaze data of an eye of a user at a plurality of
times;
classifying the gaze data of the eye of the user at the
plurality of times as one of a plurality of eye movement types
comprising a static fixation, a moving fixation, and a saccade;
determining a point of regard of the user at a particular
time subsequent to the plurality of times based at least in part
on the classification; and
displaying an image at the particular time such that a first
part of the image in a first area around the point of regard is
displayed according to a first parameter value of an image
quality parameter and a second part of the first image in at least
one second area outside the first area is displayed according to a
second parameter value of the image quality parameter.

EXAMINER’S REJECTIONS2
The Examiner has rejected claims 28, 29, 31, 36 through 40, 44, and

2 Throughout this Decision we refer to the Appeal Brief filed August 8,
2019 (“Appeal Br.”); Reply Brief filed January 28, 2020 (“Reply Br.”);
Final Office Action mailed January 24, 2019 (“Final Act.”); and the
Examiner’s Answer mailed November 29, 2019 (“Ans.”).
Appeal 2020-002294
Application 15/750,046

3
45 under 35 U.S.C. § 103 as being unpatentable over Geisler (US 6,252,989
B1 issued June 26, 2001) and Smyth (US 5,726,916 issued March 10, 1998).
Final Act. 3–6.
The Examiner has rejected claim 30 under 35 U.S.C. § 103 as being
unpatentable over Geisler, Smyth, and Qin Cheng et al., Gaze Location
Prediction for Broadcast Football Video, 22 IEEE Transactions on Image
Processing 4918–29 (Dec. 2013). Final Act. 6–7.
The Examiner has rejected claims 32, 33, 41, 42, 46, and 47 under
35 U.S.C. § 103 as being unpatentable over Geisler, Smyth, and Hein (US
9,239,956 B2 issued January 19, 2016). Final Act. 7–9.
The Examiner has rejected claims 34, 35, and 43 under 35 U.S.C.
§ 103 as being unpatentable over Geisler, Smyth, and Eckles (US
2010/0054526 A1 published March 4, 2010). Final Act. 9–11.

ANALYSIS
We have reviewed Appellant’s arguments in the Briefs, the
Examiner’s rejections, and the Examiner’s response to Appellant’s
arguments. Appellant’s arguments have not persuaded us of error in the
Examiner’s rejection of claims 28 through 47.

Independent claims 28, 38, and 44
With respect to independent claims 28, 38, and 44, Appellant argues
that the combination of Geisler and Smyth do not teach determining the
point of regard at a particular time subsequent to the plurality of times based
Appeal 2020-002294
Application 15/750,046

4
upon the classification as recited in independent claim 28. Appeal Br. 4–6.3
Appellant argues that Geisler teaches an image is foveated based upon a
current point of gaze not a predicted point of gaze. Appeal Br. 4.
Additionally, Appellant argues that there is no motivation to modify Geisler
to include a predicted point of regard as it would change the principle of
operation of Geisler. Appeal Br. 4. Further, Appellant argues that Smyth
does not make up for this deficiency in the teaching of Geisler. Appeal Br.
5. Appellant argues that Smyth teaches monitoring gaze data “to ‘determine
the start, continuation, and termination of each gaze point,’ not to predict a
‘point of regard of the user at a particular time subsequent to the plurality of
times’ based on the classification as recited in claim 28.” Appeal Br. 5.
The Examiner, in the rejection of claim 28, finds that Geisler teaches
the claim limitation of “determining a point of regard of the user at a
particular time subsequent to the plurality of times.” Final Act. 3 (citing
Geisler, col. 2, l. 21–col. 3, l. 13, col 6. l. 62– col. 7, l. 55). The Examiner
finds that Smyth teaches classifying the gaze data of the user and
determining the point of regard based in part on the classification. Final Act.
4 (citing Smyth, Figs. 1, 5, 6, col. 11, l. 65– col. 15, l. 53). In response to
Appellant’s arguments, the Examiner states that claim 28 does not recite
predicting a point of gaze as argued by Appellant and that Geisler teaches
determining the gaze of the user as a particular point in time subsequent to
the plurality of points of time as claimed. Ans. 12 (citing Geisler, col. 2, l.
21–col. 3, l. 13, col 6. l. 62– col. 7, l. 55). The Examiner finds that Geiser’s
system of determining eye position is an ongoing process and it is repeated

3 Appellant’s arguments group all the independent claim together. Appeal
Br. 6
Appeal 2020-002294
Application 15/750,046

5
in a loop. Ans. 13, 14 (citing Geiser Fig. 2, and col. 2, ll. 42–61, col. 1, ll.
37–62). Further, the Examiner finds that “as time progresses (frames of the
video/advancing images) the point of regard of the user is determined at a
particular time (of a frame/image) subsequent to the plurality of times
(previous times of previous frames/images).” Ans. 13
Appellant’s arguments have not persuaded us of error. Initially, we
concur with the Examiner that the claims do not recite predicting a future
point of gaze or point of regard. Rather, claim 28 merely requires that point
of regard is determined subsequent to at least a second time the gaze data is
determined. Further, contrary to Appellant’s arguments, Geiser is not
limited to real time processing of eye position, but does discuss using prior
measures of eye position to predict future eye position. For example,
Geisler discusses monitoring eye position of a viewer in a prior viewing and
using the measured eye position to create foveated versions for every frame
of the video for future viewing. Geisler, col. 8, ll. 16–30. We also note that
Geisler in the real time embodiment discusses using measured eye position
x0, y0 (at time 0) in part to make the determination of the current gauze point
(at time m); i.e., the prior determinations of gaze points are used as part of
the calculation to determine the current determination of gaze points.
Geisler, col. 7, ll. 10–20 (equation 6). Thus, we are not persuaded by
Appellant’s arguments that Geisler does not teach an image is foveated
based upon a predicted point of gaze or that modifying Geisler to include a
predicted point of regard would change a principle of operation.4 Further,

4 We additionally note that Smyth identifies what is useful in computer
generated imagery, and thus provides further suggestion to combine Smyth
with the systems such as those disclosed in Geisler. (Smyth, col. 1, l. 35)
Appeal 2020-002294
Application 15/750,046

6
we concur with the Examiner that Geisler teaches determining the point of
regard subsequent to at least the second time the gaze data is determined.
See Ans. 13 (“[A]s time progresses (frames of the video/advancing
images)[,] the point of regard of the user is determined at a particular time
(of a frame/image) subsequent to the plurality of times (previous times of
previous frames/images.”). As such, we are similarly not persuaded by
Appellant’s argument that Smyth does not make up for Geisler’s failure to
teach this feature.5 Accordingly we sustain the Examiner’s rejection of
independent claim 28.

CONCLUSION

5 We additionally note that Appellant acknowledges that Smyth predicts the
end point of a saccadic movement positon of gaze at end of movement.
Appeal Br. 5 (citing Smyth, col. 13, ll. 44–46). As the saccadic motion is
observed and classified prior to the calculation to predict the end point (point
of regard), Smyth predicts a gaze based upon a category. While, as
Appellant argues, Smyth includes a teaching that the prediction is checked
and can be reclassified, Smyth nonetheless teaches making the prediction
based upon the classification and prior to the reclassification. See Appeal
Br. 5 (citing Smyth, col. 14, l. 1). Thus, we consider Smyth to teach the
claimed determining a point of regard at a time subsequent to the
classification.
Appeal 2020-002294
Application 15/750,046

7
We affirm the Examiner’s rejections of claims 28 through 47.

Claims
Rejected
35 U. S. C.
§
Reference(s)/Basis Affirmed Reversed
28, 29, 31,
36–40, 44,
45
103 Geisler, Smyth 28, 29, 31,
36–40, 44,
45

30 103 Geisler, Smyth,
Cheng
30
32, 33, 41,
42, 46, 47
103 Geisler, Smyth, Hein 32, 33, 41,
42, 46, 47

34, 35, 43 103 Geisler, Smyth,
Eckles
34, 35, 43
Overall
Outcome
28–47

TIME PERIOD FOR RESPONSE

No time period for taking any subsequent action in connection with
this Appeal may be extended under 37 C.F.R. § 1.136(a).
See 37 C.F.R. § 1.136(a)(1)(iv).

AFFIRMED