Ex Parte Jahnke et al

Patent Trial and Appeal Board

Oct 31, 2016

14023942 (P.T.A.B. Oct. 31, 2016)

UNITED STA TES p A TENT AND TRADEMARK OFFICE
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR
14/023,942 09/11/2013 Nathan A. JAHNKE
23505 7590 11/02/2016
CONLEY ROSE, P.C.
Jonathan M. Harris
1001 Mckinney
Suite 1800
HOUSTON, TX 77002-6417
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O. Box 1450
Alexandria, Virginia 22313-1450
www .uspto.gov
ATTORNEY DOCKET NO. CONFIRMATION NO.
7090-00900 1292
EXAMINER
HARRIS, DOROTHY H
ART UNIT PAPER NUMBER
2625
NOTIFICATION DATE DELIVERY MODE
11/02/2016 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address( es):
pathou@conleyrose.com
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
BEFORE THE PATENT TRIAL AND APPEAL BOARD
ExparteNATHAN A. JAHNKE andJ. BRIAN STILL
Appeal2015-006242
Application 14/023,942
Technology Center 2600
Before: ELENI MANTIS MERCADER, NORMAN H. BEAMER, and
ALEX S. YAP, Administrative Patent Judges.
MANTIS MERCADER, Administrative Patent Judge.
DECISION ON APPEAL
Appeal2015-006242
Application 14/023,942
STATEMENT OF CASE
Appellants appeal under 35 U.S.C. § 134 from a rejection of claims 2-
10 and 12-18. Claims 1, 11 and 19-24 are cancelled. We have jurisdiction
under 35 U.S.C. § 6(b ).
We affirm.
THE INVENTION
The claimed invention is directed to correlating pupil position to gaze
location within a scene. Abstract.
Claims 2 and 5, reproduced below, are illustrative of the claimed
subject matter:
2. A method comprising:
receiving, by a first computer system, a first video stream
depicting an eye of a user, the first video stream comprising a
first plurality of frames;
receiving, by the first computer system, a second video
stream depicting a scene in front of the user, the second video
stream comprising a second plurality of frames;
determining, by the first computer system, pupil position
within the first plurality of frames;
calculating, by the first computer system, gaze location in
the second plurality of frames based on pupil position within the
first plurality of frames; and
sending an indication of the gaze location to a second
computer system, the second computer system distinct from the
first computer system, and the sending in real-time with creation
of the first video stream;
calibrating a relationship between pupil position within the
first plurality of frames and gaze location in the second plurality
of frames, the calibrating prior to the calculating and sending,
and the calibrating by:
displaying a plurality of calibration features in the scene
in front of the user;
2
Appeal2015-006242
Application 14/023,942
determining, by the first computer system, location of the
calibration features within the second plurality of frames;
relating, by the first computer system, pupil position in the
first plurality of frames to location of the calibration features in
the second plurality of frames; and thereby
creating, by the first computer system, a homography that
relates pupil position in the first plurality of frames to gaze
location in the second plurality of frames.
5. The method of claim 2 wherein relating further
comprises:
clustering, by the first computer system, indications of
pupil position derived from
the first plurality of frames, the clustering creates a
plurality of pupil clusters;
clustering, by the first computer system, indications of
location of the calibration features in the second plurality of
frames, the clustering indications of location creates a plurality
of feature clusters; and
correlating the pupil clusters to the feature clusters.
REFERENCES
The prior art relied upon by the Examiner in rejecting the claims on
appeal is:
Smyth
Durnell
Fengels
us 5,583,795
US 2004/0196433 Al
US 2006/0209013 Al
REJECTIONS
The Examiner made the following rejections:
Dec. 10, 1996
Oct. 7, 2004
Sept. 21, 2006
Claims 2--4 stand rejected under 35 U.S.C § 103(a) as being
unpatentable over Durnell in view of Smyth.
3
Appeal2015-006242
Application 14/023,942
Claim 5-10 and 12-18 stand rejected under 35 U.S.C § 103(a) as
being unpatentable over Durnell in view of Smyth, and further in view of
Fengels.
ISSUES
The pivotal issues are whether the Examiner erred in finding that the
combination of Durnell and Smyth teaches the limitations of:
"determining, by the first computer system, location of the calibration
features within the second plurality of frames" as recited in claim 2; and
"clustering, by the first computer system, indications of location of the
calibration features in the second plurality of frames, the clustering
indications of location creates a plurality of feature clusters,"
as recited in claim 5.
ANALYSIS
We adopt the Examiner's findings in the Final Action and the Answer
and we add the following primarily for emphasis.
Claims 2 and 4
Appellants argue that Durnell does not teach a calibrating step of
"determining, by the first computer system, location of the calibration
features within the second plurality of frames", as required by claim 2 (App.
Br. 19). According to Appellants, claim 2 requires the computer system to
determine the location of the calibration features within the frames created
by the scene camera (App. Br. 19). Appellants argue that the description of
the "prior calibration process" of Durnell only indicates that the user fixates
on specified objects (App. Br. 20). According to Appellants, there is no
4
Appeal2015-006242
Application 14/023,942
express teaching that the "prior calibration process" determines the location
of the specified objects in the frames of the scene camera (App. Br. 20).
More specifically, Appellants allege that in Durnell the relationship of the
pupil and spots is tied to a gaze location, but there is no need to also search
the frames of the scene camera for the object on which the user fixated (App.
Br. 20). According to Appellants, finding the location scene markers is not
part of the calibration process to relate pupil location with the scene (App.
Br. 21 ). Appellants assert that there is no indication in Durnell that the
user's eyes even need be open as part of the second calibration, much less a
need to fixate on the location markers (App. Br. 21 ).
We do not agree with Appellants' arguments. The Examiner finds,
and we agree, that Durnell teaches gain and offset factors determined by a
prior calibration process in which the user fixates on specified objects within
the scene image so that the direction in the scene image is related to the
differences in the horizontal and vertical position of the center of the
reference spot and the center of the pupil within the eye image (Ans. 2;
Darnell, para. 39). The Examiner further finds that Durnell indicates that
Figure 5 shows an eye tracking system for use as a workstation interface tool
wherein signals representative of the scene image are supplied by the frame
grabber module 8 to a module 7 4 for locating the positions of scene markers,
such as infrared light-emitting diodes in the image (Ans. 2). The positions
of the scene markers are then used in the decalibration process to calculate
the point of regard on the workstation monitor 76 (Ans. 2; Darnell, para. 41).
Output signals from the workstation may be supplied to a log file 78 storing
such points of regard, and to provide feedback 80 to the workstation (Ans.
2). In this process the scene image is analyzed to find the marker spots and
calculate the scene camera position and orientation by solving a set of
5
Appeal2015-006242
Application 14/023,942
transformation equations that use coefficients that depend upon the scene
camera field of view and the marker geometry (Ans. 2-3). The coefficients
are determined by a prior calibration with the scene camera held in known
positions and orientations and the spot locations determined within the scene
camera image (Ans. 3). The point of regard of the eye on the workstation
monitor is determined by combining the data for the eye direction relative to
the scene camera with the data for the scene camera relative to the
workstation (Ans. 3). This information provides the feedback 80 to the
workstation so that the workstation knows where on the monitor the subject
is looking, which will enable, for example, eye control of a mouse pointer
and/or icon selection (Ans. 3). Thus, we agree with the Examiner that
Durnell teaches "determining, by the first computer system, location of the
calibration features within the second plurality of frames" as recited in claim
2. Accordingly, we affirm the Examiner's rejection of claim 2 and the
rejection of claim 4, which is not argued separately (see App. Br. 12).
Claim 3
Appellants argue that attempts to rely on Smyth's video display 2a for
the "revealing of the plurality of calibration features not on a computer
monitor" is misplaced because while it is true that Smyth uses the flashes of
light associated with refresh times of the display 2a as part of creating and
detecting Purkinje reflections, the video display 2a is still part of the virtual
reality display (App. Br. 23).
We do not agree with Appellants' arguments. We agree with the
Examiner that Smyth teaches that the user may be viewing external real or
virtual images (Ans. 7; Smyth, col. 12, 11. 33-36 and col. 22, 11. 13-37).
Accordingly, we affirm the Examiner's rejection of claim 3.
6
Appeal2015-006242
Application 14/023,942
Claim 5, 6 and 14
Appellants argue that even assuming hypothetically that the blob
classification of Durnell is with respect to pupil centers, which Appellants
dispute, Durnell still fails to teach clustering with respect to calibration
features in the second plurality of frames (App. Br. 26). Appellants further
argue that F engels fails to teach why one of ordinary skill would be
motivated to change the "prior calibration process" of Durnell, which
apparently does not need or require "determining ... location of the
calibration features with the second plurality of frames" as required by claim
2, much less "clustering, by the first computer system, indications of
location of the calibration features in the second plurality of frames, the
clustering indications of location creates a plurality of feature clusters"
(App. Br. 29). Appellants conclude, Durnell, Smyth, and Fengels fail to
teach, "clustering ... indications of location of the calibration features in the
second plurality of frames, the clustering indications of location creates a
plurality of feature clusters" as recited in claim 5.
We do not agree. The Examiner relied on the calibration of the pupil
center position as taught by Durnell wherein blobs calibration is used to
identify the pupil center location and its alignment with a scene to eliminate
blobs which are outliers (i.e., clusters which are outliers) (see Fig. 8, and
respective disclosure) with the gaze clustering analysis of a plurality of
markers in a visual scene as taught by Smyth (Final Act. 11-12; 14--15).
Thus, the combination would teach both pupil and gaze corrections with
respect to an image. The Examiner relies on Fengels for the teaching of
correct alignment of the eye of the user with respect to a plurality of
images/scenes and extends the teachings of Durnell and Smyth to a plurality
of images (see Final Act. 15-16; paras. Smyth, 127-130 and Abstract).
7
Appeal2015-006242
Application 14/023,942
Accordingly, we also affirm the Examiner's rejection of claim 5 and
the rejection of claims 6 and 14, which are not argued separately.
Claims 7-10 and 12-18
Appellants substantially rely on the same arguments as those raised
with respect to claims 2 and 5, for the Examiner's rejection of claims 7-10
and 12-18 (App. Br. 29-38).
Accordingly, we also affirm the Examiner's rejection of claims 7-10
and 12-18.
CONCLUSIONS
The Examiner did not err in finding that the combination of Durnell
and Smyth teaches the limitations of:
"determining, by the first computer system, location of the calibration
features within the second plurality of frames" as recited in claim 2; and
"clustering, by the first computer system, indications of location of the
calibration features in the second plurality of frames, the clustering
indications of location creates a plurality of feature clusters,"
as recited in claim 5.
DECISION
For the above reasons, we affirm the Examiner's decision to reject
claims 2-10 and 12-18.
No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a)(l )(iv) .
AFFIRMED
8