Ex Parte Jahnke et alDownload PDFPatent Trial and Appeal BoardOct 31, 201614023942 (P.T.A.B. Oct. 31, 2016) Copy Citation UNITED STA TES p A TENT AND TRADEMARK OFFICE APPLICATION NO. FILING DATE FIRST NAMED INVENTOR 14/023,942 09/11/2013 Nathan A. JAHNKE 23505 7590 11/02/2016 CONLEY ROSE, P.C. Jonathan M. Harris 1001 Mckinney Suite 1800 HOUSTON, TX 77002-6417 UNITED STATES DEPARTMENT OF COMMERCE United States Patent and Trademark Office Address: COMMISSIONER FOR PATENTS P.O. Box 1450 Alexandria, Virginia 22313-1450 www .uspto.gov ATTORNEY DOCKET NO. CONFIRMATION NO. 7090-00900 1292 EXAMINER HARRIS, DOROTHY H ART UNIT PAPER NUMBER 2625 NOTIFICATION DATE DELIVERY MODE 11/02/2016 ELECTRONIC Please find below and/or attached an Office communication concerning this application or proceeding. The time period for reply, if any, is set in the attached communication. Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the following e-mail address( es): pathou@conleyrose.com PTOL-90A (Rev. 04/07) UNITED STATES PATENT AND TRADEMARK OFFICE BEFORE THE PATENT TRIAL AND APPEAL BOARD ExparteNATHAN A. JAHNKE andJ. BRIAN STILL Appeal2015-006242 Application 14/023,942 Technology Center 2600 Before: ELENI MANTIS MERCADER, NORMAN H. BEAMER, and ALEX S. YAP, Administrative Patent Judges. MANTIS MERCADER, Administrative Patent Judge. DECISION ON APPEAL Appeal2015-006242 Application 14/023,942 STATEMENT OF CASE Appellants appeal under 35 U.S.C. § 134 from a rejection of claims 2- 10 and 12-18. Claims 1, 11 and 19-24 are cancelled. We have jurisdiction under 35 U.S.C. § 6(b ). We affirm. THE INVENTION The claimed invention is directed to correlating pupil position to gaze location within a scene. Abstract. Claims 2 and 5, reproduced below, are illustrative of the claimed subject matter: 2. A method comprising: receiving, by a first computer system, a first video stream depicting an eye of a user, the first video stream comprising a first plurality of frames; receiving, by the first computer system, a second video stream depicting a scene in front of the user, the second video stream comprising a second plurality of frames; determining, by the first computer system, pupil position within the first plurality of frames; calculating, by the first computer system, gaze location in the second plurality of frames based on pupil position within the first plurality of frames; and sending an indication of the gaze location to a second computer system, the second computer system distinct from the first computer system, and the sending in real-time with creation of the first video stream; calibrating a relationship between pupil position within the first plurality of frames and gaze location in the second plurality of frames, the calibrating prior to the calculating and sending, and the calibrating by: displaying a plurality of calibration features in the scene in front of the user; 2 Appeal2015-006242 Application 14/023,942 determining, by the first computer system, location of the calibration features within the second plurality of frames; relating, by the first computer system, pupil position in the first plurality of frames to location of the calibration features in the second plurality of frames; and thereby creating, by the first computer system, a homography that relates pupil position in the first plurality of frames to gaze location in the second plurality of frames. 5. The method of claim 2 wherein relating further comprises: clustering, by the first computer system, indications of pupil position derived from the first plurality of frames, the clustering creates a plurality of pupil clusters; clustering, by the first computer system, indications of location of the calibration features in the second plurality of frames, the clustering indications of location creates a plurality of feature clusters; and correlating the pupil clusters to the feature clusters. REFERENCES The prior art relied upon by the Examiner in rejecting the claims on appeal is: Smyth Durnell Fengels us 5,583,795 US 2004/0196433 Al US 2006/0209013 Al REJECTIONS The Examiner made the following rejections: Dec. 10, 1996 Oct. 7, 2004 Sept. 21, 2006 Claims 2--4 stand rejected under 35 U.S.C § 103(a) as being unpatentable over Durnell in view of Smyth. 3 Appeal2015-006242 Application 14/023,942 Claim 5-10 and 12-18 stand rejected under 35 U.S.C § 103(a) as being unpatentable over Durnell in view of Smyth, and further in view of Fengels. ISSUES The pivotal issues are whether the Examiner erred in finding that the combination of Durnell and Smyth teaches the limitations of: "determining, by the first computer system, location of the calibration features within the second plurality of frames" as recited in claim 2; and "clustering, by the first computer system, indications of location of the calibration features in the second plurality of frames, the clustering indications of location creates a plurality of feature clusters," as recited in claim 5. ANALYSIS We adopt the Examiner's findings in the Final Action and the Answer and we add the following primarily for emphasis. Claims 2 and 4 Appellants argue that Durnell does not teach a calibrating step of "determining, by the first computer system, location of the calibration features within the second plurality of frames", as required by claim 2 (App. Br. 19). According to Appellants, claim 2 requires the computer system to determine the location of the calibration features within the frames created by the scene camera (App. Br. 19). Appellants argue that the description of the "prior calibration process" of Durnell only indicates that the user fixates on specified objects (App. Br. 20). According to Appellants, there is no 4 Appeal2015-006242 Application 14/023,942 express teaching that the "prior calibration process" determines the location of the specified objects in the frames of the scene camera (App. Br. 20). More specifically, Appellants allege that in Durnell the relationship of the pupil and spots is tied to a gaze location, but there is no need to also search the frames of the scene camera for the object on which the user fixated (App. Br. 20). According to Appellants, finding the location scene markers is not part of the calibration process to relate pupil location with the scene (App. Br. 21 ). Appellants assert that there is no indication in Durnell that the user's eyes even need be open as part of the second calibration, much less a need to fixate on the location markers (App. Br. 21 ). We do not agree with Appellants' arguments. The Examiner finds, and we agree, that Durnell teaches gain and offset factors determined by a prior calibration process in which the user fixates on specified objects within the scene image so that the direction in the scene image is related to the differences in the horizontal and vertical position of the center of the reference spot and the center of the pupil within the eye image (Ans. 2; Darnell, para. 39). The Examiner further finds that Durnell indicates that Figure 5 shows an eye tracking system for use as a workstation interface tool wherein signals representative of the scene image are supplied by the frame grabber module 8 to a module 7 4 for locating the positions of scene markers, such as infrared light-emitting diodes in the image (Ans. 2). The positions of the scene markers are then used in the decalibration process to calculate the point of regard on the workstation monitor 76 (Ans. 2; Darnell, para. 41). Output signals from the workstation may be supplied to a log file 78 storing such points of regard, and to provide feedback 80 to the workstation (Ans. 2). In this process the scene image is analyzed to find the marker spots and calculate the scene camera position and orientation by solving a set of 5 Appeal2015-006242 Application 14/023,942 transformation equations that use coefficients that depend upon the scene camera field of view and the marker geometry (Ans. 2-3). The coefficients are determined by a prior calibration with the scene camera held in known positions and orientations and the spot locations determined within the scene camera image (Ans. 3). The point of regard of the eye on the workstation monitor is determined by combining the data for the eye direction relative to the scene camera with the data for the scene camera relative to the workstation (Ans. 3). This information provides the feedback 80 to the workstation so that the workstation knows where on the monitor the subject is looking, which will enable, for example, eye control of a mouse pointer and/or icon selection (Ans. 3). Thus, we agree with the Examiner that Durnell teaches "determining, by the first computer system, location of the calibration features within the second plurality of frames" as recited in claim 2. Accordingly, we affirm the Examiner's rejection of claim 2 and the rejection of claim 4, which is not argued separately (see App. Br. 12). Claim 3 Appellants argue that attempts to rely on Smyth's video display 2a for the "revealing of the plurality of calibration features not on a computer monitor" is misplaced because while it is true that Smyth uses the flashes of light associated with refresh times of the display 2a as part of creating and detecting Purkinje reflections, the video display 2a is still part of the virtual reality display (App. Br. 23). We do not agree with Appellants' arguments. We agree with the Examiner that Smyth teaches that the user may be viewing external real or virtual images (Ans. 7; Smyth, col. 12, 11. 33-36 and col. 22, 11. 13-37). Accordingly, we affirm the Examiner's rejection of claim 3. 6 Appeal2015-006242 Application 14/023,942 Claim 5, 6 and 14 Appellants argue that even assuming hypothetically that the blob classification of Durnell is with respect to pupil centers, which Appellants dispute, Durnell still fails to teach clustering with respect to calibration features in the second plurality of frames (App. Br. 26). Appellants further argue that F engels fails to teach why one of ordinary skill would be motivated to change the "prior calibration process" of Durnell, which apparently does not need or require "determining ... location of the calibration features with the second plurality of frames" as required by claim 2, much less "clustering, by the first computer system, indications of location of the calibration features in the second plurality of frames, the clustering indications of location creates a plurality of feature clusters" (App. Br. 29). Appellants conclude, Durnell, Smyth, and Fengels fail to teach, "clustering ... indications of location of the calibration features in the second plurality of frames, the clustering indications of location creates a plurality of feature clusters" as recited in claim 5. We do not agree. The Examiner relied on the calibration of the pupil center position as taught by Durnell wherein blobs calibration is used to identify the pupil center location and its alignment with a scene to eliminate blobs which are outliers (i.e., clusters which are outliers) (see Fig. 8, and respective disclosure) with the gaze clustering analysis of a plurality of markers in a visual scene as taught by Smyth (Final Act. 11-12; 14--15). Thus, the combination would teach both pupil and gaze corrections with respect to an image. The Examiner relies on Fengels for the teaching of correct alignment of the eye of the user with respect to a plurality of images/scenes and extends the teachings of Durnell and Smyth to a plurality of images (see Final Act. 15-16; paras. Smyth, 127-130 and Abstract). 7 Appeal2015-006242 Application 14/023,942 Accordingly, we also affirm the Examiner's rejection of claim 5 and the rejection of claims 6 and 14, which are not argued separately. Claims 7-10 and 12-18 Appellants substantially rely on the same arguments as those raised with respect to claims 2 and 5, for the Examiner's rejection of claims 7-10 and 12-18 (App. Br. 29-38). Accordingly, we also affirm the Examiner's rejection of claims 7-10 and 12-18. CONCLUSIONS The Examiner did not err in finding that the combination of Durnell and Smyth teaches the limitations of: "determining, by the first computer system, location of the calibration features within the second plurality of frames" as recited in claim 2; and "clustering, by the first computer system, indications of location of the calibration features in the second plurality of frames, the clustering indications of location creates a plurality of feature clusters," as recited in claim 5. DECISION For the above reasons, we affirm the Examiner's decision to reject claims 2-10 and 12-18. No time period for taking any subsequent action in connection with this appeal may be extended under 37 C.F.R. § 1.136(a)(l )(iv) . AFFIRMED 8 Copy with citationCopy as parenthetical citation