TVision Insights, Inc.

Patent Trials and Appeals BoardDec 15, 2021

2020006124 (P.T.A.B. Dec. 15, 2021)

UNITED STATES PATENT AND TRADEMARK OFFICE UNITED STATES DEPARTMENT OF COMMERCE United States Patent and Trademark Office Address: COMMISSIONER FOR PATENTS P.O. Box 1450 Alexandria, Virginia 22313-1450 www.uspto.gov APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO. 15/702,229 09/12/2017 Inderbir Sidhu TVIS P001 2793 15360 7590 12/15/2021 Michael C. Martensen, P.C. 30 E. Kiowa St., Suite 101 Colorado Springs, CO 80903 EXAMINER LE, RONG ART UNIT PAPER NUMBER 2421 NOTIFICATION DATE DELIVERY MODE 12/15/2021 ELECTRONIC Please find below and/or attached an Office communication concerning this application or proceeding. The time period for reply, if any, is set in the attached communication. Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the following e-mail address(es): docket@martensenip.com julie@martensenip.com PTOL-90A (Rev. 04/07) UNITED STATES PATENT AND TRADEMARK OFFICE ____________ BEFORE THE PATENT TRIAL AND APPEAL BOARD ____________ Ex parte INDERBIR SIDHU, YANFENG LIU, and YUN FU ____________ Appeal 2020-006124 Application 15/702,229 Technology Center 2400 ____________ Before RICHARD M. LEBOVITZ, JENNIFER S. BISK, and JAMES W. DEJMEK, Administrative Patent Judges. LEBOVITZ, Administrative Patent Judge. DECISION ON APPEAL The Examiner rejected claims 1–5, 7, 8, 15–18, and 25–36 under 35 U.S.C. § 103 as obvious. Pursuant to 35 U.S.C. § 134(a), Appellant1 appeals from the Examiner’s decision to reject the claims. We have jurisdiction under 35 U.S.C. § 6(b). We AFFIRM, but designate the affirmance as a new ground of rejection under 37 C.F.R. § 41.50(b). 1 We use the word “Appellant” to refer to “applicant” as defined in 37 C.F.R. § 1.42 (2019). Appellant identifies the real party in interest as TVision Insights, Inc. Appeal Br. 2. Appeal 2020-006124 Application 15/702,229 2 STATEMENT OF THE CASE The Examiner rejected claims 1–5, 7, 8, 15–18, and 25–36 in the Final Office Action as follows: 1. Claims 1–3, 7, 8, 16–18, 32, 34, and 35 under 35 U.S.C. § 103 as obvious in view of Garza et al. (US 2014/0317646 A1, published Oct. 23, 2014) (“Garza”), Deephanphongs et al. (US 2014/0344017 A1, published Nov. 20, 2014) (“Deephanphongs”), and Aguera-Arcas (US 2018/0005062 A1, published Jan. 4, 2018) (“Aguera-Arcas”). Final Act. 3. 2. Claim 4 under 35 U.S.C. § 103 as obvious in view of Garza, Deephanphongs, Aguera-Arcas, and Spears et al. (US 2016/0080448 A1, published Mar. 17, 2016) (“Spears”). Final Act. 9. 3. Claim 5 under 35 U.S.C. § 103 as obvious in view of Garza, Deephanphongs, Aguera-Arcas, and Xie et al. (US 2008/0097749 A1, published Apr. 24, 2008) (“Xie”). Final Act. 10 4. Claim 15 under 35 U.S.C. § 103 as obvious in view of Garza, Deephanphongs, Aguera-Arcas, and Osawa (US 2003/0142745 A1, published July 31, 2003) (“Osawa”). Final Act. 11. 5. Claims 25–28 and 30 under 35 U.S.C. § 103 as obvious in view of Garza, Hicks (US 2016/0066034 A1, published Mar. 3, 2016) (“Hicks”), Deephanphongs, and Aguera-Arcas. Final Act. 12. 6. Claim 29 under 35 U.S.C. § 103 as obvious in view of Garza, Hicks, Deephanphongs, Aguera-Arcas, and Xie. Final Act. 19. 7. Claim 31 under 35 U.S.C. § 103 as obvious in view of Garza, Terrazas (US 2014/0259034 A1, published Sept. 11, 2014) (“Terrazas”), Deephanphongs, and Aguera-Arcas. Final Act. 20. Appeal 2020-006124 Application 15/702,229 3 8. Claims 32 and 34 under 35 U.S.C. § 103 as obvious in view of Garza, Deephanphongs, Aguera-Arcas, and Terrazas. Final Act. 25. 9. Claim 33 under 35 U.S.C. § 103 as obvious in view of Garza, Deephanphongs, Aguera-Arcas, and Peterson et al. (US 2014/0172579 A1, published June 19, 2014) (“Peterson”). Final Act. 28. 10. Claim 36 under 35 U.S.C. § 103 as obvious in view of Garza, Deephanphongs, Aguera-Arcas, and Hyatt (US 2010/0066485 A1, published Mar. 18, 2010) (“Hyatt”). Final Act. 28. There are three independent claims on the appeal, claims 1, 25, and 31. Claim 1 is representative and copied below (annotated with bracketed numbers for reference to the limitations in the claim): 1. A method of quantifying viewer engagement with a video shown on a display, the method comprising: [1] acquiring, with at least one camera, image data of a viewing area in front of the display while the video is being shown on the display; [2] acquiring, with a microphone, audio data representing a soundtrack of the video emitted by a speaker coupled to the display; [3] storing the image data and audio data in a memory buffer; [4] determining, with a processor operably coupled to the memory and without any connection to the display, a set-top box, or a cable connection, an audio fingerprint of the video based at least in part on the audio data; [5] transmitting the audio fingerprint to an application programming interface that provides an identity of the video based on the audio fingerprint; [6] estimating, with a classifier implemented by the processor and based at least in part on the image data, a first number of people present in the viewing area while the video is being shown on the display and a second number of people engaged with the video in the viewing area; Appeal 2020-006124 Application 15/702,229 4 [7] overwriting the image data and the audio data stored in the buffer so as to preserve privacy of the people in the viewing area; and [8] transmitting, by the processor, the first number of people present in the viewing area while the video is being shown on the display and the second number of people engaged with the video in the viewing area to a remote server without transmitting the image data or the audio data to the remote server. Appellant argues each of the independent claims separately under a separate heading. We use the same headings as Appellant does. CLAIM 1 Rejection Claim 1 is directed to a method of “quantifying viewer engagement with a video shown on a display.” Image and audio data is acquired in steps [1] and [2], and stored in a memory buffer in step [3]. In step [4], “an audio fingerprint of the video based at least in part on the audio data” is determined “with a processor operably coupled to the memory and without any connection to the display, a set-top box, or a cable connection.” In step [5], the audio fingerprint is transmitted to an application programming interface that provides an identity of the video based on the audio fingerprint. Step [6] is estimating the number of people in the viewing area while the video is being shown and the number of people “engaged with the video in the viewing area.” The image and audio data is overwritten in step [7] “to preserve privacy of the people in the viewing area.” In the last step [8] recited in the claim, the number of people estimated in step [6] is transmitted “to a remote server without transmitting the image data or the audio data to the remote server.” Appeal 2020-006124 Application 15/702,229 5 The Examiner found that Garza describes steps [1]–[3] and [6] of the claim, but not steps [4], [5], [7], or [8]. Final Act. 3–4, 5. To meet these deficiencies in Garza, the Examiner further cited Deephanphongs and Aguera-Arcas. The Examiner found that Deephanphongs discloses an audience attendance monitoring system like Garza in which audio data is received at a remote server and analyzed by determining an audio fingerprint of it, which the Examiner found meets step [4] of claim 1. Final Act. 4–5. The Examiner further found that Deephanphongs discloses that the fingerprint is transmitted to a client device where the identity of the video is determined as in step [5] of the claim. Id. at 5. The Examiner found it would have been obvious to one of ordinary skill in the art to apply Deephanphongs to Garza “to provide accurate user viewership information without relying on viewer participation in data gathering.” Id. For step [7] of the claim of overwriting the stored audio and image data, the Examiner cited the teaching in Aguera-Arcas where a user can request privacy by removing personal identifiable information from storage. Final Act. 5–6. Discussion The first argument made by Appellant is that Deephanphongs does not describe step [4] of claim 1 in which “an audio fingerprint of the video based at least in part on the audio data” is determined “with a processor operably coupled to the memory and without any connection to the display, a set-top box, or a cable connection.” Appeal Br. 13–14. Appellant argues that the client system cited by the Examiner for its function in determining the audio fingerprint is connected to a display and therefore does not meet the corresponding exclusion of step [4] of claim 1. Id. at 16. Appellant also Appeal 2020-006124 Application 15/702,229 6 argues that the “remote server” of Deephanphongs, also cited by the Examiner to meet limitation [4], is deficient because it is not described as receiving audio data or processing it to determine an audio fingerprint. Id. at 16–17. We first turn to the disclosure in Deephanphongs cited by the Examiner as meeting step [4] of claim 1. The Examiner found that the disclosure at paragraphs 117 and 163 of Deephanphongs describes determining the audio fingerprint without connection to display, a set-top box, or a cable connection. Ans. 4–5. Referencing Figure 10, client system 1002, paragraph 117 of Deephanphongs discloses: In some implementations the client system [1002] generates one or more audio or video fingerprints of the media content and then sends those fingerprints to the server system, which then matches the audio or video fingerprints to fingerprints of known content to determine up the identity of the content. Paragraph 117 discloses that the client system generates the audio fingerprint. Paragraph 163 of Deephanphongs also discloses that the client system 1002 of Figure 10 determines the audio fingerprint, and additionally discloses that the client system can “use the audio fingerprint to look up the identification of the content.” Figure 10, referenced in paragraphs 117 and 163, is a block diagram illustrating a client server environment 1000 that includes or more client systems 1002 and a server system 1020. Deephanphongs ¶ 125. Deephanphongs discloses that “[i]n some implementations, an integrated client system 1002-1 includes a display 1004-1, a camera 1006-1, facial recognition data 1014-1, and a facial recognition module 1016-1” and that [i]n some implementations, the integrated client system 1002-1 is a Appeal 2020-006124 Application 15/702,229 7 connected TV, an all-in-one desktop computer system, a laptop or tablet computer, a smart phone, or a handheld gaming system.” Id. Step [4] of claim 1 requires that processor, which determines an audio fingerprint, is “without any connection to the display, a set-top box, or a cable connection.” Deephanphongs describes embodiments in which the client system 1002 determines the audio fingerprint as in step [4] of claim 1, but it is deficient because the client system is described as being connected to a display 1004-1, TV, etc., which is explicitly excluded by step [4] of the claim. The Examiner further cites the remote server as performing “the analysis” of using the audio fingerprint and finds the remote server is not connected display, set top box, or cable connection. Ans. 5. However, step [4] requires that the processor determines the audio fingerprint, but the remote server is not disclosed in Deephanphongs as performing this function. Consequently, the Examiner’s finding that the client system and remote server expressly describe step [4] of the claim is not supported by a preponderance of the evidence cited by the Examiner. The only difference between Deephanphongs and step [4] of the claim is that the audio fingerprint is determined on a processor that has no connection to a display, set-top box, or cable connection. The absence of the connection to the display, etc., is not recited in the claim to have an effect on the determination of the audio fingerprint by the processor. The processor’s function in determining the audio fingerprint is not impacted by whether or not it is connected to a display or TV. Appellant did not provide evidence of any criticality to re-arranging the location of the processor in Deephanphongs’ system to lack a connection to a display, set-top box, or cable. Indeed, Deephanphongs does not require the client system to be Appeal 2020-006124 Application 15/702,229 8 connected to a TV, etc., but merely describes that in some implementations, the system is connected to a TV. Deephanphongs ¶ 125. It is not unobvious to make a part or device portable, separable, or to rearrange its location (In re Japikse, 181 F.2d 1019, 86 USPQ 70 (CCPA 1950) (Claims to a hydraulic power press which read on the prior art except with regard to the position of the starting switch were held unpatentable because shifting the position of the starting switch would not have modified the operation of the device.)”) when such changes do not affect the operation of the part or device. MPEP § 2144.04 (V) and (VI) (cases cited therein) (Ninth Edition, Revision 10.2019, Last Revised June 2020). Garza discloses that an imaging device could be “built in” to another device or it could be “a stand-alone device.” Garza ¶ 53. Thus, making a processor that determines an audio fingerprint as a separate or stand-alone device would have been obvious to one of ordinary skill in the art because changing its location does not affect how it operates and making it separate or stand-alone would make it more accessible when replacement or repair is necessary. Appellant argues that disconnecting Garza’s system for a display would make it unsuitable for its purpose because each of the entertainment devices must be connected to a display to “function properly.” Appeal Br. 18–19. This argument is not persuasive because the only part that is disconnected is the processor that determines “an audio fingerprint of the video based at least in part on the audio data” as in step [4] of claim 1. The Examiner relied on Aguera-Arcas for [7] “overwriting the image data and the audio data stored in the buffer so as to preserve privacy of the people in the viewing area.” Appellant argues it would make Garza unsuitable for its intended purpose because “[m]odifying Garza’s system to Appeal 2020-006124 Application 15/702,229 9 block collection of images of the viewers or to modify those images would make it difficult, if not impossible, for Garza’s system to gauge a viewer’s reaction to an advertisement or to identify that viewer.” Appeal Br. 21. Appellant also argues that Aguera-Arcas is not analogous art. Appeal Br. 21. Appellant states the field of endeavor in Aguera-Arcas is image-sharing, while the claim relates to “quantifying viewer engagement with a video shown on a display.” Id. (quoting from the claim preamble). Appellant also states that the claim addresses the shortcomings of conventional TV audience measurement while preserving audience privacy. Id. at 21–22. Appellant states that “concerns about image sharing don’t apply to the problem of quantifying viewer engagement” and therefore Aguera-Arcas is not analogous to claim 1. Id. at 22. We do not agree with Appellant that Aguera-Arcas is not reasonably pertinent to Garza. A reference is “reasonably pertinent” to the inventor’s particular problem when the reference “is one which, because of the matter with which it deals, logically would have commended itself to an inventor’s attention in considering his problem.” In re Klein, 647 F.3d 1343, 1348 (Fed. Cir. 2011) (citation omitted). The Examiner found that Aguera-Arcas describes a system in which a user can request privacy by removing personal identifiable information from storage. Final Act. 6. Although it may be true that Aguera-Arcas is in a different field of endeavor as the claim, it is relied upon by the Examiner for teaching privacy concerns that a user may have with respect to an image of the user, and thus is “reasonably pertinent” to Garza because Garza collects images of people who might have concerns about privacy. Overwriting the image data after the number of people in it have been estimated, as in steps Appeal 2020-006124 Application 15/702,229 10 [6] and [7] of claim 1, does not change how Garza operates. It merely overwrites stored data that has already been analyzed by a classifier. The Examiner gave a good reason to do it: to preserve the privacy of persons captured in the image. The Examiner’s reasoning is fact-based and logical. The relevant data from the image has been analyzed and therefore is no longer needed. Indeed, Deephanphongs also expresses a privacy concern for collected facial images, disclosing: The visual information is discarded after the facial images are extracted. In this way, image data collected from private client environments 1008-1 is never transmitted to a server 1020 or otherwise shared with anyone outside of the client system 1002- 1 itself. Deephanphongs ¶ 130. Appellant’s arguments with respect to Aguera-Arcas do not take into account what the cited reference would have reasonably suggested to one of ordinary skill in the art. “A person of ordinary skill is also a person of ordinary creativity, not an automaton.” KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 421 (2007). To assert that overwriting data that is no longer needed is nonobvious disregards that people discard files and paper when they are no longer necessary or being used. Appellant has not pointed to anything in the cited references that would indicate the image, after being used, should be retained. For the foregoing reasons, the rejection of claim 1 is affirmed. Because our reasoning differs from the Examiner’s with respect to the arguments regarding step [4] of the claim, we shall designate the affirmance as a new ground of rejection pursuant to 37 C.F.R. § 41.50(b) to provide Appellant the opportunity to respond. Appeal 2020-006124 Application 15/702,229 11 CLAIM 25 Claim 25 has similar limitations as claim 1. The Examiner relies on Garza for most of the limitations in the claim, but further cites Hicks for the limitation of claim 25 of “the audio data comprising timestamps representing when the memory stores the image data and the audio data, the timestamps enabling fixing of at least one blip in identifying at least one person in the viewing area while the video is being shown on the television.” Final Act. 14. The Examiner relies on Deephanphongs for disclosing the last limitation of the claim of “wherein the system is not connected to the television or to a set-top box.” With respect to this limitation, Appellant makes the same arguments about Deephanphongs that it did for claim 1. Appeal Br. 22–23. As explained for claim 1, it would have been obvious to make any device separate or stand-alone, without connection to a television or set-top box because neither of these are necessary for the system to operate. The same analysis for claim 1 applies to claim 25, Appellant also argues “it’s not clear from the Final Office Action how Deephanphongs could be used to modify Garza or Hicks to provide a system that is not connected to a television or set-top box, especially because Garza’s game console 412 is ‘coupled to the display device 410.’ Garza [0070].” Appeal Br. 23. Claim 25 does not exclude the system from being connected to a display. This exclusion does not appear in claim 25. Claim 25 further recites a processor which “erases the image data and the audio data from the buffer after determining the identity of the video and estimating the first number of people and the second number of people.” Appeal 2020-006124 Application 15/702,229 12 Appellant argues that Aguera-Arcas discloses blocking or modifying images and not erasing them. Appeal Br. 24. This argument is not persuasive. First, Aguera-Arcas at paragraph 20 discloses that “that personally identifiable or other information is removed,” which would reasonably be read to disclose or suggest removing the data by erasing it. Secondly, as discussed above, Deephanphongs ¶ 130 discloses discarding visual information which also reasonably be understood to erasing it (see supra at 10). Finally, as discussed above, the data from the image has been analyzed and therefore is no longer needed. For the foregoing reasons, the rejection of claim 25 is affirmed. Because our reasoning differs from the Examiner’s with respect to the arguments regarding the connection to a television or set-top box, we shall designate the affirmance as a new ground of rejection pursuant to 37 C.F.R. § 41.50(b) to provide Appellant the opportunity to respond. CLAIM 31 The Examiner rejected claim 31 over Garza, Terrazas, Deephanphongs, and Aguera-Arcas. Final Act. 20. The Examiner cites Garza, Deephanphongs, and Aguera-Arcas for most of the limitations recited in claim 31. Id. at 20–22. The Examiner further cites Terrazas to meet the limitation involving detecting a skeletal frame from the streaming image data, and in response, “generating at least one bounding box to identify head area of at least one person in the viewing area while the video is being shown on the display.” Id. at 23. Claim 31 also recites: estimating, with the processor and based at least in part on the streaming image data, a first number of people present in the Appeal 2020-006124 Application 15/702,229 13 viewing area while the video is being shown on the display without persisting the streaming image data so as to preserve privacy of the people in the viewing area Appellant makes substantially the same arguments as for claims 1 and 25 that Aguera-Arcas “does not disclose processing streaming image and audio data without persisting it. Instead, Aguera-Arcas discloses either blocking image collection or modifying collected images” which is not the same as “without persisting the streaming image data.” Appeal Br. 25. We simply do not agree that it is unobvious to not keep data (“without persisting”) after it has been used estimate the number of people in a viewing area. The relevant data from the streaming image has been analyzed and therefore is no longer needed. As indicated above, Deephanphongs and Aguera-Arcas disclose and reasonably suggest that the persistence of the streaming data is not necessary once the data has been used to estimate people number. The skilled artisan would also have had reason to discard it to address privacy concerns. For the foregoing reasons, the rejection of claim 31 is affirmed. Because our reasoning differs from the Examiner’s, we shall designate the affirmance as a new ground of rejection pursuant to 37 C.F.R. § 41.50(b) to provide Appellant the opportunity to respond. NEW GROUNDS OF REJECTION The affirmance of the rejections of independent claims 1, 25, and 31 are designated as new grounds of rejection pursuant to 37 C.F.R. § 41.50(b) because, while the same references are cited, we have expanded on the evidence and reasoning of the Examiner. However, the dependent claims are affirmed and not designated as new grounds because Appellant did not argue Appeal 2020-006124 Application 15/702,229 14 them separately. Appellant did not present arguments that the Examiner erred in rejecting the dependent claims.2 CONCLUSION In summary: Claims Rejected 35 U.S.C. § Reference(s)/ Basis Affirmed Reversed New Ground 1–3, 7, 8, 16–18, 32, 34, 35 103 Garza, Deephanphongs, Aguera-Arcas 1–3, 7, 8, 16–18, 32, 34, 35 1 4 103 Garza, Deephanphongs, Aguera-Arcas, Spears 4 5 103 Garza, Deephanphongs, Aguera-Arcas, Xie 5 15 103 Garza, Deephanphongs, Aguera-Arcas, Osawa 15 25–28, 30 103 Garza, Hicks, Deephanphongs, Aguera-Arcas 25–28, 30 25 29 103 Garza, Hicks, Deephanphongs, Aguera-Arcas, Xie 29 2 We review appealed rejections for reversible error based on the arguments and evidence the Appellant provide for each issue Appellant identifies. 37 C.F.R. § 41.37(c)(1)(iv); Ex parte Frye, 94 USPQ2d 1072, 1075 (BPAI 2010) (precedential) (cited with approval in In re Jung, 637 F.3d 1356, 1365 (Fed. Cir. 2011) (explaining that even if the Examiner had failed to make a prima facie case, “it has long been the Board’s practice to require an applicant to identify the alleged error in the examiner’s rejections”)). Appeal 2020-006124 Application 15/702,229 15 Claims Rejected 35 U.S.C. § Reference(s)/ Basis Affirmed Reversed New Ground 31 103 Garza, Terrazas, Deephanphongs, Aguera-Arcas 31 31 32, 34 103 Garza, Deephanphongs, Aguera-Arcas, Terrazas 32, 34 33 103 Garza, Deephanphongs, Aguera-Arcas, Peterson 33 36 103 Garza, Deephanphongs, Aguera-Arcas, Hyatt 36 Overall Outcome 1–5, 7, 8, 15–18, 25–36 1, 25, 31 TIME PERIOD FOR RESPONSE This decision contains a new ground of rejection pursuant to 37 C.F.R. § 41.50(b). 37 C.F.R. § 41.50(b) provides “[a] new ground of rejection pursuant to this paragraph shall not be considered final for judicial review.” 37 C.F.R. § 41.50(b) also provides that the Appellant, WITHIN TWO MONTHS FROM THE DATE OF THE DECISION, must exercise one of the following two options with respect to the new ground of rejection to avoid termination of the appeal as to the rejected claims: (1) Reopen prosecution. Submit an appropriate amendment of the claims so rejected or new Evidence relating to the claims so rejected, or both, and have the matter reconsidered by the Appeal 2020-006124 Application 15/702,229 16 examiner, in which event the prosecution will be remanded to the examiner. . . . (2) Request rehearing. Request that the proceeding be reheard under § 41.52 by the Board upon the same Record. . . . Further guidance on responding to a new ground of rejection can be found in the Manual of Patent Examining Procedure § 1214.01. AFFIRMED; 37 C.F.R. § 41.50(b)