Mimzi, LLCDownload PDFPatent Trials and Appeals BoardMar 12, 2021IPR2019-01516 (P.T.A.B. Mar. 12, 2021) Copy Citation Trials@uspto.gov Paper 29 571-272-7822 Date: March 12, 2021 UNITED STATES PATENT AND TRADEMARK OFFICE BEFORE THE PATENT TRIAL AND APPEAL BOARD MICROSOFT CORP., GOOGLE LLC, ASUSTEK COMPUTER INC., ACER INCORPORATED, HTC CORPORATION Petitioner, v. MIMZI, LLC, Patent Owner. IPR2019-01516 Patent 9,128,981 B1 Before JAMESON LEE, STACEY G. WHITE, and NABEEL U. KHAN, Administrative Patent Judges. KHAN, Administrative Patent Judge. JUDGMENT Final Written Decision Determining All Challenged Claims Unpatentable 35 U.S.C. § 318(a) IPR2019-01516 Patent 9,128,981 B1 2 I. INTRODUCTION A. Background and Summary Microsoft Corporation, Google LLC, ASUSTeK Computer Inc., Acer Incorporated, and HTC Corporation (collectively “Petitioner”) filed a Petition (Paper 6, “Pet.”) requesting an inter partes review of claims 1–19 (“the challenged claims”) of U.S. Patent No. 9,128,981 B1 (Ex. 1001, “the ’981 Patent”). Mimzi, LLC (“Patent Owner”) timely filed a Preliminary Response (Paper 10, “Prelim. Resp.”). On March 18, 2020, upon consideration of the Petition, Preliminary Response, and the evidence cited by the parties, we determined that Petitioner established a reasonable likelihood that it would prevail with respect to at least one of the claims challenged in the Petition and instituted review to determine the patentability of the challenged claims on all grounds. Paper 11 (“Dec. Inst.”), 1. After institution, Patent Owner filed a Patent Owner Response (Paper 18, “PO Resp.”), Petitioner filed a Reply (Paper 21, “Reply”), and Patent Owner filed a Sur-Reply (Paper 23, “Sur-Reply”). An oral hearing was held on December 14, 2020, and the hearing transcript is included in the record. Paper 28 (“Tr.”). We have jurisdiction under 35 U.S.C. § 6. This Final Written Decision, issued pursuant to 35 U.S.C. § 318(a) and 37 C.F.R. § 42.73 (2019), addresses issues and evidence raised during the inter partes review. For the reasons that follow, Petitioner demonstrates by a preponderance of the evidence that claims 1–19 of the ʼ981 Patent are unpatentable. B. Related Matters The parties identify the following matters as related to this case: Mimzi, LLC v. TripAdvisor Inc., et al., No. 1-18-cv-01768 (D. Del.); Mimzi, IPR2019-01516 Patent 9,128,981 B1 3 LLC v. Foursquare Labs, Inc., No. 1-18-cv-01767 (D. Del.); Mimzi, LLC v. Acer, Inc., No. 1-19-cv-00272 (D. Del.); Mimzi, LLC v. ASUSTeK Computer Inc., No. 1-19-cv-00273 (D. Del.); Mimzi, LLC v. HTC Corp., No. 1-19-cv- 00274 (D. Del.); TripAdvisor, LLC v. Mimzi, LLC, IPR2019-01080 (PTAB); Foursquare Labs, Inc. v. Mimzi, LLC, IPR2019-01287 (PTAB). Pet. 2; Paper 7, 2–3. The ’981 Patent is challenged in both IPR2019-01080 and IPR2019-01287. In IPR2019-01080, the Board determined that claims 1–4 and 7–18 of the ’981 Patent are unpatentable. IPR2019-01080, Paper 33. In IPR2019-01287, the Board determined that claims 1–19 of the ’981 are unpatentable. C. The ’981 Patent The ʼ981 Patent, titled “Phone Assisted ‘Photographic Memory,’” is directed “to using a phone to record phone calls, meeting and dictation, and for storing the metadata, audio and/or speech recognition text into a centralized content searchable repository, so that all or most of all conversations and near everything seen or read can be searched with a single query.” Ex. 1001, 1:14–19. The ʼ981 Patent describes using the phone to collect a variety of types of information, including recording phone calls, recording the audio of a face-to-face meeting, and recording an audio description of pictures and video taken from the camera of a phone (e.g. of notes on a notepad, whiteboard, and article). Id. at 3:23–36. Speech recognition is applied to the audio of the phone calls, meetings, and descriptions of pictures and videos to create transcripts so that these sources of information can be searched. Id. at 3:25–28. In addition to recording the above information, the phone also records metadata, such as the date, time, name, or location information of the recorded information. Id. at 2:31–34. IPR2019-01516 Patent 9,128,981 B1 4 For example, the phone can record the date and location of the phone when a picture is taken. Id. at 3:40–50. In this way, conversations, pictures and video can be searched by date, time, and location, as well as by the speech- recognized text of the recorded audio. Id. at 3:46–50. The ʼ981 Patent describes an embodiment where the text transcripts may contain timestamps. Id. at 5:58–60. For applications where the actual time spoken for each word is important, the start date and start time of the call or dictation may be read from the metadata, and the start date, start time, and relative timestamp may be added for each word in the transcript. Id. at 7:23–32. In addition, location coordinates also may be added next to each timestamp for each word in the transcript. Id. at 7:39–47. D. Illustrative Claims Of the challenged claims, claims 1, 10, and 16 are independent. Claims 2–9 depend from independent claim 1, claims 11–15 depend from independent claim 10, and claims 17–19 depend from independent claim 16. Claims 1 and 10 are reproduced below with bracketed annotations. 1. A system for presenting social-network-provided outputs to a mobile-electronic-device user at a location associated with the user in response to the user’s spoken request, comprising: [1.1] a data input port configured to receive speech information from the mobile-electronic-device user; [1.2] a memory configured to store a transcript of the spoken request and metadata associated with the spoken request comprising at least the location during the spoken request; [1.3] an interface port to a social network database, configured to transmit a request to mine information of the social network database based on the transcript and the metadata, and to receive social network information from the social network database based on the transmitted request; IPR2019-01516 Patent 9,128,981 B1 5 [1.4] at least one processor configured to transmit the request through the interface port dependent on at least the transcript and the metadata, to receive the social network information from the interface port; and [1.5] a communication port configured to communicate at least a portion of the social-network information to the user, [1.6] wherein the social network database comprises a plurality of social network records, the at least one processor being further configured to rank the received social network information comprising a plurality of received social network records dependent on at least one social network ranking factor, the communication port being further configured to output at least a portion of the social network records in a manner dependent on the at least one social network ranking factor. 10. A computer-implemented method for presenting social- network-provided information records from a social network database comprising a plurality of social network provided information records to a mobile-electronic-device user in response to the user’s spoken request, comprising: [10.1] receiving speech information from the mobile- electronic-device user; [10.2] storing the transcript of the spoken request and metadata associated with the spoken request comprising at least a location associated with the user during the spoken request; [10.3] automatically selecting a plurality of social- network-provided information records with at least one automated processor, by searching the social-network for information records using both the location and the transcript; [10.4] ranking the plurality of selected social network- provided information records with the at least one automated processor, dependent on at least one social network ranking factor and [10.5] communicating at least a portion of the selected social-network-provided information records to the user in a IPR2019-01516 Patent 9,128,981 B1 6 manner dependent on the at least one social network ranking factor. Ex. 1001, 17:4–26. E. Prior Art and Asserted Grounds Petitioner asserts that claims 1–19 would have been unpatentable on the following grounds: Ground Claim(s) Challenged 35 U.S.C. § Reference(s)/Basis 1 1, 10 103(a) Wu1 and Evermann2 2 1, 4–16, 18, 19 103(a) Wu, Evermann, Davis 3, and Burke4 3 1–19 103(a) Wu, Evermann, Davis, Burke, and Buyukkokten5 In addition, Petitioner relies on the Declaration and Reply Declaration of Sandeep Chatterjee (Exs. 1002, 1042 respectively) in support of the asserted grounds of unpatentability. Pet. 4. Patent Owner relies on the Declaration of Dr. Jose Luis Melendez to support its Patent Owner Response. Ex. 2006. II. ANALYSIS A. Patent Owner’s Constitutional Challenge Patent Owner argues “The Board should terminate this proceeding because Administrative Patent Judges (APJs) are unconstitutionally appointed.” PO Resp. 42. Patent Owner acknowledges that the Federal 1 Wu, US 7,895,177 B2, Feb. 22, 2011 (Ex. 1003, “Wu”) 2 Evermann, US 2008/0153465 A1, June 26, 2008 (Ex. 1004, “Evermann”) 3 Harold Davis, Building Research Tools with Google for Dummies (2005) (Ex. 1010, “Davis”) 4 Burke, US 2008/0065617 A1, March 13, 2008 (Ex. 1011, “Burke”) 5 Buyukkokten, US 2005/0159998 A1, July 21, 2005 (Ex. 1005, “Buyukkokten”) IPR2019-01516 Patent 9,128,981 B1 7 Circuit attempted to cure the constitutional defect in Arthrex, Inc. v. Smith & Nephew, Inc., 941 F.3d 1320 (Fed. Cir. 2019), cert. granted sub nom. United States v. Arthrex, Inc., 2020 WL 6037206 (Oct. 13, 2020), but argues “the solution advanced in Arthrex is insufficient because IPR proceedings are ‘formal adjudications’ under the Administrative Procedure Act.” PO Resp. 42 (citing Belden Inc. v. Berk-Tek LLC, 805 F.3d 1064, 1080 (Fed. Cir. 2015). Patent Owner argues, “Assuming that the APJs assigned to this proceeding are now removable at-will, they lack authority to conduct ‘formal adjudications[s]’ in accordance with 5 U.S.C. § 556.” PO Resp. 42. Patent Owner argues, “[a]s the APJs adjudicating this proceeding now lack the authority to preside over formal adjudications, including the present IPR, Patent Owner requests that the Board dismiss or stay this proceeding.” PO Resp. 43. We are bound by the Federal Circuit’s decision in Arthrex, which addressed this issue. See Arthrex, 941 F.3d at 1337 (“This as-applied severance . . . cures the constitutional violation.”); see also Arthrex, Inc. v. Smith & Nephew, Inc., 953 F.3d 760, 764 (Fed. Cir. 2020) (Moore, J., concurring in denial of rehearing) (“Because the APJs were constitutionally appointed as of the implementation of the severance, inter partes review decisions going forward were no longer rendered by unconstitutional panels.”). Accordingly, we do not address this issue any further. B. Level of Ordinary Skill in the Art Petitioner argues that a person of ordinary skill in the art at the time of the invention “would have possessed at least a bachelor’s degree in software engineering, computer science, computer engineering, or electrical engineering with at least one year of experience in computing systems used IPR2019-01516 Patent 9,128,981 B1 8 for recording and storing human speech and speech-to-text conversion (or equivalent degree or experience).” Pet. 8 (citing Ex. 1002 ¶¶ 13–15). Patent Owner argues that because the relevant art and technology is social network databases, a person of ordinary skill in the art must have some familiarity with social network databases in addition to speech-to-text conversion. PO Resp. 14. On this record, we agree with Petitioner’s proposed definition of person of ordinary skill in the art. Although the claims of the ʼ981 Patent recite searching a social network database, we find the fields of software engineering, computer engineering, or electrical engineering would provide knowledge related to social networks and database search at a level commensurate with the disclosure of the ʼ981 Patent. We note, however, that our analysis below would be unaffected were we to expressly require that the level of ordinary skill in the art include knowledge of social network databases, as Patent Owner proposes. C. Claim Construction In inter partes reviews, we interpret a claim “using the same claim construction standard that would be used to construe the claim in a civil action under 35 U.S.C. 282(b).” See 37 C.F.R. § 42.100(b) (2019). Under this standard, we construe the claim “in accordance with the ordinary and customary meaning of such claim as understood by one of ordinary skill in the art and the prosecution history pertaining to the patent.” Id. Only claim terms that are in controversy need to be construed and only to the extent necessary to resolve the controversy. See Nidec Motor Corp. v. Zhongshan Broad Ocean Motor Co., 868 F.3d 1013, 1017 (Fed. Cir. 2017). IPR2019-01516 Patent 9,128,981 B1 9 1. Metadata Associated with the Spoken Request Patent Owner proposes the term “metadata associated with the spoken request” be construed as “information related to the spoken request attached to the request, such as time and location.” PO Resp. 14 (citing Ex. 2006 ¶¶ 43–48). Patent Owner relies on a dictionary definition of the term “associated” as “1: joined together often in a working relationship . . . 2: related, connected, or combined together”6 (PO Resp. 14–15 (citing Ex. 2007))7 and argues that “[t]he word ‘associated’ signifies a tighter integration than being merely ‘related.’” PO Resp. 15. Patent Owner argues that a person of ordinary skill in the art “would understand that metadata associated with the spoken request . . . requires a greater connection between the metadata and spoken request (i.e. association) than the mere fact that a transcript and location metadata may exist or be used contemporaneously, without association.” Id. at 15 (citing Ex. 2006 ¶¶ 43–48). Patent Owner argues that subsequent claim limitations in claim 1, such as those that recite “transmit[ing] a request to mine information . . . based on the transcript and the metadata, and to receive social network information . . . based on the transmitted request” indicate that the metadata must be associated with the spoken request in order for these limitations to make sense. PO Resp. 16– 17. Patent Owner argues that Figures 1 and 2 of the ʼ981 Patent show time and location metadata “are associated with the spoken request by association of the corresponding metadata with the transcript of the speech at the ‘Add 6 The definition cited in Patent Owner’s response does not appear in Exhibit 2007. Instead Exhibit 2007 defines “associate” as “1: to join as a partner, friend, or companion . . . 2: to join or connect together . . . 3: to bring together or into relationship in any of various intangible ways.” IPR2019-01516 Patent 9,128,981 B1 10 Absolute Time,’ and ‘Add location’ steps indicated.” Id. at 16 (citing Ex. 2006 ¶¶ 43–48). Petitioner argues the limitation ‘metadata associated with the spoken request,’ properly construed, does not require the metadata to be ‘attached’ to the spoken request.” Reply 7. Instead, Petitioner argues the term should be construed as “information related to the spoken request, such as time and location.” Pet. 7 (emphasis removed). Petitioner points out that the specification of the ʼ981 Patent uses the term “corresponding” to describe metadata and its relationship with a dictation audio, indicating that the specification does not limit “associate” to being “attached.” Reply 7 (citing Ex. 1001, 6:59–63; Ex. 1042 ¶¶ 42–44). Similarly, Petitioner argues dictionary definitions also “do not limit ‘associate’ to physical connections, but expressly contemplate ‘relationship[s] in any of various intangible ways.’” Reply. 7–8 (citing Ex. 1037; Ex. 2007). Petitioner emphasizes that interpreting “associated” as “attached” would render other uses of the term in the specification of the ʼ981 Patent, such as “location associated with the user” nonsensical. Reply. 8 Having considered the parties’ arguments and the fully developed trial record, we do not agree with Patent Owner’s proposed construction. Patent Owner does not provide sufficient evidence that the word “associated” means “attached” in the context of the claim. The ʼ981 Patent does not use the term “attached” anywhere to describe the metadata associated with the spoken request, nor has Patent Owner identified the use of the term anywhere in the intrinsic evidence. Patent Owner cites Figures 1 and 2 of the ʼ981 Patent as supporting its construction but Figures 1 and 2 show that a time offset and location may be added to the transcript for each word in the transcript. In these embodiments the transcript itself includes the time and IPR2019-01516 Patent 9,128,981 B1 11 location information. The claim, however, recites that metadata be associated with the spoken request, not that it be added to the transcript. Similarly, the dictionary definitions relied on by Patent Owner also do not define “associated” to mean “attached,” and as Petitioner points out, at least one definition allows for a broader relationship. Ultimately, we fail to see how these embodiments show that time and location are attached to the spoken request. In light of the parties’ arguments regarding unpatentability, we determine that we need not further construe the term “metadata associated with the spoken request.” 2. Transcript Patent Owner argues that “the ʼ981 Patent requires that a ‘transcript’ be searchable electronically and capable of being generated as a conversion of speech by a speech recognition engine, where the speech is received through the data input port.” PO Resp. 16–17. According to Patent Owner this is distinct from “any mere text representation of speech.” Id. at 17. Patent Owner, however, does not provide an explicit construction for the term in its Patent Owner Response. Petitioner argues that express construction is not necessary and “any requirement Patent Owner appears to [be] impos[ing by its proposed construction] would still be met.” Reply 13–14. Further to that end, Petitioner contends that Patent Owner never disputes that the XML document disclosed in the prior art references teach a “transcript,” (id. at 9– 10) and that Patent Owner does not explain how its construction excludes Evermann’s text representation of the user’s spoken request (id. at 10–14). IPR2019-01516 Patent 9,128,981 B1 12 We agree with Petitioner that Patent Owner does not base its arguments for patentability on the construction of the term “transcript.” Patent Owner does not dispute that the XML document of prior art references such as Davis teach a transcript and, although Patent Owner argues that “Evermann does not generate a transcript” (PO Resp. 21) we have determined that Patent Owner does not explain how or why this is true based on its proposed construction of “transcript.” For the purposes of analyzing the issues before us, we see no need to further construe this term, which is neither a term of art nor a creation of the patentees, but rather an ordinary word that one of ordinary skill in the art would have understood without further elaboration. 3. Social Network Database Patent Owner proposes that “social network database” should be construed as “a centralized database that stores the interpersonal relationships among multiple members of the community and the content they contributed to the database.” PO Resp. 18 (citing Ex. 1001, 15:28–42; Ex. 2006 ¶¶ 51–60). According to Patent Owner, a “social network” is a community of individuals connected by interpersonal relationships. Id. at 17 (citing Ex. 2008). Patent Owner also argues that the ʼ981 Patent “unambiguously defines a social network database as ‘a centralized community search database.’” Id. (citing Ex. 1001, Abstract). Thus, according to Patent Owner, a social network database may be understood to be a centralized database that stores the interpersonal relationships among multiple members of the community. Id. (citing Ex. 1001, 15:28–42; Ex. 2006 ¶¶ 45–56). Patent Owner further argues that the term “social network” is not directed to the content of information, but rather to limitations IPR2019-01516 Patent 9,128,981 B1 13 regarding contributions by multiple members, accessibility by multiple members, and the relationships between. Id. at 17–18. (citing Ex. 2006 ¶¶ 51–60). Petitioner proposes that “social network database” be construed as “database used to store information contributed and subsequently searched by members of a social network.” Reply 21 (emphasis removed). Petitioner argues “[t]his construction gives effect to the words ‘social network’ while remaining consistent with the specification, which describes a ‘Centralized Community Search database’ that stores ‘content [] contributed by multiple members of the community,’ but makes no mention of storing any interpersonal relationships among members.” Reply 21. Petitioner also argues that that Patent Owner’s construction does not distinguish the prior art references relied upon in the grounds challenging the ʼ981 Patent claims, and that therefore the term need not be construed. Reply 20–21. Having considered the parties’ arguments and evidence and the fully developed trial record, we do not agree with Patent Owner’s construction. The ʼ981 Patent states that it discloses “[a] system and method for providing social-network (Centralized Community Search database)—provided outputs to a mobile-electronic-device user in response to the user’s spoken request.” Ex. 1001, Abstract. The ʼ981 Patent explains that a user may wish to transmit their “text messages, photos, videos, call and voice annotations or synopsis (with speech-recognized text and location data), to a Centralized Community Search database” and that the user may designate this content to be public and “be shared with anyone via the Internet.” Ex. 1001, 14:64– 15:6. The ʼ981 Patent calls the public sharing of information as “Location Blogging.” See Ex. 1001, 15:1–6. The ʼ981 Patent explains that the publicly shared content contributed to the Centralized Community Search IPR2019-01516 Patent 9,128,981 B1 14 database may include information such as “a review of a restaurant or business.” Ex. 1001, 15:7–15. The Centralized Community Search database “may be searched or mined based on social network or type of information, such as ‘Urgent Road Warning’ when driving or ‘Best Menu Choice’ at a restaurant location.” Ex. 1001, 15:22–26. Although the ʼ981 Patent describes information being stored in a centralized community search database, it does not restrict the community in any way nor does it describe the database as storing interpersonal relationships between community members. We find insufficient support in the ʼ981 Patent for such a restrictive reading. We, therefore, decline to import such limitations into the claim. The ʼ981 Patent describes a centralized community database that stores information contributed by members of a community, such as text messages, photos, videos, road warnings and restaurant reviews, accessible by anyone on the Internet, without disclosing or requiring the storage of interpersonal relationships. In light of the parties’ arguments regarding unpatentability, we need not further construe the term “social network database.” 4. Social Network Ranking Factor Patent Owner proposes that “social network ranking factor” should be construed as “an interpersonal metric, based upon relationships between members, that is derived from a social network database.” PO Resp. 19 (citing Ex. 2006 ¶¶ 51–60). Patent Owner argues that the term should not be construed as encompassing any ranking factor, rather the claims require the ranking factor to be derived from the social network database. PO Resp. 19. IPR2019-01516 Patent 9,128,981 B1 15 Petitioner argues “social network ranking factor” should be construed as “ranking factor used to rank social network-provided information.” Reply 17. Petitioner argues that its construction gives effect to the words “social network” and is consistent with the specification and claim language. Reply 17. Petitioner also argues that that Patent Owner’s construction would be met by Petitioner’s asserted ground that all challenged claims would have been obvious over Wu, Evermann, Davis, Burke, and Buyukkokten, and that therefore the term need not be construed. Reply 17. Having reviewed the arguments and evidence presented by the parties, we do not agree with Patent Owner’s proposed construction. Patent Owner provides insufficient evidence to support its argument that the social network ranking factor must be an interpersonal metric, based upon relationships between members. The ʼ981 Patent is devoid of any such description. Instead, the ʼ981 Patent describe several “ranking factors” such as “distance from the Phone user’s current location and the location of the original Location Blog poster,” “reverse chronological order, social network credibility or social network popularity of the Location Blog poster, and type of data (e.g. Urgent Road Warning versus restaurant ratings).” Ex. 1001, 15:27–37. Dependent claims 2, 3, and 4 indicate that social network ranking factor comprises social network popularity, social network credibility, and user review ratings. For example, language of claim 4 explicitly states that “the social network ranking factor compris[es] user review ratings of respective restaurants.” As such, no other ranking factors are required to meet the limitations of claim 4 other than the user review ratings. This indicates that social network ranking factor is broader than Patent Owner’s IPR2019-01516 Patent 9,128,981 B1 16 proposal, encompassing factors that are not limited to an interpersonal metric. For the foregoing reasons, we determine “social network ranking factor” is not limited to an interpersonal metric, based upon relationships between members. In light of the parties’ arguments regarding unpatentability, we determine that we need not further construe the “social network ranking factor.” D. Grounds under § 103(a) Petitioner contends claim 1 and 10 are obvious in view of Wu and Evermann; claims 1, 4–16, 18, 19 are obvious in view of Wu, Evermann, Davis and Burke; and claims 1–19 are obvious in view of Wu, Evermann, Davis, Burke, and Buyukkokten. Pet. 4. A claim is unpatentable under § 103(a) if the differences between the claimed subject matter and the prior art are “such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains.” KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 406 (2007). The question of obviousness is resolved on the basis of underlying factual determinations, including: (1) the scope and content of the prior art; (2) any differences between the claimed subject matter and the prior art; (3) the level of skill in the art; and (4) objective evidence of nonobviousness, i.e., secondary considerations. Graham v. John Deere Co., 383 U.S. 1, 17–18 (1966). Additionally, the obviousness inquiry typically requires an analysis of “whether there was an apparent reason to combine the known elements in the fashion claimed by the patent at issue.” KSR, 550 U.S. at 418 (citing In re Kahn, 441 F.3d 977, 988 (Fed. Cir. 2006) (requiring “articulated IPR2019-01516 Patent 9,128,981 B1 17 reasoning with some rational underpinning to support the legal conclusion of obviousness”)); see In re Warsaw Orthopedic, Inc., 832 F.3d 1327, 1333 (Fed. Cir. 2016) (citing DyStar Textilfarben GmbH & Co. Deutschland KG v. C. H. Patrick Co., 464 F.3d 1356, 1360 (Fed. Cir. 2006)). As explained below, we determine based on the present record that Petitioner has shown by a preponderance of the evidence that each of the challenged claims is obvious. 1. Objective Indicia of Non-Obviousness Factual inquiries for an obviousness determination include secondary considerations based on evaluation and crediting of objective evidence of nonobviousness. Graham, 383 U.S. at 17. Notwithstanding what the teachings of the prior art would have suggested to one with ordinary skill in the art at the time of the invention, the totality of the evidence submitted, including objective evidence of nonobviousness, may lead to a conclusion that the claimed invention would not have been obvious to one with ordinary skill in the art. In re Piasecki, 745 F.2d 1468, 1471–1472 (Fed. Cir. 1984). Patent Owner argues that “the alleged obviousness of the invention is further undercut by several objective indicia of nonobviousness, including satisfaction of a long-felt need and praise by the industry.” PO Resp. 36. Patent Owner relies on two pieces of evidence as establishing long felt need and industry praise for its invention: (1) a blog post, dated September 10, 2008, by Google’s then-Vice President of Search Products & User Experience, Marissa Mayer, (“Google Blog Post”) and (2) a 2011 Apple product presentation in which Apple demonstrated its Siri functionality (“Apple Presentation”). IPR2019-01516 Patent 9,128,981 B1 18 As to the Google Blog Post, Patent Owner relies on statements that Patent Owner argues show “the search industry needed exactly what was provided by the ʼ981 Patent Invention.” PO Resp. 36 (quoting Ex. 2001, 3). Below we quote portions of the Google Blog Post that Patent Owner relies upon: [W]hy couldn’t I do these searches right then, when I needed to? Because search still isn’t accessible enough or easy enough. Search needs to be more mobile—it should be available and easy to use in cell phones and cars and on handheld . . . one far-fetched idea: how about a [device] that does searches in the background based on the words it picks up from conversations . . . Why can’t I capture a snippet of audio and have the search engine identify and analyze it and tell me any relevant information about it? Services that do parts of that are available today, but not in an easy-to-use, integrated way . . . Ex. 2001, 3. Patent Owner further relies on the following statements as evidence that the Google Blog Post “identified location data in context (such as part of the metadata of the spoken query) as pivotal to increasing relevancy and ease of search.” Your location is one potentially useful facet of personalization information. Looking at my questions, the answers to a number of them . . . require the search engine to know that I was in Yankton, South Dakota and Crofton, Nebraska when I asked. Since location is relevant to a lot of searches, incorporating user location and context will be pivotal in increasing the relevance and ease of search in the future. Ex. 2001, 5. Patent Owner also relies upon statements that it contends show search of a social network database as important for making search efficient and relevant. For example, Patent Owner relies on the following statement from the Google Blog Post: IPR2019-01516 Patent 9,128,981 B1 19 Another element of personalization is social context. Who am I friends with, and how do I relate to them? How can I harness their knowledge more efficiently? . . . There’s a lot of expertise, knowledge, and context in user’s social graphs, so putting tools in place to make “friend-augmented” search easy could make search more efficient and more relevant. PO Resp. 37 (quoting Ex. 2001, 5–6). Turning to the Apple Presentation, Patent Owner focuses on a part of the presentation where Scott Forstall, an Apple Senior Vice President, demonstrated “the ability to speak into an iPhone and get social network reviews of a nearby Greek restaurant.” PO Resp. 52. Patent Owner relies on the following statements by Mr. Forstall: I’ve been in the AI field for a long time, and this still blows me away, the fact that it understands the words, the concepts, and even now ranks it by Yelp’s ordering. PO Resp. 38 (quoting Ex. 2004, 11). Patent Owner argues “three years after the ’981 patent application was filed, Apple presented the invention with pride and it was met with praise by the tech-savvy audience.” PO Resp. 38. a) Long Felt but Unmet Need In order to show a long-felt but unmet need for the claimed invention, the objective evidence must show that the need was a persistent one that was recognized by those of ordinary skill in the art at the time of the invention. In re Gershon, 372 F.2d 535, 538 (CCPA 1967). The Google Blog Post is dated September 2008, several weeks after the application, which ultimately resulted in the ʼ981 Patent was filed.8 Thus, the date of the Google Blog 8 In a related case, where intervening art was asserted against the’981 Patent and Patent Owner attempted to antedate that prior art, upon full briefing by the parties in that case, we determined that the ’981 Patent is not a Pre-AIA patent and that the time for assessing obviousness and objective indicia of IPR2019-01516 Patent 9,128,981 B1 20 Post itself cannot establish a long felt persistent, but unmet need. Moreover, the Google Blog Post does not provide any evidence of how long, prior to the invention, the needs expressed by Ms. Mayer existed. Patent Owner argues that Ms. Mayer’s comments show a need for the invention and that this need was “encountered extensively over nine-years of experience at Google.” PO Resp. 37. In support, Patent Owner relies on Ms. Mayer’s statement that “I’ve worked at Google on search for the past 9 years and 3 months. . . . I’m very proud that Google in its first 10 years has changed expectations around information and how quickly and easily it should be able to be retrieved. But I’m even more excited about what Google search can achieve in the future.” Id. (quoting Ex. 2001, 2, 7). Patent Owner argues that these statements “clearly suggests the challenges nonobviousness is May 22, 2013 for claims 1–15. Foursquare Labs, Inc. v. Mimzi, LLC, IPR2019-01287, Paper 34 at 27 (PTAB Jan. 13, 2021). Although the issue was not raised or briefed by either party in this proceeding, the Board has determined in IPR2019-01287 that for claims 1– 15 the effective filing date is no earlier than May 22, 2013. For completeness, in the alternative, we address the objective indicia of nonobviousness from the time perspective of May 22, 2013, for claims 1– 15. As explained in the Final Written Decision in IPR2019-01287, determining obviousness from the date of May 22, 2013, instead of July 2008, renders ineffective Patent Owner’s evidence of a 2008 Google Blog Post (Ex. 2001) and a 2011 Apple Presentation (Ex. 2004). Patent Owner asserts a nexus between the evidence and the claimed invention by contending that both exhibits show that the four alleged notable “features” of the ’981 Patent—speech recognition on a mobile device, location awareness, social networks, and social network ranking factors—all existed prior to May 22, 2013. PO Resp. 39. Assuming a nexus (which would be necessary for Patent Owner’s arguments), if there ever was a long-felt need, that need was satisfied at least by 2011, prior to the effective filing date of the ʼ981 Patent for claims 1–15, i.e., no earlier than May 22, 2013. IPR2019-01516 Patent 9,128,981 B1 21 she describes are challenges she has attempted to address but had not yet succeeded.” Id. We disagree. The Google Blog Post does not show that the ideas Ms. Mayer was describing were ones she or others at Google had been persistently trying to achieve for a long period of time. The only statements mentioning a time period are Ms. Mayer’s statement that, at the time of the blog post, she had worked at Google for nine years, and a statement at the end of the post stating that she was very proud of what Google had achieved over that time. See Ex. 2001, 2, 7. These statements are too general and disconnected to show sufficiently a long felt and persistent need. Instead, when read in the context of the Google Blog Post, Ms. Mayer’s statement that she had worked at Google for nine years, was more likely intended as an introductory statement showing that she has a strong interest in search technology. See Ex. 2001, 2 (“I am a search addict. I’m naturally inquisitive – I’ve always liked finding things out. Plus, I’ve worked at Google on search for the past 9 years and 3 months. Of course I search a lot.”) The Apple Presentation, which occurred roughly three years after the ʼ981 Patent application was filed, suffers from similar problems in that its date alone cannot be used to establish a persistent long-felt but unmet need at the time of the invention. As before, Patent Owner does not present any evidence relating to this presentation indicating that the need was long felt and persistent. Patent Owner relies on a statement during the presentation that “for decades, technologists have teased us with this dream that you’re going to be able to talk to technology, and it’ll do things for us. Haven’t we seen this before, over and over? But it never comes true.” PO Resp. 38–39 (quoting Ex. 2004, 11). Like the statements in the Google Blog Post, this IPR2019-01516 Patent 9,128,981 B1 22 statement is simply too general to sufficiently show a long felt persistent need. Other than mention the desire to “talk to technology,” the statement fails to provide any information regarding specific needs identified by Apple, what it wanted the technology to deliver (other than “it’ll do things for us”) and when such needs were specifically identified. Based on the evidence and arguments presented in the current record, we give little weight to Patent Owner’s evidence of long-felt need as supporting non-obviousness. b) Industry Praise Praise in the industry for a patented invention, and specifically praise from a competitor, tends to indicate that the invention was not obvious. Kinetic Concepts, Inc. v. Smith & Nephew, Inc., 688 F.3d 1342, 1370 (Fed. Cir. 2012). Industry praise for an invention may provide evidence of nonobviousness where the industry praise is linked to the claimed invention. See Geo. M. Martin Co. v. Alliance Mach. Sys. Intern. LLC, 618 F.3d 1294, 1305 (Fed. Cir. 2010); Asyst Tech’n, Inc., v. Emtrak, Inc., 544 F.3d 1310, 1316 (Fed. Cir. 2008). “[T]o be accorded substantial weight in the obviousness analysis, the evidence of secondary considerations must have a ‘nexus’ to the claims, i.e., there must be ‘a legally and factually sufficient connection’ between the evidence and the patented invention.” Henny Penny Corp. v. Frymaster LLC, 938 F.3d 1324, 1332 (Fed. Cir. 2019). Patent Owner points to comments by Mr. Forstall at the Apple Presentation praising Apple’s own Siri functionality. Mr. Forstall noted, “We’ve also partnered with Yelp, and so you can ask questions about businesses or restaurants, something like, ‘Find me a great Greek restaurant in Palo Alto.’ . . . Now I’ve been in the AI field for a long time, and this still IPR2019-01516 Patent 9,128,981 B1 23 blows me away, the fact that it understands the words, the concepts, and even now ranks it by Yelp’s ordering.” Ex. 2004, 40:16–24. Patent Owner uses these statements as a basis for arguing that there was a nexus to the claimed invention because these statements allegedly show (1) speech recognition on a mobile device; (2) location awareness; (3) social networks; and (4) social network ranking factors. PO Resp. 39–42. Even assuming Patent Owner’s claimed invention can be generalized to the aforementioned four categories, Mr. Forstall’s comments are simply insufficient to show a nexus to the claims of the ʼ981 Patent. For example, Patent Owner argues that Mr. Forstall’s comments show that Siri was location aware because it found restaurants in Palo Alto. But as can be seen, Mr. Forstall asked Siri to search for Greek restaurants in Palo Alto as part of the spoken search request. Because a location was provided as part of the search request, these comments alone do not indicate that the mobile device was location aware. Further, Mr. Forstall’s comments also appear to be directed at features that may go beyond the ʼ981 Patent’s invention, such as Siri’s AI capabilities of being able to understand the words and concepts being spoken, and not just its ability to transcribe them. Thus, at least to the extent the comments by Apple’s Vice President are based on the ability of Apple’s products to “understand[] the words, the concepts” being spoken, we note that, without further evidence or explanation, such comments have a tenuous nexus with the claimed invention. As to speech recognition, the ʼ981 Patent describes that transcribing the spoken request was a process that was performed by prior art speech recognition engines, “such as the Nuance Communications speech recognition engine.” Ex. 1001, 7:3–4. In order to establish a nexus, the IPR2019-01516 Patent 9,128,981 B1 24 evidence relied upon must trace its basis to a novel element in the claim, not to something in the prior art. Institut Pasteur & Universite Pierre et Marie Curie v. Focarino, 738 F.3d 1337, 1347 (Fed. Cir. 2013). Objective evidence that results from something that is not “both claimed and novel in the claim,” lacks a nexus to the merits of the invention. In re Kao, 639 F.3d 1057, 1068 (Fed. Cir. 2011). Thus, because the ʼ981 Patent acknowledges that speech recognition was known in the art at the time of the invention, and to the extent Apple’s comments are directed to its products’ ability to convert speech to text, such praise does not provide the required nexus to the claimed invention. Patent Owner argues that even if speech recognition was well known, “the novelty is the combination of speech recognition, location metadata, and social network data. It is the novel combination that has received praise in the industry . . . .” As we explained above however, Mr. Forstall’s comments do not establish location awareness. We, therefore, do not find his comments sufficient to establish a nexus to the claimed invention. Even if we were to assume a nexus between Mr. Forstall’s comments and the ʼ981 Patent, we still find the comments to be fairly weak support for industry praise. Mr. Forstall comments fail to show, for example, that any praise for the concepts described, was shared industry wide, rather than by Mr. Forstall or Apple themselves. Further, Mr. Forstall praise for Apple’s own products, at Apple’s own product launch event, is self-congratulatory and insufficiently persuasive of industry praise for the invention claimed in the ʼ981 Patent. See Bayer Healthcare v. Watson Pharms., 713 F.3d 1369, 1377 (Fed. Cir. 2013) (explaining that bare journal citations and self- referential commendation fall short of demonstrating industry praise). IPR2019-01516 Patent 9,128,981 B1 25 To the extent Patent Owner relies on the Google Blog Post as evidence of industry praise, we find such evidence unpersuasive. We find no evidence of industry praise in the relied upon sections of the Google Blog Post. The Google Blog Post is one person’s description of the possible future direction of Google’s search technology, rather than praise, let alone industry praise, for advances in that technology as described in the ʼ981 Patent. Accordingly, based on the evidence and arguments presented, we do not weigh Patent Owner’s evidence of alleged industry praise as significantly supportive of Patent Owner’s position in the obviousness analysis. E. Obviousness over Wu and Evermann Petitioner contends claims 1 and 10 are unpatentable as obvious over Wu and Evermann. Pet. 11–41. We provide a brief overview of Wu and Evermann, and then analyze whether this combination teaches the limitations of claims 1 and 10 in the sections below. 1. Overview of Wu Wu relates to enabling searching of user ratings and reviews using user profiles, location information, and social networks. Ex. 1003, Abstract. The search results are presented with aggregated weighted ratings. Ex. 1003, Abstract. Wu describes an embodiment where a user enters a “free form search query.” Ex. 1003, 3:43–44. The search query is then transparently modified using the user’s profile, the user’s social network, and the user’s location. Ex. 1003, 3:44–57. For example, if the user searches for a product but forgets its brand name, the user’s profile or social network may be used to modify the search query to add a brand name. IPR2019-01516 Patent 9,128,981 B1 26 Ex. 1003, 3:47–51. Similarly, if the search query does not include a location, the query may be transparently modified to add the user’s city or location as part of the search query. Ex. 1003, 3:51–57. The modified search request is then received by a Universal Search and Rating (USR) server, which searches “a data store that includes aggregated review and ratings data for various products and[] services.” Ex. 1003, 7:16–18. Wu provides an example of a search for “ketchup,” which results in a list including Heinz ketchup, Trader Joe’s organic ketchup, ketchup in barbecue sauce, and Safeway brand ketchup. Ex. 1003, 13:20–41. Wu features a table with the results of this search, which is reproduced below. The search results of a search for “ketchup,” depicted above, include ratings and also an indication of the number of reviews contributing to that rating. Ex. 1003, 13:42–47. The results and ratings are based, at least in part, on information from the searcher’s social network. Ex. 1003, 13:23–59. 2. Overview of Evermann Evermann relates to wireless communication devices with speech recognition capabilities. Ex. 1004 ¶ 2. Figure 1 of Evermann is reproduced below: IPR2019-01516 Patent 9,128,981 B1 27 The above figure depicts a system architecture that implements the invention of Evermann. Id. ¶ 14. As shown in Figure 1, mobile device 102 includes voice-mediated search application (VMSA) 106, which recognizes when a spoken command from the user constitutes a search request and determines whether it must be sent to a remote server for processing, in which case it opens data connection 108 to do so. See id. ¶¶ 37–39. The audio of the spoken request is sent to transaction server 110 along with metadata, such as the geographical location of device 102. Id. ¶¶ 39–40. Transaction server 110 passes the audio information and metadata to automatic speech recognizer (ASR) server 112, which performs speech recognition on the audio and returns text of the audio and other information to transaction server 110. Id. ¶ 42. Search management software 118 on transaction server 110 uses the metadata and other information to select one or more content (114a, b, c) and advertising (116a, b, c) providers to service the search request. Id. The content providers retrieve the requested content and send it to transaction server 110, which selects and prioritizes the received content using the metadata and sends it to mobile device 102. Id. VMSA 106 then presents the results to the user. Id. IPR2019-01516 Patent 9,128,981 B1 28 3. Combination of Wu with Evermann Petitioner combines Wu’s text-based search capability with Evermann’s voice recognition technique so that Wu’s system is able to accept search requests by voice rather than only through text. In particular, Petitioner argues “[t]he combination [of Wu and Evermann] would have predictably resulted in a mobile device user searching for social-network- provided information as disclosed in Wu, in which the search request is input by the user, and thereby received by Wu’s system, according to Evermann’s voice technique.” Pet. 19 (citing Ex. 1002 ¶¶ 66–71). The motivation to combine Wu’s text-based search with Evermann’s voice-based technique, Petitioner argues, stems from Evermann itself, which explains that a text-based interface on a cell phone can be “cumbersome” as is navigating through menus on a cell phone’s web browser. Pet. 19 (citing Ex. 1004 ¶¶ 3–4). Petitioner argues, “By entering a request through voice rather than text-based commands, therefore, the user is provided with a more convenient and straightforward way to search information.” Pet. 20 (citing Ex. 1002 ¶ 68). Inputting search through voice “obviates the need to type with a small keypad . . . [and a]dditionally, voice data can capture certain user traits, such as gender, which can then be used to return more relevant search results.” Pet. 20 (citing Ex. 1002 ¶ 69). We find Petitioner has identified specific disclosures in Evermann that would have motivated one of ordinary skill in the art to combine its teachings with Wu, such as the added convenience provided by voice-based search in a mobile phone with a small keypad. Pet. 19 (citing Ex. 1004 ¶¶ 3–4). Petitioner supports its arguments with Dr. Chatterjee’s testimony. See Pet. 20 (citing Ex. 1002 ¶¶ 68, 69). IPR2019-01516 Patent 9,128,981 B1 29 Based on the evidence presented, we determine Petitioner has articulated sufficient reasoning with rational underpinning to support the combination of Wu with Evermann. 4. Analysis of Claims 1 and 10 Petitioner presents a detailed explanation of how the combination of Wu and Evermann teaches each of the limitations of claims 1 and 10. Pet. 12–41. The limitations of claim 1 and 10 are fairly similar and Petitioner’s contentions for the two claims are similar as well. See id. Our discussion below focuses primarily on the limitations of claim 1 but also discusses issues regarding claim 10 where they are relevant. Further, our discussion below incorporates our consideration of Patent Owner’s objective evidence of nonobviousness as discussed above. Specifically, when discussing motivation or suggestion to combine, we have considered the entirety of the evidence, both for and against obviousness. a) Preamble: “A system for presenting social-network-provided outputs to a mobile-electronic-device user at a location associated with the user in response to the user’s spoken request” Petitioner argues Wu teaches the preamble of claim 1 except that Wu’s search request is not spoken. Pet. 12–17. For the “spoken” element of the preamble, Petitioner relies on Evermann. Pet. 17. Specifically, Petitioner alleges that Wu in combination with Evermann teaches a “system.” Pet. 31 (citing Ex. 1003, Fig. 1). Petitioner contends Wu’s system includes a number of user mobile devices in communication with a USR server. Pet. 12 (citing Ex. 1003, 4:13–17, 2:21–22, 6:48–55, Figs. 1, 3). According to Petitioner, the USR server stores and aggregates user reviews of various items that originate from and are subsequently searched by members of a social network. Pet. 12–13 (citing Ex. 1003, 11:42–48, IPR2019-01516 Patent 9,128,981 B1 30 12:32–35, 7:6–18, 6:48–55, 7:29–31, 12:39–42, 13:23–42, 13:49–52, 15:44– 46, 16:63–65, 20:14–16, Fig. 7; Ex. 1002 ¶¶ 49, 50, 58). Under Petitioner’s mapping, the USR server corresponds to a social network database, and the aggregated user reviews stored in the USR server correspond to social network information records. Pet. 15 (citing Ex. 1003, 13:36–42). Petitioner relies on Wu’s teaching of a user performing a search from the mobile devices for an item, and Wu’s system providing a result consisting of the aggregated reviews for that item, as teaching the claimed “social- network-provided outputs.” Pet. 16–17 (citing Ex. 1003, 13:23–42, 12:39– 42, 13:49–52, 3:5–37, Fig. 7); Pet. 31 (citing Ex. 1003, 7:10–28, 7:31–34, 10:12–18, 13:11–18, 13:23–42, 16:11–35, 12:39–42, 13:49–52, Fig. 7). Petitioner argues that “because those results are presented via the user’s mobile device, they are presented “at a location associated with the user.” Pet. 31 (emphasis omitted) (citing Ex. 1003, 3:61–65, Fig. 1). We analyze Petitioner’s allegations regarding the spoken request, and Evermann further below. See § II.E.4.b. Patent Owner argues that, according to the preamble, claim 1 “requires the remaining elements to be ‘in response to the user’s spoken request,’” but “Wu does not disclose or suggest spoken inputs” and Wu’s inputs are modified rather than the original input text. PO Resp. 19–20. In addition, Patent Owner argues the modification of Wu with Evermann likewise fails because the outputs would not be in response to a spoken request. PO Resp. 19–20. Generally, a preamble does not limit a claim. Allen Eng’g Corp. v. Bartell Indus., Inc., 299 F.3d 1336, 1346 (Fed. Cir. 2002). Here, we need not decide whether the preamble limits the claims because Petitioner IPR2019-01516 Patent 9,128,981 B1 31 establishes by a preponderance of the evidence that Wu and Evermann teach the preamble of claim 1. Wu teaches a system with a USR server connected to mobile devices through a network. Ex. 1003, 4:13–17, Fig. 1. Wu’s system is “directed towards using a free-form search query of user ratings and reviews and employing a user profile, location information, and/or social network information to modify at least one search query term to obtain a result having an associated universal aggregated rating.” Ex. 1003, 3:38–43. We are persuaded that the user reviews and ratings are social-network-provided outputs to a searching user. The aggregated user ratings and reviews are stored in USR server 106, which enables “management of social networking information, review and rating information for a product, service, business, or the like, and . . . receive[s] and perform[s] refined search queries employing at least some of the social networking, review/rating information, location information, and/or a user profile.” Ex. 1003, 6:50–55. Patent Owner’s argument that Wu does not disclose spoken requests or that the output would not be in response to a spoken request does not account for Petitioner’s reliance on Evermann as teaching spoken requests. Pet. 17. As to Patent Owner’s argument that Wu’s inputs are modified rather than the original input text, we agree instead with Petitioner that in the relied upon embodiment of Wu, location information is used not to change user provided inputs, but to limit the search to a particular location. Reply 4 (citing Ex. 1003, 3:51–54, 10:12–18, 5:3–5; 12:37–42, 18:31–36; Ex. 1042 ¶ 7). Petitioner’s argument is supported by Dr. Chatterjee testimony that “modifying” in Wu simply refers to identifying additional information, such as user location, while keeping the user input search terms unchanged. Ex. 1042 ¶ 7 (citing Ex. 1001, 3:43–54, 10:12–18, 18:31–36). Thus, Wu merely IPR2019-01516 Patent 9,128,981 B1 32 limits the search to the user’s location without modifying the user’s search term. We, therefore, are persuaded that the combination of Wu and Evermann teaches a system for presenting social-network-provided outputs, which consist of aggregated user reviews from members of a social network, to a mobile device at the location associated with the user of the mobile device in response to the user’s spoken request. b) 1.1: “a data input port configured to receive speech information from the mobile-electronic-device user” Petitioner acknowledges that Wu does not expressly disclose receiving search requests in the form of spoken or speech information; however, Petitioner points out Wu does include an audio interface that can transmit, receive, or otherwise process audio. Pet. 17 (citing Ex. 1003, 9:53–58). Petitioner argues Evermann teaches receiving a search request in spoken form from a mobile device. Pet. 18–20 (citing Ex. 1004 ¶¶ 6, 10, 27, 44, Fig. 1; Ex. 1002 ¶¶ 66–71). Specifically, Petitioner relies on Evermann’s teaching of an “open search” mode in which a “search query is provided in a ‘single, continuous utterance’ spoken by the user.” Pet. 18 (citing Ex. 1004 ¶ 32). Petitioner contends Wu’s USR server 106 would be adapted to incorporate Evermann’s ASR functionality, allowing speech information to be received by Wu’s USR server. Pet. 31–32 (Ex. 1002 ¶¶ 80–81). When Wu is modified by Evermann, Petitioner contends Wu’s audio interface 252, or network interface unit 310, would correspond to the claimed “data input port.” Pet. 32 (citing Ex. 1003, 8:36–41; Ex. 1002 ¶¶ 112–115). As explained above, Petitioner provides a rational reason to combine Wu’s text-based search capability with Evermann’s voice recognition technique. Pet. 19–20. IPR2019-01516 Patent 9,128,981 B1 33 Patent Owner does not expressly dispute Petitioner’s contentions regarding this limitation. Based on the evidence presented, we are persuaded that Wu’s audio interface or network interface, when modified by Evermann’s speech-based open search functionality, teaches a data input port configured to receive speech information from a mobile device user. c) 1.2: “a memory configured to store a transcript of the spoken request and metadata associated with the spoken request comprising at least the location during the spoken request” Petitioner argues that the claimed “memory configured to store a transcript” corresponds to volatile memory in Wu’s USR server and that the transmitted request in Wu not only includes the search terms but also the user’s location information. Pet. 20–21 (citing Ex. 1003, 8:4–6, 10:12–18, 3:51–54, 13:15–18, Fig. 1), 32–33 (citing Ex. 1002 ¶¶ 116–117). Petitioner argues the location information would teach the claimed “metadata” associated with the search request. Pet. 21 (citing Ex. 1002 ¶ 74) (emphasis omitted). Petitioner argues Evermann describes generating a ‘transcript of the spoken request’ and expressly confirms that the user’s location . . . would be ‘metadata associated with the spoken request.’” Pet. 21 (citing Ex. 1004, ¶¶ 38–39, 42, 44, Fig. 1; Ex. 1002 ¶¶ 78–79) (emphasis omitted). According to Petitioner, both the text of the search request and the associated metadata would be stored in the volatile memory of Wu’s server. Pet. 24 (citing Ex. 1002 ¶ 81). Based on the evidence presented, notwithstanding Patent Owner’s argument to the contrary, discussed below, we find Petitioner has demonstrated that Wu and Evermann teach limitation 1.2. Wu teaches that when a user inputs search terms, the mobile device “may also provide location information” to transparently modify the search query. Ex. 1003, IPR2019-01516 Patent 9,128,981 B1 34 10:12–18. Evermann teaches that along with transmitting audio information, Evermann also “collects other information relating to the device and the user, . . . refer[red] to as metadata, and transmits both the speech features and the metadata over data connection 108 to transaction server 110.” Ex. 1004 ¶ 38. Evermann discloses that this metadata includes data such as “the geographical location of the device.” Ex. 1004 ¶ 39. Once it receives the audio and metadata, Evermann generates a text representation of the user’s spoken request to search for content. Ex. 1004 ¶ 42. Both Wu and Evermann, therefore, teach metadata associated with the user’s request. Patent Owner argues that Wu and Evermann fail to disclose “a memory structure that stores both the transcript and the metadata, much less that an association exists between the stored metadata and the location of the spoken request that forms the transcript.” PO Resp. 20. Contrary to Patent Owner’s argument, however, under Petitioner’s combination, the volatile memory of Wu stores both the transcript of the spoken request and the metadata. Ex. 1002 ¶¶ 81, 116–117. Patent Owner also argues that Evermann does not generate a transcript and does not use location metadata. PO Resp. 21. Evermann, however, discloses an ASR that performs speech recognition on the audio and returns text of the spoken request. Ex. 1004 ¶ 42. We are persuaded that this text representation of the spoken request teaches the claimed transcript. Further, Evermann discloses that the metadata received from the mobile device is also used along with the audio of the search request to perform the search, thus contradicting Patent Owner’s argument that Evermann does not use location metadata. Ex. 1004 ¶ 42 (“ASR Server 112 performs speech recognition on the audio, using the metadata when it can in order to improve recognition accuracy.”). IPR2019-01516 Patent 9,128,981 B1 35 Finally, Patent Owner argues that Wu does not teach that metadata is associated with the spoken request because “location information in Wu is independently determined of the user’s freeform input” and is provided in a separate message. PO Resp. 21 (citing Ex. 2006 ¶¶ 69–72); see also Pet. 31 (arguing that “Wu discloses modifying searches but does not render obvious searching a social network database using both the location and the transcript” and that “Wu merely refines or modifies a search but does not utilize the location together with the search query to search.”). We disagree. Wu explains that the user’s current location may modify the search query to, for example, identify a city, and that this is used as part of the search itself. Ex. 1003, 3:50–56. Wu, therefore, discloses that the location information is associated with the spoken request. Although Wu states that “location information may be provided in a separate message,” it also states in the same sentence that location information may be “sent as part of another message, or even as a combination of the above.” Ex. 1003, 5:11–13. This clearly implies that Wu’s location information may be sent in the same message as that of the request itself. Regardless, location information being sent in a separate message does not contradict or preclude that location information from being associated with the request or from being stored in memory. Moreover, Petitioner also relies on Evermann as teaching “metadata associated with the spoken request.” Pet. 21 (citing Ex. 1004, ¶¶ 38–39, 42, 44, Fig. 1; Ex. 1002 ¶¶ 78–79). We agree with Petitioner. Evermann teaches that metadata is used along with the spoken request to perform the search. Ex. 1004 ¶ 42. The metadata can improve recognition accuracy and can be used to select certain content providers and advertising providers, which use the metadata to service the search request and to provide potential IPR2019-01516 Patent 9,128,981 B1 36 advertisements and pricing information. Ex. 1004 ¶ 42. In order to use metadata to select appropriate content and advertising providers for the search request, the metadata must be associated with the search request. Thus, Evermann also teaches “metadata associated with the spoken request.” Based on the evidence presented, we agree with Petitioner that the aforementioned location information in both Wu and Evermann teaches the claimed “metadata associated with the spoken request comprising at least a location associated with the user during the spoken request.” We also agree that Evermann’s text representation of the spoken search teaches “the transcript of the spoken request,” as claimed. d) 1.3: “an interface port to a social network database, configured to transmit a request to mine information of the social network database based on the transcript and the metadata, and to receive social network information from the social network database based on the transmitted request” Petitioner argues Wu’s database 352 in USR server 106, which stores user reviews of various items originating from members of a social network, corresponds to the claimed “social network database.” Pet. 33 (citing Ex. 1003, 11:42–48, 7:16–18, 6:47–55, 7:29–31, 12:32–35, 15:44–46, 16:63–65, Fig. 7). The search request in the Wu-Evermann combination queries the user review database for user reviews based on both the spoken request itself and metadata associated with the search request, as explained above. Pet. 34–35 (citing Ex. 1003, 7:10–28, 3:38–43, 3:51–54, 10:12–18, 13:11–18, 16:11–13, 16:7–10, 16:27–35; Ex. 1004 ¶¶ 38–39, 42). As to the claimed “interface port,” Petitioner argues that it would have been obvious to have an interface between the processor of Wu’s server 106 and database 352. Pet. 35–36 (citing Ex. 1002 ¶ 127). Alternatively, Petitioner argues that “Wu expressly discloses that components of the server 106 could also be IPR2019-01516 Patent 9,128,981 B1 37 distributed across multiple distinct computers” (Pet. 36 (citing Ex. 1003, 7:44–52), and that “it would have been obvious that each of the distinct computers . . . would include an ‘interface port’ such as a network interface unit.” Pet. 36–37 (citing Ex. 1003, 10:38–44, Fig. 3; Ex. 1002 ¶ 129–132). Patent Owner argues that there is wide difference between searching a database and mining a database. PO Resp. 23. According to Patent Owner, a “search implies receiving information stored as it is stored in the database for formatting and display, whereas mining implies adding additional value to the retrieved stored data to reveal further insights into the data.” PO Resp. 23 (citing Ex. 2006 ¶¶ 90–95). Based on the evidence presented, we are persuaded that Wu and Evermann teach limitation 1.3. Specifically, we are persuaded that Wu’s database 352 is a social network database. Wu expressly discloses that users can input user reviews and ratings for a product, such as Trader Joe’s ketchup, and that “[w]hen another person, such as the user’s spouse, friend, or the like, performs a free form search query for ‘ketchup,’ the searcher’s social network’s inputs to search/rating/review data store 352 may be used to search for and provide results.” Ex. 1003, 13:23–33. Thus, data store 352 contains information that is provided and accessible by members of a social network, and therefore teaches the claimed “social network database.” We also are persuaded that one of ordinary skill in the art would have understood that there would be an interface port between various components of a computer system, such as between Wu’s processor and database or between separate and distinct computers communicating over a network that implement the server and database. Petitioner’s argument is supported by Dr. Chatterjee’s testimony that an interface between Wu’s IPR2019-01516 Patent 9,128,981 B1 38 processor and database would enable data to be read from and written to the database. Ex. 1002 ¶ 127. Finally, Patent Owner’s argument that Wu and Evermann do not disclose “min[ing] information,” as claimed, because of a difference between the terms “mining” and “searching” fails to overcome Petitioner’s persuasive showing for this limitation. Patent Owner provides insufficient evidence that any distinction between the two terms would distinguish the teachings of Wu and Evermann from the claims. For example, Patent Owner’s expert, Dr. Melendez, relies on the ʼ981 Patent as indicating a “wide difference between searching a database and mining a database.” Ex. 2006 ¶ 91 (citing Ex. 1001, 15:22–26). We agree, however, with Dr. Chatterjee’s testimony that this portion of the ʼ981 Patent indicates instead, that the ʼ981 Patent uses the two terms synonymously. Ex. 1042 ¶ 46; see Ex. 1001 15:22–26 (“The Centralized Community Search database of Location Blogs . . . may be searched or mined based on social network or type of information.”) e) 1.4: “at least one processor configured to transmit the request through the interface port dependent on at least the transcript and the metadata, to receive the social network information from the interface port” Petitioner argues that the interface port identified above in limitation 1.3 would be used to transmit the request to database 352 from Wu’s processor and to receive the search results from database 352, at Wu’s processor. Pet. 38 (citing Ex. 1002 ¶ 133). The request, as explained above with respect to limitation 1.2, would query database 352 using the text representation of the user’s spoken request and location metadata, and thus IPR2019-01516 Patent 9,128,981 B1 39 teach that the request would be transmitted dependent on the transcript and the metadata, as claimed. Pet. 38. Patent Owner does not expressly dispute Petitioner’s arguments with respect to this limitation. Based on the evidence presented, we are persuaded that Wu’s processor is configured to transmit the request through the interface port to database 352 and to receive back the search results through the claimed interface port as well. f) 1.5: “a communication port configured to communicate at least a portion of the social-network information to the user” Petitioner argues that because search results are output to a user over a network, Wu’s server 106 would include a “communication port” such as network interface unit 310, for sending data to the mobile device. Pet. 40 (citing Ex. 1003, 10:38–44; Ex. 1002 ¶ 143). Alternatively, Petitioner argues the input/output interface on the user’s mobile device also corresponds to the claimed “communication port.” Pet. 40 (citing Ex. 1003, 8:61–66, Fig. 2). Patent Owner does not expressly dispute Petitioner’s arguments regarding the claimed “communication port.” Based on the evidence presented, we determine Petitioner has demonstrated that one of ordinary skill would understand that because search results are communicated to a user over a network, Wu’s server would include a communication port and that, in order to receive those results, the user’s mobile device would also include a communication port. See Ex. 1002 ¶¶ 143–144. g) 1.6: “wherein the social network database comprises a plurality of social network records, the at least one processor being further configured to rank the received social network information comprising a plurality of received social network records dependent on at least one social network ranking factor, the IPR2019-01516 Patent 9,128,981 B1 40 communication port being further configured to output at least a portion of the social network records in a manner dependent on the at least one social network ranking factor” As explained above, Petitioner argues that the aggregated reviews of products stored in Wu’s database 352 correspond with the claimed “social network records” and “social network database,” respectively. Pet. 38 (citing Ex. 1003, 13:36–42, 17:14–17, 11:41–49, 7:31–34, 4:1–3, 3:58–61). Petitioner argues that the combination of Wu and Evermann teaches ranking the search results dependent on at least one social network ranking factor. Pet. 39. For example, Petitioner argues that Wu ranks the search results, prioritizing those more likely to be relevant or valuable. Pet. 28 (citing Ex. 1003, 14:12–15). Petitioner also argues that it would have been obvious to one of ordinary skill in the art to use the same processor of Wu’s server that selected and retrieved the results from storage to rank the results. Pet. 28 (citing Ex. 1003, 10:28–30, Fig. 3; Ex. 1002 ¶ 97). Patent Owner argues that “‘social network ranking factor’ must be based on interpersonal relationships in a social network database.” PO Resp. 26. Patent Owner argues Wu’s user rating is not a social network ranking factor because it is not based on relationships between users. PO Resp. 26 (citing Ex. 2006 ¶¶ 97–106). Based on the evidence presented, we determine Petitioner has demonstrated that the combination of Wu and Evermann teaches limitation 1.6. For reasons already explained above, we agree that Wu’s database 352 and aggregated reviews teach the claimed “social network database” and “social network records,” respectively. We are also persuaded that Wu teaches ranking the search results based on a social network ranking factor. Wu’s aggregated reviews and rating are submitted by users and can be IPR2019-01516 Patent 9,128,981 B1 41 searched by others in the user’s social network. Ex. 1003, 13:23–33. These aggregated reviews can then be “ranked by relevance to the query, how meaningful the phrase is, and/or an averaged rating” when output in response to a query. Ex. 1003, 14:12–15. By teaching that search request results may be ranked by a rating provided by others in a user’s social network, we determine Wu teaches ranking social network information “dependent on at least one social network ranking factor.” Patent Owner’s argument that Wu’s user rating is not a social network ranking factor is based on an improper construction requiring the social network ranking factor to be based on interpersonal relationships. See supra § II.C.4. Wu explicitly teaches that the ratings are provided by the “user’s spouse, friend, or the like,” all of whom are in relationship to the user performing the search and in the user’s social network. Ex. 1003, 13:23–33. Thus, ranking the results based on these ratings teaches ranking “dependent on at least one social network ranking factor.” 5. Conclusion For the reasons explained above, we determine that Petitioner has shown sufficiently that the combination of Wu and Evermann teach all of the limitations of each of claims 1 and 10 of the ’981 Patent. We conclude, on this record, that Petitioner has shown the unpatentability of claims 1 and 10 as obvious over Wu and Evermann by a preponderance of the evidence. F. Obviousness over Wu, Evermann, Davis, and Burke Petitioner contends claims 1, 4–16, and 18–19 would have been obvious over Wu, Evermann, Davis, and Burke. Pet. 41–69. We provide a brief overview of Davis and Burke and then analyze whether the prior art IPR2019-01516 Patent 9,128,981 B1 42 references teach the limitations of claims 1, 4–16, and 18–19 in the sections below. 1. Overview of Davis Davis is titled “Building Research Tools with Google for Dummies” and is targeted at explaining to the reader how to use Google more effectively as a research tool. Ex. 1010, 19. Davis aims to help the reader “[u]nderstand the various Google tools, and how they can help satisfy your research needs”; “[u]se Google’s custom search syntax to effectively find information”; “[b]ecome a better researcher by learning tips, tricks, and techniques honed by top research professionals”; “[l]earn how Google works, and what mechanisms it uses to respond to search queries”; and “[h]arness the power of the Google APIs to build your own research tools.” Id. 2. Overview of Burke Burke discloses a “search entry system” where the search terms “can be autocompleted based on entries in a search history query log.” Ex. 1011 ¶ 12. The query log can store one or more previously submitted queries. Id. When a user submits a new query, “an autocompletion module can search the query log for queries that include the new query terms or partial query terms.” Id. ¶ 26. Figure 2 of Burke, depicted below, is a functional block diagram of an embodiment of an information retrieval system including a search client having a query log autocompletion search entry. 9 Citations to Davis refer to the original page numbers of the book. IPR2019-01516 Patent 9,128,981 B1 43 Figure 2 depicts a search client, which may submit one or more search queries to query server 220. Ex. 1011 ¶ 42. The query server 220 then stores the query in query log 230. Ex. 1011 ¶ 42. The queries stored in query log 230 can be used to “determine if an identical query has recently been processed by the query server.” Ex. 1011 ¶ 43. If so, the query server “may have access to the search results without performing an additional search.” Ex. 1011 ¶ 43. The query server can also access and search database 240 for one or more records matching search criteria. Ex. 1011 ¶ 44. 3. Analysis of Claims 1, 10 and 16 For the arguments regarding independent claims 1, 10 and 16 as obvious over Wu, Evermann, Davis, and Burke, Petitioner relies largely on the same arguments set forth above regarding claims 1 and 10 as obvious over Wu and Evermann. See Pet. 42–57. Petitioner cites to Davis and Burke, however, “to provide the claimed ‘transcript’ in the event Patent IPR2019-01516 Patent 9,128,981 B1 44 Owner attempts to distinguish [the combination of Wu and Evermann] by arguing that a ‘transcript’ must be a structured document that is persistently stored.” Pet. 41 (emphasis omitted). Thus, Petitioner relies on Davis, which it argues discloses an XML document used to encapsulate a search request, and also relies on Burke, which it argues discloses that the “XML document would be persistently stored in a ‘query log.’” Pet. 41. According to Petitioner, the XML document qualifies as a “transcript” because it is similar to the searchable text document described in the ʼ981 Patent. Pet. 41 (citing Ex. 1001, 7:54–58, 9:10–14, 12:27–31, 13:7–12). In Petitioner’s combination, Wu’s server would generate the text of a user’s spoken search using Evermann’s technique. Pet. 47. Wu would then place the text and metadata, such as location, in an XML document as taught by Davis. Pet. 47, 50 (citing Ex. 1002 ¶¶ 166, 181). The XML document would be stored in a query log in Wu’s server. Pet. 47. Claim 16 differs from claims 1 and 10 with respect to two limitations. The first recites “automatically generating a query of a social information search engine comprising user ratings of places.” Here, Petitioner adds that Wu teaches searching for user ratings of places. Pet. 51–52 (citing Ex. 1003, 1:16–19, 7:20–24). The second limitation recites “ranking the plurality of social network records . . . according to at least one social network ranking factor comprising at least a physical distance from the location.” For this limitation, Petitioner relies on Davis which it argues describes searching for a “Chinese Restaurant” and displaying results with local results listed before results further away. Pet. 54–55 (citing Ex. 1010, 17–18, Fig. 1–5, 1–6; Ex. 1002 ¶ 194). Petitioner argues that one of ordinary skill would have been motivated to combine Davis and Burke with Wu and Evermann. Pet. 47–48. For IPR2019-01516 Patent 9,128,981 B1 45 example, Petitioner argues Davis touts the advantages of XML which would have provided the motivation to combine Davis with Wu and Evermann. Pet. 47. Specifically, Petitioner argues XML was known to be “easy to understand and to use,” “read[able] by both humans and machines,” and “very flexible.” Pet. 47–48 (citing Ex. 1010, 244). Dr. Chatterjee testifies that using an XML document for sending a search request would have benefited from including improved data typing and structure, greater flexibility, and extensibility. Ex. 1002 ¶¶ 168–169. An additional advantage would have been that the metadata, such as location, could also be provided in the XML document along with the text of the search request. Pet. 48 (citing Ex. 1002 ¶ 170). Finally, Petitioner argues that Burke’s query log functionality would have been known to facilitate benefits in billing, advertising, and future searches. Pet. 48 (citing Ex. 1002 ¶ 170). Patent Owner argues that Davis and Burke do not cure the deficiencies of Wu and Evermann because “neither Burke nor Davis addresses the limitations that the transcript must be associated with metadata including location of a spoken request and that both the transcript and metadata be stored and used as an association to request mining of a social network database.” PO Resp. 21–22. Patent Owner emphasizes that a person of ordinary skill in the art “would not have stored a transcript of speech in the form of a searchable XML document, and even if they did, the searchable XML document still would not render obvious associating location metadata with the transcript or storing both in memory.” PO Resp. 22. As explained above, we find the combination of Wu and Evermann teaches associating metadata with the spoken request, and also using both to mine a social network database, and thus disagree that Wu and Evermann are deficient. See supra §§ II.E.4.b–c. Further, even when considering the IPR2019-01516 Patent 9,128,981 B1 46 additional teachings of Davis and Burke, as in Petitioner’s combination, “the text of the user’s spoken request and associated metadata, would be stored within a single XML document.” Pet. 46 (citing Ex. 1002 ¶ 166). As Petitioner point out, “Burke expressly contemplates that a user query can include metadata in addition to a text string.” Pet. 46 (citing Ex. 1011 ¶¶ 1, 27). Thus, in Petitioner’s combination the metadata would be associated with the spoken request and the motivation to do so is expressly provided by Burke. We determine Petitioner has demonstrated that the combination of Wu, Evermann, Davis, and Burke teaches all of the limitations of each of claims 1, 10 and 16. 4. Claims 7 and 11 Claim 7 depends from claim 1, and claim 11 depends from claim 10. Claim 7 recites “further comprising ranking the mined information based on criteria comprising a physical distance from the location,” and claim 11 recites “wherein said ranking comprises ranking a plurality of the selected social network information records on criteria comprising a physical distance from the mobile-electronic device user’s location.” For these claims, Petitioner relies on the same arguments made for claim 16 (see Pet. 57), namely that Wu teaches that its system may be used to search for places (Pet. 51–52 (citing Ex. 1003, 1:16–19, 7:20–24)), and that Davis provides an example of a user searching for a “Chinese Restaurant” and getting results that are listed from closest to furthest away from the user location (Pet. 54– 56 (citing Ex. 1010, 17–18, Figs. 1–5, 1–6; Ex. 1002 ¶¶ 194–196)). Patent Owner argues “the ‘location’ of claims 7 and 11 is ‘the location during the spoken request’ of claim 1, and Davis fails to disclose this IPR2019-01516 Patent 9,128,981 B1 47 location because it uses an arbitrary location assigned by the Google search engine.” PO Resp. 29. We disagree with this argument. Because claims 7 and 11 depend from independent claims 1 and 10, respectively, the recited “location” in claim 7 and the recited “the mobile-electronic device user’s location” in claim 11 are the locations recited in the independent claims. Petitioner relies on Wu and Evermann, not Davis, as disclosing these respective locations. See Pet. 20–21 (citing Ex. 1003, 10:12–18), 50. Thus, Patent Owner’s argument that the location is an “arbitrary location assigned by the Google search engine” (PO Resp. 29) does not address Petitioner’s arguments. Moreover, even in Davis, the location is provided by the user in the form of a zip code, not assigned by the Google search engine. See Ex. 1010, Fig. 1-6. Thus, one of ordinary skill in the art would understand that the distances displayed in Davis are distances from a location of interest to the user, and that when combined with Wu and Evermann, the distances are from the location of the user themselves. We are persuaded that Davis teaches ranking search results by distance (Ex. 1010, 17–18) and when combined with Wu, Evermann, and Burke, teaches ranking based on physical distance from the location of the user. Based on the evidence presented, we determine Petitioner has demonstrated the combination of Wu, Evermann, Davis, and Burke teaches the limitations of claims 7 and 11. 5. Claims 8, 9, 13, 14, and 18 Claims 8 and 9 depend from claim 1, claims 13 and 14 depend from claim 10, and claim 18 depends from claim 16. Claims 8, 13, and 18 add limitations requiring communicating the social network information within a location or spatial mapping user interface. Claims 9 and 14 add limitations IPR2019-01516 Patent 9,128,981 B1 48 requiring communication of the social network information within Google maps. For claims 8, 9, 13, 14, and 18, Petitioner relies on the Google maps application as discussed in Davis as an exemplar of the “location mapping” (or “spatial mapping”) user interface recited in claims 8, 13, and 18 and also as disclosing the recited “Google maps” limitation of claims 9 and 14. Pet. 58–59. We address all of these claims together because Petitioner’s arguments indicate that Google maps, in its view, is sufficient to meet the further limitations of each of these dependent claims. See Pet. 58–60. As before, Petitioner relies on Wu as teaching the recited “social network information” and “social network provided records.” See Pet. 58 (“Each [claim] would require the search results in Wu, i.e., ‘the social- network information,’ ‘the at least one of the selected social network provided records,’ or ‘the ranked plurality of records,’ to be communicated or displayed within” the map.). Petitioner relies on Davis as teaching displaying search results in Google maps. Pet. 58–60 (citing Ex. 1010, 18, Fig. 1-6). IPR2019-01516 Patent 9,128,981 B1 49 We agree. Figure 1-6 of Davis is depicted below: Figure 1-6 of Davis shows search results for Chinese restaurants being displayed within Google maps, as recited in claims 9 and 14, which is a spatial mapping interface as recited in claims 8, 13, and 18. When combined with the search results of Wu, and the previously relied upon teachings of Evermann, and Burke, we are persuaded that this disclosure would teach communicating the social network information of Wu in a map interface and specifically, in Google maps. Patent Owner argues “Davis does not disclose ‘communicating the social-network-information within [a] map’ or ‘on [a] map’ as claimed because Davis only maps the name of the business and its address and location—not social-network information.” PO Resp. 29 (alterations in original) (citing Ex. 2006 ¶¶ 137–141). Patent Owner’s argument is misdirected, however, as Petitioner relies on Wu as teaching the claimed social network information, not Davis. See Pet. 13–17, 51–53. Specifically, in the combination presented by Petitioner, the social network information IPR2019-01516 Patent 9,128,981 B1 50 consists of user ratings and reviews of places (e.g., restaurants) provided by members of a social network. See Pet. 51–53 (citing Ex. 1003, 7:20–24, 1:16–19). Based on the evidence presented, we determine Petitioner has demonstrated the combination of Wu, Evermann, Davis, and Burke teach the limitations of claims 8, 9, 13, 14, and 18. 6. Claim 19 Claim 19 depends from claim 18 and recites “further automatically searching a set of persistently stored transcripts and associated metadata with the at least one automated processor.” Petitioner makes the same arguments it made for claims 1 and 10, namely that the XML document would encapsulate both the text of the spoken search query and the associated metadata, as taught by Evermann and Davis. Pet. 60–61 (citing Ex. 1011 ¶¶ 42–43, 48; Ex. 1002 ¶¶ 166, 211). Petitioner contends that the XML document would be persistently stored in a query log as taught by Burke and automatically searched in response to a subsequent search request. Pet. 61 (citing Ex. 1011 ¶¶ 42–43, 48, Fig. 2; Ex. 1002, ¶ 212). We agree with Petitioner that the XML document of Davis stored in a query log, as taught by Burke, would teach the claimed “persistently stored transcripts and associated metadata” and that searching these XML documents in response to subsequent searches would teach automatically searching a set of persistently stored transcripts and associated metadata. Patent Owner focuses on the dependency between claim 19 and claim 16, arguing that “[c]laim 16 is structured such that a transcript of user- generated speech is obtained first, and then a query is separately automatically generated.” PO Resp. 35. The purpose of this claim IPR2019-01516 Patent 9,128,981 B1 51 language, according to Patent Owner, is to cover the embodiment disclosed in the specification of the ’981 Patent wherein the user submits voice generated user reviews to the social network database for later search. PO Resp. 35. Patent Owner argues, “Claim 19 thus requires searching persistently stored transcripts of user reviews—not searching persistently stored transcripts of queries.” PO Resp. 35. We disagree with Patent Owner’s argument. As Petitioner points out, claim 19 does not contain an antecedent reference to claim 16 for the “set of persistently stored transcripts.” Reply 25 (emphasizing that claim 19 recites “a set of persistently stored transcripts” not “the set of persistently stored transcripts”). Further, nothing in claims 16 or 19 requires the transcripts to be of user reviews. Based on the evidence presented, we determine Petitioner has established sufficiently that the combination of Wu, Evermann, Davis, and Burke teaches the limitations of claim. 19. 7. Claim 4 Claim 4 depends from claim 1 and further recites “the social network ranking factor comprising user review ratings of respective restaurants” where the ratings are output “dependent on the respective user review ratings and a geographical distance of the restaurants from the user.” Petitioner relies on Wu’s disclosure of searching for “where to eat” and on Davis’s disclosure of searching for restaurants as teaching the limitations of claim 4. Pet. 62 (citing Ex. 1003, 1:16–19, 7:20–24; Ex. 1010, 17–18). Petitioner notes that Wu teaches ranking search results based on user rating (Pet. 63 (citing Ex. 1003, 14:12–15)), while Davis ranks restaurants based on distance (Pet. 63 (citing Ex. 1010, 17–18, Figs. 1–5, 1–6)). Combining the IPR2019-01516 Patent 9,128,981 B1 52 two teachings, Petitioner argues that it would have been obvious to display the search results based on ranked user ratings and on distance from the searching user. Pet. 63–64 (citing Ex. 1003, 3:61–65; Ex. 1010, 17–18; Ex. 1002 ¶¶ 219–22). Patent Owner argues, as it did for the independent claims, that user review ratings “are not social network ranking factors as properly construed because rankings are based on ratings assigned by individual users and not based on interpersonal relationships of a social network database.” PO Resp. 28 (citing Ex. 2006 ¶¶ 116–123). This argument fails to overcome Petitioner’s persuasive showing for this limitation for the same reasons stated above in our analysis of claim 1 as obvious over Wu and Evermann. See supra §§ II.E.4.g. Patent Owner further argues that none of the prior art discloses geographical distances, because “Wu’s distance refers to error in the accuracy of the GPS system” and “Davis suggests a user entered zip code.” PO Resp. 28. We disagree with Patent Owner’s argument. Davis clearly discloses geographical distances of restaurants. Ex. 1010, 18, Fig. 1-6. We are persuaded that Davis when combined with Wu and Evermann would teach ranking restaurants based, in part, on their distance from the mobile device of the user. Based on the evidence presented, we determine Petitioner has established sufficiently that the combination of Wu, Evermann, Davis, and Burke teaches the limitations of claim 4. 8. Claim 15 Claim 15 depends from claim 10 and further recites “outputting the user reviews of the plurality of restaurants ranked based on at least the IPR2019-01516 Patent 9,128,981 B1 53 distance from the user and the user review ratings.” Petitioner relies on the same arguments and evidence presented for claim 4. See Pet. 61–63. Patent Owner argues the scope of claims 4 and 15 are different, in that claim 4 requires a distance of the user from a restaurant, while claim 15 requires a distance of the user from a user review. PO Resp. 32 (citing Ex. 2006 ¶¶ 162–165). Patent Owner argues that “Petitioners do not allege that ranking results based on a user review’s distance from the searching user are disclosed or suggested by the art—nor is it.” PO Resp. 32 (citing Ex. 2006 ¶¶ 162–165). Petitioner responds that an interpretation “requiring a single ranking factor in the form of a distance between the user and the location where a review author made the review . . . is contradicted by [Patent Owner’s] own expert, who interprets claim 15 as reciting two different ranking factors in a manner similar to claim 4.” Reply 24 (citing Ex. 2006 ¶ 122). The dispute between the two parties centers around whether claim 15 requires ranking based on a single factor (as argued by Patent Owner), i.e., the distance between the user’s current location and the location where the user review took place, or whether claim 15 requires ranking based on two factors (as argued by Petitioner): (1) the distance of the restaurant from the location of the user and (2) the user review ratings. We find Petitioner’s interpretation to be more reasonable. Patent Owner’s interpretation assumes that a “user review rating[]” has a location, but the phrase “user review ratings” does not imply a location on its own. If the inventor had intended for claim 15’s ranking to be based on a single factor, the claim could have made this clear, for example, by stating that the factor is the distance from the user and the location where the user review rating took place or by IPR2019-01516 Patent 9,128,981 B1 54 stating that the factor is the distance between the user and the user review rating. Because we interpret claim 15 to rank based on two factors: (1) the distance of the restaurant from the location of the user and (2) the user review ratings, we are persuaded that the combination of Wu, Evermann, Davis, and Burke teaches the limitations of claim 15 for the same reasons discussed with respect to claim 4. 9. Claims 5, 6, and 12 Claims 5 and 6 depend from claim 1. Claim 12 depends from claim 10. Claim 5 recites “outputting the mined information based on a chronological order.” Claim 6 recites “outputting the mined information based on a type of social information within a respective social network record.” Claim 12 recites “wherein the at least one social network ranking factor comprises at least one of reverse chronological order, social network credibility, social network popularity, and type of social information, and said communicating comprises outputting.” Petitioner argues Davis discloses ranking posts by users of an online group in reverse chronological order. Pet. 65 (citing Ex. 1010, 133). Petitioner also argues Davis teaches prioritizing certain types of information in a list of search results. For example, Petitioner argues Davis teaches displaying local results at the top of the list of results. Pet. 66–67 (citing Ex. 1010, 18, 58–59). Petitioner argues that this disclosure teaches outputting information based on type of social information. Pet. 68 (citing Ex. 1002 ¶¶ 230–231). With respect to claim 5, Patent Owner argues that ranking in reverse chronological order, as Petitioner alleges Davis discloses, does not teach or IPR2019-01516 Patent 9,128,981 B1 55 render obvious outputting in chronological order. PO Resp. 28. With respect to claim 6, Patent Owner argues, as it has for the independent claims, that the prior art does not disclose that the information is social information. PO Resp. 29. Patent Owner does not make any separate arguments with respect to claim 12. See PO Resp. 31. We are persuaded that Davis, which discloses “organiz[ing] posts in reverse chronological order,” combined with Wu, Evermann, and Burke, would render obvious ranking results based on “chronological order,” as recited in claim 5, and ranking based on “reverse chronological order,” as recited in claim 12. We agree with Petitioner that a teaching of reverse chronological order teaches one of skill in the art to order results based on time, either in the forward or reverse direction. We are also persuaded that Davis’s disclosure of prioritizing local results teaches “outputting the mined information based on a type of social information within a respective social network record,” as recited in claim 6. We disagree with Patent Owner’s argument that the prior art does not disclose that the information is social information. As explained above, we find Wu teaches the claimed social information. See supra §§ II.E.4.d, II.E.4.g. Based on the evidence presented, we determine Petitioner has established sufficiently that the combination of Wu, Evermann, Davis, and Burke teaches the limitations of claims 5, 6, and 12. 10. Conclusion We conclude that Petitioner has proved by a preponderance of the evidence that claims 1, 4–16, 18, and 19 would have been obvious over Wu, Evermann, Davis, and Burke. IPR2019-01516 Patent 9,128,981 B1 56 G. Obviousness over Wu, Evermann, Davis, Burke, and Buyukkokten Petitioner contends claims 1–19 would have been obvious over Wu, Evermann, Davis, Burke, and Buyukkokten. Pet. 69–77. We provide a brief overview of Buyukkokten and then analyze whether the prior art references teach the limitations of claims 1–19 in the sections below. 1. Buyukkokten Buyukkokten relates to “methods and systems for rating associated members in a social network.” Ex. 1005 ¶ 2. Buyukkokten describes an embodiment that presents a ratings interface to a user for rating a member of a social network. Ex. 1005 ¶ 5. The rating interface provides the user with the ability to rate the member in one or more categories. Ex. 1005 ¶ 5. These categories may include “humor, intelligence, fashion, trustworthiness, sexiness, and coolness.” Ex. 1005 ¶ 29. 2. Claims 1, 10, and 16 Petitioner builds on its prior arguments by adding Buyukkokten, which it cites to address a possible argument by Patent Owner that “social network ranking factor” must be an attribute or characteristic of the posting social network member, not just any ranking factor used to rank social network information. Pet. 70 (emphasis omitted). Thus, Petitioner argues that the search results presented by the prior combination of Wu and Evermann would be ranked “according to one or more characteristics of the social network members who initially provided those reviews, including their trustworthiness, popularity, and/or distance from the searching user.” Pet. 70. Ultimately, Patent Owner does not specifically argue that a “social network ranking factor” must be an attribute or characteristic of the posting IPR2019-01516 Patent 9,128,981 B1 57 social network member as predicted by Petitioner, but Patent Owner does argue that the “social network ranking factor” must be “an interpersonal metric, based upon relationships between members, that is derived from a social network database.” PO Resp. 19. In our discussion of the construction of “social network ranking factor” (supra § II.C.4), we explained that the term is broader than what Patent Owner proposes and explicitly includes, at least, “social network credibility” and “social network popularity,” which Petitioner argues is disclosed by Buyukkokten. For the reasons explained below, we determine that Byukkokten teaches the claimed social network ranking factor and that, therefore, the combination of Wu, Evermann, Davis, Burke, and Buyukkokten teaches the limitations of claims 1, 10, and 16. For example, Petitioner argues that Wu teaches that “an individual’s social network can be utilized to find more relevant connections.” Pet. 71 (citing Ex. 1003, 1:39–45, 3:5–11). According to Petitioner, Wu discloses that a user can search for reviews of products authored by members of the user’s social network and that these reviews may be ranked by relevance, which can depend on attributes of the review author. Pet. 71 (citing Ex. 1003, 1:39–45, 3:5–11, 13:23–42, 14:12–15). Wu, however, does not expressly disclose how such attributes for a member are determined, according to Petitioner. Pet. 71. Petitioner argues that this concern is addressed by Buyukkokten, “which teaches a way for a user to assign ratings for different attributes to each member in [the] social network.” Pet. 72 (citing Ex. 1005 ¶¶ 29, 47, 48). Petitioner argues that it “would have been obvious for the ranking of member-authored reviews by ‘relevance’ . . . to be weighted by each author’s attribute ratings as disclosed in Buyukkokten.” Pet. 73 (citing Ex. 1003, 14:12–15). Petitioner argues that Buyukkokten’s IPR2019-01516 Patent 9,128,981 B1 58 disclosure of weighting based on a member’s trustworthiness or popularity (Pet. 73 (citing Ex. 1005 ¶¶ 47, 51)), is similar to embodiments of the ʼ981 Patent disclosing ranking based on social network credibility or social network popularity (Pet. 73 (citing Ex. 1001, 15:33–37). For claim 16, Petitioner argues that the combination of Buyukkokten with Wu, Evermann, Davis, and Burke would teach or suggest ranking user reviews by distance of the authoring reviewer from the user. Pet. 74. Although Buyukkokten does not disclose “distance” as one of the rating categories, it does track the location of its social network members (Ex. 1005 ¶¶ 23, 28, 31), and Petitioner contends that, therefore, it would have been obvious to one of ordinary skill to use distance rating for each member of the social network. Pet. 74 (citing Ex. 1002 ¶ 244). Because Wu suggests that the relevance of a particular user review will depend on how well the user “know[s] or trust[s]” the review author, and how likely the review author is to share the user’s frame of reference (Pet. 74–75 (citing Ex. 1003, 1:39–45, 3:5–11)), Petitioner argues that one of ordinary skill in the art would have been motivated to combine Wu with Buyukkokten and use attributes such as trustworthiness and popularity disclosed in Buyukkokten, which serve as good indicators of how well that member is known and trusted. Pet. 74–75 (citing Ex. 1002 ¶ 247). Similarly, Petitioner argues, social network members living closer to the user are more likely to share the user’s frame of reference, and, thus, one of ordinary skill would have been motivated to add distance as a ranking factor. Pet. 75 (citing Ex. 1002 ¶¶ 247–248). Patent Owner relies on the same arguments it made with respect to the other grounds and does not present separate arguments with respect to obviousness of the independent claims under this ground. Based on the IPR2019-01516 Patent 9,128,981 B1 59 evidence presented, we agree with Petitioner that the combination of Wu, Evermann, Davis, Burke, and Buyukkokten teaches the limitations of independent claims 1, 10, and 16. We are persuaded that one of ordinary skill in the art would have been motivated to combine Buyukkokten with Wu, Evermann, Davis, and Burke. As explained above, Petitioner explains that Wu teaches that product reviews from members of a user’s social network would be more relevant to the user and coincide with the user’s frame of reference. Pet. 71 (citing Ex. 1003, 1:39–45, 3:5–11, 13:23–42, 14:12–15). Wu also teaches that shoppers seek reviews from others that the shopper knows and trusts. Pet. 74–75 (citing Ex. 1003, 1:39–45, 3:5–11). Petitioner explains that attributes such as the trustworthiness and popularity of a member of the social network, as taught by Buyukkokten, would help the user in trusting the reviews returned by Wu. Pet. 74–75 (citing Ex. 1003, 1:39–45, 3:5–11; Ex. 1002 ¶ 247). Petitioner has, therefore, articulated an adequate reason with rational underpinning for combining the references and also explained sufficiently how the references would be combined. Further, as explained above in our analysis of the previous grounds, we determine that the combination of Wu and Evermann, and the combination of Wu, Evermann, Davis, and Burke, teach the limitations of independent claims 1, 10, and 16, including the claimed social network ranking factor. The addition of Buyukkokten’s disclosure of a member’s trustworthiness or coolness further confirms that the prior art teaches the claimed social network ranking factor. We agree with Petitioner that Buyukkokten’s weighting of members based on their trustworthiness or coolness (Ex. 1005 ¶¶ 29, 47, 51) mirrors the ʼ981 Patent’s disclosure of “social network credibility” and “social network popularity,” and explicitly discloses social network ranking factors (Ex. 1001, 15:33–37). Therefore, IPR2019-01516 Patent 9,128,981 B1 60 we determine that Petitioner has demonstrated that the combination of Wu, Evermann, Davis, Burke, and Buyukkokten teaches each limitation of claims 1, 10 and 16. 3. Claims 2, 3 and 17 Claims 2 and 3 depend from claim 1 and add that the social network ranking factor comprises a “social network credibility” (claim 2) and “social network popularity” (claim 3), respectively. Claim 17 depends from claim 16 and adds that the social network records are ranked based at least on one factor selected from the group consisting of “social network credibility and a social network popularity.” Petitioner contends, “The claimed ‘social network credibility’ and ‘social network popularity’ in claims 2, 3, and 17 are satisfied, respectively, by the ‘trustworthiness’ and ‘popular[ity]’ attributes of social network members as taught by Buyukkokten.” Pet. 76 (alteration in original) (citing Ex. 1005 ¶¶ 47, 51) (emphasis removed). Patent Owner argues, “Trustworthiness however is not credibility and coolness is not popularity.” PO Resp. 27 (citing Ex. 2006 ¶¶ 107–115). Further, Patent Owner argues that “[e]ven if they were, Buyukkokten does not disclose ‘trustworthiness’ and ‘popularity’ as ‘social network ranking factors’ under the correct construction of this term because ranking is based on trustworthiness and popularity as assigned by individual users and not based on interpersonal metrics derived from the social network database.” PO Resp. 27 (citing Ex. 2006 ¶¶ 107–115). Based on the evidence presented, we are persuaded that Buyukkokten’s “trustworthiness” and “coolness” teach the claimed ranking factors of “social network credibility” and “social network popularity,” IPR2019-01516 Patent 9,128,981 B1 61 respectively. Buyukkokten explains that “the member can be rated or scored by other members of the social network 200 in specific categories, such as humor, intelligence, fashion, trustworthiness, sexiness, and coolness.” Ex. 1005 ¶ 29. Buyukkokten goes on to explain that categories such as “[t]rusty can indicate the trustworthiness of the member. Cool can indicate how trendy or popular the person is.” Ex. 1005 ¶ 47. We disagree with Patent Owner’s argument because, as we explained above (supra § II.C.4), we disagree that social network ranking factors must be based on interpersonal metrics. Therefore, we determine that Petitioner has demonstrated that the combination of Wu, Evermann, Davis, Burke, and Buyukkokten teaches each limitation of claims 2, 3 and 17. 4. Claim 15 As discussed above under our analysis of asserted obviousness over Wu, Evermann, Davis, and Burke, Patent Owner argues claim 15 should be interpreted to rank based on a single factor of a distance between the searching user and the location where the user review took place. PO Resp. 32 (citing Ex. 2006 ¶¶ 162–165). We disagree with Patent Owner’s interpretation, and thus find that the claim would be have been obvious over the teachings of Wu, Evermann, Davis, and Burke. See supra II.F.8. Here, Petitioner argues that even if claim 15 were to be interpreted as Patent Owner argues, the additional teachings of Buyukkokten still render the claim obvious. For example, Petitioner explained for claim 16 that the combination would teach or suggest ranking user reviews by distance of the authoring reviewer from the user. Pet. 74. According to Petitioner, although Buyukkokten does not disclose “distance” as one of the rating categories, it does track the location of its social network members. Pet. 74 (citing Ex. IPR2019-01516 Patent 9,128,981 B1 62 1005 ¶¶ 23, 28, 31). Buyukkokten would therefore be able to identify and record the location of a user at the time that user authors a review of a restaurant. According to Petitioner, it would have been obvious to one or ordinary skill to rank results based on the distance of the searching user to the location of the authoring reviewer at the time the review took place. Reply 24–25 (citing Ex. 1042 ¶ 87). We find this demonstrates that the prior art references render obvious ranking based on a distance between the searching user and the location where the user review took place, even under Patent Owner’s interpretation of claim 15. Therefore, we determine that Petitioner has demonstrated that the combination of Wu, Evermann, Davis, Burke, and Buyukkokten teaches the limitations of claim 15. 5. Claims 4–9, 11–14, and 18–19 Petitioner presents the same arguments and evidence for claims 4–9, 11–15, and 18–19 under this ground as for the previous grounds, with Buyukkokten being relied upon for teaching the “social network ranking factor” limitations of the independent claims. Pet. 77 (“For Ground 3, the mapping for these dependent claims remains the same as Ground 2, except for the ‘at least one social network ranking factor’ that first appeared in the independent claims.”). For the same reasons explained in our analysis of the previous grounds, and also in our analysis of this ground, explaining that Buyukkokten also discloses a “social network ranking factor,” we find Petitioner has established that the combination of Wu, Evermann, Davis, Burke, and Buyukkokten teaches the limitations of claims 4–9, 11–15, and 18–19. IPR2019-01516 Patent 9,128,981 B1 63 6. Conclusion We conclude that Petitioner has shown by a preponderance of the evidence that claims 1–19 would have been obvious over Wu, Evermann, Davis, Burke, and Buyukkokten. III. CONCLUSION10 Weighing all of the evidence of obviousness and nonobviousness together, including the content of the prior art, the differences between the prior art teachings and the claim limitations, and the objective indicia of nonobviousness, we determine Petitioner has shown by a preponderance of the evidence (1) that claims 1 and 10 would have been obvious over the combination of Wu and Evermann, (2) that claims 1, 4–16, 18, 19 would have been obvious over Wu, Evermann, Davis, and Burke, and (3) that claims 1–19 would have been obvious over Wu, Evermann, Davis, Burke, and Buyukkokten. In summary: 10 Should Patent Owner wish to pursue amendment of the challenged claims in a reissue or reexamination proceeding subsequent to the issuance of this decision, we draw Patent Owner’s attention to the April 2019 Notice Regarding Options for Amendments by Patent Owner Through Reissue or Reexamination During a Pending AIA Trial Proceeding. See 84 Fed. Reg. 16,654 (Apr. 22, 2019). If Patent Owner chooses to file a reissue application or a request for reexamination of the challenged patent, we remind Patent Owner of its continuing obligation to notify the Board of any such related matters in updated mandatory notices. See 37 C.F.R. § 42.8(a)(3), (b)(2). Claims 35 U.S.C. § Reference(s)/Basis Claims Shown Unpatentable Claims Not Shown Unpatentable 1, 10 103(a) Wu, Evermann 1, 10 1, 4–16, 18, 19 103(a) Wu, Evermann, Davis, Burke 1, 4–16, 18, 19 1–19 103(a) Wu, Evermann, 1–19 IPR2019-01516 Patent 9,128,981 B1 64 IV. ORDER In consideration of the foregoing, it is hereby: ORDERED that claims 1–19 of the ʼ981 Patent are held to unpatentable; and FURTHER ORDERED that, because this is a Final Written Decision, the parties to the proceeding seeking judicial review of the decision must comply with the notice and service requirements of 37 C.F.R. § 90.2. Davis, Burke, Buyukkokten Overall Outcome 1–19 IPR2019-01516 Patent 9,128,981 B1 65 FOR PETITIONER: Heidi Keefe DeAnna Allen Phillip Morton COOLEY LLP hkeefe@cooley.com dallen@cooley.com pmorton@cooley.com Mark Liang O’MELVENY & MYERS LLP mliang@omm.com FOR PATENT OWNER: Minghui Yang Bradley Liddle HARDY, PARRISH, YANG, LLP myang@hpylegal.com bdliddle@gmail.com Copy with citationCopy as parenthetical citation