Freshub, Ltd.

Patent Trials and Appeals Board

Jan 11, 2021

IPR2020-01144 (P.T.A.B. Jan. 11, 2021)

Trials@uspto.gov Paper 11
Tel: 571-272-7822 Date: January 11, 2021

UNITED STATES PATENT AND TRADEMARK OFFICE
_______________
BEFORE THE PATENT TRIAL AND APPEAL BOARD
_______________
AMAZON.COM, INC., AMAZON.COM SERVICES LLC
(formerly AMAZON DIGITAL SERVICES LLC), PRIME NOW LLC,
and WHOLE FOODS MARKET SERVICES, INC.,
Petitioner,

v.

FRESHUB, LTD.,
Patent Owner.
____________

IPR2020-01144
Patent 10,239,094 B2
____________

Before WILLIAM V. SAINDON, FRANCES L. IPPOLITO, and
ERIC C. JESCHKE, Administrative Patent Judges.

SAINDON, Administrative Patent Judge.

DECISION
Denying Institution of Inter Partes Review
35 U.S.C. § 314

IPR2020-01144
Patent 10,239,094 B2
2
I. INTRODUCTION
Amazon.com, Inc., Amazon.com Services LLC, Prime Now LLC, and
Whole Foods Market Services, Inc. (collectively, “Petitioner”) filed a
Petition (Paper 1, “Pet.”) requesting inter partes review of claims 1–24 of
U.S. Patent No. 10,239,094 B2 (Ex. 1001, “the ’094 patent”). Patent Owner
filed a Preliminary Response. Paper 6 (“Prelim. Resp.”). Pursuant to the
Board’s Order (Paper 7), the Petitioner filed a Reply to the Preliminary
Response addressing Patent Owner’s contentions directed to discretionary
denial under § 314(a) (Paper 9, “Prelim. Reply”) and Patent Owner filed a
Sur-reply to that Reply (Paper 10, “Prelim. Sur-reply”).
Under 35 U.S.C. § 314(a), an inter partes review may not be instituted
unless the information presented in the Petition and any response thereto
shows “there is a reasonable likelihood that the petitioner would prevail with
respect to at least 1 of the claims challenged in the petition.” Taking into
account the arguments presented in Patent Owner’s Preliminary Response,
we conclude that the information presented in the Petition does not establish
that there is a reasonable likelihood that Petitioner would prevail in
challenging at least one of claims 1–24 of the ’094 patent as unpatentable
under the grounds presented in the Petition. Pursuant to § 314, we hereby do
not institute an inter partes review as to these claims of the ’094 patent.
A. Related Matters
The ’094 patent is the subject of Freshub, Inc. v. Amazon.com Inc., et
al., Case No. 1:19-CV-00885-ADA (W.D. Tex. June 24, 2019) (“the
Freshub case”). Pet. 2; accord Paper 4, 1 (Patent Owner’s Mandatory
Notices).
IPR2020-01144
Patent 10,239,094 B2
3
B. Real Parties In Interest
Petitioner and Patent Owner each assert they are the only real parties
in interest. Pet. 1; Paper 4, 1.
C. The ’094 Patent
The ’094 patent is directed to a system for scanning information from
storage area contents and providing meal suggestions. Ex. 1001, codes (54),
(57). The claims of the ’094 patent are directed to receiving and processing
spoken user orders to identify an item. See, e.g., id. at 14:55–15:12. For
example, voice recognition is performed to recognize a spoken order. Id. at
13:64–14:34.
D. Challenged Claims
Petitioner challenges claims 1–24. Claims 1 and 20 are independent.
Claims 1 and 20 are reproduced below:
1. A computer-implemented method, the method comprising:
receiving over a network at a first computer system, using a
network interface, a digitized spoken user order from a second
computer system,
the second computer system comprising:
a microphone, a wireless network interface, and a digitizer
coupled to the microphone, wherein the digitizer is
configured to convert spoken words into a digital
representation, and the second computer system is
configured to transmit the digital representation over the
network to the first computer system;
translating, using a translation module executed by the second
computer system, at least a portion of the digitized spoken
order to text;
matching the text, translated from the digitized spoken order, to
text descriptions of items, wherein the text descriptions of
items are stored in a data store;
IPR2020-01144
Patent 10,239,094 B2
4
based on at least an identified match, identifying a corresponding
item;
adding the identified corresponding item to a list associated with
the user; and
enabling the list, including the identified item, to be displayed to
the user via a user display.

20. A voice processing system comprising:
a first computer;
a network interface;
non-transitory memory that stores instructions that when
executed by the first computer cause the first system to
perform operations comprising:
receive over a network, using a network interface, a digitized
spoken user order from a remote system,
the remote system comprising:
a microphone, a wireless network interface, and a
digitizer coupled to the microphone, wherein the
digitizer is configured to convert spoken words into
a digital representation, and the second computer
system is configured to transmit the digital
representation over the network to the first
computer system;
translate at least a portion of the digitized spoken order to text;
match the text, translated from the digitized spoken order, to
text descriptions of items, wherein the text descriptions of
items are stored in a data store;
based on at least an identified match, identify a corresponding
item;
include the identified corresponding item in a list associated
with the user; and
enable the list, including the identified item, to be displayed
to the user via a user display.

IPR2020-01144
Patent 10,239,094 B2
5
E. Prior Art and Asserted Grounds
Petitioner asserts the following prior art patents:
Name Patent Document Exhibit
Calderone US 2001/0056350 A1 1003
Ogasawara US 6,543,052 B1 1004
Sanchez US 2002/0194604 A1 1005
Partovi US 7,376,586 B1 1006
Cooper US 6,757,362 B1 1007
Sichelman US 2003/0235282 A1 1008

Petitioner asserts the following grounds:
Claims Challenged 35 U.S.C. § References
1, 2, 10–18, 20 103 Calderone, Ogasawara, Sanchez
3, 4, 7–9, 19, 22–24 103 Calderone, Ogasawara, Sanchez, Partovi
5 103 Calderone, Ogasawara, Sanchez, Cooper
5, 21 103 Calderone, Ogasawara, Sanchez, Sichelman

Petitioner relies on testimony from Dan R. Olsen, Jr., Ph.D, who has a
doctorate in Computing and Information and 35 years of experience in the
field of computer science and human-computer interaction. Ex. 1002 ¶ 3.
II. PATENTABILITY ANALYSIS
A. Claim Construction
This Petition receives the same claim construction standard used in
the federal courts. 37 C.F.R. § 42.100(b) (2019). However, no claim terms
require explicit construction for purposes of this Decision. See Nidec Motor
Corp. v. Zhongshan Broad Ocean Motor Co., 868 F.3d 1013, 1017 (Fed.
IPR2020-01144
Patent 10,239,094 B2
6
Cir. 2017) (“[W]e need only construe terms ‘that are in controversy, and
only to the extent necessary to resolve the controversy.’”) (quoting Vivid
Techs., Inc. v. Am. Sci. & Eng’g, Inc., 200 F.3d 795, 803 (Fed. Cir. 1999)).
B. Level of Ordinary Skill in the Art
Petitioner asserts that a person of ordinary skill in the art would have
had “at least a Bachelor-level degree in computer science, computer
engineering, electrical engineering, or a related field in computing
technology, and 2 years of experience with automatic speech recognition and
natural language understanding, or equivalent education, research
experience, or knowledge.” Pet. 3–4. Patent Owner does not offer a
competing definition at this time.
For purposes of this Decision, we apply Petitioner’s proposed
definition of the level of ordinary skill, which appears reasonable in view of
the relative level of sophistication required to read and understand the ’094
patent and the prior art disclosures.
C. Analysis of the Obviousness Grounds
The ’094 patent has two independent claims, 1 and 20. Petitioner
asserts that Calderone, Ogasawara, and Sanchez render these two claims
obvious. Pet. 14–28 (claim 1), 40–43 (claim 20). Petitioner’s other
obviousness grounds build on this first ground and are directed to claims that
depend from claims 1 and 20. Accordingly, we focus our analysis on
Petitioner’s ground with respect to independent claims 1 and 20.
Independent Claim 1
Independent claim 1 describes two computer systems. The second
computer system includes a microphone to convert a user’s spoken words
IPR2020-01144
Patent 10,239,094 B2
7
into a digital representation of the user’s order. A “module” of the second
computer system then “translat[es] . . . a portion of the digitized spoken
order to text.” The first computer system receives the digitized spoken user
order from the second computer system and then performs various
processing steps.
Petitioner’s ground begins with the networked voice recognition
system of Calderone. See generally Pet. 16–24; Ex. 1003, codes (54), (57).
Petitioner asserts that the claimed first computer system reads on the back
end system of Calderone, and the second computer system reads on the
computer at the user’s location in Calderone. Pet. 18 (“Calderone discloses
a second computer system at the user site”), 21 (“Calderone discloses a ‘first
computer system’ (central speech engine) that receives the digitized spoken
user order from the ‘second computer system’ (remote and set-top apparatus
at the user site).”). Petitioner asserts that a “speech recognition software
module” in Calderone’s back-end system (i.e., the alleged first computer
system) teaches the “translating” limitation and operates as the claimed
“translation module” performing the translating function. Id. at 24; Ex. 1003
¶¶ 40, 143–144, Fig. 3.
Claim 1 requires that the second computer system has the translation
module, but the Petition identifies the alleged first computer system of the
prior art to have the translation module. See, e.g., Ex. 1003 ¶ 40; see also
Prelim. Resp. 22 (arguing that Calderone’s “‘speech recognition software
module’ is located on a centrally located, back-end system, not the ‘remote
and set-top apparatus at the user site’ that Amazon identified as the front-end
system.”). Thus, Petitioner’s ground as set forth in the Petition does not
address adequately the “translating” step because the alleged “translation
IPR2020-01144
Patent 10,239,094 B2
8
module” in Calderone is not in the specific computer system identified by
the claim.1 For these reasons, we determine that Petitioner has not
established a reasonable likelihood of success for independent claim 1, or for
claims 2–19, which depend therefrom.
Independent Claim 20
Independent claim 20 is largely similar to independent claim 1 but
differs in that the translation step of claim 20 now occurs on the first
computer system. Thus, the issue we identified with Petitioner’s ground
addressing claim 1 does not apply to Petitioner’s ground addressing claim
20.
Claim 20 recites two computer systems, a “first system” and a
“remote system.” The remote system includes a microphone used to convert
spoken words into a digital representation that is sent to the first computer

1 Petitioner asserts in its Reply to Patent Owner’s Preliminary Response that
distinctions such as which computer performs which process are
“immaterial” because a person of ordinary skill in the art could place
functions in either system as they see fit. Pet. Reply 5. Even accepting,
arguendo, that a person of ordinary skill in the art would be able to place
different software various functions in any computer system in the abstract,
Petitioner would still have to explain why it would have been obvious to
modify the system of Calderone specifically, which, for example, does not
necessarily use general purpose computer systems but rather cable TV set
top boxes and particular centralized processing units. See, e.g., Ex. 1003
¶ 130 (describing use of “a General Instruments or a Scientific Atlanta set-
top” boxes); id. ¶ 132 (describing “high speed speech processing unit
capable of processing the data from several nodes”); see also Belden Inc. v.
Berk-Tek LLC, 805 F. 3d 1064, 1073 (Fed. Cir. 2015) (“obviousness
concerns whether a skilled artisan not only could have made but would have
been motivated to make the combinations or modifications of prior art to
arrive at the claimed invention”).
IPR2020-01144
Patent 10,239,094 B2
9
system. In relevant part, the first system then “translates [the] spoken order
to text,” “match[es] the text . . . to text descriptions of items . . . in a data
store,” and then “identif[ies] a corresponding item” “based on the . . .
match.” We focus our analysis on the “match” and “identify” steps, which
work together to take the text from the translation step, match the text to a
description, and then identify an item based on the match.
Petitioner’s ground for claim 20 generally directs our attention back to
its ground for claim 1. See Pet. 40–43. As to the “match” step, Petitioner’s
ground discusses how Calderone “respond[s] to identified speech content to
create an identified speech content response.” Id. at 24. According to
Petitioner, Calderone “searches for matching content using movie titles or
actor names, i.e. text descriptions related to the movie or program” in a
database. Id. at 25.
For the “identify” step, Petitioner directs our attention to “the previous
section,” i.e., its discussion for the “match” step. Pet. 25. Petitioner further
points to a “[Product Look Up (PLU)] Table” in Ogasawara, and asserts, “a
POSITA would have understood that Calderone and Ogasawara disclose
using the matching names or descriptions to identify the actual item to
provide in response to each user request.” Id. at 25–26.
In effect, Petitioner relies on the same disclosure of Calderone, and a
similar disclosure in Ogasawara, regarding searching databases for both the
“match” step as well as the “identify” step. These are different limitations,
however. Although one prior art structure may satisfy multiple claim
limitations, see, e.g., Powell v. Home Depot U.S.A., Inc., 663 F.3d 1221,
1231–32 (Fed. Cir. 2011), Petitioner does not explain how the same
disclosure reads on both limitations. See Bicon, Inc. v. Straumann Co., 441
IPR2020-01144
Patent 10,239,094 B2
10
F.3d 945, 950 (Fed. Cir. 2006) (holding that claims are “interpreted with an
eye toward giving effect to all terms in the claim”); see also 37 C.F.R.
§ 42.104(b) (requiring Petitioner to set forth “[h]ow the challenged claim is
to be construed” and “[h]ow the construed claim is unpatentable”). In
addition, as Patent Owner points out, the most analogous step in Calderone
to the “identify” step of claim 20 (i.e., where an item is selected) is
performed by a user on the second computer system, i.e., the user’s set top
box, rather than the first computer system as required by the claim. Prelim.
Resp. 14–21 (citing, e.g., Ex 1003 ¶¶ 166–170, wherein the remote module
sends back to the user a list of options for the user to select).
Thus, there are several problems with Petitioner’s ground as
articulated. First, Petitioner identifies effectively the same disclosure for the
“match” and “identify” steps without a claim construction or sufficient
explanation as to why both limitations are satisfied by the same effective
disclosure. Second, to the extent Petitioner’s ground implies that the actual
selection of an item in Calderone satisfies the “identify” step, then that
action is performed in Calderone at a location other than that required by the
claims, and by a user rather than a computer. Third, to the extent Petitioner
suggests that Ogasawara’s PLU table teaches the “identify” step, Petitioner
does not make clear how that disclosure is any different from Calderone’s
similar disclosure of a database lookup, or how that teaching is applied to
Calderone to satisfy the limitation in combination, because Petitioner again
effectively points to the same action for both the “match” and “identify”
steps. Even if this disclosure in Ogasawara did teach the “identify” step
(however differentiated), Petitioner does not explain why it would have been
obvious to modify Calderone in some way to meet the claim limitation.
IPR2020-01144
Patent 10,239,094 B2
11
Notably, having the user identify the item on their set top box (i.e., perform
the identify step on the second computer) is a specific error correction
mechanism in Calderone. See, e.g., Ex. 1003 ¶¶ 166–169 (stating that mis-
recognized speech is a “primary hindrance,” which is resolved by providing
user visual feedback, i.e., providing text back to the user for confirmation).
Petitioner would have needed to explain why a person of ordinary skill in the
art would have considered modifying this deliberate feature of Calderone’s
system. See Winner Int’l Royalty Corp. v. Wang, 202 F.3d 1340, 1349 n.8
(Fed. Cir. 2000) (“the benefits, both lost and gained, [from pursuing a
particular modification] should be weighed against one another.”); see also
KSR Int’l Co. v. Teleflex, Inc., 550 U.S. 398, 418 (2007) (“a patent
composed of several elements is not proved obvious merely by
demonstrating that each of its elements was, independently, known in the
prior art”).
For the above reasons, we determine that Petitioner has not
established a reasonable likelihood of success for independent claim 20. For
similar reasons, we also do not find a reasonable likelihood of success for
claims 21–24, which depend therefrom.
III. ORDER
We determine that Petitioner has not demonstrated a reasonable
likelihood of success in showing that claims 1–24 of the ’094 patent are
unpatentable under the grounds asserted in its Petition. Accordingly, it is
hereby:
ORDERED that the Petition for inter partes review is denied and that
no inter partes review is instituted.

IPR2020-01144
Patent 10,239,094 B2
12
For PETITIONER:

J. David Hadden
Saina S. Shamilov
Allen Wang
FENWICK & WEST LLP
dhadden@fenwick.com
sshamilov@fenwick.com
allen.wang@fenwick.com

For PATENT OWNER:

James Hannah
Jeffrey H. Price
Jonathan Caplan
KRAMER LEVIN NAFTALIS & FRANKEL LLP
jhannah@kramerlevin.com
jprice@kramerlevin.com
jcaplan@kramerlevin.com