Apple Inc.

Patent Trials and Appeals Board

Aug 3, 2021

2021002710 (P.T.A.B. Aug. 3, 2021)

UNITED STATES PATENT AND TRADEMARK OFFICE
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O. Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
16/116,112 08/29/2018 Thomas R. GRUBER P14301USC1/77870000144201 6114
150004 7590 08/03/2021
DENTONS US LLP - Apple
4655 Executive Dr
Suite 700
San Diego, CA 92121
EXAMINER
ABOU EL SEOUD, MOHAMED
ART UNIT PAPER NUMBER
2142
NOTIFICATION DATE DELIVERY MODE
08/03/2021 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
dentons_PAIR@firsttofile.com
patent.docket@dentons.com
patents.us@dentons.com
PTOL-90A (Rev. 04/07)

UNITED STATES PATENT AND TRADEMARK OFFICE
____________

BEFORE THE PATENT TRIAL AND APPEAL BOARD
____________

Ex parte THOMAS R. GRUBER and DONALD W. PITSCHEL
___________

Appeal 2021–002710
Application 16/116,112
Technology Center 2100

____________

Before MAHSHID D. SAADAT, CARL W. WHITEHEAD JR. and
ERIC B. CHEN, Administrative Patent Judges.

WHITEHEAD JR., Administrative Patent Judge.

DECISION ON APPEAL

STATEMENT OF THE CASE1
Appellant2 is appealing the final rejection of claims 1–18 under
35 U.S.C. § 134(a). See Appeal Brief 6. Claims 1, 17 and 18 are
independent. We have jurisdiction under 35 U.S.C. § 6(b).

1 Rather than reiterate Appellant’s arguments and the Examiner’s
determinations, we refer to the Appeal Brief (filed November 4, 2020), the
Reply Brief (filed March 12, 2021), the Final Action (mailed May 14, 2020)
and the Answer (mailed January 26, 2021), for the respective details.

2 Appellant refers to “applicant” as defined in 37 C.F.R. 1.42(a). (“The
word ‘applicant’ when used in this title refers to the inventor or all of the
joint inventors, or to the person applying for a patent as provided in
Appeal 2021-002710
Application 16/116,112

2

We affirm.
Introduction
According to Appellant, “[t]he claimed invention is generally directed
to a digital assistant that may interrupt the user when determined that an
output is urgent.” Appeal Brief 8 (citing Specification ¶ 103).
While the user is answering the phone via speech input SI2, the
device receives a speech output SO3 corresponding to a turn-by-
turn direction command indicating that the user should turn right
very soon (e.g., in this example, 100 feet). Because of the
urgency of the message, the device “barges-in” (e.g., interrupts
the user while the user is speaking) to output, “Turn right in 100
feet.”
Specification ¶ 103.
Representative Claim3 (disputed limitations emphasized)
1. A non-transitory computer readable storage medium storing
one or more programs, the one or more programs comprising
instructions, which when executed by one or more processors of an
electronic device, cause the device to:
generate a speech output to be provided to a user of the
device;
engage in a communication session with a remote device;

§§ 1.43, 1.45, or 1.46.”). Appellant identifies Apple, Inc. as the real party in
interest. Appeal Brief 3.

3 Appellant does not argue the independent claims individually. See Appeal
Brief 16 (“As discussed above, none of the cited references, alone or in
combination, disclose or suggest all of the features of claim 1 (and similarly
claims 17 and 18).”). Accordingly, we select independent claim 1 as
representative. See 37 C.F.R. § 41.37(c)(1)(iv)(“When multiple claims
subject to the same ground of rejection are argued as a group or subgroup by
appellant, the Board may select a single claim from the group or subgroup
and may decide the appeal as to the ground of rejection with respect to the
group or subgroup on the basis of the selected claim alone.”).

Appeal 2021-002710
Application 16/116,112

3

while the device is engaged in the communication session
with the remote device:
determine an urgency value of the speech output,
wherein the urgency value is based on content of the speech
output;
determine whether the urgency value of the speech
output satisfies a predetermined threshold;
upon determining that the urgency value of the speech
output satisfies the predetermined threshold, provide the
speech output to the user of the device; and
upon determining that the urgency value of the speech
output does not satisfy the predetermined threshold, forgo
providing the speech output to the user of the device.
References
Name4 Reference Date
Van Amerongen US 6,658,102 B1 December 3, 2003
Horvitz US 7,120,865 B1 October 10, 2006
Rokusek US 2007/0140187 A1 June 21, 2007
Zhang US 2009/0055088 A1 February 26, 2009

Rejections on Appeal
Claims 1–5, 10, 11 and 15–18 stand rejected under 35 U.S.C. § 103(a)
as being unpatentable over Horvitz and Van Amerongen. Final Action 2–7.
Claims 6–8 and 12–14 stand rejected under 35 U.S.C. § 103(a) as
being unpatentable over Horvitz, Van Amerongen and Rokusek. Final
Action 7–10.
Claim 9 stands rejected under 35 U.S.C. § 103(a) as being
unpatentable over Horvitz, Van Amerongen and Zhang. Final Action 10–11.

4 All reference citations are to the first named inventor only.

Appeal 2021-002710
Application 16/116,112

4

ANALYSIS
Claims 1–5, 10, 11 and 15–18
Appellant contends the Examiner errs because Horvitz and Van
Amerongen fail to disclose or suggest claim 1’s limitation “determine an
urgency value of the speech output, wherein the urgency value is based on
content of the speech output.” Appeal Brief 11. Appellant contends the
obviousness rejection relies upon Horvitz and “argues that the ‘document
classifier determine[s] an urgency value (priority) of the document (speech
output),’ and ‘words or special phrases are used in determining different
priority level.’” Appeal Brief 11 (citing Horvitz, column 9, lines 12–20, 61–
65; see Final Action 2). Appellant argues, “the cited portion at best
discloses that a priority for a received document or message may be
determined” and therefore the “received document or message is distinct
from a ‘speech output’ as required by claim 1.” Appeal Brief 11 (emphases
omitted). Appellant further argues that “Horvitz is silent with regard to
converting the text of any received document. Rather, Horvitz merely
discloses that an automated assistant may include general text-to-speech
rendering abilities.” Appeal Brief 12. Appellant argues:
Horvitz merely describes that to alert the user of a prioritized
document “a sound is played,” “an agent or automated assistant
is opened - that is, that agent appears on the screen,” “the
prioritized document is opened,” or “the document can receive
focus.” Thus, Horvitz clearly describes determining a priority of
the received document, but fails to disclose or suggest
determining a priority based on any “speech output” as required
by claim 1.
Appeal Brief 12 (citing Horvitz column 14, lines 1–10); see Final
Action 2.
Appeal 2021-002710
Application 16/116,112

5

Appellant’s argument is not persuasive of Examiner error. Horvitz
discloses, “[s]everal methods are reviewed for display and interaction that
leverage the assignment of priorities to documents, including a means for
guiding visual and auditory actions by priority of incoming messages.”
Horvitz, Abstract (emphasis added).
Claim 1 recites, “generate a speech output to be provided to a user of
the device.” Claim 1 further recites, “determine an urgency value of the
speech output, wherein the urgency value is based on content of the speech
output.” Horvitz discloses generating a speech output and determining an
urgency value — “the user may indicate that an agent or automated assistant
with speech recognition and text-to speech rendering abilities be activated,
to alert the user to the priority and to allow the user to engage further in a
dialog about hearing or seeing more about the message.” Horvitz column
16, lines 24–28 (emphasis added); see Final Action 2.
Horvitz discloses in column 9, lines 17–24 (emphasis added),
determining the urgency value of a speech output based upon content of the
speech output wherein the content of the speech output is based upon the
priority of a document constructed upon a user-configurable criteria5:
[A] document, such as an email message 300, is input into the
document classifier 200, which based thereon generates a
priority 302 for the document 300. That is, in one embodiment,
the document classifier 200 generates a priority 302, measured
as a percentage from 0 to 1 (i.e., 0% to 100%). This percentage

5 “In the explicit training, the document classifier is presented with both
time-critical and non-time-critical documents (e.g., email messages), so that
it may be able to discriminate between the two. This training set may be
provided by the user, or a standard training set may be used.” Horvitz,
column 6, lines 42–46.
Appeal 2021-002710
Application 16/116,112

6

is a measure of the likelihood that the document 300 is of high
priority, based on the previous training of the classifier 200.”
Horvitz’s disclosure aligns with the Specification because the
Specification discloses the urgency value or priority of a document
(electronic message) is based upon a user-configurable criteria also:
Upon determining that the device is receiving speech input
from the user, the device determines (538) if provision of the
speech output is urgent. In some embodiments, the speech output
is urgent (540) when the speech output meets user-configurable
criteria for immediate provision. For example, in some
embodiments, the user-configurable criteria are met (542) when
the device receives an electronic message from a person that the
user has previously identified as a very important person (VIP).
Specification ¶ 119 (emphasis added).
Accordingly, the claimed urgency value is based upon the content of
the speech output wherein the content of the speech output is based upon the
urgency or priority of a document (electronic message). See Specification
¶ 119. Horvitz also discloses that the urgency value of a speech output is
based upon the content of the speech output wherein the content of the
speech output is based upon the urgency or priority of a document. See
Horvitz column 16, lines 24–28, and column 9, lines 17–24.
Appellant contends, Van Amerongen fails to address the deficiency of
Horvitz because “Van Amerongen merely discloses interrupting a phone call
to provide a reminder based on whether a time criteria has been met.”
Appeal Brief 14. Appellant further argues that “determining whether a time
criteria has been met fails to disclose or suggest ‘determin[ing] an urgency
value of the speech output, wherein the urgency value is based on content of
Appeal 2021-002710
Application 16/116,112

7

the speech output,” or ‘determin[ing] whether the urgency value of the
speech output satisfies a predetermined threshold.’” Appeal Brief 15.
Appellant’s arguments are not persuasive of Examiner error because
we find Horvitz is not deficient as Appellant argues.
Appellant further argues, “[a] fundamental problem with the Office’s
rejection is that the Office has dissected the claim into separate elements and
examined them in isolation.” Appeal Brief 15. Appellant’s argument is not
persuasive of Examiner error. Appellant contends:
Van Amerongen merely discloses putting a call on hold to ask a
user if they would like to hear reminders, as discussed above.
Therefore, Van Amerongen fails to disclose or suggest
“determin[ing] an urgency value of the speech output, wherein
the urgency value is based on content of the speech output,”
much less the other limitations of claim 1, “while the device is
engaged in the communication session with the remote device.”
Appeal Brief 16.
The Examiner relies upon Horvitz to disclose the claimed urgency
value of the speech output and not Van Amerongen. See Final Action 3.
The test for obviousness is what the combined teachings of the references
would have suggested to one of ordinary skill in the art. See In re Kahn, 441
F.3d 977, 987–88 (Fed. Cir. 2006), In re Young, 927 F.2d 588, 591 (Fed.
Cir. 1991) and In re Keller, 642 F.2d 413, 425 (CCPA 1981). Furthermore,
Horvitz discloses, “the document classifier is such that a user is continually
watched while working, and the classifier is continually refined by training
in the background and being updated in real time for decision making”
(Horvitz, column 9, lines 9–12); “It is also noted that in order to determine
the expect cost of alerting, it is useful to infer or directly access information
about whether the user is present—and therefore can see or hear alerts from
Appeal 2021-002710
Application 16/116,112

8

the computer—or is not present” (Horvitz, column 12, lines 7–20); and “it is
reasoned about whether and the rate at which a user is working on a
computer, or whether the user is on the telephone, speaking with someone,
or at a meeting at another location” (Horvitz, column 12, lines 34–38).
Consequently, Horvitz actually teaches alerting the user when the user is
working or “engage[d] in a communication session with a remote device” as
recited in claim 1.
Accordingly, we sustain the Examiner’s obviousness rejection of
claims 1–5, 10, 11 and 15–18, not argued separately. See Appeal Brief 10,
16.
Claim 6
6. The non-transitory computer readable storage medium of
claim 1, wherein the one or more programs further comprise
instructions, which when executed by the one or more
processors, cause the device to:
upon determining that the urgency value of the speech
output does not satisfy the predetermined threshold, delay
providing the speech output for a predetermined time.
Appellant argues that Rokusek “merely discloses delaying providing
navigation audio until call audio ends” and therefore Rokusek “fails to
disclose or suggest ‘delay providing the speech output for a predetermined
time’” as recited in claim 6. Appeal Brief 17. The Specification discloses:
[I]t would be more suitable to delay (e.g., staying, at least
temporary) provision of a non-urgent speech output because a
user is speaking into the device, or because the user’s context
suggests that interruptions should be avoided (e.g., because the
user is in a meeting or is asleep) while other times it may be more
suitable to provide an urgent speech output immediately (e.g.,
“barge-in,” or interrupt the user).
Appeal 2021-002710
Application 16/116,112

9

Specification ¶ 96.
Horvitz discloses, “a scheme where documents are divided into low,
medium and high priority, according to an embodiment of the invention.”
Horvitz column 9, lines 41–43. Horvitz further discloses, that “a decision
should be made as to when and how to alert users to messages and to
provide services (for example) based on the inference of expected criticality
and user activity.” Horvitz column 12, lines 53–56. Horvitz also discloses
that “the decision as to when and how to alert users is made by employment
of a set of user-specified thresholds and parameters defining policies on
alerting.” Horvitz column 13, lines 12–15.
“As our precedents make clear, however, the analysis need not seek
out precise teachings directed to the specific subject matter of the challenged
claim, for a court can take account of the inferences and creative steps that a
person of ordinary skill in the art would employ.” KSR Int’l Co. v. Teleflex
Inc., 550 U.S. 398, 418 (2007). Here, combining the disclosure of “options
for audio arbitration” in Rokusek and how Horvitz determines the criticality
of messages discloses the disputed limitation of claim 6. See Rokusek ¶ 54;
Horvitz column 12 lines 53–56, column 13 lines 12–15. Accordingly, we
discern no meaningful difference between Horvitz’s dividing documents
based upon priority and then determining when to alert (speech output) the
user based on expected priority and user activity; and claim 6’s requirement
to delay the speech output based upon the urgency value of the speech
output. We sustain the Examiner’s obviousness rejection of claim 6.
Claim 8
8. The non-transitory computer readable storage medium of
claim 6, wherein the one or more programs further comprise
Appeal 2021-002710
Application 16/116,112

10

instructions, which when executed by the one or more
processors, cause the device to:
after delaying providing the speech output for the
predetermined time:
determine whether the communication session has
ended; and
in accordance with a determination that the
communication session has ended, provide the speech output to
the user of the device.
Appellant presents arguments similar to those raised for claim 6,
which were found to be unpersuasive of Examiner error. See Appeal Brief
18–19. As stated above, Horvitz discloses, that “a decision should be made
as to when and how to alert users to messages and to provide services (for
example) based on the inference of expected criticality and user activity.”
Horvitz column 12, lines 53–56. Consequently, determining to provide a
speech output to the user after the activity has concluded is well within the
purview of one of ordinary skill in the art in view of the cited teachings in
Horvitz, Van Amerongen and Rokusek. See KSR Int’l at 550 U.S. 418. We
sustain the Examiner’s obviousness rejection of claim 8.
Claims 13 and 14
13. The non-transitory computer readable storage medium of
claim 12, wherein determining whether the device is currently
receiving speech input from the user includes:
determining whether a previous speech input from the user
was received within a predetermined period of time.
The Specification discloses:
[T]he predetermined period of time is (516) a function of a
measure of a urgency of the output. For example, when the
device has an urgent message (“Turn right NOW!”) in an output
queue, the device will wait a shorter amount of time before
determining that the user is not speaking, thus barging-in the
Appeal 2021-002710
Application 16/116,112

11

moment the user pauses to catch his or her breath or consider
what to say next).
Specification ¶ 115.
As stated above, Horvitz discloses, that “a decision should be made as
to when and how to alert users to messages and to provide services (for
example) based on the inference of expected criticality and user activity.”
Horvitz column 12, lines 53–56. Consequently, “determining whether a
previous speech input from the user was received within a predetermined
period of time” as recited in claim 13, is well within the purview of one of
ordinary skill in the art in view of the cited teachings in Horvitz, Van
Amerongen and Rokusek. See KSR Int’l at 550 U.S. 418. We sustain the
Examiner’s obviousness rejection of claim 13.
14. The non-transitory computer readable storage medium of
claim 12, wherein determining whether the device is currently
receiving speech input from the user includes:
determining whether a strength characteristic of the speech
input exceeds a predetermined strength threshold.
The Specification discloses:
[D]etermining if the device is currently receiving speech input
includes a squelch determination (e.g., based on a particular
strength or directionality threshold at a device microphone) to
disambiguate, for example, background noise and/or speech
made by the user but not intended as speech input (e.g., during a
telephone conversation, when the user pauses the conversation to
talk to another party in person).
Specification ¶ 115.
Appellant contends, “Rokusek does not provide any disclose related to
‘a strength characteristic of the speech input,’ much less that ‘determining
whether the device is currently receiving speech input includes,’
Appeal 2021-002710
Application 16/116,112

12

‘determining whether a strength characteristic of the speech input exceeds a
predetermined threshold.’” Appeal Brief 20. However, Horvitz discloses,
“[a] user may enter commands and information into the personal computer
20 through input devices such as a keyboard 40 and pointing device 42.
Other input devices (not shown) may include a microphone, joystick, game
pad, satellite dish, scanner, or the like.” Appellant’s argument is not
persuasive of Examiner error because claim 14 merely recites generic
characteristics associated with well-known input devices such as
microphones. See Specification ¶ 115; see also KSR Int’l at 550 U.S. 418.
We sustain the Examiner’s obviousness rejection of claim 14.

CONCLUSION
Claims
Rejected
35
U.S.C. §
Reference(s)/Basis Affirmed Reversed
1–5, 10, 11,
15–18
103(a) Horvitz, Van
Amerongen
1–5, 10,
11, 15–18

6–8, 12–14 103(a) Horvitz, Van
Amerongen,
Rokusek
6–8, 12–14
9 103(a) Horvitz, Van
Amerongen, Zhang
9
Overall
Outcome
1–18

No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a)(1). See 37 C.F.R.
§ 1.136(a)(1)(v).

Appeal 2021-002710
Application 16/116,112

13

AFFIRMED