Microsoft Technology Licensing, LLC

Patent Trials and Appeals Board

Dec 10, 2021

2021000220 (P.T.A.B. Dec. 10, 2021)

UNITED STATES PATENT AND TRADEMARK OFFICE
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O. Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
15/344,379 11/04/2016 Stephan J. Zachwieja 360443-US-NP 5722
147148 7590 12/10/2021
Ray Quinney & Nebeker - Microsoft
36 South State Street
Suite 1400
Salt Lake City, UT 84111
EXAMINER
LIDDLE, JAY TRENT
ART UNIT PAPER NUMBER
3715
NOTIFICATION DATE DELIVERY MODE
12/10/2021 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
mspatent@rqn.com
usdocket@microsoft.com
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
____________

BEFORE THE PATENT TRIAL AND APPEAL BOARD
____________

Ex parte STEPHAN J. ZACHWIEJA,
WISGARY FRANCISCO TORRES MARRERO,
JOAQUIN BOLBRUGGE CHAVOLLA, SCOTT NATHANIEL MUNRO,
MARCUS MICHAEL POWERS, and JAMES LAO

____________

Appeal 2021-000220
Application 15/344,379
Technology Center 3700
____________

Before STEFAN STAICOVICI, WILLIAM A. CAPP, and
MICHAEL L. WOODS, Administrative Patent Judges.

STAICOVICI, Administrative Patent Judge.

DECISION ON APPEAL

STATEMENT OF THE CASE.
Appellant1 appeals under 35 U.S.C. § 134(a) from the Examiner’s
decision in the Final Office Action (dated Mar. 18, 2020, hereinafter “Final

1 We use the word “Appellant” to refer to “applicant” as defined in 37
C.F.R. § 1.42. MICROSOFT TECHNOLOGY LICENSING, LLC. is
identified as the real party in interest in Appellant’s Appeal Brief (filed June
17, 2020, hereinafter “Appeal Br.”). Appeal Br. 3.
Appeal 2021-000220
Application 15/344,379

2

Act.”) rejecting claims 1, 4–18, and 20–22. We have jurisdiction over this
appeal under 35 U.S.C. § 6(b).

SUMMARY OF DECISION
We AFFIRM.

INVENTION
Appellant’s invention is directed to a “video game platform based on
video game state data.” Spec. Para. 3.
Claims 1, 14, and 18 are independent. Claim 1 is illustrative of the
claimed invention and reads as follows:
1. A system, comprising:
at least one processor;
a data store configured to store video game state data for
a plurality of video game play sessions;
an intake coordinator configured to, by the at least one
processor, receive:
first video game state data representing a video game
environment over a first video game session while a video
game is being played by a first user, wherein the first video
game state data comprises a plurality of frames having time
stamps, and wherein the first video game state data specifies
some of the plurality of frames as key frames containing
sufficient information to visually recreate a video game
environment for the first video game session and some of the
plurality of frames as delta frames, wherein the respective delta
frames represent changes in the video game environment with
respect to the key frame or a previous delta frame; and
annotated video game state data for the first video game
session, the annotated video game state data comprising one or
more user annotations provided by a user to at least a
respective one of the key frames or a respective one of the
plurality of delta frames representing the video game
Appeal 2021-000220
Application 15/344,379

3

environment for the first video game session, the one or more
user annotations:
applied to at least one displayed video game object
within the video game environment, that is: 1) positioned at a
first coordinate location within the video game environment
and 2) viewable at a first user-selectable viewing angle that is
selectable within the video game environment,
visually indicating a particular aspect of importance
within the represented video game environment for the first
video game session, and
viewable when the at least one displayed video game
object within the video game environment is rendered at a
second user-selectable viewing angle within the video game
environment that is different from the first viewing angle; and
a distribution coordinator configured to, by the at least
one processor, transmit the first video game state data to a
requesting client associated with a second user, wherein the first
video game state data allows the requesting client to recreate
the video game environment, including the user annotations.

Appeal Br. 21 (Claims App.).

REJECTIONS
I. The Examiner rejects claims 1, 4–17, and 22 under 35 U.S.C.
§ 103 as being unpatentable over Norman2 and Perry.3
II. The Examiner rejects claims 18 and 20 under 35 U.S.C. § 103
as being unpatentable over Norman, Parker,4 and Perry.

2 Norman et al., US 2014/0113718 Al, published Apr. 24, 2014.
3 Perry et al., US 2014/0179424 Al, published June 26, 2014.
4 Parker et al., US 2016/0193530 Al, published July 7, 2016.
Appeal 2021-000220
Application 15/344,379

4

III. The Examiner rejects claim 21 under 35 U.S.C. § 103 as being
unpatentable over Norman, Perry, and Shaw.5,6

ANALYSIS
Rejection I
Appellant does not present arguments for the patentability of
claims 4–17 and 22 apart from claim 1. See Appeal Br. 13–16. Therefore,
in accordance with 37 C.F.R. § 41.37(c)(1)(iv), we select claim 1 as the
representative claim to decide the appeal of the rejection of these claims,
with claims 4–17 and 22 standing or falling with claim 1.
The Examiner finds that Norman discloses most of the limitations of
independent claim 1, but “does not appear to explicitly disclose a first
coordinate location for the annotation and two viewing angles.” Final
Act. 2–3 (citing Norman, paras. 23–30, 34, 35, Fig. 7). Nonetheless, the
Examiner finds Perry discloses the missing limitations. See id. at 3–4 (citing
Perry, paras. 83, 91, 95, Fig. 3c). Thus, the Examiner concludes that it
would have been obvious to a person of ordinary skill in the art “to combine
the teachings of Perry with the disclosure of Norman such that users could
tag particular areas of the video game in the game itself thus directing later
viewers to particularly noteworthy parts of the game play by using the tags.”
Id. at 7 (citing Perry, paras. 83, 91). According to the Examiner, once an

5 Shaw et al., US 2007/0173325 Al, published July 26, 2007.
6 The rejections under 35 U.S.C. § 103 of claim 23 as unpatentable over
Norman, Parker, and Perry and of claims 18 and 20 as unpatentable over
Norman, Parker, and Loeb (US 2017/0182426 Al, published June 29, 2017)
have been withdrawn by the Examiner. Examiner’s Answer (dated July 9,
2020, hereinafter “Ans.”) 3.
Appeal 2021-000220
Application 15/344,379

5

event is tagged in Norman, as modified by Perry, “a viewer will see that
event in Norman from either a first player camera angle or a second player
camera angle.” Id. at 4 (citing Norman, para. 46).
Appellant argues that the combined teachings of Norman and Perry
fail to
teach annotation of the video game state data where the video
game state data “positioned at a first coordinate location within
the video game environment and . . . viewable when the at least
one displayed video game object within the video game
environment is rendered at a second user-selectable viewing
angle” as is presently claimed.

Appeal Br. 16 (emphasis added). Appellant contends that “[t]here is no
indication that the annotation is made to the video game state data, nor is
there any indication in Perry that the annotation is associated with a location
in the video game environment.” Id. at 13. According to Appellant, Perry
“inextricably ties the tag data to the video recording and viewing angle of
the user at the time the annotations are provided and not to the video game
state data.” Id. at 15. Furthermore, Appellant asserts that even though
“Norman appears to teach the recreation of a video game environment based
on game state data and allow multiple perspectives of the recreation, there is
no mechanism by which the screen coordinate tag data of Perry would apply
to the video game environment recreation by state data of Norman.” Id.
Thus, Appellant takes the position that “it is unclear how the recreation of a
video game environment from game state data of Norman would combine
with the annotations based on video frames and screen coordinates of
Perry.” Id.
Appeal 2021-000220
Application 15/344,379

6

Appellant’s arguments are not persuasive because Appellant cannot
show nonobviousness by attacking Norman and Perry individually when the
rejection as articulated by the Examiner is based on a combination of
Norman and Perry. See In re Merck & Co. Inc., 800 F.2d 1091, 1097 (Fed.
Cir. 1986) (“Non-obviousness cannot be established by attacking references
individually where the rejection is based upon the teachings of a
combination of references. [Each reference] must be read, not in isolation,
but for what it fairly teaches in combination with the prior art as a whole.”).
In this case, Norman discloses a multi-user online gaming system
including, inter alia, an online game session server system accessed by a
plurality of users to interactively establish multiple game sessions and a
spectator server accessed by users who wish to view active games. Norman,
para. 4. When a spectator selects a particular game for viewing, Norman
discloses that the spectator server receives data chunks 150 (video game
state data) from the game session server system, such that the spectator
server “can generate the same graphics and visual representation of the
active game in real-time” (recreate game environment). Id. at paras. 32, 33.
Norman further discloses that data chunks 150 have “complete data for the
active game being viewed” and include “keyframes, which are effectively
snapshots of a particular game at a particular point in time” such that a user
can time-shift using included time stamps. Id. at para. 34. Norman also
discloses that a particular player can be selected for viewing, and, thus,
different camera views of different players can be obtained. Id. at para. 46.
In addition, Perry discloses tagging data at a location in a video frame
identified by a user, and, thus, “associat[ing] the tag data to an object shown
Appeal 2021-000220
Application 15/344,379

7

in the screen, or link the tag data to some even[t] shown in the screen.”
Perry, para. 83.
Thus, the Examiner’s combination of Norman and Perry constitutes an
improvement to Norman’s gaming system to tag data chunks 150 (video
game state data) in the same manner as taught by Perry to lead to a
predictable result, namely, to “direct[] later viewers to particularly
noteworthy parts of the game play by using the tags.” See Final Act. 7; see
also KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 417 (2007) (“[I]f a
technique has been used to improve one device, and a person of ordinary
skill in the art would recognize that it would improve similar devices in the
same way, using the technique is obvious unless its actual application is
beyond his or her skill.”).
In particular, we agree with the Examiner that each of Norman and
Perry discloses “that certain events may be of great importance to a user”,
and, thus, “the combination of Norman in view of Perry would allow for a
user to add [Perry’s] annotations within the gameplay of Norman.” See
Ans. 5, 6 (citing Norman, para. 48; Perry, para. 83). Accordingly, the
Examiner is correct that “adding in . . . [Perry’s] annotation ability would
make the playback of . . . [Norman’s] keyframes and chunks more
interesting to various users as a way to ‘comment’ on gameplay similar to
how sports commentators will highlight various actions of a previous play.”
Id. at 6. Thus, the Examiner’s modification of Norman, according to Perry,
is well within the skill of a person having ordinary skill in this art and
Appellant has not persuasively shown otherwise.
Lastly, we agree with the Examiner’s position that because “Norman
discloses that spectators of the game can vary which camera view point they
Appeal 2021-000220
Application 15/344,379

8

wish to view the game from and can switch said view point at any time,” in
the combination of Norman and Perry, “depending upon which camera view
a user chooses, the view of the annotated [video game] state will change
based on the selected camera.” Ans. 4 (citing Norman, para. 46), 5. Hence,
the Examiner is correct that “if a user selects one player’s camera view [in
the combination of Norman and Perry] the annotated event may appear on
the left, versus on the right from a different camera angle.” Id. at 5.
In conclusion, the Examiner’s findings of fact are supported by a
preponderance of the evidence and the Examiner’s legal conclusion of
unpatentability is well-founded. Therefore, for the foregoing reasons, we
sustain the rejection under 35 U.S.C. § 103 of claim 1 as unpatentable over
Norman and Perry. Claims 4–17 and 22 fall with claim 1.

Rejections II and III
Appellant relies on the same unpersuasive arguments discussed supra.
See Appeal Br. 16. Therefore, for the same reasons discussed above, we
likewise sustain the rejections under 35 U.S.C. § 103 of claims 18 and 20 as
unpatentable over Norman, Parker, and Perry and of claim 21 as
unpatentable over Norman, Perry, and Shaw.

CONCLUSION
The Examiner’s decision to reject claims 1, 4–18, and 20–22 is
affirmed.

Appeal 2021-000220
Application 15/344,379

9

DECISION SUMMARY
In summary:
Claim(s)
Rejected
35
U.S.C.
§
Reference(s)/Basis Affirmed Reversed
1, 4–17,
22
103 Norman, Perry 1, 4–17,
22

18, 20 103 Norman, Parker, Perry 18, 20
21 103 Norman, Perry, Shaw 21
Overall
outcome
1, 4–18,
20–22

TIME PERIOD FOR RESPONSE
No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a). See 37 C.F.R.
§ 1.136(a)(1)(iv) (2019).

AFFIRMED