Honeywell International Inc.

Patent Trials and Appeals Board

Dec 3, 2021

2020005842 (P.T.A.B. Dec. 3, 2021)

UNITED STATES PATENT AND TRADEMARK OFFICE
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O. Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
14/161,523 01/22/2014 Henry Chen H0038799-1456.1205101 8745
90545 7590 12/03/2021
HONEYWELL/STW
Intellectual Property Services Group
855 S. Mint Street
Charlotte, NC 28202
EXAMINER
MAHMUD, FARHAN
ART UNIT PAPER NUMBER
2483
NOTIFICATION DATE DELIVERY MODE
12/03/2021 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
Honeywell.USPTO@STWiplaw.com
patentservices-us@honeywell.com
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
BEFORE THE PATENT TRIAL AND APPEAL BOARD
Ex parte HENRY CHEN, HAO BAI, MICHELLE RAYMOND,
VIT LIBAL, JAN JELINEK, and VALERIE GURALNIK
Appeal 2020-005842
Application 14/161,523
Technology Center 2400

Before JAMES R. HUGHES, JOYCE CRAIG, and
MATTHEW J. McNEILL, Administrative Patent Judges.

HUGHES, Administrative Patent Judge.

DECISION ON APPEAL
STATEMENT OF THE CASE
Claims 1–3, 5–17, and 20 are pending, stand rejected, are appealed by
Appellant, and are the subject of our decision under 35 U.S.C. § 134(a).1
See Final Act. 1; Appeal Br. 6.2 We have jurisdiction under
35 U.S.C. § 6(b).
We AFFIRM.

1 We use the term “Appellant” to refer to “applicant” as defined in 37 C.F.R.
§ 1.42 (2019). Appellant identifies the real party in interest as Honeywell
International Inc. Appeal Br. 3.
2 We refer to Appellant’s Specification (“Spec.”), filed Jan. 22, 2014;
Appeal Brief (“Appeal Br.”), filed Mar. 4, 2020; and Reply Brief (“Reply
Br.”), filed Aug. 11, 2020. We also refer to the Examiner’s Final Office
Appeal 2020-005842
Application 14/161,523

2
CLAIMED SUBJECT MATTER
The claimed subject matter, according to Appellant, generally “relates
to devices, methods, and systems for operating a surveillance system.”
Spec. 1:4–5. Specifically, Appellant describes linking an image captured by
a video camera with a building information model (“BIM”) by projecting the
camera image onto a virtual image captured by a virtual video camera in the
BIM. See id. at 2:21–3:9. This linkage makes it possible “to determine
spaces of a facility where people and/or other items are located.” Id. at
3:10–12. It also allows for determining “spaces connected to that space and,
therefore, where the person is likely to be in the future” so that “a person’s
movement through the facility can be monitored.” Id. at 3:14–18. Claim 1
(directed to a method), claim 9 (directed to system), and claim 17 (directed
to a non-transitory computer-readable medium) are independent. Claim 1,
reproduced below, is illustrative of the claimed subject matter:
1. A method for operating a surveillance system
including a video camera installed at a particular location in a
facility, the method comprising:
displaying on a display a partially transparent projection
of an image captured by the video camera overlaid onto a virtual
image captured by a virtual video camera placed at a virtual
location in a building information model of the facility
corresponding to the particular location;
displaying on the display a position widget that allows a
user to manipulate a position of the image captured by the video
camera, wherein the position widget is displayed at or near a
center of the image captured by the video camera; and

Action (“Final Act.”), mailed Oct. 21, 2019; and Answer (“Ans.”), mailed
June 12, 2020.
Appeal 2020-005842
Application 14/161,523

3
displaying on the display a plurality of scale widgets that
allow the user to align a feature of the facility depicted in the
image captured by the video camera with a corresponding feature
of the virtual image, wherein each of the plurality of scale
widgets are displayed at a different respective corner of the
image captured by the video camera;
determining a plurality of camera parameters of the video
camera based at least in part upon inputs provided by the user
manipulating the position widget and/or one or more of the
plurality of scale widgets, wherein the plurality of camera
parameters include one or more of a position of the video camera
in the building information model, a resolution of the video
camera, a pan setting of the video camera, a tilt setting of the
video camera, a focal length of the video camera, an aspect ratio
of the video camera, and in image width of the video camera;
determining a two-dimensional geometry of the facility
based on the building information model, wherein the geometry
includes a plurality of spaces;
determining a coverage of the video camera based on a
portion of the plurality of camera parameters and the geometry;
and
determining which spaces of the plurality of spaces are
included in the coverage; and
associating each space included in the coverage with a
respective portion of the image.
Appeal Br. 17 (Claims App.).

Appeal 2020-005842
Application 14/161,523

4
REFERENCES
The prior art relied upon by the Examiner is:
Name Reference Date
Chen et al. (“Chen”) US 2009/0141966 A1 June 4, 2009
Plocher et al.
(“Plocher”)
US 2011/0285851 A1 Nov. 24, 2011
Thiruvengada et al.
(“Thiruvengada”)
US 2012/0306736 A1 Dec. 6, 2012

REJECTION3
The Examiner rejects claims 1–3, 5–17, and 20 under 35 U.S.C. § 103
as being unpatentable over Plocher, Thiruvengada, and Chen. See Final Act.
4–15.
ANALYSIS
Appellant argues claims 1–3 and 5–8 as a group based on independent
claim 1. Appeal Br. 6–10. Although Appellant provides nominally separate
arguments for independent claim 9 and dependent claims 10–16 (see Appeal
Br. 10–13), as well as independent claim 17 and dependent claim 20 (see
Appeal Br. 13–15), the arguments are substantially the same as those made
with respect to claim 1 or rely on the arguments made with respect to claim
1. We select independent claim 1 as representative of Appellant’s
arguments with respect to claims 1–3, 5–17, and 20. See 37 C.F.R.
§ 41.37(c)(1)(iv).

3 The Leahy-Smith America Invents Act (“AIA”), Pub. L. No. 112-29, 125
Stat. 284 (2011), amended 35 U.S.C. § 103. Because the present application
has an effective filing date after the AIA’s effective date (March 16, 2013),
this decision refers to the post-AIA version of 35 U.S.C. § 103.
Appeal 2020-005842
Application 14/161,523

5
The Examiner rejects independent claim 1 as obvious over Plocher,
Thiruvengada, and Chen. See Final Act. 4–6; Ans. 3. Specifically, the
Examiner finds that Plocher teaches each feature of claim 1 except for
“displaying on a display a partially transparent projection of an image
captured by the video camera overlaid onto a virtual image,” “displaying on
the display a position widget that allows a user to manipulate a position of
the image captured by the video camera, wherein the position widget is
displayed at or near a center of the image captured by the video camera,”
and “displaying on the display a plurality of scale widgets . . . wherein each
of the plurality of scale widgets are displayed at a different respective corner
of the image captured by the video camera.” Final Act. 4–5. The Examiner
relies on Thiruvengada for teaching the claimed widget features, and on
Chen for teaching the claimed partially transparent projection feature. Final
Act. 5–6.
Appellant contends that there would have been “no motivation to look
to Chen, or to even attempt to ‘include the transparency based alignment as
taught in Chen et al. in order to allow a user-friendly dynamic incremental
alignment technique’ as asserted by the Examiner.” See Appeal Br. 8. In
particular, Appellant points to the alleged failure of Plocher and
Thiruvengada “to teach or suggest a need for reconciling a ‘virtual image
captured by a virtual video camera placed at a virtual location in a building
information model’ and ‘an image captured by the video camera’” as a
reason why one would not have looked to Chen. Id. Appellant thus argues
that the Examiner’s combination is the result of improper hindsight. Id.
Appellant also contends that “Plocher cannot be seen to disclose
‘determining a plurality of camera parameters of the video camera based
Appeal 2020-005842
Application 14/161,523

6
upon inputs provided by the user manipulating the position widget and/or
one or more of the plurality of scale widgets’, as asserted by the Examiner.”
Appeal Br. 9. Appellant further argues Thiruvengada and Chen fail to cure
this deficiency. Id.
We adopt as our own (1) the findings and reasons set forth by the
Examiner in the action from which this appeal is taken (Final Act. 4–6) and
(2) the reasons set forth by the Examiner in the Examiner’s Answer (Ans. 4–
11) in response to Appellant’s Appeal Brief. We concur with the findings
and conclusions reached by the Examiner, and we provide the following
analysis for emphasis.
With respect to Appellant’s contention disputing the Examiner’s
combination, Appellant appears to argue that absent some teaching or
suggestion in Plocher or Thiruvengada, there would be no reason to combine
Chen’s alignment technique with those references. See Appeal Br. 8
(“[S]ince Plocher and Thiruvengada fail to teach or suggest a need for
reconciling a ‘virtual image . . .’ and ‘an image captured by the video
camera’, one of skill in the art would have absolutely no motivation to look
to Chen.”). But the Supreme Court has rejected such a narrow inquiry in
determining obviousness. See KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398,
419 (2007) (“The obviousness analysis cannot be confined by a formalistic
conception of the words teaching, suggestion, and motivation, or by
overemphasis on the importance of published articles and the explicit
content of issued patents.”). Rather, “the analysis need not seek out precise
teachings directed to the specific subject matter of the challenged claims, for
a court can take account of the interferences and creative steps that a person
of ordinary skill in the art would employ.” Id. at 418. We find one of
Appeal 2020-005842
Application 14/161,523

7
ordinary skill in the art would have considered Chen and have been
motivated to combine Chen’s teachings with Plocher and Thiruvengada.
As Appellant asserts, “Chen appears to be concerned with camera
calibration, and how to correlate a two dimensional image with the three
dimensional world.” Appeal Br. 8. In particular, Chen states that “[f]or
consumer applications, calibrated images are useful for geo-spatially
organizing captured photos for providing spatial context.” Chen ¶ 1. But,
Chen observes, “calibrating an image . . . is a tedious process” and often
requires that the user “supply a set of 3D-to-2D correspondences of points
and or lines so that the system can estimate the parameters that best fit these
measurements.” Id. ¶ 3. The application addresses a similar problem, in the
context of surveillance systems: “Previous approaches to surveillance
systems may not be linked to building architecture. Other previous
approaches may be linked by manual input . . . . [But] manually linking
surveillance systems to building architecture may become prohibitively
costly as architecture continues to become more complex.” Spec. 1:13–20.
“Accordingly, embodiments of the present disclosure can allow the linkage
of what is captured (e.g., seen) in a video image with a corresponding
location in a BIM.” Id. at 3:7–9. Based on the above, one of ordinary skill
in the art facing the problem described in the application would have looked
to Chen because it is analogous art, at least because it “is reasonably
pertinent to the particular problem with which the inventor [was] involved.”
In re Klein, 647 F.3d 1343, 1348 (Fed. Cir. 2011).
In addition, the creativity of an ordinarily skilled artisan would have
led one to combine Chen’s “user-friendly dynamic incremental alignment
technique,” in which “2D and 3D images are overlayed on each other and
Appeal 2020-005842
Application 14/161,523

8
are sufficiently transparent for visual alignment” (Chen ¶ 7), with Plocher
and Thiruvengada. As noted in KSR, “if a technique has been used to
improve one device, and a person of ordinary skill in the art would recognize
that it would improve similar devices in the same way, using the technique is
obvious unless its actual application is beyond his or her skill.” 550 U.S. at
417. Plocher describes using “semantic information” relating to a building’s
structure in combination with camera images to track intruders in a security
system. See Plocher ¶ 8. Plocher further describes exemplary display
implementations such as “popping up the best video (as estimated from
camera parameters, building structure and intruder location) and linking it to
the map or 3D scene, projecting the best video into the 3D scene, and so on.”
Id. ¶ 13. Here, it would have been obvious to apply Chen’s alignment
technique, in conjunction with Thiruvengada’s widgets, to carry out
Plocher’s linkage of camera video with a 3D scene. Appellant does not
specifically explain any shortcoming in the Examiner’s cited motivation
(Final Act. 6; Ans. 7), provided by Chen’s description of the alignment
technique as “user-friendly” (Chen ¶ 7), so we are not persuaded that the
combination is based on improper hindsight (see Appeal Br. 8–9; Reply Br.
2). Moreover, Appellant has not shown that it would have been beyond the
ordinary skill in the art to make such combination. See Appeal Br. 8
(“Applicants are not arguing, and have not argued, as asserted by the
Examiner in the Final Office Action, that ‘Chen could not be incorporated
into the teachings of Plocher et al. as modified by Thiruvengada et al.’”).
With respect to Appellant’s contention that Plocher fails to teach
determining camera parameters based on inputs from the claimed widgets
(see Appeal Br. 9), Appellant appears to misinterpret the Examiner’s
Appeal 2020-005842
Application 14/161,523

9
rejection. The Examiner does not rely solely on Plocher for disclosing the
disputed feature. Rather, the Examiner relies on the combination of Plocher
with Thiruvengada, which the Examiner finds teaches “a position widget”
and “a plurality of scale widgets.” Final Act. 5. Accordingly, Appellant’s
argument is not persuasive because it is not responsive to the Examiner’s
proposed combination.
In the Reply Brief, Appellant reiterates the contentions discussed
above, and further contends that the Examiner incorrectly finds “that the
‘semantic information’ referenced by Plocher et al. somehow teaches
incorporating information about a virtual image captured by a virtual
camera.” Reply Br. 3. Appellant also contends that Thiruvengada’s widgets
are “not the same as the claimed scale widgets that allow the user to align
features of a video image with those of a virtual image.” Id. at 4. Appellant,
however, forfeited these arguments by not raising them in the Appeal Brief.
See 37 C.F.R. § 41.41(b)(2) (“Any argument raised in the reply brief which
was not raised in the appeal brief, or is not responsive to an argument raised
in the examiner’s answer . . . will not be considered by the Board for
purposes of the present appeal, unless good cause is shown.”).
Nevertheless, even if considered, Appellant’s arguments are
unavailing. Appellant has not specifically explained why the semantic
information in Plocher fails to teach or suggest a virtual image captured by a
virtual video camera. That is, a virtual video camera is merely a way to
conceptualize data defining a space, and Appellant does not show that the
data of the claimed virtual image captured by the virtual video camera is any
different than Plocher’s semantic information about a building’s structure.
See Plocher ¶ 8. Further, Appellant does not specifically explain why
Appeal 2020-005842
Application 14/161,523

10
Thiruvengada’s widgets 127 and 128 for altering the shape of a footprint of a
video sensing device are any different than the claimed scale widgets. See
Thiruvengada ¶ 10. And in any case, the Examiner does not rely on
Thiruvengada alone for teaching “scale widgets that allow the user to align
features of a video image with those of a virtual image,” as Appellant
argues. Appeal Br. 4. Rather, the Examiner relies on the combination of
Plocher with Thiruvengada for this feature of claim 1. See Final Act. 4–5;
Ans. 8–11.
We note that Appellant also contends that the references fail to teach
certain additional limitations, but provides no specific arguments to support
these contentions. See, e.g., Appeal Br. 9 (“Moreover, none of the cited art,
whether taken individually or in combination, can be seen to teach or
suggest a method that includes . . . .”). Such conclusory arguments without
specific explanation are not persuasive of Examiner error. See, e.g., Ex
parte Belinne, Appeal 2009-004693, 2009 WL 2477843, at *4 (BPAI Aug.
10, 2009) (informative) (“Appellants’ argument. . . repeatedly restates
elements of the claim language[] and simply argues that the elements are
missing from the reference. However, Appellants do not present any
arguments to explain why the Examiner’s explicit fact finding is in error.”).
Therefore, for all the reasons discussed above, Appellant’s
contentions do not persuade us that the Examiner erred in rejecting claim 1
as obvious over Plocher, Thiruvengada, and Chen. Accordingly, we sustain
the Examiner’s obviousness rejection of independent claim 1, and dependent
claims 2, 3, and 5–8 not separately argued with particularity. See 37 C.F.R.
§ 41.37(c)(1)(iv). Although Appellant separately argues independent claims
9 and 17, Appellant presents substantially the same arguments as for claim
Appeal 2020-005842
Application 14/161,523

11
1. For the same reasons discussed above, we sustain the Examiner’s
obviousness rejection of independent claims 9 and 17, and dependent claims
10–16, and 20.
CONCLUSION
Appellant has not shown that the Examiner erred in rejecting claims
1–3, 5–17, and 20 as obvious under 35 U.S.C. § 103. We, therefore, sustain
the Examiner’s rejection of claims 1–3, 5–17, and 20.
DECISION SUMMARY
In summary:
Claim(s)
Rejected
35 U.S.C. § Reference(s)/Basis Affirmed Reversed
1–3, 5–17,
20
103 Plocher, Thiruvengada,
Chen
1–3, 5–17,
20

TIME PERIOD FOR RESPONSE
No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a). See 37 C.F.R.
§ 1.136(a)(1)(iv).
AFFIRMED