Rainer Graumann et al.

Patent Trials and Appeals Board

Aug 19, 2019

13285493 - (D) (P.T.A.B. Aug. 19, 2019)

UNITED STATES PATENT AND TRADEMARK OFFICE
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O. Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
13/285,493 10/31/2011 RAINER GRAUMANN 2011P10334 6074
27350 7590 08/19/2019
LERNER GREENBERG STEMER LLP
Box SA
P.O. BOX 2480
HOLLYWOOD, FL 33022-2480
EXAMINER
FERNANDEZ, KATHERINE L
ART UNIT PAPER NUMBER
3793
NOTIFICATION DATE DELIVERY MODE
08/19/2019 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
boxsa@patentusa.com
docket@patentusa.com
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
__________

BEFORE THE PATENT TRIAL AND APPEAL BOARD
__________

Ex parte RAINER GRAUMANN, GERHARD KLEINSZIG,
YOSHITO OTAKE, and JEFFREY SIEWERDSEN

__________

Appeal 2017-010151
Application 13/285,493
Technology Center 3700

__________

Before ERIC B. GRIMES, RYAN H. FLAX, and TIMOTHY G. MAJORS,
Administrative Patent Judges.

MAJORS, Administrative Patent Judge.

DECISION ON APPEAL
Appellants1 submit this appeal under 35 U.S.C. § 134 involving
claims to a method for localization of a structure including, inter alia, steps
of acquiring a preoperative 3D computer-tomography (CT) image and an
intraoperative two dimensional (2D) x-ray image. The Examiner rejected
the claims as obvious. We have jurisdiction under 35 U.S.C. § 6(b).
We AFFIRM.

1 Appellants identify the Real Party in Interest as The Johns Hopkins
University and Siemens Aktiengesellschaft. Br. 1.
Appeal 2017-010151
Application 13/285,493

2
STATEMENT OF THE CASE
Appellants’ “invention relates to a method for localization and
identification of a structure in a projection image with a system having a
known system geometry.” Spec. ¶ 2.
The Specification explains that “[w]rong site surgery is a surprisingly
common error in medical practice with major ramification to the patient and
healthcare system.” Id. ¶ 3. Using spinal surgery as an example, the
Specification further explains that a wrong site surgery may involve a
procedure that is performed at the wrong level, such as at an incorrect one of
the vertebrae, which can otherwise have a similar visual and radiographic
appearance. Id. According to the Specification, the described invention
overcomes disadvantages of known localization and identification methods
and “significantly decreases the number of wrong-level surgeries and . . . is
independent of the surgeon’s ability to localize and/or identify a target level
in the body.” Id. ¶ 4.
Claims 1, 3, and 8 are on appeal.2 Claim 1, the only independent
claim, is illustrative and reads as follows:
1. A method for localization and identification of a structure
with a system having a known system geometry, the method
comprising the following steps:

2 Appellants’ brief (see, e.g., Br. 1, 10) indicates that claim 9 was finally
rejected and is subject to this appeal. Claim 9, however, was newly
submitted via amendment dated April 28, 2016. And the Examiner indicated
in the August 10, 2016 Final Rejection that claim 9 was directed to a non-
elected invention and was, accordingly, withdrawn from consideration on
the merits. Final Act. 2–3; 37 C.F.R. § 1.142(b). Claim 9 is, thus, not part
of the final rejection on appeal and will not be further considered by the
Board in this decision.
Appeal 2017-010151
Application 13/285,493

3
a) acquiring a preoperative 3D computer-tomography
(CT) image of a structure;
b) preprocessing the 3D CT image to generate a volume
image by segmenting the structure in the 3D CT image;
c) acquiring an intraoperative two dimensional (2D) X-ray
image;
d) preprocessing the 2D X-ray image to obtain a line
integral of a linear attenuation coefficient from 2D image
intensity values to generate a fixed image;
e) estimating an approximate pose of the structure;
f) calculating a digitally reconstructed radiograph (DRR)
using:
the volume image,
the estimated approximate pose and
the system geometry;
g) comparing the calculated DRR and the fixed image by
a similarity measure based on gradient information;
h) updating the estimated approximate pose in order to
increase the similarity measure;
i) repeating steps f) to h) until the similarity measure
reaches a criterion of optimization and then displaying the
estimated approximate pose to represent a localized and
identified structure, wherein in each repetition of step f), the
updated estimated approximate pose that was previously
calculated in a most recent performance of step h) is used as the
estimated approximate pose; and
j) overlaying the calculated DRR from a last repetition of
step f) onto the fixed image.

Br. 9–10 (Claims App’x).
Appeal 2017-010151
Application 13/285,493

4
The claims stand rejected as follows:
Claims 1 and 3 under 35 U.S.C. § 103(a) as obvious over Khamene,3
Edic,4 Raanes,5 Pluim,6 and Cosman.7 Final Act. 3–11; Ans. 2–10.
Claim 8 under 35 U.S.C. § 103(a) as obvious over Khamene, Edic,
Raanes, Pluim, Cosman, and Boese.8 Final Act. 11–12; Ans. 10–11.
DISCUSSION
Claims 1 and 3
Appellants do not separately argue the patentability of claims 1 and 3.
Our discussion, thus, focuses on independent claim 1. 37 C.F.R.
§ 41.37(c)(1)(iv).
The Examiner finds that Khamene discloses a method having all the
limitations of claim 1, with four exceptions. Final Act. 3–6. First, the
Examiner finds that Khamene discloses step (d)’s preprocessing of the
intraoperatively acquired 2D x-ray image to form a fixed image,9 but does
not specify that the image processing involves obtaining a line integral of a
linear attenuation coefficient from 2D image intensity values. Id. at 6.

3 Khamene et al., US 2006/0188139 A1, published Aug. 24, 2006.
4 Edic et al., US 2004/0120449 A1, published June 24, 2004.
5 Raanes et al., US 2009/0003523 A1, published Jan. 1, 2009.
6 Josien P. W. Pluim et al., Image Registration by Maximization of
Combined Mutual Information and Gradient Information, 19:8 IEEE
TRANSACTIONS ON MEDICAL IMAGING 809–814 (2000).
7 Cosman, US 2002/0065461 A1, published May 30, 2002.
8 Boese et al., US 7,689,019 B2, issued Mar. 30, 2010.
9 According to the Examiner, a “fixed image” is represented by a segmented
2D-distance map or a “real segmented projection image” in Khamene. Final
Act. 4 (citing Khamene ¶¶ 17–19, 30).
Appeal 2017-010151
Application 13/285,493

5
According to the Examiner, however, that deficiency is remedied by Edic,
which the Examiner finds teaches a successful means of image generation
involving preprocessing and calibration to obtain a line integral of linear
attenuation coefficients from 2D image intensity values. Final Act. 7 (citing
Edic ¶ 31); see also Ans. 17 (“The processed data was then used to
successfully formulate/generate an image with improved temporal
resolution.”) (citing Edic, Abstract, ¶¶ 30–31). The Examiner reasons it
would have been obvious to use this known and successful x-ray image
preprocessing technique, which the Examiner finds provides improved
resolution, in Khamene’s method. Final Act. 6–7; Ans. 16–17.
Second, the Examiner finds that, although Khamene discloses, as in
claim 1’s step (g), comparing the calculated DRR (a simulated 2D projection
image reconstructed from a 3D CT image) and the fixed image (from an
intraoperatively acquired 2D x-ray image) based on a similarity measure,
Khamene does not specify that the measure is based on gradient information.
Final Act. 5 (citing e.g., Khamene ¶¶ 16, 18–19, 25–30, and Fig. 2). The
Examiner finds, however, that Pluim teaches that gradient information is a
useful similarity measure for registration of clinical images. Final Act. 8–9
(citing Pluim, Abstract, 810, 813). Because Pluim teaches that use of
gradient information provides a more robust and accurate similarity
measure, the Examiner reasons it would have been obvious to use gradient
information for the similarity measure of Khamene. Id.; see also Ans. 12–
13; Pluim 809 (“Image gradients by themselves have been shown to be
useful registration criteria”).
Appeal 2017-010151
Application 13/285,493

6
Third, although the Examiner finds that Khamene discloses that its
system may include a display, the Examiner states that Khamene does not
specifically teach step (i)’s “displaying” the estimated approximate pose to
represent a localized and identified structure as recited in claim 1. Final Act.
6 (citing Khamene ¶ 14); see also Khamene Fig. 1 (structure 116 (display
unit)). Nevertheless, the Examiner finds that Raanes describes image-guided
radiation treatment systems that involve comparing 2D in-treatment x-ray
images with 2D digitally reconstructed radiographs (DRRs) derived from 3D
pre-diagnostic treatment images (see Raanes ¶¶ 5–6) and discloses
displaying information, such as 2D or 3D representations, on a display to
assist the user. Final Act. 8; Raanes ¶ 91. According to the Examiner, it
would have been obvious to display the estimated approximate pose (i.e., a
2D representation) based on the combined teachings of Khamene and
Raanes in order to provide a user presentation and allow the user to visually
assess the structure of interest. Final Act. 8.
Fourth and finally, the Examiner finds that although Khamene’s
method seeks to closely match the synthetic projection image (DRR) to the
fixed image (generated from an intraoperative x-ray image), Khamene does
not disclose “overlaying” the DRR and fixed image as in claim 1’s step (j).
Final Act. 6. The Examiner turns to Cosman, finding it teaches a surgical
positioning system that uses reconstructed radiographs (DRRs) in
comparison with actual x-ray images taken intraoperatively. Final Act. 9
(citing Cosman ¶ 68, Fig. 4). According to the Examiner, Cosman teaches
the use of “overlay analysis” and that “[b]ased on the comparative images
(i.e. overlaid images) from such reconstructed and actual X-ray views,
Appeal 2017-010151
Application 13/285,493

7
further patient/beam corrective movements . . . may be made/planned.” Id.
The Examiner concludes it would have been obvious to overlay Khamene’s
DRR images onto a fixed image projection, as suggested by Cosman, to
provide a comparative image that can be used for corrective movements
(e.g., of the patient) during an operation. Final Act. 9–10.
We agree with, and adopt, the Examiner’s findings and conclusion
that claim 1 would have been obvious over Khamene, Edic, Raanes, Pluim,
and Cosman. Final Act. 4–11. On the record here, we credit and agree with
the Examiner’s findings that Khamene teaches the method of claim 1, but for
those distinctions discussed above. Final Act. 4–5 (identifying where
Khamene teaches or suggests the limitations of each of steps (a)–(i) of claim
1); see, e.g., Khamene ¶¶ 16–19, 30, Fig. 2. Those distinctions are,
however, known image processing and display techniques, for assessment
and comparison of x-ray, CT, DRR, and other clinical images, as evidenced
by the remaining references. And the Examiner has provided technical
reasoning, drawn largely from the prior art of record, to explain why the
skilled artisan would have included those techniques in Khamene’s method
to arrive at the claimed subject matter.
Appellants first argue that “Pluim concerns registering one three-
dimensional image with another three-dimensional image” and the “method
disclosed in Pluim is only applicable to registering a three-dimensional
image with another three-dimensional image.” Br. 4. According to
Appellants, there is no teaching “in Pluim [that] could be used to register a
three-dimensional image with a two-dimensional image as is the case in
Khamene.” Id. Moreover, Appellants contend, it is not clear from Pluim
Appeal 2017-010151
Application 13/285,493

8
how one would even make the required calculations between a three-
dimensional and two-dimensional space, “[n]or is there any teaching or
suggestion that such a result would be valid or give advantageous results.”
Id. at 4–5.
Appellants’ argument is unpersuasive. Appellants’ argument
misconstrues the rejection, the claims, and the prior art. Ans. 11–12. The
Examiner is not suggesting Pluim’s teachings be used to register a 3D image
with a 2D image; rather the “Examiner is suggesting that the gradient
information be used for the similarity metric . . . to compare the 2D DRR
and the 2D fixed image disclosed by Khamene.” Id. As the Examiner
explains in detail (with supporting evidence), a DRR is a two-dimensional
image that is derived from a 3D CT image. Id. at 12 (citing, e.g., Khamene
¶ 30); see also Raanes ¶ 5 (disclosing that DRRs are “2D digitally
reconstructed radiographs (DRRs) derived from three dimensional (3D) pre-
treatment diagnostic imaging data of the patient”). The Examiner also cites
disclosure in Pluim teaching more generally that “[i]mage gradients by
themselves have been shown to be useful registration criteria.” Ans. 13
(citing Pluim 809). And, the Examiner explains that Pluim teaches that
gradient information can be determined in individual “slices”—2D images.
Ans. 13 (citing Pluim Fig. 2). Appellants provide no persuasive argument or
evidence to the contrary.10

10 Appellants did not file a Reply Brief here to respond further to the
Examiner’s Answer. Moreover, insofar as the Appellants’ argument
suggests there would be something unclear or difficult for the skilled person
in applying Pluim’s teachings to Khamene, which otherwise discloses claim
1’s step (g) of comparing a DRR and fixed image based on a similarity
Appeal 2017-010151
Application 13/285,493

9
Appellants also argue the rejection is “logically flawed” because
“Khamene discloses a system and method for two-dimensional to three-
dimensional registration.” Br. 5. Putting aside that Appellants state (twice)
in their briefing that Khamene relates to registering a three-dimensional
image with a two-dimensional image,11 Appellants’ argument fails to
identify what, if any, actual limitation(s) of claim 1 are allegedly missing in
Khamene or the art as combined. The Examiner has explained, for example,
where Khamene teaches obtaining a 3D CT image, how that is processed to
generate a volume image, from which a synthetic/simulated 2D projection
image (DRR) is constructed, and also where Khamene discloses acquiring an
intraoperative 2D x-ray image, and the Examiner has explained why that
further processed x-ray image meets the “fixed image” limitation of claim 1,
against which the calculated DRR is compared. Final Act. 4; Ans. 3–4, 6–7;
see, e.g., Khamene ¶¶ 16–19, 30. Appellants do not persuasively
demonstrate error in the Examiner’s findings.
Appellants also contend that Raanes and Cosman “do not contain
anything” to suggest modifying Khamene’s method of alleged two-

measure, that argument is undercut by the Specification itself. The
Specification makes only a passing mention to “gradient information” as a
useful similarity measure and cites Pluim as support. Spec. ¶ 30 (“One
possible similarity measure that can be used here is gradient information
proposed by Pluim et al.”). If use of gradient information, as taught in
Pluim, in a method of 2D to 2D or 3D to 2D registration was beyond the
skilled artisan’s normal capabilities, one would expect a more complete
explanation in the Specification, not simply a citation to Pluim.
11 See Br. 4 (“. . . to register a three-dimensional image with a two-
dimensional image as is the case in Khamene”) (emphasis added), 5 (same).
Appeal 2017-010151
Application 13/285,493

10
dimensional to three-dimensional registration. Br. 5–6. This argument fails
for the reasons above—the Examiner has explained sufficiently on this
record where Khamene discloses the steps of claim 1 and Appellants do not
show otherwise. As for whether the skilled artisan would have considered
the teachings of Raanes and Cosman, we agree with the Examiner here that,
because each of Khamene, Raanes, and Cosman relate to acquiring and
using diagnostic images (DRRs, x-ray images) for interventional radiation
treatments, the skilled person had reasons for considering the teachings of
each reference. Ans. 14–15. The reasons for combining the relevant
teachings are discussed above.
Lastly, Appellants state that they “believe the person of ordinary skill
in the art would not have obtained any suggestion to have modified the
preprocessing step in Khamene by obtaining a line integral of a linear
attenuation coefficient from 2D image intensity values,” as taught in Edic.
Br. 6–7. According to Appellants, “[t]his is due to the fa[c]t that Khamene
needs to segment the 2D x-ray projection so that the distance map can be
extracted.” Id. at 7.
On this record, we are unpersuaded. The Examiner explains why the
proposed modification of the art (incorporating Edic’s preprocessing
technique) is consistent with Khamene’s method and teaching. Ans. 16
(explaining that “in Khamene, the segmentation process follows after a full
unsegmented [pre-processed] image is generated,” and “preprocessing of the
2D X-ray image to generate a fixed image is considered to encompass the
processing steps that occur prior to generating the ‘real segmented
projection images’ (i.e. fixed image) which is compared to the calculated
Appeal 2017-010151
Application 13/285,493

11
DRR.”). Appellants do not provide any argument or evidence to
demonstrate that the Examiner’s reasoning is technically flawed or otherwise
incorrect.
For the above reasons, we determine that the preponderance of the
evidence on this record supports the Examiner’s conclusion that claim 1
would have been obvious over Khamene, Edic, Raanes, Pluim, and Cosman.
Claim 3 was not argued separately and falls with claim 1.
Claim 8
The Examiner rejected claim 8 over Khamene, Edic, Raanes, Pluim,
Cosman, and Boese. Final Act. 11–12. According to the Examiner, Boese
discloses acquiring images with a C-arm device, and the Examiner
concludes it would have been obvious to use a C-arm device to acquire x-
rays in the method suggested by the combined references. Id.
Appellants do not provide separate, substantive argument related to
claim 8. Appellants contend claim 8 is patentable “for the reasons given
above with regard to claim 1.” Br. 7. Appellants’ arguments on claim 1 are
unpersuasive as already explained. We adopt the Examiner’s findings and
conclusion of obviousness for claim 8 on this record. Final Act. 11–12.
Conclusion of Law
The preponderance of the evidence on this record supports the
Examiner’s conclusion that claims 1, 3, and 8 would have been obvious.
SUMMARY
We affirm the rejections for obviousness on appeal.

Appeal 2017-010151
Application 13/285,493

12

TIME PERIOD FOR RESPONSE
No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a).

AFFIRMED