Ex Parte Suzuki et al

Patent Trial and Appeal Board

Jan 23, 2017

13231344 (P.T.A.B. Jan. 23, 2017)

United States Patent and Trademark Office
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O.Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
13/231,344 09/13/2011 Toshiaki SUZUKI 5298-6 7745
27562 7590 01/25/2017
NIXON & VANDERHYE, P.C.
901 NORTH GLEBE ROAD, 11TH FLOOR
ARLINGTON, VA 22203
EXAMINER
LEE, KWANG B
ART UNIT PAPER NUMBER
2617
NOTIFICATION DATE DELIVERY MODE
01/25/2017 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
PTOMAIL@nixonvan.com
pair_nixon @ firsttofile. com
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
BEFORE THE PATENT TRIAL AND APPEAL BOARD
Ex parte TOSHIAKI SUZUKI and SHIGEFUMI KAWASE
Appeal 2016-002098
Application 13/231,344
Technology Center 2600
Before ERIC B. CHEN, DANIEL N. FISHMAN, and
DAVID J. CUTITTAII, Administrative Patent Judges.
CHEN, Administrative Patent Judge.
DECISION ON APPEAL
Appeal 2016-002098
Application 13/231,344
This is an appeal under 35 U.S.C. § 134(a) from the final rejection of
claims 1 and 3—32. Claim 2 has been cancelled. We have jurisdiction under
35 U.S.C. § 6(b). We affirm-in-part.
STATEMENT OF THE CASE
Appellants’ invention relates to generating images by combining real
images and virtual images such as computer graphics images. (Abstract.)
Claims 1, 14, and 26 are exemplary with disputed limitations in
italics:
1. A non-transitory computer readable medium
storing an image processing program that is executed by a
computer which displays a synthesized image of a real world
image and a virtual image on a display device, the image
processing program causing the computer to perform
functionality comprising:
acquiring a real world image captured by a real camera;
defining first and second virtual regions in a virtual
image based on a boundary surface adjoining the first and
second virtual regions, the first virtual region located inside the
boundary surface and the second virtual region located outside
the boundary surface;
generating a synthesized image by synthesizing the real
world image and the virtual image depicting a first virtual
object, in such a manner that the first virtual object appears to
be present behind the real world image when the first virtual
object is in the second virtual region; and
displaying the synthesized image on the display device.
14. The non-transitory computer readable medium
storing the image processing program according to claim 1,
wherein the real world image formed with an opening in a
portion thereof is used for generating the synthesized image in
2
Appeal 2016-002098
Application 13/231,344
which the first virtual object is visible via this opening in the
generating the synthesized image.
26. The non-transitory computer readable medium
storing the image processing program according to claim 1,
wherein a portion of the first virtual object, appearing to be
present behind the real world image, is depicted as a silhouette
of the first virtual object.
Claims 1, 3—12, 20, 21, and 23—32 stand rejected under 35 U.S.C.
§ 103(a) as unpatentable over Tomite (US 2008/0024523 Al; Jan. 31, 2008),
Deb (US 2010/0085351 Al; Apr. 8, 2010), and Meier (US 2010/0289817
Al; Nov. 18,2010).
Claims 13—15, 18, 19, and 22 stand rejected under 35 U.S.C. § 103(a)
as unpatentable over Tomite, Deb, Meier, Bassett (US 2004/0219980 Al;
Nov. 4, 2004).1
Claims 16 and 17 stand rejected under 35 U.S.C. 103(a) as
unpatentable over Tomite, Deb, Meier, Bassett, and Cheng (US
2008/0293488 Al; Nov. 27, 2008).
ANALYSIS
§103 Rejection—Tomite, Deb, and Meier
Claims 1. 3-12. 20. 21. 23-25. and 28-32
First, we are unpersuaded by Appellants’ arguments (App. Br. 13—14)
that the combination of Tomite, Deb, and Meier would not have rendered
obvious independent claim 1, which includes the limitation “defining first
1 Appellants do not present any separate arguments with respect to the
rejection of dependent claim 13 under 35 U.S.C. § 103(a). Thus, any such
arguments are deemed to be waived.
3
Appeal 2016-002098
Application 13/231,344
and second virtual regions in a virtual image based on a boundary surface
adjoining the first and second virtual regions, the first virtual region located
inside the boundary surface and the second virtual region located outside the
boundary surface.”
The Examiner found that the head mounted display of Tomite, which
generates a virtual image of a celestial sphere, corresponds to the limitation
“defining first and second virtual regions in a virtual image based on a
boundary surface adjoining the first and second virtual regions, the first
virtual region located inside the boundary surface and the second virtual
region located outside the boundary surface.” (Final Act. 4; see also Ans. 2—
3.) We agree with the Examiner.
Tomite relates to “generating images combining real images and
virtual images such as computer graphics images (CG images)” (12), such
that “a CG image [is] combined with a real space image (i.e., a background),
according to which a user can feel as if a virtual object is present in the real
space” (14). Figure 2 of Tomite illustrates information processing
apparatus 100, which includes head mounted display 101 having a video
camera and carried by user 201. (133.) Tomite explains that “information
processing apparatus 100 can generate a virtual image of a celestial sphere
206 by connecting together (e.g., stitching) a plurality of real space images
(real images) 205 captured by the video camera.” (Id.) Furthermore, Tomite
explains that “information processing apparatus 100 generates a virtual
object 204 having natural brightness and shadow based on the estimated
light source information.” (| 34.) Because Figure 2 of Tomite illustrates
celestial sphere 206 having an interior and an exterior, Tomite teaches the
limitation “defining first and second virtual regions in a virtual image based
4
Appeal 2016-002098
Application 13/231,344
on a boundary surface adjoining the first and second virtual regions, the first
virtual region located inside the boundary surface and the second virtual
region located outside the boundary surface.”
Appellants argue “even assuming for sake of argument that the area
inside of the half-sphere in Fig. 2 constitutes a virtual region, Tomite does
not image objects from a virtual camera outside of the virtual region (e.g., in
a second virtual region beyond the half-sphere).” (App. Br. 14) In
particular, Appellants argues,
even if the area outside of the half-sphere constitutes a “second
virtual region,” Tomite does not position virtual objects in the
“second virtual region,” nor does Tomite generate a synthesized
image such that the virtual object appears behind a real world
object when the object is in the second region.
{Id. (emphasis omitted).) However, the Examiner cited to Deb, rather than
Tomite, for teaching the limitation “generating a synthesized image” with
images in the “first and second virtual regions.” (Final Act. 5; see also
Ans. 3^4.)
Thus, we agree with the Examiner that the combination of Tomite,
Deb, and Meier would have rendered obvious independent claim 1, which
includes the limitation “defining first and second virtual regions in a virtual
image based on a boundary surface adjoining the first and second virtual
regions, the first virtual region located inside the boundary surface and the
second virtual region located outside the boundary surface.”
Second, we are unpersuaded by Appellants’ arguments (App. Br. 15-
lb; see also Reply Br. 4) that the combination of Tomite, Deb, and Meier
would not have rendered obvious independent claim 1, which includes the
limitation “generating a synthesized image” with images in the “first and
second virtual regions.”
5
Appeal 2016-002098
Application 13/231,344
The Examiner found that the virtual camera of Deb, having different
fields of view for multiple objects, corresponds to the limitation “generating
a synthesized image” with images in the “first and second virtual regions.”
(Final Act. 5; see also Ans. 3—4.) We agree with the Examiner.
Deb relates to “a set of user interface tools for adjusting a region of
focus for rendering the space from a particular location within a particular
field of view.” (Abstract.) In particular, Deb explains that the field of view
includes: (i) a first region within the region of focus; and (ii) a second
region outside the region of focus. (Id.) Figure 3 of Deb illustrates
modification of the depth of field properties for virtual camera 105 (114),
including objects 111, 112, and 113, and focal plane 115 (134), such that
“objects 112 and 113 will both appear to be in focus, while blurring effects
are applied to object 111” (^j 39). Similarly, Figure 7 of Deb illustrates
“virtual camera 705, and three objects 710, 715, and 720, at different
distances from the virtual camera.” (161.) Because Deb illustrates that the
depth of field can be modified for three objects, Deb teaches the limitation
“generating a synthesized image” with images in the “first and second
virtual regions.”
Appellants argue that
Deb is not rendering objects so that they appear in front of a real
world surface when they are in a region on one side of a boundary
surface, and rendering objects so that they appear behind the real
world surface when they are in another region on the other side
of a boundary surface.
(App. Br. 15—16 (emphases omitted).) Similarly, Appellants argue that
“Deb only defines arbitrary regions for the purpose of determining whether
the virtual camera should focus on a particular object, and not whether it
should appear in front of, or behind, a real world object.” (Reply Br. 4.)
6
Appeal 2016-002098
Application 13/231,344
However, the Examiner cited Meier, rather than Deb for teaching the
limitation “the real world image and the virtual image depicting a first
virtual object, in such a manner that the first virtual object appears to be
present behind the real world image.” (Final Act. 5—6; see also Ans. 4—5.)
Thus, we agree with the Examiner that the combination of Tomite,
Deb, and Meier would have rendered obvious independent claim 1, which
includes the limitation “generating a synthesized image” with images in the
“first and second virtual regions.”
Last, we are unpersuaded by Appellants’ arguments (Reply Br. 3—5)
that the combination of Tomite, Deb, and Meier would not have rendered
obvious independent claim 1, which includes the limitation “the real world
image and the virtual image depicting a first virtual object, in such a manner
that the first virtual object appears to be present behind the real world image
when the first virtual object is in the second virtual region.”
The Examiner found that Figure 1 of Meier, which illustrates merging
real and virtual objects, such that the virtual object is behind the real object,
corresponds to the limitation “the real world image and the virtual image
depicting a first virtual object, in such a manner that the first virtual object
appears to be present behind the real world image when the first virtual
object is in the second virtual region.” (Final Act. 5—6; see also Ans. 4—5.)
We agree with the Examiner.
Meier relates to “merging the virtual object with an image of the real
environment generated by a recording device.” (11.) Figure 1 of Meier
illustrates scenery 1 in which “[t]he virtual object is executed ... in the form
of a sofa 3 which is to be placed behind a real object, here a table 2.” (138.)
Because Meier illustrates that sofa 3, a virtual object, is behind table 2, a real
7
Appeal 2016-002098
Application 13/231,344
object, Meier teaches the limitation “the real world image and the virtual
image depicting a first virtual object, in such a manner that the first virtual
object appears to be present behind the real world image.”
Appellants argue that “nothing in Meier suggests the concept of
having two regions in virtual space where an object will appear in front of a
real world object when in a particular region in the virtual space” and “Meier
appears to define a ‘room’ based on information provided from a user (e.g.,
where the location of walls are in an image) and then use image recognition
techniques to determine where real objects are in the image.” (Reply Br. 4.)
However, the Examiner cited to Deb, rather than Meier, for teaching the
limitation “generating a synthesized image” with images in the “first and
second virtual regions.” (Final Act. 5; see also Ans. 3—4.)
Thus, we agree with the Examiner that the combination of Tomite,
Deb, and Meier would have rendered obvious independent claim 1, which
includes the limitation “the real world image and the virtual image depicting
a first virtual object, in such a manner that the first virtual object appears to
be present behind the real world image when the first virtual object is in the
second virtual region.”
Accordingly, we sustain the rejection of independent claim 1 under 35
U.S.C. § 103(a). Claims 3—12, 20, 21, 28, and 29 depend from claim 1, and
Appellants have not presented any substantive arguments with respect to
these claims. Therefore, we sustain the rejection of claims 3—12, 20, 21, 28,
and 29 under 35 U.S.C. § 103(a), for the same reasons discussed with
respect to independent claim 1.
Independent claims 23, 24, 25, and 30 recite limitations similar to
those discussed with respect to independent claim 1, and Appellants have
8
Appeal 2016-002098
Application 13/231,344
not presented any additional substantive arguments with respect to these
claims. (See, e.g., App. Br. 18.) We sustain the rejection of independent
claims 23, 24, 25, and 30, as well as dependent claims 31 and 32, not argued
separately, for the same reasons discussed with respect to claim 1.
Dependent Claims 26 and 27
We are persuaded by Appellants’ arguments (App. Br. 17—18; see also
Reply Br. 7—8) that the combination of Tomite, Deb, and Meier would not
have rendered obvious dependent claim 26, which includes the limitation
“wherein a portion of the first virtual object, appearing to be present behind
the real world image, is depicted as a silhouette of the first virtual object.”
The Examiner found that the natural shadow for the virtual object of
Tomite and mapping captured images onto the rectangular parallelepiped of
Tomite, rather than the celestial sphere, collectively correspond to the
limitation “wherein a portion of the first virtual object, appearing to be
present behind the real world image, is depicted as a silhouette of the first
virtual object.” (Final Act. 19; see also Ans. 7—8.) We do not agree.
As discussed previously, Tomite explains that “information
processing apparatus 100 generates a virtual object 204 having natural
brightness and shadow based on the estimated light source information.”
(| 34.) Furthermore, Tomite explains that “in a case where modeling for
preliminarily determining the shape and layout of walls and objects in the
real space is available, the celestial sphere image generation method may
include mapping the captured image 205 on such a modeling shape” such
that “[t]he modeling shape is, for example, a rectangular parallelepiped
resembling a room.” (| 48.)
9
Appeal 2016-002098
Application 13/231,344
Although the Examiner cited to virtual object 204 of Tomite having a
natural shadow and mapping captured image 205 of Tomite onto a
rectangular parallelepiped, collectively, the Examiner has provided
insufficient evidence to support a finding that Tomite teaches the limitation
“wherein a portion of the first virtual object, appearing to be present behind
the real world image, is depicted as a silhouette of the first virtual object.”
In particular, Tomite is silent with respect to a shadow formed on the real
space rectangular parallelepiped (or celestial sphere 206), much less
placement of virtual object 204 “behind” such real space rectangular
parallelepiped (or celestial sphere 206). Accordingly, we are persuaded by
Appellants’ arguments that “Tomite does not at all indicate that objects
within any region are depicted as a silhouette image, let alone depicted as a
silhouette image when the object appears to be present behind a real world
image.” (App. Br. 17 (emphases omitted).)
Alternatively, the Examiner found that the virtual sofa of Meier, as
illustrated in Figure 1, corresponds to the limitation “wherein a portion of
the first virtual object, appearing to be present behind the real world image,
is depicted as a silhouette of the first virtual object.” (Ans. 7—8.) In
particular, the Examiner found that “a part of the virtual object is located
behind a real object and the form of a sofa to be placed behind a real object
as a silhouette of the first virtual object in Fig. 1.” {Id. at 8.) Again, we do
not agree.
As discussed previously, Figure 1 of Meier illustrates scenery 1 in
which “[t]he virtual object is executed ... in the form of a sofa 3 which is to
be placed behind a real object, here a table 2.” (| 38.) Figure 1 illustrates
10
Appeal 2016-002098
Application 13/231,344
that sofa 3 is depicted as dashed lines behind table 2, such that table 2
overlays sofa 3.
Although the Examiner cited to Figure 1 of Meier, the Examiner has
provided insufficient evidence to support a finding that “the form of a sofa to
be placed behind a real object as a silhouette of the first virtual object.”
(Ans. 8.) In particular, Figure 1 of Meier does not illustrate that sofa 3,
located “behind” table 2 is depicted as a silhouette. Instead Figure 1 of
Meier merely illustrates that table 2 overlays sofa 3. Accordingly, we are
persuaded by Appellants’ argument that “the sofa in Fig. 1 is illustrated
using dashed lines to show its physical placement in the image behind the
table” and “Meier does not at all appear to indicate that the sofa is shown as
a silhouette when it is behind the table.” (Reply Br. 7 (emphasis omitted).)
Thus, we do not agree with the Examiner that the combination of
Tomite, Deb, and Meier would have rendered obvious dependent claim 26,
which includes the limitation “wherein a portion of the first virtual object,
appearing to be present behind the real world image, is depicted as a
silhouette of the first virtual object.”
Accordingly, we do not sustain the rejection of dependent claim 26
under 35 U.S.C. § 103(a). Claim 27 depends from dependent claim 26. We
do not sustain the rejection of claim 27 under 35 U.S.C. § 103(a) for the
same reasons discussed with respect to dependent claim 26.
§103 Rejection—Tomite, Deb, Meier, and Bassett
We are also persuaded by Appellants’ arguments (App. Br. 19-21; see
also Reply Br. 8—9) that the combination of Tomite, Deb, Meier, and Bassett
would not have rendered obvious dependent claim 14, which includes the
11
Appeal 2016-002098
Application 13/231,344
limitation “wherein the real world image formed with an opening in a
portion thereof is used for generating the synthesized image in which the
first virtual object is visible via this opening in the generating the
synthesized image.”
The Examiner found that the “picture-in-picture” display of Bassett
from the first and second virtual cameras, corresponds to the limitation
“wherein the real world image formed with an opening in a portion thereof is
used for generating the synthesized image in which the first virtual object is
visible via this opening in the generating the synthesized image.” (Final
Act. 26; see also Ans. 10.) We do not agree.
Bassett relates to “dynamically manipulating camera angle to provide
special effects” for three-dimensional video game play. (12.) Figure 10 of
Bassett illustrates a “‘picture-in-picture’ display so that the video game
player can continue to watch the image from the perspective of the first
camera 308a while also having the benefit of an interesting, different image
from the perspective of the second camera 308b.” (1 86.)
Although the Examiner cited to Figure 10 of Bassett, the Examiner
has provided insufficient evidence to support a finding that Bassett teaches
the limitation “wherein the real world image formed with an opening in a
portion thereof is used for generating the synthesized image in which the
first virtual object is visible via this opening in the generating the
synthesized image.” In particular, Figure 10 of Bassett illustrates a “picture-
in-picture” display of the image from second virtual camera 308b formed in
the image from the first virtual camera 308a, rather than a virtual image in a
real image, as required by claim 14.
12
Appeal 2016-002098
Application 13/231,344
Accordingly, we are persuaded by Appellants’ arguments that
“Bassett explains that one virtual camera can be used to follow the moving
object where another virtual camera can be used to show the object at a
position lateral to the moving object” and “one of ordinary skill would not
consider a picture-in-picture window as an opening, let alone an opening in a
real world image.” (App. Br. 20 (emphases omitted); see also Reply Br. 9.)
Thus, we do not agree with the Examiner that the combination of
Tomite, Deb, Meier, and Bassett would have rendered obvious dependent
claim 14, which includes the limitation “wherein the real world image
formed with an opening in a portion thereof is used for generating the
synthesized image in which the first virtual object is visible via this opening
in the generating the synthesized image.”
Accordingly, we do not sustain the rejection of dependent claim 14
under 35 U.S.C. § 103(a). Claims 15, 18, 19, and 22 depend from dependent
claim 14. We do not sustain the rejection of claims 15, 18, 19, and 22 under
35 U.S.C. § 103(a) for the same reasons discussed with respect to dependent
claim 14.
§103 Rejection—Tomite, Deb, Meier, Bassett, and Cheng
Claims 16 and 17 depend from claim 14. Cheng was cited by the
Examiner for teaching the additional features of claims 16 and 17. (Final
Act. 30-32.) However, the Examiner’s application of Cheng does not cure
the above noted deficiencies of Tomite, Deb, Meier, and Bassett. Thus, we
do not sustain the rejection of claims 16 and 17 under 35 U.S.C. § 103(a) for
the same reasons discussed with respect to dependent claim 14.
13
Appeal 2016-002098
Application 13/231,344
DECISION
The Examiner’s decision rejecting claims 1, 3—13, 20, 21, 23—25, and
28—32 is affirmed.
The Examiner’s decision rejecting claims 14—19, 22, 26, and 27 is
reversed.
No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a)(l)(iv).
AFFIRMED-IN-PART
14