Andreas Richter et al.

Patent Trials and Appeals Board

Jun 8, 2020

14331239 - (D) (P.T.A.B. Jun. 8, 2020)

UNITED STATES PATENT AND TRADEMARK OFFICE
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O. Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
14/331,239 07/15/2014 Andreas Richter 14-160 1098
83657 7590 06/08/2020
Michael Soderman
Tauentzienstr 9-12
Berlin, 10789
GERMANY
EXAMINER
KARKI, PARAS D
ART UNIT PAPER NUMBER
2626
NOTIFICATION DATE DELIVERY MODE
06/08/2020 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
ip@soderman.us
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
BEFORE THE PATENT TRIAL AND APPEAL BOARD
Ex parte ANDREAS RICHTER, BORIS VELICHKOVSKY,
SEBASTIAN PANNASCH, JENS HELMERT, and GEORGI PASCHEW
Appeal 2018-008167
Application 14/331,239
Technology Center 2600
Before MICHAEL J. STRAUSS, IRVIN E. BRANCH, and
ADAM J. PYONIN, Administrative Patent Judges.
BRANCH, Administrative Patent Judge.
DECISION ON APPEAL
STATEMENT OF THE CASE1
Pursuant to 35 U.S.C. § 134(a), Appellant2 appeals from the
Examiner’s decision to reject claims 1–12. See Final Act. 1. We have
jurisdiction under 35 U.S.C. § 6(b).
We AFFIRM.

1 We refer to the Specification, filed July 15, 2014 (“Spec.”); Final Office
Action, mailed August 30, 2017 (“Final Act.”); Appeal Brief, filed January
18, 2018 (“Appeal Br.”); and Examiner’s Answer, mailed May 17, 2018
(“Ans.”).
2 We use the word Appellant to refer to “applicant” as defined in 37 C.F.R.
§ 1.42. Appellant identifies the real party in interest as Technische
Universität Dresden. Appeal Br. 3.
Appeal 2018-008167
Application 14/331,239
2
CLAIMED SUBJECT MATTER
The claims are directed to a process and apparatus for haptic
interaction with visually presented data. Claim 1, reproduced below with
disputed limitations emphasized in italics, is illustrative of the claimed
subject matter:
1. An intermodal 3D percept system for generating a user’s
percept that virtual three-dimensional objects are real three-
dimensional objects that can be touched by a user’s hand and
seen three-dimensionally in real time comprising:
- at least one device for capturing three-dimensional data
of a virtual object in real time,
- at least one second device for capturing in real time
three-dimensional data of the user’s hand, comprising hand pose
and position data,
- at least one data-processing device for processing and
rendering the captured three-dimensional data of the virtual
object and of the user’s hand and for generating a merged visual
representation of the three-dimensional data of the virtual object
and of the user’s hand and for determining a collision point
between the visually presented user’s hand and the visually
presented virtual object,
- at least one visual subsystem with a display device for
visual presentation of the three-dimensional data of the virtual
object and of the user’s hand, wherein the user’s hand is also
reproduced in the visual subsystem as a visual reproduction,
- at least one tactile subsystem with a haptic element for
interaction with the user comprising a tactile display unit that can
be tilted and positioned in accordance with a surface to be
touched, wherein the haptic element imitates the surface
characteristics of the object in a collision area of the user’s hand
and the virtual object and reproduces, when said surface
characteristics exist, hardness, height, textures, reliefs, edges and
tactile constraints in the tactile area which are coordinated in time
with pseudo-color presentations, textures and visual contrasts
reproducing material characteristics in the visual subsystem, and
wherein the visual subsystem is arranged above the tactile
subsystem and the distance between the visual subsystem and the
Appeal 2018-008167
Application 14/331,239
3
tactile subsystem is 50 cm > x > 15 cm, preferably 40 cm > x >
20 cm, as a special preference 35 cm > x > 25cm.
REFERENCES
The prior art relied upon by the Examiner is:
Name Reference Date
Cruz-Hernandez
(“Hernandez”)
Karlsson
Banerjee
Mikhailov
Paschew

US 2010/0231550 A1
US 2012/0188179 A1
US 2014/0088941 A1
US 2016/0129346 A1
DE 102006036867 A1

Sept. 16, 2010
July 26, 2012
Mar. 27, 2014
May 12, 2016
Feb. 7, 2008

REJECTIONS
Claims 1–3, 7, 8, and 12 stand rejected under 35 U.S.C. § 103 as
unpatentable over the combination of Banerjee, Mikhailov, and Hernandez.
Final Act. 8–17.
Claims 4–6 stand rejected under 35 U.S.C. § 103 as unpatentable over
the combination of Banerjee, Mikhailov, Hernandez, and Paschew. Final
Act. 17–18.
Claims 9–11 are rejected under 35 U.S.C. § 103 as being unpatentable
over the combination of Banerjee, Mikhailov, Hernandez, and Karlsson.
Final Act. 19–28.
ANALYSIS
We have reviewed the Examiner’s rejections in light of Appellant’s
arguments. We have considered in this Decision only those arguments
Appellant actually raised in the Briefs. Any other arguments Appellant
could have made but chose not to make in the Briefs are deemed to be
waived. See 37 C.F.R. § 41.37(c)(1)(iv).
Appeal 2018-008167
Application 14/331,239
4
To the extent consistent with our analysis herein, we adopt as our own
the findings and reasons set forth by the Examiner in (1) the action from
which this appeal is taken (Final Act. 8–28) and (2) the Examiner’s Answer
in response to Appellant’s Appeal Brief (Ans. 3–52) and concur with the
conclusions reached by the Examiner. We highlight the following for
emphasis.
OPINION
With respect to the obviousness rejection of independent claim 1,
Appellant contends that the combination of Banerjee, Mikhailov, and
Hernandez does not teach or suggest the claimed invention because these
references do not consider “at all” “the things that are required to ‘trick’ a
person and to convey an impression of genuineness to that person” of “the
‘materialization’ of virtual objects in a virtual world in a form in which they
can be touched with one’s bare hand.” Appeal Br. 8. Appellant contends
that “[t]he percept and the data on the fixed tolerances between the actual
position of one’s own hand and the projected position of one’s own hand,
and also that of the object in the virtual world, are unique features.” Id.
Specifically, Appellant argues that Banerjee’s merely providing “force
feedback” to the user imparts “no sense of actually touching a real object”
and thereby “teaches away in a sense from the idea of forming a percept that
the object is real.” Id. at 9. We are not persuaded.
According to Appellant’s Specification, “[a] perception experience,
and thus the subjectively experienced, real-life, conscious (phenomenal)
result of a perception process, is called a [‘]percept[’] in psychology. In the
process, visual and kinesthetic impressions are combined by the brain into
an overall experience that is interpreted as real by the brain.” Spec. 4–5.
Appeal 2018-008167
Application 14/331,239
5
Appellant’s contention that mere force feedback is insufficient to impart a
precept—a sense of actually touching a real object—is not supported with
sufficient persuasive argument or evidence to convince us that the
contention is true, let alone that it “teaches away” from forming a precept.
Whether a precept is formed in the mind of a beholder based on mere force
feedback is potentially unknowable outside the user’s mind. Appellant’s
mere assertion does not convince us that Banerjee’s force feedback imparts
“no sense of actually touching a real object.” Appeal Br. 9. Accordingly,
this argument fails to persuade us of reversible error.
Appellant further argues that the cited references do not teach or
suggest “the capture of the hand pose data and the tilting and position of the
tactile display unit in accordance with a surface to be touched.” Appeal
Br. 10. We find this unpersuasive in view of the Examiner’s findings as
follows:
Examiner is relying on Mikhailov to teach the concept of
hands being rendered to be able to reach into the scene to provide
the appearance that the user is actually present in the 3d scene
where the user is holing a controller and the scene presented has
a simulated object (steering as shown in Fig. 5a and 5b).
Hernandez teaches providing texture to the haptic device
as taught in Figs. 6a to 6h as disclosed in [0073] to [0081] based
on the display. Thus, it would have been obvious to one of
ordinary skill in the art, before the effective filing date of the
application, to combine the teachings of Banerjee as modified by
Mikhailov and Hernandez to provide a feel for the texture of the
simulated haptic device in order to provide a more realistic
experience for the augmented reality when using Banerjee’s
stylus 27 or holding the controller (simulated steering wheel) of
Mikhailov.
Ans. 32. We agree the combination of Mikailov’s disclosure of rendering a
user’s hands holding a simulated device and Hernandez’s texture-producing
Appeal 2018-008167
Application 14/331,239
6
hapatic device teaches or suggests the argued limitation of the capture of the
hand pose data and the tilting and positioning of the tactile display unit in
accordance with a surface to be touched. We see no error and no persuasive
rebuttal to the Examiner’s findings and conclusion.3
Appellant also contends that “Hernandez, in fact, teaches away from
the visual subsystem of the instant application; the user’s hand could not be
shown in the haptic device of Hernandez.” Appeal Br. 10. We disagree.
“Obviousness may be defeated if the prior art indicates that the invention
would not have worked for its intended purpose or otherwise teaches away
from the invention.” Meiresonne v. Google, Inc., 849 F.3d 1379, 1382 (Fed.
Cir. 2017) (citing DePuy Spine, Inc. v. Medtronic Sofamor Danek, Inc., 567
F.3d 1314, 1326 (Fed. Cir. 2009)). The prior art teaches away “when a
person of ordinary skill, upon reading the reference, would be discouraged
from following the path set out in the reference, or would be led in a
direction divergent from the path that was taken” in the claim. Galderma
Labs., L.P. v. Tolmar, Inc., 737 F.3d 731, 738 (Fed. Cir. 2013). However, a
reference that “merely expresses a general preference for an alternative
invention but does not criticize, discredit, or otherwise discourage
investigation into” the claimed invention does not teach away. Id.
Appellant does not explain how Hernandez’s device being incapable of
showing a user’s hand, even if true, criticizes, discredits, or discourages
investigation into Appellant’s claimed invention. Furthermore, the
Examiner cites Mikhailov, not Hernandez, for a user’s hands being rendered
in the 3D scene (i.e., “capturing in real time three-dimensional data of the
user’s hand, comprising hand pose and position data”). Ans. 32.

3 Appellant did not file a Reply Brief.
Appeal 2018-008167
Application 14/331,239
7
Appellant’s contention that “Mikhailov still does not teach a capture
of the hand pose data . . . because Mikhailov does not concern itself at all
with an object to be felt by the hand” (Appeal Br. 11 (citing Mikhailov Figs.
5A and 5B)) is belied by the Examiner’s finding that Mikhailov’s disclosure
of hands being rendered to be able to reach into the scene to provide the
appearance that the user is actually present in the 3D scene where the user is
holding a controller. Final Act. 12–13.
Appellant argues that “[t]he specific dimensions in claim 1 . . . for
forming the surprising result of a percept - a feeling of reality - in the user
and were developed for that reason . . . were not stated in any of the cited
references.” Appeal Br. 12. This argument is unconvincing of error because
it does not persuasively rebut the Examiner’s finding that the dimension is at
least suggested by Banerjee. Ans. 33–34 (“It would have been obvious to
one of ordinary skill in the art to have those distance[s] in a work station
similar to Banerjee’s for ergonomic reasons.”).
Appellant also argues that “[a] statement that it would have been
obvious to combine references to get to some of the missing features of the
instant claims is a conclusory statement.” Appeal Br. 14. “[R]ejections on
obviousness grounds cannot be sustained by mere conclusory statements;
instead, there must be some articulated reasoning with some rational
underpinning to support the legal conclusion of obviousness.” In re Kahn,
441 F.3d 977, 988 (Fed. Cir. 2006); KSR Int’l Co. v. Teleflex Inc., 550 U.S.
398, 418 (2007) (quoting Kahn). “The pertinence of each reference, if not
apparent, must be clearly explained.” 37 C.F.R. § 1.104(c)(2). The
Examiner finds the reason for combining the teachings of references is to
give the user an enhanced virtual reality experience and to merge actions in
the physical and virtual environments. Final Act. 14, 15. In light of the
Appeal 2018-008167
Application 14/331,239
8
absence of evidence to rebut the Examiner’s findings and conclusion, we
find the Examiner has articulated reasoning with rational underpinnings
sufficient to justify the legal conclusion of obviousness and are unpersuaded
by Appellant’s argument that the Examiner’s combination of references is
inadequate or improper. Kahn, 441 F.3d at 988.
Appellant also argues as follows:
[B]road areas of features are missing from a first cited reference
(in addition to the overall point of the instant application), and
other references are then cited to specifically fill in these broad
areas of missing features. The other references could not in
reality be joined together, as explained above, and none of them
have any teaching or suggestion of the overall objective.
Additionally, elements (e.g. collision points) are still missing,
even in the combination of all of the references. All of this
strongly suggests that a search was simply done in an attempt to
find many of the features of the instant claims and that hindsight
was employed at least in part to get to the features of the instant
claims.
Appeal Br. 14.
Appellant further argues:
[A] prior patent cited as a § 103 reference must be considered in
its entirety, “i.e. as a whole, including portions that lead away
from the invention.” Id.
That is, the Examiner must recognize and consider not
only the similarities, but also the critical differences between the
claimed invention and the prior art as one of the factual inquiries
pertinent to any obviousness inquiry under 35 U.S.C. § 103. In
re Bond, 910 F.2d 831, 834 [15 USPQ2d 1566, 1568] (Fed. Cir.
1990) (emphasis added).
. . .
With regard to the instant application, many elements
themselves are non-obvious, but the synergistic whole makes
unexpected and advantageous results possible, for instance the
creation of a realistic “percept” for the user.
Appeal 2018-008167
Application 14/331,239
9
Id. at 15 (some emphasis omitted).
We find these arguments unavailing. We understand Appellant to
argue that the features of the cited references could not be bodily
incorporated with one another. “The test for obviousness is not whether the
features of a secondary reference may be bodily incorporated into the
structure of the primary reference. . . . Rather, the test is what the combined
teachings of the references would have suggested to those of ordinary skill
in the art.” In re Keller, 642 F.2d 413, 425 (CCPA 1981); In re Nievelt, 482
F.2d 965, 968 (CCPA 1973) (“Combining the teachings of references does
not involve an ability to combine their specific structures.”).
As to Appellant’s contention that determining “collision points” is
missing from the cited reference, it is unconvincing (Appeal Br. 15, 18)
because it does not persuasively address and rebut the Examiner’s finding
that Banerjee discloses this disputed feature (Final Act. 8–9 (citing Banerjee
¶¶ 30, 38)).
We also do not find Appellant’s contention that “many elements
themselves are non-obvious, but the synergistic whole makes unexpected
and advantageous results possible, for instance the creation of a realistic
‘percept’ for the user” amounts to an argument against obviousness. Appeal
Br. 15. We do not find “many elements” to be “non-obvious” for the
reasons discussed above.
For the foregoing reasons, we do not find error in the Examiner’s
rejection of claim 1. We sustain the rejection of claim 1 and we sustain the
rejections of the remaining claims, which are either argued on a similar basis
or are not argued separately with particularity. Appeal Br. 16–22.
Appeal 2018-008167
Application 14/331,239
10
DECISION SUMMARY
In summary:
Claims
Rejected
35 U.S.C. § References Affirmed Reversed
1–3, 7, 8, 12 103 Banerjee, Mikhailov,
Hernandez
1– 3, 7, 8,
12

4–6 103 Banerjee, Mikhailov,
Hernandez, Paschew
4–6
9–11 103 Banerjee, Mikhailov,
Hernandez, Karlsson
9–11
Overall
Outcome
1–12
TIME PERIOD FOR RESPONSE
No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a). See 37 C.F.R.
§ 1.136(a)(1)(iv).
AFFIRMED