Ex Parte Mayfield et al

Patent Trial and Appeal Board

Mar 9, 2018

14152123 (P.T.A.B. Mar. 9, 2018)

United States Patent and Trademark Office
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O.Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
14/152,123 01/10/2014 Elijah Jacob Mayfield 143166.00101 1491
29880 7590 03/13/2018
FOX ROTHSCHILD LLP
PRINCETON PIKE CORPORATE CENTER
997 LENOX DRIVE
BLDG. #3
EAWRENCEVTT.TE, NJ 08648
EXAMINER
HONG, THOMAS J
ART UNIT PAPER NUMBER
3715
NOTIFICATION DATE DELIVERY MODE
03/13/2018 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
ipdocket @ foxrothschild. com
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
BEFORE THE PATENT TRIAL AND APPEAL BOARD
Ex parte ELIJAH JACOB MAYFIELD and
DAVID STUART ADAMSON
Appeal 2016-008243
Application 14/152,123
Technology Center 3700
Before DANIEL S. SONG, JAMES P. CALVE, and
BRANDON J. WARNER, Administrative Patent Judges.
CALVE, Administrative Patent Judge.
DECISION ON REQUEST FOR REHEARING
Appellants request rehearing of the Decision, mailed December 15,
2017, pursuant to 37 C.F.R. § 41.50(b). Requests for Rehearing are limited
to matters overlooked or misapprehended by the Board in rendering a
decision. See 37 C.F.R. § 41.52(a)(1).
The Request for Rehearing is DENIED.
Appeal 2016-008243
Application 14/152,123
In the Decision, we affirmed the Examiner’s rejection of claims 1—24
as directed to a judicial exception to 35 U.S.C. § 101 because the claims
were directed to mental processes, mathematical formulas, and manipulation
of data using generic computers and conventional functions. Dec. 4—10.
In their Request for Rehearing, Appellants argue that the claimed
method and system do not simply use a computer as a tool but significantly
improve upon prior art systems including the functioning of the computer.
Req. Reh’g 2. In particular, Appellants argue that prior art systems are not
fully automated, require human input, and lack artificial intelligence. Id.
Appellants argue that their improvements use extractors to extract a plurality
of feature values for each training essay, build a model by assigning a
probability to each of a plurality of combinations of class values and feature
values for the training essays, associate the model with a common prompt,
and apply the model to feature values of a candidate essay. Id. at 2—3.
Appellants’ argument that the claims “provide features of artificial
intelligence that were not available in prior essay scoring systems” is raised
for the first time in the Request for Rehearing and thus is not a persuasive
argument that we overlooked this matter in our Decision. Appellants do not
identify where they raised this issue in their Appeal Brief or Reply Brief. Id.
The term “artificial intelligence” does not appear in the Specification, which
describes methods and systems of evaluating textual responses with machine
learning. Spec. 134. However, the claims do not require machine learning.
Instead, the claims recite a computer-implemented method and system that
receive human assessments of training essays, extract features therefrom,
build a model by assigning a probability to combinations of class values and
feature values, and machine-generate a predicted score/grade for the essay.
2
Appeal 2016-008243
Application 14/152,123
Claim 1 recites “using one or more extractors to extract a plurality of
feature values for each of a plurality of features.” Appeal Br. 26 (Claims
Appendix). Claim 1 thus requires the use of only one extractor rather than a
“set of extractors.” Claim 1 does not recite how extractors are selected or
defined. Claim 1 does not recite any features of the extractors other than to
“extract a plurality of feature values for each of a plurality of features.” Id.
The Specification discloses that class and feature values can be received as
metadata or separate inputs associated with a document. Spec. 135. These
values are based on a human rating process and written rubric. Id. 141.
The claimed method and system are not fully automated, contrary to
Appellants’ suggestion. Req. Reh’g 2. They require human input, like prior
art systems, by “receiving a human assessment for the training essays,
wherein the human assessment comprises a class value that comprises a
grade or score of the training essay.” Appeal Br. 26, 29 (Claims
Appendix).1 They thus rely on a “human rating process [that] should follow
a written rubric, and the design and development of this rubric should be
iterated until humans reach a high level of inter-rater reliability.” Spec. 141.
Once the assessment is defined by humans this way, the system applies this
defined assessment to documents. Id. 142. The more closely the training
essays approximate future responses on essays, the better the model can
“replicate human evaluation of those future responses.” Id. 140; see id. 112
(prior art systems use expert-defined features for regression analysis). Any
1 Independent claim 11 recites “receive a class value for the training essay,
wherein the class value comprises a score or grade that resulted from human
evaluation of the training essay.” Id. at 30 (Claims Appendix).
3
Appeal 2016-008243
Application 14/152,123
machine-learning used in the method and system (id. ]Hf 34, 38, 42) is not
claimed.
Just as teachers traditionally identify a set of features in an essay that
provide a basis for scoring or grading the essay (id. 12), the claimed method
and system use “one or more extractors to extract a plurality of feature
values for each of a plurality of features” from training essays, build a model
that extracts this set of feature values from candidate essays, and predict a
score for each candidate essay based on mathematical probabilities assigned
to the class values (grades) and feature values (parts of answers). Thus, like
teachers in the past, the claimed method and system identify features in an
essay that are relevant to scoring an essay and determine what score to give
an essay by assigning probabilities to various combinations of features, i.e.,
that a particular set of features in an essay should receive a particular score.
The predicted score is based on human assessments of the training essays.
Such grading methods and systems are the quintessence of the mental
steps and processes that humans have undertaken in the past to grade essays.
Educators and prior art systems define a set of features that are correlated to
essay quality (e.g., essay length, text coherence) and assign values to each
feature in the set to assign scores to essays. Id. Tflf 2—3. Each “feature” is “a
unique, easily identifiable characteristic of a written text that, for a particular
document, can be associated with a numeric value.” Id. 121. That such
well-known mental steps and processes may be performed on a computer
does not transform the abstract idea into a patent eligible one.
Even the extraction of feature values using extractors can be done by
“extracting data from a document file, by receiving metadata or separate
inputs that are associated with the document, or by analyzing the document
4
Appeal 2016-008243
Application 14/152,123
through suitable methods such as optical character recognition (OCR).” Id.
135. Thus, feature values may be defined by humans. Cf. Req. Reh’g 3.
Therefore, just as teachers and prior art systems defined a rubric of
features to be used to grade essays, the claimed extraction identifies a list of
such features and incorporates those features into a model. Id. 141. If there
is novelty in the extraction process, it does not reside in the claims. To the
contrary, Appellants disclose extraction as receiving metadata or separate
input to prepare a list of feature values just as teachers and prior art systems
have used defined rubrics to grade essays in the past. Even if a document is
analyzed through OCR, the Specification does not explain how such OCR
analysis generates a list of feature values for each training essay. See id.
To the extent the claims recite computer components, the computer
components merely implement this abstract idea in conventional fashion.
No improvement to the computer functionality is claimed. The steps of data
extraction, data processing, data manipulation, calculations of probabilities,
manipulating existing information to generate additional information, and
display of results are basic functions that computers perform and do not
make such abstract ideas patent eligible. Dec. 7 (citations omitted).
The claimed model merely “builds a comprehensive set of probability
estimates for each feature with regard to a class value [wherein] [t]hese
probabilities now may estimate a feature’s likelihood of appearing in essays
that exhibit a given class value, rather than merely increasing or decreasing
the final estimated output score.” Spec. 172. A probable class value can be
either PASS or FAIL. Id. at || 82—85. The Specification discloses the use
of various undisclosed algorithms and Naive Bayes algorithms to predict
class values. However, the claims are not so limited even if these known
5
Appeal 2016-008243
Application 14/152,123
algorithms are employed in some new manner, of which we have no
evidence before us. See id. 38, 43, 53 62, 69—86.
The claimed method and system determine a “probable class value”
for an essay with a particular set of feature values and the probable class
value “comprises a machine-generated predicted grade or score for the
candidate essay.” In other words, the claimed model applies mathematical
formulas or probabilities, i.e., it performs calculations at a high level of
generality, to estimate a probable class value (score, grade). Id. 1—2, 41.
The claims are directed to the abstract idea of grading essays. That
the claims recite the use of probabilities to do so does not alter the fact that
the claims are directed to an abstract idea that uses mathematical formulas
and mental processes and mental steps that humans have long performed and
can perform using pen and paper. That the claims implement the abstract
idea on a generic computer using conventional functions does not make the
abstract idea patent eligible.2 The use of probabilities to score essays is not
shown to be an improvement over prior systems that calculate a fixed score.
Associating a model with a common prompt merely claims the use of
a particular model for a particular essay. It is known to use a selected rubric
to score a particular essay. Spec. Tflf 1—2, 41. The model’s accuracy depends
also on how well-defined a topic is so writers understand the type of essay
expected, how closely human-graded training essays approximate future
2 Appellants disclose conventional internal hardware components including
electrical bus 400, central processing unit 405 that is a processing device that
may include a single processing device or two or more processing devices,
read only memory 410, random access memory 415, controller 420, memory
devices 425, display interface 430, display 435, communication ports 440,
interfaces 445 for keyboard 450 or other inputs 455. Id. 87—91, Fig. 4.
6
Appeal 2016-008243
Application 14/152,123
responses to that essay, and whether the written rubric used to grade training
essays has a high level of inter-rater reliability. See id. 39-41.
It is not even clear that the claimed method and system improve the
abstract idea of grading essays. We have no evidence of unexpected results
or improved accuracy in predicted grades or scores. The alleged improved
“functioning of the electronic device by extracting and discovering features
without being limited to a specific set of curated features that are identified
by human authors” (Req. Reh’g 3) is contradicted by Appellants’ disclosure
that to build a model, the system receives class value and feature values by
receiving metadata or separate inputs associated with a document. Spec.
135. Thus, some human involvement can occur to create class values and
feature values for a model. In addition, a human rating process that follows
a written rubric is used to develop assessment evaluations that the system
applies to documents. Id. H 41 42. The design of the rubric should be
iterated until humans reach a high level of inter-rater reliability. Id. 141.
The Federal Circuit’s decision in Core Wireless Licensing S.A.R.L. v.
LCElec., Inc., 880 F.3d 1356 (Fed. Cir. 2018) does not alter our decision.
In fact, it supports our decision that the mere automation of an abstract idea
on generic computers that perform conventional functions of data collection,
extraction, processing, calculations, and display do not transform an abstract
idea into patent eligible subject matter absent an improvement in computer
functionality. Dec. 8—10. In Core Wireless, the claims were directed to an
improved user interface for computing devices with small screens that thus
“disclose a specific manner of displaying a limited set of information to the
user, rather than using conventional user interface methods to display a
generic index on a computer.” 880 F.3d at 1362—63. Here, the claims only
7
Appeal 2016-008243
Application 14/152,123
require a display device, audio device, or transmitter to output a predicted
grade or score of a probable class value. Appeal Br. 26 (Claims Appendix).
The claimed display, audio device, or transmitter outputs a predicted
grade or score in a known, conventional manner. Appellants disclose that a
display interface 430 may permit information from bus 400 to be displayed
on display 435 in audio, visual, graphic, or alphanumeric format. Spec.
190. Appellants provide no other information regarding the displays or the
display process (for example, display 435 is illustrated as a box in Figure 4).
Improvements to an abstract idea itself are still abstract ideas that are
not patent eligible. See Visual Memory LLC v. NVIDIA Corp., 867 F.3d
1253, 1258 (Fed. Cir. 2017) (“[W]e must. . . ask whether the claims are
directed to an improvement to computer functionality versus being directed
to an abstract idea.” (internal quotations omitted); Enfish, LLC v. Microsoft
Corp., 822 F.3d 1327, 1335-36 (Fed. Cir. 2016) (“[T]he first step in
the Alice inquiry in this case asks whether the focus of the claims [was] on
the specific asserted improvement in computer capabilities ... or, instead,
on a process that qualifies as an ‘abstract idea’ for which computers are
invoked merely as a tool.”). Limiting an invention to a technological
environment does “not make an abstract concept any less abstract under step
one.” Intellectual Ventures I LLC v. Capital One Fin. Corp., 850 F.3d 1332,
1340 (Fed. Cir. 2017). Claiming the improved speed or efficiency inherent
with applying the abstract idea on a computer does not provide a sufficient
inventive concept. Intellectual Ventures I LLC v. Symantec Corp., 838 F.3d
1307, 1315 (Fed. Cir. 2016). The claims are not directed to a solution to a
“technological problem.” In re TLI Commc’ns LLP Patent Litig., 823 F.3d
607, 613 (Fed. Cir. 2016). They are not shown to improve computers or
8
Appeal 2016-008243
Application 14/152,123
even grading systems. They rely on human assessments. As discussed
below, they are broad enough to encompass prior art automated systems.
In Finjan, Inc. v. Blue Coat Systems, Inc., 879 F.3d 1299 (Fed. Cir.
2018), the Federal Circuit based its finding of patent eligibility on the fact
that the claimed virus screening “constitutes an improvement in computer
functionality.” 879 F.3d at 1304. The claims in Finjan provided a non­
abstract improvement in computer functionality by enabling more flexible
and nuanced virus filtering including protecting against previously unknown
viruses and viruses that are cosmetically modified to avoid detection by
code-matching virus scans. Id. at 1304—05.
It is not clear how the claimed method and system improve computers
or known essay grading methods or systems that also score essays using an
applied rubric of values. The claimed method and system determine “a
probable class value” for a candidate essay, where “the probable class value
comprises a machine-generated predicted grade or score for the candidate
essay.” Appellants disclose that such “predicting” “produc[es] an output of
probability estimations for each possible class value for a document.” Spec.
132. The Specification discloses probability estimates for multiple scores
(class values) for a document may be calculated. See id. ]Hf 52, 53 Fig. 1
(step 125). In such case, the most probable class value may be used to select
a candidate value as the predicted value, or the system may output multiple
predictions with confidence values associated with each prediction. Id. 1 53.
Alternatively, the probabilities may be reduced to a single predicted class
value (the most probable of all options). Id. Still other algorithms that do
not assign probabilities to each class values may be used. Id. In view of
these disclosures, it is not clear whether the claims provide an actual grade
9
Appeal 2016-008243
Application 14/152,123
or score for an essay and, if so, exactly what type of score(s) is calculated
and displayed, or how they differ from conventional grade or score.
Appellants disclose that known automated grading schemes analyze as
few as twelve features with simple algorithms like linear regression, whereas
Appellants’ method and system can use thousands of features. Id. 1 65. As
a result, Appellants indicate that linear regression “may not be a suitable
algorithm for determining a score class given a set of features.” Spec, f 65
(emphasis added). However, the claims merely require the extraction and
use of “a plurality of features.” Appeal Br. 26 (Claims Appendix). Thus,
known linear regression algorithms may be used in the claimed method and
system. See id. 1—2, 65, 88. The claimed model and machine-generated
predicted grade or score encompass software functionality used in admitted
prior art grading systems. Id. 2, 27. The claimed probabilities can be
calculated using known regression models. See id. ]Hf 1—2, 65, 88.
The accuracy of the predictive elements also depends on how closely
training essays approximate the range of responses provided in the candidate
essays. Id. 140. However, the claims do not recite features that improve the
accuracy of predicting class values and scores in this way. Moreover, using
a larger number of features, as compared to the prior art, while potentially
increasing accuracy, does not necessarily impart patentability.
Nor do Appellants explain how the claimed method and system can
improve the functioning of a computer sufficiently to render the abstract idea
patent eligible. See Req. Reh’g. 5. The prior art and claimed method and
system grade essays by extracting feature values from essays according to a
rubric/model. Spec. Tflf 1—3. The claims do not specify a quantity or quality
of feature values to distinguish over prior art systems. The extraction can
10
Appeal 2016-008243
Application 14/152,123
receives class values and feature values from a document file, metadata, or
separate inputs versus from experts in prior art systems. Id. 2, 35.
The claimed method and system merely obtain feature values from a
set of essays that were graded by humans and apply those feature values as a
rubric to candidate essays to predict a score for those candidate essays based
on the feature values in each candidate essay. Id. Tflf 3, 32, 37, 39, Fig. 1.
Appellants have not claimed improved computer functionality in extracting
feature values from training or candidate essays. Nor do Appellants claim
technological improvements in the way a model is built from extracted class
values and feature values or the way a model is applied to candidate essays.
The predictive elements depend on training essays that vary in quality and
writer skill level and approximate responses in candidate essays. Id. 140.
Like conventional grading systems, the claimed method and system
extract feature values from candidate essays to be graded and apply a model
to the extracted feature values to determine a probable class value and a
predicted score. The claims thus recite conventional computer functions of
data extraction, calculations according to mathematical formulas, and the
display of the results of the data calculations. Unlike the claims in Finjan,
we have no evidence the claimed method and system improve the accuracy
or efficiency of the grading process or system. Finjan, 879 F.3d at 1304 (the
behavior-based scans analyze a downloadable program’s code to determine
if it performs potentially dangerous and unwanted operations like renaming
or deleting files and thus can protect against previously unknown viruses as
well as “obfuscated code” (known viruses that are cosmetically modified to
avoid detection by prior art code-matching virus scans)). In contrast, the
claimed method and system try to replicate human grading of essays. Spec.
11
Appeal 2016-008243
Application 14/152,123
140. The model’s reliability depends on the inter-relater reliability of a
written rubric used by human graders of the training essays. See id. 141.
The Examiner analyzed the claims and limitations and determined that
they were directed to steps that can be performed in the human mind, or by a
human using a pen and paper, as well as also organizing information through
mathematical correlations and mathematical relationships and algorithms.
Final Act. 2-4; Ans. 2—3. Appellants did not address these findings of the
Examiner. See Appeal Br. 11—20. We agreed with the Examiner. Dec. 5—6.
That a computer may perform data extraction or calculations more quickly
or efficiently than a human does not impart patent eligibility where the
processes and calculations are conventional ones performed using generic
computers. Id. at 6—8. As discussed above, extracting features from essays
has been done by educators for a long time to grade essays. Even automated
essay grading systems extract key features related to grading of the essays.
Spec. H1—2. To grade an essay, the grader must identify features of the
essay that relate to a grading rubric, positively, negatively, or otherwise.
If features in an essay do not match perfectly with a grading rubric, a
grader must determine how to grade the essay where responses do not match
perfectly with a rubric’s proposed answer key. The claims use probabilities
to capture this ambiguity inherent in grading essays. However, Appellants
have put forth no evidence that any of the extraction, filtering, classification,
and probability calculations produce more accurate grading of essays. The
extracted features may include words, character strings, similar elements,
semantic analysis, word size, sentence size. Id. 144. The extractor may
convert a text into a set of labeled numeric values representing the contents
12
Appeal 2016-008243
Application 14/152,123
of that text. Id. 45 46. Even the prediction process often selects several
candidate class values (grades) for a text. Id. 1 53.
We are not persuaded that the claimed method and system represent
an improvement in computer functionality or other relevant technology that
elevates the claims from an abstract idea or that provides an inventive step to
make the abstract idea patent eligible, whether the claims are analyzed
individually or as an ordered combination. See Dec. 8—10; Ans. 3—7.
To summarize, extracting feature values from essays to build a model
to score candidate essays (Req. Reh’g 2—3) is an abstract idea and process
that educators and teachers have done to develop a rubric to grade essays.
Spec. 1—2, 41. Prior art automated grading systems do this as well using
expert-designed features. Id. 11. The extraction and building steps also
involve mental processes that people undertake to build a rubric for grading
essay. They also involve standard data manipulation steps that routinely
extract pertinent data from documents that are abstract ideas.
Building a model by assigning a probability to combinations of class
values and feature values for training essays is an abstract idea that people
have performed when preparing rubrics. Using mathematical formulas to
assign probabilities merely replicates the human thought process of figuring
out what particular combinations of features yield particular scores with a
particular degree of certainty. It also addresses situations where features of
an essay do not fall squarely into a particular class value (grade). Thus, an
essay may not have sufficient combination of features to yield an A, B, C, D,
or F. The reduced probability of an essay having such a grade may equate to
a grade such as A+, A-, B+, B-, and so forth in the interstices. The claimed
probabilities mimic teachers evaluating essay answers that do not match the
13
Appeal 2016-008243
Application 14/152,123
features of a written rubric perfectly and require extrapolation from a rubric
to match actual essay answers to grade an essay with a level of certainty.
The steps of associating a model with a common prompt (essay
question) and applying the model to feature values of an essay to determine
a probable class value for the candidate essay (Req. Reh’g 3) are directed to
an abstract idea of grading essays as educators have done. The proper rubric
must be used to grade each essay. Spec. Tflf 1—2, 41. Teachers or graders
apply the rubric to essays to be scored as mental processes that also may be
performed with pen and paper. Prior art automated systems apply expert-
defined rubrics and curated features to essays to score the essays. Id. 12.
These steps also involve conventional computer functions applying formulas
and mathematical calculations to extracted data to determine a score.
Using probabilities is an abstract idea that recognizes the common
sense notion that candidate essays to be scored may present different feature
values than a rubric. Each combination may be different or unique in ways
that require extrapolation or approximation to yield a score from a written
rubric. Thus, the more accurately training essays approximate the range of
answers from candidate essays, the more likely “a classifier will may [sic] be
able to replicate human evaluation of those future responses.” Id. 140.
Prior art systems use regression analysis to approximate or extrapolate
a score from different combinations of different extracted values to emulate
the mental process of grading essays. Id. 12. Appellants also use regression
analysis for small numbers of features values, which the claimed method and
system encompass by not requiring a large numbers of features. Id. 1 65.
As discussed in the Decision, our reviewing court treats such mental
processes, data manipulation steps, mathematical formulas and traditional
14
Appeal 2016-008243
Application 14/152,123
processes automated with generic computers to perform such conventional
functions as directed to abstract ideas that are not patent eligible. Dec. 5—10.
For all of the foregoing reasons, we are not persuaded by Appellants’
arguments that we have overlooked any matter on this appeal.
TIME PERIOD FOR RESPONSE
No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a)(l)(iv).
REHEARING DENIED
15