If you prosecute patent applications in the machine learning space, you have probably encountered Section 101 rejections. One particular aspect in this space is training, and whether the United States Patent and Trademark Office (USPTO) considers training a machine learning model to be too abstract for patent protection under Section 101. This question hinges on the concept of the abstract idea test, a specific evaluation that one aspect of the determination of whether a patent claim falls within the boundaries of eligible subject matter.
While not the focus of this post, the abstract idea test established in Alice requires that a patent-eligible invention must involve more than just an abstract idea or fundamental concept. One approach to eligibility is for an invention to demonstrate a novel and practical application of the idea that solves a technical problem, thereby transforming it into something concrete and specific.
The USPTO acknowledges the challenges in navigating the patent landscape for machine learning technologies. While some machine learning-related inventions may seem abstract at first glance, the USPTO's guidelines provide a specific example where training a machine learning model can be deemed patentable. This example involves improving an existing technology or technical field, such as enhancing image processing or data analysis through a novel machine learning training method.
In Example 39 in the USPTO’s patent eligibility guidelines involves the following claim:
A computer-implemented method of training a neural network for facial detection, comprising:
collecting a set of digital facial images from a database;
applying one or more transformations to each digital facial image including mirroring, rotating, smoothing, or contrast reduction to create a modified set of digital facial images;
creating a first training set comprising the collected set of digital facial images, the modified set of digital facial images, and a set of digital non-facial images;
training the neural network in a first stage using the first training set;
creating a second training set for a second stage of training comprising the first training set and digital non-facial images that are incorrectly detected as facial images after the first stage of training; and training the neural network in a second stage using the second training set
In example 39, the preamble recites a method of training a neural network, and the body recites details of how the model is trained. According to the guidelines:
The claim does not recite any of the judicial exceptions enumerated in the 2019 PEG. For instance, the claim does not recite any mathematical relationships, formulas, or calculations. While some of the limitations may be based on mathematical concepts, the mathematical concepts are not recited in the claims. Further, the claim does not recite a mental process because the steps are not practically performed in the human mind. Finally, the claim does not recite any method of organizing human activity such as a fundamental economic concept or managing interactions between people. Thus, the claim is eligible because it does not recite a judicial exception.
Despite this guidance, some examiners and the Patent Trial and Appeal Board (PTAB) often find claims that recite training of a machine learning model to be abstract. The PTAB frequently distinguishes the USPTO's example, emphasizing that specific improvements and applications must be clearly defined and non-generic.
Consider a recent case where a Siemens patent application for a machine learning model's training method was denied by the PTAB. Appeal 2022-004881, Application 16/381,131. The claim on appeal is recited below, where the invention relates to a method of analyzing cardiac trajectory curves in medical images in the course of clinical decision making:
A method comprising:
training a machine learning model for automatically identifying regions of trajectory curves that correspond to a beginning of systole, a beginning of diastole, a middle of diastole, and an A-wave based on manually identified regions of trajectory curves;
automatically identifying regions of one or more trajectory curves that correspond to the beginning of systole, the beginning of diastole, the middle of diastole, and the A-wave using the trained machine learning model, the one or more trajectory curves representing cardiac movement;
determining features of interest associated with the identified regions, the features of interest comprising geometric measures of the identified regions of the one or more trajectory curves; and
generating a correspondence map by mapping the determined features of interest to clinical parameters, wherein a clinical decision is made based on the correspondence map.
Siemens cited example 39, but the PTAB was unconvinced. From the decision:
We do not agree with Appellant that mere recitation of training a machine learning model removes claim 1 from the realm of abstract ideas. Appellant reads Example VII too broadly. If we were to follow Appellant’s argument to its logical conclusion, every claim that recites machine learning would be patent eligible subject matter. However, contrary to Appellant’s position, we frequently distinguish Example VII and find that claims that recite machine learning or neural networks do contain abstract ideas. See e.g., Ex parte Petakov, Appeal No. 2022-004039, 2023 WL 4842002, *13 (PTAB July 10, 2023); Ex parte Wallach, Appeal No. 2018-006993, 2020 WL 2060453, *4–5 (PTAB April 23, 2020); Ex parte Beddo, Appeal No. 2017-009809, 2019 WL 3059860, *4–5 (PTAB July 8, 2019); Ex parte Medelius, Appeal No. 2018-002393, 2019 WL 2318912, *7 (PTAB May 23, 2019); Ex parte Bremel, Appeal No. 2018-007657, 2019 WL 2318982, *5 (PTAB May 20, 2019); Ex parte Wang, Appeal No. 2021-002149, 2022 WL 123230, *7–8 (PTAB Jan. 11, 2022).
First, the PTAB does not necessarily explain why the logical conclusion of Siemens’ argument is to be rejected out of hand. Every claim that recites a concrete non-abstract idea is patent eligible, so if training a machine learning model is not too abstract (per the guidelines), then there seems to be no issue with the “logical conclusion”. Nevertheless, moving on, the PTAB then cites a string of cases where training a machine learning model is found to be too abstract and thus ineligble. Again, this is not really a logical explanation, but rather just confirming a potentially false premise. As to actual reasoning, the PTAB provides the following:
…The Background Section to [example 39], which is not mentioned by Appellant in the Appeal Brief, explains that prior art facial detection methods suffer where there are shifts, distortions, and variations in scale and rotation of the face pattern in an image. In the invention of the example, this issue is addressed by using an expanded training set of facial images to train the neural network. The expanded training set is developed by applying mathematical transformation functions on an acquired set of facial images. The neural networks are then trained with this expanded training set. Introduction of an expanded training set increased false positives when classifying non-facial images. Thus, a second feature of the example invention minimizes these false positives by performing an iterative training algorithm, in which the system is retrained with an updated training set containing the false positives produced from non-facial images. The example invention provides robust facial detection while limiting the number of false positives. Id.
In contrast to Example VII, the instant application trains a machine learning model “based on manually identified regions of trajectory curves.” Claims App. claim 1. The claim explicitly contemplates that identifying regions of trajectory curves is a mental process. Id. The claimed machine learning model merely teaches a machine to do what the clinician already knows how to do. The Examiner does not err in finding that claim 1 recites
an abstract idea in the form of a mental process.
While this reasoning does not fully account for the analysis in the guidelines, it does illustrate how an applicant can structure claims, and a corresponding argument, to illustrate how the claims solves a technical problem, particularly in the area of machine learning model training.
To improve the chances that your machine learning-related patent claims are not deemed too abstract under Section 101, consider the following:
Specific Technical Improvements: Clearly outline the specific technical improvements achieved by the claimed machine learning model training to an existing technology or field, emphasizing novel and concrete applications.
Detailed Descriptions: Provide comprehensive and detailed descriptions of how your machine learning model (and the training thereof) functions, and how the specific training steps lead to the improvements noted above.
Be ware of generic language: using overly broad or generic features, such as generic training steps, can be interpreted as encompassing well-established machine learning techniques without any substantial improvements.
While specifically claiming the training of machine learning models is one way to obtain patent eligible protection for machine learning inventions, one must be careful to have sufficient elements that can be tied to specific technical problems in the field. The case above provides an outline, using the example 39 facts, for presenting arguments to examiners and the PTAB assuming there are sufficient details of the training in the claim.