Red Hat Israel, Ltd.

Patent Trials and Appeals Board

Jan 27, 2022

2020004383 (P.T.A.B. Jan. 27, 2022)

UNITED STATES PATENT AND TRADEMARK OFFICE UNITED STATES DEPARTMENT OF COMMERCE United States Patent and Trademark Office Address: COMMISSIONER FOR PATENTS P.O. Box 1450 Alexandria, Virginia 22313-1450 www.uspto.gov APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO. 15/053,899 02/25/2016 Michael Tsirkin 50833.56US01/20151249US 8663 110773 7590 01/27/2022 Haynes and Boone, LLP (49145/50833) Attn. IP Docketing 2323 Victory Avenue , Suite 700 Dallas, TX 75219 EXAMINER FIELDS, COURTNEY D ART UNIT PAPER NUMBER 2436 NOTIFICATION DATE DELIVERY MODE 01/27/2022 ELECTRONIC Please find below and/or attached an Office communication concerning this application or proceeding. The time period for reply, if any, is set in the attached communication. Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the following e-mail address(es): ipdocketing@haynesboone.com PTOL-90A (Rev. 04/07) UNITED STATES PATENT AND TRADEMARK OFFICE BEFORE THE PATENT TRIAL AND APPEAL BOARD Ex parte MICHAEL TSIRKIN and PAOLO BONZINI Appeal 2020-004383 Application 15/053,899 Technology Center 2400 Before TERRY J. OWENS, MAHSHID D. SAADAT, and JOHN A. EVANS, Administrative Patent Judges. SAADAT, Administrative Patent Judge. DECISION ON APPEAL STATEMENT OF THE CASE Pursuant to 35 U.S.C. § 134(a), Appellant1 appeals from the Examiner’s decision to reject claims 1, 2, 4-10, and 12-22. We have jurisdiction under 35 U.S.C. § 6(b). We REVERSE. 1 We use the word Appellant to refer to “applicant” as defined in 37 C.F.R. § 1.42. Appellant identifies the real party in interest as Red Hat Israel, Ltd. Appeal Br. 3. Appeal 2020-004383 Application 15/053,899 2 CLAIMED SUBJECT MATTER The claims are directed to “securing application code and protecting it from being executed while the system is in kernel mode” and more specifically, to “a method of securing an application running on a guest includes detecting, by a guest running on a virtual machine, that a set of physical memory pages is allocated to an application” by setting the physical memory pages “to an executable-by-user mode in the hypervisor’s page tables.” See Spec. ¶¶ 4-5. Claim 1, reproduced below, illustrates the claimed subject matter: 1. A method of securing an application running on a guest, comprising: detecting, by a guest running on a virtual machine, that a set of physical memory pages is allocated to an application, the virtual machine running on a hypervisor, and the application running on the guest; sending, by the guest, a request that causes the hypervisor to set the set of physical memory pages to an executable-by- user mode in the hypervisor’s page tables, wherein each memory page that is set to the executable-by-user mode in the hypervisor’s page tables is executable in user space and is not executable in kernel space; and after sending the request, detecting, by the guest, execution of a physical memory page of the set of physical memory pages by the application. REJECTION Claims 1, 2, 4-10, and 12-22 stand rejected under 35 U.S.C. § 103 as being unpatentable over Sallam (US 2012/0254995 A1, pub. Oct. 4, 2012) and Banginwar (US 2016/0364341 A1, pub. Dec. 15, 2016). Final Act. 3- 10. Appeal 2020-004383 Application 15/053,899 3 OPINION We have reviewed the Examiner’s rejection in light of Appellant’s contentions in the Appeal Brief and the Reply Brief that the Examiner has erred, as well as the Examiner’s response to Appellant’s arguments in the Appeal Brief. As discussed below, we are persuaded by Appellant’s contentions of Examiner error. Issue on Appeal Appellant’s arguments in the Appeal Brief present the following dispositive issue: Whether the Examiner erred in finding the combination of Sallam and Banginwar teaches or suggests the following limitations recited in independent claims 1, 9, and 20: (i) “sending, by the guest, a request that causes the hypervisor to set the set of physical memory pages to an executable-by-user mode in the hypervisor’s page tables,” (ii) “wherein each memory page that is set to the executable-by-user mode in the hypervisor’s page tables is executable in user space and is not executable in kernel space.” Appeal Br. 7-18 (emphases added). Analysis For the limitations at issue, the Examiner relies on Sallam as disclosing limitation (i) and on Banginwar as disclosing limitation (ii). Final Act. 3-5 (citing Sallam ¶¶ 76, 244, 248-251; Banginwar ¶¶ 35, 39, 69-70, 78-79, Fig. 2). The Examiner finds the proposed combination would have been obvious because it “would enable a platform protection technology to include a virtual machine monitor to enable an untrusted application and a trusted application to run on top of an OS, while preventing the untrusted Appeal 2020-004383 Application 15/053,899 4 application from accessing memory used by the trusted application.” Final Act. 5 (citing Banginwar Abs.). Appellant contends the Examiner erred by mapping limitation (i) to the use of security rules and/or memory map for granting access to a memory page in Sallam because the reference “fails to teach or suggest, however, the security rules or restrictions to system memory as being enforced through the use of a hypervisor’s page.” Appeal Br. 8-9 (emphases omitted). Appellant specifically argues that Sallam includes no disclosure regarding the memory map that includes information about the location of various entities in the memory, as “being enforced through the use of a hypervisor’s page tables.” Id. at 9 (citing Sallam ¶ 248) (emphases omitted). Appellant also contends the Examiner erred by mapping limitation (ii) to the memory use by a trusted application (TA) in Banginwar because “an untrusted application running in the user mode may be prevented from accessing memory used by a trusted application, where the trusted application runs in the same user mode and is further installed by the untrusted application,” which shows “two applications both running is user space” are protected from each other “without regard to an application’s memory pages not being executable in the kernel space.” Appeal Br. 11-12; see also Banginwar ¶¶ 35, 39, Fig. 3. Similarly, with respect to paragraphs 78 and 79 of Banginwar, Appellant argues that “both of the untrusted application 130 and trusted application 120 run in the same user space,” which means that part of the allocated memory pages are not executable in user space if they have no execute permission. Appeal Br. 13-14.2 2 We do not address Appellant’s other arguments because this issue is dispositive of the appeal. Appeal 2020-004383 Application 15/053,899 5 In response, the Examiner reads the recited user mode and kernel mode to Sallam’s disclosure of guest system on virtual machine and below- operating system trapping agent. Ans. 3 (citing Sallam ¶ 32, Fig. 1). The Examiner further relies on paragraph 39 of Sallam as disclosing “handling a request wherein access is not being executed in the kernel mode which will protect the application and driver from the user mode.” Id. Referring to paragraphs 76, 77, and 95 of Sallam, the Examiner explains that “[t]he teachings of a hypervisor page table (i.e. EPT) . . . is the same as the definition shown in para. 0024 of the Applicant’s specification (i.e. a hypervisor page table may also be referred to as an extended page table (EPT),” wherein “the hypervisor virtualizes all aspects of the guest operating system and will intercept too many attempted access of system resources which may be prone to malware when the guest device is running in privileged mode.” Id. at 4. With respect to Banginwar, the Examiner maps the recited kernel space mode to “View0” or the untrusted view and the recited user space mode to “View1” or the trusted view. Id. at 5 (citing Banginwar ¶¶ 35, 39, 42, 63). According to the Examiner, “Banginwar et al. further discloses VMM protecting the memory that has been allocated and managed by a hypervisor wherein the guest OS memory pages are assigned permissions and allocated read, write, or both read and write access based on the guest OS page table mappings,” which results in denying the guest OS from executing on the memory if no read/write access is allowed. Ans. 5 (citing Banginwar ¶¶ 69-70, 78-79). Based on our review of the applied prior art, specifically the passages cited by the Examiner, we find that Sallam discloses a system for protecting an electronic device where a below-operating system (O/S) trapping agent Appeal 2020-004383 Application 15/053,899 6 traps the attempts to access the device resources and memory based on a set of security rules. See Sallam ¶¶ 6-8. Sallam describes the protection measures in terms of protection rings and the privileges assigned to devices operating at those rings or levels. See id. ¶¶ 32, 39. As argued by Appellant, Reply Brief 2-3, “Sallam fails to teach or suggest the below-OS security agent (or any component disclosed in Sallam) setting a set of physical memory pages to any mode (e.g., executable-by-user mode in an extended page table) based on the attempted access of the resource” because “consulting security rules 1208 is different from setting a set of physical memory pages to an ‘. . . executable-by-user mode in the hypervisor’s page tables.’” We also agree with Appellant’s assertion that the Examiner has not pointed to any teachings in Sallam that show any relationship between the disclosed security rules and the hypervisor’s page tables. See Reply Br. 3. Sallam’s disclosure of a hypervisor in paragraphs 95 and 180 relate to virtualization of access to system resources, but includes no teachings with respect to setting page tables that are executable only in user space or the operation of the below-O/S trapping agent. In similar manner, Appellant’s contentions are persuasive of Examiner error in mapping the recited “each memory page that is set to the executable-by-user mode in the hypervisor’s page tables is executable in user space and is not executable in kernel space” to the trusted and untrusted application running in user mode in Banginwar. With respect to the Examiner’s characterization of View0 as the kernel space and View1 in Banginwar as the recited user space mode, Appellant correctly asserts that Figure 3 of Banginwar shows different views for different access types and further asserts: Appeal 2020-004383 Application 15/053,899 7 In Banginwar’s FIG. 3, the “View 1” represents the TA 120’s view, in which the host physical page A is executable in user space, and the “View 0” represents the untrusted application 130’s view, in which the host physical page A is not executable in user space. See Banginwar at para. [0063]. Here, Banginwar’s host physical pages associated with the trusted application's view are executable in user space, while these same pages associated with the untrusted application’s view are not executable in user space. Appeal Br. 12. The Examiner’s explanation regarding the kernel space and the user space (Ans. 5) is not supported by evidence. As stated by Appellant, the Examiner’s mapping of setting “the set of physical memory pages to an executable-by-user mode in the hypervisor’s page tables” as the security rules protection in Sallam and how each memory page set to be executable in user space is not executable in kernel space to the trusted and untrusted driver runs in Banginwar is not reasonable. See Reply Br. 3-4. That is, even if Sallam is modified with Banginwar to set the memory pages to be executable by trusted driver, but not by untrusted ones, the Examiner has not explained how the modification would result in teaching or suggesting the disputed limitations (i) and (ii), as discussed above. Conclusion For the above reasons and on the record before us, we agree with Appellant that the Examiner’s proposed combination does not teach or suggest the recited features of claim 1. Therefore, Appellant’s arguments have persuaded us of error in the Examiner’s position with respect to the rejections of independent claim 1, other independent claims which recite similar limitations (see claims 9 and 20), as well as the remaining claims dependent therefrom. See Supplemental Appeal Br. 3-6 (Claims App.). Appeal 2020-004383 Application 15/053,899 8 DECISION SUMMARY In summary: Claim(s) Rejected 35 U.S.C. § Reference(s)/Basis Affirmed Reversed 1, 2, 4-10, 12-22 103 Sallam, Banginwar 1, 2, 4-10, 12-22 REVERSED