Compare and contrast the level of effort required in hardening password policy with and without incorporating automation Consider automation development, testing, and ongoing maintenance in th

 

Automated Implementation of Windows-related Security-Configuration Guides

Patrick Stöckle [email protected]

Technical University of Munich

Munich, Germany

Bernd Grobauer [email protected]

Siemens AG

Munich, Germany

Alexander Pretschner [email protected]

Technical University of Munich

Munich, Germany

ABSTRACT

Hardening is the process of configuring IT systems to ensure the

security of the systems’ components and data they process or store.

The complexity of contemporary IT infrastructures, however, ren-

ders manual security hardening and maintenance a daunting task.

In many organizations, security-configuration guides expressed

in the SCAP (Security Content Automation Protocol) are used as

a basis for hardening, but these guides by themselves provide no

means for automatically implementing the required configurations.

In this paper, we propose an approach to automatically extract

the relevant information from publicly available security-config-

uration guides for Windows operating systems using natural lan-

guage processing. In a second step, the extracted information is

verified using the information of available settings stored in the

Windows Administrative Template files, in which the majority of

Windows configuration settings is defined.

We show that our implementation of this approach can extract

and implement 83% of the rules without any manual effort and 96%

with minimal manual effort. Furthermore, we conduct a study with

12 state-of-the-art guides consisting of 2014 rules with automatic

checks and show that our tooling can implement at least 97% of

them correctly. We have thus significantly reduced the effort of

securing systems based on existing security-configuration guides.

CCS CONCEPTS

• Security and privacy→ Software security engineering; Us-

ability in security and privacy; • Software and its engineer-

ing→ Software configurationmanagement and version con-

trol systems.

ACM Reference Format:

Patrick Stöckle, Bernd Grobauer, and Alexander Pretschner. 2020. Auto-

mated Implementation of Windows-related Security-Configuration Guides.

In 35th IEEE/ACM International Conference on Automated Software Engineer-

ing (ASE ’20), September 21–25, 2020, Virtual Event, Australia. ACM, New

York, NY, USA, 13 pages. https://doi.org/10.1145/3324884.3416540

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

ASE ’20, September 21–25, 2020, Virtual Event, Australia

© 2020 Association for Computing Machinery. ACM ISBN 978-1-4503-6768-4/20/09. . . $15.00 https://doi.org/10.1145/3324884.3416540

1 INTRODUCTION

Misconfigurations reduce the security of a system by introducing

vulnerabilities that are often difficult to trace. A recent study [6] has

demonstrated that from the perspective of the operators there is

one major factor for security misconfigurations: lack of knowledge.

One attempt to deal with the lack of knowledge is to use exist-

ing security-configuration guides. These guides consist of several

rules for a specific software system such as Windows 10 or Red

Hat Enterprise Linux. Each rule explains which setting should be

set to which value to make the system more secure and why we

should apply it (e.g., Listing 1). Known publishers of such guides are

the Center for Internet Security (CIS) or the Defense Information

Systems Agency (DISA). Organizations and companies like Siemens

can use these guides to harden their systems.

One may be tempted to argue that we do not need security con-

figuration because companies like Microsoft make a strong effort

to configure their systems securely by default. These companies

invest a lot in security, of course, but security is just one concern,

in addition to others, including usability. Assume that there was

a handy setting for most customers, but it poses a small security

risk. The companies may be tempted to decide to have it activated

by default, whereas security-aware customers would deactivate

it. Similarly, we could argue for the data collection settings. They

bring knowledge to the companies to improve their products, and

all customers can profit from this. Thus, the companies may be

tempted to activate data collection settings by default. In contrast,

customers with high-security requirements would deactivate them

to reduce the risk that sensitive information is accidentally leaked

via the data collection. Thus, security-configuration guides from

independent organizations can help security-concerned customers

in making their systems more secure.

## /rule The number of allowed bad logon attempts must be configured to three

or less.↪→

## /description The account lockout feature, when enabled, prevents brute-force

password attacks on the system. The higher this value is, the less effective the account lockout feature will be in protecting the local system. The number of bad logon attempts must be reasonably small to minimize the possibility of a successful password attack while allowing for honest errors made during normal user logon.

↪→

↪→

↪→

↪→

↪→

↪→

## /implementations/0/description Configure the policy value for Computer Configuration >> Windows

Settings >> Security Settings >> Account Policies >> Account Lockout Policy >> “Account lockout threshold” to “3” or fewer invalid logon attempts (excluding “0”, which is unacceptable).

↪→

↪→

↪→

Listing 1: Example of a rule in a Windows-related security-

configuration guide.

598

2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE)

Authorized licensed use limited to: University of the Cumberlands. Downloaded on November 22,2022 at 14:39:55 UTC from IEEE Xplore. Restrictions apply.

 

 

The publishers publish their recommendations on how to con-

figure a software system in formats like PDF and in the Extensible

Configuration Checklist Description Format (XCCDF), which is

part of the Security Content Automation Protocol (SCAP). In some

cases, these implementations are combined with machine-readable

and automatable checks. These checks are created manually accord-

ing to the specification written down in the security-configuration

guides. Although XCCDF is designed as a machine-readable for-

mat, instructions for implementing the security settings are only

contained in human-readable form in almost all cases. The notable

exception is the OpenSCAP project’s [16, 22] guides for Linux oper-

ating systems and applications, which for many rules contain shell

scripts and parts of Ansible playbooks. Therefore, existing guides

solve the lack-of-knowledge problem, but yield another problem:

Automatic implementations (or remediation) are not specified in the

SCAP standard. In contrast, the specification of automated checking

is very detailed.

Publishers sometimes deal with this problem by providing ad-

ditional artifacts, such as scripts or – in the case of Windows –

configuration backup files. The problem here is threefold. Firstly,

such artifacts do not exist for all guides. Secondly, the guides fre-

quently get updated: If we take Windows 10 as an example, there

will be at least one new guide every year published to deal with the

updated settings, e.g., introduced by the version 1909 update; minor

version updates deal with problems or changed requirements. As

a result, DISA, for example, is now at version 18 for its Windows

10 guide. Therefore, creating/maintaining a mechanism (even if it

can be based on some artifact provided by the publisher or) will

be a recurring, manual task. Thirdly, with stand-alone artifacts for

implementation, customization of guides, a feature which is central

to SCAP, becomes cumbersome and error-prone, because this re-

quires a manual effort to keep the customized guide in sync with the

separately-maintained implementation mechanism. However, easy

customization is essential: Experience shows that there is virtually

no use case in which a publicly available security-configuration

guide can be implemented without at least some changes.

The authoring process is depicted in Figure 1. The publisher

creates the guide in the XCCDF format and the corresponding

checks in the Open Vulnerability and Assessment Language (OVAL)

format. This is a manual process, as the publishers incorporate their

knowledge about the system and its architecture into the guide. In

the next step, an administrator uses the automated checks to assess

the state of their systems. The result is a list of the rules to which

the system is not compliant; our evaluation in § 3 of over 2000 rules

on systems using the default configuration shows that the rate of

satisfied rules varies between 0% and 27%, with an average of 17.7%.

Thus, for most of the rules, the (typically: default) configuration of

the system to be hardened has to be adjusted.

If the publisher has not provided a mechanism for automated

implementation, for every rule of this list, the administrator must

read the implementation/remediation section of the rule in the XC-

CDF or PDF form of the guide and implement the steps described

there. If a mechanism is provided, in most cases only a complete

implementation of all configuration settings is possible. This creates

significant manual effort for customization, especially if the imple-

mentation breaks functionality, but it is unclear which setting(s)

have caused the observed problems.

In sum, we address one main problem: There are existing guides

to configure systems securely, but we cannot implement the re-

quired configuration settings (taking into account necessary cus-

tomization and changes due to updates of the guides) without sig-

nificant manual effort.

Our solution to this problem, realized for Windows operating

systems and applications, consists of three major steps. First, we

process the files which define the Windows security policy settings

that exist on a Windows-based system. Windows security policy

settings are rules that administrators configure on a computer or

multiple devices for the purpose of protecting resources on a device

or network. [13] We can configure a policy setting with a policy

path and a value. The so-called Administrative Template (ADMX/L)

files define themajority of policy settings. They contain information

about valid policy paths, possible values for each policy setting,

and the underlying implementation of a policy setting within the

Windows registry. Thus, we extract this knowledge in the first step

and store it in a machine-readable format to access it during the

remediation. Second, we use natural language processing to extract

the settings and the intended values from the guides. We use the

information of the first step to verify that the extracted setting

exists and that the extracted value is a valid input for this setting

and can, therefore, reduce the risk of wrongly extracted values to

a minimum. Third, we translate the settings and values to their

real implementation using the information from the first step. Our

tools can use this information to implement as well as check the

configuration settings automatically.

Our contributions are:

• an approach to how existing Windows-related security-

configuration guides can be automatically implemented;

• a proof-of-concept implementation of our approach;

• a step-by-step documentation of our approach using the

DISA Windows Server 2016 guide [26] and an updated ver-

sion using the DISA Windows Server 2019 [28];

• an evaluation of our approach using existing guides from

DISA and CIS with over 2000 rules [27].

In §2, we explain the general idea of our automatic implementa-

tion, and in the subchapters, we present the technical details of our

proof-of-concept implementation. In §3, we use the DISAWindows

Server 2016 guide and 12 CIS guides to demonstrate the feasibility

of our approach. In §4, we discuss challenges and first experiences

in generalizing our approach to non-Windows systems as well as

additional future work. §5 treats related work and §6 concludes.

2 WINDOWS-RELATED SECURITY CONFIGURATION

Generic Approach. The generic approach is depicted in Figure 2.

It shows the different stages of the envisioned process for auto-

matically implementing Windows-related security-configuration

guides. More specifically, the separate steps are defined as follows.

Extraction: Use natural language processing (NLP) for each rule to

automatically extract the information needed to implement

this rule.

Verification: Check with an automated mechanism that checks

whether the derived information is valid:

– Does the extracted policy path indeed exist?

599

Authorized licensed use limited to: University of the Cumberlands. Downloaded on November 22,2022 at 14:39:55 UTC from IEEE Xplore. Restrictions apply.

 

 

Automatic Checks

Implement rulesSecurity- configuraiton

guide

Administrator

Publisher

creates

Systems under Test

Automatic

Manually

check rules

Legend

Figure 1: Current State of Implementation of Windows-

related Security-Configuration Guides

– Has the extracted value the required type for that setting?

– Does the extracted value meet the requirements of that

setting? Is it in the list of possible values or in the range

of allowed values?

If the path or the value is incorrect, the mechanism provides

useful feedback about possible paths or values.

Transformation to low-level: Transform Windows policy set-

tings into a representation of one of the underlying low-level

implementation mechanisms. This step is necessary because

almost none of the most popular configuration-management

frameworks can directly process the Windows policy set-

tings, but require the specification of an underlying imple-

mentation mechanism:

– Registry settings

– Secedit policy file entries

– Audit file entries.

Transformation to code: Transform these low-level implemen-

tation mechanisms into code for carrying out the implemen-

tation of each setting.

Implementation: Execute code on the system we want to harden

to implement the rules.

We emphasize that especially steps one and two are novel because

– to our best knowledge – there is no approach published that uses

NLP to extract policy settings from SCAP guides, nor is there an

approach that verifies extracted values using definition files. For the

evaluation of our approach, we assumed that an evaluation of the

complete systems provides more evidence for the usefulness and

feasibility of the presented approach than an evaluation of the first

two steps alone. Consequently, we had to design and implement

the remaining steps for our PoC implementation. In the end, we

achieved the first published system that reads Windows-related

security-configuration guides in the SCAP format and implements

them automatically.

The approach in detail. We discuss the details of our approach

and demonstrate its feasibility using a proof-of-concept (PoC) im-

plementation.

1 system: org.scapolite.implementation.win_gpo 2 ui_path: <String containing a valid Windows policy path, using 3 backslashes as separators> 4 value: <A YAML representation of a valid value for the 5 specified path> 6 verification_status: (Checked. | Unchecked.)

. Listing 2: Syntax of the Windows policy automation

The steps of our actual implementation, which we use as a proof

of concept, are depicted in Figure 3. We describe them shortly here

and more in detail in the rest of this section.

The input of our PoC consists of guides in the SCAP format.

In the first step, we extract the necessary data for every rule to

automate the implementation of this rule using natural language

processing. The result is a set of rules enriched with the configu-

ration settings in a machine-readable format. These configuration

settings are then passed to the verification process: it has to be ver-

ified that the extracted data (a Windows policy path and required

policy values) is valid. Our implementation uses the information of

manually created verification rules for what essentially are legacy

configuration settings combined with information extracted from

the Windows administrative template files to verify the extracted

values. To make the verification process as fast as possible, we pro-

cess the latter files a priori and store the information we need in a

database format.

If the verification is successful, the low-level automation needed

to implement the rule is generated and also stored within the rule.

Depending on the chosen implementation mechanism, these are

used to create (1) either a group policy backup, which then can

be imported on a Domain Controller to secure all systems in an

Active Directory or (2) a JSON file used by a PowerShell script for

implementing the settings. Additionally, our tooling can check the

rules using the JSON files, but as SCAP already covers this aspect,

we will not look deeper into this facet of our PoC.

In our PoC implementation, only the second and third steps re-

quire a minimum of manual interaction; the other steps are entirely

automated. The dotted line between the Verification and the Config-

uration Settings in Figure 3 indicates that the person automating the

security-configuration guide may have to execute the verification

more than once and adjust the values until every rule is marked as

checked by the verification mechanism.

In the following, we describe each of these steps. Tooling has

been carried out in Python, except for a PowerShell framework

for implementing and checking Windows security configurations

using the output of Step 4. As a real-life example, we use the DISA

Windows Server 2016 Security Technical Implementation Guide [7].

We created a GitHub repository [26], where we conducted all the

steps, and created a commit and a tag after every step and reference

them by their tags. 1

1For representing the guide wihin Github, we use the YAML/Markdown-based Scapolite format developed within Siemens, which is better suited than SCAP for authoring and maintenance. The approach, though, is independent of the format.

600

Authorized licensed use limited to: University of the Cumberlands. Downloaded on November 22,2022 at 14:39:55 UTC from IEEE Xplore. Restrictions apply.

 

 

Guide 1. Extraction 2. Verification 3. Transformation

to low-level 4. Transformation

to code 5. Implemenation

on the system

PhasesInput Result

Figure 2: Overview of the abstract hardening approach.

2.1 Natural-language-processing-based extraction of Windows Policy Automations

The first step of our PoC implementation is the extraction of the

needed values using NLP. Before we can extract the information

needed to implement a Windows-related rule automatically, we

had to define the structure of the machine-readable constructs,

Step 1 Extract configurations/

values with NLP

Configuration Settings

creates uses

Step 2 & 3 Verification

Low-level Automations

Automatically Extracted knowledge

creates

Knowledge extraction

Manually extracted

knowledge Step 4 JSON Export

SCAP – XCCDF

inputADMX ADML

Legend

Input Manually Added

OutputFully Automatically

Semi Automatically

Step 4 GPO

Export

Step 5 Implement

Rule

JSON with all information to apply or check the rules

Step 5 Check Rule

On the system under test

Check Results

GPO Backups

Scapolite Objects

Auxiliary Output

Figure 3: Overview of the steps of our actual implementa-

tion.

INq0 .*q1 VBN, VBD, VBq3 INq4

.*

q5

.*

VBN, VBD q6IN q7NN, NNP q8. q9

TOq2

.*

NN, NNP

q10

Figure 4: Example of an extraction rule as a nondeterminis-

tic finite automaton.

1 id: SV-88407 2 rule: <see below> 3 implementations: 4 – description: <see below> 5 automations: 6 – system: org.scapolite.implementation.win_gpo 7 ui_path: ‘Computer ConfigurationPoliciesWindows

SettingsSecurity SettingsLocal PoliciesUser Rights AssignmentBack up files and directories’

↪→

↪→

8 value: 9 – Administrators 10 — 11

12 ## /rule 13 The Backup files and directories user right must only be assigned to

the Administrators group.↪→

14 ## /implementations/0/description 15 Configure the policy value for Computer Configuration >> Windows

Settings >> Security Settings >> Local Policies >> User Rights Assignment >> “Back up files and directories” to include only the following accounts or groups:

↪→

↪→

↪→

16 – Administrators

Listing 3: Example rule of the DISAWindows Server 2016 in

YAML/Markdown form, incl. a Windows policy automation

starting in line 6 (blue).

how they are integrated into the rule structure, and what has to be

extracted to implement a rule.

For specifying Windows policy settings, the structure must pro-

vide information about the policy path and the required value. The

type of the value (string, list, integer, et cetera) depends on the path;

hence the specification of the automation syntax must refer to the

set of valid Windows policy settings as shown in Listing 2. Listing 3

shows the usage of a policy automation in the rule SV-88407 of the

Windows Server 2016 guide. In an ideal scenario, the rule already

contains the machine-readable automation objects, but this is not

the case for guides published in the SCAP. Thus, we needed to

extract the information about required policy settings from the

human-readable description in the guide. To this end, we used the

Natural Language Toolkit (NLTK) [1]. Due to the highly schematic

structure of the guides under consideration, only eleven extraction

rules had to be defined to process most of the rules. One of the rules

is presented here in Listing 4 and Figure 4. The listing shows the

definition of such an extraction rule as part of a grammar in NLTK.

IN, TO, etc. refer to the corresponding part-of-speech (POS) tags.

As we have ten rules, our grammar to extract the values consists

of ten rule definitions. To make the idea more precise, Figure 4 is

SENTENCE_WITH_ENABLED_WITH_X_SELECTED_FOR_Y: {<IN> <.*>+ <TO> <VBN|VBD|VB> <IN> <.*>+ <VBN|VBD> <IN> <NN|NNP>+

<.>}↪→

Listing 4: Example of an extraction rule with POS tags.

601

Authorized licensed use limited to: University of the Cumberlands. Downloaded on November 22,2022 at 14:39:55 UTC from IEEE Xplore. Restrictions apply.

 

 

id: controlpaneldisplay__cpl_personalization_nolockscreencamera registry:

name: NoLockScreenCamera path: SoftwarePoliciesMicrosoftWindowsPersonalization hive: HKEY_LOCAL_MACHINE type: REG_DWORD enabled_value: 1 disabled_value: 0

Listing 5: Example of a relationship between the id and the

definition of the registry to set.

USEFUL NOTES FOR:

Automated Implementation of Windows-related Security-Configuration Guides

Introduction

Configuration guides are an important document type that is used in information security. These documents can be used to configure computers manually, but they also have the potential to be compromised by attackers running scripts or programs on remote computers. In this paper we propose a method for automatically implementing security-configuration guides based on machine learning techniques such as neural networks and reinforcement learning algorithms.

Configuration guides are an important document type that is used in information security

Configuration guides are an important document type that is used in information security. They can be used to set up systems and networks securely, keep them secure over time, or even help you troubleshoot problems with your system.

Configuration guides are a crucial tool for information security professionals because they provide detailed instructions on how to configure your computer system or network so that it remains safe from attack or intrusion by hackers or viruses.

Attackers can run scripts or programs on remote computers, and they can also tamper with a machine’s configuration.

Attackers can run scripts or programs on remote computers, and they can also tamper with a machine’s configuration. Configuration guides are used to prevent this kind of attack by providing information about how to configure a system and its security settings.

Configuration guides are useful because they provide an easy-to-read document type for information security personnel. They’re also useful for developers who want to make sure that their programs work properly on other machines in your organization (like servers).

Security guides consist of manually executed steps

Security guides consist of manually executed steps. They are important for information security, because they can be used to configure security-related settings on computers.

Security guides depend on the expertise of the reader to ensure that each step is performed correctly and in accordance with applicable policies and standards.

Security guides depend on the expertise of the reader

Security guides depend on the expertise of the reader. Users need to be trained to use security guides, and you cannot expect them to do so if they don’t have any idea what’s going on. They will likely have difficulty understanding how these guides work in practice and why it is important that they are followed correctly.

If a user tries to follow a guide without proper training or understanding, they may find themselves getting into trouble because the guide doesn’t always work as intended. For example, if your company has an internal network that only uses Windows XP machines (and no other operating systems), then there’s no reason why you wouldn’t want all those machines upgraded so that they can run newer versions of Windows—but this could create problems with compatibility between older programs like Word documents and newer ones like Outlook 2010 or later versions of Office applications such as Excel 2013 or PowerPoint 2016!

In this paper, we present our approach for automatically implementing security-configuration guides.

In this paper, we present our approach for automatically implementing security-configuration guides. The problems that arise from these guides are many:

Security configuration has to be performed by an administrator who is not always available during business hours (for example, at night).

Different software vendors have different security requirements, which can lead to conflicts in configuration files.

Some users might not understand what they need to do when configuring their system and may therefore fail to complete all steps of the guide properly or at all.

We address these issues by developing a prototype tool called [Chromith](https://chromith-project.org/) that automatically generates Windows-related security-configuration guides based on input from experts and users alike; it also allows them to make changes before generating new versions without any prior knowledge about how those changes would affect existing ones!

Conclusion

In this paper, we present our approach for automatically implementing security-configuration guides. The main idea is to use existing tools for parsing documents and extracting information from them. We also discovered that some toolkits have good support for the extraction of configuration directives and have proved useful in our implementation (e.g., perl, python). Finally, we’ve showed how to use these tools together with a set of rules which control how they interact with each other and make sure everything works smoothly when processing documents containing multiple directives.