Home
Welcome to the security-compliance-score wiki!
I. INTRODUCTION The benefits of Cloud Computing have been spurring a new norm in today’s business operation solutions among various industries. Clients buy services from Cloud Service Providers (CSP) such as Amazon Web Services (AWS), Microsoft Azure, Google Cloud, etc., without implementing or maintaining high-cost physical infrastructures. Under the profound impact of cloud services, security audit is inevitable. However, traditional security auditing is undertaken manually, involving interviewing employees, performing vulnerability scans, and evaluating policies regarding authentication, authorization, accounting, etc., which is heavily time and resources consuming.
For businesses running on the cloud, periodical and manual security audits and risk assessments become more inefficient in the fast-changing cloud environment. In the cloud, a simple misconfiguration, like an open API, may lead to a data breach attack, which is a true story happening: Optus, an Australian telecommunications company, is facing a $1 million ransom from a cybercriminal calming access to over 11 million records from customers including personal information, like passports. This is just because of the misconfiguration of valid authentication policies. [3] It is often found that companies would maintain a healthy state during the evaluation period of SOC 2 examination, ISO 27001 certification, or other security compliance audits, but security states become worse after that because there is no continuous audit in place. How to mitigate it?
The answer will be automation, which provides a feasible solution for continuous auditing. Using the proper automated audit tools, security teams can conduct a constant audit with real-time monitoring to check security risks in the cloud environment and take action accordingly. This means users must quickly gather, analyze, and report security-related data for on-demand assessment. Unfortunately, most current tools like SolarWinds and Nessus are well designed for network security auditing under the on-premises landscape. The cloud-based automated security auditing tools are not prevalent or mature in this marketplace, not to mention an easy-to-use application, for that matter.
Microsoft 365 Defender measures compliance posture and provides Microsoft Secure Score based on the organization’s configurations and policies, including cloud and on-premises environment. Thousands of security rules are measured, and the result Dashboard clearly demonstrates each item with a security concern comprehensively. However, this secure score will not update until 48 hours after any modifications or security actions are made, which means the security evaluation does not necessarily represent the current situation. This feature is great but has not had much popularity among cloud users.
In this project, we will use Microsoft’s existing security audit capabilities and tools to design the new automated security audit application, which can be deployed on Microsoft 365 Marketplace and used among different tenants. This application will output assessment results as Pass or Fail against security criteria from established frameworks, such as Payment Card Industry Data Security Standard (PCI DSS), NIST, and other industry compliances. According to the failure information from the output, users can take actions in areas like User Identity Control, Device Management, etc., to improve security configurations and increase security scores.
II. METHODOLOGY AND RESOURCE The project uses Microsoft cloud Azure and Microsoft 365 as the main platform. First, we build a virtual organization on Azure and manage security configurations using applications under Microsoft 365 umbrella. Then, we utilize Microsoft Graph to fetch organizational security information set up by various applications. Finally, we create our own application to analyze the data we fetched from Microsoft Graph API and output security results according to different security rules.
The methodology being used is straightforward. We authenticate against Graph API using available libraries in a C# code project. After establishing the data access channel, we pull various data endpoints based on our research, analysis, and general documentation. Once we get hold of data containing the necessary information for assessing security, using code-based transformations, we generate a report based on individual security cases providing a comprehensive report to the platform users. Thus, this project is predominantly working on the following resources and application platforms:
• Azure Portal, Azure Active Directory (AD)
• Microsoft 365: Office 365, Purview, Defender, Intune, 365 Admin Center
• Microsoft Graph
• C# Code
III. BUILD UP DATA AND SECURITY CASES The purpose is to build up a virtual product organization from scratch to control it fully. This includes the following steps: • Create tenant and organization • Create users (employees)
• Create and assign Groups for users based on their job attributes
The purpose is to ensure employees have the necessary authority to perform their daily job while following security policies in place, such as Multi-factor Authentication (MFA), Block Legacy authentication protocols, Enable Self-service password reset, etc. Real users prefer Security Default in Azure AD, with this policy enabled by default, to avoid inadvertent misconfiguration.
The project relies on the fact that Microsoft Cloud Services consume Graph API infrastructure in the background. Microsoft Graph functions as the gateway to data and intelligence in Microsoft 365. The restful Graph API is the source of abundant information that flows through the cloud network to provide all its services, including Azure Active Directory, Microsoft Purview, Microsoft Intune, and Microsoft 365 Defender, to name a few. We rely on consuming this data securely and diligently to perform security analysis.
In this project, we went through massive documentation on the topics of Microsoft 365, Azure AD and Microsoft Graph, trying to figure out security checks (cases) regarding policies, configurations, and settings from Microsoft 365 applications. The data we fetch from the cloud includes users’ information, device information, conditional policies saved on the cloud, etc. We categorized our security cases into the following three areas: • Identity Security
• Device Security
• Data Security
For each security case, we nailed down the specific Graph API to find possible security data according to security cases. We use Graph API “Get all users in the organization” (https://graph.microsoft.com/beta/users) to find out basic employee information and security information, such as if the multifactor service is enabled or not. We use Graph API “Get all Conditional Access policies” https://graph.microsoft.com/v1.0/identity/conditionalAccess/policies) to query all conditional policies set in place, such as the block legacy authentication policy. We use Graph API “Get secure scores” (https://graph.microsoft.com/beta/security/secureScores?$top=1) to get a non-real-time update comprehensive security output, such as the user risk policy. By extracting data from Microsoft Graph, we constructed security rules as security testing cases for this project.
IV. APPLICATION CONFIGURATION First, we register a new application in the Azure portal that is needed to grant the authorization to read data from Graph API. To do this, we follow the tutorials from https://learn.microsoft.com/en-us/graph/tutorials/dotnet?tabs=aad&tutorial-step=1.
It is essential to mention that most of our research involved using the tool Microsoft Graph Explorer. We basically communicate with Graph API, from which we extract relevant data. We need Microsoft Graph to grant our “Security and Compliance Score” app Read permissions. To achieve this, we can go to API permissions in the Azure portal and request the permissions needed.
There are several ways to use the new project application, which are supported by the Microsoft Authentication Library (MSAL). The documentation (https://learn.microsoft.com/en-us/azure/active-directory/develop/msal-authentication-flows) clearly explains the authentication scenarios, and for this project, we are looking at using the Client credentials method, which is often referred to as daemons or service accounts, compared to user accounts that are needed to authenticate manually. To achieve the Client credentials authentication, we can either configure it using certificates (more secure) or Application secrets. This project used the Application secrets method.
In general, our purpose is to allow authorized users to use this application to examine their organization’s security status. There are two things to be done: • Create and configure Client Secret • Allow application for multi-tenants The tenant admin will use to grant application permissions, so users don’t have to be required to get permissions manually when using this app. Note that Client secret values cannot be viewed except immediately after creation. Be sure to save it before leaving the creation page.
Now, we have completed all basic configurations for our new application, “Security and Compliance Score”. In the next section, we will explain how to implement the C# code to analyze the data fetched from Graph API and output Pass/Fail results with necessary suggestions to improve the organization's security condition.
V. CODE IMPLEMENTATION AND OUTPUT Ideally, developing a user-friendly UI will be more attractive; for now, we only implemented as command line interface (CLI) to show the functional prototype. We use C# Code for all the programming work.
In this project, we have implemented ten security cases from the best practices on Microsoft 365 tenant security, including • MFA enabled • Block legacy protocol authentication • Enable Risky Users/Risky Sign In policies • Tenant Audit enabled • Enable Self Links • Enable Self Attachments • Enable Self-service Password Reset • Implement DMARC/DKIM protection for domain names • Do not allow users to grant consent to unmanaged apps • Regular review of security logs and reports (we should be able to see the last access date on reports) [8] The code comprises three interfaces/classes: ITestCase, TestHelper, and Command.
- ITestCase: extend all security cases
- TestHelper: handle application authentication
- Command: console runner and consumer of the security cases
According to different security cases, specific Graph API(s) will be called using the Microsoft Graph SDKs. To make it work, we follow the instructions from https://learn.microsoft.com/en-us/graph/sdks/create-requests?source=recommendations&tabs=CS.
This project application runs on CLI, which can be launched from Windows Command Prompt or PowerShell. We use .NET to run the web app implemented by C# code. Before running the application, it is necessary to add the package “UserSecrets” to use Client Secrete authentication.
Run the application by “dotnet run .NET Graph Tutorial”. There is no need for the user to do any authentication to run the app because the “secret” has already been configured for the web application. To check the result of any security cases, users must choose the number of specific security cases according to the instruction.
Note that in many cases, Microsoft Graph may not capture the settings or values directly but instead verifies the availability of relevant policy. For example, if we check the bit locker data from Graph API, it will not simply give you if a specific end-point device has bit locker enabled or not. Microsoft 365 ecosystem relies heavily on the availability of policies and status reports against policy enforcement. This is a fundamental concept in dealing with Users, Roles, Groups and Devices. This helps us categorize security test cases mainly into two aspects: • when a policy is missing • when a policy is not enforced The application can safely report a “FAIL” result if the relevant policy is missing, which should be an alert that is well worthy of attenuation from the organization’s security team.
VI. CONCLUSION During this project, we covered most areas that can be used to build a security score for production. Even though the project does not provide full coverage, we validated the implementation and identified measures to help a team continue working on this task. We explored massive APIs in Microsoft Graph and also provided code examples for accessing and analyzing related data for security case transformations.
In our findings, Microsoft 365 APIs can be safely categorized on three mechanisms: state-driven, policy-driven and log (report)-driven. Leveraging security cases on each STRIDE category involves a deep understanding of the available Graph APIs and the mechanism with which the APIs are structured.
Organizations can keep an eye on and improve the security of their Microsoft 365 identities, apps, and devices according to the security audit feedback from the project application. Secure Score assists businesses by reporting on the organization's security posture as it stands today. You can strengthen their security posture by giving them discoverability, visibility, direction, and control. Create key performance indicators and compare them against benchmarks (KPIs).
For future tasks, here are some points we suggest that could be further worked on. First, in our project, to evaluate our devices and user identities, we manually create the policies and configure settings from various Microsoft 365 application platforms. But a future extension to this project can be a scenario where the policies are scripted and automatically created and deployed using our API. This will be more efficient and better to review when running the automated security audit application, and also guarantee that all devices are evaluated against the suitable compliance standard and not the one that the organization has created. Because anyone in the organization can modify the policy, but if it has been automated to install with our API it will be tamperproof. Second, we only create CLI for our project purpose, but one can develop web API or web App in the future. More, we only constructed 42 security checks in this project, but there could be much more to dig in. Microsoft has provided us meaningful security output scattered in various application services. Organizations will benefit from having a single consolidated application covering all such security checks, providing a centralized comprehensive report, and automatically updating or deploying proper configurations and policies.