Securing Azure AD against common device platform conditional access pitfalls

How secure is your conditional access implementation? This post examines the way device platforms are used within conditional access policies which can lead to unexpected sign-in results. This could mean anything from users being able to access services from a platform you've blocked to a compromised account bypassing your requirement for multi-factor authentication.

N.B. The post assumes an understanding of Conditional Access policies.

Throughout the course of this post I'll be running through a few examples to illustrate how this can affect you and also how to design your conditional access policies to mitigate any risk but to get started we need to know a little bit about the user agent string. For those unfamiliar this is a string of text that is generated automatically by an application/browser and forms part of the web request headers when you make a call to any website. For the purpose of this post we won't be deep diving into the user agent string but it's important to know that it includes two pieces of information: browser and operating system. This information is commonly used by websites to serve up content in an format that is appropriate for your device.

Here's an example user agent string:

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36 Edg/92.0.902.55

As you can see, there's more information but I've highlighted the platform and browser, so in my case Edge on Windows 10.

So why is this relevant as an M365 admin?

Well when it comes to Microsoft 365, user agent strings are parsed by conditional access policies which serve the critical purpose of evaluating if access to services should be granted and under what conditions. The problem however, is that fundamentally user agent string manipulation isn't hard to do. Web developers rely on this function to ensure their pages are served up correctly, it's commonplace enough that the ability to change it is built into all modern browsers.

Let's run through an example to illustrate how this can be exploited. My lab setup: one conditional access policy that blocks access from Windows, targeting all cloud apps.

First lets prove that it has the intended effect with a sign in from Windows:

As you can see, my user has been blocked. All good so far! We'll also check Azure AD just to confirm the policy has kicked in.

Ok now let's change our user agent string, I'm using Edge so the instructions may vary depending on your browser of choice, Google is your friend here:

1. Open developer tools (Ctrl + shift + i)

2. Click the three dots, then "More tools" -> "Network conditions"

3. In the network conditions window, scroll down to "User agent" and uncheck the "Use browser default" box.

You can now use the drop down to select from pre-defined user agent strings for a variety of different browsers/platforms, or we can choose to set our own. For this example, I'll select Safari on macOS.

Now let's try signing in to the Office portal again and voila, the test user can now sign in!

Let's check the Azure AD sign-in logs again, this time we can see that our policy was "not applied"

Checking the Device Info tab, we can see the platform of macOS and the Safari browser:

So what's the lesson here? Well be aware that a user can quickly and easily change their user agent string. The impact of this to an organisation varies depending on how your access policies are set up, it could be inconsequential if you utilise other checks such as a compliant device but if you don't, know that a user or malicious actor could work around your policies.

Now to dive deeper, Microsoft have got a lot clearer with their documentation on this but I still think it helps to expand on. Within the Conditional Access documentation are references to supported and unsupported platforms. Supported platforms are simply the platforms you can select within a policy - great we can manage those, however what we must do is address the unsupported platforms otherwise we risk our policies being easily rendered ineffective.

Let's head back to the lab for another example to illustrate this. The configuration is again pretty easy to set up; my users are allowed to access all apps via Windows and Android (where they have to MFA), but no other platforms.

In this case we've got two policies, each applies to all cloud apps and only the device platforms are configured. You can expand the screenshots in the gallery below to see the full configuration.

So we're good right? Users will hit either of the two policies depending on where they are coming from. Well yes, if we're playing by the rules, but we're not. Heading back to our browser again on Windows, we should be allowed but have to provide MFA. We'll input our own user agent string then test:

Mozilla/5.0 Gecko/20100101 Firefox/15.0.1

The result? I'm signed right in, no multi-factor authentication. The reason? Well that's because there is no requirement to send a platform in our user agent string, in this case we've just specified that we're using Firefox. Azure AD picks the sign in up, checks our empty platform against any conditional access policies, doesn't find a match then let's us sail right on through. Here's the sign in within the logs:

This works just as well if I'm on one of our blocked platforms such as macOS - same story, Azure AD sees no platform, therefore doesn't match any of our policies and we're in. So in one case we've bypassed the requirement for multi-factor authentication and in the other we've gained access from a platform that should have been blocked, hopefully you can start to see the impact this could have.

Always remember M365/Azure AD is open by default, it is designed this way to allow users to leverage cloud resources from anywhere, a great capability but this doesn't suit all use-cases especially within regulated industries such as the financial services. I highly recommend that you set up a blanket block policy that uses the "all platforms" option and specifically exclude the platforms you wish to allow access to. You can then leverage additional platform specific policies depending on your requirements, but you can rest easy knowing that there aren't any loopholes (or at least loopholes that aren't by your own making!).

Questions to consider when shaping your block policy:

1. What platforms do users need to access? Add them to your excluded device platforms

2. Should your block policy include or exclude guest users? - Guest users could be on any platform, can you/should you be enforcing what they can or can't use? Until recently Linux was not a selectable platform within Conditional Access, this meant that devices such as Chromebooks couldn't effectively be split out from "unsupported" platforms. Now there's pretty good coverage for all the platforms a legitimate user could come from so I'd personally err on the side of enforcing a control than not.

3. Do you have any users or service accounts that use unlisted or unpopulated platforms? A common example of this is email service accounts which often utilise legacy authentication methods. Step through their use-cases and ensure they are catered for. If you are unsure who/what may be accessing there are three options detailed below to help find this information. The first two require Azure AD to be integrated with a Log Analytics workspace (set up found here), the last method can be done without this and any additional configuration/cost. The detail on how to do this is below.

Sign in Analysis methods

Continuing from point #3 above this section explores the various sign in analysis methods to help shape your conditional access policies including for the aforementioned block policy. As with all of the methods below bear in mind that they will not provide a definitive answer that no legitimate user will be blocked. Users usage and behaviour can vary dramatically over time, examples of this could be processes that only run at the end of a month or even only at financial or calendar year end.

Conditional Access Report-Only Policies (Log Analytics required)

This is a topic in it's own right but a conditional access policy can be set up in "report only" mode. This is one of the options available when saving a policy.

When utilising report only, each sign-in will be evaluated against the policy conditions but instead of applying the resulting access control e.g. block or require MFA it will instead record the action that would have been taken had the policy been enforced.

We can use this feature to set up our block policy as designed but in report-only mode. Just like regular conditional access policies, the policy will only apply to future sign-ins so once configured allow some time for sign-ins to occur then check the "Insights and Reporting" tab within conditional access.

At minimum, set your "Conditional Access Policy" to filter on your report only policy, by default this will give you information for the last 24 hours.

Scrolling down you'll see a summary of the sign-ins that have been evaluated followed by some charts. We can click one of the top categories to drill down into the specific result. In my case selecting failures (where a user would have been blocked) you can see that out of the 25 users, 23 had no platform and 2 were on Linux:

Finally we get a detailed breakdown of the users sign-ins, that allows you to further identify heavy hitters and determine if you need to re-design or make exceptions for certain accounts/scenarios.

Report-only is a really great feature, having the ability to see the potential effects of a conditional access policy is priceless with the huge impact they can have on your users if they have unforeseen outcomes. My advice would be to use this capability to iterate and refine your policy. If changes are required, change the policy keeping it in report-only mode and re-evaluate the data generated to gain confidence it is working as intended.

As previously noted you will have to wait for sign-ins to occur in order to report. If you've had Azure AD Log Analytics integration on for a while, you can also use the next option to get some instant feedback.

Log Analytics query

Within Azure AD Log Analytics we can write custom queries to pull back data we are interested in. Unlike with newly created conditional access policies, the data here is historic meaning that we can instantly query what would have been blocked using your tenant's past sign in data. N.B. The amount of time you can search over will depend will depend on your data retention settings within Log Analytics.

As a starter, the following query will find all successful sign-ins within the last 30 days where the operating system recorded was blank.

SigninLogs
| where TimeGenerated > ago(30d)
| extend operatingSystem = tostring(DeviceDetail.operatingSystem)
| where ResultType == 0 and operatingSystem == ""

Additionally you can add the following additional line to select unique users that meet the criteria:

| distinct UserPrincipalName

The query can be expanded depending on the platforms you are planning on blocking. I will issue a warning that these queries can get fairly complex quickly and are a bit fiddly especially if you've not had much experience with the Kusto query language. Microsoft do provide an introductory guide with some of the basic concepts that is worth an initial read for familiarisation.

Azure AD Sign-In Logs

The last method is a bit clunky. Unfortunately while the built in filtering available on Azure AD sign-in logs is generally pretty good one thing we can't do is filter using conditions such as "not contains" or on blank values. In order to do this we need to export our sign in data.

1. Navigate to the Azure AD sign-in logs

2. Select a time filter. This is by default set to the last 24 hours but note that there is a 100,000 row limit to exporting data using the UI so you may need to filter appropriately. This can affect your ability to find certain sign-ins if they do not happen regularly so beware.

3. Select "Download" then choose "CSV"

4. Choose "Interactive Sign-ins" (the first option in the list) and download

5. Once you've got the CSV open it up in Excel and navigate to the "Operating System" column and filter on blanks or specific platforms you are interested in

Rounding up

Lastly there are some additional items worth considering while designing a set of baseline conditional access policies:

Legacy Authentication

Don't forget about legacy authentication, ensure any blocking of legacy authentication uses the "all platforms" selection. Basic authentication is not secure, can't enforce a second factor and is consequently a common attack vector - it should be disabled and at worst restricted as much as possible. Check your Azure AD sign-in logs filtering on client app and select all the legacy authentication clients, it is likely your tenant is being peppered with authentication attempts from malicious actors. Your mileage may vary but I've seen around 35,000 attempts a week and while a lot of these are blocked by Microsoft for originating from IP addresses associated with malicious activity don't leave it to chance.

Using Conditional Access to enforce multi-factor authentication

Do you enforce multi-factor authentication using conditional access but only for specific platforms? If so check your policy coverage and determine if you have any gaps that may be fixed by a blanket "all platforms" policy that enforces MFA. Remember that a "block" will always trump an "allow" so this can work in conjunction with an all platforms block policy.

Add additional layers of checks

Strongly consider using other checks when granting access. My top recommendation would be requiring a compliant device providing your devices are enrolled into Microsoft Endpoint manager. An alternative less robust option could include a check that the IP address presented at sign-in is either from a range that your organisation owns (preferred) or from your proxy infrastructure. the latter of which is not perfect especially with many organisations using shared proxy solutions but can add just enough to guard against some scenarios.

Fin!

And that's all for now, I hope this has been useful. I can't stress how important conditional access policies are but if you take care in their design they'll be a great asset in your security arsenal.