Office 365 Digital Experience Monitoring (DEM) should be critical to every enterprise. Why? It’s pretty clear that Microsoft Office 365 is winning it all when it comes to collaboration suites with more innovation, polish and features than other platforms.

Office 365 Growth

Office 365 and its corresponding cloud services, Azure, Office Apps and everything in between is growing fast. Some of the latest figures for Office 365 and Azure growth:

• 200 Million Active Daily Users
  Wow, that’s a staggering number! On Microsoft’s’ FY20 Q1 earnings call on October 23 (link to call transcript), they updated the number from the previous 180 million mark reported in April 2019 to 200 million. That’s a number that makes every SaaS vendor jealous.
• Office 365 revenue growth at 36% YoY is weaker than Azure (59%), but there’s no doubt that Office 365 remains the big income stream for Microsoft cloud products.
• Microsoft reports $36.9 billion in Q2 2020 revenue: Azure up 62%, Surface up 6%, and LinkedIn up 24%

These are just a few of the examples of awesome growth. In short, Microsoft is crushing it with Office 365. And, in turn, most enterprise knowledge workers are sitting in front of Office Apps and services all day long. Therefore the importance of DEM solutions for monitoring Office 365 keeps rising (good for us!).

What’s It Mean to Monitor the Digital Experience of Office 365 Apps?

Its pretty simple — employees and knowledge workers are using Office 365 apps and services, so you want to monitor the performance and detect issues and outages before your users do. Your business and different departments within IT demand to be informed proactively instead of re-actively.

Often, the only way to go about monitoring Office 365 is via synthetic transaction monitoring (STM). Though there are some Real User Monitoring (RUM) solutions like Exoprise Service Watch that can passively monitor SaaS apps. For the following set of requirements, we’ll focus mostly on the synthetic monitoring aspect because its where we believe you should start.

Digital Experience for Exchange Online

Exchange Online is said to be the biggest workload for Office 365 when it comes to consumption and resources. And, while everyone loves to sound the death knell of Email, we all still continue to collaborate (at least in business) via Email. As such, organizations of all sizes need to monitor the uptime, availability and performance of Email.

When it comes to Office 365, Email monitoring means:

• Exchange Online, the core of the service
• Outbound SMTP filters if you use them (you probably do)
• Inbound SPAM/Junk/Anti-Virus providers
• Phishing Providers if they are separate from your SPAM/Junk blockers

For proactive monitoring of Exchange Online in Office 365, the following are required elements that should be checked:

• Mailbox performance, uptime and availability
  • Sign-on, Login, Folders, Inbox access
  • Client Access Server (CAS or CASArray) (when it comes to the Office 365 Suite)
  • Mailbox Server
• End-to-end network view, Round Trip Times (RTT), and latency from the end-user’s perspective. Accessing real-world protocols from the same sites and locations that users are is critical to DEM solutions. Monitoring uptime and availability from a cloud network is pointless and irrelevant.
• Microsoft Exchange Online Protect (EOP) performance, inbound and outbound mail queues. How long does it take to get an email out, in, and through the internet to a destination?
• Internet mail flow, transport timings and the SPAM performance. You want to make sure that your inbound and outbound mail delivery is efficient and timely.
• Email delivery reliability and overall availability
• Archive Mail access and performance
• Mail search performance
• Different access methods and protocols for Exchange Online and there’s lots of them including; EWS, ActiveSync, OWA, MAPI, Graph and more.
• Calendar access, availability and performance
• FreeBusy shared and hybrid lookup performance
• Appointment creation, updating and lookup performance

These are the critical elements to monitor for a proper Exchange Online SLA. DEM solutions like Exoprise CloudReady provide for testing all of these elements and more. Read up further on Exchange Online or download the Exchange Online datasheet.

Digital Experience Monitoring for SharePoint Online and OneDrive

People love to bash SharePoint and while it can become unwieldy, it remains incredibly popular and flexible. The latest version of SharePoint Online (SPO) dubbed “Modern” does more client-side interaction which makes it easier and more pleasant to work with. If you have users and teams dependent on SPO / OneDrive then you should be monitoring the SharePoint Digital Experience.

• SharePoint Site availability and uptime for Service Level Agreement management
• Permissions, Login and Sign-In performance
• SharePoint interaction with Single Sign-On vendors and tools such as Azure AD, ADFS, Okta
• SharePoint Drive and asset availability, creation and access
• Hop-by-hop network path performance for SharePoint and OneDrive from end-user locations, branch offices and varying network conditions that users experience. For a proper DEM solution, you must monitor from real user locations.
• Search performance. SharePoint sites for busy teams often require lots of search horsepower so everyone can find what their looking for (obviously)
• Upload and Download file performance, SharePoint bandwidth and ability to synchronize files.
• SPO Health scores, Internet Information Services (IIS) Latency, and SharePoint Request Duration. These are critical values for monitoring the health of your SharePoint tenant.

Bench-marking SharePoint metrics is critical for comparison purposes. It helps make sense of the metrics. By bench-marking your SharePoint site performance, you can identify poorly performing SharePoint tenancies and contact your Microsoft representative if site health is poor compared to the crowd. Read more about Exoprise SharePoint Monitoring features and Exoprise Crowd-Powered benefits.

Digital Experience for Microsoft Teams

Finally, we’ll finish up with the requirements for Microsoft Teams Digital Experience Monitoring. Teams is a large and growing platform for Office 365. Its quickly becoming the new foundation for how businesses’ collaborate. With the forced replacement of Skype for Business Online next summer (June 2021), MS Teams will become even more business critical and requires monitoring.

Monitoring the digital experience of Microsoft Teams should at least cover the following:

• Teams Site availability and uptime for Service Level Agreement (SLA) management
• Permissions, Login and Sign-In performance
• Presence and availability, latency
• Teams creation, access, messaging and notification performance
• Microsoft Teams Audio Video Application Performance
  • End-to-end network performance assurance and tests
  • Audio, Video Jitter
  • Video and Audio Frame, Bitrates
  • Packet Loss
  • Round Trip Delay Time (RTT) and latency as seen by the WebRTC layer that MS Teams utilizes
  • STUN, TURN and other WebRTC protocol performance
  • Monitoring VoIP/Video performance requires monitoring from real-user networks and locations. Monitoring VoIP availability from the cloud is useless for determining digital experience.
• Proxy Connect Times if leveraging local or cloud-based proxy systems (like Zscaler)
• TCPIP Connect
• DNS Lookup performance
• SSL Negotiate (good indicator of general network congestion)
• Page Layout, Render and DOM Loaded times
• Time to First Byte (TTFB) across all of the MS Teams connections

Capturing and comparing this performance data across your environment enables different IT teams monitor the digital experience of MS Teams proactively.

Conclusion

These are the basic requirements for a full featured Office 365 Digital Experience Monitoring solution. Along with just covering these items, a proper solution should be easy to install and easy to integrate with. Integration usually entails existing on-premise or cloud-based monitoring and ITSM solutions like Enterprise Splunk, SCOM, or ServiceNow. 

And along with monitoring the core Office apps, a proper solution should absolutely have support for additional dependent services like:

• Yammer and Skype for Business. There still here and beloved by some.
• Azure AD, ADFS and a whole compliment of critical authorization services. Ideally, covering these services should watch their interactions with the downstream services to ensure performance.
• VoIP, DNS, and other latency tests
• Dynamics, Salesforce and other CRM solutions
• Generic Web Login and Web Monitoring solution for any application.