How to Monitor Microsoft Copilot: A Practical Guide for SME Operators

TL;DR: Practical observability for M365 Copilot and Azure OpenAI. Usage dashboards, cost alerts, Purview audit logs for European SMEs.

You have deployed Microsoft 365 Copilot or Azure OpenAI services. That matters because the deployment decision is not the hard part: the hard part is knowing what those tools are doing in production. Who is using Copilot daily? How much are you spending per department? Which interactions touched sensitive data? Without answers to these questions, you are running a significant monthly subscription without operational visibility.

This guide covers the built-in observability tools Microsoft provides, what they miss, and three specific monitoring routines suited to a small business or 20-person company running Copilot in 2026.

What You Actually Need to Monitor

Observability for Microsoft AI tools breaks into four practical categories.

Usage patterns. Who is submitting prompts, how often, and in which applications (Word, Teams, Outlook, SharePoint). Low adoption signals wasted seats. Uneven adoption signals training gaps.

Cost visibility. For M365 Copilot, the cost is per-seat per month (currently around €30 per user). For Azure OpenAI, cost is token-based and can spike unexpectedly. Both require separate tracking.

Adoption rates by department. A legal team using Copilot for contract drafting has different risk exposure than a sales team using it for email. Knowing where adoption is highest tells you where governance pressure is greatest.

Compliance logging. What data did Copilot access to generate a response? What was the output? This is not optional if you handle GDPR-regulated personal data in Microsoft 365.

Microsoft's Built-In Observability Tools

Microsoft provides four native tools. Each covers part of the picture.

Microsoft 365 Admin Center: Copilot Usage Reports

Navigate to Reports, then Microsoft 365 Copilot. You get per-user activity (days active, prompts submitted, apps used), tenant-wide adoption totals, and trend lines over 7, 30, or 90 days.

What works: the per-user breakdown is genuinely useful for identifying inactive seats. If 12 of your 20 licensed users have submitted fewer than 5 prompts in 30 days, that is an ROI problem worth addressing before the next renewal.

What the report does not show: prompt content, response quality, or whether any particular interaction produced a useful business outcome.

Azure Monitor: Token Consumption and Rate Limit Alerts

If you are running Azure OpenAI Service (for custom deployments or Copilot Studio backends), Azure Monitor is your cost and reliability dashboard. You can track token consumption by model, by deployment, and over time. Set metric alerts on token rate thresholds to catch runaway consumption before it hits your bill.

The setup takes about 30 minutes: create an alert rule targeting your Azure OpenAI resource, set the metric to "Total Token Transactions," and configure a threshold at 20% above your 30-day baseline. Route the alert to a Slack channel or email inbox an operations leader checks daily.

Microsoft Purview: Audit Logs for Copilot Interactions

Purview is the compliance layer. It logs what Copilot accessed and what it generated, tied to the user who made the request. You can export audit logs via the Purview compliance portal or the Microsoft 365 Management Activity API.

For any growing software team or professional services firm handling client data in SharePoint or Teams, Purview audit logs are not optional. GDPR requires you to account for how personal data was processed. Copilot interactions that touch personal data in SharePoint documents constitute processing. Your data retention policy determines how long these logs must be kept (typically 12 to 36 months depending on your sector).

If you have not enabled Purview audit logging for Copilot, do that before anything else on this list.

Copilot Studio: Session Analytics

If you have built custom agents in Copilot Studio, the built-in analytics panel shows session volume, conversation success rate, topic escalation rate, and handoff rate to human agents.

The conversation success rate metric deserves attention. It reflects whether the session ended with the user completing their intent without abandoning or escalating. A rate below 70% on a deployed agent is a signal to review conversation design, not just training data.

What Built-In Tools Miss

Microsoft's native tooling answers "who used Copilot and when." It does not answer "did the Copilot response lead to the right action?"

Quality signals require a different approach. The practical options for a small business without a dedicated AI evaluation team:

Manual spot-checks. Designate one person per department to review three to five Copilot outputs per week. Record whether the output was used as-is, edited significantly, or discarded. This is low-tech but it builds the pattern recognition you need.

User feedback mechanisms. In Copilot Studio, you can add a thumbs-up/thumbs-down prompt at the end of agent sessions. Even a rough satisfaction signal beats no signal at all.

External evaluation tools. For Azure OpenAI deployments, Azure AI Studio includes an evaluation harness where you can run your prompts against ground-truth outputs. This is more relevant for a technical team running custom models than for a standard M365 Copilot rollout.

Three Monitoring Routines for a 20-Person Company

These are weekly, monthly, and quarterly cadences that a founder-led company or operations lead can run without dedicated tooling.

Weekly: Cost review (15 minutes). Open Azure Cost Management if you run Azure OpenAI. Check M365 billing for any seat changes. Flag any line item more than 20% above the prior week.

Monthly: Adoption check (30 minutes). Pull the M365 Admin Center Copilot usage report. Identify users with fewer than 5 active days in the past 30. Contact their manager to determine whether they need training or whether the seat should be reallocated.

Quarterly: Compliance audit (2 hours). Export Purview audit logs covering Copilot interactions for the quarter. Confirm that log retention settings match your data retention policy. Review any flagged interactions involving sensitive data categories (health, financial, personal identification).

Alert Thresholds Worth Setting Now

Four alerts that take under an hour to configure and prevent larger problems:

1. Cost spike alert: Azure Monitor metric alert triggering at 120% of your 30-day token average.
2. High per-user token rate: A single user consuming more than 3x the tenant median in a 24-hour period can indicate prompt injection attempts or policy misuse.
3. Inactive seat alert: Any licensed M365 Copilot seat with zero activity for 21 consecutive days.
4. Purview log gap: A simple script or Logic App that alerts if the Purview audit export produces fewer records than expected for a given week (a sign that logging may have been disrupted).

EU Compliance Considerations

For European SMEs, two compliance points are non-negotiable.

First, Purview audit logging must be active before any Copilot deployment that touches GDPR-regulated personal data. This is not a best-practice recommendation; it is a practical requirement for demonstrating compliance to a supervisory authority.

Second, if you use Azure OpenAI Service and have selected European data residency, verify this in the Azure portal under your resource's geographic configuration. Microsoft offers EU Data Boundary commitments for M365, but Azure OpenAI regional availability and data residency settings are configured separately at the resource level.

FAQ

Do I need Purview if I only use M365 Copilot for internal documents?

Yes, if those internal documents contain personal data. Copilot accesses SharePoint, Teams, and Exchange content when generating responses. That constitutes processing under GDPR. Purview audit logs provide the record of what was accessed and by whom.

How do I identify which Copilot seats are worth keeping?

Pull the per-user activity report from M365 Admin Center. Any user with fewer than 5 active days and fewer than 20 prompts in the past 30 days is a low-utilisation seat. Cross-reference with their manager before deprovisioning to confirm whether the issue is awareness, access, or fit.

Can I monitor Copilot quality without Purview?

You can get partial quality signals through Copilot Studio session analytics (if you use custom agents) and through manual spot-check processes. Purview does not measure output quality: it logs what Copilot accessed and generated. Quality monitoring requires a separate process.

What is the biggest monitoring gap for Azure OpenAI deployments?

Cost attribution by department or project. Azure billing rolls up to the resource level by default. If multiple teams share one Azure OpenAI deployment, you cannot easily separate their costs without adding tags to each API call or creating separate deployments per team. Plan for this before usage scales.

Further Reading

• Microsoft 365 Copilot Governance for European SMEs: Governance foundations before you scale Copilot adoption.
• Copilot Studio vs Power Automate: Decision Guide: Which Microsoft automation tool fits which workflow.
• Copilot Studio Human-in-Loop Governance: When and how to require human review of agent outputs.
• AI Governance Framework for European SMEs: The broader governance structure that observability feeds into.