Microsoft Fabric Monitoring with Metrics App and Data Activator

By:   |   Updated: 2024-05-13   |   Comments (2)   |   Related: > Microsoft Fabric


Problem

With Microsoft Fabric now generally available, organizations are interested in implementing this flagship Unified Data and AI Intelligence Platform for several reasons. Its native integration within the Azure stack provides seamless and secure access to widely used technologies for data integration, business intelligence, and advanced analytics. Microsoft Fabric's storage and compute capacity is utilized by resources within this unified analytics platform, including storage repositories, such as data warehouses and data lakes, and compute capacity for Power BI, Pipelines, DW processing, and artificial intelligence (AI)/machine learning (ML) workloads.

Fabric capacity can be purchased on Azure with a pay-as-you-go model, and a 60-day free trial (64 CUs) is offered to test the platform. Organizations that have an existing Power BI Premium capacity can easily enable access to Fabric by using the Microsoft Fabric admin switch. Enabling Fabric in Power BI Premium as opposed to Azure Portal creates a problem: there is no easy way to monitor and set alerts on your Fabric capacity metrics in the Azure Portal.

Solution

The Microsoft Fabric Capacity Metrics App is designed to provide monitoring capabilities for Fabric and Power BI Premium capacities. Data Activator in Microsoft Fabric is a no-code tool that automatically monitors data in Power BI reports and Eventstreams items and takes actions, such as alerting users or initiating Power Automate workflows when the data hits certain thresholds or matches specified patterns. When combined, the Metrics App and Data Activator provide a comprehensive suite of tools for monitoring and taking necessary administrative action to control your organization's Fabric capacity. In this article, you will learn more about both the Metrics App and Data Activator for Microsoft Fabric.

Installing the Metrics App

The Metrics App can be installed from within your Power BI apps by searching for 'Fabric.'

PBI Apps Selecting Fabric Metrics App

Note: The Metrics App is free. However, you need to be a Fabric Capacity Admin to install this app. You can learn more about the detailed steps required for installing and upgrading the Metrics App from Microsoft's article: Install the Microsoft Fabric capacity metrics app.

Download Metrics App Steps to get the app

After the installation, remember to connect your data by providing your Capacity ID to the app. You can find the capacity ID in the URL of the capacity management page. In Microsoft Fabric, go to Settings > Admin center > Capacity settings, then select a capacity. You also need to provide your Authentication Method (OAuth2 is the default) and set your privacy level. Select Organizational to enable app access to all the data sources in your organization.

As the Metrics App loads for the first time, you might see a page like the figure below. Once loaded, you can select your organization's capacity from the 'Capacity Name' drop-down. Remember to consistently check for new upgrades to the Metrics App to stay up-to-date with the latest version.

MetricsAppInitialize Metrics app is installing and rendering data.

Understanding the Metrics App

The Metrics App renders as a Power BI report within your Fabric workspace. The app contains two pages: Storage and Compute.

The Storage page provides insights into storage (GB) by workspace. This can be useful to identify storage trends, high-cost workspaces, and those reaching storage capacity limits. You also gain access to OneLake in Fabric, a unified, logical data lake for your organization. It is like OneDrive for your analytics data, improving collaboration by eliminating the need to manage multiple resources. Each workspace in OneLake is part of a capacity tied to a specific region and billed separately. Built on Azure Data Lake Storage Gen2 (ADLS Gen2), OneLake supports any file type and automatically stores all Fabric data items in Delta Parquet format. It is compatible with existing ADLS Gen2 applications, including Azure Databricks.

Storage Page Visual of the storage page in the metrics app

The Compute page is useful for Fabric Admins to troubleshoot and monitor capacity usage. Microsoft has a detailed article to help you understand the Metrics App compute page. You can also learn more about Fabric capacities from Microsoft's blog.

Capacity management in Fabric involves features like bursting and smoothing to optimize performance and manage compute spikes without throttling, ensuring efficient job execution. Bursting allows you to consume extra compute resources beyond what you have purchased to speed up the execution of a workload. For instance, a job that would normally run on 64 Capacity Units (CUs) and complete in 60 seconds could, with bursting, use 256 CUs to complete the job in only 15 seconds.

On the other hand, smoothing offers relief for customers who create sudden spikes during their peak times while they have a lot of idle capacity that is unused. It simplifies capacity management by spreading the evaluation of compute to ensure that customer jobs run smoothly and efficiently. For example, if a customer's peak usage is at 10:00 AM and they have idle capacity at 3:00 PM, smoothing would distribute some of the compute from 10:00 AM to 3:00 PM, ensuring a more efficient use of resources.

The visuals within the Compute page of the Metrics App give insight into when the largest capacity unit spend occurred. Selecting a day within the 'Multi metrics ribbon chart' also enables the other visuals and tabular data to filter for that day.

Metrics App View View of the fully loaded and rendered metrics app for a selected capacity.

By hovering over the '?' icon within each card on the report (shown below), you will see detailed definitions of the metrics that are displayed on the app. The CU % over time visual shows hourly capacity usage, including processing time, operations, and users. It also displays CU usage over time and the percentage of CU consumption for different operations and capacities. The app helps monitor capacity consumption, providing metrics like storage usage and CU seconds consumption, aiding in capacity management decisions. It is important to note that Interactive % (Columns in Red) is smoothed over 5 minutes, and Background % (Columns in Blue) is smoothed over 24 hours. A significant amount of blue could mean that background operations need to be optimized, and red could mean bad models, DAX, high concurrency, etc.

CU Over time definition details of the definition of CU over time measures in the card.

The Items visual is a matrix table that provides detailed information about each item's usage in your capacity. It displays metrics such as CU processing time in seconds, the number of operations that took place, and the number of users that performed operations. This visual helps you understand the consumption of CU seconds for a specific item over the past 14 days. It is a valuable tool for monitoring and managing your capacity resources effectively.

Item definition details of the definition of Items measures in the card.

You can also hover over the tooltip details within the 'Items' section to know the operation level CU usage. If the CUs are mostly tied to Queries, you might need to explore optimization opportunities for data models, DAX, and report design. On the other hand, if CUs are higher for Dataset refreshes, then you might need to think about enhancement opportunities around incremental refreshes, alternate data connections to OneLake, and leveraging the new Fabric Mirroring capabilities to minimize or eliminate excessive dataset refreshes.

Item ToolTip 
Details of the Item tool tip showing operations detail

Enabling Data Activator

This section will describe how to enable notifications and alerts on metrics and visuals within the Metrics App with Data Activator. Fabric Data Activator is a no-code feature in Microsoft Fabric that automatically takes actions when patterns or conditions are detected in changing data. It monitors data in Power BI reports and Eventstreams items, and can trigger actions such as alerts or Power Automate workflows when data hits certain thresholds or matches patterns.

Data Activator Selecting data activator

Within Data Activator, you can create a Reflex, which is a rule that monitors data and triggers actions when certain conditions are met. Remember that you will need a premium Fabric workspace to enable this.

Selecting Reflex how to select reflex in Data Activator

Reflex is built into Power BI by opening your report and selecting the 'Trigger action' to create a new Reflex.

Connecting Reflex to PBI instruction on how to set data activator alerts for PBI

For example, you can navigate to 'More actions (…)' within your 'CU % over time' visual and then 'Trigger action' to set an alert to receive an email when the CU percentage exceeds 100%.

CU over time alert step to create alert for exceeding CU threshold 100%

You can do the same for other measures, including those in the 'Items' card, to create automated alerts for selected measures and define conditions, thresholds, and notification types (email or Teams).

Item Alert step to create alert when perf delta exceeds -15.

Depending on your Notification Type selection, you can add one or more recipients, including a header, optional message, and additional information.

Teams Notify Configure notification details.

Once the alert is started, a Data Activator Reflex will be created that allows you to monitor the alert in real time with the built-in visuals.

DataActivator Monitor Visual to monitor the alerts in real time

When the trigger condition is met, you will get an email with the alert details and a link to open the report in question.

Reflex Alert Email Email that you will receive once the data activator reflex alert is triggered.

Similarly, if you opt to receive a Teams notification, then you will receive a Teams message with the alert.

Reflex Alert Teams Teams message that you will receive once the data activator reflex alert is triggered.

Customizing the Metrics App

Finally, you have the option to view the semantic model of the Capacity Metrics App, allowing you to build your own custom reports or create other custom processes using the data model.

Semantic Model Step to view the metrics app semantic model.

You may need to gain access to the semantic model by clicking 'Request access,' which can be granted by your Fabric Admin.

Request access step to request access to the semantic model.

An example of a custom report could be an alert being received for CUs that exceed the maximum autoscale capacity units. You can locate your allocated base and Autoscale capacity units within 'Capacity settings' in the Admin portal. If you have a P3 Power BI premium SKU with 256 base CUs and additional 48 Autoscale CUs available, then you could create a report that sends an alert when your Autoscale CUs exceed 48.

PBi Capacity Settings how to view your PBI capacity settings

To create this custom report and alert, you will need the following data points: sum of autoscale capacity units by Timepoint.

Custom Metrics Report How to create a custom metrics report.

Once the report is created, you can set an alert when the autoscale CU threshold of 48 is exceeded.

Setting custom alerts step to set a custom alert for the customer report.

You are now equipped with the skills to monitor your Fabric compute and storage capacity with the Fabric Metrics App. You also learned to access the Fabric Metrics semantic data model to create custom reports and alerts. Finally, using Data Activator and Reflex, you learned to create and trigger email and Teams alerts for metrics that are out of bounds for pre-defined thresholds.

Next Steps


sql server categories

sql server webinars

subscribe to mssqltips

sql server tutorials

sql server white papers

next tip



About the author
MSSQLTips author Ron L'Esteve Ron L'Esteve is a trusted information technology thought leader and professional Author residing in Illinois. He brings over 20 years of IT experience and is well-known for his impactful books and article publications on Data & AI Architecture, Engineering, and Cloud Leadership. Ron completed his Masterís in Business Administration and Finance from Loyola University in Chicago. Ron brings deep tec

This author pledges the content of this article is based on professional experience and not AI generated.

View all my tips


Article Last Updated: 2024-05-13

Comments For This Article




Friday, May 17, 2024 - 10:54:50 PM - INGO R JOBLING Back To Top (92250)
Excellent article, thank you!

Wednesday, May 15, 2024 - 7:26:31 AM - Koen Verbeeck Back To Top (92237)
You can also learn about Fabric capacities in this tip:
https://www.mssqltips.com/sqlservertip/7754/microsoft-fabric-capacities-and-cost-structure/
And about data activator in this tip:
https://www.mssqltips.com/sqlservertip/7912/what-is-data-activator-in-microsoft-fabric/














get free sql tips
agree to terms