Home Nimbus Nimbus Integrations Historic Reporting Power BI

PBI Incremental Refresh

Incremental Refresh in the Nimbus Power BI template

Starting with template version 1.96, the Nimbus Power BI Template enables users to set up the report so that it can be refreshed incrementally. This page shows specific aspects of the Incremental Refresh functionality applicable to the Nimbus template.

About Incremental Refresh in Power BI

Incremental Refresh is a functionality in the Power BI software that partitions the semantic model and enables incremental loading of new data instead of the entire dataset at each refresh.

This Power BI feature that can be enabled optionally, if needed to regularly refresh the report for rolling updates (e.g. you need to refresh the report for the last x number of days, months etc.).

💡 The template can continue to be used also for ad-hoc refresh operations, without incremental refresh. If not making use of incremental refresh, users must remember to set the required date range in the parameters before each refresh to ensure that the desired date range is loaded into the report

🔍 For additional information on what it is and how it works, refer to the Incremental Refresh documentation on the Microsoft website.

IMPORTANT TO KNOW

Although incremental refresh policy is set up on Power BI desktop, it comes into force only once the report is exported to the Power BI Services (online). In order to take advantage of the incremental refresh functionality, after the initial set up, the report must be uploaded and distributed via Power BI Services. This is how the Incremental Refresh functionality is designed to work by Microsoft.

Setting Up Incremental Refresh on the Template

PREREQUISITES

Note the prerequisites below before setting up incremental refresh:

Licensing plans

Incremental refresh is supported for Power BI Premium, Premium per user, Power BI Pro, and Power BI Embedded models. To understand what your Power BI license allows, please refer to the Microsoft Power BI plans or contact your Microsoft representative.

Downloads to .pbix format

Incremental refresh is set up in Power BI desktop, however, it is only activated once the report is loaded and refreshed once in Power BI online. Once the Incremental Refresh is set up and running on Power BI Online, the download to *.pbix to Power BI desktop is disabled. This is by design by Microsoft.

Nimbus Template version updates

Once the template is set up and running with incremental refresh, a semantic model is saved in Power BI online and the data will be incrementally refreshed. From then on, changes to the template can only be done in the Power BI online environment and the model cannot be edited easily.

If a new template is loaded with the same name, a warning will be displayed asking if you want to replace it.

A screenshot of a computer

Description automatically generated

When overwriting the model, the data archived will be lost and the incremental refresh will have to be started again (incremental refresh will have to be set up again from scratch). This needs to be considered when deciding about uploading a new template version with the latest changes.

Advanced techniques for managing the partitions and model changes can be explored using external tools. It is recommended that those who intend to use these advanced techniques and tools have some minimum technical knowledge of data management to understand the concepts and that they familiarize themselves with the Microsoft documentation on advanced incremental refresh as well as other sources available online.

Initial refresh time limits

When setting up the Incremental Refresh, the report will need to be refreshed once to retrieve the initial archive. Depending on the size of your data and the length of the archive you set in the incremental refresh policy this could take some time.

Regardless of whether incremental refresh is enabled or not, Power BI Pro models have a refresh time limit of two hours. For models in a Premium capacity, the time limit is five hours. If the initial refresh for your tenant hits these limits, advanced incremental refresh techniques for loading the initial archive will have to be adopted. For more information refer the following Microsoft documentation and guidelines:

Memory size and capacity

Depending on the incremental refresh policy set, the number of sessions generated by your tenant and the amount of data you intend to archive in the report, the semantic model may grow into large sizes.

Irrespective of whether you decide to take advantage of the incremental refresh or not, your Power BI license and Power BI environment settings will impact the amount of data you can load into your semantic model. When limits are reached, errors may occur.

As explained by Microsoft, what you can do in the Power BI service depends on three things:

The type of license and subscription you are using.
The type of workspace that stores the content.
The roles and permissions you are assigned in a workspace.

If you are not familiar with the license and capabilities of your Power BI deployment, please consult with your IT department. If you need advice on which Power BI license is best to fulfil your business requirements, please contact Microsoft.

Set up the Incremental Refresh policy

✅ The Power BI Template must be set up. If you have not set it up yet, go through the steps in Setting Up Power BI, then return to this page once you have the template saved as Nimbus.pbix.

Navigate to the model view in the Nimbus template in Power BI desktop.
Go to the tab “Inc Refresh Facts”.
Each of the tables visible in this tab can be set for incremental refresh. To do so, start by clicking on the 3 dots on the right-hand corner of the first table.
Select Incremental Refresh from the dropdown menu.
⮑ The window will open to show the incremental refresh policy window and the selected table.
Toggle on “Incrementally refresh this table”.
⮑ The options to incrementally refresh the table appear.
Toggle on the “Set import and refresh ranges”:
⮑ You can now define the policy to incrementally refresh this table.
Repeat all these steps for each of the fact tables shown in this tab and save the Nimbus.pbix report.

Incremental Refresh policy options

All the steps for setting the incremental refresh policy are explained in the Microsoft documentation Define Policy.

💡 We recommend you to set the policy accordingly to your requirements. However, when selecting the archive data range, consider the capacity and memory allocated to your environment, based on the licensing and Power BI deployed by your organization. At the moment, Incremental refresh is supported for Power BI Premium, Premium per user, Power BI Pro, and Power BI Embedded models. In case of doubts about what license is required for your organization's needs, consult your IT department or your Microsoft support contact.

💡Optional settings are not possible with the Nimbus OData connection/data model. You can ignore these.

Learn about the initial messages…

You can ignore the warning about being unable to fold the M query.
Direct query is not applicable with OData connections. This option is not available.
Follow the “Learn more” link provided by Microsoft in this window to learn more about Incremental Refresh.

Publish the report to Power BI Services

Save the report and publish to a Power BI workspace. If you are not sure about this step, refer to our KB page Publish to Power BI Service.

Activate the incremental Refresh policy

Once publishing to Power BI service is complete, go to the semantic model settings.
Ensure that the connection credentials are correct.
Check if the parameters are set as required (enable optional tables if needed).
💡 Notice that RangeStart and RangeEnd are no longer visible. This is because Power BI will automatically set the dates based on the Incremental Refresh policy you have set.
Set the Large semantic model storage format to ON if you have a premium license.

💡 This must be done before the first refresh. It is required if your tenant generates huge amount of data and you are loading an extensive range of dates into your archive which may grow your semantic model beyond 1GB. This setting can improve refresh operation performance and ensure the model does not max out size limits.
🔍 Refer to the Microsoft documentation about large semantic models in Power BI premium for more information.
Once the above is set, perform an initial refresh operation on the model. This refresh should be an individual (manual) refresh so that you can monitor the progress. This will load the initial archive based on the set incremental refresh policy.
💡The initial refresh operation can take quite a while to complete. Partitions must be created, historical data loaded, objects such as relationships and hierarchies built or rebuilt, and calculated objects recalculated.

Subsequent Data Refresh

At this point, you can set a scheduled refresh or you can refresh the report manually when you need it.

From now on, each time the report is refreshed, Power BI will follow the original Incremental Refresh policy and will incrementally refresh the data since the previous refresh.

Luware Knowledge

Nimbus

Recording

Resource Library

PBI Incremental Refresh

Incremental Refresh in the Nimbus Power BI template