Power BI DevOps
Best Practices for DevOps in Power BI
With any system development, following good DevOps processes is very important, and Power BI is no exception.
What is DevOps? It's a process of managing changes to software or systems in a controlled fashion, usually involving source code control, version control, testing and deployment processes. Wikipedia describes it as:
"DevOps is a combination of software development and information technology operations. DevOps is a set of software development practices that aim to shorten the systems development life cycle while delivering features, fixes, and updates frequently in close alignment with business objectives." (Wikipedia)
Yes it's true that Power BI doesn't have any native support for DevOps processes, but we can manage it ourselves with good development practices.
First of all you need a source control system; you could use a number of tools, including Azure Devops Server (formally TFS) or Git, if you want an easy way of getting started with Git then try GitHub.
The point of this is every time you make a change to a Power BI Desktop file, you save the resulting PBIX file in your source control repository, with a comment explaining what has changed and why. You can then roll back to any version of the file in the future should you need to.
Note that PBIX files can become very large when you import data, so you may want to limit the size by saving the file as a PBIT, a Power BI Template. When you next load the PBIT it will load the data again. It's not ideal, but saves a lot of space. Just choose 'Save As' and change the type to 'Power BI template'.
After making a change, you need to test it. the first step is to test it locally in Power BI Desktop, this should include testing to make sure new functionality works as expected, but also regression testing to make sure that new functionality has not broken or compromised existing functionality.
Then we need to deploy a version for UAT (user acceptance testing). This requires publishing the file to PowerBI.com, however we don't want to overwrite the old version of the report just yet, in case there are any problems.
So create a new Power BI workspace, called something suitable like 'UAT Workspace' or similar. This should be secured so that only authorised testers can access it, to prevent it from being used in production.
The UAT testers can then view this version of the report in parallel with the production report, and compare the difference.
Once the new report has been signed off, deploy the new report from Power BI Desktop into the main production workspace. Don't forget to then delete the report (and associated dataset) from the UAT workspace, you don't want this one to be used going forward.
Do make sure that production reports are deployed into a shared App Workspace and not into your own personal Power BI workspace, yes it's easy to use 'My Workspace', but what if you leave your job, your account gets compromised, you want other people to modify the report, etc. etc. etc. - just don't do it!
Can this be automated?
Yes! If you'd like help in automating the process through Azure DevOps then let us know and we can discuss options with you.
Alternative Solution - Apps
If you use Power BI Apps, then you can perform UAT directly in your production Workspace, so long as your users don't have access to it, you only want your testers to see it.
Then once it's been tested and approved, you can update the App which will distribute it to the users.
So what doesn't this cover?
The process above helps the testing and change control process for your process driven development team. However, how do you apply this to your Power BI power users around the organisation; Mary in finance, Tim in production, Grace in marketing etc. - none of them have Git, or will ever want to follow good DevOps practice.
So you end up with a partial DevOps implementation. This is still better than nothing, but you need something else, another level of protection when things go wrong.
And what about backups; you religiously adhere to DevOps processes for you database development, but you still back up your production environment don't you?
You also need to know which other reports may be impacted by the report/dataset/datasource that you're about to change. You can't easily see this from within Power BI.
Power BI Sentinel - To The Rescue!
Power BI Sentinel will automate your Power BI backups, giving you peace of mind that you have a roll-back plan and an audit/investigation process should you ever need it; for both your DevOps users and the power users who don't follow your process.
The change tracking feature helps identify when something was broken, pointing you to which version to roll back to.
You can use Power Bi Sentinel's data lineage tool to determine which reports use a specific dataset or datasource. You can then very quickly assess the potential impact of making a change, without having to scour through hundreds of reports.
With any development process, the documentation is usually the part most disliked by developers; and it's almost universally bypassed by non-developers! Power BI Sentinel automates your dataset documentation, leaving your developers to focus on what they enjoy doing, and filling in the missing gaps from your business users.
Power BI, DevOps & Power BI Sentinel - a good match?
Using DevOps processes where you can will help your developers create, test and deploy good functional Power BI reports in a controlled way.
Power BI Sentinel provides an additional layer of functionality, rollback, disaster recovery and documentation to your environment, for both your DevOps team and your business users.
Together they provide complete control, and comfort that your Power BI estate has minimal risk.
We're also looking into developing a Restful API into Sentinel, to allow you to easily query the data lineage, documentation, etc. - if you're interested in this, let us know.