How I use Azure VMs for Power BI development
For once this is not a technical blog about a feature or trick in Microsoft Power BI.
In this blog post, I want to explain the benefits and downsides of using Azure VMs instead of regular hardware. This might be especially interesting for people using MS Power BI.
What is an Azure Virtual Machine?
I am by no means a hardware expert but to me an Azure VM is a virtual computer that is highly flexible in its scaling (processing power, storage etc.) and at the same time offers everything a normal computer has to offer. You can think of it as a computer that is running somewhere which you can always access via remote desktop connection (like some people use their desktop computers)
There are a lot of things you can define when deploying (think of it as building) an Azure VM, the most important things are:
-Operating system (Windows server or client)
-Location (Azure data center, IP address)
-Hardware specs (vCPUs, RAM, storage)
What are benefits of using a VM for Power BI (or other MS applications)?
1) You do not have to buy any hardware, you can use a Mac or any windows machine, even a tablet to acces the VM (via Remote Desktop Connection).
2) Costs are quite OK, as you can see, the machines themselves (you pay per minute of using them) are the biggest cost but similarly high is the storage (if you want a fast SSD with 128 or 256 GB of permanent storage space at least)
Obviously it depends on a lot of factors but what I am using is this size (hardware specs):
B4ms (4 vCPUs and 16GB RAM)
B4ms costs me about 15-25€ per month per machine depending on how much you use it and which region you pick. As soon as you deallocate it (this is not the same as turning it off) you won't for it anymore.
Then we have storage, which is about the same price per month per machine for my default 128GB Premium SSD. This is a fixed cost since it will not get deleted when you shut it down.
The third component is the virtual network, if you do not pay for this you will not have a static ip and every day when connecting to your VM you need to update it.
Overall you can calculate with 30-40€ per month for one VM. If you have an Office license of F3 or higher, you will not have to pay for any Windows licensing which is a major plus as well.
If I compare this to a laptop for 1.500€ you can pay for the VM for 40-45 months until it gets more expensive than this laptop. Also do not forget the flexibility.
3) The third big plus for me is the flexibility it offers. You can use one VM for every major client project you have. You can put the VM in the region of the client and you will not need any VPN or anything since you have a static IP and the client can authorize that. Even though you might be in Bali laying in the sun (still planning on doing this one day) you could connect to your VM having the same IP as always not running into any trouble.
4) For me security is also a big benefit since all data will safely be stored on the VM. No need to send around anything, no need to manage multiple folders or OneDrives.
One VM, one OneDrive, one Teams account, one Outlook account etc.
When the project ends, you can be 100% sure that you will destroy all the data your client gave you. Who else can say that?
I am not sure if it is easier or harder to hack a Azure VM than a normal computer but my guess is it is harder since the chance that you get or have any malware/virus installed there is much lower.
If you use a public WiFi you will know that any connection from the VM to the client will be as safe as always and data interception seems like it is not an issue.
There are three security layers to access/login to a VM
First is a login and password including the IP address and port of the VM which anyone who wants to connect requires.
The second one is that you can define a single IP address that can login, your ip address. Any other address even though they might have the login and pw will not be able to log on.
Third is that when you turn off the VM it is 100% off (unless anyone can start it without you knowing it using your Azure credentials of course)
I think having everything in one place without any client data getting mixed with data of another client is the biggest plus for me in terms of security.
5) Being able to access a co-workers or employee's machine is an additional benefit. This fosters cooperation and can be quite handy sometimes. There are obvious downsides of course which I will go into more detail in the cons.
6) No annyong wait times for transformations, data loads or laptops overheating. Since all the processing and hard work is done on the VM, your laptop will be nice and cool and you can do other things while waiting for transformations to finish or large datasets being refreshed without
7) It is an absolute must (at least try it out) for remote Teams, nothing to add!
8) The VM cannot crash (at least it has never happened to me) and it does happen to my desktop/laptops. So whatever you are working on, even if you have a power outage or any kind of disaster, the VM will stay up and running.
9) Even in a slow internet environment you can work. The VM will transfer all the data for you, do updates. Downloading the latest version of Power BI or any other software will still be super fast even though your internet might be slow.
10) If you need more processing power you can easily upgrade to a bigger size (more performant hardware) this has some limits though.
11) Logins, accounts etc. will be on the VM and you do not have to remember or pull passwords. When you start the VM it will log into the correct Teams, Power BI or Outlook (via your browser).
12) Get started within 5 minutes, it really is that fast.
What are downsides of using a VM for Power BI (or other MS applications)?
1) You still need some kind of hardware and might end up spending more than before (at least if you are flying solo)
2) The VM will be slower than a normal 4 core CPU (apparently vCPUs are about half the speed of physical CPUs so a 4 core is basically just a 2 core)
3) Memory on a VM is not like physical memory. This learning came by surprise to me. I reached the limits of the physical memory on my machine with a large Power BI dataset due to the clients vast transformations and inefficient datamodel. Which means I got a memory overflow error every time. I had to do transformations outside the VM to fix this which were not problem on a physical machine with 16 GB RAM.
4) There is some kind of delay when you are working on the VM just because of internet latency. So you will have to get used to that and it can be annoying at the beginning. I would not recommend playing any games there ;) The extent of the delay will depend on your internet connection and how far you are from the region the VM is located it.
5) Learning the inside out of Azure and its VMs, sizes, regions, storage etc. was quite tricky I found among other things can be annoying at first. So be ready for learning more about this technology. I think Microsoft could have been more user friendly and less technical here.
6) You might waste some money at first because you will deploy your VM wrong or forget to de-allocate it but rather just shut it down and then you will keep on paying every minute. The good thing is thougt that you get 200€ for free (in the first month) after signing up. This was also my main motivation of writing this blog post.
7) Creating images for new VMs could be better since you do not want to start from 0 every time you deploy a new VM. I have not figured out a good way yet how to update my image in some way without going through a whole process. Now every time I set up a new VM I will have Power BI, Teams etc. installed but older versions of it since I created that image months ago. But I also cannot create an image from a current VM since it would have client data on it.
8) When your internet connection is a complete disaster or you are offline it is game over.
9) Doing meetings can be challenging, in the sense that the VM will not be able to access your laptop's microphone? At least I couldn't figure that out. When having meetings you might have to get creative using your phone and VM at the same time or just doing it through your actual laptop and then share the VMs window etc.
10) Ongoing but idle projects will incur costs even though the VMs might be offline most of the time. This is only the case of course if you stick with separating projects/clients by VM.
The storage costs are running permanently until you destroy the VM so this could add some 20€ per month without any real benefit if the project is just idle. Maybe that is some extra motivation there to end things when they should end ;)
11) For some use cases VMs are not suitable. Like that client that requires you to use their hardware.
Would I do it again?
After all these months of learning and experiencing the ins and outs of Azure VMs I think it was a good decision and I am continuing to rely on this technology. I think everyone should give it a shot (you have free money to spend the first month and play around). It is the only way to find out if it can be a suitable add-on or all out solution for your requirements.
I think especially people working on a Mac who also work in the Microsoft environment using tools like Power BI where there is no native version for Mac should take a good look at this. It could be better than local virtualization software but see for yourself.