Azure

Cloud Security via Security Center

by

Another critical part of managing any cloud is security. In Azure Microsoft has a service called Security Center. I am going to cover Security Center at a high level here in this post as Security Center itself is a big topic and is frequently changing with new improvements. This provides continuous assessment of your clouds security posture. Security Center gives you a central place to monitor and manage your security. Security Center can even covers Hybrid Cloud with the ability to extend on-premises. With Security Center you can apply security policies to your cloud workloads and respond to attacks that occur.

Security Center has a “free” tier that can be used with any Azure subscription. In fact if you are running Azure you should at a minimum be utilizing the free tier of Security Center. The tiers are:

Not covered = not monitored by Security Center.

Basic Coverage = subscriptions under this “free” tier are under the limited, free level of Security Center.

Standard Coverage =  subscriptions under this “standard” tier have the maximum level coverage by Security Center.

Key features in Security Center are:

– Security policy, assessment, and recommendations / free / Security Center performs continuous assessment and recommendations based on security policies that are set. This is the core feature of Security Center.

– Event collection and search / standard / Security Center can store security events in a Log Analytics (LA) workspace. The events also are available in the LA workspace for searching.

– Threat Protection / standard / visibility into any detected security alerts and their severity level.

– Just in time VM access / standard / Just in time VM access locks down inbound traffic to IaaS VM’s. With this feature users are required to request access to the VM for a specified amount of time. A firewall rule is opened on an NSG allowing the access and then the ports are closed after the allotted window of access time. This can reduce the attack surface on VM’s.

NOTE: (Automate Just In Time VM Access Request With PowerShell – https://github.com/CharbelNemnom/Power-MVP-Elite/tree/master/Request%20JIT%20VM%20Access by Microsoft MVP Charbel Nemnom)

– Adaptive application controls / standard /  This feature allows you to choose what applications are allowed to run on your VMs. This feature uses machine learning to analyze the applications running in the VM and then you whitelist the ones you want to allow to run.

– Custom alerts / standard /  Security Center has a bunch of default alerts. Alerts fire when a threat, or suspicious activity occurs. You can find the list of the default alerts here: security alerts. Security Center also has custom alerts that you can setup. With these you define the conditions upon which an alert is fired.

– Threat intelligence  / standard /  this feature watches for known bad actors using threat intelligence data from Microsoft’s global products and services such as Azure, Office 365, Microsoft CRM online, Microsoft Dynamics AX, outlook.com, MSN.com, and Microsoft’s Digital Crimes Unit (DCU) and Microsoft Security Response Center (MSRC) .

It is important to note that Security Center leverages many other Azure services to power its services. Some of these other Azure services include:

  • Azure Policy
  • Log Analytics
  • Logic Apps
  • Machine Learning

Now that we looked at key features of Security Center let’s take a tour of Security Center. The best way to navigate Security Center is via the navigation on the left hand side and that is the way I will break it down. The menu sections are shown in the following table:

When you first click into Security Center you will see the Overview. Overview is also the first section under “General”. Here is a screenshot of the overview pain.

Essentially the overview pane gives you a summary of your security posture pulling in data from several sections in Security Center. Getting started is where you can launch a 60 day trial on the standard plan. Events brings you to a log analytics workspace dashboard to give you another display and search capabilities on your security data. Search will bring you directly to the log analytics search screen where you can search on your security data.

Read more

Azure & Azure Stack Resource Group Cleanup Script

by

When building things in Azure & Azure Stack I tend to create a lot of temporary resources groups. I like to remove these when I am done. I have been using a PowerShell script for a while to help make this easier. I have decided to upload this script hoping others will find it useful as well. The script is named CleanupResourceGroups.ps1 and can be downloaded here:
https://gallery.technet.microsoft.com/Cleanup-Azure-Resource-d95fc34e

The script can be used two ways:

#1 the script can be run using -Like with an expression like where {$_.ResourceGroupName -like (‘*MySQL*’) in which the script would remove any resource group with MySQL in it. To use this option just un-comment the code in SECTION 1- Uses -Like, change MySQL to whatever you want, comment SECTION 2- Interactive RG selection code, and then run the script.

#2 the script can be run interactively allowing you to select multiple resource groups you want to remove. By default the SECTION 2- Interactive RG selection code is un-commented. If you run the script it will run interactively as shown in the following steps/screenshots.

After running the script it will prompt you to select an Azure subscription.

Next the script will give you a list of resource groups in the subscription you selected. Select the resource groups you want to remove and click Ok.

The script will loop through and remove the resource groups you selected. Note that script is using -Force so it will not prompt to ensure you intend to remove the resource groups. Make sure you want to remove the resource groups before running this script.

NOTE: When running this for Azure Stack ensure you are logged into the Azure Stack environment. For info on how to do this visit: https://bit.ly/2LkvddG

That is it. It is a simple script to make removing many resource groups easier. I hope you find this script useful as I have!

Read more

The “argument is null or empty” error in Azure Automation Runbook

by

I was recently working on an Azure Automation runbook that provisions an empty resource group in Azure. I was running into an issue when the runbook ran that the variable being used with New-AzureRmRoleAssignment was null. The errors I was receiving are:

New-AzureRmRoleAssignment : Cannot validate argument on parameter ‘SignInName’. The argument is null or empty. Provide
an argument that is not null or empty, and then try the command again.
At line:96 char:39
+ New-AzureRmRoleAssignment -SignInName $RequesterSignIn -RoleDefinitio …
+ ~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidData: (:) [New-AzureRmRoleAssignment], ParameterBindingValidationException
+ FullyQualifiedErrorId :
ParameterArgumentValidationError,Microsoft.Azure.Commands.Resources.NewAzureRoleAssignmentCommand

and

New-AzureRmRoleAssignment : Cannot validate argument on parameter ‘ObjectId’. Specify a parameter of type ‘System.Guid’
and try again.
At line:97 char:37
+ New-AzureRmRoleAssignment -ObjectID $RequesterID -RoleDefinitionName  …
+                                     ~~~~~~~~~~~~
+ CategoryInfo          : InvalidData: (:) [New-AzureRmRoleAssignment], ParameterBindingValidationException
+ FullyQualifiedErrorId :
ParameterArgumentValidationError,Microsoft.Azure.Commands.Resources.NewAzureRoleAssignmentCommand

It turned out to be a permission issue with AzureRM.Resources CMDLETS not being able to talk to AAD specifically Get-AzureRmADUser that I was using for a variable.

To fix this I had to give the following permissions for the AAD directory to the AzureServicePrincipal Run As Account:

Windows Azure Active Directory (AAD)
Application Permissions

·       Read/Write directory data
·       Read directory data

Delegated Permissions
·       Read directory data
·       Read all users’ full profiles
·       Read all users’ basic profiles

Microsoft Graph
App Permissions
·       Read directory data

In your runbook code you will typically have

# Authenticate to Azure resources
$connectionName = “AzureRunAsConnection”

# Get the connection “AzureRunAsConnection “
$servicePrincipalConnection = Get-AutomationConnection -Name $connectionName
“Logging in to Azure…”
Login-AzureRmAccount `
-ServicePrincipal `
-TenantId $servicePrincipalConnection.TenantId `
-ApplicationId $servicePrincipalConnection.ApplicationId `
-CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint

You may have a some differences like the connection variable and the name of the runasconnection. The point here is that the runas connection is what needs to have the proper permissions. You can find this account here to get the name and ApplicationID:

To give the permissions go to Azure Active Directory>the directory you are using in this automation>App registrations>and search based on the ApplicationID. Don’t forget to select All apps in the drop down.

Click on Add first and add the AAD and then Microsoft Graph permissions.

After you add the proper permissions make sure you click on Grant Permissions. The permissions are not actually applied until you do this. Once you click on Grant permissions you will see the prompt shown in the screenshot. Click Yes.

Verify the permissions have been added properly. In AAD go to All applications>select All applications. Find your service principle application.

Click on the service principle applications permissions.

Verify the AAD and graph permissions are listed. If the AAD and graph permissions are listed then the runbook should be good to go.

Read more

Unlink yourself from unused AAD directories

by

Working in the world of consulting I am often added to other Azure Active Directories that are managed by someone else. After a while these can pile up like in the following screenshot.

I like to clean these up as the inviting organizations typically don’t remove you. Here is a quick way to do this. In a browser go to https://myapps.microsoft.com. It will look like this:

Click on your name and select the directory you want to remove. Click on your name again and click on the cog for the settings.

You should then see the option to “Leave the organization”. Click the link.

You will see the following pop-up. Click on Leave.

That’s it. You will no longer see the directory you just removed listed in the Azure portal under your directory list.

Read more

Microsoft Professional Program for DevOps Finished!

by

I am a firm believer that no matter how old you are, how far along you are in your career, and regardless of the industry you are in it is important to continue educating yourself. This helps you expand your skillset, stay relevant, and sets you up for new opportunities as they come along. My field of information technology has been changing at a rapid pace and so for a while, I wanted a good way to ramp up on DevOps as a whole. A while back I found out that Microsoft added a new track to their Professional Program for DevOps. When I checked it out I found it to be very thorough and it was not just focused on Microsoft’s DevOps tooling but included non-Microsoft as well. I jumped in without hesitation and started learning.

I finally completed the program last week. Here is my certificate https://academy.microsoft.com/en-us/certificates/67284e84-8afe-4f13-b477-d7620949fb18. I am planning to dive into the program they have for cloud next. If you have not heard of Microsoft’s Professional Program DevOps before here more information about it:

“DevOps is the union of people, process, and products to enable continuous delivery of value to end users. This program helps the student learn about continuous integration and deployment, infrastructure as code, testing, databases, containers, and application monitoring: skills necessary for a DevOps culture in today’s workplace. This program focuses on Microsoft DevOps technologies as well as some OSS (Open Source Software) DevOps tools. Some of the Microsoft DevOps technologies covered in this course consist of Azure, Azure Resource Manager, IaaS, PaaS, IIS, Azure App Service, DevTest labs, Desired State Configuration (DSC), Azure Automation, OMS, Application Insights, SQL, Nuget, TFS, VSTS, and Visual Studio. Some of the OSS DevOps tools covered in this course consist of Jenkins, Git, Github, New Relic, Nagios, Chef, Docker, DC/OS, swarm, and Kubernetes.”

Here is a link to it:  https://academy.microsoft.com/en-us/tracks/devops

This program consists of 8 required courses. Each course runs for three months and starts at the beginning of a quarter. In the end, there is a capstone that has to be completed. This capstone course is the 8th one. You have four weeks to complete the capstone. The capstone is a bunch of hands-on stuff you have to do. Courses average 16-32 hours per course to complete and are taken via the edX.org platform https://www.edx.org/microsoft-professional-program-devops.

Here is a list of all of the DevOps program courses:

      • Introduction to Dev Ops Practices
      • Infrastructure as Code
      • Continuous Integration and Continuous Deployment
      • Configuration Management for Containerized Delivery
      • DevOps Testing
      • DevOps for Databases
      • Application Monitoring and Feedback Loops
      • Microsoft Professional DevOps Capstone Project
      • The DevOps Capstone Project contains:
        • Automation
          • Use ARM templates to deploy and configure Infrastructure in Azure
        • Continuous Integration
          • Implement Continuous Integration solution using Visual Studio Team services (VSTS)
        • Continuous deployment
          • Implement Continuous Deployment solution using Visual Studio Team Services (VSTS)
        • Testing
          • Implement Unit tests
          • Implement Testing in Production
        • Application Monitoring
          • Implement application monitoring solution using Application Insights

As you can see from that list this program is not just all about VSTS. There is a lot of Azure baked in as well as other non-Microsoft DevOps tooling. I highly recommend this course for anyone jumping into DevOps, or CloudOps and especially for folks with an IT pro background. If CloudOps is foreign to you here are a couple of blogs related to this topic: Sys Admin to Cloud Admin…ITSM to CloudOps…On-Prem to Azure Stack/Azure and Native Cloud Management in Azure.

My personal opinion is that Microsoft should move away from the certifications as they are and to this format. This format combines training and testing. When Microsoft first started the Professional Program for they only had a track for data scientists. They have added more and more tracks over time. Today there are tracks also for Big Data, Web Development, Software Development, AI, IT Support, and Cloud Administration.

Here is a link for all the tracks so you can check them out: https://academy.microsoft.com/en-us/professional-program/tracks. These programs are a great way to expand your learning. Check them out!

Read more

Native Cloud Management in Azure

by

For those that know me know that I have been a System Center expert for some time focused on helping organizations manage their IT along with their ITSM needs. I have been working with Azure since it was released off and on but started to get serious about Azure after Microsoft’s move to resource manager. And even more recently I have re-focused completely to Azure and DevOps along with ITSM in the context of the cloud. I consider this combination CloudOps.

CloudOps is important when it comes to cloud and supporting DevOps. A part of CloudOps is cloud management. More specifically the tooling name for cloud management is often referred to as Cloud Management Platform (CMP).  CMP’s can be a CloudOps architect and engineers best friend or worst nightmare. There are many CMP solutions out there in the market that can be used to manage Azure and other clouds as well. Microsoft has done a nice job building and bringing in native solutions that can be used to manage Azure. The following image depicts the areas of cloud management that are in focus for Microsoft.

I am sure the plan for native cloud management will change and expand over time as Azure and its management needs continue to grow. The native set of cloud management tools in Azure can be viewed as a CMP. I am going to put together a group of blogs that at a high level cover the native solutions that exist for managing and securing Azure. There are so many areas in this topic that it has to be broken out into a blog series. This is the first time I am doing a blog series. It will cover the following:

Check back on this post soon. As I create more blog posts in this series they will be linked on the list above.

Read more

Azure Cost Management (Cloudyn)

by

IT financial management (ITFM) is an important part of IT operations as business dependency on IT continues to grow in the age of digital transformation. ITFM is a part of ITIL as a Service Strategy element in the framework. ITFM is a key part of CloudOps as well because spending in the cloud is based on an OPEX model and every single cost is tracked. ITFM and cost management in the cloud should be used to effectively and concisely connect the dollars spent on IT to the value delivered to the business. We can do this with Azure Cost Management. In this post, I am going to give an overview of Azure cost management highlighting many of the things you can do with it. Let’s dive into the solution now.

Overview

In June of 2017, Microsoft acquired Cloudyn a startup that had tooling for cloud monitoring and analytics tools focused on cloud financial management. Cloudyn’s solution is multi-cloud covering Azure, Azure Stack, AWS, and GCP. Through the acquisition of Cloudyn Microsoft was able to bring the tooling into the Azure ecosystem giving Azure customers an enhanced way to track and control cloud spend improving the improving the Azure cloud governance story.  As of right now, there is a free level and a paid level for Azure cost management. The following table lists what features are available with each level.

FREE capabilities:

Reporting Report on cost and usage
Data enrichment Categorize by resource tags
Budgets Create and manage cost and usage budgets
Alerting Create alerts on cost and usage budgets
Recommendations Eliminate idle cloud resources

Right-size cloud resources
PAID capabilities:

Chargeback features including cost markup, redistribution, and custom charges
Import external budgets
Customize recommendation thresholds
Categorize costs with custom meta-tags

Since the acquisition, Microsoft has added a link to the Cloudyn portal directly in Azure and integration with your Azure subscriptions giving you the ability to launch a new Cloudyn account that is tied to your subscription. Microsoft added Cost Management in Azure and this is where you will find Cloudyn and sign up. As shown in the following screenshot you can see the “Go to Cost Management” button. After clicking on that you will go the Cloudyn portal and will be able to add your various cloud accounts.  The thing that I really like about Azure cost management is that there is a ton of data and dashboards that are available right out of the box after adding a cloud account. There is not a bunch of configuration that you need to do to get the default dashboards and optimization tools.

After you are all signed up and have your cloud accounts added your dashboards will start to show data. The next two screenshots show a couple of the default dashboards.

The management dashboard gives a good summary of your cloud financials on one pane of glass.

 

The cost controller dashboard shows cost trends, some forecasting info, a breakdown of costs and more.

As you can see from the previous screenshots there are several other dashboards with other content. You can modify any of these dashboards adding or removing widgets. You also can create your own dashboard adding whatever widgets you want to it.

In Azure cost management, you can add cost centers known as Cost Entities. Entities are intended to mirror your organization’s hierarchical structure such as business units, divisions, departments, or teams within your organization some examples are engineering, R&D, development, marketing etc. The goal of the entities is to give you a way to track cloud spend by the entities. Keep in mind the cost entities can be anything that fits the way you want to structure and track cloud costs. You also can leverage tags, add budgets, and then associate costs and or budgets to the cost entities into cost models. Cost models give you a way to distribute and allocate costs. You can track costs back to these cost entities and you can track costs against budgets for showback or chargeback scenarios. Below is a screenshot of the cost entities screen. Keep an eye out for a detailed blog from me walking through how to structure and set up this part of Azure cost management. This area of Azure cost management warrants its own dedicated blog.

Here is an example of a budget set on a cost entity.

Read more

Azure Policy

by

A key component of cloud governance in Azure is being able to apply policies across cloud resources. In Azure, there is a  service called Azure Policy that can be used to define policies and enforce them across your cloud resources. Azure Policy can be used to create, assign and, manage, and apply policy definitions. Azure Policy can be set to just evaluate when resources are out of compliance or remediate when resources are out of compliance. These two modes are known as audit effect and deny effect.

Azure policies can be applied to Management Groups, subscriptions, or resources.

Azure Policy has been around for a while but recently it has revamped to make it enterprise ready. Azure Policy is in preview but it won’t be long before it will go GA and can be used to help manage your Azure. There is no pricing yet while Policy is in preview.

Azure Policy is not RBAC. RBAC deals with user access and user actions such as what users can access what resources and what they can do with them. Azure Policy deals with existing resources and resource properties during the deployment of them.

In Azure Policy you have something known as definitions. Definitions are essentially compliance rules that can be assigned to Azure resources. These definitions can just check to see if items are compliant or not and can enforce compliance. Definitions can be used to set conventions for resources, for example, all resources in a subscription should have a certain tag when created. Definitions are also used to evaluate something and take an action based on the result of the evaluation. A good example of this is that you could use a policy definition to evaluate if virtual machines are using managed disks or not. Azure Policies are used to help control costs and manage resources across your Azure subscriptions.

There are two types of definitions called Policy and Initiative. A Policy definition is a single definition. An Initiative definition is a group of Policy definitions. Initiative definitions are used to help achieve larger compliance need. To gain a better understanding of Initiative definitions you can look at Security Center as it leverages Initiative definitions. Security Center has a built-in Initiative definition named [Preview]: Enable Monitoring in Azure Security Center. This built-in Initiative definition for Security Center contains 13 Policy definitions related to security as shown in the following screenshot.

In Azure policy there are built-in and custom definitions. The built-in definitions have been created by Microsoft and are ready to be used to help with common needs in cloud. There are 36 built-in policy definitions today. Custom definitions are built by you. All Azure policies are JSON so writing custom polices is similar to writing ARM templates. Templates for Azure policies can be found in the Repository for Azure Resource Policy samples here: https://github.com/Azure/azure-policy. You can use these samples as a starting point when building your own. Here is an example of an Azure policies JSON:

Read more

Azure Management Groups

by

If your company is like most organizations that are using the cloud, then you have many subscriptions floating around. This is often due to “shadow IT”. However, sometimes organizations simply use many subscriptions as a way to put boundaries around cloud services for departments, teams or other reasons.

Microsoft has built a new service in Azure to help with the governance of your cloud. This new service is called Management Groups. Management Groups is still in preview but it is something I highly recommend you start trying out or using now as it is going to be as big for cloud as group policy was for on-premises AD based environments.

Management Groups sit above subscriptions. This allows Management Groups to be at the highest level in the chain so they can be used to effectively manage access, policies, and compliance for any subscriptions that belong to your organization. Within Management Groups you can set access controls (RBAC) and Azure policy to be applied to subscriptions. Subscriptions are organized in logical containers and the containers are the “management groups”. Your governance conditions are then applied to the management groups. This is the much-needed enterprise level type of management that has been needed in Azure for a while.

Management Groups will eventually become the starting point of governance when organizations embark on the cloud. Management Groups also can be used for organizations that are already in the cloud. I am going to dive into Management Groups giving you a high-level tour but first I need to give some more background on the components of Management Groups.

Each directory has a “root management group”. This root management group is at the top level of the management group hierarchy. All other management groups and subscriptions fold up to the root management group. Access and policies can be applied at the directory level via this root management group.

A couple of other things to note about management groups are that you can only have up to 10,000 management groups in a single directory, a management group tree can go six levels deep not including the root management group, and each management group can have multiple children management groups but only one parent management group.

Now let’s explore how I have structured my management groups to give some examples of how this works. Note that all the examples I show in this blog post are for my Azure environments but yours will be different based on many factors such as your organizational structure of departments, teams, etc.

You can find management groups under All Services>>Management Groups.

When you first access Management Groups you will need to create a root MG. Note that the root MG cant deleted or moved. You can rename the root MG. In the following screenshot, I am showing the creation of a sub MG in my root MG. Also, notice on the left-hand side you can set Access controls (RBAC) on this MG.

In order to configure Azure Policies and apply it to a management group, you do that within the Azure Policy itself. You can see in the following screenshot that I have an Azure policy and I am scoping it to the Prod01 MG. Whatever subscription/s and resources in those subscriptions will inherit the policy unless an exclusion is set in the policy or I am breaking inheritance at the resource group level.

In the following screenshot, I am showing the addition of an existing resource. The resources you can add are other MG’s or subscriptions.

In the following screenshot, you can see that I am going to add one of my subscriptions to my Dev01 management group. After doing this I can configure development related access and development related policies to this subscription. I also can do the same thing with my production environments/subscriptions.

Here is what my Management Groups hierarchy looks like:

In my hierarchy I have 3 subscriptions I split into two for production and 1 for development. I have created a root management group and placed all other management groups in it. I created a parent management group for my prod subscriptions and 1 for my development subscriptions in case I add more in the future. I then created a prod01 and prod02 pulling a subscription into each one. Doing this allows me to have separate access and policies per subscription. One thing you could do is pull multiple subscriptions into a single management group.

Note that I also could apply access and policies at the root level or at one of my environment management groups i.e. Prod_Env/Dev_Env and the sub-management groups would inherit the access and policies that are set at the environment management group level.

Also if you need to you can move management groups to a new parent management groups.

Thanks for reading this post. As I mentioned at the beginning of this post Azure Management Groups are currently in preview but they are worth checking out and potentially using now as these are going to become a critical part of the Azure governance story.

Read more

Azure Mobile App

by

Microsoft has a mobile Azure mobile app for Android and IOS. At first I was skeptical about the need of a mobile app for cloud but I found myself actually using it a few times for various tasks that I did not want to log onto my computer to do. In this blog post I am going to give one example. Before I jump into the example let’s explore the app.

First off you can load the app from Itunes or Google Play. You also can check it out here: https://azure.microsoft.com/en-us/features/azure-portal/mobile-app/ and here: https://play.google.com/store/apps/details?id=com.microsoft.azure&hl=en.

With the app in general you can see your Azure resources, their metrics, their health along with alerts, and diagnose and fix some issues through some actions you can perform on the resources via the mobile app. Some of the actions you can perform are Restart a web app or connect to a VM. Something else you can do with the app is access the Azure cloud shell. It supports Bash and PowerShell.  The following are some screenshots from the app.

Here is the app on my Android:

After the app launches for the first time you will be prompted to log into your subscription. Once you are logged in you will see all of your resources.

You can actually click on the filter icon to scope down to a specific type of resources.

The last screenshot here is of the Azure cloud shell in the mobile app.

Now lets talk about one reason you may use the app. I host an Azure user group website on WordPress on Azure. I have an availability monitor in Application Insights monitoring the site. If the site goes down I get an email from Application Insights as shown in the following screenshot.

I also get a notification in the UG board Slack channel by Logic Apps if the site is down. Well one day I got the notification from Slack on my phone.

I was not at my computer and did not want to go to it just to see what was going on with the site. I checked and sure enough the site was down.

Instead of logging onto my computer to troubleshoot I just used the app on my phone. Logging in I was able to see the site was up.

After clicking on the web app I was able to quickly restart it. It was up after that and I did it all from my phone.

I know restarting a web app is a basic thing. It saves time not having to log all the way into a computer to do this. I recommend trying out the mobile app. You never know when it might come in handy for a quick way to get info about one of your Azure resources and even help you troubleshoot something.

Read more