Add support for reading XML files

[ronaldbosma]

I would like to be able to read XML files so I can create custom rules to check the XML. In particular, I want to check my Azure API Management policies. We define our policies in separate XML files, so I would like to be able to load these and check them with PSRule.

An extra XML option for Input.Format would most likely be the best approach. Handling both elements & attributes might be a bit tricky, but both should be supported. Also note that elements with the same name can occur multiple times, while attribute names should be unique for a given element.

Alternatives I looked at:

The PSRule.Rules.Azure suite provides the Azure.APIM.PolicyBase rule that asserts the contents of a policy. This rule expects the policy XML to be defined directly in the Bicep file. We use separate XML files for our policies because these are more readable and maintainable. So, the approach used for Azure.APIM.PolicyBase doesn't work for us.

I've tried converting the Bicep into an JSON ARM template and then using a similar approach as is done by Azure.APIM.PolicyBase, but the policy contents is set via a generated variable as you can see in the sample below. Most likely because we use the loadTextContent Bicep function when loading the policy XML file.

"variables": {
  "$fxv#1": "<policies>\r\n   ... \r\n</policies>"
},
"resources": [
  {
    "type": "Microsoft.ApiManagement/service/apis/policies",
    "apiVersion": "2022-08-01",
    "name": "[format('{0}/{1}/{2}', parameters('apimResourceName'), parameters('apiName'), 'policy')]",
    "properties": {
      "format": "rawxml",
      "value": "[variables('$fxv#1')]"
    },
    "dependsOn": [
      "[resourceId('Microsoft.ApiManagement/service/apis', parameters('apimResourceName'), parameters('apiName'))]"
    ]
  },

[github-actions]

Thanks for raising your first issue, the team appreciates the time you have taken 😉

[BernieWhite]

@ronaldbosma Thanks for raising the issue. We'd like to improve XML support too since we found a few things that could be better such as #1518.

Currently you can use PowerShell based rules to work with XML by using a convention. Similar to this https://github.com/microsoft/PSRule-samples/tree/main/samples/azure/BicepModuleRequires

For example:

# Synopsis: Imports XML policy in for analysis.
Export-PSRuleConvention 'APIMPolicy.Import' -Initialize {
    $policies = @(Get-ChildItem -Path 'policies/' -Include '*.xml' -Recurse -File | ForEach-Object {
        $name = $_.Name
        [PSCustomObject]@{
            Name = $name
            Content = [Xml](Get-Content -Path $_.FullName -Raw)
        }
    })
    $PSRule.ImportWithType('APIMPolicy', $policies);
}

# Synopsis: Your rule
Rule 'Rule001' -Type 'APIMPolicy' {
    $policy = $TargetObject.Content
}

See conventions

[ronaldbosma]

Hi @BernieWhite. Thanks for answering and for your suggestion to use a convention. I was working a workaround which had several issues, but using a convention that includes the file name and importing them as custom types fixed all of them.

[ronaldbosma]

Hi @BernieWhite I was wondering if the above still applies for v3. Do I still need to use a custom convention with PowerShell based rules to check XML?

[BernieWhite]

@ronaldbosma The current plan for this is v3.1.0, as shipping the breaking changes we need to move forward is the main priority for v3.0, which is very close now. So yes, at GA drop v3.0 will still require the convention.

If you are trying with the current v3 prerelease you may need to set Input.FileObjects to true if there is no files found. This will be defaulted to true in the next prerelease.

See: https://microsoft.github.io/PSRule/v3/concepts/PSRule/en-US/about_PSRule_Options/#inputfileobjects

[sterankin]

I am trying to follow this thread - but does PSRule actually validate the APIM policy syntax is correct?

I am trying to find a way to check the syntax of the entire apim policy (and replicate the experience in the portal when you have a syntax error).

For example, in a policy expression which is the C sharp bases syntax, its easy to add an extra comma or bracket which can make the syntax invalid. The portal will show an error when you try to save the policy, e.g.

Error in element 'set-variable' on line 16, column 10: Invalid expression term ')'

Is there anyway to replicate this experience in CICD with PSRule or some other cli tool? Microsoft are obviously using something behind the scenes to check the syntax is valid.

Since all our policies are in source control and deployed using Bicep, I was hoping we could check this kind of issue as part of our CI builds.

[ronaldbosma]

Hi @sterankin,

That wasn’t the goal of my request. My aim was to create custom rules that ensure the standards and guidelines we’ve defined for our policies are being followed. I recently wrote a blog post on this topic: Validate API Management Policies with PSRule.

The solution in the post checks that the XML is valid. However, it won’t raise an error if you use an unknown policy or if your C# code is invalid. You could create your own custom rule(s) to handle those cases, but that might be a lot of work.

[sterankin]

@ronaldbosma Thanks for the link to your post, I was having a read of your blog and your articles are excellent and very helpful. I have sent them to our team as there is a lot we can learn and implement. I didn't know we could use a .cshtml file extension for our policy files.

I am surprised MS do not have something that validates an APIM policy. For example, using the Azure portal, if I try to add an extra comma or bracket to the policy expression code - it won't let me save the file and will give me an error. I would love to be able to do this in a CI build - I wonder what they use in the background to check the syntax of the policy.

[ronaldbosma]

@sterankin you could download the Self-hosted gateway docker container, extract the API Management assemblies from the container and have a look with ILSpy. I've done this for another issue I had, but I'm not sure if you can find the code you're looking for.