Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support multiple certificate and different number of empty lines for netsh #1153

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

yepeekai
Copy link

Support multiple certificate and different number of empty lines for netsh. fix #919

…t of netsh. Also support if there is more than one certificate. fix microsoft#919
@yepeekai
Copy link
Author

@microsoft-github-policy-service agree

Copy link

@coolhome coolhome left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

100% confirmed this is my issue. Great fix!

image

@coolhome
Copy link

coolhome commented May 22, 2023

@LeftTwixWand @starkmsu @manolerazvan

👋 I hate to bug you guys, but can we merge this crucial fix? Depending on your version of Windows it will cause IIS deployment issues due to different netsh outputs based on its version.

Windows Server 2012 - netsh.exe @ 6.3.9600.17415 works with this extension ✔️
Windows Server 2022 - netsh.exe @ 10.0.20348.1 does not work until this is merged. ⛔

Thank you!

@coolhome
Copy link

Related Issues:
Oct 20, 2021 - microsoft/azure-pipelines-tasks#15419
Nov 25, 2021 - #1008
Apr 22, 2022 - #1039
Jan 10, 2022 - microsoft/azure-pipelines-tasks#15686
May 20, 2022 - #1058
Dec 12, 2022 - microsoft/azure-pipelines-tasks#17462

Duplicated Pull Requests:
Oct 27, 2021 - #993
Mar 13, 2023 - #1134

@LeftTwixWand
Copy link
Contributor

LeftTwixWand commented May 23, 2023

Hello Preston and @yepeekai!

Thank you so much for your efforts, it's great to see when community takes over some issues.
Unfortunately, I don't have much experience with TaskModules, so I need some to check how it works.
Also, I don't know about the versioning in TaskModules, so I have to clarify it too.

I'll keep you up to date😊

@LeftTwixWand
Copy link
Contributor

Hi @coolhome and @yepeekai👋

Generally, PR looks good, I will just add a one more test here: TaskModules/powershell/Tests to cover the new functionality.
It shouldn't take to long.

@coolhome
Copy link

I just noticed this line will need to be updated to find by key to get the certificate hash -

$isItSameCert = $result.Get(5).ToLower().Contains($certhash.ToLower())

The same fixes also need to be applied for IISWebAppDeployment - https://github.com/microsoft/azure-pipelines-tasks/blob/065dfd9c37ff3e0bae7bd385bfa523fc17f98407/Tasks/IISWebAppDeployment/MsDeployOnTargetMachines.ps1#L277-L298

@yepeekai
Copy link
Author

yepeekai commented Jul 10, 2023

I committed a change to validate the certificate hash. I also changed the binding search to ensure there is no false positive. For example if we search for port 443 and there is a certificate for 443 and 44311, both would have matched in the previous version and would have cause problem.

@LeftTwixWand
Copy link
Contributor

Hi @yepeekai @coolhome

I tested the refactored method, and everything looks good.
The only one concern, which I see I'm not sure, is that this is the correct place for the fix.
Because we have TaskModules/powershell, where you made a fix. And I'm not sure that the task is not using this module.
Because the tasks have their own AppCmdOnTargetMachines.ps1 scripts.

I'll try to test these changes in the real environment.

@CJohannimloh
Copy link

Would be great to get this MR merged in the near future...We are running into this issue aswell

@coolhome
Copy link

@LeftTwixWand I can confirm this is the wrong file modified! 🤦 I wonder what the history is for why IIS Tasks didn't inherit from this other AppCmdOnTargetMachines.ps1. I'm a big fan of this the new functions introduced in this PR.

Additional related PRs that can likely be closed once this is merged:
#1134
https://github.com/microsoft/azure-pipelines-extensions/pull/993/files

@yepeekai
Copy link
Author

That's sad. Maybe still begin with this PR and then replicate the changes over to the other place

@LeftTwixWand
Copy link
Contributor

Hey @yepeekai @coolhome @CJohannimloh I know that this issue is taken buy few colleagues from my team. Let me ask them about the progress.

@LeftTwixWand
Copy link
Contributor

Hey @yepeekai @coolhome @CJohannimloh can some of you provide the next information and I'll try to complete this PR ASAP:

  • Extension name
  • Affected tasks names
  • Describe how I can set up the environment to test the changes, like this:
    -- Create a hosted agent
    -- Preinstall something on agent
    -- Run tasks with such parameters.

This would help me a lot. Because I still have other high priority work to do.
But I really want to unblock you, and this would extremely speed up the process.

@LeftTwixWand LeftTwixWand self-assigned this Oct 17, 2023
@CJohannimloh
Copy link

Hey @LeftTwixWand,

sure i can give some information.

How to reproduce:

  • Create a hosted Server 2022 Agent

  • Install Basic IIS (Powershell)
    Install-WindowsFeature Web-Server,Web-WebServer,Web-Common-Http,Web-Default-Doc,Web-Dir-Browsing,Web-Http-Errors,Web-Static-Content,Web-Health,Web-Http-Logging,Web-Performance,Web-Stat-Compression,Web-Security,Web-Filtering,Web-Basic-Auth,Web-Windows-Auth,Web-App-Dev,Web-Net-Ext45,Web-ASP,Web-Asp-Net45,Web-ISAPI-Ext,Web-ISAPI-Filter,Web-Mgmt-Tools,Web-Mgmt-Console,Web-Mgmt-Service,NET-Framework-45-Features,NET-Framework-45-Core,NET-Framework-45-ASPNET,NET-WCF-Services45,NET-WCF-HTTP-Activation45,NET-WCF-TCP-PortSharing45,WAS,WAS-Process-Model,WAS-Config-APIs

  • Run this Task twice (requires a SSL Thumbprint)

  • First one should succeed and then it fails every time because BindingConfiguration can't run

          - task: IISWebAppManagementOnMachineGroup@0
            inputs:
                IISDeploymentType: 'IISWebsite'
                ActionIISWebsite: 'CreateOrUpdateWebsite'
                WebsiteName: 'testwebsite'
                WebsitePhysicalPath: '%SystemDrive%\inetpub\wwwroot\testwebsite'
                WebsitePhysicalPathAuth: 'WebsiteUserPassThrough'
                CreateOrUpdateAppPoolForWebsite: true
                AppPoolNameForWebsite: 'testapppoool'
                DotNetVersionForWebsite: 'No Managed Code'
                ConfigureAuthenticationForWebsite: true
                AnonymousAuthenticationForWebsite: true
                AddBinding: true
                Bindings: |
                    {
                        bindings:[
                            {
                                "protocol":"http",
                                "ipAddress":"*",
                                "hostname":"",
                                "port":"80",
                                "sslThumbprint":"",
                                "sniFlag":false
                            },
                            {
                                "protocol":"https",
                                "ipAddress":"*",
                                "hostname":"",
                                "port":"443",
                                "sslThumbprint":"$(Thumbprint)",
                                "sniFlag":false
                            }
                        ]
                    }

@LeftTwixWand
Copy link
Contributor

Hey @CJohannimloh thank you so much for your help. We're already working on it.

@coolhome
Copy link

coolhome commented Oct 18, 2023

@LeftTwixWand in addition this is the area we forget.

I'm using the same as above, but you need to enter in the remote host information. The one above was for Deployment Groups where this is using WinRM to deploy.

https://github.com/microsoft/azure-pipelines-extensions/tree/master/Extensions/IISWebAppDeploy/Src/Tasks/IISWebAppMgmt/IISWebAppMgmtV3

- task: IISWebAppMgmt@3
  inputs:
    machinesList: '$(RemoteHostname)'
    AdminUserName: '$(RemoteUser)'
    AdminPassword: '$(RemotePass)'
    WinRMProtocol: Http
    WebsiteName: 'testwebsite'
    WebsitePhysicalPath: '%SystemDrive%\inetpub\wwwroot\testwebsite'
    AddBinding: true
    CreateOrUpdateAppPoolForWebsite: true
    ConfigureAuthenticationForWebsite: true
    Protocol: https
    Port: 443
    ServerNameIndication: false
    SSLCertThumbPrint: '$(Thumbprint)'
    AppPoolNameForWebsite: 'testapppoool'
    AnonymousAuthenticationForWebsite: true
    WindowsAuthenticationForWebsite: false

@LeftTwixWand
Copy link
Contributor

LeftTwixWand commented Nov 13, 2023

Hey @yepeekai and @coolhome may I ask a question about this code changes.
Why they've done in the TaskModules/powershell/TaskModuleIISManageUtility/AppCmdOnTargetMachines.ps1?
I don't see that this module been used in IIS Web App Deployment Using WinRM.
Did I miss something?

Because when I go to the task folder on the build agent I can see this:
image

But this file is part of the task code, no the TaskModules:
image

@LeftTwixWand LeftTwixWand added enhancement Area: IISWebAppDeploy Changes related to IISWebAppDeploy extension labels Nov 13, 2023
@yepeekai
Copy link
Author

My understanding of azure tasks stuff is limited... LeftTwixWand made a comment in that regard on july 26th if you go up in the history of this PR. It in fact appears that I did modify at the wrong place, this peace of code is duplicated and I choose the wrong one when I searched for specific patterns. On my side I am not able to test azure directly with this, so I copied the part that I though was used and executed it on my machine (and a dev server we have). I suggested to correct it here and then port the solution to the other place.

@LeftTwixWand
Copy link
Contributor

@yepeekai thank you so much for this response.
I was also confused, because AppCmdOnTargetMachines exist in multiple places.
So, I thought we were missing something.
Looks like TaskModules was used in the past, but now each task has its own AppCmdOnTargetMachines.

Anyway, thank you for this pr, I'm sure it will be merged soon.
Just a few more tests needed.

@jonascarlbaum
Copy link

jonascarlbaum commented Feb 1, 2024

Hey @LeftTwixWand,

sure i can give some information.

How to reproduce:

  • Create a hosted Server 2022 Agent
  • Install Basic IIS (Powershell)
    Install-WindowsFeature Web-Server,Web-WebServer,Web-Common-Http,Web-Default-Doc,Web-Dir-Browsing,Web-Http-Errors,Web-Static-Content,Web-Health,Web-Http-Logging,Web-Performance,Web-Stat-Compression,Web-Security,Web-Filtering,Web-Basic-Auth,Web-Windows-Auth,Web-App-Dev,Web-Net-Ext45,Web-ASP,Web-Asp-Net45,Web-ISAPI-Ext,Web-ISAPI-Filter,Web-Mgmt-Tools,Web-Mgmt-Console,Web-Mgmt-Service,NET-Framework-45-Features,NET-Framework-45-Core,NET-Framework-45-ASPNET,NET-WCF-Services45,NET-WCF-HTTP-Activation45,NET-WCF-TCP-PortSharing45,WAS,WAS-Process-Model,WAS-Config-APIs
  • Run this Task twice (requires a SSL Thumbprint)
  • First one should succeed and then it fails every time because BindingConfiguration can't run
          - task: IISWebAppManagementOnMachineGroup@0
            inputs:
                IISDeploymentType: 'IISWebsite'
                ActionIISWebsite: 'CreateOrUpdateWebsite'
                WebsiteName: 'testwebsite'
                WebsitePhysicalPath: '%SystemDrive%\inetpub\wwwroot\testwebsite'
                WebsitePhysicalPathAuth: 'WebsiteUserPassThrough'
                CreateOrUpdateAppPoolForWebsite: true
                AppPoolNameForWebsite: 'testapppoool'
                DotNetVersionForWebsite: 'No Managed Code'
                ConfigureAuthenticationForWebsite: true
                AnonymousAuthenticationForWebsite: true
                AddBinding: true
                Bindings: |
                    {
                        bindings:[
                            {
                                "protocol":"http",
                                "ipAddress":"*",
                                "hostname":"",
                                "port":"80",
                                "sslThumbprint":"",
                                "sniFlag":false
                            },
                            {
                                "protocol":"https",
                                "ipAddress":"*",
                                "hostname":"",
                                "port":"443",
                                "sslThumbprint":"$(Thumbprint)",
                                "sniFlag":false
                            }
                        ]
                    }

I have somewhat identical setup, one http and one https with "sniFlag": true. Both with a specified hostname, otherwise pretty much the same.

The first run everything is setup just fine, all following runs results in.

SSL Certificate add failed, Error: 183
Cannot create a file when that file already exists.

##[error]Process 'netsh' exited with code '1'.

Is this because of the same issues mentioned in this thread?

I have something more like

- task: IISWebAppManagementOnMachineGroup@0
    displayName: 'Create: Website & AppPool'
    inputs:
        IISDeploymentType: 'IISWebsite'
        ActionIISWebsite: 'CreateOrUpdateWebsite'
        ActionIISApplicationPool: 'CreateOrUpdateAppPool'
        WebsiteName: '$(iis.siteName)'
        WebsitePhysicalPath: '$(iis.wwwRootPath)\${{ parameters.versionedFolderName }}\' 
        WebsitePhysicalPathAuth: 'WebsiteUserPassThrough'
        CreateOrUpdateAppPoolForWebsite: true
        AppPoolNameForWebsite: '$(iis.appPoolName)'
        DotNetVersionForWebsite: 'No Managed Code'
        PipeLineModeForWebsite: 'Integrated'
        AppPoolIdentityForWebsite: '${{ parameters.appPoolIdentity }}'
        AppPoolUsernameForWebsite: '$(iis.appPoolUsername)'
        AppPoolPasswordForWebsite: '$(iis.appPoolPassword)'
        AddBinding: true
        Bindings:|
            {   
              bindings: [
                {
                  "protocol":"http",
                  "ipAddress":"*",
                  "hostname":"thedomainname.com",
                  "port":"80",
                  "sslThumbprint":"",
                  "sniFlag":false
                },
                {
                  "protocol":"http",
                  "ipAddress":"*",
                  "hostname":"localhost",
                  "port":"80",
                  "sslThumbprint":"",
                  "sniFlag":false
                },
                {
                  "protocol":"https",
                  "ipAddress":"*",
                  "hostname":"thedomainname.com",
                  "port":"443",
                  "sslThumbprint":"$(iis.sslThumbprint)",
                  "sniFlag":true
                }
              ]
            }

@scsloan
Copy link

scsloan commented Feb 8, 2024

Running into the same issue with the same setup as @jonascarlbaum First time always works, second time get the error

SSL Certificate add failed, Error: 183
Cannot create a file when that file already exists.

##[error]Process 'netsh' exited with code '1'.

Any update on this @LeftTwixWand

@jonascarlbaum
Copy link

Yep, this is really bad @scsloan. Did a workaround, can’t copy on my phone right now, but can screenshot the outline of the task I run prior this one now.

The missing part on bottom line is =thedomainname.com:443’

image

@jonascarlbaum
Copy link

Yep, this is really bad @scsloan. Did a workaround, can’t copy on my phone right now, but can screenshot the outline of the task I run prior this one now.

The missing part on bottom line is =thedomainname.com:443’

image

I feel this workaround is not a valid solution in the long run…

@scsloan
Copy link

scsloan commented Feb 9, 2024

Thanks for sharing your workaround @jonascarlbaum.

@jonascarlbaum
Copy link

Thanks for sharing your workaround @jonascarlbaum.

No problems @scsloan!

But I would love a permanent fix!

@LeftTwixWand is these issues going to be addressed and fixed any time soon?

@krilbe
Copy link

krilbe commented Feb 15, 2024

Why hasn't this PR been merged? I'm having this problem and would really appreciate to have it fixed.

@jonascarlbaum
Copy link

Why hasn't this PR been merged? I'm having this problem and would really appreciate to have it fixed.

Think the fix isn’t in correct files. But I had hopes someone had solves this after such long time… Since the issue is easily reproducable it would be something that just should be solved. More and more people will get trouble using these tasks…

@krilbe
Copy link

krilbe commented Feb 16, 2024

Why hasn't this PR been merged? I'm having this problem and would really appreciate to have it fixed.

Think the fix isn’t in correct files. But I had hopes someone had solves this after such long time… Since the issue is easily reproducable it would be something that just should be solved. More and more people will get trouble using these tasks…

Yes. Frankly, how do we request priority on these issues? The task seems to have been abandoned 5 years ago - no new commits for five years!? But as far as I know the deployment group/agent approach for DevOps release pipelines hasn't been deprecated, so the task needs to keep up.

@colhouse-hakh
Copy link

@coolhome @LeftTwixWand any update on this?
We are waiting to move from TeamCity/Octopus to AzurePipelines and this is a major roadblock for us as we would like to be able to target Windows Server 2022 for our deployments.

@CJohannimloh
Copy link

Any updates on this topic?

@colhouse-hakh
Copy link

@CJohannimloh I haven't heard anything back. So instead we are looking at just disable Add Bindings as part of this task and instead use Powershell script to add bindings.

@CJohannimloh
Copy link

@colhouse-hakh: Ye sure. We are doing the same but still want a better solution

@v-schhabra
Copy link

Hi @colhouse-hakh @CJohannimloh,
Can anyone of you please share the debugged logs of the pipeline?
It would be beneficial for us to know the issue in more detail.

@CJohannimloh
Copy link

Hi @colhouse-hakh @CJohannimloh, Can anyone of you please share the debugged logs of the pipeline? It would be beneficial for us to know the issue in more detail.

What exactly do you need?
I guessed all the informations are available in the PR

@v-schhabra
Copy link

Hi @colhouse-hakh @CJohannimloh, Can anyone of you please share the debugged logs of the pipeline? It would be beneficial for us to know the issue in more detail.

What exactly do you need? I guessed all the informations are available in the PR

We need pipeline logs as similar fix we did for the issue but still many of you are still having some issues.
#1220

@jonascarlbaum
Copy link

Of course we can’t mess with current production environments. It’s easier doing such things during the time you build the pipeline.

But the only relevant part of the logs seems still being the part

SSL Certificate add failed, Error: 183
Cannot create a file when that file already exists.

##[error]Process 'netsh' exited with code '1'.

@colhouse-hakh
Copy link

colhouse-hakh commented May 2, 2024

@v-schhabra
Here are the logs for a deployment that failed

2024-05-01T10:26:00.6238324Z ##[section]Starting: Create/Update Web Site
2024-05-01T10:26:01.1665499Z ==============================================================================
2024-05-01T10:26:01.1667818Z Task : IIS web app manage
2024-05-01T10:26:01.1668987Z Description : Create or update websites, web apps, virtual directories, or application pools
2024-05-01T10:26:01.1669561Z Version : 0.238.0
2024-05-01T10:26:01.1673061Z Author : Microsoft Corporation
2024-05-01T10:26:01.1674335Z Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/deploy/iis-web-app-management-on-machine-group
2024-05-01T10:26:01.1675215Z ==============================================================================
2024-05-01T10:26:19.3145903Z ##[command]"C:\Windows\system32\inetsrv\appcmd.exe" list apppool /name:"WebSite"
2024-05-01T10:26:20.4391239Z ##[command]"C:\Windows\system32\inetsrv\appcmd.exe" add apppool /name:"WebSite"
2024-05-01T10:26:20.7692289Z APPPOOL object "WebSite" added
2024-05-01T10:26:20.9568474Z ##[command]"C:\Windows\system32\inetsrv\appcmd.exe" set apppool /apppool.name:"WebSite" -managedRuntimeVersion:v4.0 -managedPipelineMode:Integrated -processModel.identityType:SpecificUser -processModel.userName:"username" -processModel.password:""
2024-05-01T10:26:21.3089032Z APPPOOL object "WebSite" changed
2024-05-01T10:26:21.3641320Z ##[command]"C:\Windows\system32\inetsrv\appcmd.exe" list site /name:"WebSite"
2024-05-01T10:26:21.6888764Z ##[command]"C:\Windows\system32\inetsrv\appcmd.exe" add site /name:"WebSite" /physicalPath:"D:\IISSites\WebSite"
2024-05-01T10:26:22.0018947Z SITE object "WebSite" added
2024-05-01T10:26:22.0019766Z APP object "WebSite/" added
2024-05-01T10:26:22.0022428Z VDIR object "WebSite/" added
2024-05-01T10:26:22.2290692Z
2024-05-01T10:26:22.2439282Z
2024-05-01T10:26:22.2746937Z Directory: D:\IISSites
2024-05-01T10:26:22.2747951Z
2024-05-01T10:26:22.2750072Z
2024-05-01T10:26:22.3004034Z Mode LastWriteTime Length Name
2024-05-01T10:26:22.3034457Z ---- ------------- ------ ----
2024-05-01T10:26:22.3099087Z d----- 5/1/2024 8:26 PM WebSite
2024-05-01T10:26:22.3672388Z ##[command]"C:\Windows\system32\inetsrv\appcmd.exe" set site /site.name:"WebSite" -applicationDefaults.applicationPool:"WebSite" -[path='/'].[path='/'].physicalPath:"D:\IISSites\WebSite" -[path='/'].[path='/'].userName: -[path='/'].[path='/'].password:
2024-05-01T10:26:22.7064599Z SITE object "WebSite" changed
2024-05-01T10:26:22.8631466Z ##[command]"C:\Windows\system32\inetsrv\appcmd.exe" list sites
2024-05-01T10:26:23.2581176Z ##[command]"C:\Windows\system32\inetsrv\appcmd.exe" set site /site.name:"WebSite" /+bindings.[protocol='https',bindingInformation='
:443:website.xxxxxx.com.au']
2024-05-01T10:26:23.6242453Z SITE object "WebSite" changed
2024-05-01T10:26:23.7042966Z ##[command]"netsh" http show sslcert ipport=0.0.0.0:443
2024-05-01T10:26:24.0984427Z ##[command]"netsh" http add sslcert ipport=0.0.0.0:443 certhash=
* appid={----} certstorename=MY
2024-05-01T10:26:24.5099595Z
2024-05-01T10:26:24.5100361Z SSL Certificate add failed, Error: 183
2024-05-01T10:26:24.5110360Z Cannot create a file when that file already exists.
2024-05-01T10:26:24.5110801Z
2024-05-01T10:26:24.5110975Z
2024-05-01T10:26:25.2302820Z ##[error]Process 'netsh' exited with code '1'.
2024-05-01T10:26:25.5587658Z ##[section]Finishing: Create/Update WebSite Site

The error is described in more details in issue #1008 has all the details you require for this ticket. This ticket was also mentioned by @coolhome in this PR on 23rd May 2023.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: IISWebAppDeploy Changes related to IISWebAppDeploy extension enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SSL Certificate add failed, Error: 183 Cannot create a file when that file already exists
9 participants