Cake Frosting - cancelling build within Azure

[tapika]

Prerequisites

• I have written a descriptive issue title
• I have searched issues to ensure it has not already been reported

Cake runner

Cake Frosting

Cake version

2.0.0

Operating system

Windows

Operating system architecture

64-Bit

CI Server

No response

What are you seeing?

I have made by own custom launcher of devenv.com - to be able to build solution.

It works as expected - it launches Visual studio's devenv.com and compiles entire solution.

However - during compilation we have some amount of C++ projects, which are compiling as in parallel and eating a lot of CPU.

When C++ compilation is in progress - and you try to cancel build job from azure - cakebuilder is cancelled, but devenv.com remains running.

What I see from log files - cakebuilder receives cancel signal in form of Ctrl+C or Ctrl+Break (seeing from logs), but result is that cakebuild is cancelled / killed / terminated, but devenv.com remains running.

I think this has something to do with C++ parallel compilation, as 8 CPU's are working on compiling C++, and maybe because of that cakebuilder does not gets execution priority to actually cancel devenv.com. I have tested this locally and manually pressed Ctrl+C - everything works correctly. Maybe on Azure's there is faster timeout happening, and because of that azure decides to kill cakebuilder without terminating it's child compilation processes.

This could be also classified as bug of Azure itself, as from my perspective it should also kill child processes, not only main builder process.

For now I have coded somehow similar killbuildtasks.ps1:

param ( 
    [string]$categories = "build,test",

    [Alias('v')]
    [switch]$verbose=$false
)

$procs2kill = New-Object System.Collections.Generic.List[System.Object]

foreach( $category in $categories.Split(",;".ToCharArray()) )
{
    if($category -eq 'build')
    {
        $procs2kill.AddRange( 
            @(
                "devenv.com",
                # launched by devenv.com
                "devenv.exe",
                "msbuild.exe",
                "cmake.exe",
                "ninja.exe",
                "cl.exe",
                "link.exe",
                "vcpkg.exe",
                "mspdbsrv.exe"
            )
        )
    }

    if($category -eq 'test')
    {
        $procs2kill.AddRange( 
            @(
                "vstest.console.exe",
                "testhost.x86.exe",
                "testhost.exe",
                "testhost.net472.exe"
            )
        )
    }
}

# $procs2kill.Add('notepad.exe')

if($verbose -eq $true)
{
    "Searching following processes:"
    $procs2kill
}

Get-Process | ForEach-Object {

    if([string]::IsNullOrEmpty($_.Path))
    {
        # Ignore where we cannot get path
        return
    }

    $file = [System.IO.Path]::GetFileName($_.Path)
    if( $procs2kill.Contains($file) )
    {
        # Don't kill Visual studio instances, but if building solution
        if( $file -eq 'devenv.exe')
        {
            $query = "processid = '" + $_.Id + "'"
            $process = Get-CimInstance Win32_Process -Filter $query | Select-Object CommandLine
            $cmdLine = $process.CommandLine
            if(-not $cmdLine.Contains('/build') )
            {
                return
            }
        }

        "Killing $file (" + $_.Id + ")"

        # Does not always works
        #Stop-Process -Id $_.Id -Force

        $cmd = "cmd.exe"
        $cmdArgs = @( '/c', 'wmic', 'process', 'where', ('processid="' + $_.Id + '"'), 'call', 'terminate', '>', 'nul')

        #$cmdArgs
        & $cmd $cmdArgs
    }
}

and added into yaml something like this:

    -   script: |
            @echo on
            powershell -executionpolicy bypass -file killbuildtasks.ps1
        displayName: Kill build tasks
        condition: always()

But this feels like a crack solution.

Maybe someone can propose a better solution on this one ?

One approach could be:

• Put cake builder itself to higher priority over child processes, so it can process cancel signal faster, and would not end up into process termination.

What is expected?

All child compilation tasks gets terminated before terminating main frosten cake builder task.

Steps to Reproduce

Really difficult to reproduce - need to have heavy c++ project with cakebuilder adapted to it.
Need to run cake builder task in azure.
Perform cancel in Azure while compilation is up and running.

Cannot provide my own build environment due to privacy of the project.

Output log

No response