Native single-binary with .Net Core 3.0 and Azure Pipeline

At G DATA we track and fight cybercrime in different areas. One of them is command-and-control (CC) server infrastructure and other malware related infrastructure. We share our data with partners and customers (paid service). To enable our customers to fetch our newest CCs, we provide a HTTPS API. Internally we have a command line tool, called CCGetter, written in C# to pull the current list of CCs from that HTTPS API. Recently we wanted to share the command line tool on Github as an example for interaction with our HTTPS API. However, during this process we ran into some problems, which I will show how to solve in this article.

The command line tool CCGetter is written in C# and targets .NET Core 3.0 preview. This enables us to serve internal Windows and Linux customers at once, because .NET Core is cross-platform per default. There are two major ways to achieve cross-platform run-ability of a .NET Core app which we will look at.

The first way is to compile the executable as shown below, which creates a folder with the executable and all needed dependencies (DLLs and config files).

dotnet build -c Release

In the case of CCGetter, 11 files are created with a total size of 1.21 MB. The executable will run on Windows, Linux and MacOS as long as the needed version of the .NET Core runtime is installed. The last part is a problem for us. It is very unlikely that our users have the required .NET Core runtime installed on their system and therefore the tool won’t run.

The second option is to publish a self-contained executable. In this case the runtime needed to execute the tool is included, but we lose the cross-platform capability. This means we need to publish explicitly for every platform we want to support. In our case that’s Windows and Linux. Running the commands below creates a stand-alone command line tool with all dependencies in a folder.

dotnet publish -c Release -r win-x64
dotnet publish -c Release -r linux-x64

If we copy this folder to any Windows or Linux computer, it will run without the need for any installed dependencies. This solution has a drawback, though: the whole runtime environment is now a dependency which we have to distribute. In the case of Windows that means 225 files with a total size of 65.7 MB and 193 files for Linux with a total size of 73.4 MB. Would you like a command line tool consisting of roughly 200 files? Definitely not!
Luckily, there is a third, not so well-known option.

Besides the .NET Core runtime which uses a just-in-time (JIT) compiler, there exists an ahead-of-time (AOT) compiler which produces a native single binary for each platform. The project is an official Microsoft project called CoreRT and in an early development state but already usable for small applications.

I wrote about AOT compilation a while back in my blog, but never tried it with .NET Core 3.0 preview and with both Windows and Linux as a target - but it looks to me like that this would solve all my problems for the CCGetter tool.
To get native compilation, we need to change only two minor things in our project.

First, we add the Intermediate-Language compiler (ILCompiler) which translates IL into native code to our package references.

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp3.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="CommandLineParser" Version="2.4.3" />
    <PackageReference Include="Microsoft.DotNet.ILCompiler" Version="1.0.0-alpha-27507-02" />
    <PackageReference Include="Tiny.RestClient" Version="1.6.1" />
  </ItemGroup>

</Project>

The important addition here is the line:

<PackageReference Include="Microsoft.DotNet.ILCompiler" Version="1.0.0-alpha-27507-02" />

Because the ILCompiler package is not available on NuGet, due to its alpha status, we need to add a reference to .NET MyGet. To do so, we add a file called NuGet.config with the following content to our project root.

<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <packageSources>
        <clear />
        <add key="dotnet-core" value="https://dotnet.myget.org/F/dotnet-core/api/v3/index.json" />
        <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
    </packageSources>
</configuration>

To compile a native single-binary Windows executable we need to run the same command as in the self-contained executable. But this time we get an AOT compiled executable instead of an JIT compiled one.

dotnet publish -c Release -r win-x64

This creates an 20.6 MB single executable without any dependencies. That's exactly what we need to distribute the command line tool. Now lets compile the same for Linux.

> dotnet publish -c Release -r linux-x64
Microsoft (R) Build Engine version 16.0.443+g5775d0d6bb for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Persisting no-op dg to C:\CCGetter.csproj.nuget.dgspec.json
  Restore completed in 40,38 sec for C:\CCGetter\src\CCGetter\CCGetter.csproj.

C:\Program Files\dotnet\sdk\3.0.100-preview3-010431\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.RuntimeIdentifierInference.targets(151,5): message NETSDK1057: You are using a preview version of .NET Core. See: aka.ms/dotnet-core-preview 

C:\Users\xxx\.nuget\packages\microsoft.dotnet.ilcompiler\1.0.0-alpha-27507-02\build\Microsoft.NETCore.Native.Publish.targets(63,5): 
error : Cross-compilation is not supported yet. github.com/dotnet/corert/issues/5458

Well - that did not go as planned. We got an error that tells us that cross-compilation is not supported yet. This means we need a Linux host to compile the binary for Linux. We will solve this problem with the help of an Azure Pipeline.

Before we jump into the part where we build CI, lets compare the three methods to show the advantages of AOT compilation with .NET Core.

Azure Pipelines have a great and free GitHub integration and support CI for Windows, Linux and MacOS. What we need is the ability to compile a .NET Core 3.0 application on Windows and on Linux to a native executable and then publish both executables on the corresponding GitHub Release page. This already gives us the job structure of our build pipeline. We create one job that compiles the Windows binary on a Windows host and one job that runs in parallel and compiles the Linux binary on a Linux host. The last job depends on the output of the both previous jobs and publishes the build artifacts to the GitHub Release page. Because there is currently no build image available with .NET Core 3.0 preview installed, we have to install it manually on the hosts.

Let's start with the Windows build:

- job: Build_Windows_Binary

  pool:
    vmImage: 'windows-2019'

  steps:
    - powershell: choco install dotnetcore-sdk --pre
      displayName: Install .NET Core 3.0 preview

    - powershell:  dotnet publish -r win-x64 -c release 
      displayName: Build native Windows executable

    - powershell: cp .\src\CCGetter\bin\Release\netcoreapp3.0\win-x64\native\CCGetter.exe $(Build.ArtifactStagingDirectory)
      displayName: Copy CCGetter.exe to "Artifacts Staging Directory"

    - task: PublishBuildArtifacts@1
      displayName: Publish Windows executable
      inputs:
        pathtoPublish: '$(Build.ArtifactStagingDirectory)'
        artifactName: winExe

We run our build on a Windows 2019 server and install the latest .NET Core 3.0 preview with the Chocolatey package manager. The next step is the actual compilation of our code to a native binary. The next two tasks copy our executable in the artifact staging directory and publish the artifact, such that we can use it in another job later.

The Linux build is a bit more complicated, because we do not have a package manager available where we can install .NET Core 3.0 preview from. Instead we have to download and unpack .NET Core 3.0 manually. Furthermore a lot of dependencies are missing such that we need to install them, too.

- job: Build_Linux_Binary
  
  pool:
    vmImage: 'ubuntu-16.04'

  variables:
    dotnetUrl: 'https://download.visualstudio.microsoft.com/download/pr/35c9c95a-535e-4f00-ace0-4e1686e33c6e/b9787e68747a7e8a2cf8cc530f4b2f88/dotnet-sdk-3.0.100-preview3-010431-linux-x64.tar.gz'

  steps:
    - script: 'sudo apt install -y liblttng-ust0 libcurl3 libssl1.0.0 libkrb5-3 zlib1g libicu55 wget clang-3.9 libcurl4-openssl-dev zlib1g-dev libkrb5-dev'
      displayName: Install dependencies

    - script: 'wget $(dotnetUrl)'
      displayName: Download .NET Core 3.0 preview

    - script: 'mkdir -p $HOME/dotnet && tar zxf dotnet-sdk-3.0.100-preview3-010431-linux-x64.tar.gz -C $HOME/dotnet'
      displayName: Unpack .NET Core 3.0 preview

    - script: '$HOME/dotnet/dotnet publish -r linux-x64 -c release'
      displayName: Build native Linux executable

    - script: 'cp ./src/CCGetter/bin/Release/netcoreapp3.0/linux-x64/native/CCGetter $(Build.ArtifactStagingDirectory)'
      displayName: Copy CCGetter to "Artifacts Staging Directory"

    - task: PublishBuildArtifacts@1
      displayName: Publish Linux executable
      inputs:
        pathtoPublish: '$(Build.ArtifactStagingDirectory)'
        artifactName: linExe

The last job takes the artifacts from both build jobs and publishes them on Github.

- job: Release_To_Github
  dependsOn:
  - Build_Windows_Binary
  - Build_Linux_Binary

  pool:
    vmImage: 'windows-2019'

  steps:
    - task: DownloadBuildArtifacts@0
      displayName: Download Linux executable
      inputs:
        buildType: 'current'
        downloadType: 'single'
        artifactName: 'linExe'
        downloadPath: '$(System.ArtifactsDirectory)'
    
    - task: DownloadBuildArtifacts@0
      displayName: Download Windows executable
      inputs:
        buildType: 'current'
        downloadType: 'single'
        artifactName: 'winExe'
        downloadPath: '$(System.ArtifactsDirectory)'

    - task: GitHubRelease@0
      displayName: Create GitHub Release
      condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
      inputs:
        gitHubConnection: 'CCGetterRelease'
        repositoryName: 'GDATASoftwareAG/CCGetter'
        title: 'CCGetter Release'
        addChangeLog: false
        assets: |
          $(System.ArtifactsDirectory)\linExe\*
          $(System.ArtifactsDirectory)\winExe\*.exe

And that's it. We now have a working CI to publish native Windows and Linux .NET Core 3.0 binaries to Github.

You can find the full code and pipeline in the CCGetter GitHub repository.

We have succesfully solved our problem of publishing a command line tool for Windows and Linux to GitHub by using .NET Core 3.0 preview, CoreRT and an Azure Pipeline. There is no notable impact on the final executable due to preview and alpha tooling. I hope that this article helps CoreRT to get some traction, because it has a lot of potential and allows new use-cases for .NET Core besides the cloud.

If you are interested in command-and-control servers and malware related infrastructure, please contact us: info@remove-this.gdata.de