.Net 6 Pipeline with SQL Server, Postgresql, & Publishing to Nuget Example

Helpful Resources Tips, Tricks, and Hacks
1

I wanted to share this pipeline I built for one of my SDK projects which has been very useful to me and solved a number of publishing & testing problems. It uses a few intermediate and (at least one unconventional) techniques to achieve the following:

  • Checkout and build a .NET 6 C# Project.
  • Run Unit Tests.
  • Run Integration tests with SQL Server & Postgresql databases.
  • Disintegration tests (more on this later)
  • Determine Nuget package version from branch name and Publish to Artifactory/Nuget

This is the repository here:
https://github.com/mathtone/mathtone-magic/

Yml:
https://github.com/mathtone/mathtone-magic/blob/main/.circleci/config.yml

A few features here:

This step looks for branches named something like: release/1.0.1-whatever and sets a tag for that package version in an env. variable:

commands:
  get-version:
    steps:
      - run:
         name: Getting Package Version
         command: |
          branch=$CIRCLE_BRANCH
          ver=${branch/"release/"/""}
          suffix=${ver#*-}
          build=${ver%-*}
          vertag=""
          if  [ $suffix == "" ] || [ $suffix == $build ]; then
            echo NO SUFFIX
            echo "export PKG_SFX_TAG=""" >> $BASH_ENV
            vertag=NOPE
          else
            echo $"SUFFIX FOUND- {$suffix}"
            vertag=$"--version-suffix $suffix"
            echo $"VERSION TAG- $vertag"
            echo "export PKG_SFX_TAG='$vertag'" >> $BASH_ENV
          fi
          
          echo "export PKG_VER=$build" >> $BASH_ENV
          echo "export PKG_SFX=$suffix" >> $BASH_ENV

This one spools up container images with SQL Server and Postgresql and runs the integration tests:

  integration-tests:
    docker:
      - image: mcr.microsoft.com/dotnet/sdk:6.0
      - image: postgres
        environment:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: test!1234
      - image: mcr.microsoft.com/mssql/server:2022-latest
        environment:
          MSSQL_SA_PASSWORD: Test!1234
          ACCEPT_EULA: Y
    working_directory: /mnt/ramdisk
    steps:
      - attach_workspace:
          at: .
      - run: dotnet test src/mathtone-magic.sln --configuration Release

These two publish to Artifactory and (in my case with approval) to Nuget.org:

publish-artifactory:
    docker:
      - image: mcr.microsoft.com/dotnet/sdk:6.0
    steps:
      - attach_workspace:
          at: .
      - run:
          name: "Add Artifactory"
          command: dotnet nuget add source https://mathtone.jfrog.io/artifactory/api/nuget/v3/mathtone-dev --name Mathtone-Dev --username ${ARTIFACTORY_USER} --password ${ARTIFACTORY_PWD} --store-password-in-clear-text
      - run:
         name: Publish to Artifactory
         command: |
            ls -l packages
            dotnet nuget push packages/*.nupkg --source Mathtone-Dev
  publish-nuget:
    docker:
      - image: mcr.microsoft.com/dotnet/sdk:6.0
    steps:
      - attach_workspace:
          at: .
      - run:
         name: Publish to Nuget
         command: dotnet nuget push packages/*.nupkg --api-key ${NUGET_PUBLISH_KEY} --source https://api.nuget.org/v3/index.json

Like So:

image
image1051×472 18.7 KB

The "Disintegration Tests step is incredibly useful to me, this step uses a utility program included in the repository as code (which of course C/CI can build and run) to analyze the solution, identify all project references to projects that will be published as packages, organize that into a hierarchy and create a script to convert all those project references to package references. Once that script executes C/CI performs the “Test” step again. This is VERY USEFUL to me in managing this SDK that’s composed of a dozen or so interrelated package references. It allows me to work on the SDK with the projects referencing each other and PUBLISH with all package references.

 disintegration-tests:
    docker:
      - image: mcr.microsoft.com/dotnet/sdk:6.0
      - image: postgres
        environment:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: test!1234
      - image: mcr.microsoft.com/mssql/server:2022-latest
        environment:
          MSSQL_SA_PASSWORD: Test!1234
          ACCEPT_EULA: Y
    working_directory: /mnt/ramdisk
    steps:
      - attach_workspace:
          at: .
      - get-version
      - run: |
          mkdir temp
          mkdir packages
          ls -l
      - run: dotnet build tools/build-util/build-util.csproj --output temp
      - run: dotnet temp/build-util.dll src/mathtone-magic.sln
      - run: cat temp/process-sln.sh
      - run: |
          echo SUFFIX - $PKG_SFX_TAG
          dotnet restore src/mathtone-magic.sln
          dotnet nuget remove source nuget.org
          dotnet nuget add source /mnt/ramdisk/packages -n local
          bash temp/process-sln.sh
      - run: dotnet test src/mathtone-magic.sln --configuration Release -- xunit.parallelizeAssembly=true
      - persist_to_workspace:
          root: .
          paths:
            - packages

Anyway, this is still a bit rough but there are a few things here that people might find useful so I thought I would share this. In the end you get something like this:

image
image2368×289 22.9 KB