I wanted to share this pipeline I built for one of my SDK projects which has been very useful to me and solved a number of publishing & testing problems. It uses a few intermediate and (at least one unconventional) techniques to achieve the following:
- Checkout and build a .NET 6 C# Project.
- Run Unit Tests.
- Run Integration tests with SQL Server & Postgresql databases.
- Disintegration tests (more on this later)
- Determine Nuget package version from branch name and Publish to Artifactory/Nuget
This is the repository here:
https://github.com/mathtone/mathtone-magic/
Yml:
https://github.com/mathtone/mathtone-magic/blob/main/.circleci/config.yml
A few features here:
This step looks for branches named something like: release/1.0.1-whatever and sets a tag for that package version in an env. variable:
commands:
get-version:
steps:
- run:
name: Getting Package Version
command: |
branch=$CIRCLE_BRANCH
ver=${branch/"release/"/""}
suffix=${ver#*-}
build=${ver%-*}
vertag=""
if [ $suffix == "" ] || [ $suffix == $build ]; then
echo NO SUFFIX
echo "export PKG_SFX_TAG=""" >> $BASH_ENV
vertag=NOPE
else
echo $"SUFFIX FOUND- {$suffix}"
vertag=$"--version-suffix $suffix"
echo $"VERSION TAG- $vertag"
echo "export PKG_SFX_TAG='$vertag'" >> $BASH_ENV
fi
echo "export PKG_VER=$build" >> $BASH_ENV
echo "export PKG_SFX=$suffix" >> $BASH_ENV
This one spools up container images with SQL Server and Postgresql and runs the integration tests:
integration-tests:
docker:
- image: mcr.microsoft.com/dotnet/sdk:6.0
- image: postgres
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: test!1234
- image: mcr.microsoft.com/mssql/server:2022-latest
environment:
MSSQL_SA_PASSWORD: Test!1234
ACCEPT_EULA: Y
working_directory: /mnt/ramdisk
steps:
- attach_workspace:
at: .
- run: dotnet test src/mathtone-magic.sln --configuration Release
These two publish to Artifactory and (in my case with approval) to Nuget.org:
publish-artifactory:
docker:
- image: mcr.microsoft.com/dotnet/sdk:6.0
steps:
- attach_workspace:
at: .
- run:
name: "Add Artifactory"
command: dotnet nuget add source https://mathtone.jfrog.io/artifactory/api/nuget/v3/mathtone-dev --name Mathtone-Dev --username ${ARTIFACTORY_USER} --password ${ARTIFACTORY_PWD} --store-password-in-clear-text
- run:
name: Publish to Artifactory
command: |
ls -l packages
dotnet nuget push packages/*.nupkg --source Mathtone-Dev
publish-nuget:
docker:
- image: mcr.microsoft.com/dotnet/sdk:6.0
steps:
- attach_workspace:
at: .
- run:
name: Publish to Nuget
command: dotnet nuget push packages/*.nupkg --api-key ${NUGET_PUBLISH_KEY} --source https://api.nuget.org/v3/index.json
Like So:
The "Disintegration Tests step is incredibly useful to me, this step uses a utility program included in the repository as code (which of course C/CI can build and run) to analyze the solution, identify all project references to projects that will be published as packages, organize that into a hierarchy and create a script to convert all those project references to package references. Once that script executes C/CI performs the “Test” step again. This is VERY USEFUL to me in managing this SDK that’s composed of a dozen or so interrelated package references. It allows me to work on the SDK with the projects referencing each other and PUBLISH with all package references.
disintegration-tests:
docker:
- image: mcr.microsoft.com/dotnet/sdk:6.0
- image: postgres
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: test!1234
- image: mcr.microsoft.com/mssql/server:2022-latest
environment:
MSSQL_SA_PASSWORD: Test!1234
ACCEPT_EULA: Y
working_directory: /mnt/ramdisk
steps:
- attach_workspace:
at: .
- get-version
- run: |
mkdir temp
mkdir packages
ls -l
- run: dotnet build tools/build-util/build-util.csproj --output temp
- run: dotnet temp/build-util.dll src/mathtone-magic.sln
- run: cat temp/process-sln.sh
- run: |
echo SUFFIX - $PKG_SFX_TAG
dotnet restore src/mathtone-magic.sln
dotnet nuget remove source nuget.org
dotnet nuget add source /mnt/ramdisk/packages -n local
bash temp/process-sln.sh
- run: dotnet test src/mathtone-magic.sln --configuration Release -- xunit.parallelizeAssembly=true
- persist_to_workspace:
root: .
paths:
- packages
Anyway, this is still a bit rough but there are a few things here that people might find useful so I thought I would share this. In the end you get something like this: