Build Automation for Dotnet Core Apps

In a previous blog post I documented how I built a “Build Server” to deploy .NET 4.6+ apps on windows 2012 server.

While this worked, and was a reasonably good way to do it, It wasn’t without it’s problems. During it’s use for example I frequently had timing problems, where just one little change to some JS code would cause NPM to overrun a time out by half a second, or where an SSH connection timed out just slightly before the build server completed it’s login, and on top of all that, it regularly used to take about 15 minutes to build and deploy the project it was being used for.

Since then however, Iv’e started to explore Dotnet Core further, and Iv’e now started to build a lot of my apps using the SPA templates for Aurelia with an ASP.NET Core web api back end.

Since my application template has now gotten a huge amount leaner and more efficient, I thought I should really do the same with my chosen automated build process, this post documents how I ended up putting that together.

Step 1 – Provision the machine

I have my own onsite cloud server, so it was a simple case of firing up MS-Hyper-V manager and creating a small VM, I allocated one virtual CPU to the instance and configured it with 2gb’s of ram and a 30gb virtual hard disk.

Once the VM was up and running I booted it with an Ubuntu 16.04 64bit server instance, and went through the installation.  At the end of the install when I was asked which packages I wanted to install, I selected nothing except an OpenSSH server, and I configured the install to use one partition for the entire machine.

How you actually configure the hardware, partitions etc really doesn’t matter, just as long as you give yourself enough horse power to run the tools you need.

Once all this was done I closed down Hyper-V manager and the console, and logged in using Putty via SSH.

Step 2 – Setting up the required Software

The next step was going to be configuring the software the system was to use.  For this build I choose to use Jenkins rather than Cruise Control.NET.

The primary motivation for this is that Jenkins is Java based, and thus it means if you want to do this on a windows server you can easily do so, for me I’m building this on a Linux OS so I wanted something that would work there too.  On top of that, CCNet has been unmaintained for quite some time now, and while it’s a veteran of the CI/CD world, it is unfortunately starting to show it’s age.  Jenkins in comparison seems to have a huge amount more traction and support.

Beacuse Jenkins is Java based you first need to make sure that you have an up to data Java virtual machine installed on the target server.  Unfortunately as of the date of this blog post, the current version available in the Ubuntu software repository was too old, so to get the most up to date version you need to add the Java PPA repo.  This is not nearly as scary as it sounds.

As sudo run the following command:

This will ask you for your superuser password, and should go ahead and install the requested package.

Once the package has installed, you then need to use it to update your package repository list by using the following “add-apt-repository” command:

As you can see, this will add the PPA for openjdk and the required RSA signitures.

once, you get to this point, you just need to issue an apt-get update command, followed by the install command for the JDK you need:

(Thats: sudo apt-get install openjdk-7-jdk if the writing in the above image is a little too small)

Once youv’e gotten the jdk installed a quick “java –version” at the command line should return something that has a version number similar to “1.7.0_95” in it.  This is the minimum version you need as of the date of this post, depending on how far in the future you read this, you may need a further version, unfortunately Iv’e not yet gotten my time machine working, so I can’t answer that one for you just yet

Once you have the correct version, your now ready to install Jenkins, and surprise surprise, thats also in a different repo too …..

First thing you need to do is to get and add the repo details

Which then finally puts you in a position to actually install Jenkins

Once your screen has stopped chucking out lots of text and returned you to your command prompt, you should be able to start the Jenkins CI server with the following command:

If all goes well you should be able to put “http://<server ip address>:8080/” into your browser and see the following:

If you see this then congratulations, you have Jenkins installed.  If you cat the file mentioned in the browser window to your linux console,

and copy and paste the contents of it into the box provided, you’ll go to the next step, please follow the instructions and set up any users and plug-in’s you need.  Come back here once your done.

Adding Dotnet Core

Once you have Jenkins set up, you need the Dotnet core build tools installing on the machine, you can see a video and get copy paste instructions from Miscrosoft’s official site here:

https://www.microsoft.com/net/core#linuxubuntu

Remember to follow the instructions for “Ubuntu 16.04 / Linux Mint 18” if your installing the same base that I am here.

Once you’ve done the “sudo apt-get install dotnet-sdk-2.0.0” part, and gotten this:

Then your ready to create your first build script using Jenkins.

To do that you should basically be able to do a “dotnet restore”, “dotnet build” and “dotnet publish” for your application without a problem.

One problem you MIGHT encounter, especially if your using the Dotnet core SPA templates is the version of NodeJS that ships by default with Ubuntu 16.04.  To fix this, you need to import the updated NodeJS repository using:

curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash –

Followed by a normal sudo apt-get install nodejs

I’ll cover setting up a basic Aurelia JS application with .NET back end in another blog post, for now we’ll just assume you have one ready to go already, and it’s sat in your git version control server ready to be deployed.

Setting up your build job

Fire up your blowser, head to “http://<server ip here>:8080/” and log in using the user name and password you created when you did the final set-up step of jenkins.

Once your logged in, click on “Create new Jobs” in your dash board view to get started.

When prompted, enter a name for your build project and choose “Freestyle Project”

Then click the “Ok” button at the bottom left of the page.

Fill in your Source code managment section:

and your build triggers (In this case Iv’e set it to check my git server every 5 minutes):

Under the “Build” section, add 3 execute shell steps, and fill them in as follows:

Optionally on the build and publish commands you can specify the build profile EG: “dotnet build –c Release”

At this point, you can leave the job running, and go back to your regular work machine, editing your app in Visual Studio for example and checking it into the specified Git repository.

Every 5 minutes, Jenkins will check the repo, and if it’s changed, check it out and perform a restore and build on it, showing fails and successes in the Jenkins dash board as it does.  For me however, I went one step further.

Deployment

I installed an apache web server on the machine, then set it up to forward proxy to my .NET app.  For those of you unfamiliar with the term, it means you can go to say “http://myserver.local/mywebapp” in your browser, and make it connect to “http://myserver.local:5000” or which ever port your Dotnet core application is listening on.

Unfortunately going into the configuration and set up of Apache in this post would make it far to long, and there’s plenty of tutorials elsewhere out there on the web showing how to do this, so I’m just going to skip over that part and explain how to deploy your now compiled app to another folder, ready to run from your fresh build.

First step is to add a 4th and final “execute shell” step to the main build process, just after “dotnet publish”

As you’ll see from the image, iv’e put in the command “echo ‘APP BUILT’”

The text you put in here can be anything you want, as it’s used as a marker to decide if the post build step should be triggered.

Your post build step (Which we’ll configure in just a moment) will always be run, so to control it, it looks for a string in your output.  In my first version of this, I had mine looking at output from the dotnet cli command, and that caused me a few problems, as it also appeared even when my app failed to build.

By echoing a string as the very last step in the build pipeline, you can be sure that if any build step fails before that one, you’ll never get to the point where you execute that final step, and as a result, your post build step won’t find it’s marker, and won’t run, saving you from deploying an old build, or worse getting lots of errors in your deploy script beacuse something is missing.

In your “Post Build” actions, you need to add a “Post Build Task”, and then set it up as follows:

As you can see the Log Text is the same string you put in the last echo statement above, and in the script box, we have the name of the shell script that performs our deploy, prefixed with “sudo” to make it run as a root user.

My script has to be run as root beacuse it uses supervisord (As shown in Scott Hanslemans post [Step 5] HERE)

Since i need to start and stop the service, I need to run it prefixed with sudo, but there’s a small problem with this.

When you run something with sudo on a linux install, generally you get asked to enter the superuser password in order to allow it to run, and in something like Jenkins this is not possible, as the shell is not interactive.

You could just run Jenkins as the “root” user, but this is a VERY, VERY, VERY, VERY BAD IDEA (Let me repeat that 10 more times in 1 foot high bold lettering)…..

So how do you run it?

Well it turns out, that you can configure sudo in a surprising number of ways, one of which allows you to specify exactly one user who is allowed to run one thing (and one thing only) as a root user.

You do this by editing your sudoers file, which can usually be found in “/etc/sudoers”, under Ubuntu you normaly use the “visudo” command to edit the file, but in reality, as long as you have root access you can use any plain text editor.

In the user privilege section, add a line for jenkins as follows:

As you can see, Iv’e specified the jenkins user, told it no password, but ONLY for that one script file.  You really DO NOT want to check that file into your version control, and you want to make it so the ONLY user that can read/write or execute it on the server is root, again there are plenty of tutorials available on Linux file permissions but usually something like “chmod 700 deployfromjenkins.sh” will do the trick.

As for the contents of the script, well for me it’s quite simply the following:

and yes, this is a different server to the temp one above

I already had this one set up, so it was easier just to screen shot that one.  The contents of your deploy file will depend very much on how you plan on deploying your application.  For me, I wanted to copy it from the folder that Jenkins had built and published it to, into the folder where my Apache web server was looking for it to be running, I also needed to stop supervisor (Very similar to stopping and starting stand alone services on windows) before I replaced the files, followed by starting it again afterwards.

I still need to come up with a way to make sure no run-time errors prevented the application from re-starting, but that’s something Iv’e not figured out yet, for now however using this set-up means I can make changes in VS2017, check my application into my git server, and within 5 minutes or less it’ll be built and deployed, ready to just serve on a regular port 80 request curtsy of Apache web server.

This whole process is also fairly portable with very minor changes, and there’s no reason why you couldn’t make it run on MacOS or Windows too.