You have a substantial development project with several people working on it and you expect to be around for a while. Having your source built by Teamcity/TFS/Jenkins etc is a good start, but there is plenty more you should probably be doing.

To help out, here is my checklist of a good quality continuous integration/continuous delivery pipeline. It takes a while and some experience to set it all up but the long term benefits to developer productivity are well worth it.

How many do you score?

  • All code in source control
  • Documented how to build and how to deploy readme in root of project.
  • Automated build runs automatically on check-in for every branch
  • Build artefacts can be uniquely linked to a revision in source control
  • Automated build does not requires a special build chain or configuration of the build server
  • Each contributors work can be built and deployed separately from others and is testable in isolation
  • Unit tests detected and run automatically and can fail the build
  • Build and unit testing takes less than 5 minutes
  • Each build has a unique identifier, ideally a SemVer number
  • Documented up to date build and deployment dependencies in the source
  • Any developer can run a scripted one click build after checkout. No special manual configuration needed.
  • Developer can run a one click deploy and have a full working app on their machine including db, files, data etc
  • Build artefacts contains no sensitive passwords
  • Build generates code coverage metrics for server side code (and client side too if you have it)
  • Code coverage below team agreed threshold fails the build
  • Build identifies code duplication
  • Code duplication above threshold fails the build
  • Any build can be deployed in one click to a test environment
  • Deployment/environment related configuration can be changed without needing a new build
  • Each passing build is deployed automatically into a test environment
  • Build takes under 10 minutes
  • Deploy takes under 5 minutes
  • Test environment does not have production data in it and is created from scratch on each deploy
  • Smoke tests run after deploy automatically - validating all deployed components are up and running
  • Integration tests cover key component integration points and run as part of the build pipeline
  • Acceptance tests run automatically against deployed system to identify regression
  • Acceptance tests complete in under 20 minutes
  • A passing build can be promoted to a staging environment with one click, and then promoted to production when ready
  • Source control is tagged with live deploy version
  • Backups are taken before the deployment
  • Deploys to live have zero downtime so you can deploy as often as you like
  • When the deploy fails the previous version stays in place or you can rollback in one click
  • Data can be restored to the pre deploy state with one click/command