Cupermind

Continuous Integration process for the static site

A short time ago we’ve presented our site. As you might expect, everything we do, we try to do DevOps style. That is one of shiny examples of how things can be automated for a thing as small as our company blog.

1. As you already might know, we are using awesome static site generator called Jekyll - the most popular tool from static site generators family.
2. Git and GitLab SaaS has been selected to empower “Distributed Version Control System” DVCS needs for this project.
3. Continuous Integration process is up and running.
4. We get all the above as a free service!

Follow the steps from this article to achieve similar results for your site - we will try to cover in detail what we’ve done and how.

Ideas

Making thing work DevOps style - how we see it:

1. Site should be placed to DVCS, and workflow working with code should be defined. Usually it implies some techniques like code reviews, feature branches, pull/merge requests. There are many DVCS out there, but usually teams tend to use either Git or Mercurial.
2. Local development must be automated to some extent, for example running local server, watching file changes & reloading/refreshing content and many other things which are relevant to your site should be automated. Some frameworks include some useful commands to automate things, but if not, please refer to our article on Paver and Fabric
3. Automated tests must be in place.
4. Both Production and Staging endpoints must be preserved.
5. Developer should be able to deploy to production/staging easily.
6. We will also show how to use Continuous Integration (CI) to deploy master branch to production on a change trigger.

Automation with Make

Jekyll speaks Ruby. So we need to use Bundler for it. Our Makefile:

NAME := "cupermind.com"
JEKYLL := "./bin/jekyll"
PROD_ENDPOINT := "[email protected]:~/htdocs/"
PROD_PORT := 12345
STAGING_ENDPOINT := "[email protected]:~/htdocs/"
STAGING_PORT := 54321
SSH_OPTIONS := "-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null"

.PHONY: help, prepare, serve, build, build_staging, deploy, deploy_staging, clean, test

PROJECTDIR := $(shell /bin/bash -c pwd)

help: ## Help
		@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'

prepare: ## Prepare local Ruby environment
		bundler install --path vendor/bundle --binstubs

serve: ## Start local-dev server
		@$(JEKYLL) serve --config _config.yml,_config_dev.yml

build: ## Build prod version of site
		JEKYLL_ENV=production $(JEKYLL) build

build_staging: ## Build staging version of site
		@$(JEKYLL) build --config _config.yml,_config_staging.yml

deploy: ## Deploy site to production
		rsync -avz  -e "ssh $(SSH_OPTIONS) -p $(PROD_PORT)" _site/ $(PROD_ENDPOINT)

deploy_staging: ## Deploy site to staging
		rsync -avz  -e "ssh $(SSH_OPTIONS) -p $(STAGING_PORT)" _site/ $(STAGING_ENDPOINT)

clean: ## Clean
		@$(JEKYLL) clean

test: ## Test the site
		@./bin/htmlproofer ./_site --disable-external --assume-extension --url-ignore "/#.*/"

Local development

1. make prepare
2. make serve

First command installs all gems needed for the site and second starts local web server.

Tests

html-proofer is being used to do some basic tests.

Deploy changes to production/staging

To deploy site to staging developer should:

1. Build static site for staging: make build_staging.
2. Test it: make test.
3. Deploy it: make deploy_staging.

Doing the same on production is very similar:

1. make build
2. make test
3. make deploy

But actually we would recommend using Continuous Integration doing the same, but more reliable and easy way (read next ↓).

Continuous integration

For Cupermind site we’ve decided to use GitLab pipelines to implement CI, and here is our .gitlab-ci.yml:

image: ruby:2.3

before_script:
# install ssh-agent
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
# install rsync
- 'which rsync || ( apt-get update -y && apt-get install rsync -y )'
# run ssh-agent
- eval $(ssh-agent -s)
# add ssh key stored in SSH_PRIVATE_KEY variable to the agent store
- ssh-add <(echo "$SSH_PRIVATE_KEY")
- make prepare

stages:
- build
- deploy

staging_deploy:
  stage: deploy
  script:
  - make build_staging
  - make test
  - make deploy_staging
  except:
  - master

prod_deploy:
  stage: deploy
  script:
  - make build
  - make test
  - make deploy
  only:
  - master

create_artifact:
  stage: build
  script:
  - make build
  - make test
  artifacts:
    paths:
    - _site/
  only:
  - tags

Now, every push to non-master branch triggers the event of staging server automated deploy, and any push to master triggers deploy to prod.

Additionally, for any tag in master branch artifact is being created. Then, we are using rsync over ssh to actually transfer files to destination server (production/staging).

Conclusion

All components pass to each other and are working perfectly. Sometimes there can be jitter with pipeline running on GitLab free shared runners, meaning sometime CI process takes longer time, but usually it does it in around 5 minutes.