Static site CI
by Alexander Svyrydov (October 17, 2016)
Continuous Integration process for the static site
A short time ago we’ve presented our site. As you might expect, everything we do, we try to do DevOps style. That is one of shiny examples of how things can be automated for a thing as small as our company blog.
- As you already might know, we are using awesome static site generator called Jekyll - the most popular tool from static site generators family.
- Git and GitLab SaaS has been selected to empower “Distributed Version Control System” DVCS needs for this project.
- Continuous Integration process is up and running.
- We get all the above as a free service!
Follow the steps from this article to achieve similar results for your site - we will try to cover in detail what we’ve done and how.
Ideas
Making thing work DevOps style - how we see it:
- Site should be placed to DVCS, and workflow working with code should be defined. Usually it implies some techniques like code reviews, feature branches, pull/merge requests. There are many DVCS out there, but usually teams tend to use either Git or Mercurial.
- Local development must be automated to some extent, for example running local server, watching file changes & reloading/refreshing content and many other things which are relevant to your site should be automated. Some frameworks include some useful commands to automate things, but if not, please refer to our article on Paver and Fabric
- Automated tests must be in place.
- Both Production and Staging endpoints must be preserved.
- Developer should be able to deploy to production/staging easily.
- We will also show how to use Continuous Integration (CI) to deploy
master
branch to production on a change trigger.
Automation with Make
Jekyll speaks Ruby. So we need to use Bundler for it. Our Makefile
:
NAME := "cupermind.com"
JEKYLL := "./bin/jekyll"
PROD_ENDPOINT := "[email protected]:~/htdocs/"
PROD_PORT := 12345
STAGING_ENDPOINT := "[email protected]:~/htdocs/"
STAGING_PORT := 54321
SSH_OPTIONS := "-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null"
.PHONY: help, prepare, serve, build, build_staging, deploy, deploy_staging, clean, test
PROJECTDIR := $(shell /bin/bash -c pwd)
help: ## Help
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
prepare: ## Prepare local Ruby environment
bundler install --path vendor/bundle --binstubs
serve: ## Start local-dev server
@$(JEKYLL) serve --config _config.yml,_config_dev.yml
build: ## Build prod version of site
JEKYLL_ENV=production $(JEKYLL) build
build_staging: ## Build staging version of site
@$(JEKYLL) build --config _config.yml,_config_staging.yml
deploy: ## Deploy site to production
rsync -avz -e "ssh $(SSH_OPTIONS) -p $(PROD_PORT)" _site/ $(PROD_ENDPOINT)
deploy_staging: ## Deploy site to staging
rsync -avz -e "ssh $(SSH_OPTIONS) -p $(STAGING_PORT)" _site/ $(STAGING_ENDPOINT)
clean: ## Clean
@$(JEKYLL) clean
test: ## Test the site
@./bin/htmlproofer ./_site --disable-external --assume-extension --url-ignore "/#.*/"
Local development
make prepare
make serve
First command installs all gems
needed for the site and second starts local web server.
Tests
html-proofer is being used to do some basic tests.
Deploy changes to production/staging
To deploy site to staging developer should:
- Build static site for staging:
make build_staging
. - Test it:
make test
. - Deploy it:
make deploy_staging
.
Doing the same on production is very similar:
make build
make test
make deploy
But actually we would recommend using Continuous Integration doing the same, but more reliable and easy way (read next ↓).
Continuous integration
For Cupermind site we’ve decided to use GitLab pipelines
to implement CI, and here is our .gitlab-ci.yml
:
image: ruby:2.3
before_script:
# install ssh-agent
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
# install rsync
- 'which rsync || ( apt-get update -y && apt-get install rsync -y )'
# run ssh-agent
- eval $(ssh-agent -s)
# add ssh key stored in SSH_PRIVATE_KEY variable to the agent store
- ssh-add <(echo "$SSH_PRIVATE_KEY")
- make prepare
stages:
- build
- deploy
staging_deploy:
stage: deploy
script:
- make build_staging
- make test
- make deploy_staging
except:
- master
prod_deploy:
stage: deploy
script:
- make build
- make test
- make deploy
only:
- master
create_artifact:
stage: build
script:
- make build
- make test
artifacts:
paths:
- _site/
only:
- tags
Now, every push to non-master branch triggers the event of staging server automated deploy, and any push to master triggers deploy to prod.
Additionally, for any tag in master branch artifact is being created.
Then, we are using rsync over ssh
to actually transfer files to destination server (production/staging).
Conclusion
All components pass to each other and are working perfectly. Sometimes there can be jitter with pipeline running on GitLab free shared runners, meaning sometime CI process takes longer time, but usually it does it in around 5 minutes.