Elixir continious integration with CircleCI

2018-06-19 · [neiro]

Elixir programming language has gained popularity and now it is supported at many platforms, including plenty of CI services. In this article we will see how we can achieve seamless and (almost) dead simple continious integration by using CircleCI in our Elixir projects.

CircleCI 2.0

CircleCI is one of the most popular and user-friendly continious integration solutions. It supports many programming languages and tools, including Elixir and Erlang/OTP.

CircleCI is entirely free when it comes to open-source GitHub repositories, but it also provides free 1500 minutes a month for any private repos.

Starting version 2.0 CircleCI can create jobs based on any images from DockerHub. This feature makes possible to build any programming language or platform that can be placed in Docker image.

Imagine you have a standard Elixir Phoenix / Ecto application. You need to run it on the latest versions of Elixir and Erlang/OTP and run the tests on PostgreSQL database.

Let’s start by creating a CircleCI configuration file in .circleci/config.yml:

 1  version: 2  # use CircleCI 2.0 instead of CircleCI Classic
 2  jobs:  # basic units of work in a run
 3    build:  # runs not using Workflows must have a `build` job as entry point
 4      parallelism: 1  # run only one instance of this job in parallel
 5      docker:  # run the steps with Docker
 6        - image: circleci/elixir:1.6 # ...with this image as the primary container; this is where all `steps` will run
 7          environment:  # environment variables for primary container
 8            MIX_ENV: test
 9            SHELL: /bin/bash
10        - image: mdillon/postgis:9.6-alpine  # database image
11          environment:  # environment variables for database
12            POSTGRES_DB: app_test
13
14      steps:  # commands that comprise the `build` job
15        - checkout  # check out source code to working directory
16
17        - run: mix local.hex --force  # install Hex locally (without prompt)
18        - run: mix local.rebar --force  # fetch a copy of rebar (without prompt)

As you can see here, we are declaring the build continious integration job. Basically we will use the Elixir 1.6 with PostgreSQL 9.6 to run tests on the app_test database. After that we will checkout source code base to fetch our recent changes into the build. mix local tasks are also necessary in order to use any of Mix tasks later.

Running tests and code quality

All of us want to run all common continious integration steps such as:

Fetch dependencies and compile application
Run code quality tools and checks (you can read more about it here)
Execute all tests to make sure that our build is successful and infallible
Run heavy and bulky static analysis tools

Also we want to make our builds as fast as possible, so we definitely need caching. Let’s continue with our config and implement the steps above:

{% raw %}

 1        - restore_cache:  # restores saved mix cache
 2            keys:  # list of cache keys, in decreasing specificity
 3              - v1-mix-cache-{{ .Branch }}-{{ checksum "mix.lock" }}
 4              - v1-mix-cache-{{ .Branch }}
 5              - v1-mix-cache
 6        - restore_cache:  # restores saved build cache
 7            keys:
 8              - v1-build-cache-{{ .Branch }}
 9              - v1-build-cache
10        - restore_cache:  # restores saved plt cache
11            keys:
12              - dialyzer-cache
13
14        - run: mix do deps.get, compile # get updated dependencies & compile them
15
16        - save_cache:  # generate and store cache so `restore_cache` works
17            key: v1-mix-cache-{{ .Branch }}-{{ checksum "mix.lock" }}
18            paths: "deps"
19        - save_cache:  # make another less specific cache
20            key: v1-mix-cache-{{ .Branch }}
21            paths: "deps"
22        - save_cache:  # you should really save one more cache just in case
23            key: v1-mix-cache
24            paths: "deps"
25        - save_cache: # don't forget to save a *build* cache, too
26            key: v1-build-cache-{{ .Branch }}
27            paths: "_build"
28        - save_cache: # and one more build cache for good measure
29            key: v1-build-cache
30            paths: "_build"
31
32        - run: mix do format --check-formatted, credo --strict, security
33        - run: mix do xref deprecated --include-siblings, xref unreachable --include-siblings, xref graph --format stats
34
35        - run:  # special utility that stalls main process until DB is ready
36            name: Wait for DB
37            command: dockerize -wait tcp://localhost:5432 -timeout 1m
38
39        - run: mix do ecto.migrations, ecto.load
40        - run: mix test  # run all tests in project
41
42        - run: mix dialyzer --halt-exit-status
43        - save_cache:
44            key: dialyzer-cache
45            paths: "_build/test/dialyxir*.plt"
46
47        - store_test_results:  # upload test results for display in Test Summary
48            path: _build/test/lib/app/results.xml

{% endraw %}

Now you can start new builds by signing up into CircleCI as it will run the configuration and steps from your config. Every commit in any branch will run the build job and you will know if something is wrong with your code.

Deploying

However, having only one build job is not enough even for the simplest CI process. Most often we need to make a staging/production release by using Distillery.

Let’s continue filling up our configuration file by adding a new deploy job:

 1    deploy:
 2      docker:
 3        - image: circleci/elixir:1.6
 4          environment:  # environment variables for primary container
 5            SHELL: /bin/bash
 6            MIX_ENV: staging
 7      steps:
 8        - checkout  # check out source code to working directory
 9
10        - run: mix local.hex --force  # install Hex locally (without prompt)
11        - run: mix local.rebar --force  # fetch a copy of rebar (without prompt)
12
13        - run: mix do deps.get, compile # get updated dependencies & compile them
14
15        # set MIX_ENV to prod or staging value according to the source branch
16        - run:
17            name: Update MIX_ENV environment variable
18            command: |
19              echo "export MIX_ENV=$(if [ '$CIRCLE_BRANCH' '==' 'master' ]; then echo 'prod'; else echo 'staging'; fi)" >> $BASH_ENV
20              source $BASH_ENV
21
22        - run: cd deps/argon2_elixir && make clean && make && cd -
23        - run: MIX_ENV=staging mix release --env $MIX_ENV
24
25        - run: tar -zcvf $CIRCLE_SHA1.tar.gz bin appspec.yml VERSION _build/$MIX_ENV/rel/app/releases/$(cat VERSION)/app.tar.gz

This will be enough to create a separate deploy job that will run on a separate Docker image. However, we will need to run it only on develop and master branches in order to upload staging/production releases accordingly. We can achieve this by using CircleCI workflow and providing a simple configuration at the bottom of our config file:

 1  workflows:
 2    version: 2
 3    build-and-deploy:
 4      jobs:
 5        - build
 6        - deploy:
 7            requires:
 8              - build
 9            filters:
10              branches:
11                only:
12                  - develop
13                  - master

After that you are free to upload the built release to any server or any platform you want. You can use Edeliver, Ansible, Chef, Docker - it’s up to you.

Conclusion

As you can see above, it’s not so hard to build and deploy Elixir applications with CircleCI 2.0. This platform is flexible and fast enough to make your continious integration bright and shiny.

If you want to discover even more on the topic then let’s read CircleCI 2.0 documentation and Elixir Language Guide.

Happy hacking, everyone!

#elixir #quality #ci