How we are using Elixir

14 March, 2017

This post is a continuation of series of posts started in previous article. Previously we’ve talked about one of the most awesome, innovative and pleasant projects that we had the honor to develop here in FlatStack. In this article I will tell you about the problems we’ve faced and how the right choice of technologies stack helped us to solve them.

Concerns that we’ve faced

When we started this project we knew right away that the features that we need to implement aren’t that simple. It’s not a typical website with predefined functionality and common list of features, it’s not a web store or blog, but rather a more complex and solid service. The heart of our project is the time lapse build process.

We’re operating the huge amount of photos distributed in time by hours, days, months or even years. It would be nice if we could see these photos through time to see progress, changes or development of something. Therefore we can just clue these images into one video — time lapse. However, this feature isn’t that simple as it sounds. The entire build process is multi-staged and sophisticated because it requires operations with external resources, data storages, system utilities. We need to make several steps before we get the time lapse:

So these features were not easy and straightforward. We’ve faced many concerns and problems regarding these tasks:

These issues are not so easy to solve when you’re trying to fix them using popular programming languages or technologies. Most of them are not so fault-tolerant, not process-oriented and requires much more complexity to build such things. But we were free in selecting technologies stack so we’ve decided to give a chance to Elixir. And after successful development of time lapse service we certainly sure that this was the right choice.

What is an Elixir?

It’s not just a new fancy, shiny programming language. It’s an entire platform based on mature, robust Erlang VM and it’s ecosystem. What are key benefits of Erlang runtime system? Well, the are plenty of them:

Elixir comes with pleasant Ruby-like syntax and fully compatibility with any Erlang code. But it also extends standard Erlang library and provides more tools and libraries:

We’ve tried Elixir before in our hackathon projects and it established itself as a powerful and eloquent tool for developing complex thread-based applications, e.g servers, process supervisors. Therefore we’ve decided that core features and advantages of Elixir as both programming language and platform will doubtless help us to solve our problems so we’ve decided to give it a chance.

How we’ve actually used Elixir?

At the first, we’ve created a base skeleton for our new Elixir applications: https://github.com/fs/elixir-base. In this repository we’ve developed just a simple template that contains required libraries, scripts and tools that we like to use to.

The heart of project, time lapse service is based on this skeleton. Basically it’s just a simple server written in Elixir, but it contains multiple supervisors, processes, distribution steps. OTP tools are just great for building such applications, they allow us to write complex multi-thread logic in elegant and simple way. So how it help us with solving main problems and concerns?

Conclusion

Developing process of Elixir micro service was not that fast at first. When we created the first fully-functional prototype we thought that it was already completed and reliable. But after some time we saw the endless possibilities to improve our current implementation and making it more robust and stable.

We did a stress testing of our service and understood that we need more flexibility and steadiness for each step of time lapse build process. Therefore we tried to rethink our entire architecture and the build flow and we’ve ended up reconstructing our application using GenStages. This approach finally resolved our main concerns regarding application stability and performance bottlenecks and we’ll share with you some more information about this in our next post.