CircleCi 2.0 has introduced workflows. A workflow is a set of declarations that define a number of jobs, their run order and helps you orchestrate and troubleshoot those jobs. It has a lot of benefits compared to CircleCi 1.0 that doesn’t support workflows. However, our team switched to CircleCi 2.0, getting notifications on our continuous integration process became a problem. I get Slack alerts from projects on CircleCI that I am not following. In a large org, this can get incredibly noisy. Is there any way I can only get alerts for projects that I am following?

(I’m new to the CircleCI platform, so please correct me if any assumptions below are incorrect)

We’ve recently been notified that the “legacy Slack Notification settings will not be available in the new UI” via this post.

I attempted to follow step 1 (“Remove the Slack webhook url from the legacy chat notification page”) by editing the URL to contain our organization name and project names. However I cannot seem to get to the old UI. Clicking on links on this page results in the following message:

Docker on Linux allows Xdebug to automatically connect back to the host system so you can just set xdebug.remoteconnectback=1 and leave out the xdebug.remotehost in your xdebug.ini. Xdebug php docker.

Error! GraphQL error: Received a 404 from the /api/v2/project/github/ORG/PROJECT/envvar api.

I’ve decided to skip that step for now.

I’ve enabled the new Slack integration as per step 2, but the new integration seems to be missing a lot of information that made the status notifications useful.

Here’s what the old and new notifications look like:

Notice the lack of links to the repo, branch, and commit as well as it not showing the commit message or author.

The division 2 steam. I’ve looked at the documentation for the orb here but I don’t see any way to have it show all of the information that the old integration used to show.

Am I missing something here?

Hi, I’m @celia, an iOS Engineer working at Merpay. Last time, I wrote about the different aspects of infrastructure in Mercari’s mobile team, and introduced a few tools we made that makes our team more productive (article is available here). Today I want to share with you about some improvements we made recently, on automating code integration between Merpay and Mercari iOS.

Background

Let me first explain the problem we are trying to solve. As you may have guessed (because we have a code integration problem), Merpay and Mercari’s iOS codebase is stored in different GitHub repositories. Merpay’s code exists in Mercari app as a git submodule, and it is managed using a dependency manager called Carthage. Other than develop, commit and push, updating Merpay code in Mercari is an integral part of our tasks.

Circleci Slack Orb

If you are an iOS developer who is familiar with Carthage, feel free to skip this part. For those who don’t, let me explain how Carthage update works. It uses a file named Cartfile to keep track of dependencies for your project, pointing to the corresponding version or git position that it’s checking out from. In Cartfile, the dependency information is specified in the following format:

To update it, overwrite the Cartfile to the latest position, and run command carthage update in the console under your project directory. You can find details in Carthage’s documentation.

The Mercari iOS app is usually released on a bi-weekly basis. Before, we assign one member from Merpay iOS team to handle all the tasks, mainly:

• Confirm all pull requests under this release target is merged
• Update version number and add git tag for it
• Merge into main branch
• Update Carthage dependency in Mercari

We had a detailed checklist stored in GitHub’s issue template, and for every release we open a new issue to manage it. It was a series of tasks that took a few hours to do, and if something went wrong, might take half a day of our member’s time. This means the member we assigned to work on it needs to stop their tasks at hand, and deal with integration on high priority.

To automate this process, the rough requirements of our automated solution are:

• Increment version
• Create pull requests and merge branches
• Add git tag
• Make request to Mercari CI to trigger a job to update Merpay
• Overwrite Cartfile and run carthage update, submit a pull request in Mercari GitHub repository

We also found time was mostly spent on waiting for CI to run tests, which happens when a branch is merged or a new commit is pushed. Since the only relevant code change was bumping up the version number, which has no effect on the test results, it should skip all CI checks to speed up the entire process.

Solution

We set our goal to shorten the integration time to be within 40 minutes, and automate all the manual steps. We also consider this an opportunity to build a tool that’s versatile, so we can use it to build more automation features in the future. In addition, it should be easy for anyone to maintain and debug.

After discussing the idea with people across different teams, we figured out we can build a Slackbot using Google App Engine, that it parses simple incoming arguments and makes HTTP requests to CircleCI pipeline to trigger a workflow, which runs custom scripts to perform different steps in the integration tasks. We can write scripts in Ruby using fastlane to achieve steps that can be done using shell scripts and GitHub API, as long as we are able to connect the workflow so it starts next task in the sequence when the previous one is finished, and we are able to make a request from Merpay’s CI to Mercari’s CI to trigger a job to update Merpay, we can achieve our goal. Previously, we already solved the problem of overwriting Cartfile and running carthage update using scripts, you can find more about it in my previous blog post.

Here is a flow chart representing each step in the automated integration process:

A bot command prepare_next is triggered on Slack. It sends a message to Google App Engine (GAE). Then GAE parses the message and makes a request with needed parameters to CircleCI using pipeline. CircleCI then runs custom scripts written in fastlane, that it checks out the latest Merpay code, bumps up version number, adds tag for it, and sends another request to Mercari’s CircleCI using pipeline to trigger a job to run scripts to update Merpay.

While the job is running, it will post progress to the Slack channel mentioning Merpay iOS team, and when the workflow is finished, it will generate a pull request, updating Merpay to the latest version. The integration is finished when this pull request is merged.

Example of Slack notifications, certain information has been masked due to security concern.

Result

With the help of this tool, we are able to decrease the time needed for integration in release from 168 minutes to only 12 minutes, which is more than 10 times faster. We are also able to remove all the manual steps so the workflow is triggered by one single bot command.

Circleci Slack

The significant decrease of time is made possible by skipping as many CI checks as we can safely, and avoiding the cost of engineers frequently switching between tasks and the need to work in both project directories.

At Mercari/Merpay, we are always looking for better ways to deliver our product safer, and faster. Automation not only reduces repetitive manual work and lets engineers focus more on actual product development, it also reduces risks brought by possible human error. In the future, we’d like to add more features to the bot, such as deploy beta apps or triggering tests running on CI. It has the potential to simplify other aspects of our daily work as iOS engineers, and I’m excited to see how it will evolve in the future.

Circleci Slack Notifications

Conclusion

Circleci Slack Webhook

At the end, I want to thank the following people for contributing to this project:

• @vvakame (advisor)
• @kenmaz (bot engineer)
• @celia (CI engineer)
• @takeshi.sato (bot name suggestion)
• @kaito.watanabe, @daichiro (code reviewer)

I hope this can inspire you to find areas around your daily work that can be improved by automation, and try to solve it with modern tools such as GitHub actions, CI, webhook and API provided by different platforms. It’s a great way to improve velocity of your software project, and a chance to try technology that otherwise you won’t come across.