How to Setup and Use the Gremlin Slackbot

Introduction

Slack is a communications platform to enable teamwork. If you use Slack, it’s useful to create a channel for your Engineering team to see the Gremlins in action when they are running attacks. This tutorial will explain how to setup and use the Gremlin Slackbot.

Prerequisites

Before you begin this tutorial, you’ll need the following:

  • A Gremlin account
  • A Slack account
  • Gremlins unleashed on your infrastructure

Step 1 - Setting up the Gremlin Slackbot

In this step, you’ll setup the Gremlin Slackbot application.

Login to the Gremlin Control Panel using your Company name and sign-on credentials. These details were emailed to you when you signed up to start using Gremlin.

Next click on your name and select Settings in the Gremlin Control Panel. Then click on the “Add to Slack” button.

Select the Slack room for the Gremlin Slackbot to join using the dropdown menu in the top right of the screen. In this example, we are connecting the Slackbot to the Chaos Engineering Community Slack. Next, select which channel the bot can post to, in this example we have selected the #gremlin-attacks channel. Then click Authorize.

Step 2 - Validating Installation

After installing the Gremlin Slackbot you will see a message explaining the Slack integration is connected.The Gremlin Slackbot will join the channel you selected, in this example #gremlin-attacks.

Step 3 - Receive Slack notifications for Gremlin attacks

Your Gremlin Slackbot is now ready for you to use.

You will now be notified in Slack for:

  • Started Gremlin attacks
  • Successful Gremlin attacks
  • Halted Gremlin attacks

You can now click from within Slack to:

  • Rerun Gremlin attacks
  • Show logs for Gremlin attacks
  • Halt Gremlin attacks

Example: Create a Gremlin CPU Attack

Select Create Attack in the Gremlin Control Panel.

The “Hello World” of Chaos Engineering is the CPU Resource Attack. To create a CPU Resource Attack select “Resource” and then “CPU” in the dropdown menu.

The CPU Resource Attack will consume CPU resources based on the settings you select. The most popular default settings for a CPU Resource Attack are pre-selected, a default attack will utilize 1 core for 60 seconds. Before you can run the Gremlin attack you will need to click either Exact hosts to run the attack on or click the Random attack option.

Click Exact and select a Gremlin Client in the list.

Finally, click Create. Your attack will begin to run, you will be able to view its progress via Gremlin Attacks in the Gremlin Control Panel.

On your server, run top to check the impact of the Gremlin Attack:

$ top

top - 06:26:47 up 7 days,  7:00,  1 user,  load average: 0.28, 0.07, 0.02
Tasks: 105 total,   1 running, 104 sleeping,   0 stopped,   0 zombie
%Cpu(s): 79.7 us, 20.3 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :  1016120 total,   127140 free,    93956 used,   795024 buff/cache
KiB Swap:        0 total,        0 free,        0 used.   712192 avail Mem 

  PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND     
23768 gremlin   20   0   13268  11136   3576 S 99.3  1.1   0:14.05 gremlin     
23766 root      20   0   40388   3600   3072 R  0.3  0.4   0:00.03 top         
    1 root      20   0   37760   5760   3940 S  0.0  0.6   0:13.74 systemd     
    2 root      20   0       0      0      0 S  0.0  0.0   0:00.00 kthreadd    
    3 root      20   0       0      0      0 S  0.0  0.0   0:01.28 ksoftirqd/0 
    5 root       0 -20       0      0      0 S  0.0  0.0   0:00.00 kworker/0:0H
    7 root      20   0       0      0      0 S  0.0  0.0   0:06.14 rcu_sched   
    8 root      20   0       0      0      0 S  0.0  0.0   0:00.00 rcu_bh      
    9 root      rt   0       0      0      0 S  0.0  0.0   0:00.00 migration/0 
   10 root      rt   0       0      0      0 S  0.0  0.0   0:04.09 watchdog/0  

When your attack is finished it will move to Completed Attacks.

View and interact with the attack using the Gremlin Slackbot

Open the Slack channel where you enabled the Gremlin Slackbot to post. Then click on Show. The Gremlin app will open and you can now see the configuration, details and logs related to your CPU Attack.

Next, click on “Rerun” in the Slack post. This will open the Gremlin app and enable you to rerun the same CPU attack.

When the attack succesfully completes it will the Gremlin Slackbot will post a message in the channel.

It is also possible to halt the attack from within Slack. To halt the attack, click on halt in the Slack post. This will open a halt popup box as shown below. Next click ok, the attack will be halted.

Conclusion

You’ve setup the Gremlin Slackbot and validated that it works by creating a CPU attack. You now possess tools that make it possible for you to share the Gremlin attacks that are running on your infrastructure with your team via Slack.

Gremlin’s Developer Guide is a great resource and reference for using Gremlin to do Chaos Engineering. You can view additional Gremlin Attacks including attacks that impact State and Network. You can also explore the Gremlin Blog for more information on how to use Chaos Engineering with your application infrastructure.