ConsoleMe: A Central Control Plane for AWS Permissions and Access

By Curtis Castrapel, Patrick Sanders, and Hee Won Kim

At AWS re:Invent 2020, we open sourced two new tools for managing multi-account AWS permissions and access. We’re very excited to bring you ConsoleMe (pronounced: kuhn-soul-mee), and its CLI utility, Weep (pun intended)!

If you missed the talk, check it out here.

Motivation

Growth in the cloud has exploded, and it is now easier than ever to create infrastructure on the fly. Groups beyond software engineering teams are standing up their own systems and automation. This is an amazing movement providing numerous opportunities for product innovation, but managing this growth has introduced a support burden of ensuring proper security authentication & authorization, cloud hygiene, and scalable processes.

At many companies, managing cloud hygiene and security usually falls under the infrastructure or security teams. They are the one-stop-shop for cloud permissions and access. As the company scales, this centralized and manual management approach falls over, becoming impractical for both operations teams and their users.

This happened for us at Netflix. Our Cloud Infrastructure Security team is the arbiter of AWS permissions, handling numerous requests from Netflix employees for cloud permissions and access. Our historical approach of helping Netflix internal cloud users looks something like this:

1. A user messages us in our support channel
2. We clarify what the user needs, and why. Risks are analyzed, and we make suggestions of better approaches if applicable.
3. We hand-craft an IAM policy for the end-user
4. We log into the AWS account with the applicable IAM role and manipulate the policy
5. If the request involves a cross-account resource, we log out of the AWS account, log in to the account with the resource, and manipulate the resource policy
6. We ask the user to test
7. The user comes back with an additional permissions error
8. We play a game of permission whack-a-mole (Steps 3–7) until we resolve all of the user’s issues
9. We repeat this multiple times a day with different users.

This process is manual, time-consuming, inconsistent, and often a game of trial and error.

At Netflix, we’re firm believers in empowering our employees and providing low-friction systems that allow users to get their jobs done in a safe way. By integrating best practices such as least privilege into an IAM pipeline, we transitioned the security team from being gatekeepers of the cloud into cloud development accelerators.

What is ConsoleMe?

ConsoleMe is a self-service tool for AWS that provides an easier way of managing permissions and access across multiple accounts, while encouraging least-privilege permissions. Users can use the following features:

• Access the AWS console
• Retrieve and utilize short-lived AWS credentials through Weep
• Request IAM permissions through a self-service wizard
• Utilize ConsoleMe’s native policy editors for more advanced requests
• Quickly locate and navigate to AWS resources within an organization

In addition, cloud administrators can use ConsoleMe to:

• Manage IAM and resource policies without logging in to the AWS Console
• Create or clone IAM roles across accounts

Check out the demos in our documentation, then try it locally with your own account.

Access the AWS console

(docs, talk)

ConsoleMe allows users to access the AWS console through the use of temporary IAM role credentials. After the user authenticates, ConsoleMe determines which roles they’re authorized to access based on their identity and group memberships.

ConsoleMe generates an authorization mapping that is used to determine which users/groups are allowed to access a given IAM role. This mapping can be generated through role tags that indicate which users/groups are allowed to retrieve credentials for the role, ConsoleMe’s Dynamic Configuration, or through an organization’s custom logic. (docs).

Users have a number of ways they can log in to the AWS console. The simplest way is by browsing to ConsoleMe and clicking on the desired role via the web interface. They can also use URL parameters to log into a particular role, access a specific region, AWS service, or AWS resource without having to navigate around ConsoleMe’s web interface.

At Netflix, we’ve seen users integrate ConsoleMe with productivity tools like Alfred, chat bots, and custom browser search engines.

Retrieve and serve short-lived AWS credentials through Weep

(docs, talk)

Weep is ConsoleMe’s CLI utility. It retrieves temporary (1-hour) AWS credentials from ConsoleMe, and offers a number of different ways to serve them locally. Weep can automatically refresh credentials. This ensures that long-lived AWS actions are successful (Like an s3:GetObject action taking longer than an hour). Weep can also transparently perform nested AssumeRole calls, and serve the assumed role credentials to the local user. Credentials are discoverable by the AWS CLI and AWS SDKs through the default credential provider chain.

Weep supports the following methods of serving credentials:

1. Write credentials to a user’s ~/.aws/credentials file
2. Export credentials as environment variables
3. Emulate the EC2 instance metadata proxy
4. Emulate the ECS credential provider
5. Generate and provide credential_process commands to source credentials

Request IAM permissions through a self-service wizard

(docs, talk)

ConsoleMe provides a step-by-step self-service wizard to help users request AWS IAM permissions.

Users no longer need to worry about the IAM JSON permissions syntax. They can simply search for their role and choose the permissions they need. ConsoleMe will generate an IAM policy and, if required, cross-account resource policies that are applicable to the request. Users can modify the generated policy if they desire, and then submit for approval.

ConsoleMe’s configurable self-service wizard offers the following features:

1. Fully configurable based on an organization’s most common requests
2. Typeaheads against all known AWS permissions and resource ARNs across an organization
3. Automatic approval of low-risk permission requests, governed by ConsoleMe’s configuration and powered by Zelkova

ConsoleMe’s self-service wizard has reduced our response time in servicing access requests, provided more consistency in our IAM policies, and simplified AWS permissions for our users.

Utilize ConsoleMe’s native policy editors for advanced requests

(docs, talk)

ConsoleMe offers a native policy editor for popular resource types. Administrators use it to manage permissions and tags for common resource types. End-users can manipulate a resource and submit policy change requests.

The policy editor offers the following features:

1. Cloud administrators can manage resource policies and tags directly
2. End-users can manipulate policies and tags, then submit changes for approval
3. Code editors provide typeaheads for AWS permissions and known AWS resources
4. Policy templates make it easy to generate new inline policies consistently
5. Users can view recent CloudTrail errors for a given resource

Today, ConsoleMe supports a small number of popular resource types. We’d love your help with adding support for new resource types. Reach out to us on Discord or better yet, create an issue or submit a pull request on GitHub.

Quickly locate and navigate to AWS resources within an organization

(docs, talk)

ConsoleMe provides a centralized, filterable view of your most critical cloud resources, synchronized from AWS Config. It allows users to quickly find an AWS resource across all of the accounts within an organization.

For resource types that ConsoleMe doesn’t have native policy editors for, ConsoleMe provides a link that will both log users into the AWS console and redirect them to the appropriate resource.

Create or clone IAM roles across accounts

(docs)

ConsoleMe makes it easy for cloud administrators to create or clone new IAM roles across multiple AWS accounts. We created this feature because we found ourselves in the AWS Console copying and pasting various policies by hand.

The clone feature can copy one or more of the following to a new role:

1. IAM role Trust Policies (Assume Role Policy Document)
2. Description
3. Inline Policies
4. Managed Policies
5. Tags

How does ConsoleMe encourage least-privilege permissions?

At Netflix, we use IAM roles instead of IAM users because roles do not allow long-lived, static credentials. IAM user credentials are more vulnerable to accidental exposure, difficult to rotate, and generally harder to secure.

In addition, we prefer using inline policies instead of managed policies for our IAM roles because it’s easier to enforce least-privilege as inline policies are specific to an IAM role while managed policies can be attached to multiple roles. It’s hard to remove permissions from shared managed policies because some roles may be using permissions from the policy that other roles are not.

We use ConsoleMe in conjunction with RepoKid to remove unused permissions, and then to make the process of requesting them back as painless as possible.

Getting Started

ConsoleMe is available on GitHub (Give us a ★!). You can try out ConsoleMe using Docker. A quick start guide is available in our documentation.

ConsoleMe has example Terraform files that you can reference when you’re ready to deploy.

Contributing

ConsoleMe still has a long way to go, and we could use your help. ConsoleMe and Weep work great for us here at Netflix, and we want them to work great for everyone else too. The best way to get started is to read through the documentation and code, install ConsoleMe, and take a look at our open issues to see what work needs to be done, or submit issues yourself.

Not a coder or an IAM expert? No problem. We have a lot of documentation that could use proofreading and clarifying to make it more approachable.

For more information on how you can get involved, check out our Contributing guide.

Also, we’re hiring! If you’re interested in these sorts of problems, take a look at https://jobs.netflix.com/teams/security, and apply.

What’s next?

Over the last couple of years, we’ve battle tested ConsoleMe and have added features to scale it with our needs at Netflix. We’ve now brought ConsoleMe out in the open. As companies adopt ConsoleMe, we want to continue growing it to address the unique challenges of large-scale cloud permissions management that many of us face.

We have a lot of plans for the future of ConsoleMe. Many of these goals are ambitious, and we can’t do it without your support. If any of these excite you, please reach out to us on our Discord channel or submit feature enhancements on GitHub.

Some of the ideas we have in mind are:

Easier Permissions Debugging

AWS permissions can be hard to debug with opaque Access Denied errors. We aim to simplify and automate the debugging process. This might include exposing and connecting information from the following sources:

• CloudTrail logs
• Service Control Policies
• Resource policies
• Permission boundaries
• Session policies
• Inline Policies
• Managed Policies

Ideally, users would be able to ask ConsoleMe whether an IAM role can take a specific action on a given resource. If not, ConsoleMe would provide an explanation and context about any policies that are preventing the action.

Support for Team Roles

We plan to add features supporting the creation and management of team roles. Team roles are IAM roles that an entire team has access to. These roles can be propagated across multiple accounts, and can have differing permissions on each account. A simplified management interface will make it easy to create, request, or modify a team role.

Enhanced Cross-Account Policy Generation

ConsoleMe only supports cross-account policy generation for a subset of resource types. We hope to expand this in the future and make generated policies as accurate as possible by adding awareness of permission boundaries and service control policies.

Decentralized Policy Request management

Cloud administrators should have the option to no longer manage and review all policy requests. If ConsoleMe has context on the owner of a resource, and is able to determine that the policy is within a set of defined safety limits, policy requests should be routed to the owners of the resources affected by the policy.

Policy Rollback

On occasion, we need to rollback policy changes that either break an IAM role or prevent new functionality from working. ConsoleMe should allow users to revert a role to an older snapshot.

Multi-Cloud Support

Centrally manage access and permissions across all of your clouds.

Where can I learn more?

Here are some helpful resources:

• ConsoleMe source code on GitHub
• Weep source code on GitHub
• Documentation for ConsoleMe and Weep, including our Contributing guide
• Chat with us on Discord
• Join our team!

Special Thanks

We would like to give a special thanks to Srinath Kuruvadi, Jay Dhulia, the Cloud Infrastructure Security Team at Netflix, the Infosec team at Netflix, and our AWS partners.

We’d also like to thank our contributors for both ConsoleMe and Weep.