OH MY DC: OIDC Misconfigurations in CI/CD

Executive Summary

In the course of investigating the use of OpenID Connect (OIDC) within continuous integration and continuous deployment (CI/CD) environments, Unit 42 researchers discovered problematic patterns and implementations that could be leveraged by threat actors to gain access to restricted resources. One instance of such an implementation was identified in CircleCI's OIDC.

OIDC extends the OAuth protocol by adding a new token to the protocol, enabling applications to verify user identities and authorize access to resources using that token. It plays a crucial role in ensuring secure and seamless authentication and authorization during CI/CD processes. Securing these implementations is critical, as OIDC is rapidly being adopted as the primary foundation for modern cloud authentication workflows. In this article, we discuss potential critical security risks in OIDC implementation and usage.

Our analysis revealed three key threat vectors:

• Loosely configured policies used by identity federations
• Reliance on user-controllable claim values
• Ability to leverage poisoned pipeline execution (PPE) in combination with permissive identity federation

We urge organizations to review and strengthen their OIDC policies, implement strict claim validation and enhance CI/CD security practices.

CircleCI also shared a response describing how they addressed the issue and offering recommendations to improve security.

Palo Alto Networks' Cortex Cloud and Prisma Cloud offerings provide protection against these threats through comprehensive cloud security features.

If you suspect a compromise or have an urgent security matter, contact the Unit 42 Incident Response team immediately.

OIDC Overview

This paper accompanies the presentation "Oh-My-DC," delivered at DEF CON 32 in August 2024. This article assumes a basic familiarity with OAuth and CI/CD pipelines, including concepts like authorization grants, access tokens and the different stages of a CI/CD workflow.

Readers unfamiliar with these concepts may find the following resources helpful:

What Is OIDC?

OIDC extends the OAuth protocol by adding a new token to the protocol, enabling applications to verify user identities and authorize access to resources using that token. The protocol comprises two key parts:

• Authentication
• Authorization

Considering the authentication, the OIDC protocol performs the same OAuth authentication flow, with an additional requirement that the identity provider (IdP) issues an ID token to the client after successful authentication.

This ID token, formatted as a JSON Web Token (JWT), contains values (called claims) about the just-authenticated identity (e.g., their username, email) and some other vendor-specific information we will touch on later.

Per the authorization part, it occurs when the client attempts to access a resource, usually on the cloud side. At this point, the resource's identity federation policy comes into play. This policy verifies the claims within the received ID token against a predefined set of rules. If the claims in the ID token satisfy the policy, the client is granted access to the resource.

Figure 1 illustrates the OIDC protocol, highlighting the authentication and authorization components.

Having covered the basic OIDC flow, we can now examine its application within CI/CD environments.

OIDC in CI/CD

Credential leaks and authentication vulnerabilities within CI/CD systems have led to numerous breaches in recent years. These incidents directly correlate with the increasing interactions between CI pipelines and external resources, particularly cloud resources.

The OIDC protocol addresses this challenge by eliminating the need for password-based authentication when CI machines interact with external resources. This follows the idea of "if you don't have a password, you can't leak it."

Instead, the protocol relies on identity tokens that CI machines acquire through the OIDC flow. In essence, OIDC enables passwordless interaction between our CIs and resources.

A key challenge in implementing OIDC within CI/CD lies in the ephemeral nature of CI runners, which lack persistent credentials. How then do these runners obtain identity tokens?

The answer lies in the fundamental CI/CD architecture. The CI/CD vendor is the only entity that can authoritatively identify a runner, as it's the sole party aware of the newly provisioned ephemeral machine.

This architectural reality positions the CI/CD vendor as the natural IdP in the OIDC flow.

This vendor-as-IdP model has significant security implications, namely bypassing the authentication phases as well as requiring at least two more configuration points. These will be addressed later in the case study.

Let's summarize how OIDC integrates with CI environments. When a workflow is triggered, the CI/CD vendor provisions a runner. Acting as an IdP, it then issues a signed ID token (i.e., OIDC). The machine will later use this token to identify itself when attempting to gain authorization to access protected resources throughout the build lifecycle.

Understanding OIDC Claims

As mentioned above, the ID token is a JWT that contains values called claims. Down the line, the identity federation will assess these claims to determine whether the incoming token grants access to a requested resource. Generally speaking, claims provide essential information about the authentication event and the authenticating user.

Required claims include the information listed in Table 1.

iss	The token issuer (the IdP)
sub	The token's subject identifier
aud	The expected audience
exp	Token's expiration time
iat	Issuance time

Table 1. The required claims as defined by the OpenID Connect Core 1.0 specification.

Additional claims can convey the following information:

• User attributes (e.g., email, name)
• Context-specific information (e.g., repository details, pipeline data)
• Role-based access control (RBAC) attributes

These claims form the foundation of trust between the IdP and relying applications, enabling user authentication and authorization decisions.

The Risks of OIDC Misconfigurations

OIDC misconfigurations can be critical, as we've just learned. OIDC's role is to ultimately safeguard our potentially valuable resources. Therefore, if OIDC is misconfigured at any of its points (either the authentication or authorization), an attacker can potentially abuse misconfigurations and access resources with the same permissions as authorized users.

This is particularly concerning in CI environments, where build systems typically have broad access to sensitive resources and systems like the organization's cloud environment or back-office software. A single misconfiguration could potentially expose multiple downstream resources to unauthorized access.

Therefore, proper OIDC configuration requires careful attention to both the authentication and authorization components, as a weakness in either can compromise the entire security model.

OIDC Misconfigurations

When examining OIDC misconfigurations and their potential exploits, we can see that the authorization phase is the Achilles' heel of CI-based OIDC implementations.

As Figure 3 shows, the CI vendor serves as both the IdP and the machine provisioner. This dual role means the authentication happens automatically when the CI vendor creates a runner.

This architectural design has an important implication. Every CI runner that supports OIDC automatically receives an identity token signed by the vendor. However, this is true not just for our runners, but for every user of that CI platform. In other words, any customer of the same CI vendor can obtain machine-identity tokens bearing the vendor's signature.

From a security standpoint, this means that any other customer of the CI/CD vendor automatically satisfies the authentication requirements of your OIDC configuration. This is precisely why authorization becomes our most sensitive security control. While not inherently vulnerable, it is susceptible to misconfigurations that can have severe consequences.

To understand these architectural implications, let's examine common OIDC misconfigurations and their potential impact.

OIDC Misconfiguration #1: Missing or Permissive Identity Federation Policies

A critical OIDC misconfiguration occurs when identity federation policies are either missing or too permissive. This happens when policies exist but fail to enforce meaningful validation on the OIDC token claims.

We can observe this in two main scenarios:

• Overly permissive conditions, such as accepting any audclaim value or asserting the existence of a subclaim
• Validating claims that are always true within the given context, such as checking if the subclaim starts with repowhen all tokens from that issuer inherently have this prefix

These scenarios, shown in Figure 4, demonstrate how the default or seemingly valid configurations might offer no real security guarantees as they fail to properly validate the token's claims.

For further information on this topic, please consult the following videos:

Exploiting Lax Federation With PPE

Combining poisoned pipeline execution (PPE) with lax OIDC federation policies allows attackers to escalate privileges within organizations that rely on overly broad trust relationships. This attack exploits remote code execution (RCE) vulnerabilities in a CI/CD pipeline to obtain OIDC tokens that meet the lax federation requirements, potentially granting unauthorized access to sensitive resources.

Let's examine how this works in practice with a real-world scenario.

Imagine an organization with two repositories, each having a distinct security posture and purpose:

• Repository A: This is a standard development project with a CI/CD pipeline vulnerable to PPE. This vulnerability might initially appear low-risk, as the repository has limited permissions and no direct access to sensitive resources. The vulnerability could be as simple as a script injection flaw in a build step.
• Repository B: This is a critical production repository that manages sensitive cloud resources via OIDC authentication. The security issue arises from the OIDC federation configuration for Repository B's cloud resources. Although Repository B's pipeline is secure, its federation policy, using the pattern repo:my_org/*is overly permissive. This broad access pattern, likely chosen for convenience, allows any repository within the organization to potentially meet the federation requirements. This creates a significant security risk.

The attack flow proceeds as follows:

• Exploit the PPE vulnerability in Repository A's pipeline
• Execute arbitrary code in the compromised CI/CD environment
• Obtain the ID token from the exploited machine. This token now has a subclaim of the format repo:my_org/repo_A:...
• Use the fraudulently obtained token to access Repository B's cloud resources by claiming the identity due to the lax identity policy that grants access to any repository in the organization

This attack demonstrates why granular OIDC federation policies are crucial. A single vulnerable pipeline, even in a seemingly low-risk repository, combined with broad federation patterns can compromise your entire security boundary.

Organizations should implement strict, repository-specific federation policies. Avoid relying on organization-wide patterns or patterns derived from user input, as these significantly increase the risk of unauthorized access.

OIDC Misconfigurations #2: User Side - Relying on User-Input Claims

A dangerous OIDC misconfiguration occurs when identity federation policies trust claims that end users can manipulate. While these claims are legitimately part of the OIDC token, their values should not be used for critical security decisions since they originate from user-controlled inputs.

A classic example is trusting the workflowclaim in GitHub Actions OIDC tokens. While this claim exists in the token and is signed by GitHub, its value is simply derived from the workflow filename. This is a parameter that any user in GitHub can create and by this gain access to the resource as well. Basing access decisions on such claims is equivalent to letting users write their own permissions.

Other examples might include:

• The branch name (ref claim): An attacker could create a branch with a name that matches a privileged access pattern, thereby gaining unauthorized access to resources protected by that pattern
• The environment name: Similar to the branch name, an attacker could manipulate the environment name to escalate privileges
• A specific audvalue: The audvalue is mostly predictable or user-controllable, it can be spoofed to bypass authentication checks

The key issue isn't about the validity of these claims. They are all properly validated by the IdP. Rather, it's about relying on claims whose values other users of the same platform can freely choose.

For a comprehensive analysis of GitHub Actions OIDC claims and their safety assessment, we've created a reference project. This project provides a detailed breakdown of each claim's safety for identity federation, helping you make informed decisions about which claims to trust in your OIDC configurations.

OIDC Misconfigurations #3: Risks of Custom Sub Claims

While studying user-input claims in OIDC, we discovered a security implication regarding the custom subclaim feature. This feature, intended to provide flexibility, allows users to construct their subclaim using various ID token claims. However, this flexibility can inadvertently introduce security risks.

Although designed to enhance security by providing granular control over token validation, the custom subclaim feature can be misused. For example, GitHub's implementation allows users to construct their subclaim using any available claim, regardless of its origin or security implications. This can inadvertently expose sensitive resources.

Consider two scenarios that highlight this risk:

• User-controllable claims: If a repository owner sets their custom subto only include the value of the workflowclaim, they're essentially validating user-controlled input (as previously explained) and are vulnerable. (This fact is true for other claims as well and not unique to the workflowclaim.)
• Claim ordering sensitivity: The order of claims in the custom subbecomes security-critical. This is because if a user-controllable claim is placed at the beginning of the substring, an attacker can potentially manipulate it to mimic a legitimate subclaim. For instance:
  • Safe: repo:org_name/repo_name:workflow:my_workflow
  • Unsafe: workflow:my_workflow:repo:org_name/repo_name

The second example format (Unsafe) is vulnerable because an attacker could create a workflow named my_workflow:repo:org_name/repo_nameand set their OIDC subformat to workflow. This generates a token with a subclaim of workflow:my_workflow:repo:org_name/repo_nameand allows the attacker to gain a token that mimics that of the target.

Note that the workflowclaim used for the above example is not the only unsafe claim and there are more that one needs to be aware of.

For users of the custom sub claim feature, we've developed a tool called GitHub OIDC Utils that can help organizations to assess their sub claim format for this type of misconfiguration. The tool highlights custom claims that attackers can abuse. Consult the project's documentation for usage instructions.

OIDC Misconfigurations #4: Vendor-Side Credential Handling

Let's look at an interesting case from CircleCI's initial OIDC implementation that demonstrates how vendor-side misconfigurations can impact customers' security.

Our investigation into CircleCI's OIDC implementation revealed unexpected behavior in fork-based pull requests. During a fork pull request, the CI naturally runs in the target repository's context. As a result, CircleCI would generate OIDC tokens containing the target's identity and their repository information, and then it would provide them to the fork's workflow. This effectively grants fork owners the same level of access as the target repository.

This meant that anyone could:

• Fork any public repository using CircleCI
• Submit a Pull Request
• Obtain OIDC tokens with the target repository's identity
• Access any resources configured to trust these tokens

After reporting this security issue to CircleCI, they resolved this vulnerability by disabling OIDC token generation in fork workflows by default. Tokens are now generated only when explicitly enabled via the Pass Secrets to Forked Builds setting.

This case serves as an excellent example of how vendor implementations are also prone to misconfigurations when setting up OIDC as the CI vendor.

Conclusion

OIDC in CI represents a significant advancement in securing CI/CD pipelines by eliminating the need for stored credentials. OIDC can help to reduce the impact in instances like the recent tj-actions/changed-filesincident, where pipeline secrets were printed to the logs. However, our research reveals several critical security implications that organizations need to consider when implementing OIDC authentication in their CI environments.

The key findings from our investigation demonstrate that:

• The dual role of CI vendors as both runner provisioners and IdPs creates unique security considerations
• Authorization configurations are particularly prone to misconfigurations for OIDC in the context of CI
• Lax policies sometimes equal vulnerable policies
• User-controlled claims and custom configurations require careful validation

Looking ahead, as organizations increasingly adopt OIDC for CI/CD security, we anticipate these challenges will become more prevalent. The combination of PPE vulnerabilities with permissive OIDC policies particularly highlights the need for defense-in-depth approaches.

For organizations implementing OIDC in CI environments, we recommend:

• Using repository-specific federation rules instead of organization-wide patterns
• Implementing strict claim validation, especially for user-controllable claims
• Regularly auditing OIDC configurations, focusing on federation policies and custom claim formats
• Following CI security best practices to prevent PPE vulnerabilities from being exploited via OIDC

Palo Alto Networks Protection and Mitigation

For existing customers, we have updated our Infrastructure as Code (IaC) policies to be able to identify these types of OICD misconfigurations, and to alert the user if any are detected.

As customers are upgraded from Prisma Cloud to Cortex Cloud, they can benefit from all existing protections. For example, the screenshot in Figure 8 below demonstrates the tool detecting a claim format that could be utilized for fraudulent purposes.

If you think you may have been compromised or have an urgent matter, get in touch with the Unit 42 Incident Response team or call:

• North America: Toll Free: +1 (866) 486-4842 (866.4.UNIT42)
• UK: +44.20.3743.3660
• Europe and Middle East: +31.20.299.3130
• Asia: +65.6983.8730
• Japan: +81.50.1790.0200
• Australia: +61.2.4062.7950
• India: 00080005045107

Palo Alto Networks has shared these findings with our fellow Cyber Threat Alliance (CTA) members. CTA members use this intelligence to rapidly deploy protections to their customers and to systematically disrupt malicious cyber actors. Learn more about the Cyber Threat Alliance.

CircleCI's Response

CircleCI prioritizes providing customers with default settings that make it intuitive to restrict access to all jobs on our platform.

At the time Palo Alto Networks Unit 42 researchers first reported their findings to CircleCI, CircleCI's default offering was an OIDC environment variable for all authorized CircleCI jobs, requiring additional action by the customer to grant or deny access. No vulnerabilities were identified by our customers.

On June 13, 2023, CircleCI introduced the ability to exclude OIDC tokens from forked builds by default. This default setting locks down the permission to invoke, which forces the customer to take an action if they want to share the access more broadly. For those customers who wish to allow OIDC tokens in forked builds, CircleCI has provided documentation that explains how to do this:

CircleCI advises all customers to employ the best practices of managing identity and access roles outside of their environmental variables.

References