Pipeline Groups in Dynatrace OpenPipeline: Enterprise-grade governance explained

As organizations scale their observability practice, a familiar tension emerges: platform engineering teams need to enforce consistent configuration for pipeline ingestion, such as security context, cost allocation, and compliance-driven routing, while application teams need the freedom to tailor data processing for their specific workloads.

In practice, most organizations attempt to solve this with one of two compromises: lock everything down centrally and become a bottleneck for every pipeline change, or copy rules across pipelines and hope nothing drifts. Neither approach scales. And as data requirements shift faster than ever to address new services, new regulatory mandates, and new object types to observe, the cost of such compromise keeps growing.

Pipeline Groups change that dynamic. Generally available now in Dynatrace SaaS version 1.332, Pipeline Groups let platform engineering teams, the teams with admin permissions over the observability platform's configuration, mandate and standardize ingest behavior across many pipelines, while safely delegating day-to-day configuration of individual pipelines to the teams best suited to own them.

The problem: Pipeline governance at scale

In large enterprises, dozens of teams may each operate their own custom pipelines. Common requirements, security context enrichment, cost allocation tagging, and storage bucket assignment must be applied consistently across the board. However, without a structured mechanism to enforce such standards, platform teams face a choice that gets harder with every new team, every new data source, and every new regulation:

• Lock everything down and become the gatekeeper for every pipeline change. Every adjustment, no matter how small, becomes a ticket, a review, a delay. Innovation stalls. Teams wait days for changes that should take minutes.
• Copy rules everywhere and accept the risk of configuration drift. What starts as a manageable set of shared rules gradually fragments-one team forgets to apply the latest cost allocation tag, another skips the security enrichment step, a third routes data to the wrong bucket. The inconsistencies compound silently until they surface as compliance gaps or billing surprises.

Customers consistently tell us that this approach doesn't scale. What they need is a way to separate what must always happen from what teams should be free to decide, and to encode that separation directly into the platform, not just into process documents that team members forget to follow.

What are Pipeline Groups?

A Pipeline Group is a first-class configuration object that separates a global pipeline list into distinct sets; the pipelines in these sets are then members of the group. A Pipeline Group orchestrates a special type of reusable pipeline to define what happens before or after the stages of the member pipelines.

Pipeline Groups allow you to:

• Organize pipelines into groups with clearly defined membership.
• Define execution order so processing happens in a predictable, layered sequence.
• Control stage execution for the member pipelines so that teams can turn on or turn off specific pipeline stages.
• Enforce mandatory global processing that no team can bypass or override.
• Allow team-level customization within boundaries defined by the platform team.

Pipeline Groups determine how data flows through your pipelines. This ensures that governance isn't an afterthought bolted onto a pipeline configuration, but rather it's built into the execution model itself.

How Pipeline Group ownership works

The design behind Pipeline Groups draws a deliberate line between group-level configuration and member-pipeline configuration. How organizations map this division to team ownership is up to them. Pipeline Groups provide the mechanism, not a prescriptive policy. The pattern we see most often is straightforward: platform or SRE teams own the pipeline groups, and application teams own their member pipelines within those groups.

To make this more concrete, consider a platform team that's responsible for observability across multiple business units. They create a Pipeline Group that adds a business segment field to every record for organizational attribution, applies cost allocation tags for accurate chargeback, sets security context so sensitive data is handled consistently, and assigns data to the correct storage bucket based on retention and compliance needs. These are the rules that must always apply, and because they operate at the group level, no member pipeline can bypass or override them.

Member pipeline ownership can be more granular than a simple admin-vs-team split. While a group is always admin-owned, the individual pipelines that make up the Pipeline Group can each have different owners. For example, a pipeline that enriches every record with organizational metadata or classifies data sensitivity might be globally relevant and owned by the platform team, or by a specialist team like the security team. A pipeline that handles domain-specific compliance logic, such as PCI field masking for a financial services division, might be owned by that compliance team because they have the expertise that the platform team lacks. This arrangement reflects how responsibility within that organization is distributed.

Within that group, application teams each get their own member pipeline. One team configures parsing for a specific log format. Another sets up filtering to reduce noise. A third extracts metrics tailored to their services. Each team works independently within its own pipeline without touching the platform-level rules above and without needing to coordinate with other teams or the platform team for routine changes.

The value of this separation becomes clear when things change, as they do frequently in large enterprises. Say that one of those business units onboards a new microservice that generates a completely new log format. Under a centralized-only model, the team would file a request, wait for the platform team to update the pipeline, validate, and iterate. With Pipeline Groups, the team simply adds their parsing and filtering rules to their own member pipeline. The mandatory enrichment, cost tagging, and routing are already guaranteed by the group. The new service is observable in hours, not weeks, and the platform team doesn't need to be involved at all.

Or picture the reverse direction: a new regulatory requirement lands, say, a mandate that all log data from EU-based services must be assigned to region-specific storage. The platform team updates the group configuration once. Every member pipeline inherits the change immediately.

Why the flexibility of Pipeline Groups matters

The pace at which enterprise data requirements evolve has fundamentally changed. New services are spun up in days, not months. Regulatory landscapes shift across jurisdictions. The types of software entities that organizations need to observe, from traditional infrastructure to AI model outputs, edge devices, and third-party SaaS telemetry, keep expanding. In this environment, a rigid, centralized-only pipeline configuration becomes a constraint rather than a benefit.

Pipeline Groups are built for this reality. They give enterprises a mechanism that:

1. Creates room for innovation. Application teams can iterate on their pipeline configurations independently, experimenting with new parsing rules, extracting new metrics, and adapting to new data shapes without waiting for central approval on every change.
2. Eliminates bottlenecks. Platform teams define the rules once and let the system enforce them. Teams are freed from serving as gatekeepers for routine changes and can focus on architecture, standards, and strategy.
3. Ensures that ever-changing guardrails are applied. As compliance requirements, security policies, or cost structures evolve, platform teams can update them all in one place. The changes propagate automatically to every pipeline in the Pipeline Group.

Pipeline Groups give platform teams the governance controls and flexibility they need

Pipeline Groups give enterprise platform teams the governance controls they've been asking for, mandatory processing, centralized ownership of sensitive stages, and structured delegation, all without taking away the flexibility that makes Dynatrace OpenPipeline® valuable to individual teams in the first place.

Ready to get started with OpenPipeline and Pipeline Groups?

If your organization manages observability across multiple teams and struggles with consistency, Pipeline Groups are now generally available for you to try.

Learn how to define global guardrails once and empower your application teams to build their own pipelines confidently within standardized boundaries.