Native Observability

🚀 Overview

Native Observability brings real observability into Drupal.
Trace requests, inspect execution, analyze performance, export telemetry, and understand runtime behavior directly inside your application.

No core patches required. Install, enable, and start seeing what actually happens inside your Drupal system.

✨ What you get

• 🔁 Request tracing with correlation IDs
• 🧩 Execution spans with timeline visualization
• 🌐 External HTTP (Guzzle) tracing
• 📊 Performance metrics including latency, throughput, P95 and P99
• 📈 Dashboard with cache, database, runtime, technical insights, and route-level actions
• 📝 Structured reports with printable HTML output
• 📡 Telemetry export for OpenTelemetry and Prometheus-based stacks
• 🔕 Request-log exclusion rules to suppress noisy or unwanted traffic from observability data
• 🔔 ECA integration to react to observability events inside Drupal

🆕 Native Observability 1.0.7

Native Observability 1.0.7 introduces a full request-log exclusion workflow and improves the dashboard operator experience.

This release focuses on three major areas:

• 🔕 Request-log exclusion as a first-class core feature with centralized rule management, runtime-safe matching, and suppression across traces, spans, metrics, and exports
• 📈 Better dashboard usability with direct route actions, re-enable flows, deep-linking to exclusions, and improved handling of unresolved external requests
• 📚 Stronger documentation and runtime safety with deeper operator guidance, PHPUnit coverage, and a cache-first architecture for exclusion rules

🔕 Request-log exclusion

Version 1.0.7 adds a dedicated request-log exclusion system directly in the Native Observability core.
This makes it possible to prevent noisy, irrelevant, synthetic, or infrastructure-driven requests from polluting observability data.

Exclusion rules are managed from the Native Observability settings area and are designed to work across the full request-correlated pipeline.

• 🧭 Match requests by HTTP method
• 🛣 Match routes and paths by exact, prefix, or regex pattern
• 🔎 Match query parameters using operators such as exists, missing, equals, regex, gte, lte, and between
• 🚫 Suppress excluded traffic from traces, spans, metrics, observers, and telemetry exports
• ⚡ Use a cache-first runtime model so exclusion checks do not perform continuous database queries on every request

📊 Dashboard improvements

The dashboard now provides a more operational workflow for investigating and managing traffic directly from the Technical Insights area.

It also continues to provide an extended report of about 40 pages, designed as a descriptive snapshot of the most relevant runtime, performance, cache, database, and external request issues detected in the application.

• 📉 Latency, throughput, and error rate
• 🗄 Database and cache observability
• 🌐 External request analysis
• 🧩 Span breakdown and technical insights
• 📈 Charts powered by Apache ECharts
• 🔕 Direct row actions such as Exclude from logs and Re-enable logs
• 🧭 Deep-linking from dashboard rows to exclusion rule management with prefilled filters
• 🌍 Better labeling for unresolved incoming requests, including privacy-aware external request identification

🔔 ECA integration

Native Observability includes integration with ECA, bringing observability events into Drupal’s event-condition-action ecosystem.

This opens the door to automation scenarios such as reacting to slow requests, failed requests, AJAX activity, trace deletion, or other runtime signals captured by Native Observability.

• 🔌 native_observability_eca_bridge module
• 🧠 Native Observability events exposed to ECA
• 🧪 Unit coverage for ECA event metadata
• 🧰 Optional demo module with sample ECA configurations
• 🚨 Example workflows for 500 errors, slow requests, AJAX requests, trace logging, and trace deletion

🧠 How it works

1. A request enters Drupal
2. A unique request ID is assigned
3. Native Observability evaluates whether the request should be excluded
4. If not excluded, execution is traced and stored
5. Spans capture internal operations
6. Metrics are aggregated
7. Data is visualized in dashboard and reports
8. Telemetry can be exported to external observability platforms
9. Runtime events can be used by ECA workflows

This lets you understand:

• ✔ what happened
• ✔ how long it took
• ✔ where time was spent
• ✔ which routes, services, requests, or queries slowed things down
• ✔ which traffic should be excluded because it adds noise rather than value
• ✔ which runtime events should trigger automation or alerts

🧱 Modules

• native_observability – tracing core and request-log exclusion engine
• native_observability_execution – execution monitoring
• native_observability_spans – span structure
• native_observability_metrics – performance analytics
• native_observability_dashboard – UI dashboard
• native_observability_report – reporting
• native_observability_otel – OpenTelemetry and Prometheus export
• native_observability_eca_bridge – ECA integration
• native_observability_eca_bridge_demo – optional demo ECA workflows for testing and examples

📚 Requirements

Dashboard and reports use Apache ECharts for chart rendering.

"merge-plugin": {
    "include": [
      "[Module_ROOT_PATH]/native_observability/modules/native_observability_dashboard/composer.libraries.json"
    ]
  }

The ECA bridge requires the ECA module only when you want to enable event-driven automation based on observability events.

🔥 Why it matters

Logs tell you what happened.
Observability shows you why.

Native Observability gives Drupal real runtime visibility, Prometheus/Grafana-ready telemetry, automation support through ECA, and now a robust way to exclude noisy traffic before it distorts the data you rely on.

⚖ Performance

Approximate overhead: ~10% when full tracing is enabled.

The request-log exclusion system introduced in 1.0.7 is designed to keep runtime overhead low.
Exclusion rules are stored in the database as the authoritative source, then normalized and saved in cache so the application does not continuously query the database on every request.

During a request lifecycle, Native Observability also reuses the already loaded rule set in memory, reducing repeated work and keeping exclusion checks lightweight.

⚠️ Important usage notes

Native Observability is designed for diagnostics, investigation, performance analysis, and controlled monitoring windows.
Because it can collect a significant amount of runtime data, it is not recommended to keep full tracing enabled for long periods without limits.

If left active without a retention strategy, the database may grow quickly. Before using it in production or on busy environments, make sure to configure the available retention and record-limit options so that stored traces, spans, metrics, and reports remain under control.

The new request-log exclusion feature should be used to suppress health checks, synthetic traffic, probing requests, obsolete endpoints, and other known noise sources that would otherwise reduce the clarity of the collected observability data.

When enabling OpenTelemetry or Prometheus export, pay particular attention to security and access configuration. Telemetry endpoints and exported runtime data may expose sensitive operational information if they are not protected correctly.

Detailed OpenTelemetry and Prometheus setup instructions are available in the dedicated native_observability_otel README, including guidance on how to connect external tools safely.

📌 Status

Stable 1.0.7 release.
Actively maintained and evolving.

🗺️ What’s next

Native Observability will continue evolving around better external observability integrations, richer dashboard insights, stronger operator workflows, and deeper automation scenarios with ECA.

If you have ideas, suggestions, or real-world monitoring use cases, feel free to share them in the issues.