Problem/Motivation

LangFuseClient::syncTraces() refuses to sync under CLI:

public function syncTraces(bool $force = FALSE): void {
  if (empty($this->pendingTraceIds) || (!$force && !$this->shouldSyncNow())) {
    return;
  }
  // ...
}

protected function shouldSyncNow(): bool {
  // Check if the request is ending or other conditions that would
  // indicate it's appropriate to sync traces now.
  return php_sapi_name() !== 'cli';
}

So any trace created from Drush, from cron, or from a queue worker is
silently dropped unless the caller happens to pass
$force = TRUE. There is no log line, no warning, and no exception —
the method simply returns and the accumulated traces are discarded when the
process ends.

This matters more than it might appear, because a large share of AI work in
Drupal runs exactly there: queued indexing, scheduled agent runs, batch
content operations, and anything driven by cron. Those are the operations whose
cost and latency people most want observability on, and they are the ones that
produce nothing.

The failure mode is the worst kind: it looks like it worked. The trace object
is created successfully, code that inspects it behaves normally, and no error
surfaces anywhere.

Steps to reproduce

  1. Configure the module against a reachable Langfuse instance.
  2. From Drush, create a trace via langfuse.client, add an
    observation, and call syncTraces() without force.
  3. Nothing is sent. The trace never appears in Langfuse, and Drupal logs
    contain nothing about it.
  4. Call syncTraces(TRUE) instead and the same trace lands
    correctly.

Proposed resolution

The intent behind shouldSyncNow() looks like "only flush when the
request is ending" — the docblock says as much. Keying that on
php_sapi_name() is the wrong test: under CLI there is no kernel
terminate event to defer to, so "not now" effectively means "never".

Options, roughly in order of preference:

  • Flush on drush_shutdown / register_shutdown_function
    under CLI, so CLI gets an end-of-process flush equivalent to the
    kernel-terminate flush that web requests get.
  • Or drop the SAPI check entirely and let the existing sync subscriber own
    the timing, with an explicit flush at process end.

Whichever is chosen: a skipped sync must be logged. Silently
discarding collected telemetry is not an acceptable outcome in any branch of
this method.

Remaining tasks

  • Decide between the shutdown-hook and subscriber-owned approaches.
  • Kernel test asserting traces created under CLI are actually flushed.
  • Add a log line for any path that discards pending traces.
  • Manual verification with a real Drush-triggered AI operation.

User interface changes

None.

API changes

None expected. syncTraces(bool $force) keeps its signature; the
$force parameter becomes a genuine override rather than the only
way to make CLI work at all.

Data model changes

None.

Release notes snippet

Traces created from Drush, cron and queue workers are now sent to Langfuse.
Previously they were collected and then silently discarded at the end of the
process.

Issue fork langfuse-3616634

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

nikro created an issue. See original summary.

nikro’s picture

nikro commented

Ran a test with Claude:

Confirming this empirically, because the natural first objection is that Langfuse ingests asynchronously and the trace simply had not propagated yet. It is not propagation.

1. The method returns before sending anything. Inspecting the client under Drush via reflection:

php_sapi_name()   : cli
shouldSyncNow()   : false
pending BEFORE    : 1
pending AFTER     : 1

pendingTraceIds is only cleared after a successful send. It still holds the trace after syncTraces() returns, which means execution stopped at the guard. No HTTP request is issued at all, so there is nothing in flight to propagate.

2. Timed against a control. Two traces were created from the same Drush process — one synced normally, one with syncTraces(TRUE) — then both polled against GET /api/public/traces/{id} every 8 seconds:

t=  0s   forced=200   unforced=404
t= 52s   forced=200   unforced=404
t=104s   forced=200   unforced=404
t=156s   forced=200   unforced=404
t=199s   forced=200   unforced=404

The forced control was already retrievable at t=0 — ingestion on this instance is effectively instant. The unforced trace never appeared across 24 polls spanning 200 seconds, long after the Drush process had exited.

3. The log actively reports success. The only entry written for the discarded run was:

langfuse   Info   Successfully synced 0 LangFuse traces.

That message is itself misleading for a second reason:
syncTraces() logs
count($this->pendingTraceIds) after emptying the array, so it reports 0 on every run including successful ones. Between the two, the logs cannot distinguish "synced nothing because CLI" from "synced successfully" — which is why this went unnoticed.

That count bug is already fixed on the #3594090 branch and is not part of this issue; noting it here only so the two are not conflated while debugging.

Environment: Drupal 11.4.5, dropsolid/langfuse-php-sdk v1.2.0,
Langfuse v3.225.2.

nikro’s picture

nikro commented

Pushed a fix. shouldSyncNow() returned php_sapi_name() !== 'cli', so traces created by Drush, cron and queue workers were collected and then dropped when the process ended, with no error and nothing in the log. The guard turned out to be redundant: syncing only runs from the kernel terminate subscriber, and Drush terminates the kernel too, so the request is already ending in both cases. It now returns TRUE, and the branch that refuses to sync logs a warning rather than discarding silently.

How to review it

  • Configure LangFuse, then from Drush create a trace and sync it without forcing:
    drush php:eval '$t = \Drupal::service("langfuse.client")->createTrace("cli check"); print $t->getId();'
  • On 1.x that trace never appears in LangFuse. On the branch it lands within a few seconds.
  • Give it a minute before concluding it is missing. Ingestion is asynchronous, so an absent trace and a slow one look the same at first.
  • Run tests/src/Unit/CliSyncTest.php. PHPUnit runs under CLI, so it asserts the previously broken case directly.
  • Check the diff is only the two hunks in LangFuseClient::syncTraces() and shouldSyncNow(), plus the new test.

One thing not fixed here, so it does not look like an oversight: the success message logs count($this->pendingTraceIds) after emptying the array, so it always reads "Successfully synced 0 LangFuse traces". That is fixed on the #3594090 branch.

nikro’s picture

nikro commented
Status: Active » Needs review

nikro opened merge request !10

abhisekmazumdar made their first commit to this issue’s fork.

abhisekmazumdar’s picture

abhisekmazumdar
he/him
Primary language English
Location India
commented
Status: Needs review » Reviewed & tested by the community

Reviewed. The fix is correct: shouldSyncNow() no longer blocks CLI, so Drush/cron/queue traces sync instead of getting silently dropped. Diff is exactly the two hunks it should be, plus a matching test.

The failing phpstan job is unrelated, those same 4 errors already fail on 1.x today, in files this MR doesn't touch.

Pushed a tiny follow-up commit: two comment/docblock wording fixes, no logic changes.

RTBC from my side.

nikro’s picture

nikro commented

Pushed a one-line follow-up (770fee0) to get the phpcs job green. The docblock reword in 8eb1364 left the long description starting with a lowercase shouldSyncNow(), which PHPCS rejects as Doc comment long description must start with a capital letter. The wording was an improvement so I kept it as-is and only changed the opening to "The shouldSyncNow() method ...", rewrapping the three lines. No code or test behaviour touched.

nikro’s picture

nikro commented
Status: Reviewed & tested by the community » Fixed

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

  • nikro committed 704827da on 1.x 
    fix: #3616634 sync traces created outside a web request instead of...