Abstract token usage support

There is no code here to be reviewed, so setting back to Active.

There is a fork but no merge request: see https://www.drupal.org/docs/develop/git/using-gitlab-to-contribute-to-dr... which may assist.

Code looks good to me: cannot be properly tested or merged until https://www.drupal.org/project/ai/issues/3517618 has been approved and merged, so this will probably sit for a while.

Tested this and came across the following issues

While testing it with the ai_ckeditor module:
Cannot use object of type OpenAI\Responses\StreamResponse as array in Drupal\ai_provider_openai\Plugin\AiProvider\OpenAiProvider->chat() (line 363 of modules/contrib/ai_provider_openai/src/Plugin/AiProvider/OpenAiProvider.php)

The code now assumes that the $response will be an array, and if I'm correctly, the $respons will only be an array if the response is not streamed (see try catch above)

While testing a combination of modules:
Argument #1 ($tokens) must be of type int, null given, called in /var/www/html/web/modules/contrib/ai_provider_openai/src/Plugin/AiProvider/OpenAiProvider.php
So either the setter should allow null as a value (issue in the other patch), or this patch should be adapted to 0 if it's null

Adding some logic to the MR, in an attempt to also provide support for token usage, in case of streamed responses.
This needs https://www.drupal.org/project/ai/issues/3524435 to be fixed first.

Added related issues in AI module, that need to be fixed first, before both token usage for both regular chat and streamed chat can be captured.

Minor comment and before I merge this - it is compatible from my testing with AI Core 1.1.0, correct?

Fixed this for 1.2.x release, getting merged.