Today, I started getting the error: Rejected by Microsoft Translator: TranslateApiExceptionMethod: TranslateArray()Message: The Azure Market Place Translator Subscription associated with the request credentials has zero balance.message id=1116.V2_Rest.TranslateArray.4182143E

Some digging revealed https://datamarket.azure.com/dataset/bing/microsofttranslator the Microsoft data market is closed, and has been moved to https://portal.azure.com/#create/Microsoft.CognitiveServices/apitype/Tex...

I've created new API credentials on the Azure Portal, but the two-key system does not seem to mesh with the TMGMT "Client ID" and "Client Secret" -- is there a way to use the new Azure Portal "Translate Text" API credentials with TMGMT?

CommentFileSizeAuthor
#88 2840876-88.patch37.98 KBbceyssens
#86 screenshot-wangjueju.cn-2019.02.17-00-16-24.png46.77 KBjjwfcd
#84 2840876-84.patch48.2 KBPeacog
#84 interdiff-2840876-81-84.txt498 bytesPeacog
#81 2840876-81.patch48.32 KBheddn
#81 interdiff_68-81.txt3.74 KBheddn
#79 2840876-79.patch3.54 KBheddn
#77 2840876-77.patch3.52 KBheddn
#75 2840876-75.patch3.07 KBheddn
#72 2840876-72.patch48.65 KBheddn
#72 2840876-72.patch48.65 KBheddn
#70 2840876-70.patch48.51 KBheddn
#68 2840876-68.patch48.43 KBheddn
#68 interdiff_66-68.txt622 bytesheddn
#66 2840876-65.patch48.43 KBheddn
#66 interdiff_64-65.txt1.72 KBheddn
#64 interdiff_62-64.txt6.41 KBheddn
#64 2840876-64.patch48.37 KBheddn
#62 2840876-62.patch47.82 KBheddn
#62 interdiff_61-62.txt2.43 KBheddn
#61 2840876-61.patch47.96 KBheddn
#61 interdiff_58-61.txt19.86 KBheddn
#58 2840876-58.patch43.67 KBheddn
#56 2840876-56.patch44.48 KBheddn
#54 2840876-54.patch44.08 KBheddn
#54 interdiff_49-54.txt11.66 KBheddn
#40 2840876-39.patch31.75 KBheddn
#37 2840876-37.patch31.72 KBheddn
#36 2840876-36.patch32.66 KBheddn
#34 interdiff_32-34.txt3.48 KBheddn
#34 2840876-34.patch32.16 KBheddn
#32 interdiff_30-32.txt10.46 KBheddn
#32 2840876-32.patch31.75 KBheddn
#30 interdiff_28-30.txt5.51 KBheddn
#30 2840876-30.patch29.86 KBheddn
#28 interdiff_27-28.txt8.09 KBheddn
#28 2840876-28.patch28.94 KBheddn
#27 2840876-27.patch27.51 KBheddn
#27 interdiff_23-27.txt20.5 KBheddn
#23 2840876-23.patch13.4 KBheddn
#23 interdiff_17-23.txt4.58 KBheddn
#17 tmgmt_microsoft-azure-congnitive-api-update-2840876-17.patch8.48 KBpaulmartin84
#13 tmgmt_microsoft-azure-congnitive-api-update-2840876-13-interdiff.txt8.32 KBBerdir
#13 tmgmt_microsoft-azure-congnitive-api-update-2840876-13.patch12.32 KBBerdir
#6 tmgmt_microsoft-azure-congnitive-api-update-2840876-6.patch6.86 KBccrawford91
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

charlesj created an issue. See original summary.

firfin’s picture

Experincing the same issue. To elaborate: In the Azure Portal, go to 'All resources' somewhere in this list you can click on your Cognitive Services account for the current project. Under the 'resource management' vertical tab you can find 'keys' section.
They you can find three values:
- Account name
- KEY 1
- KEY 2

Haven't found a way to get these working yet. Which I think makes sense probably, I can imagine there is a completely new auth system in place now. Probably API changed too.

firfin’s picture

Datamarket/translator will stop working end of april. See https://translatorbusiness.uservoice.com/knowledgebase/articles/1078534-...
This module will probably stop functioning completely at that point, therefore setting priority to critical.
According to that page:

To begin using the Microsoft Translator subscription from Azure, you will need to replace the app authentication using your Microsoft Data Market Client ID and Client Secret with your new Azure key. The single Azure key is used to request an Azure token for the Microsoft Translator service.

firfin’s picture

Priority: Normal » Critical
heddn’s picture

There is a replacement in Azure. We might just need to updated some endpoints and use a single key, instead of the combo key/secret. Anyone up to rolling a patch. See https://cognitive.uservoice.com/knowledgebase/articles/1128340-announcem...

New endpoint: https://api.cognitive.microsoft.com/sts/v1.0/issueToken

ccrawford91’s picture

I've patched the module to use the new endpoint and API key.

heddn’s picture

Status: Active » Needs work

Do we need an update path? Me thinks yes.

heddn’s picture

Status: Needs work » Needs review

Or maybe not... since the old api keys are going to be non-functional. Moving to NR for now.

devwebcms’s picture

I'm receiving the error message "The "Client ID" or "Client secret" is not valid." Please let me know if the patch (tmgmt_microsoft-azure-congnitive-api-update-2840876-6.patch) is included with 7.x-1.x-dev or 7.x-1.0-alpha2 or if it has to be applied. Thank you very much.

jollysolutions’s picture

Status: Needs review » Reviewed & tested by the community

Please apply ASAP

heddn’s picture

Status: Reviewed & tested by the community » Needs work

Needs an update path. Marking as NW. Or a @TODO and an opened issue to add an update path.

jollysolutions’s picture

Status: Needs work » Reviewed & tested by the community

What update path? They are completely different systems with different keys.

Berdir’s picture

Agreed that an upgrade path is not possible.

I worked a bit on this to fix/update tests and also quite a bit of cleanup, the code was getting pretty strange with 3 different api methods implemented but everything except the new API is no longer working, so there's no point in trying to support them still.

  • Berdir committed 23b6028 on 7.x-1.x authored by christian3244
    Issue #2840876 by Berdir, christian3244: Azure Datamarket has changed
    
miro_dietiker’s picture

A commit without status change?

Does this need 8.x care?

Berdir’s picture

Version: 7.x-1.x-dev » 8.x-1.x-dev
Status: Reviewed & tested by the community » Patch (to be ported)

Looks like my comment got lost.

paulmartin84’s picture

This gets it working in d8, I haven't had time to adjust the tests yet.

Grimreaper’s picture

Status: Patch (to be ported) » Needs work

Hello,

I wanted to test tmgmt with Microsoft, so I applied patch from comment 17. It fixed the issue.

Thanks.

Berdir’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 17: tmgmt_microsoft-azure-congnitive-api-update-2840876-17.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

sealionking’s picture

patched the #17 patch

but

Client error: `GET http://api.microsofttranslator.com/v2/Http.svc/GetLanguagesForTranslate` resulted in a `400 Bad Request` response:
Argument Exception

Method: GetLanguagesForTranslate()

Parameter:

Message: The API ty (truncated...)

k_zoltan’s picture

Checkout the back-port of the #17 patch for he 7.x version #2962022: Microsoft data market is closed, Moving to Microsoft Azure CognitiveServices

heddn’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 23: 2840876-23.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

heddn’s picture

Hmm, this seems to use v2, which is already deprecated. If we are going to land this, wouldn't it be better to use V3? https://docs.microsoft.com/en-us/azure/cognitive-services/translator/ref...

Here's a sample example in PHP:
https://github.com/MicrosoftTranslator/Text-Translation-API-V3-PHP/blob/...

heddn’s picture

FileSize
20.5 KB
27.51 KB

There's still some test failures, but less.

heddn’s picture

Status: Needs work » Needs review
FileSize
28.94 KB
8.09 KB

Manual testing, this worked. Not to get all the automated tests working.

Status: Needs review » Needs work

The last submitted patch, 28: 2840876-28.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

heddn’s picture

Status: Needs work » Needs review
FileSize
29.86 KB
5.51 KB

Tests passed on local.

Status: Needs review » Needs work

The last submitted patch, 30: 2840876-30.patch, failed testing. View results

heddn’s picture

Status: Needs work » Needs review
FileSize
31.75 KB
10.46 KB

Fix up a few more CS things. Tried to reproduce the bugs, but not sure I was able to. I tested locally against 8.5 and 8.6, just to be sure it wasn't a core thing.

Status: Needs review » Needs work

The last submitted patch, 32: 2840876-32.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

heddn’s picture

Status: Needs work » Needs review
FileSize
32.16 KB
3.48 KB

Some debugging added to figure out what is displayed on the testbot.

Status: Needs review » Needs work

The last submitted patch, 34: 2840876-34.patch, failed testing. View results

heddn’s picture

Status: Needs work » Needs review
FileSize
32.66 KB
heddn’s picture

The last submitted patch, 36: 2840876-36.patch, failed testing. View results

Status: Needs review » Needs work

The last submitted patch, 37: 2840876-37.patch, failed testing. View results

heddn’s picture

Status: Needs work » Needs review
FileSize
31.75 KB

Status: Needs review » Needs work

The last submitted patch, 40: 2840876-39.patch, failed testing. View results

heddn’s picture

OK, let's forget simpletest and try a proper functional test. This one also passes on local, but so did the simpletest. Fingers crossed.

For reviewers, the test is basically a straight port to functional w/ as little changed as possible.

Status: Needs review » Needs work

The last submitted patch, 42: 2840876-42.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

heddn’s picture

Status: Needs work » Needs review
FileSize
3.17 KB
39.33 KB

Code standards + possibly we are facing concurrency? Let's try that.

Status: Needs review » Needs work

The last submitted patch, 44: 2840876-44.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

heddn’s picture

Status: Needs work » Needs review
FileSize
2.5 KB
39.39 KB

drupal_static? Let's see.

Status: Needs review » Needs work

The last submitted patch, 46: 2840876-46.patch, failed testing. View results

heddn’s picture

Status: Needs work » Needs review
FileSize
261 bytes

I think tests are currently failing. Let's see.

Status: Needs review » Needs work

The last submitted patch, 49: 2840876-49.patch, failed testing. View results

heddn’s picture

I give up. Tests pass just fine on local. I was hoping I could get this to pass tests, but I need to move on. I'll just keep using the patch.

mpp’s picture

Thanks for the patch and your perseverance :-)

Why are we testing PHP 5.5, it's deprecated. Let's see what 5.6 and 7.2 make of it.

mpp’s picture

heddn’s picture

Status: Needs review » Needs work

The last submitted patch, 54: 2840876-54.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

heddn’s picture

Status: Needs work » Needs review
FileSize
44.48 KB

Status: Needs review » Needs work

The last submitted patch, 56: 2840876-56.patch, failed testing. View results

heddn’s picture

Status: Needs work » Needs review
FileSize
43.67 KB

Status: Needs review » Needs work

The last submitted patch, 58: 2840876-58.patch, failed testing. View results

mpp’s picture

You're getting there!

heddn’s picture

FileSize
19.86 KB
47.96 KB

Different approach to things. Big interdiff.

heddn’s picture

Status: Needs work » Needs review
FileSize
2.43 KB
47.82 KB

Cleaned up things I spotted.

Status: Needs review » Needs work

The last submitted patch, 62: 2840876-62.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

heddn’s picture

Status: Needs work » Needs review
FileSize
48.37 KB
6.41 KB

Status: Needs review » Needs work

The last submitted patch, 64: 2840876-64.patch, failed testing. View results

heddn’s picture

Status: Needs work » Needs review
FileSize
1.72 KB
48.43 KB

Status: Needs review » Needs work

The last submitted patch, 66: 2840876-65.patch, failed testing. View results

heddn’s picture

Status: Needs work » Needs review
FileSize
622 bytes
48.43 KB

Status: Needs review » Needs work

The last submitted patch, 68: 2840876-68.patch, failed testing. View results

heddn’s picture

Status: Needs work » Needs review
FileSize
48.51 KB

Status: Needs review » Needs work

The last submitted patch, 70: 2840876-70.patch, failed testing. View results

heddn’s picture

Status: Needs work » Needs review
FileSize
48.65 KB
48.65 KB

For some reason, the following returns empty string on the testbot. Any suggestions, would be much appreciated.

  public function getLanguages(Request $request) {
    $query = $request->query;
    if (!$query->has('api-version')) {
      return new Response('Bad request - missing: api-version', 400);
    }
    $json['translation']['de'] = [
      'name' => 'German',
      'nativeName' => 'German',
      'dir' => 'ltr',
    ];
    $json['translation']['en'] = [
      'name' => 'English',
      'nativeName' => 'English',
      'dir' => 'ltr',
    ];
    $json['translation']['es'] = [
      'name' => 'Spanish',
      'nativeName' => 'Espanol',
      'dir' => 'ltr',
    ];
    return new JsonResponse($json);
  }
    $languages = [
      'route' => 'tmgmt_microsoft_test.languages',
      'verb' => 'GET',
      'key' => $good_key,
      'query' => [
        'scope' => 'translation',
        'api-version' => '3.0',
      ],
    ];
    $response = $this->assertHttpRequest('languages', $languages);
    $json = json_decode((string) $response->getBody()->getContents(), TRUE);

The last submitted patch, 72: 2840876-72.patch, failed testing. View results

Status: Needs review » Needs work

The last submitted patch, 72: 2840876-72.patch, failed testing. View results

heddn’s picture

Status: Needs work » Needs review
FileSize
3.07 KB

The most simple test possible.

Status: Needs review » Needs work

The last submitted patch, 75: 2840876-75.patch, failed testing. View results

heddn’s picture

Status: Needs work » Needs review
FileSize
3.52 KB

Too slimmed down.

Status: Needs review » Needs work

The last submitted patch, 77: 2840876-77.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

heddn’s picture

Status: Needs work » Needs review
FileSize
3.54 KB

Status: Needs review » Needs work

The last submitted patch, 79: 2840876-79.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

heddn’s picture

So, after consulting with mixologic and berdir in slack, we figured out it was an issue with how run-tests.sh does things vs phpunit. Its something to do with `BROWSERTEST_OUTPUT_DIRECTORY` being set in run-tests.sh as an env var when the test is executed.

is the patch using $response->getBody()->getContent() or so
don't do that
use (string) $response->getBody()
the response body has an internal pointer

Ignore all the chatter in between 68 and 81. It was just trying to pin down what the issue was. This should now pass.

heddn’s picture

Yeah. Tests pass green.

mpp’s picture

Whoohoo!!! @heddn, you win a gold medal for this one, heck I'd give you a diamond one if I could :-)

Big thanks to mixologic and berdir as well.

Peacog’s picture

Thanks for the patch! I've tweaked it slightly to change the max supported characters to 5000 as per the current API specification.

Status: Needs review » Needs work

The last submitted patch, 84: 2840876-84.patch, failed testing. View results

jjwfcd’s picture

3ks you guys

the patch applied,

And need help to make it work when a node body exceeds 5000 characters

jacktonkin’s picture

Status: Needs work » Needs review

I think #84 failed because TMGMT was broken on 8.7 at the time. Setting back to needs review.

bceyssens’s picture

Even with patch #84, we couldn't connect the provider. Apparently we need to use a regional service URL: https://docs.microsoft.com/azure/cognitive-services/translator/reference....

  • heddn committed 73e9f8e on 8.x-1.x
    Issue #2840876 by heddn, Berdir, Peacog, ccrawford91, paulmartin84,...
heddn’s picture

Status: Needs review » Fixed

Everyone, thanks for your contributions.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.