Convert request method to POST to get past 5000 character limit

Did you have any luck with a patch for D8?

The character limit was easy enough to workaround in D7 but struggling to get my head around it in D8 without falling back to refactoring my content types.

Just wanted to offer my thanks for the prototype, it works very well but as you say needs some refinement for more graceful error handling.

I'll see what I can do to extend that functionality now I have the base to work from.

Hi, I need the same functionality for 7.x version of the module.

Anyone has a patch or knows how to fix this issue for 7.x version?

Thanks in advance!

ajFernandex - Within 7.x this patch addressed majority of issues for me: https://www.drupal.org/project/tmgmt_google/issues/1799502#comment-7710247

Thanks for your response @westerix!

Which version should I apply the patch to? alpha2 or dev? I tried to apply the patch on the dev version but it gave me an error.

Thanks again!

A patch for @lpeabody's work is created. The patch creates 2 files: composer.json to install google/cloud-translate:^1.2 library and a new tmgmt translation provider uses Google batch translation.

Adding PHPCS coding standards.

The patch works.

But one thing is still missing: When there is already a job item with more than 5000 characters, the job form can't be saved because of the checkTranslatable() override. This overrride and the $maxCharacters variable should be just removed if there is no limit any more.

Thanks for the previous patch, it helped us to find out what are actual problems with the 'character limit'.

BTW, it patch doesn't work with Continuous jobs at all. It missing parts of code related to Continuous jobs. But it works fine for simple page translate, and there is no 5000 characters limit. We needed Continuous jobs working, so I get deeper into it and realized that limits are originating from the GET method used to send Guzzle HTTP requests to Google Translate API.

The previous patch includes an additional library google/cloud-translate, which sends POST requests instead of GET, so it fixed the characters limit issue. Also, google/cloud-translate's `translateBatch` method doesn't work anything 'fancy', just supporting translating multiple strings (array) with one request, and it's already supported by TMGMT_Google. Example: `$q = ['Title', 'Body text', etc]`. It doesn't send multiple requests or so

So, I thought it would be nice to have it working within the original TMGMT Provider (Google), and without including another library dependency. So I have created a simple patch which fixes 'character limit' issues. It changes GET requests to POST, removing a limit function, etc. Please find attached patch, probably it would be helpful for someone :)

I'm not sure about Google Translate API limits, their documentation is a bit confusing, so if someone realizes what are actual limits, let us know please :) Documentation says hard limit for requests is 30K Unicode characters, but in my case, I managed to translate 35.000+ characters without any problems using this patch. Looks like the Google Cloud Translation API is migrating to a new quota system in these days

When/If we know actual Google Translate API limits (number of characters per request, number of requests in some time period, etc) - it makes a sense to work on improvements here, like splitting content into parts (this wouldn't be easy I guess), add Drupal Batch Jobs, etc

Hi, lpeabody! It's completely fine to use the official Google library, but I think, in that case, most of the code/methods should be re-written and adapted to a new library. I have seen here is already some good work done and created a working system with GuzzleHTTP & official Google Translate API, so just tried to do some minimal changes and make it improved. And it's not any reduplicate work, just a couple lines of code added. Just my 2 cents :)
Cheers

updated patch attached, fixed test error

this is great 👍. I will test #20

The patch works well. Passed all the tests and it should be safe to say it's RTBC.

Works Great!!!

Great, it works also for us so far. Probably this line needs to be removed before RTBC:
// TODO: Investigate if there are any other Google Translate API limits after switching to POST method
I don't think there is one?

Makes sense. The checkTranslatable method is now empty and has no validations to check.

I believe all Google Translate API limits and quotes - other than the 5000 characters limit on GET requests - are managed from the Google APIs console. Which means those limits will be checked after the request is sent to Google, as it differs from a Google console app to another.

I have removed the checkTranslatable method. Hope this patch passes the tests now.

Both D8.9 and D9 tests passed.

Since last patch was submitted by me I hope someone else can test it and mark as RBTC.

+++ b/src/Plugin/tmgmt/Translator/GoogleTranslator.php
@@ -266,16 +246,9 @@ class GoogleTranslator extends TranslatorPluginBase implements ContainerFactoryP
-    // Prepare Guzzle Object.
-    $request = new Request('GET', $url);
-
     // Build the query.
     $query_string .= '&key=' . $translator->getSetting('api_key');
-    if (isset($request_query['q'])) {
-      foreach ($request_query['q'] as $source_text) {
-        $query_string .= '&q=' . urlencode($source_text);
-      }
-    }
+
     if (isset($request_query['source'])) {
       $query_string .= '&source=' . $request_query['source'];

I was confused why the tests were still passing, when we completely change how we pass things along.

Turns out the tests are broken. Specifically \Drupal\tmgmt_google_test\Controller\GoogleTranslatorTestController::translate(), we check for certain query arguments and attempt to return an error if missing, but we don't actaully *return* the error.

So lets fix those checks so that the test can actually fail and then update it to look for post data. Maybe we should also add an explicit check for the expected translatable string.

Sorry for putting that on this issue, but I don't want to get the tests even further from what they should do and we need to adjust them anyway here.

#26 is working really well for me. Great work!

#26 did not work for me. It removed my inability to submit due to the rate limit, but then after submitting, I received this error:

"Translation has been rejected with following error: Google Translate service returned following error: User Rate Limit Exceeded"

Do I also need to require google/cloud-translate with patch #26?

Thanks!

#26 doesn't apply on D9.4.7
I created a new version of this patch for pass tests on D9.

Sorry, I did mistake. Patch #26 applied on D9.4.7. I had some other patches for this module and forgot to check this.

Here's an attempt at fixing the tests. The main issue was simply not returning the JSON missing the translatedText item with error messages, etc.. I also added looking for the POST data, the request method and cleaned up some assertions. I'm not a phpunit expert, so there may be more to do. Hopefully, this moves the ball forward.

Fix linting issue.

Thanks for improving the tests, didn't test very large characters but seems to work at least as well as before.

Hello.
As I understood well, these changes were applied in tmgmt_google 8.x-1.1
Thank you.