Resubmitting a failed form always fails with ObscureUrl enabled

I added dpm($this->url_elements); to function isSpam() in the ObscureURL recipe, and it appears that while the URL parameter does not change, the expected field name and value change, causing the resubmission to fail.

public function isSpam($form, $form_state) {
$isSpam = parent::isSpam($form, $form_state); dpm($this->url_elements);
foreach ($this->url_elements as $field => $value) { dpm($_GET[$field]);
$url_field = isset($_GET[$field]) ? $_GET[$field] : FALSE;
unset($_GET[$field]); dpm($url_field);
if (isset($value['!valid_token']) && $url_field !== $value['!valid_token']) {
$isSpam = TRUE;
break;
}
}
return $isSpam;
}

dpm() results - first submission:
... (Array, 1 element)
3618_name (Array, 3 elements)
#type (String, 9 characters ) textfield
#default_value (String, 0 characters )
!valid_token (String, 31 characters ) 896a8bca03618fd7da1c2b4dad_form

dpm() results - second submission:
... (Array, 1 element)
52d8_name (Array, 3 elements)
#type (String, 9 characters ) textfield
#default_value (String, 0 characters )
!valid_token (String, 31 characters ) 8aa673a1f9362c72b5dbf9d002_form

URL parameter for both submissions:
3618_name=896a8bca03618fd7da1c2b4dad_form

This behavior would be consistent with a duplicate chunk of JavaScript code being added to the second form.

Ok, the resubmission does appear to generate additional fields, but it looks like the real problem is probably a variable substitution issue. The URL is generated with two botcha parameters for the resubmission, but the second parameter includes a variable name that did not get processed:

?a8fb_name=b6e53c126e8839d02791c7816f_form&a100_name=07'+v+'b22093bb686fc116e4_form

The '+v+' segment is added in function function getJsValue() in this segment:
// Secure_token.
$submit = preg_replace(
'/__replace__/',
$js['chops'][$chops_positions[0]] . '\'+v+\'' . $js['chops'][$chops_positions[1]],
$submit
);

I have not yet tracked down how the javascript gets added on resubmission to see what might be different (does something happen first that strips the backslashes?).

I was wrong (not surprising). The '+v+' bit is supposed to be there, and it looks like the previous URL parameter is probably ok (I did some testing with it stripped out, and the results were the same).

I stripped down my dpm() calls when I realized isSpam() was somehow running for both the search form and the form I was working with (a node form in one case, and a comment form in another).

dpm($this->url_elements); dpm($_GET);

On the first form submission, the $_GET element for the field matched the !valid_token. On the second submission, the original $_GET parameter was still there, but the token was new. On the third submission, the $_GET parameter has the second token, but the !valid_token is new. I continued the process for a few additional submissions, and I always got a $_GET parameter that was one submission behind the current !valid_token.

I believe I have a fix, though it isn't in the right place, and I have not yet tested in IE, so I can't really submit a patch just yet.

BotchaRecipeUsingJsAbstract uses drupal_add_js() in method generateFormElements, which is called from hook_form_alter(). On validation failure, the form is reloaded without adding the javascript (though apparently the updated javascript gets cached so it loads on the next submission).

My usual solution to this problem is to add an #after_build callback in hook_form_alter() and call drupal_add_js() from there. I couldn't figure out a way to get the callback to call a particular instance (it has been a LONG time since I did any object oriented programming). However, I was able to call drupal_add_js() indirectly during form validation by adding the below code (copied from BotchaRecipeUsingJsAbstract) in the isSpam() method for BotchaRecipeObscureUrl.

$js_value = $this->getProperty(
$this->settings['js']['value'],
'getJsValue'
);
if (!empty($js_value)) {
// @todo Abstract it.
drupal_add_js($js_value, 'inline');
//drupal_add_js($js_value, array('type' => 'inline', 'preprocess' => FALSE));
}

I suspect that if I dig a bit, there will be an actual form validation callback that I could use directly in BotchaRecipeUsingJsAbstract to make the issue disappear completely, but it is time to go home and get some sleep.

I am sorry to say that your proposal did not work for my installation.

I noticed, that $this->url_elements at line botcha.recipe.controller.inc:781 is on resubmitting one value "ahead" of the actual URL (the value read there is the value actually used after next submit). Maybe this helps debugging?

#1958222: Submitting a preview is not allowed for ObscureURL is probably just a duplicate of this issue, but since it also fails on preview, not just a resubmitted and failed form I opened it for further tracking.

I hate to admit it, but I think you may be right. I'm going to need to switch logging methods and probably spend another day testing soon. The fix above allowed the forms to submit for me, but I think I am getting just about as much spam with the fix as I was with the ObscureURL recipe disabled.

Tagged "ObscureURL"

Are you working off 6.x-3.x-dev or 6.x-3.0 release? If you fighting 6.x-3.0 (which is suggested by the current issue settings), then try 6.x-3.x-dev as there were recent commits backported from 7.x that address some js issues and report here.

I'm fairly certain I was working from the 3.0 release with a patch or two applied from other issues I have posted on, but I have not had time to work on it in a while, so I'm not certain anymore. Next chance I get, I will try the dev version.

Yes, this is still a problem in 6.x-3.x-dev. I do see the patch in dev to allow recipes to properly apply to node forms, but they still fail on resubmit after an error. I checked comments as well, since I don't have the time today to troubleshoot in-depth, and if comment preview is enabled, it still fails the second submit (clicking Save after previewing a comment).

FTR: This also affects 7.x-3.x-dev

After disabling ObscureUrl, the error disappeared, however lots of spam are passing through which defeats the purpose.
The question is; what does ObscureUrl do in the code? can it be fixed to eliminate this issue?

Confirmed: Also occurs in 7.x-3.x-dev.

Experienced the same issue in 7.x-3.2 version also. All you need to do is just disable the Obscure URL from the recipe. Now everything works fine. Thankyou all!!

Now everything works fine??

wait a little and you will get spam that contains URLs.
Disabling ObscureURL line is just a band-aid and we need that line. Hope there will be a permanent solution for this issue.

We're also experiencing this in 7.x-3.2. Looking forward to a fix.

I'm also using the 7.x-3.2 version with the same problem occurring. I don't know if it's related, but mine was rejecting me on the user login form. I found that if I tried to login using the form entries prefilled out by the browser, it rejected me. If I erased the username/password fields, then typed it in manually, it accepted me.

#18 sounds like a different issue. ObscureURL always fails on resubmit. #18 case seems to re-submit ok. Please open a new ticket for this and add as much detail as possible (OS/platform, browser, version, browser plugins, etc.)

Same issue with 7.3.2 for user_register. Disabling ObscureUrl works around the problem.

Am I following the code correctly?

It looks to me like hook_form_alter() will cause generateFormElements() to execute in class BotchaRecipeObscureUrl, which extends BotchaRecipeUsingJsAbstract. generateFormElements() assembles some bits and pieces, including some Javascript, then calls generateFormElements() in the parent (BotchaRecipeUsingJsAbstract).

In BotchaRecipeUsingJsAbstract, generateFormElements() calls drupal_add_js().

Assuming the above is correct (I am not at all sure I'm following it correctly), then drupal_add_js() is called during hook_form_alter(), not during an #after_build callback, so when the form is reloaded, the JavaScript may not be attached.

There is apparently a $cache parameter for drupal_add_js(), but it didn't change the behavior. I also tried to fit an #after_build callback in using a global variable to hold $js_value (and disabled all recipes except ObscureURL because it is not the only recipe descended from BotchaRecipeUsingJsAbstract). The behavior still didn't change - though it did break the first submit until I remembered to actually disable all other recipes.

Hopefully I'm just missing something simple in my admittedly hastily built test code, but I have seen similar behavior enough times in my own modules when I used drupal_add_js in hook_form_alter(), so it really seems worth a bit more investigation.

I had a bit more time for experiments, but I'm no closer to a solution. Hopefully the results of the experiments will help someone who has more time than I do.

In controller/application/botcha.application.controller.inc

class Botcha has a form_alter() method.

I looked through the moopapi code and figured out that it automatically creates functions that can be used in callbacks based on the methods available, so I added public function form_after_build(&$form, &$form_state), which seems to work just like normal #after_build callbacks. However, I was not able to figure out which parts of form_alter() could be moved to form_after_build(), and moving all of form_alter() did not work. I played with bits and pieces quite a lot, but don't have a list of what I tried.

If anyone wants to experiment with #after_build, I added the below line to form_alter() after if ($botcha_form->isEnabled()):
$form['#after_build'][] = 'botcha_form_after_build';

Here is a minimal form_after_build() method that does nothing (I verified it gets called by calling watchdog()):
public function form_after_build(&$form, &$form_state) {
return $form;
}

--

Once I gave up on that strategy, looked at $form_state['rebuild'] (drupal.org/node/1850410), using form_get_errors() in the formValidate() method. My thought was that if the form is rebuilt from scratch, then any JavaScript added in form_alter() should be added again.

$form_errs = form_get_errors();
if (!empty($form_errs)) {
$form_state['rebuild'] = true;
}

This also appears to have no effect.

I also have this issue on different forms.

added link to 1958222

The correct way to add JS and CSS to a form in D7 is to use the #attached property. Surprised no one has mentioned that here. I'm going to amend this recipe accordingly and try it.

Using #attached does not seem to be the answer, at least not on its own. :(

@ #19: I'm not sure #18 is a different issue. I found this thread (amongst others) after receiving the OP's error message. In my case, it was received by authenticated users trying to add a new topic to the website's (core)Forum+Advanced Forum. Disabling ObscureURL solved that swiftly, first try. From this thread and others, it feels like it's something in how the recipe is handled versus Drupal's form submission. (I've no idea how to dig around in the code, like others have, or I'd take a further look myself.)

I'm also having this problem with my forms. I just enabled botcha 7.x-3.3 and didn't change anything.

I too am seeing this issue with 7.x-3.3 when resubmitting a form having not field out all of the required fields the first time around.

Just hit this too - lots of good investigating but no fix apart from disabling ObscureURL?

I have the same issue. Anyone can explain the risk without using ObscureUrl?

Like csc4, any fix apart from disabling ObscureURL?

hosais

I've been trying to get the bottom of this, and after some hours I think I've have got to the root of the problem, though I don't have a solution yet:

The BotchaRecipe class contains a "seed" that's used to generate the pseudo-random field names etc. The seed is derived from the class "secret" which is derived from the Drupal form build_id. When the form is first generated (1st build) the new form build_id is used to generate the seed. But when the form is submitted and rebuilt (2nd build) the build_id of the submitted form (1st build) is used, so that the seed matches the one used when the original form was generated. This allows the BOTCHA validation to regenerate the original field names and values.

Unfortunately it all goes wrong if the form validation fails for some other reason (e.g. required field): If the form is submitted a second time, BOTCHA uses the submitted build_id to generate the seed, as above. This is the build_id of the 2nd build of the form. But when the 2nd build of the form was built BOTCHA actually used the submitted build_id from the 1st build to create the seed. As a result the validate functions look for fields that are not there. In the case of ObscureURL this causes the recipe to fail validation. But the other recipes just fail the other way: They pass without actually doing any checking!

To test this, load a form with only one recipe configured eg Honeypot. Turn off CSS so you can see the hidden field. Then
Step 1) Normal BOTCHA rejects bot
Load the user register form. With css off, enter something into the hidden field, enter "a" into the username and email fields, and submit the form -> BOTCHA correctly detects the honeypot has been violated

Step 2) BOTCHA doesn't reject bot
Then (with css off) enter something else in the hidden honeypot field and submit again.
-> This time BOTCHA fails to detect the honeypot violation.

It's questionable whether any bot will resubmit a failed form, but if it does then BOTCHA does nothing (except ObscureURL which fails every time).

A solution could be based on having two seeds, one (derived from the build_id) used to build a form, and the other (derived from the submitted form's build_id) used in form validation.

Unfortunately things get even more complex because in addition to the seed, mt_rand() is used in various places when generating fields and values, so I don't have a clear solution yet.