Null fields are not synchronized [#1096712]

I am unable to sync null fields from Drupal to Salesforce or vice versa.

For example, if I have a field "profile_city", and it has a value of "Chicago", that will sync fine to Salesforce. I can alter the value to "Denver" and that syncs fine. But if I edit a user's profile and clear out the field, the change is not synched over to Salesforce. Similarly, if I clear out the "city" field in the Salesforce Contact, that change is not reflected back over to Drupal.

Looking at sf_user around lines 581-585, where we are building the $changes array, we have this:

elseif (isset($sf_data->$sf_fieldname) && isset($account->$drupal_fieldname)) {
      if ($account->$drupal_fieldname != $sf_data->$sf_fieldname) {
        $changes[$drupal_fieldname] = $sf_data->$sf_fieldname;
      }
    }

If $sf_data->$sf_fieldname is set to null, then the code won't clear out the relevant field in Drupal.

I see that in salesforce_api_export_create() there is some code to handle exporting NULL fields:

    // Ignore null values for non-nillable fields
    if (is_null($value) && !$nillable) {
      continue;
    } elseif (is_null($value)) {
      // Enterprise client can only nullify one field per transaction even with
      // the proper values set in fieldsToNull. This is a bug in a 3rd-party
      // library, beyond our control. In other words, this should be an array
      // but in fact must be a string. This is going to get overwritten on
      // every iteration of this loop, meaning that only one null field gets
      // nullified per transaction.
      $fieldsToNull = $sf_fieldname;
      continue;
    }

How are other people dealing with this issue? If someone can point out a good direction for development, then I can work on a patch.

Comment	File	Size	Author
#14	Screen shot 2011-06-30 at 1.05.57 PM.png	61.74 KB	rjacobs
#9	fieldsToNull-1096712-9.patch	1019 bytes	kostajh

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Comment #1

kostajh CreditAttribution: kostajh commented 28 March 2011 at 15:36

Looking at the PHP Toolkit, it seems like we should be able to pass an array:

  /**
   * Updates one or more new individual objects to your organization's data.
   * @param array sObjects    Array of sObjects
   * @param AssignmentRuleHeader $assignment_header is optional.  Defaults to NULL
   * @param MruHeader $mru_header is optional.  Defaults to NULL
   * @return UpdateResult
   */
  public function update($sObjects, $type, $assignment_header = NULL, $mru_header = NULL) {

    foreach ($sObjects as &$sObject) {

    	// FIX for fieldsToNull issue (STEP #1)
   		$xmlStr = '';
    	if(isset($sObject->fieldsToNull) && is_array($sObject->fieldsToNull)) {
    		foreach($sObject->fieldsToNull as $fieldToNull) {
    			$xmlStr .= '<fieldsToNull>' . $fieldToNull . '</fieldsToNull>';
    		}
    	}
    	// ------
    	
    	$sObject = new SoapVar($sObject, SOAP_ENC_OBJECT, $type, $this->namespace);
    	
    	// FIX for fieldsToNull issue (STEP #2)
    	if($xmlStr != '') {
    		$sObject->enc_value->fieldsToNull = new SoapVar(new SoapVar($xmlStr, XSD_ANYXML), SOAP_ENC_ARRAY);
    	}
    	// ------
    }
    $arg->sObjects = $sObjects;
    return parent::_update($arg);
  }

So in salesforce_api_export_create, I changed $fieldsToNull = $sf_fieldname; to $fieldsToNull[] = $sf_fieldname;

But on exporting I get this error:

salesforce Exception while attempting to export user: Unexpected element {http://www.w3.org/2001/XMLSchema}string during simple type deserialization

Any ideas?

Comment #2

AaronBauman

he/him

CreditAttribution: AaronBauman commented 28 March 2011 at 16:41

fieldsToNull is broken for the Enterprise client
http://boards.developerforce.com/t5/Perl-PHP-Python-Ruby-Development/PHP...
http://boards.developerforce.com/t5/Perl-PHP-Python-Ruby-Development/PHP...

Comment #3

kostajh CreditAttribution: kostajh commented 31 March 2011 at 21:00

I see. Thanks for letting me know.

For anyone else searching on this issue, here is how I resolved the problem when handling fields that have been cleared out in Salesforce:

In the pre_import hook take the incoming object $sf_object.
Create a test outbound object based on the uid, fieldmap name etc in sf_object.
In a foreach loop, go through each field in the sf_object and unset it from the outbound object.
Remaining fields in the $outbound_object can be assumed to be NULL. So tack on their keys to the original $sf_object.
When user_save gets called, it should successfully save the fields as NULL.

I've implemented this and it works okay although it's not the most efficient approach.

For handling fields that are set to null in Drupal, I suppose one could take the opposite approach and deal with manually unsetting SF fields in the post_export hook.

Comment #4

AaronBauman

he/him

CreditAttribution: AaronBauman commented 1 April 2011 at 22:01

I just noticed that the code you posted is for update(), but we're using upsert() in sf_user_export() and sf_node_export()

I'm not sure why this fix is not applied to update AND upsert, so I submitted this issue to the toolkit on github:
https://github.com/developerforce/Force.com-Toolkit-for-Facebook/issues#...

I didn't have time to test the fix on upsert(), but it might be worth looking into

Comment #5

kostajh CreditAttribution: kostajh commented 6 April 2011 at 19:17

Thanks for filing that issue - it should go on the PHP-Toolkit project though right?

I'll try testing and see what results I get.

Comment #6

AaronBauman

he/him

CreditAttribution: AaronBauman commented 6 April 2011 at 19:26

oh, duh.
go vote for this one instead.
https://github.com/developerforce/Force.com-Toolkit-for-PHP/issues/4

Comment #7

kostajh CreditAttribution: kostajh commented 6 April 2011 at 20:29

Thanks Aaron. I voted on it. I did some testing and can confirm that the fix for update also works for upsert - make the change in SforceEnterpriseClient.php then at the bottom of salesforce_api_fieldmap_export_create, set $object->fieldsToNull to an array of Salesforce field names to see it in action.

The trouble I'm having now is correctly populating the fieldsToNull array. Will post back here when I have something working.

Comment #8

kostajh CreditAttribution: kostajh commented 7 April 2011 at 16:18

The issue in #6 is now committed (https://github.com/developerforce/Force.com-Toolkit-for-PHP/commit/c6e47...), thanks Aaron!

Now we have to adjust the code in salesforce_api_field_map_export_create.

What about something like:

    // Ignore null values for non-nillable fields
    if (!$value && !$nillable) {
      continue;
    } elseif (!$value) {
      $fieldsToNull[] = $sf_fieldname;
      continue;
    }

Comment #9

kostajh CreditAttribution: kostajh commented 7 April 2011 at 19:31

Status:

Active

» Needs review

File	Size
fieldsToNull-1096712-9.patch	1019 bytes

Ok, how about this? I did some testing and it works fine. I have to use !$value instead of is_null($value) to correctly grab all the fields that should be set to NULL in Salesforce.

Comment #10

kostajh CreditAttribution: kostajh commented 18 April 2011 at 14:47

Anyone else had a chance to test this? It is working well for me so far.

Comment #11

amariotti CreditAttribution: amariotti commented 20 April 2011 at 14:59

Another confirmation here that the patch does work both ways. Thanks.

Comment #12

kostajh CreditAttribution: kostajh commented 20 April 2011 at 18:24

Status:

Needs review

» Fixed

Thanks @amariotti, this is now committed to 6.x-2.x-dev.

Commit http://drupalcode.org/project/salesforce.git/commit/baea330

Comment #13

4 May 2011 at 18:32

Status:

Fixed

» Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

Comment #14

rjacobs CreditAttribution: rjacobs commented 30 June 2011 at 18:11

Component:	sf_user	» salesforce_api
Status:	Closed (fixed)	» Needs review

File	Size
Screen shot 2011-06-30 at 1.05.57 PM.png	61.74 KB

Sorry to re-open this, but I'm encountering some issues with syncing empty fields, and this seemed to be the most relevant issue thread (of several) on the topic.

I have the most recent dev version of the api and the most recent version of the php toolkit from https://github.com/developerforce/Force.com-Toolkit-for-PHP (it looks like the soapclient was last updated June 7th 2011, though I have no idea where I can access specific version info on it). I've also got some custom fields defined using hook_fieldmap_objects_alter(), a couple of which return a simple numeric value which can be 0.

The problem I'm finding is that whenever the value of my custom field is indeed 0, and a new object is being created in SF, the export fails with an error of "Unexpected element {http://www.w3.org/2001/XMLSchema}string during simple type deserialization". This seems to be the error that's been reported before, and which has seemingly been fixed?

If my custom field is non-zero on create, and then I later do a sync while changing the value of this field to 0, the export is successful. The problem only seems to relate to the creation of new objects when the value is 0.

I put some dpm statements in salesforce_api.module to check that my custom field (called "fees") is nillable and to show the object that's returned at the end of salesforce_api_fieldmap_export_create() (line 1066). The output of this is attached.

I should also note that the problematic export is being called via uc_salesforce, but as best I can tell (given that I'm using hook_fieldmap_objects_alter() to define these specific custom fields), the issue seems to stem from the salesforce API or salesforce/sf-toolkit.

Any insight is certainly appreciated!

Ryan

Comment #15

EvanDonovan CreditAttribution: EvanDonovan commented 2 July 2011 at 17:23

Issue tags:

+7.x-2.x

Tracking for 7.x-2.x.

Comment #16

tayzlor CreditAttribution: tayzlor commented 12 July 2011 at 15:22

FYI - I had to patch the salesforce toolkit inside SforceEnterPriseClient.php to behave similar to update() function -

  /**
   * Adds one or more new individual objects to your organization's data.
   * @param array $sObjects    Array of one or more sObjects (up to 200) to create.
   * @param AssignmentRuleHeader $assignment_header is optional.  Defaults to NULL
   * @param MruHeader $mru_header is optional.  Defaults to NULL
   * @return SaveResult
   */
  public function create($sObjects, $type) {
    foreach ($sObjects as &$sObject) {
      // FIX for fieldsToNull issue - allow array in fieldsToNull (STEP #1)
   		$xmlStr = '';
    	if(isset($sObject->fieldsToNull) && is_array($sObject->fieldsToNull)) {
    		foreach($sObject->fieldsToNull as $fieldToNull) {
    			$xmlStr .= '<fieldsToNull>' . $fieldToNull . '</fieldsToNull>';
    		}
    	}
      // ------
      
      $sObject = new SoapVar($sObject, SOAP_ENC_OBJECT, $type, $this->namespace);

      // FIX for fieldsToNull issue - allow array in fieldsToNull (STEP #2)
    	if($xmlStr != '') {
    		$sObject->enc_value->fieldsToNull = new SoapVar(new SoapVar($xmlStr, XSD_ANYXML), SOAP_ENC_ARRAY);
    	}
    	// ------
    }
    $arg = $sObjects;

    return parent::_create(new SoapParam($arg, "sObjects"));
  }

This sorted the problem with the null fields error on create. I need to look at getting a patch over on github.

Comment #17

longwave

he/him

English

CreditAttribution: longwave commented 21 July 2011 at 10:09

This should be fixed in the Salesforce toolkit shortly, see https://github.com/developerforce/Force.com-Toolkit-for-PHP/issues/11 (and the end of issue 4 where I originally reported it)

Comment #18

EvanDonovan CreditAttribution: EvanDonovan commented 28 October 2011 at 06:32

Was this ever fixed in the PHP Toolkit?

Comment #19

EvanDonovan CreditAttribution: EvanDonovan commented 28 October 2011 at 22:49

It looks like this is in the PHP toolkit now. I just saw the error from #1 on a Webform submission (6.x), but I am wondering if it was caused by something else.

Comment #20

rjacobs CreditAttribution: rjacobs commented 29 October 2011 at 02:07

Which version of sf_webform are you using? If I recall correctly you were once using a version that explicitly removed the $fieldsToNull part of the export object (independent of the API handling). In the most recent version that I have been working on (in your sandbox project) this is now handled differently... though I still can't confirm that it is 100% correct.