Background: RDF_sync module worked well before I accidentally removed the Virtuoso container and lost all the synchronized RDF data. I re-run a new Virtuoso container and connected it with Drupal. I tried manually batch write all my mapped nodes into Virtuoso as below:
vendor/bin/drush rdf_sync:synchronize node
It showed error as below:

In ProcessBase.php line 171:
                                                                                                                                                                                                                                
  Unable to decode output into JSON: Syntax error                                                                                                                                                                               
                                                                                                                                                                                                                                
  TypeError: Drupal\rdf_sync\RdfSyncSynchronizer::Drupal\rdf_sync\{closure}(): Return value must be of type string, null returned in Drupal\rdf_sync\RdfSyncSynchronizer->Drupal\rdf_sync\{closure}() (line 122 of /var/www/ht  
  ml/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php).                                                                                                                                                        
                                                                                                                                                                                                                                
  Fatal error: Uncaught TypeError: Drupal\rdf_sync\RdfSyncSynchronizer::Drupal\rdf_sync\{closure}(): Return value must be of type string, null returned in /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSync  
  hronizer.php:122                                                                                                                                                                                                              
  Stack trace:                                                                                                                                                                                                                  
  #0 [internal function]: Drupal\rdf_sync\RdfSyncSynchronizer->Drupal\rdf_sync\{closure}()                                                                                                                                      
  #1 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php(119): array_map()                                                                                                                          
  #2 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php(104): Drupal\rdf_sync\RdfSyncSynchronizer->doSynchronize()                                                                                 
  #3 /var/www/html/drupal/web/core/lib/Drupal/Core/EventSubscriber/KernelDestructionSubscriber.php(51): Drupal\rdf_sync\RdfSyncSynchronizer->destruct()                                                                       
  #4 [internal function]: Drupal\Core\EventSubscriber\KernelDestructionSubscriber->onKernelTerminate()                                                                                                                          
  #5 /var/www/html/drupal/web/core/lib/Drupal/Component/EventDispatcher/ContainerAwareEventDispatcher.php(111): call_user_func()                                                                                              
  #6 /var/www/html/drupal/vendor/symfony/http-kernel/HttpKernel.php(115): Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch()                                                                          
  #7 /var/www/html/drupal/web/core/lib/Drupal/Core/StackMiddleware/StackedHttpKernel.php(63): Symfony\Component\HttpKernel\HttpKernel->terminate()                                                                            
  #8 /var/www/html/drupal/web/core/lib/Drupal/Core/DrupalKernel.php(688): Drupal\Core\StackMiddleware\StackedHttpKernel->terminate()                                                                                          
  #9 /var/www/html/drupal/vendor/drush/drush/src/Boot/DrupalBoot8.php(326): Drupal\Core\DrupalKernel->terminate()                                                                                                             
  #10 [internal function]: Drush\Boot\DrupalBoot8->terminate()                                                                                                                                                                  
  #11 {main}                                                                                                                                                                                                                    
    thrown in /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php on line 122

When I created a node in Drupal Web UI, it was synchronized into Virtuoso.
Is it because there are many entity_reference fields between bundles? (Edit: Because when I manually added a new node, it did write RDF into Virtuoso.)

Issue fork rdf_sync-3472816

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

fishfree created an issue. See original summary.

claudiu.cristea’s picture

claudiu.cristea
Primary language Romanian
Location Arad 🇷🇴
commented
Category: Bug report » Support request
Status: Active » Postponed (maintainer needs more info)

I need more context to understand your case

claudiu.cristea’s picture

claudiu.cristea
Primary language Romanian
Location Arad 🇷🇴
commented

Also, could you, please, enter a proper title for this issue and not paste the error message. Thanks

fishfree’s picture

fishfree commented
Title: TypeError: Drupal\rdf_sync\RdfSyncSynchronizer::Drupal\rdf_sync\{closure}(): Return value must be of type string, null returned in Drupal\rdf_sync\RdfSyncSynchronizer->Drupal\rdf_sync\{closure}() (line 122 of /var/www/html/drupal/web/modules/con » "Return value must be of type string, null returned" when running vendor/bin/drush rdf_sync:synchronize node
Issue summary: View changes
fishfree’s picture

fishfree commented
Issue summary: View changes
claudiu.cristea’s picture

claudiu.cristea
Primary language Romanian
Location Arad 🇷🇴
commented
Status: Postponed (maintainer needs more info) » Fixed

I tried manually batch write all my mapped nodes into Virtuoso as below:

This happens when one ore more nodes don't have a corresponding record in the rdf_sync_uri table (or for some reason the uri column is empty). Check and fix those cases

claudiu.cristea’s picture

claudiu.cristea
Primary language Romanian
Location Arad 🇷🇴
commented
Category: Support request » Bug report
Status: Fixed » Needs work

I have decided to give a 2nd look at this. I think it would be correct to handle the case when, accidentally, some entities URIs were lost and prevent any ugly error.

claudiu.cristea opened merge request !24

  • claudiu.cristea committed bbfae774 on 1.x 
    Issue #3472816 by claudiu.cristea, fishfree: "Return value must be of...
claudiu.cristea’s picture

claudiu.cristea
Primary language Romanian
Location Arad 🇷🇴
commented
Status: Needs work » Fixed

Thank you for reporting

fishfree’s picture

fishfree commented

@claudiu Thank you! After apply your patch, my error turned into:

In ProcessBase.php line 171:
                                                                                                                                                                
  Unable to decode output into JSON: Syntax error                                                                                                               
                                                                                                                                                                
  Fatal error: Uncaught InvalidArgumentException: $uri should be a string and cannot be null or empty in /var/www/html/drupal/vendor/sweetrdf/easyrdf/lib/Re  
  source.php:69                                                                                                                                                 
  Stack trace:                                                                                                                                                  
  #0 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/Normalizer/RdfSyncNormalizer.php(61): EasyRdf\Resource->__construct()                              
  #1 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php(133): Drupal\rdf_sync\Normalizer\RdfSyncNormalizer->normalize()            
  #2 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php(104): Drupal\rdf_sync\RdfSyncSynchronizer->doSynchronize()                 
  #3 /var/www/html/drupal/web/core/lib/Drupal/Core/EventSubscriber/KernelDestructionSubscriber.php(51): Drupal\rdf_sync\RdfSyncSynchronizer->destruct()       
  #4 [internal function]: Drupal\Core\EventSubscriber\KernelDestructionSubscriber->onKernelTerminate()                                                          
  #5 /var/www/html/drupal/web/core/lib/Drupal/Component/EventDispatcher/ContainerAwareEventDispatcher.php(111): call_user_func()                              
  #6 /var/www/html/drupal/vendor/symfony/http-kernel/HttpKernel.php(115): Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch()          
  #7 /var/www/html/drupal/web/core/lib/Drupal/Core/StackMiddleware/StackedHttpKernel.php(63): Symfony\Component\HttpKernel\HttpKernel->terminate()            
  #8 /var/www/html/drupal/web/core/lib/Drupal/Core/DrupalKernel.php(688): Drupal\Core\StackMiddleware\StackedHttpKernel->terminate()                          
  #9 /var/www/html/drupal/vendor/drush/drush/src/Boot/DrupalBoot8.php(326): Drupal\Core\DrupalKernel->terminate()                                             
  #10 [internal function]: Drush\Boot\DrupalBoot8->terminate()                                                                                                  
  #11 {main}                                                                                                                                                    
    thrown in /var/www/html/drupal/vendor/sweetrdf/easyrdf/lib/Resource.php on line 69
claudiu.cristea’s picture

claudiu.cristea
Primary language Romanian
Location Arad 🇷🇴
commented
Status: Fixed » Needs work

I see the same bug in my project :(

claudiu.cristea opened merge request !25

claudiu.cristea’s picture

claudiu.cristea
Primary language Romanian
Location Arad 🇷🇴
commented
Status: Needs work » Needs review

@fishfree, could you, please, test the latest MR?

fishfree’s picture

fishfree commented

@claudiucristea Thank you very much! Would you pls share a single patch file against the version https://www.drupal.org/project/rdf_sync/releases/1.0.0-alpha12 ? Sorry I'm not able to manually merge your multiple MRs.

claudiu.cristea’s picture

claudiu.cristea
Primary language Romanian
Location Arad 🇷🇴
commented

You should just go to MR and append .diff to URL

fishfree’s picture

fishfree commented

@claudiucristea Thank you very much! After applying https://git.drupalcode.org/project/rdf_sync/-/merge_requests/25.diff based on https://www.drupal.org/project/rdf_sync/releases/1.0.0-alpha12, I re-run vendor/bin/drush rdf_sync:synchronize node
It showed as below:

> ......
 > [notice] Synchronized 1200 out of 1965 entities
>  [notice] Synchronized 1250 out of 1965 entities
>  [notice] Synchronized 1300 out of 1965 entities
>  [notice] Synchronized 1350 out of 1965 entities
> 
> In Resource.php line 69:
>                                                        
>   $uri should be a string and cannot be null or empty  
>                                                        
> 
> PHP Fatal error:  Uncaught InvalidArgumentException: $uri should be a string and cannot be null or empty in /var/www/html/drupal/vendor/sweetrdf/easyrdf/lib/Resource.php:69
> Stack trace:
> #0 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/Normalizer/RdfSyncNormalizer.php(108): EasyRdf\Resource->__construct()
> #1 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php(122): Drupal\rdf_sync\Normalizer\RdfSyncNormalizer->normalize()
> #2 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php(88): Drupal\rdf_sync\RdfSyncSynchronizer->doSynchronize()
> #3 /var/www/html/drupal/web/core/lib/Drupal/Core/EventSubscriber/KernelDestructionSubscriber.php(51): Drupal\rdf_sync\RdfSyncSynchronizer->destruct()
> #4 [internal function]: Drupal\Core\EventSubscriber\KernelDestructionSubscriber->onKernelTerminate()
> #5 /var/www/html/drupal/web/core/lib/Drupal/Component/EventDispatcher/ContainerAwareEventDispatcher.php(111): call_user_func()
> #6 /var/www/html/drupal/vendor/symfony/http-kernel/HttpKernel.php(115): Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch()
> #7 /var/www/html/drupal/web/core/lib/Drupal/Core/StackMiddleware/StackedHttpKernel.php(63): Symfony\Component\HttpKernel\HttpKernel->terminate()
> #8 /var/www/html/drupal/web/core/lib/Drupal/Core/DrupalKernel.php(688): Drupal\Core\StackMiddleware\StackedHttpKernel->terminate()
> #9 /var/www/html/drupal/vendor/drush/drush/src/Boot/DrupalBoot8.php(326): Drupal\Core\DrupalKernel->terminate()
> #10 [internal function]: Drush\Boot\DrupalBoot8->terminate()
> #11 {main}
>   thrown in /var/www/html/drupal/vendor/sweetrdf/easyrdf/lib/Resource.php on line 69

In ProcessBase.php line 171:
                                                                                                    
  Unable to decode output into JSON: Syntax error                                                   
                                                                                                    
  Fatal error: Uncaught InvalidArgumentException: $uri should be a string and cannot be null or em  
  pty in /var/www/html/drupal/vendor/sweetrdf/easyrdf/lib/Resource.php:69                         
  Stack trace:                                                                                      
  #0 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/Normalizer/RdfSyncNormalizer.php(108)  
  : EasyRdf\Resource->__construct()                                                                 
  #1 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php(122): Drupal\  
  rdf_sync\Normalizer\RdfSyncNormalizer->normalize()                                                
  #2 /var/www/html/drupal/web/modules/contrib/rdf_sync/src/RdfSyncSynchronizer.php(88): Drupal\r  
  df_sync\RdfSyncSynchronizer->doSynchronize()                                                      
  #3 /var/www/html/drupal/web/core/lib/Drupal/Core/EventSubscriber/KernelDestructionSubscriber.p  
  hp(51): Drupal\rdf_sync\RdfSyncSynchronizer->destruct()                                           
  #4 [internal function]: Drupal\Core\EventSubscriber\KernelDestructionSubscriber->onKernelTermina  
  te()                                                                                              
  #5 /var/www/html/drupal/web/core/lib/Drupal/Component/EventDispatcher/ContainerAwareEventDispa  
  tcher.php(111): call_user_func()                                                                  
  #6 /var/www/html/drupal/vendor/symfony/http-kernel/HttpKernel.php(115): Drupal\Component\Event  
  Dispatcher\ContainerAwareEventDispatcher->dispatch()                                              
  #7 /var/www/html/drupal/web/core/lib/Drupal/Core/StackMiddleware/StackedHttpKernel.php(63): Sy  
  mfony\Component\HttpKernel\HttpKernel->terminate()                                                
  #8 /var/www/html/drupal/web/core/lib/Drupal/Core/DrupalKernel.php(688): Drupal\Core\StackMiddl  
  eware\StackedHttpKernel->terminate()                                                              
  #9 /var/www/html/drupal/vendor/drush/drush/src/Boot/DrupalBoot8.php(326): Drupal\Core\DrupalKe  
  rnel->terminate()                                                                                 
  #10 [internal function]: Drush\Boot\DrupalBoot8->terminate()                                      
  #11 {main}                                                                                        
    thrown in /var/www/html/drupal/vendor/sweetrdf/easyrdf/lib/Resource.php on line 69

I checked in my Virtuoso, it seems all the 1965 entities had been synchonized. Why still the error occured?

dimilias’s picture

dimilias commented

Hello @fishfree

From what I see, it seems that you might not have a URI for all entities. Probably you can check directly by running a quick query like 

select * from node as n
left join rdf_sync_uri as r on n.nid = r.entity_id
where r.uri is NULL and n.type = 'my_bundle'

if we are talking about nodes and re run this for every bundle to see which IDs are empty.

Furthermore, I created the following script (works for me with nodes) to iterate through all entity types in order to check all of them for empty URIs. Sorry for all the container calls, it was created in haste. After running it with drush php:script ./my-script.php and you verify if I am correct, you can also uncomment the commented section designated, to also create URIs for the IDs that are missing them and update the entities. Please, note, that this will actually update the entities. If you do not wish them updated, adjust accordingly or manually force the values in the rdf_sync_uri table.

<?php

use Drupal\Core\Entity\ContentEntityInterface;

$rdfMapper = \Drupal::getContainer()->get('rdf_sync.mapper');
$entityTypeManager = \Drupal::entityTypeManager();
$entityFieldManager = \Drupal::service('entity_field.manager');
$messages = [];
foreach ($entityTypeManager->getDefinitions() as $entityTypeId => $entityType) {
  if (!$entityType->entityClassImplements(ContentEntityInterface::class) || !($bundleEntityTypeId = $entityType->getBundleEntityType())) {
    continue;
  }

  $bundleStorage = $entityTypeManager->getStorage($bundleEntityTypeId);
  foreach ($bundleStorage->loadMultiple() as $bundleId => $bundle) {
    if ($rdfSyncSettings = $bundle->getThirdPartySettings('rdf_sync')) {
      // Get the base table of the entity.
      $baseTable = $entityType->getBaseTable();
      // Get the entity key for the ID.
      $idKey = $entityType->getKey('id');
      // Get the bundle key.
      $bundleIdKey = $entityType->getKey('bundle');

      // Load from the database IDs of the entity type that do not have do not
      // have a URI stored. Select IDs that in the `rdf_sync_uri` table, they
      // do not have a mapping to the columns `entity_type`, `entity_id`, and
      // `bundle`, or do, but the `uri` is empty.
      $query = \Drupal::database()->select($baseTable, 'e');
      $query->leftJoin('rdf_sync_uri', 'u', "e.{$idKey} = u.entity_id");
      $query->fields('e', [$idKey]);
      $query->condition("e.{$bundleIdKey}", $bundleId);
      $query->isNull('u.uri');
      $entityIds = $query->execute()->fetchCol();
      if ($entityIds) {
        $messages[] = "Entity type: $entityTypeId, bundle: $bundleId, IDs with missing URIs: " . implode(', ', $entityIds);
      }

      // Uncomment below to also appoint new URIs.
//      foreach ($entityTypeManager->getStorage($entityTypeId)->loadMultiple($entityIds) as $entity) {
//        $pluginId = $rdfMapper->getRdfUriPluginId($entityTypeId, $bundleId);
//        $plugin = \Drupal::getContainer()->get('plugin.manager.rdf_sync.uri_generator')->createInstance($pluginId);
//        $uri = $plugin->setEntity($entity)->generate();
//        $fieldName = \Drupal::getContainer()->get('rdf_sync.mapper')->getRdfUriFieldName(entity: $entity);
//        $entity->get($fieldName)->setValue($uri);
//        $entity->save();
//        $messages[] = "Entity type: $entityTypeId, bundle: $bundleId, ID: {$entity->id()}, New URI appointed: $uri";
//      }
    }
  }
}

print implode("\n", $messages);

Further more, in https://www.drupal.org/project/rdf_sync/issues/3478179, we are working on allowing a subscriber to tamper with the entities before syncing them. That means, that using this subscriber, and a simple check whether the entity does have a URI, you can either identify or block the entity from being synced.

Please, consider that, the error you are getting is based on the idea that having a bundle synced, all entities should have a URI. That is why if a bundle is mapped, the getUri should always return a value and we do not allow it to return null. That is because a URI is like a persistent identifier in RDF data and thus we thought we should be strict with it.

I checked in my Virtuoso, it seems all the 1965 entities had been synchonized. Why still the error occured?

This might be that the URI was removed somewhere along the way or that Virtuoso already had the data before you sync. If you are using docker, and the container crashed, it doesn't mean data were deleted if the volume remained intact.
Also, if the functionality was enabled after a few entities were created, and are left without a URI from the beginning, the whole rdf_sync would work in general, but the mass export would fail.
I mean, we would need more information to identify further issues.

In terms of this ticket, I will review the second MR because in the first one, the deletion of triples was removed and now data cannot be removed when the entity is deleted.

dimilias’s picture

dimilias commented

We have discussed with Claudiu, I will merge MR 25 and will continue to https://www.drupal.org/project/rdf_sync/issues/3478179 to conclude the tests.

dimilias’s picture

dimilias commented

Sorry, here is an updated script that works. I tested it in my case (I managed to replicate it by tampering with the data in the table)

<?php

use Drupal\Core\Entity\ContentEntityInterface;

$rdfMapper = \Drupal::getContainer()->get('rdf_sync.mapper');
$entityTypeManager = \Drupal::entityTypeManager();
$entityFieldManager = \Drupal::service('entity_field.manager');
$messages = [];
foreach ($entityTypeManager->getDefinitions() as $entityTypeId => $entityType) {
  if (!$entityType->entityClassImplements(ContentEntityInterface::class) || !($bundleEntityTypeId = $entityType->getBundleEntityType())) {
    continue;
  }

  $bundleStorage = $entityTypeManager->getStorage($bundleEntityTypeId);
  foreach ($bundleStorage->loadMultiple() as $bundleId => $bundle) {
    if ($rdfSyncSettings = $bundle->getThirdPartySettings('rdf_sync')) {
      // Get the base table of the entity.
      $baseTable = $entityType->getBaseTable();
      // Get the entity key for the ID.
      $idKey = $entityType->getKey('id');
      // Get the bundle key.
      $bundleIdKey = $entityType->getKey('bundle');

      // Load from the database IDs of the entity type that do not have do not
      // have a URI stored. Select IDs that in the `rdf_sync_uri` table, they
      // do not have a mapping to the columns `entity_type`, `entity_id`, and
      // `bundle`, or do, but the `uri` is empty.
      $query = \Drupal::database()->select($baseTable, 'e');
      $query->leftJoin('rdf_sync_uri', 'u', "e.{$idKey} = u.entity_id");
      $query->fields('e', [$idKey]);
      $query->condition("e.{$bundleIdKey}", $bundleId);
      $query->isNull('u.uri');
      $entityIds = $query->execute()->fetchCol();
      if ($entityIds) {
        $messages[] = "Entity type: $entityTypeId, bundle: $bundleId, IDs with missing URIs: " . implode(', ', $entityIds);
      }

      // Uncomment below to also appoint new URIs.
      foreach ($entityTypeManager->getStorage($entityTypeId)->loadMultiple($entityIds) as $entity) {
        $pluginId = $rdfMapper->getRdfUriPluginId($entityTypeId, $bundleId);
        $plugin = \Drupal::getContainer()->get('plugin.manager.rdf_sync.uri_generator')->createInstance($pluginId);
        $uri = $plugin->setEntity($entity)->generate();
        \Drupal::database()
          ->insert('rdf_sync_uri')
          ->fields(['entity_type', 'entity_id', 'uri', 'bundle'])
          ->values([
            'entity_type' => $entity->getEntityTypeId(),
            'entity_id' => $entity->id(),
            'uri' => $uri,
            'bundle' => $entity->bundle(),
          ])->execute();
        $messages[] = "Entity type: $entityTypeId, bundle: $bundleId, ID: {$entity->id()}, New URI appointed: $uri";
      }
    }
  }
}

print implode("\n", $messages);
claudiu.cristea’s picture

claudiu.cristea
Primary language Romanian
Location Arad 🇷🇴
commented

@fishfree, did you try to test with suggestions from @idimopoulos? I want to close this issue.

fishfree’s picture

fishfree commented

For me, the problem still exists after upgrading to the latest dev version of rdf_sync and running vendor/bin/drush php:script ./my-script.php.

claudiu.cristea’s picture

claudiu.cristea
Primary language Romanian
Location Arad 🇷🇴
commented
Status: Needs review » Postponed (maintainer needs more info)

We cannot do anything else here. If you can give us more details, let us know