Remaining tasks since Drupal Developer Days 2015 and all the work done after that (and continuing) :

  • In the plugins configuration form, a validation error triggered with ajax is not removed when the form is re-submitted
  • Check the log for probable undefined index issues
  • We have too many createInstance plugin calls, we probably want to move those to a Utility Class (search in search_api code for ideas :))
  • All Remaining non tested D7 features:
    • field collection attachment indexation: Needs test
    • entityreference attachment indexation: Needs test
    • ...
  • Validate with http://pareview.sh/pareview/httpgitdrupalorgprojectsearchapiattachmentsg...
Members fund testing for the Drupal project. Drupal Association Learn more

Comments

Grimreaper’s picture

Issue summary: View changes
Grimreaper’s picture

Patch to allows txt parsing.

Grimreaper’s picture

Now it works with the tika library and for image files.

Two patches :
- one with two commits (the patch from comment #2 and another one)
- one with one commit

TODO :
path to tika and tika jar filename should be configurable.

Thytane’s picture

Two patches:
- One with two commit
- One with one commit

Note: Need to add a specific *jar on solr server to have the correct configuration.
File is solr-cell.jar (http://www.java2s.com/Code/JarDownload/apache-solr/apache-solr-cell-1.4....)
Put the file inside the folder: contrib/extraction/lib/

WIP - Extracting method raise an exception.

izus’s picture

Hi,
Here is a POC that needs more love and work from the Drupal Dev days Montpellier
We worked on that POC Bès and myself.
i'm adding the patch here for information.

Thanks

  • izus committed c826909 on 8.x-1.x
    Issue #2470553 by Bès, izus: D8 Port
    
Grimreaper’s picture

Issue tags: +#drupaldevdays

  • izus committed 0f1f858 on 8.x-1.x authored by Grimreaper
    Issue #2470553 by Grimreaper, izus: D8 POC add tika support
    

  • izus committed af2564b on 8.x-1.x authored by Grimreaper
    Issue #2470553 by Grimreaper, izus: D8 Port, Handle tika extractor.
    
izus’s picture

From Drupal Dev Days Montpellier, We merged needed things from #2 and #3

yannickoo’s picture

Issue tags: -#drupaldevdays +drupaldevdays
Grimreaper’s picture

Issue summary: View changes

  • izus committed 1fed547 on 8.x-1.x
    Issue #2470553 by izus: D8 POC add solr extract support
    
Grimreaper’s picture

Hello,

Here is a patch that add a tika live check and allows to empty the tika path variable.

  • izus committed 660ce6f on 8.x-1.x
    Issue #2470553 by izus: First attemption to implement solr extraction
    
izus’s picture

throwing a note to not forget that #14 needs to be adapted to D8

Grimreaper’s picture

Sorry, I uploaded the wrong patch. Because I made the D8 port. I will upload it this evening.

Grimreaper’s picture

The good patch that should have been in the 14th comment.

Grimreaper’s picture

Remade the patch in comment #18 to add the "-V" option as mentionned in https://www.drupal.org/node/2478925

  • izus committed a862bbd on 8.x-1.x authored by Grimreaper
    Issue #2470553 by Grimreaper, izus: Validate tika file execution.
    
izus’s picture

merged #19 after moving the code to thevalidate form handler instead of teh builder.
i will also push to use form Error instead of drupal_set_message
Thanks

  • izus committed eb070d4 on 8.x-1.x
    Issue #2470553 by izus: Validate tika file execution depending on...

  • izus committed 82809cd on 8.x-1.x
    Issue #2470553 by izus: Fix Solr extraction.
    

  • izus committed 20fa790 on 8.x-1.x
    Issue #2470553 by izus: Handle empty form value when submitted with ajax...

  • izus committed 18d7825 on 8.x-1.x
    Issue #2470553 by izus: Initialize README.txt file.
    

  • izus committed 04fb10f on 8.x-1.x
    Issue #2470553 by izus: Add real extraction testing upon configuration...

  • izus committed d39649e on 8.x-1.x
    Issue #2470553 by izus: Follow Core API changes about strings utility.
    

  • izus committed 06b2731 on 8.x-1.x
    Issue #2470553 by izus: Feed up the README.txt file.
    
  • izus committed a757378 on 8.x-1.x
    Issue #2470553 by izus: Add a real live extraction test upon form...
izus’s picture

Version: 7.x-1.x-dev » 8.x-1.x-dev

release 8.x-1.x-dev is now available

izus’s picture

Issue summary: View changes

  • izus committed b54dc77 on 8.x-1.x
    Issue #2470553 by izus: Solr index only what is in body tag.
    
Grimreaper’s picture

Hi Izus,

I have done a review of all your last commits.

I made a patch that:
- fix some typos
- remove statements highlighted as unused by my ide
- fix missing variable $extractor_plugin_id in buildTextExtractorTestResultForm() in /src/Form/TextExtractorFormSettings.php
- fix ide warnings on protected properties in getFileFields() in /src/Plugin/search_api/processor/FilesFieldsProcessorPlugin.php

After the review, I have some questions:
- in /src/Plugin/SearchApiAttachmentsTextExtractor/SolrExtractor.php: there is

$htmlencoder = new XmlEncoder();

Is the variable well named ? Or should it be new HTMLEncoder() ?

- in /src/Plugin/search_api/processor/FilesFieldsProcessorPlugin.php, in create():

/** @var \Drupal\Core\StringTranslation\TranslationInterface $translation */
    $translation = $container->get('string_translation');
    $plugin->setStringTranslation($translation);

What is it for ? I think it is the way to be able to translate the strings marked with @translation in the comments like

/**
 * @SearchApiAttachmentsTextExtractor(
 *   id = "solr_extractor",
 *   label = @Translation("Solr Extractor"),
 *   description = @Translation("Adds Solr extractor support."),
 * )
 */

Thanks for the review and the response.

Grimreaper’s picture

Hello,

I wanted to remove the todo: //@todo test connection live from SolrExtractor.php

But when I saved the config form the test extraction raised an error :

Solarium\Exception\HttpException: Solr HTTP error: OK (500) {"error":{"msg":"lazy loading error","trace":"org.apache.solr.common.SolrException: lazy loading error\n\tat org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.getWrappedHandler(RequestHandlers.java:262)\n\tat org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:244)\n\tat org.apache.solr.core.SolrCore.execute(SolrCore.java:1976)\n\tat org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:777)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)\n\tat org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)\n\tat org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:368)\n\tat org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)\n\tat org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)\n\tat org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)\n\tat org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)\n\tat org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:647)\n\tat org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)\n\tat org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)\n\tat org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)\n\tat java.lang.Thread.run(Thread.java:745)\nCaused by: org.apache.solr.common.SolrException: Error loading class 'solr.extraction.ExtractingRequestHandler'\n\tat org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:490)\n\tat org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:421)\n\tat org.apache.solr.core.SolrCore.createInstance(SolrCore.java:551)\n\tat org.apache.solr.core.SolrCore.createRequestHandler(SolrCore.java:624)\n\tat org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.getWrappedHandler(RequestHandlers.java:253)\n\t... 30 more\nCaused by: java.lang.ClassNotFoundException: solr.extraction.ExtractingRequestHandler\n\tat java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat java.security.AccessController.doPrivileged(Native Method)\n\tat java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:789)\n\tat java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat java.lang.Class.forName0(Native Method)\n\tat java.lang.Class.forName(Class.java:274)\n\tat org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:474)\n\t... 34 more\n","code":500}} in Solarium\Core\Query\Result\Result->__construct() (line 103 of modules/search_api_solr/vendor/solarium/solarium/library/Solarium/Core/Query/Result/Result.php).

Solarium\Core\Client\Client->createResult(Object, Object)
Solarium\Core\Client\Client->execute(Object, NULL)
Solarium\Core\Client\Client->extract(Object)
Drupal\search_api_attachments\Plugin\SearchApiAttachmentsTextExtractor\SolrExtractor->extract(Object)
Drupal\search_api_attachments\Form\TextExtractorFormSettings->submitForm(Array, Object)
call_user_func_array(Array, Array)
Drupal\Core\Form\FormSubmitter->executeSubmitHandlers(Array, Object)
Drupal\Core\Form\FormSubmitter->doSubmitForm(Array, Object)
Drupal\Core\Form\FormBuilder->processForm('search_api_attachments_admin_form', Array, Object)
Drupal\Core\Form\FormBuilder->buildForm(Object, Object)
Drupal\Core\Controller\FormController->getContentResult(Object, Object)
call_user_func_array(Array, Array)
Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object, 1)
Symfony\Component\HttpKernel\HttpKernel->handle(Object, 1, 1)
Drupal\Core\DependencyInjection\Container\prod\Symfony_Component_HttpKernel_HttpKernel_Proxy->handle(Object, 1, 1)
Drupal\Core\StackMiddleware\Session->handle(Object, 1, 1)
Drupal\Core\DependencyInjection\Container\prod\Drupal_Core_StackMiddleware_Session_Proxy->handle(Object, 1, 1)
Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object, 1, 1)
Drupal\Core\DependencyInjection\Container\prod\Drupal_Core_StackMiddleware_KernelPreHandle_Proxy->handle(Object, 1, 1)
Drupal\page_cache\StackMiddleware\PageCache->pass(Object, 1, 1)
Drupal\page_cache\StackMiddleware\PageCache->handle(Object, 1, 1)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object, 1, 1)
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object, 1, 1)
Drupal\webprofiler\WebprofilerMiddleware->handle(Object, 1, 1)
Stack\StackedHttpKernel->handle(Object, 1, 1)
Drupal\Core\DrupalKernel->handle(Object)

So I tried the what Thytane said in comment 4, and I got the following error:

Solarium\Exception\HttpException: Solr HTTP error: OK (500) {"error":{"msg":"java.lang.AbstractMethodError: org.apache.solr.handler.ContentStreamHandlerBase.newLoader(Lorg/apache/solr/request/SolrQueryRequest;Lorg/apache/solr/update/processor/UpdateRequestProcessor;)Lorg/apache/solr/handler/loader/ContentStreamLoader;","trace":"java.lang.RuntimeException: java.lang.AbstractMethodError: org.apache.solr.handler.ContentStreamHandlerBase.newLoader(Lorg/apache/solr/request/SolrQueryRequest;Lorg/apache/solr/update/processor/UpdateRequestProcessor;)Lorg/apache/solr/handler/loader/ContentStreamLoader;\n\tat org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:793)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:434)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)\n\tat org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)\n\tat org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:368)\n\tat org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)\n\tat org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)\n\tat org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)\n\tat org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)\n\tat org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:647)\n\tat org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)\n\tat org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)\n\tat org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)\n\tat java.lang.Thread.run(Thread.java:745)\nCaused by: java.lang.AbstractMethodError: org.apache.solr.handler.ContentStreamHandlerBase.newLoader(Lorg/apache/solr/request/SolrQueryRequest;Lorg/apache/solr/update/processor/UpdateRequestProcessor;)Lorg/apache/solr/handler/loader/ContentStreamLoader;\n\tat org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:63)\n\tat org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)\n\tat org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:246)\n\tat org.apache.solr.core.SolrCore.execute(SolrCore.java:1976)\n\tat org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:777)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)\n\t... 26 more\n","code":500}} in Solarium\Core\Query\Result\Result->__construct() (line 103 of modules/search_api_solr/vendor/solarium/solarium/library/Solarium/Core/Query/Result/Result.php).

Solarium\Core\Client\Client->createResult(Object, Object)
Solarium\Core\Client\Client->execute(Object, NULL)
Solarium\Core\Client\Client->extract(Object)
Drupal\search_api_attachments\Plugin\SearchApiAttachmentsTextExtractor\SolrExtractor->extract(Object)
Drupal\search_api_attachments\Form\TextExtractorFormSettings->submitForm(Array, Object)
call_user_func_array(Array, Array)
Drupal\Core\Form\FormSubmitter->executeSubmitHandlers(Array, Object)
Drupal\Core\Form\FormSubmitter->doSubmitForm(Array, Object)
Drupal\Core\Form\FormBuilder->processForm('search_api_attachments_admin_form', Array, Object)
Drupal\Core\Form\FormBuilder->buildForm(Object, Object)
Drupal\Core\Controller\FormController->getContentResult(Object, Object)
call_user_func_array(Array, Array)
Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object, 1)
Symfony\Component\HttpKernel\HttpKernel->handle(Object, 1, 1)
Drupal\Core\DependencyInjection\Container\prod\Symfony_Component_HttpKernel_HttpKernel_Proxy->handle(Object, 1, 1)
Drupal\Core\StackMiddleware\Session->handle(Object, 1, 1)
Drupal\Core\DependencyInjection\Container\prod\Drupal_Core_StackMiddleware_Session_Proxy->handle(Object, 1, 1)
Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object, 1, 1)
Drupal\Core\DependencyInjection\Container\prod\Drupal_Core_StackMiddleware_KernelPreHandle_Proxy->handle(Object, 1, 1)
Drupal\page_cache\StackMiddleware\PageCache->pass(Object, 1, 1)
Drupal\page_cache\StackMiddleware\PageCache->handle(Object, 1, 1)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object, 1, 1)
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object, 1, 1)
Drupal\webprofiler\WebprofilerMiddleware->handle(Object, 1, 1)
Stack\StackedHttpKernel->handle(Object, 1, 1)
Drupal\Core\DrupalKernel->handle(Object)

My solr version is 4.10.4. Is there something missing to document?

izus’s picture

for #33
Here is what i added to the README.txt file

Note 2: "lazy loading error"
If you obtain this error, you may need some extra configuration of solr:
Per example with solr 4.10.4, in addition to the configuration suggested in
search_api_solr README.txt, you need to update your solrconfig.xml file
(full path can look like example/solr/collection1/conf/solrconfig.xml)
Change the /update/extract request Handler class like this :

class="org.apache.solr.handler.extraction.ExtractingRequestHandler" >

This means that you delete this part:
- startup="lazy"
- class="solr.extraction.ExtractingRequestHandler" >

Then in example folder:
cp -r ../contrib/extraction/lib solr/collection1/lib
cp ../dist/solr-cell-4.10.4.jar solr/collection1/lib/

Please try it and confirm if it solves your issue.
it's just a way to solve it, i think #2352889: Update solrconfig.xml to support external libraries can make it better
(also some other ways to test may be found in #2254129: Lazy Loading Error )

  • izus committed 6db9979 on 8.x-1.x
    Issue #2470553 by izus: Better documentation for apache solr...
izus’s picture

merged #32 with a little fix

  • izus committed cb4bbf8 on 8.x-1.x authored by Grimreaper
    Issue #2470553 by Grimreaper, izus: Cleanup code
    
izus’s picture

answers for #32

- in /src/Plugin/SearchApiAttachmentsTextExtractor/SolrExtractor.php: there is

$htmlencoder = new XmlEncoder();

Is the variable well named ? Or should it be new HTMLEncoder() ?

there is only the XmlEncoder class
actually it can be called $xmlencoder2, i just avoided to have numbered variables once for encoding and the for decoding and well...html is a family member of xml (cousins i think :))
and also because naming variables is a hard problem ;)

- in /src/Plugin/search_api/processor/FilesFieldsProcessorPlugin.php, in create():

/** @var \Drupal\Core\StringTranslation\TranslationInterface $translation */
    $translation = $container->get('string_translation');
    $plugin->setStringTranslation($translation);

What is it for ? I think it is the way to be able to translate the strings marked with @translation in the comments like

/**
 * @SearchApiAttachmentsTextExtractor(
 *   id = "solr_extractor",
 *   label = @Translation("Solr Extractor"),
 *   description = @Translation("Adds Solr extractor support."),
 * )
 */

we inject the 'string_translation' service from the container to our Processor.
actually as we extend ProcessorPluginBase and i needed to inject our custom service plugin.manager.search_api_attachments.text_extractor, we just did what the parent class does + injecting our service :)

Grimreaper’s picture

Thanks for the answers. No problem about the variable name, it was a simple question :)

I test with the instructions in the README and it is ok thanks.

  • izus committed 5375c96 on 8.x-1.x
    Issue #2470553 by izus: Add excluded extensions management and file...

  • izus committed 2ad7945 on 8.x-1.x
    Issue #2470553 by izus: Add limitation to N items for multivalues fields...

  • izus committed bf5e368 on 8.x-1.x
    Issue #2470553 by izus: Limit indexation to permanent files.
    

  • izus committed 35a8fdc on 8.x-1.x
    Issue #2470553 by izus: Handle private files.
    

  • izus committed 811778c on 8.x-1.x
    Issue #2470553 by izus: Handle file size limitation before indexing.
    
izus’s picture

Issue summary: View changes
izus’s picture

Issue summary: View changes

  • izus committed a9c8378 on 8.x-1.x
    Issue #2470553 by izus: Make isFileIndexable return FALSE result as...

  • izus committed 8ae4d51 on 8.x-1.x
    Issue #2470553 by izus: Redirect to configuration page when no...

  • izus committed 15b5ce2 on 8.x-1.x
    Issue #2470553 by izus: Create a cache bin and store extracted data in...

  • izus committed 635c6fa on 8.x-1.x
    Issue #2470553 by izus: Delete cached extracted data of a file upon...
izus’s picture

Issue summary: View changes
izus’s picture

Category: Task » Plan

  • izus committed 69b5941 on 8.x-1.x
    Issue #2470553 by izus: Change SafeMarkup::checkPlain to Html::escape.
    
izus’s picture

Issue summary: View changes
izus’s picture

Issue summary: View changes
izus’s picture

Issue summary: View changes
izus’s picture

Issue summary: View changes
izus’s picture

Issue summary: View changes
izus’s picture

izus’s picture

Issue summary: View changes
Grimreaper’s picture

Status: Active » Needs review
FileSize
632 bytes

Hello,

Here is a small patch that add a composer.json.

Thanks for the review.

izus’s picture

is this required now ? can you please drop a link to some reading (#lazy_mode)
as we will not use an extra dependency (search_api_solr per example needs solarium project) i wonder if we need it as placeholder or it is required for some reason

Grimreaper’s picture

Yes,

On https://github.com/drupal-composer/drupal-project, you can see in their composer.json

"extra": {
        "installer-paths": {
            "web/core": ["type:drupal-core"],
            "web/modules/contrib/{$name}": ["type:drupal-module"],
            "web/profiles/contrib/{$name}": ["type:drupal-profile"],
            "web/themes/contrib/{$name}": ["type:drupal-theme"],
            "drush/contrib/{$name}": ["type:drupal-drush"]
        }
    }

So projects should have a composer.json with the appropriate type to help projects using composer to be able to place the contrib modules in the appropriate directory.

On many blog posts they invite every contrib project to do so. By example: https://blog.erdfisch.de/2014/08/composer-based-workflow-drupal-8 just after "Drupal 8 modules"

I don't find a page on the drupal.org. Do you want me to search more?

izus’s picture

  • izus committed 5ae3648 on 8.x-1.x
    #2470553 by izus: Fix code style.
    

  • izus committed 3b0678c on 8.x-1.x authored by Grimreaper
    Issue #2470553 by Grimreaper, izus: Add composer.json file.
    

  • izus committed 80f9fe2 on 8.x-1.x
    #2470553 by izus: Fix code style.
    

  • izus committed 3b9d106 on 8.x-1.x
    #2470553 by izus: Fix code style.
    

  • izus committed 590cc85 on 8.x-1.x
    #2470553 by izus: Fix code style.
    
izus’s picture

Issue summary: View changes

  • izus committed 3012698 on 8.x-1.x
    #2470553 by izus: Fix installation errors.
    

  • izus committed 6475889 on 8.x-1.x
    #2470553 by izus: Check if a field belongs to entity before indexing..
    
izus’s picture

Status: Needs review » Active

i made it work with last Drupal core version and last alpha version of search api + some little fixes.
Here is the Alpha 1 release :)
https://www.drupal.org/node/2634062

drunken monkey’s picture

Status: Active » Needs review
FileSize
4.01 KB

Great job here, seems to work really well!
However, there were recently some changes in both Drupal Core and the Search API which this doesn't yet seem to reflect. Please see the attached patch, which got this working as intended again for me.
In any case, thanks for the port!

borisson_’s picture

The patch in #74 looks great, we'll probably need more changes when #2638116: Clean up caching of Index class method results (especially fields) lands.

izus’s picture

Status: Needs review » Active

Thanks for the path and the review.
#74 is now merged in 8.x-1.x

Thank you again :)

  • izus committed 33462c1 on 8.x-1.x
    Issue #2470553 by izus: Fix errors.
    
izus’s picture

Issue summary: View changes
Grimreaper’s picture

Hello,

Sorry, I don't have time to provide a patch but I suggest that src/Plugin/SearchApiAttachmentsTextExtractor should be moved into src/Plugin/search_api_attachments/text_extractor

marco-s’s picture

It was not possible to index file content from file entity with the current Search API attachments version.
I've fixed that with the attached patch (Version 1!)
After adding the patch you can add the field "Search api attachments: File Entity" to your search index.

izus’s picture

patch in #81 is not accurate at all, the feature request was added in #2765601: Index content of files not attached to nodes

izus’s picture

Issue summary: View changes
Grimreaper’s picture

Hello,

Here is a patch that make a code cleanup and add dependency injection.

I am really sorry but I made the patch yesterday and I though I had submitted it. I hope it can still be applied. Thanks @izus for asking me in private.

I tested search_api_attachments yesterday with and without the patch on the following setup :
- drupal core 8.1.7-dev
- search_api 8.1.0-alpha16
- search_api_solr 8.1.0-alpha5
- search_api_page 8.1.0-alpha11 (as the views integration is broken)

izus’s picture

Status: Active » Needs work

hi,
the @file headers are needed for pareview.sh validations, so we should have them.
Also the patch doesn't apply (actually FilesFieldsExtrator is replaced by FilesExtrator (file and class name)) and the code there was altered changed a lot these 24 hours !

Replacements everywhere to use DI is wonderful ! Thanks

Grimreaper’s picture

Status: Needs work » Needs review
FileSize
29.63 KB

Hello,

Here is the rebased patch.

I tested it against a content index. I see that search_api_attachments should now support entity files, really cool thanks.

The @file doc is not needed for namespaced files.

Namespaced classes, interfaces and traits should not begin with a file doc comment

: http://cgit.drupalcode.org/coder/tree/coder_sniffer/Drupal/Sniffs/Commen...

Thanks for the review.

  • izus committed c898972 on 8.x-1.x authored by Grimreaper
    Issue #2470553 by Grimreaper, izus: Use DI everywhere.
    
izus’s picture

Status: Needs review » Active

merged #86
thanks

izus’s picture

Issue summary: View changes

  • izus committed dba7764 on 8.x-1.x
    Issue #2470553 by izus: Fix pareview.sh report.
    
izus’s picture

ohthehugemanatee’s picture

Is there a roadmap anywhere for getting to beta or stable release? Sorry, I haven't been able to find one, and this ticket seems like a good place for it.

drakythe’s picture

Asking the same question: Have we got a current status/roadmap update? I see a few RTBC patches but I'm unsure of where to get started, and the last alpha release was back in July, while search_api_solr module has been updated as recently as a week ago. Just looking for an update/place to jump in.

izus’s picture

Hi,
there is now alpha 5 release : https://www.drupal.org/project/search_api_attachments/releases/8.x-1.0-a...

a suggestion of roadmap (all suggestions of all contributors are welcome))

beta releases :
why not, to follow the search_api progress, but let's first reduce open issues in https://www.drupal.org/project/issues/search_api_attachments?text=&statu... (As of today there is 0 RTBTC issue)
and also, if people that tested search_api_attachments can drop a comment here to say if it worth going to beta versions or not yet (and what would should be done), or if people are happy with it and want it as beta as it's enough tested, then go for it :).

Stable releases:
No stable release before search_api itself has one :)

A realy vital need for the module's future: tests, tests and tests, we can't refactor anything/add a lot of new features without tests (all contributions are welcome)

Thanks

izus’s picture

Hi,
With the last merged patches, people that tested and the search_api general progress. i think we can move to a first beta release today.
i'll publish it today

Thanks to all contributors and testers