As far as I understand, Embed.ly supports embedding any url, not just the url of it's 'featured' providers. I realize you can create a 'custom' provider and use the generic embed.ly endpoint (http://api.embed.ly/1/oembed)....my question is: Why is the "Schemes" field even necessary? Why not the ability to leave it blank (or just put in *) to allow oembeds of any url?

I'm just not sure if there's something I'm not understanding that makes adding individual urls necessary when, it seems in many (most?) cases, site owners would use this to allow embedding of anything. (in my case, I'd have to add hundreds of urls to allow...)

The 2 parts of this request/question are:

1. Question: Is there a simple 'accept all urls' token I can put in that field so I don't have to specify each domain?

2. If this is possible, I'm suggesting is a 'catch all' provider in the module that accepts any url when you enable the embed.ly integration module.

CommentFileSizeAuthor
#27 1539632-oembedembedly-additional-schemes-27.patch1.44 KBdixon_
#3 oembed-catchall-1539632.patch1.47 KBmuriqui
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

jacobpov’s picture

jacobpov CreditAttribution: jacobpov commented

I don't understand the reason why this hasn't been implemented yet ?

Rob_Feature’s picture

Rob_Feature CreditAttribution: Rob_Feature commented
Priority: Normal » Critical

Marking this critical. There's really no way to use this module if you have to add every single url pattern you ever want to use. Any thoughts on this from the devs?

muriqui’s picture

muriqui CreditAttribution: muriqui commented
Category: support » feature
Status: Active » Needs review
FileSize
oembed-catchall-1539632.patch1.47 KB

An easy solution might be to designate one existing provider as the "catch all" in oembedcore's settings (admin/config/media/oembed/settings), and then fail back to it in oembedcore_get_provider() when no other providers match the given URL... See attached patch. I've left the default catch all set to "none," so the module's processing doesn't change after applying the patch. Two nice things about this approach: 1) it's a relatively tidy change (only 4 lines, plus the settings form), and 2) it's not specific to embed.ly, so other services could be used as the catch all instead. Thoughts?

Rob_Feature’s picture

Rob_Feature CreditAttribution: Rob_Feature commented

I installed the patch and I think you may have misunderstood the issue: We basically need a provider that will work with any url without having to enter them all in. The patch provides a fallback provider, but the problem is, there's no fallback we can create which allows any url.

Am I missing something?

muriqui’s picture

muriqui CreditAttribution: muriqui commented

Actually, the patch is doing what you wanted; I just didn't document it well enough. :) Basically, when you designate a catch all provider in the settings, oEmbed is going to use that provider regardless of what patterns are set in the provider's Schemes box. So if you set it to your embed.ly provider, you can blank out the Schemes box on that provider (or leave them in, it doesn't matter) and it'll still trigger for every URL.

Rob_Feature’s picture

Rob_Feature CreditAttribution: Rob_Feature commented

Errr...actually, maybe you DID work magic :) I was now able to enter in http://* for my 'schemes' in a custom provider and it worked.

I don't understand why this didn't work before, though. Why can I enter in this scheme now and it's allowed, where before it wasn't. I don't see in your code where that would change.

Rob_Feature’s picture

Rob_Feature CreditAttribution: Rob_Feature commented
Status: Needs review » Reviewed & tested by the community

Brilliant. Marking tested and to be committed.

muriqui’s picture

muriqui CreditAttribution: muriqui commented
Status: Reviewed & tested by the community » Needs review

See previous comment. :) Entering http://* into Schemes really has no effect, and the module couldn't be made to work that way without a significant rewrite. Designating a catch all simply means that the Schemes patterns for that provider will be ignored... It will use the catch all for any URL that doesn't match the Scheme patterns on any other provider. You can put whatever you like in the catch all's Schemes, it won't make a difference.

muriqui’s picture

muriqui CreditAttribution: muriqui commented
Status: Needs review » Reviewed & tested by the community

Fixing status... darn cross-posting.

jjclint’s picture

jjclint CreditAttribution: jjclint commented

I think there might be some unforeseen bugs with the patch at #3.

I use Feeds+Media Feeds to import nodes with image and video content from a remote website.

In the feed mappers for both image and video fields I have it set on 'media-internet' rather than 'by fie name from library'. This was all working quite well except for certain youtube urls.

After installing the patch I was able to embed those youtube urls as well, but now I get the following error message:

"Missing bundle property on entity of type file."

I've checked the file_managed table in the sql and those files were indeed missing the 'filemyme' and the 'type' values.

Also the image file wasn't present at my local server as it should have been.

The uri was also wrong as it was leading to 'oembed://http...' and not to 'public://...' as it should.

I think the patch might be a little bit aggressive.

Rob_Feature’s picture

Rob_Feature CreditAttribution: Rob_Feature commented

My guess is that if you tried to embed youtube content and it ended up using the 'fall back' provider from this patch then there's actaully something wrong with the youtube provider implementation...otherwise, why would it fall back to this? (it should only do that if it can't find a provider, which means it didn't pickup the youtube provider) Is there a url pattern missing from the embed.ly youtube provider? (do you have the embed.ly integration enabled?)

It should be that if it makes it to using the fallback, it's because everything else failed to process.

muriqui’s picture

muriqui CreditAttribution: muriqui commented

Hi JJ... I have to agree with Bob on this. All the patch is doing is allowing one oembed provider to kick in when none of the others match; in effect, if you picked one of your oembed providers and typed into its Schemes field every conceivable URL that you might want to use on your site, you'd get the same result as you get from this patch... All the patch is doing is saving you from having to type a million URLs if you have a provider that works for everything, like embed.ly. If you look at the patch code, you'll see that it's literally just a few lines added after the URL pattern matching function that basically say "if no provider matched, pick this one."

The patch doesn't do anything with bundles, entities, or the file_managed table, and it certainly doesn't do anything with oembed:// vs. public:// stream wrappers. So wherever those errors are coming from on your site, I'd wager it's a separate issue between oEmbed and Feeds/Media Feeds. If you remove the patch and just add the URL pattern of whatever you were trying to embed to whichever provider you were using as the catch all, you should get the same errors.

jjclint’s picture

jjclint CreditAttribution: jjclint commented

Reading my comment again I realize that I haven't made my self clear enough, sorry for that.

The youtube video works fine, it's the image content that I'm having a problem with. Both the video and the image are being pulled from a 3rd party website (not youtube) the video is a url to a youtube video and the image is stored locally on the 3rd party website.

As I've said the video embeds perfectly but the image is missing it's file myme and type, does not download to my server as it should and has the wrong stream wrapper.

At first I thought that the patch falls back for the image as well leading to the wrong wrapper being applied and the other stuff which I've mentioned. But, after reading your comment I'm not so sure and well have to check it further.

At the moment I'm resorting to not using the embed.ly endpoint (I use the youtube/oembed endpoint) and regex changing all youtube's url patterns to 'www.youtube.com/watch?=' for single videos and to 'www.youtube.com/playlist?list=PL' for playlists. As for some reason those 2 patterns are the only ones that seem to work.

I have to admit that I don't exactly understand the difference between the embed.ly endpoint to the other endpoints.

Rob_Feature’s picture

Rob_Feature CreditAttribution: Rob_Feature commented

I think jjclint makes an important point here that's worth future users of this patch knowing about: We're really only testing the 'fallback' endpoint as using the embed.ly service. Unless the fallback enpoint will actually process any url, then this is going to break.

For example, let's say you put the youtube endpoint in as your fallback. It's basically going to fail on anything that's not a youtube url.

The key that you're missing, jjclint, is that the embed.ly endpoint is built to process any url. Youtube is only built to process youtube urls.

So one requirement of using this patch is that the endpoint you put in as your fallback can actually process any url that doesn't match others (and remember, if processing fails it's not because this patch didn't work...it's because your fallback endpoint didn't process it properly)

Rob_Feature’s picture

Rob_Feature CreditAttribution: Rob_Feature commented

This patch is demod in this video podcast.

joeysantiago’s picture

joeysantiago CreditAttribution: joeysantiago commented

Is there anything for the 6.x version of the module?

Rob_Feature’s picture

Rob_Feature CreditAttribution: Rob_Feature commented

We're only working with the D7 version, but I'd think someone could probably back port this pretty easily (?)

muriqui’s picture

muriqui CreditAttribution: muriqui commented

If/when the D7 patch gets committed, I might be willing to roll a backport for D6.

Anonymous’s picture

Anonymous (not verified) CreditAttribution: Anonymous commented
Status: Reviewed & tested by the community » Needs work

I am not against this patch, but let's be clear, it's not oEmbed. This is embed.ly taking advantage of widespread support for OpenGraph and simply returning a link representation based on that metadata. You could easily ask `drupal_http_request` to do this too!

Use of schemas allows two important things: matching links to providers and allowing site owners to control the content embedded by users on their site. This patch complements another one #1387386: Consolidate embed.ly into a single oembed provider quite well. That patch replaces all of embedly's providers with a single one that offers a menu of services.

This patch should be a added to the oEmbed Embed.ly module and not to the core module of this project. The neglected oEmbed spec could never have predicted embed.ly's wicked magic, but I think we should try to keep some order around here.

I've not tested this patch, but I will review it and if necessary commit it if it's made on oEmbed embed.ly. And let's do something about 1.0 too. Can someone look at my other patch please?

seb.pisarski’s picture

seb.pisarski CreditAttribution: seb.pisarski commented

What about the possibility of excluding internal url's from "catch all". My site has restricted access so if a member posts a link pointing to one of my site pages (for instance reporting broken link etc.) embed.ly won't be able to reach the page. I see it already in my logs. I guess one external request won't hurt performance too much but if this escalates ...?

jacobpov’s picture

jacobpov CreditAttribution: jacobpov commented

thanks patch working great just what I needed but the problem is that it just displays title and URL which is not enough the module has so much more potential to extract more data .

Joe_O’s picture

Joe_O CreditAttribution: Joe_O commented

I'm a bit confused by the patch for the module itself. Line 240 does not start with what is mentioned, did something change causing the patch to need updating?

Anonymous’s picture

Anonymous (not verified) CreditAttribution: Anonymous commented
Status: Needs work » Fixed

With the patch I committed today, this is now possible.

There is now only one embedly provider rather than one for each site embedly supports. The unified provider has its own configuration screen to choose which services you want the Drupal module's provider to support. There's also a place to add additional schemas which you want Embedly to handle. If you add these two schemes, you're telling the embedly provider to match every URL it sees.

http://*
https://*

This may or may not be a good idea, but it's supported now. Many times, you'll get back a `link` type oembed response, which can contain lots of stuff you can't see on the surface. Use the test interface to see how embedly handles different links.

You can create special theme implementations for specific providers and types by overriding the theme_oembed function and naming with more specificity. The pattern is this: theme_oembed__TYPE__PROVIDER__SUBTYPE. So for embedly links that are handled by additional schemas, the theme function `MYTHEME__oembed__link__embedly__embedly` will let you customize the output.

See http://drupal.org/commitlog/commit/10488/4092dfea8c8f507b41f291a58eae53c... and wait for the next development release.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

jddeli’s picture

jddeli CreditAttribution: jddeli commented
Title: Embedding any URL » Bookmarklets
Category: feature » support
Priority: Critical » Normal
Status: Closed (fixed) » Active

Sorry Rob_Feature but your videopodcast this video podcast creating bookmarklets for mysite is very difficult.
There is no explanations for creating bookmarklets.
This is make the solution for embed not working.

muriqui’s picture

muriqui CreditAttribution: muriqui commented
Title: Bookmarklets » Embedding any URL
Category: support » feature
Priority: Normal » Critical
Status: Active » Closed (fixed)

jddeli, please don't hijack existing issues like that. This feature request was resolved and closed. If you need help using the module on your site, please open a new issue.

dixon_’s picture

dixon_
he/him
Primary language English
Location France
CreditAttribution: dixon_ commented
Category: feature » bug
Priority: Critical » Major
Status: Closed (fixed) » Needs review
FileSize
1539632-oembedembedly-additional-schemes-27.patch1.44 KB

This functionality seems to have got lost somewhere since the commit in #23. The form element for entering the additional schemes are there, but it doesn't do anything.

The attached patch is against the 7.x-1.x branch in git and should fix the problem.

BarwonHack’s picture

BarwonHack CreditAttribution: BarwonHack commented

Who is willing to fix this issue in the two dev streams and then take it to a formal release?
I'll put some money on the table.

jfrederick’s picture

jfrederick CreditAttribution: jfrederick commented
Issue summary: View changes

The patch in #27 allowed me to retrieve expected response data from arbitrary URLs entered on the oEmbed test page.

That is, if I have the oembedembedly submodule setting "Providers enabled by default" set to Enabled. If I set it to Disabled, the regexes in the 'Additional Schemes' box are not applied.

It seems like they should be applied regardless of that setting. Or, there should be a checkbox to apply the additional schemes.

pierrozone’s picture

pierrozone CreditAttribution: pierrozone commented

This patch: https://www.drupal.org/node/2578937#comment-10402353 enables functionality to blacklist domains.

dankung1’s picture

dankung1 CreditAttribution: dankung1 as a volunteer commented

Hi, I found the #3 patch is for old edition, not for the '7.x-1.0-rc2' . So not use it with '7.x-1.0-rc2' edition.
and I also tried #27, not work too . Probably embed.ly has changed the api (It's now November 24, 2015).

I contacted embed.ly about 'Embed.ly', who suggest the 'EMBED CARD' at http://embed.ly/docs/products/cards

I defined a custom field format for 'Link' field. It seems now it works for embed ANY URL.
My formatter:
[node:field-link:title]

Related modules

https://www.drupal.org/project/custom_formatters

https://www.drupal.org/project/link

Before that, you have to install the Install Platform.js at http://embed.ly/docs/products/cards.

I used this method to install Platform.js

=====
Using the theme's .info file (Drupal 6 & 7)

You can add your script path in the .info file of your theme and Drupal will include them automatically.

name = My theme
description = Theme developed by me.
core = 7.x
engine = phptemplate

scripts[] = Platform.js
=========

Gentle remind, within your Platform.js, not include tags. Just include those codes between and

Hope that helps.

November 24, 2015

TheodorosPloumis’s picture

TheodorosPloumis
he/him
Location Greece
CreditAttribution: TheodorosPloumis commented

Patch from #27 works normally for 7.x-1.x-dev version.

jemisond’s picture

jemisond CreditAttribution: jemisond commented

I'm confused... I have 7.x-1.x-dev installed, but the #3 patch fails (no oembedcore.admin.inc or oembedcore.module files).