iOS 4 and Android 2.3 and below (webkit based browsers) return "Unsupported Media Type" on XML in stock 8.x

That's highly weird. It sounds like the Simulator is sending the wrong Accept header. Can you drop some debug code into the negotiation library to see what it's actually sending?

We want to change that mechanism anyway once we have the ability to, but it's good to know what broken systems we have to deal with. :-/

I was using the iOS Simulator throughout Drupalcon (as part of working on the mobile initiative for 8.x), re-installing regularly, and didn't notice this.

I wonder if it's particular to the version? I'm using version 5.1 on Mountain Lion and $this->negotiation->getContentType($request); returns 'html'.

I get the same error on iOS Simulator Snow Leopard.

This doesn't happen with a real iPhone 4s

I have access to a device running ios3.1 and I get the same error on that.

I'll download the iOS 4 and 3 simulators and test on those.

Ah! On Mountain Lion, doesn't appear to be an (easy) way to get hold of the older simulators.

Managed to test on an actual iPod Touch running 4.2.1 and can confirm the problem exists there also. Have changed the issue title as it's an iOS issue in general, not just a simulator one.

So let me understand this...

Older versions of iOS asked for an HTML page using a mime type of 'text/xml', 'application/xml', or 'application/x-xml' (the three mime types that would map to "xml" right now) when what they want to get back is text/html? That cannot possibly be right, or it would have broken a few zillion other systems by now.

What's the actual mime type that's being sent/received? Can you throw some debug lines into the code to get the raw Accept header?

iOS 4.2.1 request headers:

Host: 10.0.1.14:81
User-Agent: Mozilla/5.0 (iPod; U; CPU iPhone OS 4_2_1 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8C148 Safari/6533.18.5
Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Cache-Control: max-age=0
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: keep-alive

iOS 5.1 request headers:

Host: 127.0.0.1:81
User-Agent: Mozilla/5.0 (iPhone Simulator; CPU iPhone OS 5_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9B176 Safari/7534.48.3
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us
Accept-Encoding: gzip, deflate
If-Modified-Since: Fri, 31 Aug 2012 19:42:15 GMT
If-None-Match: "1346442135"
Cookie: Drupal.toolbar.collapsed=0
Connection: keep-alive

The main difference in the accept header is the ordering: in 4.2.1 application/xml and application/xhtml+xml come before text/html.

Looking into this further, iOS 4.2.1 returns the following for $request->getAcceptableContentTypes() in the getContentType method of the ContentNegotiation class:

    [0] => application/xml
    [1] => application/xhtml+xml
    [2] => image/png
    [3] => text/html
    [4] => text/plain
    [5] => */*

and $format = $request->getFormat($mime_type); is not null, so xml gets returned.

Oh Apple, you so silly... And by silly, I mean *expletive deleted*

I believe the current logic will match the first listed type. Why Safari Mobile is asking for raw undefined XML data as a higher preference to HTML I have no bloody idea, but we may need to hard code text/html as the first check for now. We still need to clean up that negotiation library, but... *sigh*

I just logged the same issue on Android devices at version 2.3.3

#1802882: Pages requested on older Android devices (verified on 2.3.3) return a page with the error "Unsupported Media Type"

I'll close as a duplicate. I'm trying to get header info out of Charles now.

tag

This is a webkit problem, caused by a 'strange' design decision, for some reason they choose to use the accept headers from FF 2.0, it is fixed in later version (FF fixed it in FF 3.0)

• mobilizer has the same problem, AFAIK they use webkit internally.
• I checked on my mobile (Android JB) using FF and Chrome and they work.

Related issues:

• https://bugs.webkit.org/show_bug.cgi?id=27267
• http://trac.webkit.org/changeset/80776
• http://trac.webkit.org/changeset/35167

Bumping this to major, but no idea if this is 'our' problem.

It looks like we'll have to figure out some way to work around this webkit bug, even if it's not our problem per se. :-( Probably as part of whatever cleanup we do to the content negotiation system.

Which, I think Fabpot just said he's going to look into, so I'll make sure he's aware of this issue. :-)

I found a closed issue for symfony at https://github.com/symfony/symfony/pull/564

I've experienced this bug on each and every Android mobile device I got into my hands in the past six months or so. (testing the web site of my local DUG)

D7 sites are working fine, so this presents a really critical regression.

It's a little strange that the same bug does not occur on symfony.com. I therefore wonder whether this really is a upstream bug in Symfony, or whether it isn't caused by our own code.

In any case, we cannot release with this.

This isn't a symfony problem, but it would be nice if they got a solution.

Problem is caused by this code , it returns as soon as it finds a valid format:

     foreach ($request->getAcceptableContentTypes() as $mime_type) {
       $format = $request->getFormat($mime_type);
       if (!is_null($format)) {
         return $format;
       }
     }

Very ugly working fix, see http://picturefill8.h011.attiks.com:

       if (FALSE && !is_null($format)) {

If I were fixing this bug I would hardwire "^application/xml,application/xhtml+xml,text/html" and parse it as HTML.

I think it's safer to check only for text/html since all browsers will be sending this. If you add others you will complicate things for services.

something like this

+    if (strpos($request->headers->get('Accept'), 'text/html') !== FALSE) {
+      return 'html';
+    }

#20: Needs work as my Android sends:

application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
text/xml, text/html, application/xhtml+xml, image/png, text/plain, */*;q=0.8

So it tries hard :-D.

#21:

Works, BUT:

+++ b/core/lib/Drupal/Core/ContentNegotiation.phpundefined
@@ -41,6 +41,11 @@ public function getContentType(Request $request) {
+    if (strpos($request->headers->get('Accept'), 'text/html') !== FALSE) {

I am not sure this will work in all circumstances, because it completely ignores the order.

#22 patch in #21 ignores the order, AFAIK this is the only way to make sure we always find the text/html part. I'm assuming that non-browser clients aren't going to send text/html, but not tested.

I collected the browser Accept headers mentioned in this and all referenced issues, and turned them into a regression test.

Chrome 2009 fails, too.

Some examples of browsers sending the wrong headers (list of UA):

• Nokia2720a-2b/2.0 (08.64) Profile/MIDP-2.1 Configuration/CLDC-1.1
• Mozilla/5.0 (Linux; U; Android 1.6; en-us; T-Mobile G1 Build/DMD64) AppleWebKit/528.5+ (KHTML, like Gecko) Version/3.1.2 Mobile Safari/525.20.1
• Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; it-it) AppleWebKit/533.17.8 (KHTML, like Gecko) Version/5.0.1 Safari/533.17.8
• Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.472.59 Safari/534.3
• Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8B117 Safari/6531.22.7
• Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-us) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5
• Mozilla/5.0 (iPad; U; CPU OS 3_2 like Mac OS X; en-us) AppleWebKit/531.21.10 (KHTML, like Gecko) Version/4.0.4 Mobile/7B367 Safari/531.21.10
• Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.472.59 Safari/534.3
• Mozilla/5.0 (Linux; U; Android 2.1-update1; en-in; HTC_Wildfire_A3333 Build/ERE27) AppleWebKit/530.17 (KHTML, like Gecko) Version/4.0 Mobile Safari/530.17

Source: http://whatheaders.com/data/

PS: In the same data files are even request without any accept header :/

Didn't see #22 yet. Incorporated.

That said, #22 is not entirely clear - those are two separate Accept headers from separate requests, right?

@sun: My Android (Nexus One, Froyo 2.2) sends first the first header (application), then if that fails it tries again with the second header (text). Only the first header needs to be supported IMHO.

If the header is accepted, then the second header (text) is never send.

I analyzed the data a bit further:

In total the xml file contains 3508 request, from those 354 (10%) are not starting with */*, text/* or text/html
Those 354 request (wrong_headers.csv) contain 33 unique accept headers (unique_accept.txt)

Combined patch from #21 and #26

Please refactor this so that we use the parsed $request->getAcceptableContentTypes() instead of this ugly (and incorrect) string parsing.

#31 you mean something like this:

    $acceptable_content_types = $request->getAcceptableContentTypes();
    // Return HTML without looking at the weight.
    if (in_array('text/html', $acceptable_content_types)) {
      return 'html';
    }

Care to clarify the " incorrect"

I agree with #31.

Pseudo-Code (not working):

$content_types = $request->getAcceptableContentTypes();
array_map($content_types, $request->getFormat);
if (in_array('html', $content_types)) {
  return 'html';
}
...

In general I agree with the approach of "if there's text/html in there anywhere, it wins". Browsers tend to be crap at Accept headers, but anything else should be fairly sane with what it sends so we can trust them to not be stupid.

Since there's no way to map them all at once, we have to loop through all mime types.

If HTML is found, it is returned, otherwise the first valid found is returned.

Patch #36 works on Sony Ericsson android device and ipad simulator 4.3
Before the patch is applied, get an error "Unsupported Media Type". Able to access the site after applying the patch.

We need to decide first:

1. Use the fast detection from #33 or #31 and only look for text/html, not text/* and/or */*
2. Use the more correct but slow version of #36

Keep in mind that this code gets executed for every request.

I wouldn't worry about micro-optimising this at this point. For now let's just make it work, since this code path may end up rarely used if we get proper conneg in place and SCOTCH has its way (which I hope it does). Focus on most reliable/robust for now.

Bah, stupid issue forms...

so that means #36 is up for review

+1 for #36
Patch #36 works on Sony Ericsson android device and ipad simulator 4.3
Before the patch is applied, get an error "Unsupported Media Type". Able to access the site after applying the patch.

Performance is fine:

=== text/html..Android#1 compared (508d67f44548d..508d697f390b6):

fc  : 17,532|17,548|16|0.1%
wt  : 74,344|74,466|122|0.2%
cpu : 72,005|76,005|4,000|5.6%
mu  : 8,705,120|8,705,032|-88|-0.0%
pmu : 8,754,240|8,754,056|-184|-0.0%

---
fc = function calls, wt = wall time, cpu = cpu time used, mu = memory usage, pmu = peak memory usage

This compares core requested via "Accept: text/html,application/xml,application/xhtml+xml;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5" vs. #36 with "Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5".

Such 2 additional calls to getFormat, which result in 12 calls to strpos. One call to getFormat takes around 22 microsecs.

=> to scale this to a worst case: 22 microsecs * number of formats in accept header. Lets assume 20 => 400 microsecs => 0.4 ms.

=> Micro Optimization as Crell said. The rest is measuring delta.

=> RTBC!