Comments

mikeytown2’s picture

Status:Active» Needs review
StatusFileSize
new802 bytes

Do something like this. We wait 1 second for any stream to be available; if none are available for I/O after 1 second, we then go into polling every 25ms 100 times. If nothing has happened by this point and no other streams are running we exit the loop.

hass’s picture

mikeytown2’s picture

It is related to #1437658-4: options timeout set to (Float) 28.88858. The connection isn't happening but stream_select doesn't know that per say. Luckily stream_socket_get_name() seems that it can be used to detect when the server refuses the connection or drops it (I discovered this via trial and error).

Can you test this patch below? Should make HTTPRL faster for this use case.

mikeytown2’s picture

Status:Needs review» Fixed

Above patch has been committed.

hass’s picture

Status:Fixed» Needs work

I'm not a fan of your custom status codes at all and this is not like core. Can you use common codes and error messages that are Googleable too, please? Normally this is one of the below. Negative so it does not clash with the other between 100-5xx.

See http://technet.microsoft.com/en-us/library/cc723430.aspx for some of these well known, please.

10060 Connection timed out
10061 Connection refused by destination
10065 Host unreachable
11001 Host not found
mikeytown2’s picture

mikeytown2’s picture

Status:Needs work» Needs review
StatusFileSize
new9.33 KB

More descriptive listing http://msdn.microsoft.com/en-us/library/aa924071.aspx

This is the patch, let me know what you think.

mikeytown2’s picture

StatusFileSize
new12.24 KB

More work on error codes.

hass’s picture

What will happen if the server returns any other winsock error? Let's say 10065? Adding more and more constants seems to be the wrong way to me.

mikeytown2’s picture

Odds are that will be caught by HTTPRL_CONNECTION_REFUSED -10061. There isn't a way to get all winsock errors when using the ASYNC flag; best I can do is infer what the problem might be. I've tried most of the http://php.net/stream functions seeing if I can get the error state from them and I haven't found a function that will do this for me. The fact that http://php.net/stream-socket-get-name can be used to detect errors like this is a nice side effect.

hass’s picture

While looking on older comments... It's so inconsistent what we are doing... See http://drupal.org/node/1428474#comment-5590436 where i've got a winsock 10035. Just logic wise this implementation is really no fun... One status code here, others there and so on... really hard to develop and test :-((( it's not possible to grab it always the same way? Good night.

mikeytown2’s picture

10035 is what was returned from stream_socket_client(). It is a https connection thus the ASYNC flag may not be used as I will try again without ASYNC if this is a https connection. Your complaining about something I don't have any control over in short. If stream_socket_client() gives me an error I pass that along. If it doesn't I keep going, relying on stream_socket_get_name and timeouts in short to see if the stream is in an error state. For your case, you can use drupal_http_request to get the error; or we could have an option to set a sync flag if you want, just be advised that HTTPRL's speed will be reduced significantly on the connection side, in this function httprl_request(). The speed of httprl_send_request() will not be affected by the ASYNC flag.

mikeytown2’s picture

StatusFileSize
new18.62 KB

Patch adds in the error codes, re-worked the $flags variable, added in the async_connect option, optimized the foreach ($streams as $id => $fp) loop.

Testing the new flag.

  $urls = array(
    'http://www.netzgegenrechts.de',
    'http://www.nrwgegendiskriminierung.de/',
    'http://www.statewatch.org',
    'http://europa.eu.int/comm/dgs/justice_home/index_de.htm',
    'http://europa.eu.int/comm/justice_home/unit/charte/index_en.html',
    'http://book.coe.fr/conv/de/ui/frm/f5-d.htm',
    'http://www.curia.eu.int/de/',
    'http://www4.karlsruhe.de/kultur/ausstellungen',
    'http://www.museum.gov.my/english/home.htm',
  );

Time
Async: 2.89282
Sync: 3.85309

*   - async_connect: default is TRUE. FALSE may give more info on errors but is
*     generally slower.
mikeytown2’s picture

StatusFileSize
new18.62 KB

fixed a small typo in the documentation.

mikeytown2’s picture

#14 has been committed.

mikeytown2’s picture

hass’s picture

How can I have link failures like this if it works with wget properly? I also see very many 10060.

http://www.profamilia.de/, HEAD, -10061, Connection refused by destination. Write.
mikeytown2’s picture

That url works correctly on my test server.

(
    [http://www.profamilia.de/] => stdClass Object
        (
            [url] => http://www.profamilia.de/
            [request] => HEAD / HTTP/1.0
User-Agent: User-Agent: Drupal (+http://drupal.org/)
Host: www.profamilia.de
Connection: close

            [status] => Done.
            [data] =>
            [options] => Array
                (
                    [blocking] => 1
                    [headers] => Array
                        (
                            [User-Agent] => User-Agent: Drupal (+http://drupal.org/)
                            [Host] => www.profamilia.de
                            [Connection] => close
                        )

                    [method] => HEAD
                    [max_redirects] => 1
                    [timeout] => 14.44986
                    [async_connect] =>
                    [data] =>
                    [context] =>
                    [version] => 1.0
                    [referrer] =>
                    [domain_connections] => 8
                    [global_connections] => 128
                    [global_timeout] => 120
                    [chunk_size_read] => 32768
                    [chunk_size_write] => 1024
                )

            [chunk_size] => 32768
            [protocol] => HTTP/1.1
            [status_message] => OK
            [headers] => Array
                (
                    [server] => nginx/1.0.4
                    [date] => Fri, 24 Feb 2012 22:13:55 GMT
                    [content-type] => text/html; charset=iso-8859-1
                    [connection] => close
                    [x-powered-by] => PHP/5.3.2-1ubuntu4.14
                    [expires] => Thu, 19 Nov 1981 08:52:00 GMT
                    [cache-control] => no-store, no-cache, must-revalidate, post-check=0, pre-check=0
                    [pragma] => no-cache
                    [set-cookie] => PHPSESSID=19ob0epf51mn37chpe1b2r9492; path=/,fe_typo_user=b109ba9bfe5b43b621dd75d35e2c50ce; path=/; domain=.profamilia.de
                    [vary] => Accept-Encoding
                )

            [cookies] => Array
                (
                    [0] => Array
                        (
                            [name] => PHPSESSID
                            [value] => 19ob0epf51mn37chpe1b2r9492
                            [path] => /
                        )

                    [1] => Array
                        (
                            [name] => fe_typo_user
                            [value] => b109ba9bfe5b43b621dd75d35e2c50ce
                            [path] => /
                            [domain] => .profamilia.de
                        )

                )

            [code] => 200
        )

)

mikeytown2’s picture

StatusFileSize
new982 bytes

fix for async connections. This has been committed.

Status:Fixed» Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.