Php: cannot make multiple fread () calls on php: // input

I am sending data back and forth to a PHP application using content-encoding: chunked via POST. I need my PHP application to read some data, work on it, send a response, read some more data, etc. I cannot read all the data right away, as it will not be available. Imagine a large file download when a checksum is sent as a response at regular intervals.

The problem is that although I can read a few bytes from php://input , subsequent calls to fread do not return new content.

I am currently using the Docker container for PHP . I tried both php:7.0-apache and php:5-apache with the same result.

The PoC client below generates random strings and sends them in fragments to the server with an interval of 3 seconds. The server reads with php://input with an interval of 1 second and prints the contents. At the server output, only the first three lines are displayed; the server also “blocks” until the first three are read.

Things I tried to no avail:

  • Using fseek
  • Using stream_select doesn't seem to work with php://input stream. I have no idea why, since that would be ideal for me, but given how poorly designed and implemented PHP is, I'm not surprised.
  • Closing and reopening php://input
  • Using fgetc

Client Output:

  $ python poc.py Sending: --- POST /poc.php HTTP/1.1 Host: localhost accept-encoding: *;q=0 Transfer-Encoding: chunked Content-Type: application/octet-stream --- After sending headers, response: HTTP/1.1 200 OK Date: Mon, 29 May 2017 14:25:52 GMT Server: Apache/2.4.10 (Debian) X-Powered-By: PHP/5.6.30 transfer-encoding: chunked Content-Type: application/octet-stream 4 OK Waiting 3 seconds Sending string: AuVuvsyGJc Waiting 3 seconds Sending string: LfKouYzccV Waiting 3 seconds Sending string: WmpPspYqiR Waiting 3 seconds Sending string: IApMOjoaIv Waiting 3 seconds Sending string: tuGrVklcVy Waiting 3 seconds Sending string: btUVIezCow Waiting 3 seconds Sending string: XUPOrEidyd Traceback (most recent call last): File "poc.py", line 33, in <module> websock.send(to_chunk(rnd)) socket.error: [Errno 32] Broken pipe 

Server output:

 Connected Read: AuVuvsyGJc LfKouYzccV WmpPspYqiR Read: Read: Read: Read: 172.17.0.1 - - [29/May/2017:14:25:52 +0000] "POST /poc.php HTTP/1.1" 200 191 "-" "-" 

PHP server:

 <?php header("transfer-encoding: chunked"); header("content-type: application/octet-stream"); flush(); /** * Useful to print debug messages in the Apache logs */ function _log($what) { file_put_contents("php://stderr", print_r($what, true) . "\n"); } _log("Connected"); /** * To send data as chunks */ function _ch($chunk) { echo sprintf("%x\r\n", strlen($chunk)); echo $chunk; echo "\r\n"; flush(); } // Test chunks _ch("OK\r\n"); $web_php_input = fopen("php://input", 'r'); $continue = 5; while ($continue-- > 0) { $contents = fread($web_php_input, 1024); _log("Read: " . $contents); sleep(1); } fclose($web_php_input); ?> 

Python client:

 from __future__ import print_function import random import socket import string import time def to_chunk(what): return format(len(what), 'X') + "\r\n" + what + "\r\n" websock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) websock.connect(("localhost", 8080)) # Send the initial chunked POST header connect_string = ''.join(( "POST /poc.php HTTP/1.1\r\n", "Host: localhost\r\n", "accept-encoding: *;q=0\r\n", # ,gzip;q=0,deflate;q=0\r\n", "Transfer-Encoding: chunked\r\n", "Content-Type: application/octet-stream\r\n", # "Connection: keep-alive\r\n", "\r\n", )) print("Sending:\n---\n{}\n---\n".format(connect_string)) websock.sendall(connect_string) print("After sending headers, response:\n {}".format(websock.recv(1024))) c = True while c: print("Waiting 3 seconds") time.sleep(3) rnd = ''.join(random.choice(string.ascii_letters) for _ in range(10)) rnd += '\r\n' print("Sending string: {}".format(rnd)) websock.send(to_chunk(rnd)) print("done") 

Dockerfile:

 FROM php:5-apache COPY custom.ini /usr/local/etc/php/conf.d 

Docker command line:

 docker build -t listener . docker run -i --rm -p 8080:80 -v $(pwd):/var/www/html --name listener listener 

custom.ini so that PHP knows that the POST body should not be buffered:

 enable_post_data_reading=false 

Before someone suggests using another language or framework or something else: it must be PHP; It cannot rely on a third-party library or PECL; and that’s exactly what I need.

As a side note, this behavior is compatible with the HTTP specification ; the server does not have to read all incoming data before returning part of the response to the client. See Also RFC6202 .

+5
source share
1 answer

To understand why this happens, you need to know how HTTP works, which, unfortunately, is not the way you think. Transcoded transfer coding and PHP also do not work the way you think. I will try to explain in a way that is relevant to what I think you are trying to do.

If I understand correctly, you are trying to send request and response fragments in interlaced mode or send data back and forth as they are described. This is a violation of the HTTP specification. Therefore, you cannot do this because requests are processed directly by the HTTP server, not PHP.

HTTP

HTTP is a request / response protocol (RFC2616, section 1.4), which has a simple operation:

  • The client sends an HTTP request message to the server.
  • After receiving and interpreting the request message, the server responds with an HTTP message. (RFC2616, clause 6).

Note that step 2 says “After,” not “Bye,” which means the server must wait for the request to complete before it can send a response. This is why the "server seems to be blocking."

The HTTP Longing and Streaming HTTP life cycles described in RFC6202 actually work the same without violating the HTTP specification. They do not send data back and forth (without interleaving).

Striped Copy

If the request has the Transfer-Encoding: chunked header, the server should wait for the last fragment. This is described in at least two places:

  • In BNF Section 3.6.1 . Note that the Chunked-Body must have last-chunk .
  • In pseudo-code Section 19.4.6 . Please note that “there is no response to the client” or something similar inside the loop (in the whole pseudo-code, really).

In short, movement is not allowed. Chunked transfer-encoding does not introduce interleaving and therefore does not change the way HTTP works.

Php

Since the server has to wait for the request, PHP will not be called until the request completes. Therefore, when you send chunks of data with 3 second delays, your PHP script is not running yet.

As for the PHP enable_post_data_rendering configuration enable_post_data_rendering , it does not exist. The closest to it is enable_post_data_reading , which simply means that the request body will not be parsed, and therefore $ _FILES and $ _POST will be empty. This is because of efficiency: there is no time to parse the request body and the lack of memory used to store the values ​​of $ _FILES and $ _POST. This has nothing to do with buffering the POST body.

Let me know if there is anything you are still unclear about.

Update

This is the result of my own experiment with 3 second intervals between events and a 15 second socket timeout. Timestamps are useful for determining which events are related.

Please note that reading from the server is always timed out before sending the last fragment. Also observe the time 13:43:03 when sending the last fragment, which also appears when PHP is called. It shows that the server waited for the last fragment before calling PHP.

  client 13:40:54 opening socket ... opened
 client 13:40:57 sending request ... 130 bytes sent
 client 13:41:00 reading from server ...
 client 13:41:15 timed out
 client 13:41:18 sending chunk 0 ... 14 bytes sent
 client 13:41:21 reading from server ...
 client 13:41:36 timed out
 client 13:41:39 sending chunk 1 ... 14 bytes sent
 client 13:41:42 reading from server ...
 client 13:41:57 timed out
 client 13:42:00 sending chunk 2 ... 14 bytes sent
 client 13:42:03 reading from server ...
 client 13:42:18 timed out
 client 13:42:21 sending chunk 3 ... 14 bytes sent
 client 13:42:24 reading from server ...
 client 13:42:39 timed out
 client 13:42:42 sending chunk 4 ... 14 bytes sent
 client 13:42:45 reading from server ...
 client 13:43:00 timed out
 client 13:43:03 sending last chunk ... 5 bytes sent
 client 13:43:06 reading from server ...
 client 13:43:06 279 bytes read
 client 13:43:06 ---------- start of response
 HTTP / 1.1 200 OK
 Host: localhost
 Connection: close
 X-Powered-By: PHP / 7.0.12
 Transfer-Encoding: chunked
 Content-Type: application / octet-stream

 20
 server 2017-06-16 13:43:03 start
 2d
 13:41:18
 13:41:39
 13:42:00
 13:42:21
 13:42:42

 1e
 server 2017-06-16 13:43:03 end
 0

 client 13:43:06 ---------- end of response
 client 13:43:06 done

This is server.php :

 <?php while(@ob_end_flush()); header("Transfer-Encoding: chunked"); header("Content-Type: application/octet-stream"); echo chunk("server ".gmdate("Ymd H:i:s ")."start"); if($f = fopen("php://input", "r")){ while($s = fread($f, 1024)){ echo chunk($s); } fclose($f); } echo chunk("server ".gmdate("Ymd H:i:s ")."end"); echo chunk(""); function chunk($s){ return dechex(strlen($s))."\r\n".$s."\r\n"; } 

This is client.php :

 <?php out("opening socket... "); if($socket = fsockopen("localhost", 80, $errno, $error)){ echo "opened\n"; //set socket timeout to 15 seconds stream_set_timeout($socket, 15); sleep(3); out("sending request... "); $n = fwrite($socket, "POST http://localhost/server.php HTTP/1.1\r\n" ."Host: localhost\r\n" ."Transfer-Encoding: chunked\r\n" ."Content-Type: application/octet-stream\r\n" ."\r\n" ); echo "$n bytes sent\n"; sleep(3); readFromServer($socket); sleep(3); for($i=0; $i<5; $i++){ out("sending chunk {$i}... "); $n = fwrite($socket, chunk(gmdate("H:i:s\n"))); echo "$n bytes sent\n"; sleep(3); readFromServer($socket); sleep(3); } out("sending last chunk... "); $n = fwrite($socket, chunk("")); echo "$n bytes sent\n"; sleep(3); readFromServer($socket); fclose($socket); }else{ echo "error\n"; } out("done\n"); function out($s){ echo "client ".gmdate("H:i:s ").$s; } function chunk($s){ return dechex(strlen($s))."\r\n".$s."\r\n"; } function readFromServer($socket){ out("reading from server... \n"); $response = fread($socket, 1024); $info = stream_get_meta_data($socket); if($info['timed_out']){ out("timed out\n"); }else{ out(strlen($response)." bytes read\n"); if($response){ out("---------- start of response\n"); echo $response; out("---------- end of response\n"); } } } 
+5
source

Source: https://habr.com/ru/post/1268329/


All Articles