Http Request and Response

May 23, 2018
HTTP Chrome DevTools

HTTP overview

Hypertext Transfer Protocol (HTTP) is an application-layer protocol for transmitting hypermedia documents, such as HTML. It was designed for communication between web browsers and web servers, but it can also be used for other purposes. HTTP follows a classical client-server model, with a client opening a connection to make a request, then waiting until it receives a response.

Clients and servers communicate by exchanging individual messages (as opposed to a stream of data). The messages sent by the client, usually a Web browser, are called requests and the messages sent by the server as an answer are called responses. Requests and responses are also called HTTP messages. HTTP makes use of the Uniform Resource Identifier (URI) to identify a given resource and to establish a connection. Once the connection is established, HTTP messages are passed in a format/structure for transferring the required data.

HTTP request

An HTTP client sends an HTTP request to a server in the form of a request message which includes following format:

  1. A Request-line

  2. Zero or more header (General|Request|Entity) fields

  3. An empty line indicating the end of the header fields

  4. Optionally a message-body

Now let’s make a few requests using curl in terminal as examples.

Send requests with curl

Curl is a linux utility that is used to make HTTP requests to a given url. It outputs HTTP response to standard output and is actually very easy to use.

  1. GET request example:

    This line below sends an HTTP GET request to the url “https://www.google.com". -H allows adding extra headers to use when getting a web page. For detailed explanations of other parameters in this command please check out explainshell.

    $ ~ curl -s -v -H "Jade: halo world" -- "https://www.google.com"
    

    From all the things we received, these lines are the actual request content we made from curl (note that all the ‘>’ symbols are only for readability use):

    > GET / HTTP/2
    > Host: www.google.com
    > User-Agent: curl/7.54.0
    > Accept: */*
    > Jade: halo world
    >
    
  2. POST request example:

    Here we are making a POST request and the -d parameter specifies the data we want to post to the server. In this case, we are sending a string ‘meow’ with length of 4.

    $ curl -X POST -d "meow" -s -v -H "Jade: halo world" -- "https://www.google.com" 
    

    Request content:

    > POST / HTTP/2
    > Host: www.google.com
    > User-Agent: curl/7.54.0
    > Accept: */*
    > Jade: halo world
    > Content-Length: 4
    > Content-Type: application/x-www-form-urlencoded
    >
    

HTTP request structure

There are maximum four parts and minimum three parts in an HTTP request message, each of the entities can be used in this structure:

1 Request-Method SPACE Request-URI SPACE HTTP-Version CRLF
2 Key1: value1
2 Key2: value2
2 Key3: value3
2 ...
3 
4 Optional data to send

HTTP response

After receiving and interpreting a request message, a server responds with an HTTP response message:

  1. A Status-line

  2. Zero or more header (General|Response|Entity)

  3. An empty line indicating the end of the header fields

  4. Optionally a message-body

Receive responses with curl

  1. Response for the GET request example:

    The ‘<’ symbols are also for readability uses.

    < HTTP/2 200
    < date: Wed, 23 May 2018 16:02:40 GMT
    < expires: -1
    < cache-control: private, max-age=0
    < content-type: text/html; charset=ISO-8859-1
    < p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
    < server: gws
    < x-xss-protection: 1; mode=block
    < x-frame-options: SAMEORIGIN
    < set-cookie: 1P_JAR=2018-05-23-16; expires=Fri, 22-Jun-2018 16:02:40 GMT; path=/; domain=.google.com
    < set-cookie: NID=130=WiAnlBdoQAOkGLE5cKKLLmb3bqK3y9mcKglhtQX8B6wPIiK9bIqGu2tWsyBvJySvqhHB40dRJqthccXAo6NUlGZ2FB3vXQF9iyz0gvhKA9tt8Yg-VN5GZ7eZR6JtB07i; expires=Thu, 22-Nov-2018 16:02:40 GMT; path=/; domain=.google.com; HttpOnly
    < alt-svc: hq=":443"; ma=2592000; quic=51303433; quic=51303432; quic=51303431; quic=51303339; quic=51303335,quic=":443"; ma=2592000; v="43,42,41,39,35"
    < accept-ranges: none
    < vary: Accept-Encoding
    <
    <!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="en"><head><meta content="Search the world's information, including webpages, images, videos and more. Google has many special features to help you find exactly what you're looking for." ... 
    
  2. Response for the POST request example:

    < HTTP/2 405
    < allow: GET, HEAD
    < date: Wed, 23 May 2018 22:50:05 GMT
    < content-type: text/html; charset=UTF-8
    < server: gws
    < content-length: 1589
    < x-xss-protection: 1; mode=block
    < x-frame-options: SAMEORIGIN
    < alt-svc: hq=":443"; ma=2592000; quic=51303433; quic=51303432; quic=51303431; quic=51303339; quic=51303335,quic=":443"; ma=2592000; v="43,42,41,39,35"
    <
    <!DOCTYPE html>
    <html lang=en>
     <meta charset=utf-8>
     <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
     <title>Error 405 (Method Not Allowed)!!1</title> ...       
    

HTTP response structure

The response structure is similar to the request one, it also consists of maximum four parts:

1 HTTP-Version SPACE Status-Code SPACE Reason-Phrase CRLF
2 Key1: value1
2 Key2: value2
2 Key3: value3
2 ...
3 
4 Optional data to fetch

Inspect HTTP requests and responses with Chrome DevTools

We can also send an HTTP request to a web server somewhere using Chrome Developer Tools.

  1. Type in a URL in Chrome’s address bar (I’ll use https://github.com as an example).
  2. Press Command+Option+I (Mac) or Control+Shift+I (Windows, Linux) to open DevTools on the page.
  3. In DevTools, click the Network tab.
  4. Press Command+R (Mac) or Control+R (Windows, Linux) to reload the page. Note that the new stuff happened on this panel is a set of requests that were generated in order to render what you’ve seen on the screen. As your application runs and additional requests are sent to the server, those requests are going to show up here as well.
  5. Click on the first document listed on the panel (github.com in this case), then select the Headers tab.
  6. Expand the Request Headers list, and click view source (it will turn into ‘view parsed’ after clicking). The first three parts of the request message can be viewed in the Request Headers list. The fourth part (that is the optional message body after the CRLF) will be shown in FormData or Payload when sending POST or PUT request, the detailed difference can be found here.

    request with chrome

  7. Expand the Response Headers list, and click view source. In the list, we have the 1st and 2nd part of the response message.

  8. The 4th part of the response message can be viewed in Preview or Response tab.

    response with chrome

comments powered by Disqus