Request and response glossary

Introduction

This page lists the various terms that relate to request and response. Only necessary terms are covered that show up when handling of REST calls.

Remote procedure call, or RPC

See article about RPC at W3 and Wikipedia. RPC is a technique for building distributed systems by allowing a program on one machine (client) to call a subroutine on another machine (server) without knowing that the invoked subroutine is occurring on a different machine. Thus, it provides concepts for achieving a request–response communication. On the client side, it is coded as if it the call being made is a normal (local) procedure call and without the programmer explicitly coding the details for the remote interaction, or hardware and OS details of remote server. That is, the programmer writes essentially the same code whether the subroutine is local to the executing program, or remote. Note that RPC is not a transport protocol.

RPC can be achieved by locally (local, meaning at the client application side) combining all information needed by the server some a message format and sending it to remote server. The remote server unpacks the message to identify the code to run and parameters to the code. It executes the method and then returns the result back to the client application (Reference: Wikipedia). The process of combining necessary information in the message, done by client application when it sends the initial request, or by the server when it sends the response, is called "marshalling". The reverse process of retrieving the information contained in the message, done by the server when it receives the request from client application, and by the client application when it receives the response from the server, is called "unmarshalling" where the parameters are recreated from the message. The common rule agreed upon by both the client and the server, determining how to pack or unpack necessary information in a message is called the "protocol" of the communication. Marshalling of information is needed to convert data that exists in a client application computer's memory into a binary form that can then be sent to another computer, i.e., the server, over the network. For example, consider a date object formed by code new Date("2019-12-20"). This is an "object" which exists in computer's memory and has access to various object methods, like, isBefore(...), isAfter(...), etc. One way to marshall this date object is to convert the date to string 2019-12-20, which UTF-8 encoded to bytes 50, 48, 49, 57, 45, 49, 50, 45, 50, 48, that can be transferred over network as 00110010 for 50, 00110000 for 48, etc. The server converts the sequence of 0s and 1s to bytes, which are then UTF-8 encoded to form the string that can be used to create a similar date "object" at server side.

Remote method invocation, or RMI

See article about RMI at Wikipedia, Oracle. With the development of object oriented programming, an extension of RPC was developed wherein the complete objects are passed between client and server. This bypasses the overhead of marshalling/unmarshalling and makes the communication more performant. However, it also constrains the interaction to be between a client and server running same programming language because both the client and server must understand the class definition and corresponding binary data representing the object being exchanged. Should one of them choose to use a different language at any time an for any reason, the communication breaks. This restricts the popularity of RMIs.

Web services

See article about web service in Wikipedia. Stating simply, a web service is a RPC that uses Hypertext Transfer Protocol or HTTP as a protocol for communication between the client application and the server. This allows a client application to send a request to server for processing data, reusing the same infrastructure as used for internet. Since the server is using HTTP, so it must listen for incoming HTTP requests, execute methods depending on url and method of the request, and read parameters from url and body of the request. The client application also must make a properly structured HTTP request to a server to achieve the desired result.

Simple Object Access Protocol, or SOAP

See article about SOAP at Wikipedia. Consider a RPC where the protocol for arranging information in a message is separated from the protocol used for communication between two machines. The former protocol controls the arrangement of input parameters in request body before it is then sent to server, and also decides on how to extract the response data from response returned by the server, including, to identify if the processing failed with an error. SOAP is one such protocol for communication with web service. It is an official messaging protocol specification used for exchanging structured information in the implementation of web services and maintained by W3C. In SOAP, the request and response message is structured in a standard XML format, and the and the message is transmitted between machines using application layer protocols, most often HTTP. Since the communication between the client and server is done via a XML message, so, unlike RMI, it is allowed for the two to be running different programming languages, and still be able to communicate with each other.

Representational State Transfer, or REST

See article about REST at Wikipedia, Medium. To begin with, realize that REST is an architecture and not a protocol. It provides a set of guidelines for designing web APIs enabling a client to discover and execute relevant services provided by a server. However, it does not police adherence to the architecture considerations. A web service designed with REST architectural considerations are called RESTful web services. I've always seen RESTful services being mentioned in context of communication over HTTP and I am not aware if they can be used in other scenarios. Personally, I find that the best way to understand and appreciate REST designs is by understanding the Richardson maturity model. An example of migrating from zero REST to Level-3 REST and the benfit it provides is covered in this article from Martin Fowler. Some articles comparing SOAP (recall, SOAP is a protocol) vs REST can be found at Stackify and Red Hat. Currently, a large percentage of web APIs follow REST because of its intuitiveness, simplicity and ease of change. That being said, many business prefer having a list of available APIs that can be used. To meet this requirement, OpenAPI specifications and Swagger can be used (They are further discussed in a section within the housekeeping glossary).

Web servers and Application servers

See article about web servers at Mozilla. As mentioned above, a web service can be used to request data and/or trigger a processing at a server using HTTP for communication. The data being served can either be static files, like, images, html, css, etc. files. It can also be dynamic data, like, calendar page for logged in user, which depends on the user making the call. The requests made to the server can also trigger some processing, like, making a new calendar entry. Generally, when talking about a web server, it means a static web server that serves static content. Separate from it is an application server which is a combination of web server and additional business logic that allows returning dynamic content to users in response to HTTP calls. An application server can also manage database connections and also support calls other that via HTTP (i.e. RMI calls). For more difference between the web and application servers, see articles at StackOverflow and IBM.

HyperText Transfer Proptocol, or HTTP

See article about HTTP at IETF, Wikipedia, Mozilla. HTTP is an application layer protocol designed within the framework of the Internet protocol suite, i.e. as a protocol enabling a request–response communication. Its definition presumes an underlying and reliable transport layer protocol, for which the Transmission Control Protocol (TCP) is commonly used. Servers and esources that can be communicated to via HTTP are identified and located on the network by Uniform Resource Locators (URLs). HTTP allows sending/receiving headers as part of request/response containing different metadat about the request/response. HTTP in itself is stateless and there is no link between two requests being successively carried out on the same connection. However, it allows servers to set HTTP cookies via headers which can then be used to have stateful sessions. HTTPS is an extension of http wherein the communication is encrypted to secure it against various attacks. It is covered in a section under the security glossary.

Request URL

For details about URL, see IETF specification and Wikipedia. URLs are used to identify servers and resources on the web so that a request-response communication can then be initiated. As an example, consider the url https://www.example.com:8000/root/parent/1234/child/?param1=value1&param2=value2. Here, the initial https is called "url-scheme", www.example.com is the "domain" of the website, 8000 is the "port" of server that is listening for HTTP requests (it may not be necessary to provide a port if the domain is configured accordingly), /root/parent/1234/child/ is the "url path" (by convention, the last "/" in url path is optional) and param1=value1&param2=value2 are "url query parameters". Additional discussion on url is covered below.

Context root

One physical machine / computer can be made to host multiple web applications. This may be done either to achieve proper resource utilization, or, maybe because multiple applications were bundled together and all deployed at the same time. In either case, it becomes necessary to add a mechanism to identify that if a HTTP request comes to the server, then to which application amongst the many on the server should the request be sent to for further processing. This is done by adding a context root to each application which is the portion of url the comes immediately after the domain name. For most cases, it is left empty (i.e., just "/") because a single application is deployed in the server. In the url https://www.example.com/root/parent/1234/child/ABCD/, the context root would be /root, if configured so for the application. For more details, see here and here (Disclosure: I've seen context root being used only with java web applications).

Path parameter

Consider the url path https://www.example.com/root/parent/1234/child/ABCD/ for which the application server responds by returning a text "1234 ABCD". It does so by comparing the url path against a configured template, like https://www.example.com/root/parent/{parent-id}/child/{child-id}/, identifying the values "1234" for parent-id and "ABCD" for child-id, and forming a final response by coming the parent and child id(s) together. In this example, {parent-id} and {child-id} are called "path parameters" because these values are obtained from the url-path and used as parameters by the method that is executed on the application server. It is also possible to provide object-valued or multiple entries in a single path parameter by separating the different values in a path parameter using semicolon, or period, or comma (Reference: Swagger docs identifying different ways to express object or array valued path parameter).

Query parameter

Consider the url https://www.example.com/root/parent/?start=12. Let's say this url intends to look for all "parent" entries for which the "parent-id" starts with "12", like, "1234" (as used in previous example, and also..), "1244", "1289", etc. but not entries like "2378", "2147", etc. When a response to a url can return multiple resources, then query parameter(s) is used to filter down the results. The query paramters are added after url-path. Adding a "?" identifies the beginning of query-parameter and so, the url path finishes before it. Multiple query parameters can be used to filter on different criteria and are joined together by "&", like ?start=12&end=34. This should now return "1234" but not return "1244", nor "1289". It is also possible to pass multiple values for a single query parameter by repeating the parameter, like, ?start=12&start=23. Now, this should also return "2378" which was previously discarded, but still not return "2147". (Reference: Swagger)

Slug

I consider this StackOverflow answer as the best reference for about the need for a "slug". The term "slug", as used in relation to a url, comes from the world of newspaper production. It's an informal name given to a story during the production process. As the story winds its path from the reporter through to editor through to the "printing presses", this is the name it is referenced by, e.g., "Have you fixed those errors in the 'kate-and-william' story?", where, 'kate-and-william' is the slug. The slug can be used to make a more human readable url that can be used to locate the resource, like https://www.example.com/archives/kate-and-william.

Request method

See details about HTTP Request methods, also called HTTP verbs, at IETF, Mozilla and Wikipedia. The request method indicates a desired action to be performed for a given resource (resource identified by the url). The most commonly used verbs are POST, GET, PUT, PATCH, DELETE. "GET" is used to fetch a resource, "POST" to create a new resource, "PUT" is to replace the existing resource with the newly provided one, "PATCH" to update an existing resource and "DELETE" to delete a resource. A request body can be sent with POST, PUT, PATCH calls. Although not technically disallowed, sending a request body with DELETE calls is very rare (Reference: StackOverflow). For GET calls, the request body is conventionally ignored and should not be sent (Reference: StackOverflow). Instead, url, with path-parameters and query-parameters should be constructed in a manner that the HTTP GET call returns desired results.

Idempotent request method

See details about idempotent HTTP methods at IETF, Mozilla, Wikipedia. Requests methods are labeled as being idempotent if an identical request can be made once or several times in a row and it leaves the server in the same state. GET, PUT, DELETE fall in this category. In calling "DELETE" multiple times, it will return 200 response status (discussed later) for the first time and 404 for subsequent calls. It is still considered as being idempotent because the state of data on server should always remain the same. Note that "idempotence" for corresponding request method isn't a constraint enforced by HTTP, but is a conventional guidance on how the application method should behave.

Safe request method

See details about safe request method at IETF, Mozilla, Wikipedia. A HTTP method is safe if it doesn't alter the state of the server, i.e. if it is a read-only operation. These methods, by definition, are also idempotent. "GET" calls fall into the category of safe method. Note that "safety" for corresponding request method isn't a constraint enforced by HTTP, but is a conventional guidance on how the application method should behave.

Headers

See details about headers at IETF, Mozilla, Wikipedia. The request-header fields allow the client to pass additional information about the request and the client itself, to the server. Similarly, there are response-header fields that allow the server to pass additional information about the response, access to resource and server. It is highly recommended to go throught the list of standard request and response headers. Custom headers (i.e., ones specific to the web application being used, and not not standard to HTTP) can be added in request or returned in response if needed. It used to be customary to prefix the name of these header with "X-" but the recommendation has since been removed and instead, it is suggested to simply name the custom headers properly (references: StackOverflow, Mozilla). Note that choosing to still use "X-" prefix isn't something that will break HTTP communication, but is just something not recommended anymore. Some security related headers (for authentication, security response headers, cors) are discussed in sections of the security glossary.

Cookies

See details about HTTP Cookies at Mozilla, Kapersky, Wikipedia and at this article. A HTTP cookie is a small piece of data that a server sends to the user's web browser. It is a special kind of request header in that a browser stores it and sends it back to the server every time a request is made to same website, thereby enabling identification if two requests came from the same browser. In doing so, cookies enables adding statefulness by remembering stateful data in a client-server interaction over a stateless HTTP protocol. For example, a user may want their previous settings, preferences to be persisted across multiple HTTP calls so that they are asked about it just once and not every time. A cookie can also be used to check if the user has logged in (IMPORTANT: Note that a simple cookie-only based authentication is susceptible to CSRF attacks, as discussed in security glossary). The server can set new cookies or modify the value of existing cookies by sending one or multiple "Set-cookie" response headers to set/update the cookies. Various attributes and expiry period can be added to customize the behavior and scope of a cookie. As mentioned here, a cookie must be secured using HttpOnly, Samesite, Secure attributes whenever possible. With the passing of data privacy laws, like GDPR, one must be careful in storing and using cookies and not use them indiscriminately.

Request and response body

See details about request and response body at IETF. Request body can be used to transfer message from the client application to server when sending a request. Response body can be used to transfer message from the server to client application when sending a response. In either case, request or response headers like Content-Type, Content-Length, Content-Encoding, etc. are used to provide information on how to convert the bytes in request or response body into a meaningful message.

Response status

See details about response status code at IETF, Mozilla. A response status code along with a small text decription is provided with every HTTP response. They are grouped in 5 classes: informational responses (100–199), successful responses (200–299), redirects (300–399), client errors (400–499), server errors (500–599). It is up to the application to return proper response code for proper communication to user. Each response code also maps to a corresponding text that gives a brief description about the response code. The text is also sent in the response. For example: 201 Created, 400 Bad Request, etc. In this example, "Created" and "Bad Request" is the text corresponding to response code "201" and "400".

Advanced HTTP based communication

HTTP based request-response type communication is generally used in a way where a client sends a request to server, which is processed at server (generally within 0.1- 1 second) and then a response is returned. This section covers some advanced communication techniques using HTTP.

Long Polling

Polling refers to actively and synchronously sampling the status of an external device/application by a client program. In doing so, the client program wants to keeping checking the external application for any new entries.. Think of the scenario where you're driving to some destination and a fellow passenger keeps asking "are we there yet?". In this case, that passenger is the "client", who is "polling" the driver to identify a status update for the resource. In this example, the driver is the "server", and the resource is status to the question if the destination has been reached. The delay between successive calls is the polling interval. Long polling (reference: IETF, Wikipedia) is when server accepts the request and keeps it pending till a response is available, at which time it is sent. Once the client receives the response, they immediately open another long polling request. In doing so, the client is able to retrieve all message updates from the server. Consider same example as above, but after the first question, the "client" passenger and "server" driver have an understanding that the driver will respond once they have an update, and in the mean time, the passenger will eagerly wait for response from driver. Long polling techniques can be quick to add but they have their own drawbacks (reference: StackOverflow).

Webhook

A Webhook is a HTTP callback that is trigged by some application event. Let's say there are two applications, APP1 and APP2. Let's say APP2 gave some data and processing instructions to APP1, with the expectation that latter will process the data and return the result. To check that the processing is complete, one option for APP2 is to "poll" APP1 by continuously caling it till APP1 returns a successful response. The webhook option is where APP2 tells APP1 of a HTTP URL where latter should make a call after the processing is complete. In doing so, unlike the polling scenario, APP2 is free to do other tasks till it receives a callback from APP1, which itself doesn't have to spend resources in repeatedly responding to APP2 that the processing is still underway and hasn't completed. Wherever possible, having a webhook can be a much better alternative than polling based solutions.

Server Sent Events (or, SSE)

See details about SSE at Mozilla, Wikipedia and here. Some implementation examples can be seen at Medium, Spring, here and at Mozilla. Unlike the traditional web request where the client receives only a single response from server after a request in made, in SSE a client receives a "stream" of response updates from a server from a single request, and via the same HTTP connection. The Server-Sent Events EventSource API is standardized as part of HTML5 by the W3C. However, the restriction from HTTP communication still applies that before a server should have received the complete request object before it can start sending the response stream.

Websocket

See details about websockets at Wikipedia, Mozilla. WebSocket is a computer communications protocol, providing full-duplex communication channels over a single TCP connection. WebSocket is distinct from HTTP, for one, it uses "ws://" and "wss://" scheme instead of HTTP's "http://" and "https://". However, despite being different, a WebSocket "is designed to work over HTTP ports 443 and 80 as well as to support HTTP proxies and intermediaries," making it compatible with the HTTP protocol (reference: RFC 6455). To achieve compatibility, the WebSocket handshake uses the HTTP Upgrade header to change from the HTTP protocol to the WebSocket protocol. Doing so also enables using websocket in environments that block non-web Internet connections by using a firewall. The other big difference is that unlike tradition HTTP response or even SSE, the server can start sending response stream in a websocket even if the request body hasn't been fully sent by the client. This is what enables a back and forth communication between the client and server. The client can read the response from server, based on which it can form a new entry that is sent to server, and to which the server may respond by sending some other data. In a websocket, it is not necessary that client sends something to server before the server can start responding, nor is it necessary that for a single "frame" of data sent by client, the server responds by sending only one "frame" of response data. Hence, WebSocket protocol enables two-way client-server interaction with lower overhead than half-duplex alternatives such as HTTP polling and also facilitates real-time data transfer from and to the server. More example of websocket can be seen in this article at Spring, here, SessionStack, Treehouse.

Long polling vs SSE vs Websocket

Various articles can be found over the internet comparing long polling vs websocket vs SSE, for example, at Medium, Codeburst, Educative (paywall). Briefly, long polling can be good to have in initial development stages since it is nothing different from ordinary HTTP calls, except allowing for long connections. The drawback is that this adds larger server overhead and isn't very reliable. SSE should be preferred when a streaming response is needed from server after client has initiated a request, but client will no longer be sending any data from their side. For example, you ordered pizza - and want to get a very detailed second-by-second update status. Here, you as the "client" will not be sending any information after making the initial call, but the response requires streamed response from server (..just to mention, if the business case indicates that user would be ok with receiving response, say, every 3 minutes, then rather than using SSE, or even long polling, it'd be preferable to simply have the website or app make a request every 3 minutes). However, if a "chat" like feature is needed and where the data transferred can be very frequent, then a websocket is useful (think of ordering pizza, and then being able to talk with the chef, kitchen staff!! Fancy pizza!).

HTTP/1.1 vs HTTP/2 vs HTTP/3

(Last updated: Jan, 2022) HTTP/2 and HTTP/3 are both enhancements to the HTTP/1.1 protocol that is commonly used (..and is commonly meant whenever there is a discussion of webservices). HTTP/3 is still new to me, and I am unaware of the benefits that it brings and how those improvements contribute to in a web application. So, I won't go into its details. NOTE: The subsequent text in the paragraph relates to HTTP/2. On my side, I am not very sure of whether I understand things fully and properly! I also don't have a hands-on experience (yet) working with HTTP/2. Plus, there are just so many conflicting opinions on internet! So, please take the upcoming comments with a health dose of skepticism. For HTTP/2, what I understand is that it moves from a request-response model to a client-response model, adding performance optimization when multiple requests are made by a single client to the server. However, the design premise that a server can only start responding after the complete request body has been read, still holds in HTTP/2. Thus, even with the performance improvement, HTTP/2 cannot (yet) replace the use of Websockets. This article has a good summary on the improvements made in HTTP/2. Another new feature introduced in HTTP/2 is Server Push, which is exemplified in this article. One important aspect to note about Server Push, is that the pushed resources don't make it all the way to the web application, but only to the browser cache. Meaning, the javascript code in your web application will not see the pushed resource, but if you make an API call to get the resource that was pushed by server, then the browser will not make another call to server, and instead respond immediately with the data that was server-push'd with the previous request. gRPC is a server-to-server RPC framework that is built on top of HTTP/2 and boasts high performance. However, I'm not sure that if (1) REST like calls are done from one server to another, (2) with very few headers being exchanged, and each header having a small value, (3) data is exchanged in binary format, (4) the server handling the incoming request is able to leverage HTTP/2, then, whether gRPC would be much more faster than ordinary REST like calls done today. In other words, I am not sure if usual REST request-response structure is slow simply because of ailments of HTTP/1.1, or if the gRPC framework uses some novel design.

OLTP and OLAP

The difference between OLTP vs OLAP is covered in articles at StackOverflow and here. OLTP, or "On-line Transaction Processing", is characterized by a large number of short on-line transactions. Such operations have a strong emphasis being on "short", i.e. the user sees a response very soon after making a request. On the other hand, OLAP, or "On-line Analytical Processing" involves processing of historical data via complex aggregations. Such operations are not made very often because, (1) nobody wants to make a request and wait for a long time before getting response, and, (2) the actual reason - the response data only changes negligibly within a small time, and so, there's no reason to make such requests often. Note that OLTP and OLAP is categorization of request being processed by the server, based on the efforts needed to process the request. However, both requests can be web requests, sent using HTTP, etc., and those are not the factors that differentiate OLTP vs OLAP.