Making HTTP requests is core functionality for modern languages and one of the first things many developers learn when acclimating to new environments. When it comes to Node.js there are a fair amount of solutions to this problem both built into the language and by the community. Let’s take a look at some of the most popular ones.
A few years ago, I wrote a similar post on this topic. But now that async/await functionality is more pervasive and mainstream in JavaScript code, making network requests is more straightforward than ever. On top of that, Request, the previously most popular HTTP library for Node has been deprecated. So it's time for an updated guide!
As in the other post, we’ll be using NASA’s Astronomy Picture of the Day API as the JSON API that we are interacting with in all of these examples because space is the coolest thing ever.
Before moving on, make sure you have up to date versions of Node.js and npm installed on your machine.
CF worker to log all REST API activity in a non-blocking way to an external service - including response time. The fetch requires only one parameter which is the URL of the resource that you want to fetch: let response = fetch(url); The fetch method returns a Promise so you can use the then and catch methods to handle it. Await your turn. Async/await is a new approach that was added to JavaScript in ES2017. It actually uses promises behind the scenes, so you can think of async/await as syntactic sugar over promises. Thus, you still need to understand promises to use async/await. Like promises, async/await is non-blocking. The fetch API is a native JavaScript function that we can use to interact with web services. How can we use fetch with async and await? And how can we use this with TypeScript to get a strongly-typed response? Let’s find out Making a simple request. Fetch supports async and await out of the box.
HTTP – the Standard Library
For the purpose of making comparisons, let's start by taking a look at the default HTTP module without Promises and async/await. With this module, you have the advantage of not relying on any dependencies, but it isn't very developer-friendly compared to other solutions.
The following code will send a
GET
request to NASA’s API and print out the URL for the astronomy picture of the day as well as an explanation:Much of the
http
and the https
module’s functionality is fairly low-level. You’re required to receive response data in chunks and need to explicitly listen for when all of the data is received. You also need to parse the response data manually. This module also does not support HTTPS by default, so we need to require the https
module instead if the API we are using communicates over HTTPS. Although you can't use the async/await feature for the HTTP requests made with this library, you could potentially use asynchronous streams for chunking the data.It may take a bit more effort to get the data you want, but is a great utility if you don’t want to add dependencies to your codebase or want access to its low level functionality.
Got
Got is a great choice if you want a more lightweight library. It is designed to be human-friendly and is also available to use by default in Twilio Functions.
Install Got with npm:
Because Got uses promises, you’ll see that you need much less code to accomplish the same task that we did above, and can also take advantage of the async and await keywords:
This library does not parse JSON by default, so we needed to add
{ json: true }
as an argument when making the request. Got is a fantastic option if you just want an easy to use library that deals with HTTP requests in a sane way.Axios
Axios is another Promise based HTTP client that works for the browser as well as node.js.
To install Axios from npm, enter the following command in your terminal:
The following code will accomplish the same task of logging the URL and explaining the astronomy picture of the day:
Axios even parses JSON responses by default. Pretty convenient! You can even make multiple concurrent requests with
axios.all
if you wanted to do something like get the astronomy picture of two different days at once:SuperAgent
Similarly to Axios, SuperAgent is another popular library primarily used for making asynchronous requests in the browser but works in Node.js as well. Install SuperAgent with the following command:
What is cool about SuperAgent is that you have other useful functions that you can chain onto requests such as
query()
to add parameters to the request rather than passing them through as an options
object. We’ve been manually adding them in the URL in the previous examples, but notice how SuperAgent gives you a function to do this:Just like with Axios you don’t have to parse the JSON response yourself, which is pretty cool.
node-fetch
node-fetch is a light weight module that brings the browser library
window.fetch
to Node.js with minimal code.As with the previous examples, install node-fetch with the following:
Recent versions of this library use promises, so we're able to use async/await syntax with it as well:
This library does have a built in function for converting the response to JSON, but it does not do it automatically in the same way that Axios and SuperAgent do. This is a great library for people who are used to using the Fetch API in the browser.
Final Thoughts
This doesn’t cover all of the solutions, but now you see how the basic functionality works in a few of the most popular HTTP libraries in Node.
Other languages have a similar variety of libraries to tackle this problem. Check out these other tutorials in Swift, Python and Ruby. Also, check out our Node.js Quickstarts for a place to apply your new skills.
What are your favorite ways to send HTTP requests? Feel free to reach out and let me know or ask any questions:
- Email: [email protected]
- Twitter: @Sagnewshreds
- Github: Sagnew
- Twitch (streaming live code): Sagnewshreds
JavaScript can send network requests to the server and load new information whenever it’s needed.
For example, we can use a network request to:
- Submit an order,
- Load user information,
- Receive latest updates from the server,
- …etc.
…And all of that without reloading the page!
There’s an umbrella term “AJAX” (abbreviated Asynchronous JavaScript And XML) for network requests from JavaScript. We don’t have to use XML though: the term comes from old times, that’s why that word is there. You may have heard that term already.
There are multiple ways to send a network request and get information from the server.
The
fetch()
method is modern and versatile, so we’ll start with it. It’s not supported by old browsers (can be polyfilled), but very well supported among the modern ones.The basic syntax is:
url
– the URL to access.options
– optional parameters: method, headers etc.
Without
options
, that is a simple GET request, downloading the contents of the url
.The browser starts the request right away and returns a promise that the calling code should use to get the result.
Getting a response is usually a two-stage process.
First, the
promise
, returned by fetch
, resolves with an object of the built-in Response class as soon as the server responds with headers.At this stage we can check HTTP status, to see whether it is successful or not, check headers, but don’t have the body yet.
The promise rejects if the
fetch
was unable to make HTTP-request, e.g. network problems, or there’s no such site. Abnormal HTTP-statuses, such as 404 or 500 do not cause an error.We can see HTTP-status in response properties:
Typescript Fetch Api
status
– HTTP status code, e.g. 200.ok
– boolean,true
if the HTTP status code is 200-299.
For example:
Second, to get the response body, we need to use an additional method call. Cadintosh x 8 5 1.
Response
provides multiple promise-based methods to access the body in various formats:response.text()
– read the response and return as text,response.json()
– parse the response as JSON,response.formData()
– return the response asFormData
object (explained in the next chapter),response.blob()
– return the response as Blob (binary data with type),response.arrayBuffer()
– return the response as ArrayBuffer (low-level representation of binary data),- additionally,
response.body
is a ReadableStream object, it allows you to read the body chunk-by-chunk, we’ll see an example later.
For instance, let’s get a JSON-object with latest commits from GitHub:
Or, the same without
await
, using pure promises syntax:To get the response text,
await response.text()
instead of .json()
:As a show-case for reading in binary format, let’s fetch and show a logo image of “fetch” specification (see chapter Blob for details about operations on
Blob
):We can choose only one body-reading method.
If we’ve already got the response with
response.text()
, then response.json()
won’t work, as the body content has already been processed.Response headers
The response headers are available in a Map-like headers object in
response.headers
.It’s not exactly a Map, but it has similar methods to get individual headers by name or iterate over them:
Request headers
To set a request header in
fetch
, we can use the headers
option. It has an object with outgoing headers, like this:…But there’s a list of forbidden HTTP headers that we can’t set:
React Fetch Await
Accept-Charset
,Accept-Encoding
Access-Control-Request-Headers
Access-Control-Request-Method
Connection
Content-Length
Cookie
,Cookie2
Date
DNT
Expect
Host
Keep-Alive
Origin
Referer
TE
Trailer
Transfer-Encoding
Upgrade
Via
Proxy-*
Sec-*
These headers ensure proper and safe HTTP, so they are controlled exclusively by the browser.
POST requests
To make a
POST
request, or a request with another method, we need to use fetch
options:method
– HTTP-method, e.g.POST
,body
– the request body, one of:- a string (e.g. JSON-encoded),
FormData
object, to submit the data asform/multipart
,Blob
/BufferSource
to send binary data,- URLSearchParams, to submit the data in
x-www-form-urlencoded
encoding, rarely used.
The JSON format is used most of the time.
For example, this code submits
user
object as JSON:Please note, if the request
body
is a string, then Content-Type
header is set to text/plain;charset=UTF-8
by default.But, as we’re going to send JSON, we use
headers
option to send application/json
instead, the correct Content-Type
for JSON-encoded data.Sending an image
We can also submit binary data with
fetch
using Blob
or BufferSource
objects.In this example, there’s a
<canvas>
Bettertouchtool 3 344. where we can draw by moving a mouse over it. A click on the “submit” button sends the image to the server:Please note, here we don’t set
Content-Type
header manually, because a Blob
object has a built-in type (here image/png
, as generated by toBlob
). For Blob
objects that type becomes the value of Content-Type
.The
submit()
function can be rewritten without async/await
like this:Summary
A typical fetch request consists of two
await
calls:Or, without
await
:Response properties:
response.status
– HTTP code of the response,response.ok
–true
is the status is 200-299.response.headers
– Map-like object with HTTP headers.
Methods to get response body:
Await Fetch
response.text()
– return the response as text,response.json()
– parse the response as JSON object,response.formData()
– return the response asFormData
object (form/multipart encoding, see the next chapter),response.blob()
– return the response as Blob (binary data with type),response.arrayBuffer()
– return the response as ArrayBuffer (low-level binary data),
Fetch options so far:
Await Fetch Error
method
– HTTP-method,headers
– an object with request headers (not any header is allowed),body
– the data to send (request body) asstring
,FormData
,BufferSource
,Blob
orUrlSearchParams
object.
Async Await With Fetch
In the next chapters we’ll see more options and use cases of
fetch
.