Module http | Tarantool
Sign in Sign up
Download

Version:

latest (2.x)
Reference Built-in modules reference Module http

Module http

The http module, specifically the http.client submodule, provides the functionality of an HTTP client with support for HTTPS and keepalive. The HTTP client uses the libcurl library under the hood and takes into account the environment variables libcurl understands.

Creating a client

To create an HTTP client, call the http.client.new() function:

local http_client = require('http.client').new()

Optionally, this function can accept specific client configuration options.

Making requests

After creating the client, you can make HTTP requests.

HTTP method

The main way of making HTTP requests is the request method, which accepts the following arguments:

  • An HTTP method, such as GET, POST, PUT, and so on.
  • A request URL. You can use the uri module to construct a URL from its components.
  • (Optional) a request body for the POST, PUT, and PATCH methods.
  • (Optional) request options, such as request headers, SSL settings, and so on.

The example below shows how to make the GET request to the https://httpbin.org/get URL:

local http_client = require('http.client').new()
local response = http_client:request('GET', 'https://httpbin.org/get')

In addition to request, the HTTP client provides the API for particular HTTP methods: get, post, put, and so on. For example, you can replace the request above by calling get as follows:

local http_client = require('http.client').new()
local response = http_client:get('https://httpbin.org/get')

Query parameters

To add query string parameters, use the params option exposed by the request_options object:

local http_client = require('http.client').new()
local response = http_client:get('https://httpbin.org/get', {
    params = { page = 1 },
})
print('URL: '..response.url)

In the example above, the requested URL is https://httpbin.org/get?page=1.

Note

If a parameter name or value contains a reserved character (for example, & or =), the HTTP client encodes a query string.

Headers

To add headers to the request, use the headers option:

local http_client = require('http.client').new()
local response = http_client:get('https://httpbin.org/headers', {
    headers = {
        ['User-Agent'] = 'Tarantool HTTP client',
        ['Authorization'] = 'Bearer abc123'
    }
})
print('Authorization: '..response:decode()['headers']['Authorization'])

Cookies

You can add cookies to the request using the headers option:

local http_client = require('http.client').new()
local response = http_client:get('https://httpbin.org/cookies', {
    headers = {
        ['Cookie'] = 'session_id=abc123; csrftoken=u32t4o;',
    }
})
print(response.body)

To learn how to obtain cookies passed in the Set-Cookie response header, see Response cookies.

Body

Serialization

The HTTP client automatically serializes the content in a specific format when sending a request based on the specified Content-Type header. By default, the client uses the application/json content type and sends data serialized as JSON:

local http_client = require('http.client').new()
local response = http_client:post('https://httpbin.org/anything', {
    user_id = 123,
    user_name = 'John Smith'
})
print('Posted data: '..response:decode()['data'])

The body for the request above might look like this:

{
    "user_id": 123,
    "user_name": "John Smith"
}

To send data in the YAML or MsgPack format, set the Content-Type header explicitly to application/yaml or application/msgpack, for example:

local http_client = require('http.client').new()
local response = http_client:post('https://httpbin.org/anything', {
    user_id = 123,
    user_name = 'John Smith'
}, {
    headers = {
        ['Content-Type'] = 'application/yaml',
    }
})
print('Posted data:\n'..response:decode()['data'])

In this case, the request body is serialized to YAML:

user_id: 123
user_name: John Smith

Form parameters

To send form parameters using the application/x-www-form-urlencoded type, use the params option:

local http_client = require('http.client').new()
local response = http_client:post('https://httpbin.org/anything', nil, {
    params = { user_id = 123, user_name = 'John Smith' },
})
print('User ID: '..response:decode()['form']['user_id'])

Streaming upload

The HTTP client supports chunked writing of request data. This can be achieved as follows:

  1. Set the chunked option to true. In this case, a request method returns io_object instead of response_object.
  2. Use the io_object.write() method to write a chunk of data.
  3. Call the io_object.finish() method to finish writing data and make a request.

The example below shows how to upload data in two chunks:

local http_client = require('http.client').new()
local json = require('json')

local io = http_client:post('https://httpbin.org/anything', nil, {chunked = true})
io:write('Data part 1')
io:write('Data part 2')
io:finish()
response = io:read('\r\n')
decoded_data = json.decode(response)
print('Posted data: '..decoded_data['data'])

Receiving responses

All methods that are used to make an HTTP request (request, get, post, etc.) receive response_object. response_object exposes the API required to get a response body and obtain response parameters, such as a status code, headers, and so on.

Status code

To get a response’s status code and text, use the response_object.status and response_object.reason options, respectively:

local http_client = require('http.client').new()
local response = http_client:get('https://httpbin.org/get')
print('Status: '..response.status..' '.. response.reason)

Headers

The response_object.headers option returns a set of response headers. The example below shows how to obtain the ETag header value:

local http_client = require('http.client').new()
local response = http_client:get('https://httpbin.org/etag/7c876b7e')
print('ETag header value: '..response.headers['etag'])

Cookies

To obtain response cookies, use response_object.cookies. This option returns a Lua table where a cookie name is the key. The value is an array of two elements where the first one is the cookie value and the second one is an array with the cookie’s options.

The example below shows how to obtain the session_id cookie value:

local http_client = require('http.client').new()
local response = http_client:get('https://httpbin.org/cookies/set?session_id=abc123&csrftoken=u32t4o&', {follow_location = false})
print("'session_id' cookie value: "..response.cookies['session_id'][1])

Response body

Deserialization

The HTTP client can deserialize response data to a Lua object based on the Content-Type response header value. To deserialize data, call the response_object.decode() method. In the example below, the JSON response is deserialized into a Lua object:

local http_client = require('http.client').new()
local response = http_client:get('https://httpbin.org/json')
local document = response:decode()
print("'title' value: "..document['slideshow']['title'])

The following content types are supported out of the box:

  • application/json
  • application/msgpack
  • application/yaml

If the response doesn’t have the Content-Type header, the client uses application/json.

To deserialize other content types, you need to provide a custom deserializer using the client_object.decoders property. In the example below, application/xml responses are decoded using the luarapidxml library:

local http_client = require('http.client').new()
local xml = require("luarapidxml")

http_client.decoders = {
    ['application/xml'] = function(body, _content_type)
        return xml.decode(body)
    end,
}

local response = http_client:get('https://httpbin.org/xml')
local document = response:decode()
print("'title' value: "..document['attr']['title'])

The output for the code sample above should look as follows:

'title' value: Sample Slide Show

Decompressing

The HTTP client can automatically decompress a response body based on the Content-Encoding header value. To enable this capability, pass the required formats using the request_options.accept_encoding option:

local http_client = require('http.client').new()
local response = http_client:get('https://httpbin.org/gzip', {accept_encoding = "br, gzip, deflate"})
print('Is response gzipped: '..tostring(response:decode()['gzipped']))

Streaming download

The HTTP client supports chunked reading of request data. This can be achieved as follows:

  1. Set the chunked option to true. In this case, a request method returns io_object instead of response_object.
  2. Use the io_object.read() method to read data in chunks of a specified length or up to a specific delimiter.
  3. Call the io_object.finish() method to finish reading data.

The example below shows how to get chunks of a JSON response sequentially instead of waiting for the entire response:

local http_client = require('http.client').new()
local json = require('json')

local io = http_client:get('https://httpbin.org/stream/5', {chunked = true})
local chunk_ids = ''
while data ~= '' do
    local data = io:read('\n')
    if data == '' then break end
    local decoded_data = json.decode(data)
    chunk_ids = chunk_ids..decoded_data['id']..' '
end
print('IDs of received chunks: '..chunk_ids)
io:finish()

Redirects

By default, the HTTP client redirects to a URL provided in the Location header of a 3xx response. If required, you can disable redirection using the follow_location option:

local http_client = require('http.client').new()
local response = http_client:get('https://httpbin.org/cookies/set?session_id=abc123&csrftoken=u32t4o&', {follow_location = false})

API Reference

Functions  
http.client.new() Create an HTTP client instance
Objects  
client_options Configuration options of the client
client_object An HTTP client instance
request_options Options passed to a request
response_object A response object
io_object An IO object used to read/write data in chunks

http.client.new()

http.client.new([options])

Create an HTTP client instance.

Parameters:
Return:

a new HTTP client instance (see client_object)
Rtype:

userdata

Example

local http_client = require('http.client').new()

client_options

object client_options

Configuration options of the client. These options can be passed to the http.client.new() function.

client_options.max_connections

Specifies the maximum number of entries in the cache. This option affects libcurl CURLMOPT_MAXCONNECTS. The default is -1.

Example

local http_client = require('http.client').new({max_connections = 5})

Note

Do not set max_connections to less than max_total_connections unless you are confident about your actions. If max_connections is less than max_total_connections, libcurl doesn’t reuse sockets in some cases for requests that go to the same host. If the limit is reached and a new request occurs, then libcurl creates a new socket first, sends the request, waits for the first connection to be free, and closes it to avoid exceeding the max_connections cache size. In the worst case, libcurl creates a new socket for every request, even if all requests go to the same host.

client_options.max_total_connections

Specifies the maximum number of active connections. This option affects libcurl CURLMOPT_MAX_TOTAL_CONNECTIONS.

Note

You may want to control the maximum number of sockets that a particular HTTP client uses simultaneously. If a system passes many requests to distinct hosts, then libcurl cannot reuse sockets. In this case, setting max_total_connections may be useful since it causes curl to avoid creating too many sockets, which would not be used anyway.

client_object

object client_object

An HTTP client instance that exposes the API for making requests. To create the client, call http.client.new().

client_object:request(method, url, body, opts)

Make an HTTP request and receive a response.

Parameters:
  • method (string) – a request HTTP method. Possible values: GET, POST, PUT, PATCH, OPTIONS, HEAD, DELETE, TRACE, CONNECT.
  • url (string) – a request URL, for example, https://httpbin.org/get
  • body (string) – a request body (see Body)
  • opts (table) – request options (see request_options)
Return:

This method returns one of the following objects:
Rtype:

table

Example

local http_client = require('http.client').new()
local response = http_client:request('GET', 'https://httpbin.org/get')

See also: Making requests, Receiving responses

client_object:get(url, opts)

Make a GET request and receive a response.

Parameters:
Return:

This method might return one of the following objects:
Rtype:

table

Example

local http_client = require('http.client').new()
local response = http_client:get('https://httpbin.org/get')

See also: Making requests, Receiving responses

client_object:post(url, body, opts)

Make a POST request and receive a response.

Parameters:
Return:

This method might return one of the following objects:
Rtype:

table

Example

local http_client = require('http.client').new()
local response = http_client:post('https://httpbin.org/anything', {
    user_id = 123,
    user_name = 'John Smith'
})
print('Posted data: '..response:decode()['data'])

See also: Making requests, Receiving responses

client_object:put(url, body, opts)

Make a PUT request and receive a response.

Parameters:
Return:

This method might return one of the following objects:
Rtype:

table

See also: Making requests, Receiving responses

client_object:patch(url, body, opts)

Make a PATCH request and receive a response.

Parameters:
Return:

This method might return one of the following objects:
Rtype:

table

See also: Making requests, Receiving responses

client_object:delete(url, opts)

Make a DELETE request and receive a response.

Parameters:
Return:

This method might return one of the following objects:
Rtype:

table

See also: Making requests, Receiving responses

client_object:head(url, opts)

Make a HEAD request and receive a response.

Parameters:
Return:

This method might return one of the following objects:
Rtype:

table

See also: Making requests, Receiving responses

client_object:options(url, opts)

Make an OPTIONS request and receive a response.

Parameters:
Return:

This method might return one of the following objects:
Rtype:

table

See also: Making requests, Receiving responses

client_object:trace(url, opts)

Make a TRACE request and receive a response.

Parameters:
Return:

This method might return one of the following objects:
Rtype:

table

See also: Making requests, Receiving responses

client_object:connect(url, opts)

Make a CONNECT request and receive a response.

Parameters:
Return:

This method might return one of the following objects:
Rtype:

table

See also: Making requests, Receiving responses

client_object:stat()

Get a table with statistics for the HTTP client:

  • active_requests – the number of currently executing requests
  • sockets_added – the total number of sockets added into an event loop
  • sockets_deleted – the total number of sockets from an event loop
  • total_requests – the total number of requests
  • http_200_responses – the total number of requests that returned HTTP 200 OK responses
  • http_other_responses – the total number of requests that returned non-200 OK responses
  • failed_requests – the total number of failed requests, including system, curl, and HTTP errors
client_object.decoders

Since: 2.11.0

Decoders used to deserialize response data based on the Content-Type header value. Learn more from Deserialization.

request_options

object request_options

Options passed to a request method (request, get, post, and so on).

See also: Making requests

request_options.ca_file

The path to an SSL certificate file to verify the peer with.

Rtype:string
request_options.ca_path

The path to a directory holding one or more certificates to verify the peer with.

Rtype:string
request_options.chunked

Since: 2.11.0

Specifies whether an HTTP client should return the full response (response_object) or an IO object (io_object) used for streaming download/upload.

Rtype:boolean

See also: Streaming download, Streaming upload

request_options.headers

A table of HTTP headers passed to a request.

Rtype:table
request_options.params

Since: 2.11.0

A table of parameters passed to a request. The behavior of this option depends on the request type, for example:

Rtype:table
request_options.keepalive_idle

A delay (in seconds) the operating system waits while the connection is idle before sending keepalive probes.

Rtype:integer

See also: CURLOPT_TCP_KEEPIDLE, keepalive_interval

request_options.keepalive_interval

The interval (in seconds) the operating system waits between sending keepalive probes. If both keepalive_idle and keepalive_interval are set, then Tarantool also sets the HTTP keepalive headers: Connection:Keep-Alive and Keep-Alive:timeout=<keepalive_idle>. Otherwise, Tarantool sends Connection:close.

Rtype:integer

See also: CURLOPT_TCP_KEEPINTVL

request_options.low_speed_limit

The average transfer speed in bytes per second that the transfer should be below during “low speed time” seconds for the library to consider it to be too slow and abort.

Rtype:integer

See also: CURLOPT_LOW_SPEED_LIMIT

request_options.low_speed_time

The time that the transfer speed should be below the “low speed limit” for the library to consider it too slow and abort.

Rtype:integer

See also: CURLOPT_LOW_SPEED_TIME

request_options.max_header_name_len

The maximum length of a header name. If a header name length exceeds this value, it is truncated to this length. The default value is 32.

Rtype:integer
request_options.follow_location

Specify whether the HTTP client follows redirect URLs provided in the Location header for 3xx responses. When a non-3xx response is received, the client returns it as a result. If you set this option to false, the client returns the first 3xx response.

Rtype:boolean

See also: Redirects

request_options.no_proxy

A comma-separated list of hosts that do not require proxies, or *, or ''.

  • Set no_proxy = host [, host ...] to specify hosts that can be reached without requiring a proxy, even if proxy is set to a non-blank value and/or if a proxy-related environment variable has been set.
  • Set no__proxy = '*' to specify that all hosts can be reached without requiring a proxy, which is equivalent to setting proxy=''.
  • Set no_proxy = '' to specify that no hosts can be reached without requiring a proxy, even if a proxy-related environment variable (HTTP_PROXY) is used.

If no_proxy is not set, then a proxy-related environment variable (HTTP_PROXY) may be used.

Rtype:string

See also: CURLOPT_NOPROXY

request_options.proxy

A proxy server host or IP address, or ''.

  • If proxy is a host or IP address, then it may begin with a scheme, for example, https:// for an HTTPS proxy or http:// for an HTTP proxy.
  • If proxy is set to '' an empty string, then proxy use is disabled, and no proxy-related environment variable is used.
  • If proxy is not set, then a proxy-related environment variable may be used, such as HTTP_PROXY or HTTPS_PROXY or FTP_PROXY, or ALL_PROXY if the protocol can be any protocol.
Rtype:string

See also: CURLOPT_PROXY

request_options.proxy_port

A proxy server port. The default is 443 for an HTTPS proxy and 1080 for a non-HTTPS proxy.

Rtype:integer

See also: CURLOPT_PROXYPORT

request_options.proxy_user_pwd

A proxy server username and password. This option might have one of the following formats:

  • proxy_user_pwd = user_name:
  • proxy_user_pwd = :password
  • proxy_user_pwd = user_name:password
Rtype:string

See also: CURLOPT_USERPWD

request_options.ssl_cert

A path to an SSL client certificate file.

Rtype:string

See also: CURLOPT_SSLCERT

request_options.ssl_key

A path to a private key file for a TLS and SSL client certificate.

Rtype:string

See also: CURLOPT_SSLKEY

request_options.timeout

The number of seconds to wait for a curl API read request before timing out. The default timeout is set to infinity (36586400100 seconds).

Rtype:integer
request_options.unix_socket

A socket name to use instead of an Internet address for a local connection.

Rtype:string

Example: /tmp/unix_domain_socket.sock

request_options.verbose

Turn on/off a verbose mode.

Rtype:boolean
request_options.verify_host

Enable verification of the certificate’s name (CN) against the specified host.

Rtype:integer

See also: CURLOPT_SSL_VERIFYHOST

request_options.verify_peer

Set on/off verification of the peer’s SSL certificate.

Rtype:integer

See also: CURLOPT_SSL_VERIFYPEER

request_options.accept_encoding

Enable decompression of an HTTP response data based on the specified Accept-Encoding request header. You can pass the following values to this option:

  • '' – if an empty string is passed, the Accept-Encoding contains all the supported encodings (identity, deflate, gzip, and br).
  • br, gzip, deflate – a comma-separated list of encodings passed in Accept-Encoding.
Rtype:string

See also: CURLOPT_ACCEPT_ENCODING

response_object

object response_object

A response object returned by a request method (request, get, post, and so on).

See also: io_object

response_object.status

A response status code.

Rtype:integer

See also: Status code

response_object.reason

A response status text.

Rtype:string

See also: Status code

response_object.headers

Response headers.

Rtype:table

See also: Headers

response_object.cookies

Response cookies. The value is an array of two elements where the first one is the cookie value and the second one is an array with the cookie’s options.

Rtype:table

See also: Cookies

response_object.body

A response body. Use decode to decode the response body.

Rtype:table

See also: Response body

response_object.proto

An HTTP protocol version.

Rtype:string
response_object:decode()

Since: 2.11.0

Decode the response body to a Lua object based on the content type.

Return:a decoded body
Rtype:table

See also: Deserialization

io_object

object io_object

Since: 2.11.0

An IO object used to read or write data in chunks. To get an IO object instead of the full response (response_object), you need to set the chunked request option to true.

io_object:read(chunk[, timeout])
io_object:read(delimiter[, timeout])
io_object:read({chunk = chunk, delimiter = delimiter}[, timeout])

Read request data in chunks of a specified length or up to a specific delimiter.

Parameters:
  • chunk (integer) – the maximum number of bytes to read
  • delimiter (string) – the delimiter used to stop reading data
  • timeout (integer) – the number of seconds to wait. The default is 10.
Return:

A chunk of read data. Returns an empty string if there is nothing more to read.
Rtype:

string

See also: Streaming download

io_object:write(data[, timeout])

Write the specified chunk of data.

Parameters:
  • data (table) – data to be written
  • timeout (integer) – the number of seconds to wait. The default is 10.

See also: Streaming upload

io_object:finish([timeout])

Finish reading or writing data.

Parameters:
  • timeout (integer) – the number of seconds to wait. The default is 10.

See also: Streaming download, Streaming upload

Found what you were looking for?
Feedback
Module fun
Module iconv
Single-page version