Module fiber
With the fiber
module, you can:
- create, run and manage fibers,
- send and receive messages between different processes (i.e. different connections, sessions, or fibers) via channels, and
- use a synchronization mechanism for fibers,
similar to “condition variables” and similar to operating-system functions
such as
pthread_cond_wait()
pluspthread_cond_signal()
.
Below is a list of all fiber
functions and members.
Name | Use |
---|---|
fiber.create() | Create and start a fiber |
fiber.new() | Create but do not start a fiber |
fiber.self() | Get a fiber object |
fiber.find() | Get a fiber object by ID |
fiber.sleep() | Make a fiber go to sleep |
fiber.yield() | Yield control |
fiber.status() | Get the current fiber’s status |
fiber.info() | Get information about all fibers |
fiber.kill() | Cancel a fiber |
fiber.testcancel() | Check if the current fiber has been cancelled |
fiber_object:id() | Get a fiber’s ID |
fiber_object:name() | Get a fiber’s name |
fiber_object:name(name) | Set a fiber’s name |
fiber_object:status() | Get a fiber’s status |
fiber_object:cancel() | Cancel a fiber |
fiber_object.storage | Local storage within the fiber |
fiber_object:set_joinable() | Make it possible for a new fiber to join |
fiber_object:join() | Wait for a fiber’s state to become ‘dead’ |
fiber.time() | Get the system time in seconds |
fiber.time64() | Get the system time in microseconds |
fiber.clock() | Get the monotonic time in seconds |
fiber.clock64() | Get the monotonic time in microseconds |
fiber.channel() | Create a communication channel |
channel_object:put() | Send a message via a channel |
channel_object:close() | Close a channel |
channel_object:get() | Fetch a message from a channel |
channel_object:is_empty() | Check if a channel is empty |
channel_object:count() | Count messages in a channel |
channel_object:is_full() | Check if a channel is full |
channel_object:has_readers() | Check if an empty channel has any readers waiting |
channel_object:has_writers() | Check if a full channel has any writers waiting |
channel_object:is_closed() | Check if a channel is closed |
fiber.cond() | Create a condition variable |
cond_object:wait() | Make a fiber go to sleep until woken by another fiber |
cond_object:signal() | Wake up a single fiber |
cond_object:broadcast() | Wake up all fibers |
A fiber is a set of instructions which are executed with cooperative multitasking. Fibers managed by the fiber module are associated with a user-supplied function called the fiber function.
A fiber has three possible states: running, suspended or dead. When a fiber is created with fiber.create(), it is running. When a fiber is created with fiber.new() or yields control with fiber.sleep(), it is suspended. When a fiber ends (because the fiber function ends), it is dead.
All fibers are part of the fiber registry. This registry can be searched with fiber.find() - via fiber id (fid), which is a numeric identifier.
A runaway fiber can be stopped with fiber_object.cancel.
However, fiber_object.cancel is advisory — it works
only if the runaway fiber calls fiber.testcancel()
occasionally. Most box.*
functions, such as
box.space…delete() or
box.space…update(), do call
fiber.testcancel() but
box.space…select{} does not. In practice, a runaway
fiber can only become unresponsive if it does many computations and does not
check whether it has been cancelled.
The other potential problem comes from fibers which never get scheduled, because they are not subscribed to any events, or because no relevant events occur. Such morphing fibers can be killed with fiber.kill() at any time, since fiber.kill() sends an asynchronous wakeup event to the fiber, and fiber.testcancel() is checked whenever such a wakeup event occurs.
Like all Lua objects, dead fibers are garbage collected. The Lua garbage collector frees pool allocator memory owned by the fiber, resets all fiber data, and returns the fiber (now called a fiber carcass) to the fiber pool. The carcass can be reused when another fiber is created.
A fiber has all the features of a Lua coroutine and all the programming concepts that apply for Lua coroutines will apply for fibers as well. However, Tarantool has made some enhancements for fibers and has used fibers internally. So, although use of coroutines is possible and supported, use of fibers is recommended.
-
fiber.
create
(function[, function-arguments]) Create and start a fiber. The fiber is created and begins to run immediately.
Parameters: - function – the function to be associated with the fiber
- function-arguments – what will be passed to function
Return: created fiber object
Rtype: userdata
Example:
tarantool> fiber = require('fiber') --- ... tarantool> function function_name() > print("I'm a fiber") > end --- ... tarantool> fiber_object = fiber.create(function_name); print("Fiber started") I'm a fiber Fiber started --- ...
-
fiber.
new
(function[, function-arguments]) Create but do not start a fiber: the fiber is created but does not begin to run immediately – it starts after the fiber creator (that is, the job that is calling
fiber.new()
) yields, under transaction control. The initial fiber state is ‘suspended’. Thusfiber.new()
differs slightly from fiber.create().Ordinarily
fiber.new()
is used in conjunction with fiber_object:set_joinable() and fiber_object:join().Parameters: - function – the function to be associated with the fiber
- function-arguments – what will be passed to function
Return: created fiber object
Rtype: userdata
Example:
tarantool> fiber = require('fiber') --- ... tarantool> function function_name() > print("I'm a fiber") > end --- ... tarantool> fiber_object = fiber.new(function_name); print("Fiber not started yet") Fiber not started yet --- ... tarantool> I'm a fiber --- ...
-
fiber.
self
() Return: fiber object for the currently scheduled fiber. Rtype: userdata Example:
tarantool> fiber.self() --- - status: running name: interactive id: 101 ...
-
fiber.
find
(id) Parameters: - id – numeric identifier of the fiber.
Return: fiber object for the specified fiber.
Rtype: userdata
Example:
tarantool> fiber.find(101) --- - status: running name: interactive id: 101 ...
-
fiber.
sleep
(time) Yield control to the scheduler and sleep for the specified number of seconds. Only the current fiber can be made to sleep.
Parameters: - time – number of seconds to sleep.
Exception: see the Example of yield failure.
Example:
tarantool> fiber.sleep(1.5) --- ...
-
fiber.
yield
() Yield control to the scheduler. Equivalent to fiber.sleep(0).
Exception: see the Example of yield failure. Example:
tarantool> fiber.yield() --- ...
-
fiber.
status
([fiber_object]) Return the status of the current fiber. Or, if optional fiber_object is passed, return the status of the specified fiber.
Return: the status of fiber
. One of: “dead”, “suspended”, or “running”.Rtype: string Example:
tarantool> fiber.status() --- - running ...
-
fiber.
info
() Return information about all fibers.
Return: number of context switches, backtrace, id, total memory, used memory, name for each fiber. Rtype: table Example:
tarantool> fiber.info() --- - 101: csw: 7 backtrace: [] fid: 101 memory: total: 65776 used: 0 name: interactive ...
-
fiber.
kill
(id) Locate a fiber by its numeric id and cancel it. In other words, fiber.kill() combines fiber.find() and fiber_object:cancel().
Parameters: - id – the id of the fiber to be cancelled.
Exception: the specified fiber does not exist or cancel is not permitted.
Example:
tarantool> fiber.kill(fiber.id()) -- kill self, may make program end --- - error: fiber is cancelled ...
-
fiber.
testcancel
() Check if the current fiber has been cancelled and throw an exception if this is the case.
Note
Even if you catch the exception, the fiber will remain cancelled. Most types of calls will check
fiber.testcancel()
. However, some functions (id
,status
,join
etc.) will return no error. We recommend application developers to implement occasional checks with fiber.testcancel() and to end fiber’s execution as soon as possible in case it has been cancelled.Example:
tarantool> fiber.testcancel() --- - error: fiber is cancelled ...
-
object
fiber_object
-
fiber_object:
id
() Parameters: - fiber_object – generally this is an object referenced in the return from fiber.create or fiber.self or fiber.find
Return: id of the fiber.
Rtype: number
fiber.self():id()
can also be expressed asfiber.id()
.Example:
tarantool> fiber_object = fiber.self() --- ... tarantool> fiber_object:id() --- - 101 ...
-
fiber_object:
name
() Parameters: - fiber_object – generally this is an object referenced in the return from fiber.create or fiber.self or fiber.find
Return: name of the fiber.
Rtype: string
fiber.self():name()
can also be expressed asfiber.name()
.Example:
tarantool> fiber.self():name() --- - interactive ...
-
fiber_object:
name
(name[, options]) Change the fiber name. By default a Tarantool server’s interactive-mode fiber is named ‘interactive’ and new fibers created due to fiber.create are named ‘lua’. Giving fibers distinct names makes it easier to distinguish them when using fiber.info. Max length is 32.
Parameters: - fiber_object – generally this is an object referenced in the return from fiber.create or fiber.self or fiber.find
- name (string) – the new name of the fiber.
- options –
truncate=true
– truncates the name to the max length if it is too long. If this option is false (the default),fiber.name(new_name)
fails with an exception if a new name is too long.
Return: nil
Example:
tarantool> fiber.self():name('non-interactive') --- ...
-
fiber_object:
status
() Return the status of the specified fiber.
Parameters: - fiber_object – generally this is an object referenced in the return from fiber.create or fiber.self or fiber.find
Return: the status of fiber. One of: “dead”, “suspended”, or “running”.
Rtype: string
fiber.self():status(
can also be expressed asfiber.status()
.Example:
tarantool> fiber.self():status() --- - running ...
-
fiber_object:
cancel
() Cancel a fiber. Running and suspended fibers can be cancelled. After a fiber has been cancelled, attempts to operate on it will cause errors, for example fiber_object:name() will cause
error: the fiber is dead
. But a dead fiber can still report its id and status.Parameters: - fiber_object – generally this is an object referenced in the return from fiber.create or fiber.self or fiber.find
Return: nil
Possible errors: cancel is not permitted for the specified fiber object.
Example:
tarantool> fiber.self():cancel() -- kill self, may make program end --- ... tarantool> fiber.self():cancel() --- - error: fiber is cancelled ... tarantool> fiber.self:id() --- - 163 ... tarantool> fiber.self:status() --- - dead ...
-
fiber_object.
storage
Local storage within the fiber. It is a Lua table created when it is first accessed. The storage can contain any number of named values, subject to memory limitations. Naming may be done with
fiber_object.storage.name
orfiber_object.storage['name'].
or with a numberfiber_object.storage[number]
. Values may be either numbers or strings.fiber.storage
is destroyed when the fiber is finished, regardless of how is it finished – viafiber_object:cancel()
, or the fiber’s function did ‘return’. Moreover, the storage is cleaned up even for pooled fibers used to serve IProto requests. Pooled fibers never really die, but nonetheless their storage is cleaned up after each request. That makes possible to usefiber.storage
as a full featured request-local storage.This storage may be created for a fiber, no matter how the fiber itself was created – from C or from Lua. For example, a fiber can be created in C using
fiber_new()
, then it can insert into a space, which has Luaon_replace
triggers, and one of the triggers can createfiber.storage
. That storage will be deleted when the fiber is stopped.Example:
tarantool> fiber = require('fiber') --- ... tarantool> function f () fiber.sleep(1000); end --- ... tarantool> fiber_function = fiber.create(f) --- ... tarantool> fiber_function.storage.str1 = 'string' --- ... tarantool> fiber_function.storage['str1'] --- - string ... tarantool> fiber_function:cancel() --- ... tarantool> fiber_function.storage['str1'] --- - error: '[string "return fiber_function.storage[''str1'']"]:1: the fiber is dead' ...
See also box.session.storage.
-
fiber_object:
set_joinable
(true_or_false) fiber_object:set_joinable(true)
makes a fiber joinable;fiber_object:set_joinable(false)
makes a fiber not joinable; the default is false.A joinable fiber can be waited for, with fiber_object:join().
Best practice is to call
fiber_object:set_joinable()
before the fiber function begins to execute, because otherwise the fiber could become ‘dead’ beforefiber_object:set_joinable()
takes effect. The usual sequence could be:Call
fiber.new()
instead offiber.create()
to create a new fiber_object.Do not yield at this point, because that will cause the fiber function to begin.
Call
fiber_object:set_joinable(true)
to make the new fiber_object joinable.Now it is safe to yield.
Call
fiber_object:join()
.Usually
fiber_object:join()
should be called, otherwise the fiber’s status may become ‘suspended’ when the fiber function ends, instead of ‘dead’.
Parameters: - true_or_false – the boolean value that changes the
set_joinable
flag
Return: nil
Example:
The result of the following sequence of requests is:
- the global variable
d
will be 6 (which proves that the function was not executed until afterd
was set to 1, whenfiber.sleep(1)
caused a yield); fiber.status(fi2)
will be ‘suspended’ (which proves that after the function was executed the fiber status did not change to ‘dead’).
fiber=require('fiber') d=0 function fu2() d=d+5 end fi2=fiber.new(fu2) fi2:set_joinable(true) d=1 fiber.sleep(1) print(d) fiber.status(fi2)
-
fiber_object:
join
() “Join” a joinable fiber. That is, let the fiber’s function run and wait until the fiber’s status is ‘dead’ (normally a status becomes ‘dead’ when the function execution finishes). Joining will cause a yield, therefore, if the fiber is currently in a suspended state, execution of its fiber function will resume.
This kind of waiting is more convenient than going into a loop and periodically checking the status; however, it works only if the fiber was created with fiber.new() and was made joinable with fiber_object:set_joinable().
Return: two values. The first value is boolean. If the first value is true, then the join succeeded because the fiber’s function ended normally and the second result has the return value from the fiber’s function. If the first value is false, then the join succeeded because the fiber’s function ended abnormally and the second result has the details about the error, which one can unpack in the same way that one unpacks a pcall result. Rtype: boolean +result type, or boolean + struct error Example:
The result of the following sequence of requests is:
- the first
fiber.status()
call returns ‘suspended’, - the
join()
call returns true, - the elapsed time is usually 5 seconds, and
- the second
fiber.status()
call returns ‘dead’.
This proves that the
join()
does not return until the function – which sleeps 5 seconds – is ‘dead’.fiber=require('fiber') function fu2() fiber.sleep(5) end fi2=fiber.new(fu2) fi2:set_joinable(true) start_time = os.time() fiber.status(fi2) fi2:join() print('elapsed = ' .. os.time() - start_time) fiber.status(fi2)
- the first
-
-
fiber.
time
() Return: current system time (in seconds since the epoch) as a Lua number. The time is taken from the event loop clock, which makes this call very cheap, but still useful for constructing artificial tuple keys. Rtype: number Example:
tarantool> fiber.time(), fiber.time() --- - 1448466279.2415 - 1448466279.2415 ...
-
fiber.
time64
() Return: current system time (in microseconds since the epoch) as a 64-bit integer. The time is taken from the event loop clock. Rtype: cdata Example:
tarantool> fiber.time(), fiber.time64() --- - 1448466351.2708 - 1448466351270762 ...
-
fiber.
clock
() Get the monotonic time in seconds. It is better to use
fiber.clock()
for calculating timeouts instead of fiber.time() becausefiber.time()
reports real time so it is affected by system time changes.Return: a floating-point number of seconds, representing elapsed wall-clock time since some time in the past that is guaranteed not to change during the life of the process Rtype: number Example:
tarantool> start = fiber.clock() --- ... tarantool> print(start) 248700.58805 --- ... tarantool> print(fiber.time(), fiber.time()-start) 1600785979.8291 1600537279.241 --- ...
-
fiber.
clock64
() Same as fiber.clock() but in microseconds.
Return: a number of seconds as 64-bit integer, representing elapsed wall-clock time since some time in the past that is guaranteed not to change during the life of the process Rtype: cdata
Make the function which will be associated with the fiber. This function contains an infinite loop. Each iteration of the loop adds 1 to a global variable named gvar, then goes to sleep for 2 seconds. The sleep causes an implicit fiber.yield().
tarantool> fiber = require('fiber')
tarantool> function function_x()
> gvar = 0
> while true do
> gvar = gvar + 1
> fiber.sleep(2)
> end
> end
---
...
Make a fiber, associate function_x with the fiber, and start function_x. It will immediately “detach” so it will be running independently of the caller.
tarantool> gvar = 0
tarantool> fiber_of_x = fiber.create(function_x)
---
...
Get the id of the fiber (fid), to be used in later displays.
tarantool> fid = fiber_of_x:id()
---
...
Pause for a while, while the detached function runs. Then … Display the fiber id, the fiber status, and gvar (gvar will have gone up a bit depending how long the pause lasted). The status is suspended because the fiber spends almost all its time sleeping or yielding.
tarantool> print('#', fid, '. ', fiber_of_x:status(), '. gvar=', gvar)
# 102 . suspended . gvar= 399
---
...
Pause for a while, while the detached function runs. Then … Cancel the fiber. Then, once again … Display the fiber id, the fiber status, and gvar (gvar will have gone up a bit more depending how long the pause lasted). This time the status is dead because the cancel worked.
tarantool> fiber_of_x:cancel()
---
...
tarantool> print('#', fid, '. ', fiber_of_x:status(), '. gvar=', gvar)
# 102 . dead . gvar= 421
---
...
Warning: yield() and any function which implicitly yields (such as sleep()) can fail (raise an exception).
For example, this function has a loop which repeats until cancel() happens.
The last thing that it will print is ‘before yield’, which demonstrates
that yield()
failed, the loop did not continue until testcancel() failed.
fiber = require('fiber')
function function_name()
while true do
print('before testcancel')
fiber.testcancel()
print('before yield')
fiber.yield()
end
end
fiber_object = fiber.create(function_name)
fiber.sleep(.1)
fiber_object:cancel()
Call fiber.channel()
to allocate space and get a channel object, which will
be called channel for examples in this section.
Call the other routines, via channel, to send messages, receive messages, or check channel status.
Message exchange is synchronous. The Lua garbage collector will mark or free the
channel when no one is
using it, as with any other Lua object. Use object-oriented syntax, for example
channel:put(message)
rather than fiber.channel.put(message)
.
-
fiber.
channel
([capacity]) Create a new communication channel.
Parameters: - capacity (int) – the maximum number of slots (spaces for
channel:put
messages) that can be in use at once. The default is 0.
Return: new channel.
Rtype: userdata, possibly including the string “channel …”.
- capacity (int) – the maximum number of slots (spaces for
-
object
channel_object
-
channel_object:
put
(message[, timeout]) Send a message using a channel. If the channel is full,
channel:put()
waits until there is a free slot in the channel.Parameters: - message (lua-value) – what will be sent, usually a string or number or table
- timeout (number) – maximum number of seconds to wait for a slot to become free
Return: If timeout is specified, and there is no free slot in the channel for the duration of the timeout, then the return value is
false
. If the channel is closed, then the return value isfalse
. Otherwise, the return value istrue
, indicating success.Rtype: boolean
-
channel_object:
close
() Close the channel. All waiters in the channel will stop waiting. All following
channel:get()
operations will returnnil
, and all followingchannel:put()
operations will returnfalse
.
-
channel_object:
get
([timeout]) Fetch and remove a message from a channel. If the channel is empty,
channel:get()
waits for a message.Parameters: - timeout (number) – maximum number of seconds to wait for a message
Return: If timeout is specified, and there is no message in the channel for the duration of the timeout, then the return value is
nil
. If the channel is closed, then the return value isnil
. Otherwise, the return value is the message placed on the channel bychannel:put()
.Rtype: usually string or number or table, as determined by
channel:put
-
channel_object:
is_empty
() Check whether the channel is empty (has no messages).
Return: true
if the channel is empty. Otherwisefalse
.Rtype: boolean
-
channel_object:
count
() Find out how many messages are in the channel.
Return: the number of messages. Rtype: number
-
channel_object:
is_full
() Check whether the channel is full.
Return: true
if the channel is full (the number of messages in the channel equals the number of slots so there is no room for a new message). Otherwisefalse
.Rtype: boolean
-
channel_object:
has_readers
() Check whether readers are waiting for a message because they have issued
channel:get()
and the channel is empty.Return: true
if readers are waiting. Otherwisefalse
.Rtype: boolean
-
This example should give a rough idea of what some functions for fibers should look like. It’s assumed that the functions would be referenced in fiber.create().
fiber = require('fiber')
channel = fiber.channel(10)
function consumer_fiber()
while true do
local task = channel:get()
...
end
end
function consumer2_fiber()
while true do
-- 10 seconds
local task = channel:get(10)
if task ~= nil then
...
else
-- timeout
end
end
end
function producer_fiber()
while true do
task = box.space...:select{...}
...
if channel:is_empty() then
-- channel is empty
end
if channel:is_full() then
-- channel is full
end
...
if channel:has_readers() then
-- there are some fibers
-- that are waiting for data
end
...
if channel:has_writers() then
-- there are some fibers
-- that are waiting for readers
end
channel:put(task)
end
end
function producer2_fiber()
while true do
task = box.space...select{...}
-- 10 seconds
if channel:put(task, 10) then
...
else
-- timeout
end
end
end
Call fiber.cond()
to create a named condition variable, which will be called
‘cond’ for examples in this section.
Call cond:wait()
to make a fiber wait for a signal via a condition variable.
Call cond:signal()
to send a signal to wake up a single fiber that has
executed cond:wait()
.
Call cond:broadcast()
to send a signal to all fibers that have executed
cond:wait()
.
-
object
cond_object
-
cond_object:
wait
([timeout]) Make the current fiber go to sleep, waiting until another fiber invokes the
signal()
orbroadcast()
method on the cond object. The sleep causes an implicit fiber.yield().Parameters: - timeout – number of seconds to wait, default = forever.
Return: If timeout is provided, and a signal doesn’t happen for the duration of the timeout,
wait()
returns false. If a signal or broadcast happens,wait()
returns true.Rtype: boolean
-
Assume that a tarantool instance is running and listening for connections on localhost port 3301. Assume that guest users have privileges to connect. We will use the tarantoolctl utility to start two clients.
On terminal #1, say
$ tarantoolctl connect '3301'
tarantool> fiber = require('fiber')
tarantool> cond = fiber.cond()
tarantool> cond:wait()
The job will hang because cond:wait()
– without an optional timeout
argument – will go to sleep until the condition variable changes.
On terminal #2, say
$ tarantoolctl connect '3301'
tarantool> cond:signal()
Now look again at terminal #1. It will show that the waiting stopped, and the
cond:wait()
function returned true
.
This example depended on the use of a global conditional variable with the
arbitrary name cond
. In real life, programmers would make sure to use
different conditional variable names for different applications.