Fix decimal values in vinyl spaces when upgrading to 2.10.1
This is an upgrade guide for fixing one specific problem which could happen with decimal values in vinyl spaces. It’s only relevant when you’re upgrading from Tarantool version <= 2.10.0 to anything >= 2.10.1.
Before gh-6377 was fixed, decimal
and double
values in a scalar
or number
index
could end up in the wrong order after the update.
If such an index has been built for a space that uses the vinyl
storage engine,
the index is persisted and is not rebuilt even after the upgrade.
If this is the case, the user has to rebuild the affected indexes manually.
Here are the rules to determine whether your installation was affected.
If all of the statements listed below are true, you have to rebuild indexes for the affected vinyl
spaces manually.
- You were running Tarantool version 2.10.0 and below.
- You have spaces with the
vinyl
storage engine. - The
vinyl
spaces havenumber
orscalar
indexes. - The tuples in these spaces may contain both
decimal
anddouble
Inf
orNaN
values.
If this is the case for you, you can run the following script, which will find all the affected indices:
local fiber = require('fiber')
local decimal = require('decimal')
local function isnan(val)
return type(val) == 'number' and val ~= val
end
local function isinf(val)
return val == math.huge or val == -math.huge
end
local function vinyl(id)
return box.space[id].engine == 'vinyl'
end
require_rebuild = {}
local iters = 0
for _, v in box.space._index:pairs({512, 0}, {iterator='GE'}) do
local id = v[1]
iters = iters + 1
if iters % 1000 == 0 then
fiber.yield()
end
if vinyl(id) then
local format = v[6]
local check_fields = {}
for _, fmt in pairs(v[6]) do
if fmt[2] == 'number' or fmt[2] == 'scalar' then
table.insert(check_fields, fmt[1] + 1)
end
end
local have_decimal = {}
local have_nan = {}
if #check_fields > 0 then
for k, tuple in box.space[id]:pairs() do
for _, i in pairs(check_fields) do
iters = iters + 1
if iters % 1000 == 0 then
fiber.yield()
end
have_decimal[i] = have_decimal[i] or
decimal.is_decimal(tuple[i])
have_nan[i] = have_nan[i] or isnan(tuple[i]) or
isinf(tuple[i])
if have_decimal[i] and have_nan[i] then
table.insert(require_rebuild, v)
goto out
end
end
end
end
end
::out::
end
The indices requiring a rebuild will be stored in the require_rebuild
table.
If the table is empty, you’re safe and can continue using Tarantool as before.
If the require_rebuild
table contains some entries,
you can rebuild the affected indices with the following script.
Примечание
Please run the script below only on the master node and only after all the nodes are upgraded to the new Tarantool version.
local log = require('log')
local function rebuild_index(idx)
local index_name = idx[3]
local space_name = box.space[idx[1]].name
log.info("Rebuilding index %s on space %s", index_name, space_name)
if (idx[2] == 0) then
log.error("Cannot rebuild primary index %s on space %s. Please, "..
"recreate the space manually", index_name, space_name)
return
end
log.info("Deleting index %s on space %s", index_name, space_name)
local v = box.space._index:delete{idx[1], idx[2]}
if v == nil then
log.error("Couldn't find index %s on space %s", index_name, space_name)
return
end
log.info("Done")
log.info("Creating index %s on space %s", index_name, space_name)
box.space._index:insert(v)
end
for _, idx in pairs(require_rebuild) do
rebuild_index(idx)
end
The script might fail on some of the indices with the following error: «Cannot rebuild primary index index_name on space space_name. Please, recreate the space manually». If this happens, automatic index rebuild is impossible, and you have to manually re-create the space to ensure data integrity:
- Create a new space with the same format as the existing one.
- Define the same indices on the freshly created space.
- Iterate over the old space’s primary key and insert all the data into the new space.
- Drop the old space.