Lua Table Persistence · 2010-01-27 14:56 by Black in Programming Scripts
Lua is a very flexible scripting language for embedding into programs. It’s standard API is very slim, it lacks all but basic functions. Adding them is easy though.
The persistence code here requires nothing but lua’s standard io.open for reading and writing files. It can handle loops, multiple references to the same table in both keys and values, and most standard value types.
Not supported are userdata, threads and many types of functions. Exporting simple lua functions works, but the exported byte code is not portable. The result from the export is itself lua code, it can be executed and returns data structures equivalent to those that were exported.
The core for the export is a simple recursion with a dispatcher method and writers for all types. When unsupported types are encountered, nil is written. This can cause problems on import when those unsupported values are used as table keys, but in most cases it is more desirable than to fail the export.
persistence.lua [5.50 kB]
- -- Format items for the purpose of restoring
- writers = {
- ["nil"] = function (file, item)
- file:write("nil");
- end;
- ["number"] = function (file, item)
- file:write(tostring(item));
- end;
- ["string"] = function (file, item)
- file:write(string.format("%q", item));
- end;
- ["boolean"] = function (file, item)
- if item then
- file:write("true");
- else
- file:write("false");
- end
- end;
- ["table"] = function (file, item, level, objRefNames)
- local refIdx = objRefNames[item];
- if refIdx then
- -- Table with multiple references
- file:write("multiRefObjects["..refIdx.."]");
- else
- -- Single use table
- file:write("{\n");
- for k, v in pairs(item) do
- writeIndent(file, level+1);
- file:write("[");
- write(file, k, level+1, objRefNames);
- file:write("] = ");
- write(file, v, level+1, objRefNames);
- file:write(";\n");
- end
- writeIndent(file, level);
- file:write("}");
- end;
- end;
- ["function"] = function (file, item)
- -- Does only work for "normal" functions, not those
- -- with upvalues or c functions
- local dInfo = debug.getinfo(item, "uS");
- if dInfo.nups > 0 then
- file:write("nil --[[functions with upvalue not supported]]");
- elseif dInfo.what ~= "Lua" then
- file:write("nil --[[non-lua function not supported]]");
- else
- local r, s = pcall(string.dump,item);
- if r then
- file:write(string.format("loadstring(%q)", s));
- else
- file:write("nil --[[function could not be dumped]]");
- end
- end
- end;
- ["thread"] = function (file, item)
- file:write("nil --[[thread]]\n");
- end;
- ["userdata"] = function (file, item)
- file:write("nil --[[userdata]]\n");
- end;
- }
To be able to export tables that are referenced several times (be it a cycle in the data structure, or just one that is inserted several times), the structures that are to be written are examined first and the numbers or references to each table are counted.
All tables that have multiple references to them are created at the start in the export file before they are filled with content. This is required, since they could contain themselves or other multi-ref tables.
After all those temporary tables are created, they are filled with content. The writer for tables uses a lookup table for multi-ref tables, instead of creating the table constructor for them, they are assigned from the table created at the start. Last but not least, the passed arguments themselves are created in the same way.
persistence.lua [5.50 kB]
- store = function (path, ...)
- local file, e;
- if type(path) == "string" then
- -- Path, open a file
- file, e = io.open(path, "w");
- if not file then
- return error(e);
- end
- else
- -- Just treat it as file
- file = path;
- end
- local n = select("#", ...);
- -- Count references
- local objRefCount = {}; -- Stores reference that will be exported
- for i = 1, n do
- refCount(objRefCount, (select(i,...)));
- end;
- -- Export Objects with more than one ref and assign name
- -- First, create empty tables for each
- local objRefNames = {};
- local objRefIdx = 0;
- file:write("-- Persistent Data\n");
- file:write("local multiRefObjects = {\n");
- for obj, count in pairs(objRefCount) do
- if count > 1 then
- objRefIdx = objRefIdx + 1;
- objRefNames[obj] = objRefIdx;
- file:write("{};"); -- table objRefIdx
- end;
- end;
- file:write("\n} -- multiRefObjects\n");
- -- Then fill them (this requires all empty multiRefObjects to exist)
- for obj, idx in pairs(objRefNames) do
- for k, v in pairs(obj) do
- file:write("multiRefObjects["..idx.."][");
- write(file, k, 0, objRefNames);
- file:write("] = ");
- write(file, v, 0, objRefNames);
- file:write(";\n");
- end;
- end;
- -- Create the remaining objects
- for i = 1, n do
- file:write("local ".."obj"..i.." = ");
- write(file, (select(i,...)), 0, objRefNames);
- file:write("\n");
- end
- -- Return them
- if n > 0 then
- file:write("return obj1");
- for i = 2, n do
- file:write(" ,obj"..i);
- end;
- file:write("\n");
- else
- file:write("return\n");
- end;
- file:close();
- end;
Loading the exported data is simple, but the provided method performs some error checking.
persistence.lua [5.50 kB]
- load = function (path)
- local f, e = loadfile(path);
- if f then
- return f();
- else
- return nil, e;
- end;
- end;
I hope this code is useful for someone, use it as you wish, it is licensed under the MIT license.
thanks for yr excellent work. i made this of it: git://github.com/hipe/lua-table-persistence.git
as of release 0.0.2 it’s yr work plus some unit tests, README, etc. I might make it a luarock. i might change its api.
i hope to make it part of an orbit (mvc) session data solution.
thx again!