Go to content Go to navigation

Lua String Writer · 2010-02-04 16:35 by Black in

Lua strings are opaque byte streams. They are constant, and can only be manipulated by using the string api to create new strings. This can be expensive, especially when creating a string by appending new values at the end. While Lua contains optimizations for direct concatenation, successive appending has a high overhead.

This StringWriter class reduces the overhead by aggregating string concatenations in a table and executing them when requested. It was originally designed to serve as an efficient drop-in replacement for files as created by io.open, but it can also be used standalone.

The class itself is built with a protected shared metatable and state inside a table. The state itself is not protected (it would be possible by using individual metatables or an internal database in a weak table, but this is more elegant). The metatable contains entries to redirect reads to the method table, redirect new writes to nothing and prevent changing or reading the metatable. The concatenation operator is also overloaded, but since it has value semantic, and is not allowed to change the object itself, the implementation is less efficient than StringWriter:write(). Converting a StringWriter with tostring() gives the contained string, equivalent to StringWriter:get().

stringwriter.lua [4.77 kB]

  1. -- MetaTable for string writers
  2. local StringWriter_Meta = {
  3.   ["__index"] = StringWriter_Methods;
  4.   ["__newindex"] = function ()
  5.       -- Don't allow setting values
  6.     end;
  7.   ["__metatable"] = StringWriter_ID;
  8.   ["__tostring"] = StringWriter_Methods.get;
  9.   ["__concat"] = function (this, str)
  10.       str = tostring(str);
  11.       local sw = StringWriter();
  12.       sw.string_ = {};
  13.       for _, v in ipairs(this.string_) do
  14.         table.insert(sw.string_, v);
  15.       end
  16.       table.insert(sw.string_, str);
  17.       sw.len_ = this.len_ + #str;
  18.       sw.pos_ = sw.len_;
  19.       return sw;
  20.     end;
  21. }

The method table itself contains all methods the StringWriter supports. It was modeled after the file class, so many methods are placeholders that do nothing. The methods that are supported are seeking and writing. Seeking simply sets an internal position value. Writing in the context of files means overwriting and extending. When the position is at the end, the contents that are to be written can simply be appended to the contents table. Otherwise, the string has to be baked, split, and recomposed.

stringwriter.lua [4.77 kB]

  1. -- Methods for string writers
  2. local StringWriter_Methods = {
  3.   ["close"] = voidFunc;
  4.   ["flush"] = voidFunc;
  5.   ["lines"] = voidFunc;
  6.   ["read"] = voidFunc;
  7.   ["seek"] = function (this, base, offset)
  8.       -- Only act on StringWriters
  9.       if not StringWriter_Check(this) then
  10.         return nil, "Invalid StringWriter";
  11.       end;
  12.       -- Default offset
  13.       if type(base) == "number" then
  14.         offset = base; -- Not done in file, but reasonable
  15.       else
  16.         offset = offset or 0;
  17.       end
  18.       -- Set position and return it
  19.       if base == "set" then
  20.         this.pos_ = math.clamp(offset,0, this.len_);
  21.       elseif base == "end" then
  22.         this.pos_ = math.clamp(#this.string_+offset,0, this.len_);
  23.       else -- "cur"
  24.         this.pos_ = math.clamp(this.pos_+offset,0, this.len_);
  25.       end
  26.       return this.pos_;
  27.     end;
  28.   ["setvbuf"] = voidFunc;
  29.   ["write"] = function (this, ...)
  30.       -- Only act on StringWriters
  31.       if not StringWriter_Check(this) then return end;
  32.       -- Concat all arguments (assuming they are valid)
  33.       local s = table.concat({...});
  34.       -- Concat argument string with current string
  35.       if this.pos_ == -1 or this.pos_ == this.len_ then
  36.         -- Just append
  37.         table.insert(this.string_, s);
  38.       else
  39.         -- Insert, merge into a string
  40.         local sFull = table.concat(this.string_);
  41.         -- Split it up
  42.         local sLeft = string.sub(sFull, 1, this.pos_);
  43.         local sRight = string.sub(sFull, this.pos_+1+#s, -1)
  44.         -- And put it back in
  45.         this.string_ = {sLeft, s, sRight};
  46.       end
  47.       -- Update position
  48.       this.pos_ = this.pos_ + #s;
  49.       if this.pos_ > this.len_ then
  50.         this.len_ = this.pos_;
  51.       end;
  52.     end;
  53.   ["get"] = function (this)
  54.       if not StringWriter_Check(this) then
  55.         return nil, "Invalid StringWriter";
  56.       else
  57.         this.string_ = {table.concat(this.string_)};
  58.         return this.string_[1];
  59.       end;
  60.     end;
  61. }

StringWriter instances are created by a factory method. It initializes the state and sets the metatable.

stringwriter.lua [4.77 kB]

  1. -- StringWriter factory
  2. StringWriter = function ()
  3.   local sw = {
  4.     string_ = {""};
  5.     len_  = 0;
  6.     pos_  = 0;
  7.   }
  8.   setmetatable(sw, StringWriter_Meta);
  9.   return sw;
  10. end

I hope this code is useful for someone, use it as you wish, it is licensed under the MIT license.

  Textile help