doc updates; problem with @usage with scripts; formal arguments now always take precendence; warnings cleaned up

config.ld

@@ -3,4 +3,5 @@ title='LDoc documentation'
 description='A Lua documentation tool'
 format='discount'
 file='ldoc.lua'
+dir='out'
 readme='docs/doc.md'

docs/doc.md

@@ -274,20 +274,7 @@ One added convenience is that it is easier to name entities:
 
 This is because type names (like 'function', 'module', 'table', etc) can function as tags. LDoc also provides a means to add new types (e.g. 'macro') using a configuration file which can be shipped with the source. If you become bored with typing 'param' repeatedly then you can define an alias for it, such as 'p'. This can also be specified in the configuration file.
 
-LDoc will also work with C/C++ files, since extension writers clearly have the same documentation needs as Lua module writers:
-
-    @plain
-    /***
-    Create a table with given array and hash slots.
-    @function createtable
-    @param narr initial array slots, default 0
-    @param nrec initial hash slots, default 0
-    @return the new table
-    */
-    static int l_createtable (lua_State *L) {
-    ....
-
-LDoc does not pretend to understand C/C++, so in this case it is necessary to specify the name and type explicitly.
+LDoc will also work with C/C++ files, since extension writers clearly have the same documentation needs as Lua module writers.
 
 LDoc gives the documenter the option to use Markdown to parse the contents of comments.
 
@@ -364,6 +351,7 @@ As always, explicit tags can override this behaviour if it is inappropriate.
 
 LDoc can process C/C++ files:
 
+    @plain
     /***
     Create a table with given array and hash slots.
     @function createtable
@@ -551,7 +539,7 @@ It is also very much how the Lua documentation is ordered. For instance, this co
 
 Generally, using Markdown gives you the opportunity to structure your documentation in any way you want; particularly if using lua-discount and its [table syntax](http://michelf.com/projects/php-markdown/extra/#table); the desired result can often be achieved then by using a custom style sheet.
 
-## Examples and readme files
+## Examples and Readme files
 
 It has been long known that documentation generated just from the source is not really adequate to explain _how_ to use a library.  People like reading narrative documentation, and they like looking at examples.  Previously I found myself dealing with source-generated and writer-generated documentation using different tools, and having to match these up.
 
@@ -571,7 +559,26 @@ Like all good Github projects, Winapi has a `readme.md`:
 
 This goes under the 'Topics' global section; the 'Contents' of this document is generated from the second-level (##) headings of the readme.
 
-Readme files are always processed with Markdown, but may also contain @\{} references back to the documentation and to example files. As with doc comments, a link to a standard Lua function like @\{os.execute} will work as well. Any code sections will be pretty-printed as well, unless the first indented line is '@plain'.
+Readme files are always processed with Markdown, but may also contain @\{} references back to the documentation and to example files. As with doc comments, a link to a standard Lua function like @\{os.execute} will work as well. Any code sections will be pretty-printed as Lua, unless the first indented line is '@plain'.
+
+## Tag Modifiers
+
+New with this release is the idea of _tag modifiers_. For instance, you may say @\param[type=number] and this associates the modifier `type` with value `number` with this particular param tag. A shorthand can be introduced for this common case, which is "@\tparam <type> <parmname> <comment>"; in the same way @\treturn is defined.
+
+The exact form of `<type>` is not defined, but here is a suggested scheme:
+
+    number  -- a plain type
+    Bonzo   -- a known type; a reference link will be generated
+    {string,number}  -- a 'list' tuple, built from type expressions
+    {A=string,N=number} -- a 'struct' tuple, ditto
+    {Bonzo,...}   -- an array of Bonzo objects
+    {[string]=Bonzo,...} -- a map of Bonzo objects with string keys
+
+Currently the `type` modifier is the only one known and used by LDoc when generating HTML output. However, any other modifiers are allowed and are available for use with your own templates or for extraction by your own tools.
+
+The `alias` function within configuration files has been extended so that alias tags can be defined as a tag plus a set of modifiers.  So `tparam` is defined as:
+
+    alias('tparam',{'param',modifiers={type="$1"}})
 
 ## Fields allowed in `config.ld`
 
@@ -603,6 +610,29 @@ Available functions are:
   - `add_section`
   - `new_type(tag,header,project_level)` used to add new tags, which are put in their own section `header`. They may be 'project level'.
 
+## Annotations and Searching for Tags
+
+Annotations are special tags that can be used to keep track of internal development status. The known annotations are 'todo', 'fixme' and 'warning'. They may occur in regular function/table doc comments, or on their own anywhere in the code.
+
+    --- Testing annotations
+    -- @module annot1
+    ...
+    --- first function.
+    -- @todo check if this works!
+    function annot1.first ()
+        if boo then
+
+        end
+        --- @fixme what about else?
+    end
+
+Although not currently rendered by the template as HTML, they can be extracted by the `--tags` flag, which is given a comma-separated list of tags to list.
+
+    @plain
+    D:\dev\lua\LDoc\tests> ldoc --tags todo,fixme annot1.lua
+    d:\dev\lua\ldoc\tests\annot1.lua:14: first: todo check if this works!
+    d:\dev\lua\ldoc\tests\annot1.lua:19: first-fixme1: fixme what about else?
+
 
 ## Generating HTML
 
@@ -617,7 +647,7 @@ LDoc, like LuaDoc, generates output HTML using a template, in this case `ldoc_lt
 
 This is then styled with `ldoc.css`. Currently the template and stylesheet is very much based on LuaDoc, so the results are mostly equivalent; the main change that the template has been more generalized. The default location (indicated by '!') is the directory of `ldoc.lua`.
 
-You may customize how you generate your documentation by specifying an alternative style sheet and/or template, which can be deployed with your project. The parameters are `--style` and `--template`, which give the directories where `ldoc.css` and `ldoc.ltp` are to be found. If `config.ld` contains these variables, they are interpreted slightly differently; if they are true, then it means 'use the same directory as config.ld'; otherwise they must be a valid directory relative to the ldoc invocation. An example of fully customized documentation is `tests/example/style': this is what you could call 'minimal Markdown style' where there is no attempt to tag things (except emphasizing parameter names). The narrative ought to be sufficient, if it is written appropriately.
+You may customize how you generate your documentation by specifying an alternative style sheet and/or template, which can be deployed with your project. The parameters are `--style` and `--template`, which give the directories where `ldoc.css` and `ldoc.ltp` are to be found. If `config.ld` contains these variables, they are interpreted slightly differently; if they are true, then it means 'use the same directory as config.ld'; otherwise they must be a valid directory relative to the ldoc invocation. An example of fully customized documentation is `tests/example/style`: this is what you could call 'minimal Markdown style' where there is no attempt to tag things (except emphasizing parameter names). The narrative ought to be sufficient, if it is written appropriately.
 
 Of course, there's no reason why LDoc must always generate HTML. `--ext` defines what output extension to use; this can also be set in the configuration file. So it's possible to write a template that converts LDoc output to LaTex, for instance.
 
@@ -662,4 +692,6 @@ For instance, to find all functions which don't have a @return tag:
 
 The internal naming is not always so consistent; `ret` corresponds to @return, and `params` corresponds to @param.  `item.params` is an array of the function parameters, in order; it is also a map from these names to the individual descriptions of the parameters.
 
-`item.modifiers` is a table where the keys are the tags and the values are arrays of modifier tables.  The standard tag aliases `tparam` and `treturn` attach a `type` modifier to their tags.  So
+`item.modifiers` is a table where the keys are the tags and the values are arrays of modifier tables.  The standard tag aliases `tparam` and `treturn` attach a `type` modifier to their tags.
+
+

ldoc.lua

@@ -22,7 +22,7 @@ app.require_here()
 
 --- @usage
 local usage = [[
-ldoc, a documentation generator for Lua, vs 1.0.0
+ldoc, a documentation generator for Lua, vs 1.1.0
   -d,--dir (default docs) output directory
   -o,--output  (default 'index') output name
   -v,--verbose          verbose
@@ -43,7 +43,8 @@ ldoc, a documentation generator for Lua, vs 1.0.0
   --tags (default none) show all references to given tags, comma-separated
   <file> (string) source file or directory containing source
 
-  `ldoc .` means read options from an `config.ld` file in same directory.
+  `ldoc .` reads options from an `config.ld` file in same directory;
+  `ldoc -c path/to/myconfig.ld .` reads options from `path/to/myconfig.ld`
 ]]
 local args = lapp(usage)
 

ldoc/doc.lua

@@ -259,7 +259,7 @@ function Item:set_tag (tag,value)
    elseif ttype == TAG_FLAG then
       self.tags[tag] = true
    else
-      self:warning ("unknown tag: '"..tag.."' "..tostring(ttype))
+      Item.warning(self,"unknown tag: '"..tag.."' "..tostring(ttype))
    end
 end
 
@@ -373,19 +373,27 @@ function Item:finish()
             comments:append(comment)
          end
       end
-      -- not all arguments may be commented --
+      -- not all arguments may be commented: we use the formal arguments
+      -- if available as the authoritative list, and warn if there's an inconsistency.
+      -- The _exception_ is if the formal args are just ... !
       if self.formal_args then
-         -- however, ldoc allows comments in the arg list to be used
          local fargs = self.formal_args
-         for i,name in ipairs(fargs) do
-            if params then -- explicit set of param tags
-               if names[i] ~= name then
-                  self:warning("param and formal argument name mismatch: '"..name.."' '"..tostring(names[i]).."'")
+         if #fargs ~= 1 or fargs[1] ~= '...' then
+            local pnames, pcomments = names, comments
+            names, comments = List(),List()
+            for i,name in ipairs(fargs) do
+               if params then -- explicit set of param tags
+                  if pnames[i] ~= name then
+                     if pnames[i] then
+                        self:warning("param and formal argument name mismatch: '"..name.."' '"..pnames[i].."'")
+                     else
+                        self:warning("undocumented formal argument: '"..name.."'")
+                     end
+                  end
                end
-            else
                names:append(name)
-               local comment = fargs.comments[name]
-               comments:append (comment or '')
+               -- ldoc allows comments in the formal arg list to be used
+               comments:append (fargs.comments[name] or pcomments[i] or '')
             end
          end
       end
@@ -440,10 +448,10 @@ end
 
 
 function Item:warning(msg)
-   local name = self.file and self.file.filename
-   if type(name) == 'table' then pretty.dump(name); name = '?' end
-   name = name or '?'
-   io.stderr:write(name,':',self.lineno or '1',': ',msg,'\n')
+   local file = self.file and self.file.filename
+   if type(file) == 'table' then pretty.dump(file); file = '?' end
+   file = file or '?'
+   io.stderr:write(file,':',self.lineno or '1',': ',self.name or '?',': ',msg,'\n')
    return nil
 end
 
@@ -562,7 +570,7 @@ end
 function Item:dump_tags (taglist)
    for tag, value in pairs(self.tags) do
       if not taglist or taglist[tag] then
-         Item.warning(self,self.name..' '..tag..' '..tostring(value))
+         Item.warning(self,tag..' '..tostring(value))
       end
    end
 end

ldoc/html/ldoc_ltp.lua

@@ -212,7 +212,7 @@ return [==[
 </div> <!-- id="content" -->
 </div> <!-- id="main" -->
 <div id="about">
-<i>generated by <a href="http://github.com/stevedonovan/LDoc">LDoc 1.0</a></i>
+<i>generated by <a href="http://github.com/stevedonovan/LDoc">LDoc 1.1</a></i>
 </div> <!-- id="about" -->
 </div> <!-- id="container" -->
 </body>

ldoc/lang.lua

@@ -203,8 +203,8 @@ function Lua:parse_module_modifier (tags, tok, F)
    if tags.usage then
       if tags.class ~= 'field' then return nil,"cannot deduce @usage" end
       local t1= tnext(tok)
-      local t2 = tok()
-      if t1 ~= '[' or t2 ~= '[' then return nil, 'not a long string' end
+      --local t2 = tok()
+      if t1 ~= '[' then return nil, t1..' '..': not a long string' end
       t, v = tools.grab_block_comment('',tok,'%]%]')
       return true, v, 'usage'
    elseif tags.export then

ldoc/parse.lua

@@ -114,11 +114,12 @@ local function parse_file(fname,lang, package)
    function F:warning (msg,kind,line)
       kind = kind or 'warning'
       line = line or lineno()
-      io.stderr:write(kind..' '..fname..':'..line..': '..msg,'\n')
+      io.stderr:write(fname..':'..line..': '..msg,'\n')
    end
 
    function F:error (msg)
       self:warning(msg,'error')
+      io.stderr:write('LDoc error\n')
       os.exit(1)
    end
 
@@ -208,10 +209,10 @@ local function parse_file(fname,lang, package)
                   end
                   item_follows(tags,tok)
                   local res, value, tagname = lang:parse_module_modifier(tags,tok,F)
-                  if not res then F:warning(fname,value,1); break
+                  if not res then F:warning(value); break
                   else
                      if tagname then
-                        module_item.set_tag(tagname,value)
+                        module_item:set_tag(tagname,value)
                      end
                      -- don't continue to make an item!
                      ldoc_comment = false
@@ -255,6 +256,12 @@ local function parse_file(fname,lang, package)
             if tags.name then
                current_item = F:new_item(tags,line)
                current_item.inferred = item_follows ~= nil
+               if doc.project_level(tags.class) then
+                  if module_item then
+                     F:error("Module already declared!")
+                  end
+                  module_item = current_item
+               end
             end
             if not t then break end
          end

tests/luadoc/config.ld

@@ -1,4 +1,5 @@
 project = 'LuaDoc'
 title = "LuaDoc with LDoc"
 package = 'luadoc'
-file = [[c:\users\steve\luadist\share\lua\lmod\luadoc]]
+-- point this to your Luarocks directory
+file = [[c:\lua\lua\luadoc]]