The current _runner.lua expects a table containing steps to be passed
in. However, not all tests look like this. This commits adds an API to
the runner that allows tests to run however they like. They just have to
call run_direct() initially and call done() when they are finished.
Signed-off-by: Uli Schlachter <psychon@znc.in>
The first step of a test is called 20 times before a failure is
diagnosed. Later steps only get five calls to finish. I guess the idea
is that the first step sets everything up and opens all necessary
clients, while later steps then do something with these clients.
However, looking at our existing tests, they are not structured like
that. Later steps open more clients etc.
Since this limit has no negative impact on the run-time of tests, but
only means that failures are detected later, let's just remove this
limit of five calls.
Signed-off-by: Uli Schlachter <psychon@znc.in>
- add colors
- count test files
- display error summary
- filter out a_dbus_connect warnings and "Test finished successfully." message
- print current step with VERBOSE=1
- kill clients at the end of tests in an extra step
* Move table functions out of awful.util into new gears.table
* travis: Use v9999 prefix for full requests
Make sure no newly deprecated functions are used
* Move all `awful.util.table.*` calls to `gears.table.*` calls
Move table test functions from awful/util_spec to new gears/table_spec
Change awful.util.subsets call to gears.math.subsets in awful/key.lua
Before this, the test runner used a timer which fired every 0.1 seconds
to "do its thing". Many of these waits seem unnecessary.
This commit makes the test runner wait 0 seconds for the first call of a
step function. Only following calls will have a timeout of 0.1 seconds
applied.
A full run of the test suite (tests/run.sh without further arguments)
took about 100 seconds before this change. After this change, we are
down to 60 seconds. This is almost factor two faster! (Well, five thirds
is the exact number, so factor 1.66)
(The numbers are best out of three runs. The "before" number is rounded
down while the "after" number is rounded up.)
Signed-off-by: Uli Schlachter <psychon@znc.in>
When e.g. test-leaks.lua fails, it will cause a Lua error before starting the
test runner. This means that the test will just hang, because nothing causes
awesome to quit.
Handle this by starting a timer when the test runner is loaded and quitting
awesome in there if no test run was started yet. This only works if all tests
load the runner before doing anything that could fail, so the require("_runner")
is moved to the beginning in every test.
Signed-off-by: Uli Schlachter <psychon@znc.in>
Currently, an error in the default config in the right place isn't noticed. Fix
this by doing two things:
- Also grep for "error" (this catches runtime errors with a stack trace)
- Make _runner print a "success" message at the end and also grep for that
Fixes: https://github.com/awesomeWM/awesome/issues/689
Signed-off-by: Uli Schlachter <psychon@znc.in>
This tried to use awful, but didn't load awful itself. Kids, this is why you
should make your variables local!
Signed-off-by: Uli Schlachter <psychon@znc.in>
Uli Schlachter
Daniel Hahler
Daniel Hahler
Kevin Zander