Make simpletest report running time

In the same note, I would also love to add a "estimated time of arrival" in the Batch API progress report.

subscript.

Estimating would be very hard since each of the test cases takes very different amounts of time.

For instance the db tests take a long time, whereas the search test is virtually instant.

Total time is doable, but just depends on the implementation. Either a variable that records start time or something stored in the results table, or passed through batch API.

How about the idea from HowTo: Benchmark Drupal code?

For sure that simpletest go though most cases for each modules, but seems it can't demonstrate the real-life situation.

Another idea for brainstorming: should we create another set of simpletest class for benchmarking, which will:

1. Employ the idea of HowTo: Benchmark Drupal code
2. Create required user accounts, taxonomies, nodes, path aliases and comment on-the-fly with a given standard (let's say bind together with devel, or just implement the core programming logic of those dummy record generation within simpletest).
3. Automatically generate the benchmarking report, again, with a given standard formatting as reference.

As I remember that there is an existing contribute module which try to implement the above idea within a single click, but I am not able to remember its name...

Which could be easily done by assigning time() to a variable and just outputting the difference.

...I would also love to add a "estimated time of arrival" in the Batch API progress report.

I love this idea too, and here is the patch for it. Patch detail:

1. Count and store elapsed time within additional $current_set['elapsed'] variable.
2. Estimated complete time based on: 1. elapsed time, and 2. complete percentage. (so estimate based on statistic)
3. Add 2 new placeholder @elapsed and @estimated for customization.
4. Modify simpletest's progress_message in order to utilize above placeholders.

Screenshot as reference:

Update estimation based on RAW percentage (in floating format), so able to report estimated time even percentage is now in zero.

This will happened when we have total tasks more than 100, e.g. running all simpletest at once (132 tasks). After first task complete, percentage < 1%; if estimate with percentage in integer format (i.e. float($percentage) == 0), the estimated time will report as zero, too (which shouldn't be).

Also update with some documentation and comment.

This is a neat feature when generalized to the batch API.

My guess is that the estimation portion will be fairly inaccurate as the tests take significantly different amounts of time to run, but in general it would be neat.

Some more update:

1. Save elapsed time within $batch_set['results']['elapsed'], so able to fetch out during hook_batch_finished().
2. Estimate "remaining time", rather than "complete time" (clone idea from 7zip and MS windows). Therefore redefine @remaining meaning.
3. Use format_interval() instead of gmdate().
4. Also display elapsed time when batch process complete.
5. Other minor documentation fix.

I also give some study for "estimate simpletest running time" individually. The idea is similar as above: we need to collect enough data in order to estimate. For most ideal case, we should collect each testing class running time.

BTW, its coming with some drawback:

1. {simpletest} target for storing "pass, error or exception" message. It is not suitable to store "the time elapsed between each action".
2. Similar as above concern, we will need additional table for storing each class's time elapsed.
3. The setup() also consume some among of time (especially for database tests). Should we collect its information, too?

Finally, the logic seems to be too complicated. Implement estimation and elapsed time within batch API may benefit for more different cases :S

Screenshot as reference:

I like the idea but I suggest to change the "Remaining time" by "Estimated remaining time".

...I want to see the running time on the result page.

Is that means the running time of each test, or the total running time of all tests?

Get it. Patch reroll.

Remove the additional $batch['progress_message'] so no "Elapsed time... Remaining time..." message display on the progress bar page. "Display total running time on the result page" is already included in patch from #10, so keep it.

Also keep the ability of batch API report "Elapsed time... Remaining time..." on-the-fly, unless we specify the placeholders within $batch['progress_message'] or $batch['init_message']. It is just a souvenir of total elapsed time, and maybe useful for some elsewhere?

-1 on renaming variable $remaining to $operations : not needed, + less clear IMO

Also,

$batch_set['results']['elapsed'] = format_interval($batch_set['results']['elapsed'] / 1000);

is dangerous, because 'results' is left entirely free for batch operations to use in whichever way they see fit.
If for some reason some batch process chooses to store it's 'results' as a string, this will cause errors.

Also note that the elapsed time computed here is the effective time spent in actual batch operations, not taking drupal bootstrap into account. What you get is not the actual time spent since batch processing was launched.
This is in fact probably OK for what Dries initially requested, but it can indeed hardly be used to compute an 'estimated end time'.

This should looks better. Call "estimate remaining" as "@estimate". Add additional variable at tail for hook_batch_finished() so don't need to alter existing implementation. Most likely other should keep untouched.

I get an elapsed time once the tests complete, but the batch operation shows no timer at all. Cleared all caches, browser and site. What am I missing?

Edit: Oh, it was taken out. I liked it, personally, but hey.

The 'estimate' value is still wrong - see my comment #17.
If you do want an estimate for remaining time, you'll need to base it on (timetamp of the batch start) - time(), not on 'elapsed'.

It might indeed be useful for some batch processes where 'operations' are more similar one to another, but precisely doesn't make sense for tests batches. Since this is outside of Dries original request, this could be left to a separate patch IMO.

I'd leave estimates to a separate patch - for downloads, installing windows, anything else - they're always wrong. Like progress bars that get to 99% then hang.

Reporting total time on the results page, +1.

FYI, scripts/run-tests.sh already reports this - like a wall clock.

Patch reroll:

1. Record elapsed time based on wall clock time. Clone handling from timer_start() and timer_read().
2. Sync complete time message with scripts/run-tests.sh, as 'Test run duration: @elapsed.

due to: #74645: modify file_scan_directory to include a regex for the nomask.

Anyway... Just reroll the patch for testing.

i would really like to log/see how long a testcase needed to run,

so there could be performancetests on the testserver, before and after for certain patches

would it be possible to just write the starttime foreach tests, than we could get the difference between two testsCases and get running time

i attached a reroled patch

sure very easy

The output is "% sec", because of the use of format_interval, see the screenshot

I much prefer "The tests finished in @elapsed." I think (haven't made a patch for a while, worth double checking...) this patch reflects that change.