Ticket UUID: | 623787 | |||
Title: | programmatic access to summary data | |||
Type: | RFE | Version: | None | |
Submitter: | davidw | Created on: | 2002-10-15 22:05:20 | |
Subsystem: | 34. tcltest Package | Assigned To: | dgp | |
Priority: | 5 Medium | Severity: | ||
Status: | Open | Last Modified: | 2004-02-24 03:10:50 | |
Resolution: | None | Closed By: | ||
Closed on: | ||||
Description: |
If one is running *lots* of tests, it might be nice to have an option to display the results in a different way, like: foo: pass bar: fail baz: pass fcopy: pass and so forth. Maybe it would be better to give some sort of programmatic interface to the summary information, such as test name, result, desired result, test text, etc... that way it would be possible to write ones own ways of collecting the information. | |||
User Comments: |
dgp added on 2004-02-24 03:10:50:
File Added - 77708: 623787.patch dgp added on 2004-02-24 03:10:49: Logged In: YES user_id=80530 here's an updated patch against the current HEAD. dkf added on 2003-12-18 23:23:21: Logged In: YES user_id=79902 I approve of a programmatic way of getting test results in general and especially so if it works across processes (there are a few spots in the core test suite which could make good use of it) but I have no specific expertise to offer on how to make it happen or whether davidw's patch is a good one. davidw added on 2003-11-10 22:57:42: File Added - 67031: latest.diff.gz davidw added on 2003-11-10 22:57:41: Logged In: YES user_id=240 It wasn't ok with the previous upload. Let's try again and compress it this time. davidw added on 2003-11-10 22:53:14: Logged In: YES user_id=240 I have an initial cut at some changes. They are pretty far reaching, so before going any further, I would like to submit them for people to have a look at. Known failures: 1) One of the rename tests renames 'incr', which is used by the new version of the test suite. 2) A few of the tests access the 'testConstraints' variable directly. There are probably other things, but I'd rather find out sooner or later that some aspect of the direction I'm taking needs reworking or abbandoning. davidw added on 2003-08-30 21:12:52: Logged In: YES user_id=240 Upon looking through the code, it seems as if tcltest really was designed as an application rather than a library. 'puts', 'exit' and the like are used throughout. dgp added on 2003-06-28 00:17:47: Logged In: YES user_id=80530 Tcl Bugs 761334 and 761344 suggest other needs along these lines. Currently summary information about the test suite is printed by [cleanupTests], but the data that makes up that report is not available. Additional interfaces to access the internal values like $numTests(Failed) would allow for combining results from multiple interps (the issue in those bug reports) and would allow for reporting the same data in alternate formats, as requested here. dgp added on 2002-10-16 12:18:22: Logged In: YES user_id=80530 seems like the core of a good idea. If you can flesh it out a bit (what interface are you looking for, etc.) we can get a TIP going on it. davidw added on 2002-10-16 05:19:52: Logged In: YES user_id=240 I guess some of this is available, maybe another option to verbose is enough to keep it from giving you more information than just a plain 'fail'? |