How to execute tests

[edgargreen]

Executing tests

I am looking at the sample DB and I am trying to see how to actually execute the tests that are in the DB and I just cant see that.

I am also trying to see how to delete the results and to do a second run of the same test suite.

I see how to add testers and how to assign tests but I cant see how to run the test in order to get results (I am only using the sample DB)

[David (PassMark)]

To start with I would suggest you download and have a quick look in the Testlog Users Guide.

Test cases exist in two 'flavours' in TestLog. This is to facilitate reuse of test cases. Test cases are first created as Generic test cases. This type of test case exists outside any projects and contains no information concerned with the actual execution or results of the test. Generic test cases can then be added to any TestLog project. Once added to a project, the test case becomes a Project test case. The test case now belongs to it's parent project and is updated by testers with the results of the test.

By dragging Generic test cases into projects, all the results fields are cleared, ready for test execution.

Testlog isn't an automated test execution / scripting platform however.

[dvdtknsn]

Running test cases

I had the same concerns when I first started using TestLog. Other test case tracking systems have a concept of test runs, where you can run the same test case multiple times within the same.

With TestLog, when you run the test case again, you are effectively editing the previous test run. If there has been a change of status, this is recorded in the Test History tab. However, take care not to overwrite any of your previous test results if you want to preserve this data. What I do is to simply append my second run information to the Results Obtained box. If you use some of the fields such as Build Tested, remember that if you change this, it will be overwritten and you may lose the record of the build on which you first ran the test. So best to write this sort of information in the body of the Results Obtained so you can be sure it is preserved.

Although this is a bit of a TestLog design flaw, we find that running the same test case is fairly unusual. If we are running a test case on a different configuration, we will duplicate the test suite to a new folder (naming it appropriately) and run it there, rather than reusing an existing test suite. This also allows for easier viewing of historical test runs, as each test suite then becomes representative of a run in a point in time.

It would be nice to get a view from Passmark as to the recommended way of re-running test cases. My way works but still feels like a workaround.

David

[Mike McKinley]

running a test multiple times

Have just started using Testlog and find its very easy and intuitive to use. Had the same problem of wanting to run the same test multiple times, e.g. the test failed the first time so need to re-run a 2nd time when the problem has been fixed. An easy way to do this is to right click on the 1st test and create a copy. Then record the 2nd test results on the copy, and so on until the test finally passes. Not the ideal solution but a good work around that gives you the total picture when the test is finally passed.
Mike

[Mike McKinley]

better still, just right click and select "Duplicate"

[Vince]

Executing tests

I'm evaluating this tool for my company.
So far I like what I see but one big drawback is not being able to run the same test case during multiple runs.

For example, our testing is localisation based testing.
We perform the following test passes:
1. Run test case per language
2. Repeat test case per configuration e.g. Windows, Linux etc...

Therefore we could have 4 languages being tested and each of these across 4 different configurations.
It would be great if I could do this in TestLog without having to create multiple suites and then duplicate generic test cases into each one.
Ideally I would like to be able to create a test case and then apply custom fields to it. Then when the tester executes the case and logs a result they can simply select the lang/config that they have executed the case against.

What are my options?

Thanks,
Vince

[Tim (PassMark)]

There currently isn't a way to store multiple results with multiple configurations in the same test case. This would require a very significant change to the internal structures of test cases and TestLog in general, especially seeing there would need to be multiple status's for each test case, and multiple histories to match the the results, essentially a test case that contains test cases.

It's something that will be looked at in future but due to the nature of the changes isn't likely to be a high priority. What is more likely to be added is a way of storing multiple results for a test case (but not multiple test setups).

Continuing to use the current setup of a test case per lang/config is the best current way.