Test Matix

[colin]

Hey All,

Since I have recently joined a test development group at work and am thinking a lot these days on how to break software I wondered if those who are regulars at testing Sub's new releases are interested in writing up a core set of tests to work through?

I think this would benefit GBPVR and help find problems and verify fixes. When you consider GBPVR works in all timezones, with how many different types of cards (individually used and combined), with manual, scheduled one off or season recordings with the ability to watch live TV (and record with multiple cards) and a heck of a lot of other stuff some formalised testing would be good.

If you are interested let me know,

cheers,
Colin.

Edit: Fortunately there was not a spelling test for this job, of course Matix should be Matrix

[Guest]

Colin,

I think that is a great idea and I would like to help. I am in the US and my configuration consists of two PVR-250s and two MVPs (one is wireless).

I have worked with Sub on some of his other pre-releases and I think he has done a remarkable job with GB-PVR. If a few of us can help by implementing a coordinated testing program, then everyone should benefit.

Bryan

[colin]

Hey Bryan,

great stuff. just start noting down what you do to install/configure /use GBPVR. A simple test plan would be of the format

Inputs:
Outputs:
Expected outcome:

Gets a little bit more complicated with this type of application but this general idea should apply. Then we can start putting together a plan,

cheers,
Colin.

[MaBo]

Colin,
May I add two very importants line to your suggestion:

actual outcome:

(is evident)

conditions:

The same test performed as the first one after a fresh reboot or after two days of intensive use may render completely different outcomes. The problem is what to write in this line. At least mention other programs running, whether its the first thing done after program start.

Also, please note that for proper testing your goal shouldnot be to find as many things that went correctly, but to find as many errors as possible!!

To come in the right mood, the "testing" of a Toyota Hi-Lux done by BBC's topgear might serve useful: BBC's topgear downloadpage ;-).

BTW, for a proper testplan, we should first decide on the wanted behaviour...

GReetz,
MaBo

[colin]

Hey Mabo,

Add as many suggestions as you like. My suggestion was only a starting point

And yes before we can truly test the funtionality of the software its needs to be defined. I think for some the of software we can pretty much determine this,

Cheers,
Colin.

[Guest]

Colin,

Another thought.

Maybe we should divide up our testing areas. I have multiple MVPs and MVP functionality is very important to me. So when I get a new GB-PVR release, the first thing I try is how well do the MVPs work. I'am also in the US, so I can test Zap2it, but I can't test Bleb.

So your test matrix might have tester on one axis and functionality on the other. That way we can feed Sub's TODO list even faster. Just kidding!

I also have two PVR-250s. So I'm willing to agree to test MVP functionality, multiple MVP functionality and recording or watching LiveTV from two PVR-250s at the same time.

How does that sound?

Bryan

[colin]

Hey Bryan,

good idea. It would be a bit much to expect one person to attempt to test everything.

It might be an idea to define the core functionality/behaviour which everyone will touch on, the each person can branch off in the direction of their setup

Colin.