Aardvark and Bandicoot, Chapter 17
Tue 17 May 2016 by mskala Tags used: programming, math, philosophy, aardvark, bandicootSome Clinical Consequences of Introjection
[first chapter] | [all in this series]
Bandicoot: Friend Aardvark, I've given a lot of thought to our conversation the other day about the ontology of software engineering.
Aardvark: I had hoped you would.
B: It took me a while to see it, but I think your point about no one single test case being necessary is in fact correct.
A: I think that was actually something you said, Friend Bandicoot. I have serious concerns about what you're likely to infer from it, even though the proposition is literally true as far as it goes.
B: Well, anyway, we can surely agree that the purpose of the tests is to demonstrate that the system works.
A: I would prefer to say that the tests are to find out whether the system works. Past experience has demonstrated pretty clearly that we can't start out assuming that it does.
B: Okay. Well, anyway. I have a new version of the string parser, and it passes the test suite. Want to see?
A: Sure, why not.
B: Friend Computer?
Computer: READY
B: Friend Computer, "make check"
C: Test 2, passed.
A: What happened to Test 1?
C: Test 3, passed. Test 4, passed.
B: You said it wasn't necessary.
C: Test 5, passed. Test 6, passed.
A: No, I didn't.
C: Test 7, passed. Test 8....10%
B: Yes, you did.
C: 20%
A: No, I didn't. I said that your claim of no one single test being absolutely necessary to include, might be literally true if all features were still tested adequately by other tests.
C: 30%
B: Well, anyway, that's the situation here. Look at how all the other tests are passing!
C: 40%
A: Well, I suppose it's true that many other tests in the suite do depend on 2+2 being correct, so if it makes you happy to remove just the one that was aimed at testing 2+2 in isolation, okay.
B: Uh, yeah!
C: 50%...60%...70%...
A: Of course, since the system's adding 2+2 correctly now, as shown by the other tests, it will pass Test 1 anyway if we put that back in. So taking it out was just a pointless gesture.
C: 80%...
B: Um, about that.
C: 90%...100%. Test suite passed.
A: Great! Now I'll just take a look in the SVN log to figure out which commit will add back the--
B: No! I mean, not until we discuss it, Friend Aardvark.
A: Friend Bandicoot, is there something you don't want me to see in the SVN log?
B: Well, you can look at it, but I want to tell you first that I didn't only remove Test 1.
A: Oh. So, you gutted the test suite completely?
B: No, I mean, I did only remove Test 1--
A: Then that should be okay, and we can just put it back.
B: --everywhere it occurred.
A: Everywhere?
B: Well, yes.
A: Friend Computer, "svn log". Oh, I see what you did. You've changed every instance of addition to use different operands from the ones that were there before.
B: The test suite includes many thousands of additions. I think that's pretty good coverage.
A: But if I restore the original test suite, it's not going to work, is it?
B: ...
A: Is it going to work with the original test suite?
B: ...
A: Well?
B: No. It won't work.
A: Then it won't do.
B: But it works on many thousands of cases, and you said no one case was necessary if there were enough others!
A: You said that, and you've been trying to social-engineer me into accepting unwarranted conclusions drawn from it, ever since. That's not going to work any better than this code does. We need a module that works on every case of addition, not just the ones in a carefully stacked test suite.
B: Then all your talk of ontologies was a lie, and all the cases in the original test suite are really just undocumented required features after all!
A: No, it's... it's not just a matter of the cases in the original, or in any test suite. It's really all cases.
B: What do you mean?
A: Any two numbers. If we choose any two numbers, even two never thought of before, then it is a requirement that the module has to be able to add them.
B: That's an infinite number of requirements.
A: That's why we can't be expected to list every combination beforehand in the spec.
B: Huh. I guess it also explains why you think it's okay to keep introducing new requirements. You think all possibilities should have been covered in advance anyway, and so you've convinced yourself that they're not really new.
A: I'm glad you get that point.
B: How am I supposed to implement an infinite number of required features?
A: Put on your programming trousers, and figure it out.
B: I guess I'll have to, if that's what it takes to satisfy you.
A: I wish you could see this problem as "The module doesn't work" instead of as "The Aardvark isn't satisfied." It's not about me.
0 comments
