Unit Tests and Contracts

[paul_ray]

I have read the samples, documentation, and the wiki about unit test and contracts. I am glad the language supports these as first class citizens. I use unit test and asserts now in other languages.

Having the unit test and contracts in the program helps the next person who has to do maintenance (which could be me six months or a year later).

I understand (at least I believe I understand) contracts.

The question I have (trying to explain to a coworker), is the contract evaluated at compile time? Does it actually perform the action (calling the function and having the function run at compile time) and return a value to test?

The unit test is a separate compiler switch, correct? The unit test is done at runtime (just running the unit tests) and not at compile time, correct?

On the surface, it seems that a unit test and a contract are performing the same action. What is the real difference between the two?

[helium]

The contracts are tested when the method gets called. The "require" part tests that the method gets only valid arguments, the "ensure" part test that the method returns only expected values.

Google for "design by contract".

[todd.a]

I agree. The first class nature of tests in Cobra is a strong selling point. To me it makes developing and testing seamless and encourages you to write more tests (for me at least).

To get your test report run using the -test command

Code: Select all

cobra -test file[s]

I believe preconditions, invariants, etc. also increase the quality of your code right off the bat.

[Charles]

Tests and contracts both occur at run-time:

-- Cobra automatically runs all unit tests at the start of your program. (You can turn this off with: cobra -include-tests:no)

-- Cobra automatically asserts the conditions in "require" when a method is called. Those conditions can speak to the arguments as well as the current state of the object. Cobra automatically asserts the conditions in "ensure" when a method exits. Those conditions can speak to the return value, the current state of the object and the "old" state of the object.

Cobra does not invoke any methods or contracts at compile-time. Contract conditions are error checked as ordinary boolean expressions and evaluated only at run-time. You can turn contracts off with: cobra -contracts:none

Unit tests and contracts both provide quality control and documentation, but using two different approaches:

(1) Unit tests drive quality with examples.

(2) Contracts drive quality with specification of behavior.

For example, an "ensure" of ".count == old .count + 1" specifies the behavior of the method without providing any example of its use. The specification is very strong in that it is always invoked when the method exits whether you are running a unit test or using the method in development.

A unit test might say something like:

test
    t = List<of int>()
    assert t.count == 0
    t.add(5)
    assert t[0] == 5 and t.count == 1

Which is an example that is run once at the beginning of the program.

I feel that contracts are fundamentally stronger because they describe the method at all times--they define it--and they are checked for the life time of the process. Unit tests provide quality control when they are run, but afterwards they contribute nothing explicit.

Unit tests are possibly more instructive as people generally learn from examples better than they do from strict definitions.

And sometimes I find it hard to define my methods with boolean conditions. In those cases, unit tests can be easier.

Of course, you can have both on the same method.

[torial]

I just wanted to chime in and say two things:

1) I've written more unit tests w/ Cobra in the last two days than I have in the previous 10 years! The nice thing about Cobra is that it makes the concept very natural, and once I started doing it, I was like "Gee, this is what the hype is about!". I had been looking at NUnit, xUnit, Gallio, etc.. for a while, but never got to experience that moment. This is akin to my experience with Python -- even though I had worked in an OO .Net language for a number of years, it wasn't until I worked with Python that I understood polymorphism and also MVC.

Anyways -- what Cobra does is cool. I didn't know about the various test switches to get a test report or suppress testing. I will have to look at this report.

2) One feature I think would be a nice improvement is for the assert exception to not get thrown all the way to the application level (e.g. to the point where you get the MS Problem report that the application had an issue and do you want to report it or debug it.) If a nice way of getting it to run all the tests, and then do an Application.exit if there were exceptions would be a subtle but nice improvement.

[Charles]

In your point (2) did you mean all asserts found anywhere? Or just inside of tests?

I agree that our "test runner" needs a mode to continue tests. I find the current behavior useful for debugging, but as you build up more tests, it can get annoying.

Would you like to add a ticket? See http://cobra-language.com/trac/cobra/report/1 for examples.

[torial]

In your point (2) did you mean all asserts found anywhere? Or just inside of tests?

....

Would you like to add a ticket? See http://cobra-language.com/trac/cobra/report/1 for examples.

Just inside of tests. Asserts elsewhere I presume are just a standard exception throwing practice and if a person puts them in their run-time (non-test) code, then by all means raising them would be good.

I'll be happy to add a ticket!

[torial]

Added as ticket 195:
http://cobra-language.com/trac/cobra/ticket/195

[Charles]

Thanks. I added a question via a comment on the ticket.