I have tried several applications to convert videos from one format to another, but to my taste HandBrake beats all the alternatives, by far: better quality, faster, smaller output, smarter (it doesn't try to upscale your videos as many apps do), and on top of all that it is free.
The only problem that I have always had with HandBrake is that it does not support batch processing on the Mac. You can queue several video conversions, which will be performed sequentially, but you can't simply drop a bunch of files and let it go. You need to add all videos one by one.
Here comes HandBrakeBatch. I had some time in the weekend, and I have put together a Cocoa applications to batch video conversions using HandBrake. I will release it shortly (most likely tomorrow) under a GPL license. The application is simple, if you want additional features don't hesitate to let me know, or simply fork the repository (available shortly on github) and drop me a pull request.