Ignoring exceptions handled by other assembly

I have an application for smoke testing several key services. I want these tests to be easily written and self discovering. To this end, I have created an attribute for each method in the style of [TestMethod] from MSTest. I have also created a object that will find all these methods and execute them within a try-catch. If the method throws an exception I report as a failure, otherwise as a success.

This is all very unit test familiar and that is the intention. So a test would look like...

[MyTestAttribute]
Public void TestingTimesAhead()
{
  var d = DateTime.MaxValue.AddDays(1);
}

When ran in isolation this will cause an exception and the test fails. Perfect.

However, when I wrap the attributes and test running object up into a dll and reference from a new project, VS throws the exception because the exception is created and not handled in the solution - even though it is handled by the now external component.

I can suppress exceptions on a type by type basis, but that isn't going to work for reuse.

Does anyone know how I can either quickly configure my project to ignore all these scenarios, or better still, have my external code run as intended?