Should Unit Tests test against the REST API?

For a while now I have unit tested my business logic separately from the REST Api layer.
In the integration tests I've tested the service itself against its api.

Naturally, the integration tests did not include all the edge cases that the unit tests included. It felt like an overkill and duplicate test code.

But in fact, I'm left with a whole layer that is not covered. I can't really be sure that the return value is serialized as expected and that the error codes are correct or if the parameters are deserialized as I want.

My question is, should I just forgo testing the business logic through the objects that implement them, and test through the service in order to incorporate all the cases without duplication of effort?

Notes:

• I don't mean dropping all unit tests. Complex enough units that do not touch the API layer directly should still be tested separately.
• Test runtime is not an issue here.