How can I get coveralls to ignore code from gems my gem has as dependencies?

I've made a small gem (SmsSafe), and it currently has 100% code coverage (according to SimpleCov).

However, according to Coveralls, it only has 41% coverage (report here).

The reason for that difference seems to be that coveralls is taking the code from my gem's dependencies as part of my code, and complaining that those aren't covered...

I haven't seem it do this in any other gems I've looked into, and I haven't seem any special coveralls configuration in those gems' codebases...

The way I'm calling coveralls is by having this in my Rakefile:

require 'coveralls/rake/task'
Coveralls::RakeTask.new
task :test_with_coveralls => ["test", "coveralls:push"]

and by having Travis evecute "test_with_coveralls":

script: bundle exec rake test_with_coveralls

Which is the right way of adding it according to their docs

Any ideas why this might be happening?