How can I regression/feature test the inclusion of non-python files in a python package?

I am creating a command line application which includes non-python files such as yaml config files. I would like to distribute these as part of the python package and therefore would like to ensure that they are part of the installed package. I would like to use feature tests for this.

I am having a problem where I run my feature tests against the tox installed sdist of the package and all the feature tests pass, including those that require the config files. However when I install the package on to a server I get errors that it can't find the config files. I think know how to fix these problems, but what I am specifically asking for is help with testing that these config files are installed and present etc. when running my feature tests, as currently this appears to be not the case.