Test works when debugging using VS, throws System.IO.IOException when using mstest.exe

I noticed a strange behaviour during automated execution of regression tests. They fail when I execute them with mstest.exe /test:XYZ /... but they work fine when I debug them in Visual Studio (2015 Enterprise). Reason:

Test method XYZ threw exception: 
System.IO.IOException:  not found as file or resource.

The test creates some files and folders and I try to read a newly created file. As I just pointed out: It works fine if I use Visual Studio to run the test - when using mstest.exe it doesn't work.

Any idea why?