jeudi 24 août 2017

Xcode crashes when running all tests, Fastlane tests crash and restart causing low code coverage

I have around 600 unit tests that I run by using CMD+U, or using the scan action in Fastlane. It seems some tests are causing a crash or similar which either stops the tests altogether (when run in Xcode) or restarted and start fresh from the next method (when run in FastLane), causing the code coverage to be considerably less than it should.

Also the tests seem really inconsistent, sometimes they pass, sometimes they don't.

Are there any useful debugging tools to see what tests could be interfering with one another, still running asynchronously or not clearing their system state correctly?

An example of the crash in Fastlane shows this error, and then restarts from the next method:

2017-08-24 11:03:23.648 xcodebuild[22051:407936] IDETestOperationsObserverDebug: Writing diagnostic log for test session to: [11:03:23]: ▸ /Users/chris.merron/Library/Developer/Xcode/DerivedData/ND_360-foqmobymnlbonedpxdhsagbuefgm/Logs/Test/9C91A406-37AC-485B-B253-E30139A5B52B/Session-ND Vision Unit Tests-2017-08-24_110323-2GcUA5.log [11:03:23]: ▸ 2017-08-24 11:03:23.648 xcodebuild[22051:385325] [MT] IDETestOperationsObserverDebug: (BFE781F0-C1D3-47AA-B386-7171B89DF4EA) Beginning test session ND Vision Unit Tests-BFE781F0-C1D3-47AA-B386-7171B89DF4EA at 2017-08-24 11:03:23.645 with Xcode 8C1002 on target { [11:03:23]: ▸ SimDevice: SimDevice : iPhone 6 (F369E845-94F6-422E-8FE1-03363CD835FC) : state={ Booted } deviceType={ SimDeviceType : com.apple.CoreSimulator.SimDeviceType.iPhone-6 } runtime={ SimRuntime : 10.2 (14C89) - com.apple.CoreSimulator.SimRuntime.iOS-10-2 } [11:03:23]: ▸ } (10.2 (14C89))

As a result of this issue above, my code coverage (via XCov) came in at 0%.

After disabling certain test classes I've noticed the crashes only seem to happen after a certain class. Perhaps there could be something in there that doesn't finish before other tests run...

Aucun commentaire:

Enregistrer un commentaire