I am trying to set up automated tests for my web app on Travis CI. I am talking about automated browser tests - running unit tests is simple, but I need a process/system test that will start a headless browser and perform scripted tests of various use cases.
My application is in PHP so I decided to go with PHPUnit, Selenium and headless Firefox.
After a lot of research and trial and error I ended with following .travis.yml file:
language: php
php:
- '7.1'
services:
- mysql
addons:
firefox: latest
env:
- MOZ_HEADLESS=1
DISPLAY=:99.0
SELENIUM_FIREFOX_DRIVER=/home/travis/build/lotcz/zSample/geckodriver
before_install:
- sudo apt-get update > /dev/null
- wget https://selenium/download/url -O selenium-server.jar
- wget https://github.com/mozilla/geckodriver/releases/download/v0.21.0/geckodriver-v0.21.0-linux32.tar.gz
- tar -xzf geckodriver-v0.21.0-linux32.tar.gz
install:
- sudo apt-get install apache2
- sudo service apache2 start
- mysql -e 'CREATE DATABASE IF NOT EXISTS zsample;'
before_script:
- nohup java -jar -Dwebdriver.gecko.driver=$SELENIUM_FIREFOX_DRIVER selenium-server.jar &
- composer install
script:
- phpunit --fail-on-risky --fail-on-warning --stop-on-skipped --stop-on-incomplete --verbose --debug --colors
after_failure:
- cat nohup.out
I edited out some pieces specific to my application. Just believe me that I set my application correctly before running the test.
Now a very simple test may look something like this:
class VisitorLoginTest extends PHPUnit_Extensions_Selenium2TestCase {
public function setUp() {
$this->setHost('localhost');
$this->setPort(4444);
$this->setBrowserUrl('http://localhost');
$this->setBrowser('firefox');
}
public function tearDown() {
$this->stop();
}
public function testFrontPage() {
$this->url('/');
$content = $this->byClass('main-title')->text();
$this->assertEquals('Hello', $content);
}
}
When my test is run I get this:
The Selenium Server is not active on host localhost at port 4444.
OK, but incomplete, skipped, or risky tests!
Tests: 1, Assertions: 0, Skipped: 1.
The command "make test" exited with 0.
Now my problems are these:
- this file seems to be really long to me and too complicated given that I want to do something rather standard (I assume that automated browser tests are common these days). I would expect Travis CI to provide an easier way to perform automated browser tests. Here I have to download, install and start Apache, Selenium driver, Selenium server, use composer to get PHPUnit Selenium plugin etc...
- Selenium server doesn't seem to be running and I can't find out why.
- In the end, because PHPUnit ingeniously return 0 even when it couldn't even run the tests, Travis reports this test as successful. All those flags like
--fail-on-risky
or--stop-on-skipped
still don't force PHPUnit to report a test failure which is in my opinion the only logical result as test clearly failed.
I know this is too broad and contains multiple questions. I am afraid that I took wrong direction somewhere and I am probably trying to do something simple in a complicated way.
Can somebody provide working example of .travis.yml file for automated browser tests? My application is in PHP, but I can write tests in Node.js, Python, Java or anything else as long as tests will really work and failure will be reported if anything goes wrong.
Aucun commentaire:
Enregistrer un commentaire