Symfony2 testing: Why should I use fixtures instead of managing data directly in test?

I'm writing some functional tests for an API in Symfony that depend on having data in the database. It seems the generally accepted method for doing this is to use Fixtures and load the fixtures before testing.

It seems pretty daunting and impractical to create a robust library of Fixture classes that are suitable for all my tests. I am using the LiipFunctionalTestBundle so I only load the Fixtures I need, but it only makes things slightly easier.

For instance, for some tests I may need 1 user to exist in the database, and other tests I may need 3. Beyond that, I may need each of those users to have slightly different properties, but it all depends on the test.

I'd really like to just create the data I need for each test on-demand as I need it. I don't want to pollute the database with any data I don't need which may be a side effect of using Fixtures.

My solution is to use the container to get access to Doctrine and set up my objects in each test before running assertions.

Is this a terrible decision for any reason which I cannot foresee? This seems like a pretty big problem and is making writing tests a pain.

Another possibility it to try and use this port of Factory Girl for PHP, but it doesn't seem to have a large following, even though Factory Girl is used so widely in the Ruby community to solve this same problem.