laravel testing with data provider not working with refreshDatabase

I have the following tests and validation rules:

/** @var \App\Http\Requests\StorePopularClientRequest*/
private $rules;

/** @var \Illuminate\Validation\Validator */
private $validator;

public function __construct($name = null, array $data = array(), $dataName = '') {
    parent::__construct($name, $data, $dataName);

    $this->createApplication();
}

public function setUp(): void
{
    parent::setUp();
    $this->adminUser = $this->createUser();
    $this->businessUser = $this->createUser('Business');

    $this->popularClients = factory(PopularClient::class, 20)->create()->each(function($client) {
        static $order = 1;
        $client->order = $order;
        $client->save();
        $order++;
    });

    $this->validator = app()->get('validator');

    $this->rules = (new StorePopularClientRequest())->rules();
}

public function validationProvider()
{
    /* WithFaker trait doesn't work in the dataProvider */
    $faker = Factory::create(Factory::DEFAULT_LOCALE);
    $file = factory(Attachment::class)->create(['category' => 'logo']);
    return [
        'request_should_pass_when_data_is_provided' => [
            'passed' => true,
            'data' => [
                'name' => $faker->company(),
                'description' => $faker->sentence($nbWords = 20, $variableNbWords = true),
                'order' => 1,
                'file' => [
                    "id" => $file->id,
                    "name" => $file->name,
                ]
            ]
        ]
    ];
}

/**
 * @test
 * @dataProvider validationProvider
 * @param bool $shouldPass
 * @param array $mockedRequestData
 */
public function validation_results_as_expected($shouldPass, $mockedRequestData)
{
    $this->assertEquals(
        $shouldPass,
        $this->validate($mockedRequestData)
    );
}

protected function validate($mockedRequestData)
{
    return $this->validator
        ->make($mockedRequestData, $this->rules)
        ->passes();
}

And inside StorePopularClientRequest I have the following rules

public function rules()
{
    return [
        "name" => [
            'required','string', 
        ],
        "description" => [
            'required', 'string', 'min:5',
        ],
        'order' => ['required', 'integer'],
        'file' => ['required', 'array'],
        'file.id' => ['required', 'exists:attachments,id'],
        'file.name' => ['required', 'exists:attachments,name'],
    ];
}

All of those rules working properly except exists:attachments, name, exists:attachments,id, because of using RefreshDatabase, I if comment RefreshDatabase it works. So is there any way to not refresh the database before all tests finished, and then refresh the database?

Now the attachment factory works, and create attachment data in attachments table, but when I run the test, attachments table is refreshed before test, and the exists rule for attachment is invalid