RSpec and Factory Girl - factory test gives me connect error port 587

I'm working with legacy code (rails) for the first time, also my first time with postgresql (which may or may not be related to the problem). The application came with no tests. I installed rspec and factory_girl_rails. I started writing some warm up tests.

Context:

I have a model Users and a model Building. A user has many buildings and a building has many users, through the UserBuilding join table. The UserBuilding model belongs to a user and a building. It only has two attributes, a user_id and a building_id.

Now, I wrote a factory for UserBuilding:

factory :user_building do
   association :user
   association :building
end

This is user_building_spec.rb:

    # spec/models/user_building_spec.rb
require 'rails_helper'

describe UserBuilding, :type => :model do

  let(:user) { build(:user) }
  let(:building) { build(:building) }
  let(:user_building) { create(:user_building, user_id: user.id, building_id: building.id) }


  it "has a valid factory" do
    expect(build(:user_building)).to be_valid
  end

  it "has a user" do
    expect(user.name).to eq("Filip")
    expect(user.id).to eq(user_building.user_id)
  end

  it "has a building" do
    expect(building.name).to eq("Test Building")
    expect(building.id).to eq(user_building.building_id)
  end
end

As you can see, when I create the user and the building, and then associate their IDs with user_building, it works fine. The "has a user" and "has a building" tests work. However, the "has a valid factory" test doesn't work, and it gives me a strange error that I can't figure out. Here it is:

Failures:

1) UserBuilding has a valid factory Failure/Error: expect(build(:user_building)).to be_valid

Errno::ECONNREFUSED:
  Connection refused - connect(2) for "" port 587

port 587 is for sending emails, what does that have to do with anything? Googling this error doesn't help because of that.

Previously, I had written the factory in a different way:

   factory :user_building do
     after(:create) do |user_building|
         user_building.user << FactoryGirl.build(:user)
         user_building.building << FactoryGirl.build(:building)
     end
   end

With this version, the "has a valid factory" test passes, I guess because there's no create, so the rest doesn't happen. Using create, I get this error (curiously, it's just this one, no mention of "building"):

Failures:

1) UserBuilding has a valid factory Failure/Error: user_building.user << FactoryGirl.build(:user)

NoMethodError:
  undefined method `user' for #<UserBuilding user_id: nil, >building_id: nil>

Makes sense, since UserBuilding doesn't have a "user" attribute, it has a user_id one. That's why I changed this for the version up in the beginning of the question.

Anyway, that port 587 error is confusing me. Notice it's not for "localhost" or anything like that. It's empty between the quotes. I use the "has a valid factory" test on other models and it works fine, so clearly this one must be due to the associations.

EDIT: I put the postgresql tag because the problem might have to do with the database, due to the use of create instead of build.