Rails factories having multiple associations to same object

I have a model which have associations like this:-

 class Order < ApplicationRecord
  belongs_to :customer, inverse_of: :orders
  belongs_to :user, inverse_of: :orders
  belongs_to :shipping_address, class_name: "Customer::Address", inverse_of: :shipped_orders
  belongs_to :billing_address, class_name: "Customer::Address", inverse_of: :billed_orders
end

Also, customer_address have a field customer_id. My factory is like this:-

FactoryGirl.define do
  factory :order do 
    customer
    user
    association :shipping_address, factory: :customer_address, customer_id: customer.id 
    association :billing_address, factory: :customer_address, customer_id: customer.id
  end
end

But I am not able to access customer.id. I am getting this error:-

undefined method `id' for #<FactoryGirl::Declaration::Implicit:0x007fa3e6979d70>

How can I pass customer.id to shipping_address and billing_address association?