CircleCI或Codeship持续集成服务上的功能规格测试失败

13

我的基本功能测试在本地可以正常通过,但在CircleCI和Codeship上失败。正在失败的测试:

require 'spec_helper'

describe 'Authentication' do

  describe "sign in" do
    it "is the landing page for non-authenticated users" do
      user = create(:user)
      visit root_path

      expect( page ).to have_content 'LOG IN' # On sign-in page
      fill_in 'Email', with: user.email
      fill_in "Password", with: user.password
      click_button 'Sign in'

      expect( current_path ).to eq user_path(user)
    end
  end

  describe 'registration' do
    it "allows new users to register" do
      visit root_path

      click_link 'Sign up'
      fill_in 'Email', with: 'myfake@email.com'
      fill_in 'Password', with: 'password'
      fill_in 'Password confirmation', with: 'password'
      fill_in 'First name', with: 'John'
      fill_in 'Last name', with: 'Doe'
      click_button "Sign up"

      expect( current_path ).to include '/users/'
      expect( page ).to have_content "Welcome! You have signed up successfully."
    end
  end
end

这两个测试都在设置页面期望值的第一行失败了(分别为expect( page ).to have_content "LOG IN"click_link "Sign up"),错误提示页面HTML完全为空白:
expected to find text "LOG IN" in ""

我在CircleCI上保存了截图,它们显示了一个完全的空白页面。

这里有个有趣的事情。我尝试通过使用VNC在Circle上运行/观察spec文件来调试问题。当我a)为测试设置driver: :seleniumb)在测试之前加入1或2个sleep,并c)通过在VNC上SSH到他们的服务器后手动运行测试时,我可以看到Selenium中的测试运行(它们在VNC中打开浏览器),并且它们完美地通过了测试。

然而,在VNC之外,这些测试在两台CI服务器中始终失败。无论是否有大量的sleepdriver: :selenium。对于常规的CircleCI/Codeship服务器和他们的VCN/本地测试环境之间存在什么差异引起了这种差异,您是否有任何想法?我联系了CircleCI的人,但他们暂时束手无策。

如果相关的话,我正在运行Ruby 2.2.0、Rails 4.2、Capybara 2.4.4、Capybara-Webkit 1.4.1和Selenium-Webdriver 2.44.0

一些潜在相关的文件:

spec_helper.rb

ENV["RAILS_ENV"] = "test"

require File.expand_path("../../config/environment", __FILE__)

require "rspec/rails"
require "shoulda/matchers"
require "webmock/rspec"
require 'vcr'

Dir[Rails.root.join("spec/support/**/*.rb")].each { |file| require file }

module Features
  include Warden::Test::Helpers
  Warden.test_mode!

  def sign_in(user)
    login_as(user, scope: :user)
  end
end

module Controllers
  # Pre-parse controller responses for easy access
  def response_body
    body = JSON.parse(response.body)
    body.is_a?(Hash) ? body.to_sh : body.map(&:to_sh)
  end
end

module Mock
  def disable_webmock(&block)
    WebMock.disable!
    yield
    WebMock.enable!
  end
end

RSpec.configure do |config|
  config.expect_with :rspec do |c|
    c.syntax = :expect
  end

  # Save a screenshot to CircleCI when a feature test fails
  config.after(:each, :type => :feature) do |example|
    if example.exception
      artifact = save_page
      puts "\"#{example.description}\" failed. Page saved to #{artifact}"
    end
  end

  config.include Features, type: :feature
  config.include Controllers, type: :controller
  config.include Mock
  config.include Formulaic::Dsl, type: :feature
  config.infer_spec_type_from_file_location!
  config.infer_base_class_for_anonymous_controllers = false
  config.order = "random"
  config.use_transactional_fixtures = false
end

RSpec::Matchers.define :hash_eq do |expected|
  match do |actual|
    actual.recursive_symbolize_keys == expected.recursive_symbolize_keys
  end
end

VCR.configure do |c|
  c.cassette_library_dir = 'spec/fixtures/vcr_cassettes'
  c.hook_into :webmock
  c.allow_http_connections_when_no_cassette = true
  c.configure_rspec_metadata!
  c.ignore_hosts '127.0.0.1', 'localhost:3000'
end

ActiveRecord::Migration.maintain_test_schema!
Capybara.javascript_driver = :webkit

if ENV['CIRCLE_ARTIFACTS']
  Capybara.save_and_open_page_path = ENV['CIRCLE_ARTIFACTS']
end

WebMock.disable_net_connect!(allow_localhost: true)

database_cleaner.rb

RSpec.configure do |config|
  config.use_transactional_fixtures = false

  config.before(:suite) do
    DatabaseCleaner.strategy = :truncation
  end

  config.before(:each) do
    DatabaseCleaner.start
  end

  config.after(:each) do
    DatabaseCleaner.clean
  end

end

这可能是一个CircleCI配置问题。你能分享一下你的cicle.yml文件吗? - Jan Klimo
你是使用无头浏览器还是 Saas 服务?大多数 CI 程序都是纯粹的无头浏览器,不能像 selenium 那样运行,除非进行一些真正的工作。 - TIMBERings
如果您使用Selenium,可以尝试使用Firefox 28.0。 - Juanito Fatas
你找到解决方案了吗?我也遇到了同样的问题。 - Arnaud
2个回答

2

我可以想到几件事情可以尝试:

  • If you want to get your specs to run headless, try using poltergeist rather than capybara-webkit. poltergeist has several advantages which I described here: https://dev59.com/jmAg5IYBdhLWcg3wG3_O#24108439
  • Selenium might just be a distraction, but, if you want to get it to work, be sure that your CI environment has the right version of Firefox. Each version of selenium-webdriver seems to need a narrow range of versions of Firefox. On CircleCI, you can configure circle.yml to install a specific version of Firefox. Right now we use selenium-webdriver 2.46.2 and Firefox 39.0.3, installed with the following in circle.yml:

    dependencies:
      pre:
        - sudo apt-get update; sudo apt-get install firefox=39.0.3+build2-0ubuntu0.12.04.1
    

    I don't know about Codeship.


0

我在CircleCI上运行Selenium测试时遇到了不稳定的情况,直到我看到了这篇文章https://discuss.circleci.com/t/selenium-tests-timing-out/7765/2

添加以下内容后:

machine:
  environment:
    DBUS_SESSION_BUS_ADDRESS: /dev/null

对于我的circle.yml文件,我的测试已经变得稳定了。


网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接