Selenium无头测试在EC2实例上运行非常慢

编程问答 2022-06-03

问题描述

感谢您阅读我的文章。我有一个问题想引起你的注意。我有许多用Java编写的自动化Selenium测试。他们利用JUnit进行测试。我的任务是优化这些测试在Amazon EC2 CentOS服务器上的运行时间。这些测试是从Github存储库中提取的，并存储在EC2实例上的Docker Container中。同样，这些测试也使用Cucumber从客户端的便携式计算机通过Eclipse运行。在笔记本电脑上，测试只花费了在EC2实例上进行的一小部分时间。 EC2实例和便携式计算机具有相同的RAM量和足够的cpu处理能力来运行测试，因此我认为对EC2实例的测试所需的时间几乎不会比它们长。 EC2实例和便携式计算机都必须通过代理才能连接到Internet。我已经确保正确设置了此部分，因为我必须将HTTP_PROXY，HTTPS_PROXY和no_proxy设置为特定的IP地址。

在本节中，我将提供chromedriver在被调用时实例化的代码。我还将提供用于运行测试的确切命令。请看下面：

这是实例化时调用的方法：

public static ChromeOptions setupChromeOptions() {
    ChromeOptions options = new ChromeOptions();
    Map<String,Object> p refs = new HashMap<String,Object>();
    prefs.put("profile.default_content_setting_values.notifications",2);
    options.setHeadless(true);
    options.setProxy(null);
    options.setExperimentalOption("prefs",prefs);
    options.addArguments("--disable-dev-shm-usage");
    options.addArguments("--disable-extensions");
    options.addArguments("--disable-gpu");
    options.addArguments("--headless");
    options.addArguments("--no-proxy-server");
    options.addArguments("--no-sandBox");   
    options.addArguments("--proxy-bypass-list=*");
    options.addArguments("--proxy-server=");
    options.addArguments("--proxy-server='direct://'");
    options.addArguments("--start-maximized");  
    options.addArguments("--window-size=1280,720");
    return options;
}

以下是在EC2实例上通过Jenkins运行的命令：

#!/bin/bash
cd /home/ec2-user/Git/REPOS_NAME                                                            # Navigate to where the repos are at on the EC2.
git fetch --all
git reset --hard origin/master
git pull                                                                                    # Pull down the latest.
sudo docker container ls                                                                    # display the running containers.
sudo docker exec 0967d39b1967 bash -c 'cd /Selenium/ ; rm -rf REPO_ROOT_FOLDER/ ; exit'     # Delete the automated test folder so that it gets copied over fresh.
sudo docker cp /home/ec2-user/Git/REPO_NAME/REPO_ROOT_FOLDER 0967d39b1967:/Selenium/        # copy the latest from the Github Repository to the Docker container.
sudo docker exec 0967d39b1967 bash -c 'cd /Selenium/REPO_ROOT_FOLDER/ANOTHER_FOLDER/driver ; chmod 0777 chromedriver ; cd .. ; mvn clean test'  # Activate the Testrunner that is enabled in pom.xml. This kicks off the automated tests.

最后，这是我正在运行的版本号和操作系统详细信息：

Amazon EC2: CentOS 6.10
Docker Container OS: Ubuntu 16.04
Selenium: selenium-server-standalone-3.141.59.jar
Chromedriver: 84.0.4147.30
Google Chrome: 84.0.4147.105
Java: OpenJDK 1.8.0_252
Cucumber: 2.0.0
I cannot provide links to an HTML Page that I'm scrapping because it's all inside of a Salesforce environment.

EC2 Specs:

AWS T2.XLarge
4 vcpus
64-Bit
16 GB RAM
54 cpu Credits per hour

Laptop Specs:

Intel Core i7-8665U @ 1.90GHz,2112 MHz,4 cores,8 logocal Processors
16 GB RAM
Windows 10

Group of Tests 1:
EC2: 50 minutes
Laptop through Eclipse: 20 minutes

Group of Tests 2:
EC2: 2 hours 43 minutes
Laptop through Eclipse: 1 hour 15 minutes

两组测试的时序有两个细分。如您所见，EC2的使用时间比笔记本电脑的使用时间长得多。

最后，这是一组需要50分钟运行的测试的日志。我必须更改一些目录和路径名以保护客户端。

Started by user Scott-IM
Running as SYstem
[EnvInject] - Loading node environment variables.
Building remotely on EC2_INSTANCE in workspace /var/lib/jenkins/workspace/Run mvn test
[Run mvn test] $ /bin/bash /tmp/jenkins166122467778007381.sh
Fetching origin
FIPS mode initialized
HEAD is Now at 506f4ef Setting proxy to null for Chromeoptions
FIPS mode initialized
Already up to date.
CONTAINER ID        IMAGE               COMMAND                CREATED             STATUS              PORTS                    NAMES
0967d39b1967        IMAGE_NAME          "/Selenium/setup.sh"   4 hours ago         Up 4 hours          0.0.0.0:8080->8080/tcp   CONTAINER_NAME
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[INFO] Scanning for projects...
[INFO] 
[INFO] --------------< test:automation >--------------
[INFO] Building test-automation 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ automation ---
[INFO] Deleting /Selenium/automation/automation/target
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ automation ---
[WARNING] Using platform encoding (UTF8 actually) to copy filtered resources,i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /Selenium/automation/automation/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ automation ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ automation ---
[WARNING] Using platform encoding (UTF8 actually) to copy filtered resources,i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /Selenium/automation/automation/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ automation ---
[INFO] Changes detected - recompiling the module!
[WARNING] File encoding has not been set,using platform encoding UTF8,i.e. build is platform dependent!
[INFO] Compiling 111 source files to /Selenium/salesforce-automation/salesforce-automation/target/test-classes
[WARNING] /Selenium/automation/automation/src/test/java/com/test/mapper/OIDP_Flow/OIDP_Email_Auto_Approval_Flow_2.java:[797,128] unmappable character for encoding UTF8
[WARNING] /Selenium/automation/automation/src/test/java/com/test/mapper/OIDP_Flow/OIDP_Email_Auto_Approval_Flow_2.java:[1842,130] unmappable character for encoding UTF8
[WARNING] /Selenium/automation/automation/src/test/java/com/test/shared/ExtentReporter.java:[126,33] unmappable character for encoding UTF8
[WARNING] /Selenium/automation/automation/src/test/java/com/test/shared/ExtentReporter.java:[127,32] unmappable character for encoding UTF8
[WARNING] /Selenium/automation/automation/src/test/java/com/test/mapper/rsiaa/HD_ISO_VSC_Auto_Approval_of_Resubmitted_Si.java:[4977,95] unmappable character for encoding UTF8
[WARNING] /Selenium/automation/automation/src/test/java/com/test/pages/aao/TrackingRecord.java:[27,39] unmappable character for encoding UTF8
[WARNING] /Selenium/automation/automation/src/test/java/com/test/pages/aao/TrackingRecord.java:[117,84] unmappable character for encoding UTF8
[WARNING] /Selenium/automation/automation/src/test/java/com/test/pages/aao/TrackingRecord.java:[156,92] unmappable character for encoding UTF8
[WARNING] /Selenium/automation/automation/src/test/java/com/test/pages/aao/TrackingRecord.java:[195,84] unmappable character for encoding UTF8
[WARNING] /Selenium/automation/automation/src/test/java/restApiPOC/PrivacyRestApiConnection.java: Some input files use unchecked or unsafe operations.
[WARNING] /Selenium/automation/automation/src/test/java/restApiPOC/PrivacyRestApiConnection.java: Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-surefire-plugin:2.19:test (default-test) @ automation ---
[WARNING] The parameter forkMode is deprecated since version 2.14. Use forkCount and reuseForks instead.
 
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
Running com.test.runner.TestRunner2
Starting ChromeDriver 84.0.4147.30 (48b3e868b4cc0aa7e8149519690b6f6949e110a8-refs/branch-heads/4147@{#310}) on port 23815
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
[1604696323.888][SEVERE]: bind() Failed: Cannot assign requested address (99)
Nov 06,2020 8:58:44 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
Frame count :3
Frame count :4
Frame count :6
Newly Generated Service Item Number is :15355359
Frame count :3
Frame count :3
Clicked on new service item :15355359
Frame count :2
Frame count :2
Frame count :4
Frame count :3
Frame count :3
 
1 Scenarios (1 passed)
33 Steps (33 passed)
44m57.274s
 
Tests run: 0,Failures: 0,Errors: 0,Skipped: 0,Time elapsed: 2,698.236 sec - in com.salesforcetest.runner.TestRunner2
 
Results :
 
Tests run: 0,Skipped: 0
 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  45:05 min
[INFO] Finished at: 2020-11-06T21:43:40Z
[INFO] ------------------------------------------------------------------------
Finished: SUCCESS

如果有任何不清楚或不合理的地方，请告诉我，我将解决此问题。我在整个Internet上寻找解决方案，但还没有发现任何东西。我几乎不知所措，因为我不完全确定缓慢的确切原因是什么。如果有人可以提供一些线索或尝试的方法，我将更愿意尝试一下。

非常感谢您提供的所有帮助。

解决方法

本质上不是解决方案，但是有一些值得尝试的地方。

由于您正在比较Chrome的运行时间，因此建议您保持版本同步。在Windows上很难做到这一点，因为您通常无法控制Chrome更新的时间。这并不是说可以解决您的问题，但可以帮助消除变量。
Chrome（以及chromedriver）经历了相当多的更改，而这些更改不是同一版本的CAN和HAS对我有所帮助。

在过去的两年中，我不得不使用以下几种Chrome选项来使事情顺利进行。我注意到有3个您没有。

--disable-extensions-您永远都不知道他们在本地版本的chrome中可能拥有什么。

--dns-prefetch-disable-在较旧版本的chromedriver中，如果没有此脚本，脚本将挂起。
https://bugs.chromium.org/p/chromedriver/issues/detail?id=402#c128

--enable-features=NetworkService,NetworkServiceInProcess-冻结脚本
https://groups.google.com/forum/m/#!topic/chromedriver-users/ktp-s_0M5NM[21-40]

amazon-ec2dockerheadlessseleniumselenium-chromedriver