AMTSO最新修订“动态测试”指南
发布者:cheesejust 发布时间:2010-05-31 11-37-24
反恶意软件测试标准组织AMTSO 于五月的会议中,根据当前的恶意软件以及杀毒软件的发展趋势,修订了于2008年发布的“动态测试指南”。在拟修订的指南版本中,AMTSO成员提出测试物理环境的多样化,除了在戴尔,宏基,惠普的电脑上进行测试,还需要拓展硬件环境以更贴近现实状况,同时,在测试之前,必须将所有的安装在测试机上的软件移除以控制风险。
VB100的测试总监John Hawes在“动态测试指南”中加入了对于适当的测试设置的要求:提出了测试中采用产品的多种设置,如默认设置,最佳设置等来测试产品的性能,而在动态测试中,对产品设置的调整可能会影响到测试的准确性,因此在调整测试的环节中,测试机构要特别引起重视。他还提到了在测试前完全更新产品的重要性,以及产品更新在性能测试中的特例等问题。
- Appropriate configuration For most types of test, the tester may choose to use default, out-of-the-box settings, ‘best-possible’ settings designed to measure the maximum detection levels of a solution, or the most appropriate configuration for a given environment, often specified by the solution developer. Default settings are the most common and generally most appropriate for measuring performance. However, testers should be aware of certain differences between performance and other types of testing in this regard. For many other types of testing of anti-malware solutions, such as measuring protection levels, it is appropriate for the purposes of the test to adjust some settings which will not affect the data being recorded. Most notably, many tests will require logging levels to be adjusted to ensure adequate information is gathered. For performance testing, any such adjustments to logging or other features may impact the performance of the solution under test, and testers should take such possibilities into consideration when designing tests and selecting configuration adjustment procedures. Updating of products should also be considered; for many solutions, access to external resources will be necessary to fully test the real-world performance or performance impact of the solution, and most solutions will require updating of some kind. It is normally desirable to ensure products are fully updated prior to commencing performance tests, as some solutions may initially install with minimal content which will allow them to operate more quickly than when fully updated and operational. Testers should also be aware of the potential impact of scheduled update attempts on performance tests; in some circumstances it may be appropriate to disable scheduled updates to avoid having the additional activity influence the performance measures, while in other types of tests it will be necessary to allow updates to operate normally in order to measure their contribution to the data recorded.