TY - CPAPER TI - Towards a Robustness Evaluation Framework for BPEL Engines AU - Harrer, Simon AU - Wirtz, Guido AU - Nizamic, Faris AU - Lazovik, Alexander PY - 2014 T2 - Proceedings of the 7th IEEE International Conference on Service-Oriented Computing and Applications (SOCA'14), Matsue, Japan, November 17-19, 2014 SP - 199--206 PB - IEEE CY - Washington, DC, USA DO - 10.1109/SOCA.2014.40 UR - https://fis.uni-bamberg.de/handle/uniba/21381 LA - eng AB - The selection of the best fitting process engine for a specific project requires the evaluation of engines according to various requirements. We focus on the non-functional requirement robustness, which is critical in production environments but hard to determine. Thus, we propose an evaluation framework to reveal important robustness criteria of process engines. In this work, we focus on message robustness, i.e., the ability to handle the receipt of invalid messages appropriately. In a case study comprising five open source BPEL engines, we determine message robustness by injecting faults into robustly designed processes as a reply to a previously sent request from an external virtual service and assert their behavior. The results show that the degree of message robustness significantly differs, hence, robustly designed processes do not necessarily lead to robust runtime behavior, the selected engines still play a major role. KW - virtual services KW - BPEL KW - process engines KW - testing KW - robustness ER -