Bottleneck Analysis via Grammar-based Performance Fuzzing*

Yavuz Koroglu; Franz Wotawa

doi:10.1109/ICSTW58534.2023.00041

Bottleneck Analysis via Grammar-based Performance Fuzzing^*

Yavuz Koroglu, Franz Wotawa

Graz University of Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Performance is a general quality attribute of every software system that developers always want to improve. A performance fuzzer helps developers in this task by automatically generating inputs hitting performance bottlenecks. However, a developer must still manually localize the root causes of these bottlenecks. In this study, we perform grammar-based performance fuzzing on an example System Under Test (SUT), focusing on response time for determining problematic grammar constructs with the highest likelihood of causing bottlenecks. We show that replacing these constructs creates an average of 40.53x speedup on 24 bottleneck cases out of 50. Furthermore, avoiding the problematic constructs in the input generation provides an average of 1.46x speedup. These preliminary results suggest a measurable link between grammar constructs and performance bottlenecks, opening up the possibility of high-level categorization and analysis.

Original language	English
Title of host publication	Proceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2023
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	180-185
Number of pages	6
ISBN (Electronic)	9798350333350
DOIs	https://doi.org/10.1109/ICSTW58534.2023.00041
Publication status	Published - 2023
Externally published	Yes
Event	16th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2023 - Dublin, Ireland Duration: 16 Apr 2023 → 20 Apr 2023

Publication series

Name	Proceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2023

Conference

Conference	16th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2023
Country/Territory	Ireland
City	Dublin
Period	16/04/23 → 20/04/23

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Keywords

bottleneck analysis
grammar-based fuzzing
performance testing

Access to Document

10.1109/ICSTW58534.2023.00041

Cite this

Koroglu, Y., & Wotawa, F. (2023). Bottleneck Analysis via Grammar-based Performance Fuzzing^*. In Proceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2023 (pp. 180-185). (Proceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2023). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICSTW58534.2023.00041

Koroglu, Yavuz ; Wotawa, Franz. / Bottleneck Analysis via Grammar-based Performance Fuzzing^*. Proceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2023. Institute of Electrical and Electronics Engineers Inc., 2023. pp. 180-185 (Proceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2023).

@inproceedings{af45b521d663410db87503ff1a119e70,

title = "Bottleneck Analysis via Grammar-based Performance Fuzzing*",

abstract = "Performance is a general quality attribute of every software system that developers always want to improve. A performance fuzzer helps developers in this task by automatically generating inputs hitting performance bottlenecks. However, a developer must still manually localize the root causes of these bottlenecks. In this study, we perform grammar-based performance fuzzing on an example System Under Test (SUT), focusing on response time for determining problematic grammar constructs with the highest likelihood of causing bottlenecks. We show that replacing these constructs creates an average of 40.53x speedup on 24 bottleneck cases out of 50. Furthermore, avoiding the problematic constructs in the input generation provides an average of 1.46x speedup. These preliminary results suggest a measurable link between grammar constructs and performance bottlenecks, opening up the possibility of high-level categorization and analysis.",

keywords = "bottleneck analysis, grammar-based fuzzing, performance testing",

author = "Yavuz Koroglu and Franz Wotawa",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 16th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2023 ; Conference date: 16-04-2023 Through 20-04-2023",

year = "2023",

doi = "10.1109/ICSTW58534.2023.00041",

language = "English",

series = "Proceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2023",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "180--185",

booktitle = "Proceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2023",

address = "United States",

}

Koroglu, Y & Wotawa, F 2023, Bottleneck Analysis via Grammar-based Performance Fuzzing^*. in Proceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2023. Proceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2023, Institute of Electrical and Electronics Engineers Inc., pp. 180-185, 16th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2023, Dublin, Ireland, 16/04/23. https://doi.org/10.1109/ICSTW58534.2023.00041

Bottleneck Analysis via Grammar-based Performance Fuzzing^*. / Koroglu, Yavuz; Wotawa, Franz.
Proceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2023. Institute of Electrical and Electronics Engineers Inc., 2023. p. 180-185 (Proceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2023).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Bottleneck Analysis via Grammar-based Performance Fuzzing*

AU - Koroglu, Yavuz

AU - Wotawa, Franz

PY - 2023

Y1 - 2023

N2 - Performance is a general quality attribute of every software system that developers always want to improve. A performance fuzzer helps developers in this task by automatically generating inputs hitting performance bottlenecks. However, a developer must still manually localize the root causes of these bottlenecks. In this study, we perform grammar-based performance fuzzing on an example System Under Test (SUT), focusing on response time for determining problematic grammar constructs with the highest likelihood of causing bottlenecks. We show that replacing these constructs creates an average of 40.53x speedup on 24 bottleneck cases out of 50. Furthermore, avoiding the problematic constructs in the input generation provides an average of 1.46x speedup. These preliminary results suggest a measurable link between grammar constructs and performance bottlenecks, opening up the possibility of high-level categorization and analysis.

AB - Performance is a general quality attribute of every software system that developers always want to improve. A performance fuzzer helps developers in this task by automatically generating inputs hitting performance bottlenecks. However, a developer must still manually localize the root causes of these bottlenecks. In this study, we perform grammar-based performance fuzzing on an example System Under Test (SUT), focusing on response time for determining problematic grammar constructs with the highest likelihood of causing bottlenecks. We show that replacing these constructs creates an average of 40.53x speedup on 24 bottleneck cases out of 50. Furthermore, avoiding the problematic constructs in the input generation provides an average of 1.46x speedup. These preliminary results suggest a measurable link between grammar constructs and performance bottlenecks, opening up the possibility of high-level categorization and analysis.

KW - bottleneck analysis

KW - grammar-based fuzzing

KW - performance testing

UR - https://www.scopus.com/pages/publications/85163121750

U2 - 10.1109/ICSTW58534.2023.00041

DO - 10.1109/ICSTW58534.2023.00041

M3 - Conference contribution

AN - SCOPUS:85163121750

T3 - Proceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2023

SP - 180

EP - 185

BT - Proceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2023

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 16th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2023

Y2 - 16 April 2023 through 20 April 2023

ER -

Koroglu Y, Wotawa F. Bottleneck Analysis via Grammar-based Performance Fuzzing^*. In Proceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2023. Institute of Electrical and Electronics Engineers Inc. 2023. p. 180-185. (Proceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2023). doi: 10.1109/ICSTW58534.2023.00041