Number of the records: 1
Revealing data leakage in protein interaction benchmarks
- 1.0585960 - ÚOCHB 2025 RIV AT eng C - Conference Paper (international conference)
Bushuiev, A. - Bushuiev, Roman - Sedlář, J. - Pluskal, Tomáš - Damborský, J. - Mazurenko, S. - Šivic, J.
Revealing data leakage in protein interaction benchmarks.
GEM-2024. ICLR 2024 Workshop on Generative and Experimental Perspectives for Biomolecular Design. Vienna: ICLR, 2024.
[ICLR 2024. International Conference on Learning Representations /12./. Vienna (AT), 07.05.2024-11.05.2024]
R&D Projects: GA MŠMT(CZ) LM2023055; GA ČR(CZ) GM21-11563M
EU Projects: European Commission(XE) 891397 - KavaTarget
Research Infrastructure: e-INFRA CZ II - 90254; RECETOX RI II - 90269
Institutional support: RVO:61388963
Keywords : protein-protein interactions * data leakage * benchmarks * data splits
OECD category: Other biological topics
https://openreview.net/forum?id=ORMXYUK5IY
In recent years, there has been remarkable progress in machine learning for protein-protein interactions. However, prior work has predominantly focused on improving learning algorithms, with less attention paid to evaluation strategies and data preparation. Here, we demonstrate that further development of machine learning methods may be hindered by the quality of existing train-test splits. Specifically, we find that commonly used splitting strategies for protein complexes, based on protein sequence or metadata similarity, introduce major data leakage. This may result in overoptimistic evaluation of generalization, as well as unfair benchmarking of the models, biased towards assessing their overfitting capacity rather than practical utility. To overcome the data leakage, we recommend constructing data splits based on 3D structural similarity of protein-protein interfaces and suggest corresponding algorithms. We believe that addressing the data leakage problem is critical for further progress in this research area.
Permanent Link: https://hdl.handle.net/11104/0353594
File Download Size Commentary Version Access 0585960.pdf 0 1.7 MB Publisher’s postprint open-access
Number of the records: 1