{"id":1264,"date":"2026-01-25T19:34:33","date_gmt":"2026-01-25T19:34:33","guid":{"rendered":"https:\/\/www.cs.ubbcluj.ro\/~meco\/evaluating-deep-learning-models-for-cross-platform-ui-component-detection-a-study-across-web-desktop-and-mobile-interfaces-2025\/"},"modified":"2026-02-01T12:06:26","modified_gmt":"2026-02-01T12:06:26","slug":"evaluating-deep-learning-models-for-cross-platform-ui-component-detection-a-study-across-web-desktop-and-mobile-interfaces-2025","status":"publish","type":"post","link":"https:\/\/www.cs.ubbcluj.ro\/~meco\/evaluating-deep-learning-models-for-cross-platform-ui-component-detection-a-study-across-web-desktop-and-mobile-interfaces-2025\/","title":{"rendered":"Evaluating Deep Learning Models for Cross-Platform UI Component Detection: A Study Across Web, Desktop, and Mobile Interfaces (2025)"},"content":{"rendered":"<div class=\"entry-content\">\n<p>Procedia Computer Science<\/p>\n<h2>Authors<\/h2>\n<p>Madalina Dicu, Camelia Chira<\/p>\n<h2>Abstract<\/h2>\n<p>User interfaces look different across web, desktop, and mobile platforms \u2014 not just in layout, but in how buttons, icons, and text appear. This makes it hard for deep learning models trained on one platform to accurately detect UI components on another. In this paper, we evaluate the cross-domain generalization of three modern object detectors \u2014 YOLOv8, YOLOv9, and Faster R-CNN \u2014 trained on one or more GUI platforms using three datasets: GENGUI (web), UICVD (desktop), and VINS (mobile). We focus on three common UI classes: Text, Button, and Icon, and compare model performance across four scenarios: in-domain training, domain adaptation, fine-tuning, and combined training. Our results show that YOLOv9 consistently delivers the best cross-domain performance, especially when fine-tuned \u2014 achieving up to 95.5% mAP when adapted from desktop to web interfaces. We also fnd that Text is the most transferable class, while Button and Icon require adaptation to new visual styles. Fine-tuning emerges as the most effective strategy for improving generalization with limited data.<\/p>\n<h2>Citation<\/h2>\n<pre class=\"wp-block-preformatted\">@Inproceedings{Dicu2025EvaluatingDL,\n author = {Madalina Dicu and Camelia Chira},\n booktitle = {Procedia Computer Science},\n title = {Evaluating Deep Learning Models for Cross-Platform UI Component Detection: A Study Across Web, Desktop, and Mobile Interfaces},\n year = {2025}\n}<\/pre>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>User interfaces look different across web, desktop, and mobile platforms \u2014 not just in layout, but in how buttons, icons, and text appear. This makes it hard for deep learning models trained on one platform to accurately detect UI components on another. In this paper, we evaluate the cross-domain generalization of three modern object detectors \u2014 YOLOv8, YOLOv9, and Faster R-CNN \u2014 trained on one or more GUI platforms using three datasets: GENGUI (web), UICVD (desktop), and VINS (mobile). We focus on three common UI classes: Text, Button, and Icon, and compare model performance across four scenarios: in-domain training, domain adaptation, fine-tuning, and combined training. Our results show that YOLOv9 consistently delivers the best cross-domain performance, especially when fine-tuned \u2014 achieving up to 95.5% mAP when adapted from desktop to web interfaces. We also fnd that Text is the most transferable class, while Button and Icon require adaptation to new visual styles. Fine-tuning emerges as the most effective strategy for improving generalization with limited data.<\/p>\n","protected":false},"author":6,"featured_media":818,"comment_status":"closed","ping_status":"","sticky":false,"template":"","format":"standard","meta":[],"categories":[4],"tags":[73,11,27,76,32],"_links":{"self":[{"href":"https:\/\/www.cs.ubbcluj.ro\/~meco\/wp-json\/wp\/v2\/posts\/1264"}],"collection":[{"href":"https:\/\/www.cs.ubbcluj.ro\/~meco\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.cs.ubbcluj.ro\/~meco\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.cs.ubbcluj.ro\/~meco\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/www.cs.ubbcluj.ro\/~meco\/wp-json\/wp\/v2\/comments?post=1264"}],"version-history":[{"count":2,"href":"https:\/\/www.cs.ubbcluj.ro\/~meco\/wp-json\/wp\/v2\/posts\/1264\/revisions"}],"predecessor-version":[{"id":1417,"href":"https:\/\/www.cs.ubbcluj.ro\/~meco\/wp-json\/wp\/v2\/posts\/1264\/revisions\/1417"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.cs.ubbcluj.ro\/~meco\/wp-json\/wp\/v2\/media\/818"}],"wp:attachment":[{"href":"https:\/\/www.cs.ubbcluj.ro\/~meco\/wp-json\/wp\/v2\/media?parent=1264"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.cs.ubbcluj.ro\/~meco\/wp-json\/wp\/v2\/categories?post=1264"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.cs.ubbcluj.ro\/~meco\/wp-json\/wp\/v2\/tags?post=1264"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}