Touch interface as a co-author of a multimedia text narrative
DOI:
https://doi.org/10.28925/2311-259x.2026.1.13Keywords:
multimedia text, gesture semantics, digital age, artificial intelligence, mediaAbstract
The article analyzes the evolution of narrative practices in digital journalism through haptic technologies of interaction with multimedia content. The relevance of the study is determined by the proliferation of devices with touch displays and the transformation of content interaction patterns. The advancement of touch displays has opened new possibilities for ordinary users, making touch one of the essential elements of storytelling. The subject matter comprises the characteristics of tactile interfaces as elements of textual structural organization. The objective is to identify the mechanisms that enable the sensory interface to transform from a neutral intermediary into an active participant in narrative construction within the Ukrainian digital space. The research uses a methodology that integrates media linguistics, sensory interface theory, cognitive science, semiotics, and cognitive studies. A structural-functional analysis of multimedia texts from Ukrainian publications has been applied. Semiotic analysis showed the nature of gestural interaction and enabled the interpretation of specific gesture types. The case study method was used to examine interactive projects by Ukrainian media professionals in detail. The empirical foundation of the research comprises eye-tracking data and results from experiments measuring cognitive demand.
The results of the study led to the conclusion that the sensory interface has transformed from an intermediary into a participant of narrative creation. It was established that gestures acquire intuitive semantic meanings during interaction with multimedia content. The structure of multimedia texts is divided by authors into modules separated by visual elements or empty space, which facilitates convenient navigation. Ukrainian publications employ parallax technology, creating effects of depth and volume. Promising developments in this direction include more sophisticated haptics for tactile feedback with varying intensity and rhythm. The application of AR/VR technologies will enable opportunities for creating spatial narratives. The development of flexible displays in devices will require new approaches to modifying multimedia texts.
Downloads
References
Drozdova, Ye., Bodnar, P., & Kelm, N. (2022, August 18). Pid udarom. Shcho y koly obstriliuvala Rosiia [Under attack. What and when Russia shelled]. Texty.org.ua. https://texty.org.ua/projects/107440/pid-udarom-sho-i-koly-obstrilyuvala-rosiya/
Gatcho, A. R. G., Manuel, J. P. G., & Sarasua, R. J. G. (2024). Eye tracking research on readers’ interactions with multimodal texts: A mini-review. Frontiers in Communication, 9, Article 1482105. https://doi.org/10.3389/fcomm.2024.1482105
HO “Interniuz-Ukraina”. (2025, September 23). Ukrainski media: spozhyvannia novyn i dovira u 2025 rotsi [Ukrainian media: News consumption and trust in 2025]. https://internews.ua/opportunity/media_trust_consumption_2025_release
Huth, F., Koch, M., Awad-Mohammed, M., Weiskopf, D., & Kurzhals, K. (2024). Eye tracking on text reading with visual enhancements. In Proceedings of the 2024 Symposium on Eye Tracking Research and Applications (ETRA’24). https://doi.org/10.1145/3649902.3653521
Lee, S. (2024). Future trends: The evolution of storytelling in mobile apps. https://thisisglance.com/blog/future-trends-the-evolution-of-storytelling-in-mobile-apps
Lei, Y., Wang, Y., Caslin, T., Wisowaty, A., Zhu, X., Khamis, M., & Ye, J. (2023). DynamicRead: Exploring robust gaze interaction methods for reading on handheld mobile devices under dynamic conditions. Proceedings of the ACM on Human-Computer Interaction, 7(ETRA), Article 158. https://doi.org/10.1145/3591127
Mayer, A.-T., Ohme, J., Maslowska, E., & Segijn, C. M. (2024). Headlines, pictures, likes: Attention to social media newsfeed post elements on smartphones and in public. Social Media + Society, 10(2), 1–16. https://doi.org/10.1177/20563051241245666
McKenna, S., Henry Riche, N., Lee, B., Boy, J., & Meyer, M. (2017). Visual narrative flow: Exploring factors shaping data visualization story reading experiences. Computer Graphics Forum, 36(3), 377–387. https://doi.org/10.1111/cgf.13195
Mika, A., & Chu, M. (2025). Multimodal Interfaces in App Design. https://www.ramotion.com/blog/multimodal-interfaces/
Mykhailyshyn, Ya., Solodko, P., & Kulchynskyi, R. (2016). Svydovets. Anatomiia Karpat [Svydovets. Anatomy of the Carpathians]. Texty.org.ua. https://texty.org.ua/d/carpathians-3d/
Ovsianyi, K. (2023). Do i pislia. Pohliad z suputnyka na mista Ukrainy, zruinovani Rosiieiu [Before and after. A view from a satellite of Ukrainian cities destroyed by Russia]. Radio Svoboda. https://www.radiosvoboda.org/a/skhemy-rik-vtorhnennya-suputnyk/32279385.html
Pardos, Z. A., Rosenbaum, L. F., & Abrahamson, D. (2022). Characterizing learner behavior from touchscreen data. https://doi.org/10.1016/j.ijcci.2021.100357
Seyser, D., Zeiller, M. (2018). Scrollytelling — An analysis of visual storytelling in online journalism. 2018 22nd International Conference Information Visualisation (IV) (pp. 401–406). https://doi.org/10.1109/iV.2018.00075
Shchotkina, K. (2023). Dolia UPTs MP v Ukraini: doroha vid khramu [The fate of the UOC MP in Ukraine: The road from the temple]. Dzerkalo tyzhnia. https://zn.ua/project/church/
Zhang, K., Luo, J., & Pan, Y. (2025). Design and evaluation of touch and touchless interfaces for interactive digital heritage experiences. Herit. Sci, 13, 313. https://doi.org/10.1038/s40494-025-01889-3
Zozulia, Yu., Matiash, D., Matiash, Yu., Omelchenko, M., Orlyk, S., & Syvak, O. (2025). Viina Vprytul [War Up Close]. https://war.city/uk/objects/
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Dmytro Honchar
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
