AR-assisted children book for smart teaching and learning of Turkish alphabets

Abstract

Background

Augmented reality (AR), virtual reality (VR), and remote-controlled devices are driving the need for a better 5G infrastructure to support faster data transmission. In this study, mobile AR is emphasized as a viable and widespread solution that can be easily scaled to millions of end-users and educators because it is lightweight and low-cost and can be implemented in a cross-platform manner. Low-efficiency smart devices and high latencies for real-time interactions via regular mobile networks are primary barriers to the use of AR in education. New 5G cellular networks can mitigate some of these issues via network slicing, device-to-device communication, and mobile edge computing.

Methods

In this study, we use a new technology to solve some of these problems. The proposed software monitors image targets on a printed book and renders 3D objects and alphabetic models. In addition, the application considers phonetics. The sound (phonetic) and 3D representation of a letter are played as soon as the image target is detected. 3D models of the Turkish alphabet are created by using Adobe Photoshop with Unity3D and Vuforia SDK.

Results

The proposed application teaches Turkish alphabets and phonetics by using 3D object models, 3D letters, and 3D phrases, including letters and sounds.

Keyword

Human computer interaction ; 5G ; Augmented reality ; IOT ; Turkish alphabet ; Smart teaching ; AR assisted book

Cite this article

Ahmed L ALYOUSIFY, Ramadhan J MSTAFA. AR-assisted children book for smart teaching and learning of Turkish alphabets. Virtual Reality & Intelligent Hardware, 2022, 4(3): 263-277 DOI:10.1016/j.vrih.2022.05.002

References

1. Rahmat R F, Akbar F, Syahputra M F, Budiman M A, Hizriadi A. An interactive augmented reality implementation of hijaiyah alphabet for children education. Journal of Physics: Conference Series, 2018, 978: 012102 DOI:10.1088/1742-6596/978/1/012102

2. Zhao X, Tang F, Wu Y. Real-time human segmentation by BowtieNet and a SLAM-based human AR system. Virtual Reality & Intelligent Hardware, 2019, 1(5): 511–524

3. Yin J H, Chng C B, Wong P M, Ho N, Chua M, Chui C K. VR and AR in human performance research―An NUS experience. Virtual Reality & Intelligent Hardware, 2020, 2(5): 381–393 DOI:10.1016/j.vrih.2020.07.009

4. Stull A T, Gainer M J, Hegarty M. Learning by enacting: the role of embodiment in chemistry education. Learning and Instruction, 2018, 55: 80–92 DOI:10.1016/j.learninstruc.2017.09.008

5. Harley J M, Lajoie S P, Tressel T, Jarrell A. Fostering positive emotions and history knowledge with location-based augmented reality and tour-guide prompts. Learning and Instruction, 2020, 70: 101163 DOI:10.1016/j.learninstruc.2018.09.001

6. Yip J, Wong S H, Yick K L, Chan K, Wong K H. Improving quality of teaching and learning in classes by using augmented reality video. Computers & Education, 2019, 128: 88–101 DOI:10.1016/j.compedu.2018.09.014

7. Chang S C, Hwang G J. Impacts of an augmented reality-based flipped learning guiding approach on students’ scientific project performance and perceptions. Computers & Education, 2018, 125: 226–239 DOI:10.1016/j.compedu.2018.06.007

8. Zhou X, Tang L Y, Lin D, Han W. Virtual & augmented reality for biological microscope in experiment education. Virtual Reality & Intelligent Hardware, 2020, 2(4): 316–329 DOI:10.1016/j.vrih.2020.07.004

9. Biró K, Molnár G, Pap D, Szűts Z. The effects of virtual and augmented learning environments on the learning process in secondary school. In: 2017 8th IEEE International Conference on Cognitive Infocommunications (CogInfoCom). Debrecen, Hungary, IEEE, 2017, 371–376 DOI:10.1109/coginfocom.2017.8268273

10. Li J Y, Yang B B, Chen D P, Wang N, Zhang G F, Bao H J. Survey and evaluation of monocular visual-inertial SLAM algorithms for augmented reality. Virtual Reality & Intelligent Hardware, 2019, 1(4): 386–410 DOI:10.1016/j.vrih.2019.07.002

11. Cheng Y. Edge caching and computing in 5G for mobile augmented reality and haptic internet. Computer Communications, 2020, 158, 24–31

12. Milgram P, Kishino F. A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems, 1994, 77(12), 1321–1329

13. Gil K, Rhim J, Ha T, Doh Y Y, Woo W. AR Petite Theater: augmented reality storybook for supporting children's empathy behavior. In: 2014 IEEE International Symposium on Mixed and Augmented Reality-Media, Art, Social Science, Humanities and Design. Munich, Germany, IEEE, 2014,13–20 DOI:10.1109/ismar-amh.2014.6935433

14. Al-Ali H, Bazzaza M W, Zemerly M J, Ng J W P. MyVision AIR: an augmented interactive reality book mobile application. In: 2016 IEEE Global Engineering Education Conference. Abu Dhabi, United Arab Emirates, IEEE, 2016, 741–745 DOI:10.1109/educon.2016.7474634

15. Zainuddin N, Idrus R M. The use of augmented reality enhanced flashcards for Arabic vocabulary acquisition. In: 2016 13th Learning and Technology Conference (L&T). Jeddah, Saudi Arabia, IEEE, 2016, 1–5 DOI:10.1109/lt.2016.7562857

16. Dalim C S C, Piumsomboon T, Dey A, Billinghurst M, Sunar S. TeachAR: an interactive augmented reality tool for teaching basic English to non-native children. In: 2016 IEEE International Symposium on Mixed and Augmented Reality. Merida, Mexico, IEEE, 2016, 344–345 DOI:10.1109/ismar-adjunct.2016.0113

17. Majid N A A, Yunus F, Arshad H, Johari M F. Development framework based on mobile augmented reality for pre-literacy kit. World Academy of Science, Engineering and Technology, International Journal of Social, Behavioral, Educational, Economic, Business and Industrial Engineering, 2016, 10: 2915–2918

18. Hsu T C. Learning English with augmented reality: do learning styles matter? Computers & Education, 2017, 106: 137–149 DOI:10.1016/j.compedu.2016.12.007

19. Hashim N C, Majid N A A, Arshad H, Nizam S S M, Putra H M. Mobile augmented reality application for early Arabic language education: ARabic. In: 2017 8th International Conference on Information Technology (ICIT). Amman, Jordan, IEEE, 2017, 761–766 DOI:10.1109/icitech.2017.8079942

20. Lee L K, Chau C H, Chau C H, Ng C T. Using augmented reality to teach kindergarten students English vocabulary. In: 2017 International Symposium on Educational Technology (ISET). Hong Kong, China, IEEE, 2017, 53–57 DOI:10.1109/iset.2017.20

21. Sidi J, Yee L F, Chai W Y. Interactive English Phonics Learning for Kindergarten Consonant-Vowel-Consonant (CVC) Word Using Augmented Reality. Journal of Telecommunication, Electronic and Computer Engineering (JTEC), 2017, 9(3-11), 85–91

22. Zainuddin N, Sahrir M S, Idrus R M, Najib M. Scaffolding a conceptual support for personalized Arabic vocabulary learning using augmented reality (AR) enhanced flashcards. 2016

23. MacCallum K, Kumar M. Exploring ARMobile in early childhood literacy learning. In Proceedings: Flexible Learning Association (FLANZ) 2018 Conference, Palmerston North, New Zealand, 2018, 137–142

24. Cieza E, Lujan D. Educational mobile application of augmented reality based on markers to improve the learning of vowel usage and numbers for children of a kindergarten in Trujillo. Procedia Computer Science, 2018, 130: 352–358 DOI:10.1016/j.procs.2018.04.051

25. KMuhammad Afnan, Khan N, Lee M Y, Sajjad M. School of the future: a comprehensive study on the effectiveness of augmented reality as a tool for primary school children's education. Applied Sciences, 2021, 11(11): 5277

26. Amin D, Govilkar S. Comparative study of augmented reality sdk's. International Journal on Computational Science & Applications, 2015, 5(1): 11–26 DOI:10.5121/ijcsa.2015.5102

27. DigitalPromise. 2021. Types of AR-Digital Promise. 2021

28. Romilly M. 12-best-augmented-reality-sdks, dzone.com, 2019

29. Jack G. Vuforia SDK+GRAVITY Jack′s browsar code stack is an augmented reality developer′s dream, gravityjack.com, 2013

30. O'Donnell B, 2021. Columnist, Accessed, 2021

2. Jun-Hao YIN, Chin-Boon CHNG, Pooi-Mun WONG, Nicholas HO, Matthew CHUA, Chee-Kong CHUI, VR and AR in human performance research―An NUS experience Virtual Reality & Intelligent Hardware 2020, 2(5): 381-393

3. Jiaxin LIU, Hongxin ZHANG, Chuankang LI, COMTIS: Customizable touchless interaction system for large screen visualization Virtual Reality & Intelligent Hardware 2020, 2(2): 162-174

4. Carlos ARCE-LOPERA, María José ARIAS, Gustavo CORRALES, Training birdsong recognition using virtual reality Virtual Reality & Intelligent Hardware 2021, 3(5): 397-406

5. Vivian GÓMEZ, Kelly PEÑARANDA, Pablo FIGUEROA, Lessons learned from requirements gathering for virtual reality simulators Virtual Reality & Intelligent Hardware 2021, 3(5): 407-422

6. Mengting XIAO, Zhiquan FENG, Xiaohui YANG, Tao XU, Qingbei GUO, Multimodal interaction design and application in augmented reality for chemical experiment Virtual Reality & Intelligent Hardware 2020, 2(4): 291-304

7. Mingxuan CHEN, Ping ZHANG, Zebo WU, Xiaodan CHEN, A multichannel human-swarm robot interaction system in augmented reality Virtual Reality & Intelligent Hardware 2020, 2(6): 518-533

8. Yonghang TAI, Junsheng SHI, Junjun PAN, Aimin HAO, Victor CHANG, Augmented reality-based visual-haptic modeling for thoracoscopic surgery training systems Virtual Reality & Intelligent Hardware 2021, 3(4): 274-286

9. Yukang YAN, Xin YI, Chun YU, Yuanchun SHI, Gesture-based target acquisition in virtual and augmented reality Virtual Reality & Intelligent Hardware 2019, 1(3): 276-289

10. Yuan GAO, Le XIE, A review on the application of augmented reality in craniomaxillofacial surgery Virtual Reality & Intelligent Hardware 2019, 1(1): 113-120

11. Jinyu LI, Bangbang YANG, Danpeng CHEN, Nan WANG, Guofeng ZHANG, Hujun BAO, Survey and evaluation of monocular visual-inertial SLAM algorithms for augmented reality Virtual Reality & Intelligent Hardware 2019, 1(4): 386-410

12. Xiaomei ZHAO, Fulin TANG, Yihong WU, Real-time human segmentation by BowtieNet and a SLAM-based human AR system Virtual Reality & Intelligent Hardware 2019, 1(5): 511-524

13. Chan QIU, Shien ZHOU, Zhenyu LIU, Qi GAO, Jianrong TAN, Digital assembly technology based on augmented reality and digital twins: a review Virtual Reality & Intelligent Hardware 2019, 1(6): 597-610

14. Wang LI, Junfeng WANG, Sichen JIAO, Meng WANG, Shiqi LI, Research on the visual elements of augmented reality assembly processes Virtual Reality & Intelligent Hardware 2019, 1(6): 622-634

15. Pengfei HAN, Gang ZHAO, A review of edge-based 3D tracking of rigid objects Virtual Reality & Intelligent Hardware 2019, 1(6): 580-596

16. Shenze WANG, Kaikai DU, Ningfang SONG, Dongfeng ZHAO, Di FENG, Zhengqian TU, Study on the adaptability of augmented reality smartglasses for astigmatism based on holographic waveguide grating Virtual Reality & Intelligent Hardware 2020, 2(1): 79-85

17. Lingfei ZHU, Qi CAO, Yiyu CAI, Development of augmented reality serious games with a vibrotactile feedback jacket Virtual Reality & Intelligent Hardware 2020, 2(5): 454-470

18. Xiang ZHOU, Liyu TANG, Ding LIN, Wei HAN, Virtual & augmented reality for biological microscope in experiment education Virtual Reality & Intelligent Hardware 2020, 2(4): 316-329

19. Jie REN, Chun YU, Yueting WENG, Chengchi ZHOU, Yuanchun SHI, Design and evaluation of window management operations in AR headset+smartphone interface Virtual Reality & Intelligent Hardware 2022, 4(2): 115-131

20. Yun-Han LEE, Tao ZHAN, Shin-Tson WU, Prospects and challenges in augmented reality displays Virtual Reality & Intelligent Hardware 2019, 1(1): 10-20

21. Zike YAN, Hongbin ZHA, Flow-based SLAM: From geometry computation to learning Virtual Reality & Intelligent Hardware 2019, 1(5): 435-460

22. Mohib ULLAH, Sareer Ul AMIN, Muhammad MUNSIF, Muhammad Mudassar YAMIN, Utkurbek SAFAEV, Habib KHAN, Salman KHAN, Habib ULLAH, Serious games in science education: a systematic literature review Virtual Reality & Intelligent Hardware 2022, 4(3): 189-209

23. Rafik HAMZA, Minh-Son DAO, Privacy-preserving deep learning techniques for wearable sensor-based big data applications Virtual Reality & Intelligent Hardware 2022, 4(3): 210-222

24. Hayat ULLAH, Sitara AFZAL, Imran Ullah KHAN, Perceptual quality assessment of panoramic stitched contents for immersive applications: a prospective survey Virtual Reality & Intelligent Hardware 2022, 4(3): 223-246