Current Research Positions

Ph.D. Research Opportunities (from Jan 2024)

Project Overview

These opportunities are funded under the Laureat Research program of the Irish Research Council (2023); the funded project is PRIVI-SENSE (Privacy-Responsive Integrated computer Vision for Intelligent Safe sENsing and Situational Enhancement)

This project will investigate novel combinations of multimodal imaging – near-Infrared (NIR), thermal-Infrared (LWIR) and neuro-morphic (event) camera technologies processed by AI networks, to infer actions or detect subtle thermal or biometric events related to a human subject. The combination of AI with multi-modal imaging of a scene/subject enables subtle understanding and sensing without creating images or video of a human subject that might accidently violate their data privacy.

There are three Ph.D. research opportunities associated with this project.

Ph.D. Opportunity #1 – Multimodal User Authentication and Personalization

Building on current C3I research on authenticating users [1]–[4] this work package will explore the extension of existing embedded, in-device authentication methods, based on conventional imaging, to employ methods based on multi-modal imaging data. An additional objective of this research will be to determine some unique advantages of multimodal authentication over conventional imaging and to personalize some of the sensing methods explored in Ph.D. project #2. Recent  works in the C3I group have progressed some aspects of this research topic, notably some of our work to develop tools for synthesis of child data [5]–[7], on the effects of ageing on authentication [8], [9] and recent work on neuromorphic vision systems for sensing [10]–[13]. Based on our most recent experience it is likely that this research topic will focus on the use of event-cameras combined with sparse/spiking neural network architectures.

Ph.D. Opportunity #2 - Physiological, Emotional and Cognitive Sensing

Thermal data can determine breathing patterns and temperature variations in the face can be linked with subject comfort and was demonstrated via the Heliaus project [14]–[17]. Similarly, eye and facial movements are more easily distinguished by neuromorphic imaging [18], [19]. A key challenge here will be to better correlate these involuntary behavioural patterns of a subject with their cognitive and emotional state. A secondary objective will be to investigate the potential of multi-modal imaging to achieve physiological sensing of, for example, a subject’s pulse rate. Other behavioural patterns, e.g. eye-movements or facial tics, could be linked to underlying physiological conditions. A key end goal is to advance the use of smart-sensing in support of the well-being of home workers – a field where research is growing following the broad adoption of home-working during the pandemic [20]–[23]. Also, this research topic may encompass speech analysis to enhance computer-vision based sensing. C3I has built significant know-how on child speech synthesis and recognition in the past 3 years [6], [24]–[26].

Ph.D. Opportunity #3 - Assistive Supports for Older Adults at Home

Human action recognition from video is a rapidly emerging field [27]–[29]. This work package will move a step beyond basic human actions and focus on Activities for Daily Living (ADL) . [16], [29], [30]. This is an area of application where data privacy is essential [31], [32] and where smart-sensing, based on multi-modal imaging can offer a novel approach. Our key goal in this work package is to deliver a working framework which will leverage work undertaken by the three Ph.D. researchers in order to deliver a working Edge-AI system that can be field-tested and validated in user trials during the final 18 months of the project.

Requirements

Potential candidates should be highly motivated and ideally have completed a Master level degree. A strong background in computer vision and neural-network (NN) based data analysis is essential. A strong programming background is essential with knowledge of Linux and tools such as Pytorch, OpenCV, Docker of high relevance. Experience with advance neural architectures such as GANs, Transformers, generative data methods, Diffusion Models or equivalents is an advantage. Configuring and operating large-scale experiments with state-of-art datasets for training such models is an added advantage. Experience working with thermal, NIR, or event camera data is also an advantage.

Applicants can write to peter.corcoran@universityofgalway.ie in the first instance – add “IRC PhD #N” (N is your preferred choice from above) in the subject of your e-mail and include a CV and evidence of English proficiency with your e-mail. (NB: Do not send multiple e-mails if you wish to be considered!)

References

[1]             S. Bazrafkan, T. Nedelcu, P. Filipczuk, and P. Corcoran, “Deep learning for facial expression recognition: A step closer to a smartphone that knows your moods,” in 2017 IEEE International Conference on Consumer Electronics (ICCE), Jan. 2017, pp. 217–220. doi: 10.1109/ICCE.2017.7889290.

[2]             S. Bazrafkan and P. Corcoran, “Enhancing iris authentication on handheld devices using deep learning derived segmentation techniques,” in 2018 IEEE International Conference on Consumer Electronics (ICCE), Jan. 2018, pp. 1–2. doi: 10.1109/ICCE.2018.8326219.

[3]             A. Mitra, D. Bigioi, S. P. Mohanty, P. Corcoran, and E. Kougianos, “iFace 1.1: A Proof-of-Concept of a Facial Authentication Based Digital ID for Smart Cities,” IEEE Access, vol. 10, pp. 71791–71804, 2022.

[4]             W. Yao, V. Varkarakis, G. Costache, J. Lemley, and P. Corcoran, “Towards Robust Facial Authentication for Low-Power Edge-AI Consumer Devices,” IEEE Access, pp. 1–1, 2022, doi: 10.1109/ACCESS.2022.3224437.

[5]             M. A. Farooq, W. Yao, G. Costache, and P. Corcoran, “ChildGAN: Large Scale Synthetic Child Facial Data Using Domain Adaptation in StyleGAN,” IEEE Access, vol. 11, pp. 108775–108791, 2023, doi: 10.1109/ACCESS.2023.3321149.

[6]             M. Ali Farooq, D. Bigioi, R. Jain, W. Yao, M. Yiwere, and P. Corcoran, “Synthetic Speaking Children – Why We Need Them and How to Make Them,” in 2023 International Conference on Speech Technology and Human-Computer Dialogue (SpeD), Oct. 2023, pp. 36–41. doi: 10.1109/SpeD59241.2023.10314943.

[7]             W. Yao, M. A. Farooq, J. Lemley, and P. Corcoran, “A Comparative Study of Image-to-Image Translation Using GANs for Synthetic Child Race Data,” Aug. 2023, doi: 10.5281/ZENODO.8208491.

[8]             W. Yao, M. A. Farooq, J. Lemley, and P. Corcoran, “A Study on the Effect of Ageing in Facial Authentication and the Utility of Data Augmentation to Reduce Performance Bias Across Age Groups,” IEEE Access, vol. 11, pp. 97118–97134, 2023, doi: 10.1109/ACCESS.2023.3312612.

[9]             Will your Doorbell Camera still recognize you as you grow old? Zenodo, 2023. doi: 10.5281/zenodo.8208368.

[10]           P. Kielty, M. S. Dilmaghani, W. Shariff, C. Ryan, J. Lemley, and P. Corcoran, “Neuromorphic Driver Monitoring Systems: A Proof-of-Concept for Yawn Detection and Seatbelt State Detection Using an Event Camera,” IEEE Access, vol. 11, pp. 96363–96373, 2023, doi: 10.1109/ACCESS.2023.3312190.

[11]           W. Shariff et al., “Neuromorphic Driver Monitoring Systems: A Computationally Efficient Proof-of-Concept for Driver Distraction Detection,” IEEE Open J. Veh. Technol., vol. 4, pp. 836–848, 2023, doi: 10.1109/OJVT.2023.3325656.

[12]           M. S. Dilmaghani, W. Shariff, C. Ryan, J. Lemley, and P. Corcoran, “Control and evaluation of event cameras output sharpness via bias,” in Fifteenth International Conference on Machine Vision (ICMV 2022), SPIE, Jun. 2023, pp. 455–462. doi: 10.1117/12.2679755.

[13]           C. Ryan et al., “Real-Time Multi-Task Facial Analytics With Event Cameras,” IEEE Access, vol. 11, pp. 76964–76976, 2023, doi: 10.1109/ACCESS.2023.3297500.

[14]           D. Cardone et al., “Driver Stress State Evaluation by Means of Thermal Imaging: A Supervised Machine Learning Approach Based on ECG Signal,” Appl. Sci., vol. 10, no. 16, Art. no. 16, Jan. 2020, doi: 10.3390/app10165673.

[15]           D. Cardone et al., “Driver drowsiness evaluation by means of thermal infrared imaging: preliminary results,” in Infrared Sensors, Devices, and Applications XI, A. K. Sood, P. Wijewarnasuriya, and A. I. D’Souza, Eds., San Diego, United States: SPIE, Aug. 2021, p. 25. doi: 10.1117/12.2594504.

[16]           D. Perpetuini et al., “Can Functional Infrared Thermal Imaging Estimate Mental Workload in Drivers as Evaluated by Sample Entropy of the fNIRS Signal?,” in 8th European Medical and Biological Engineering Conference, T. Jarm, A. Cvetkoska, S. Mahnič-Kalamiza, and D. Miklavcic, Eds., in IFMBE Proceedings. Cham: Springer International Publishing, 2021, pp. 223–232. doi: 10.1007/978-3-030-64610-3_26.

[17]           D. Cardone et al., “Classification of Drivers’ Mental Workload Levels: Comparison of Machine Learning Methods Based on ECG and Infrared Thermal Signals,” Sensors, vol. 22, no. 19, Art. no. 19, Jan. 2022, doi: 10.3390/s22197300.

[18]           C. Ryan et al., “Real-time face & eye tracking and blink detection using event cameras,” Neural Netw., vol. 141, pp. 87–97, Sep. 2021, doi: 10.1016/j.neunet.2021.03.019.

[19]           M. S. Dilmaghani, W. Shariff, C. Ryan, J. Lemley, and P. Corcoran, “Control and Evaluation of Event Cameras Output Sharpness via Bias.” arXiv, Oct. 25, 2022. doi: 10.48550/arXiv.2210.13929.

[20]           C. Belletier, M. Charkhabi, G. Pires de Andrade Silva, K. Ametepe, M. Lutz, and M. Izaute, “Wearable cognitive assistants in a factory setting: a critical review of a promising way of enhancing cognitive performance and well-being,” Cogn. Technol. Work, vol. 23, no. 1, pp. 103–116, Feb. 2021, doi: 10.1007/s10111-019-00610-2.

[21]           L. Marino and V. Capone, “Smart Working and Well-Being before and during the COVID-19 Pandemic: A Scoping Review,” Eur. J. Investig. Health Psychol. Educ., vol. 11, no. 4, Art. no. 4, Dec. 2021, doi: 10.3390/ejihpe11040108.

[22]           A. Papetti, F. Gregori, M. Pandolfi, M. Peruzzini, and M. Germani, “A method to improve workers’ well-being toward human-centered connected factories,” J. Comput. Des. Eng., vol. 7, no. 5, pp. 630–643, Oct. 2020, doi: 10.1093/jcde/qwaa047.

[23]           M. Shamsi, T. Iakovleva, E. Olsen, and R. P. Bagozzi, “Employees’ Work-Related Well-Being during COVID-19 Pandemic: An Integrated Perspective of Technology Acceptance Model and JD-R Theory,” Int. J. Environ. Res. Public. Health, vol. 18, no. 22, Art. no. 22, Jan. 2021, doi: 10.3390/ijerph182211888.

[24]           R. Jain and P. Corcoran, “Improved Child Text-to-Speech Synthesis through Fastpitch-based Transfer Learning,” in 2023 International Conference on Speech Technology and Human-Computer Dialogue (SpeD), Oct. 2023, pp. 54–59. doi: 10.1109/SpeD59241.2023.10314899.

[25]           R. Jain, M. Y. Yiwere, D. Bigioi, P. Corcoran, and H. Cucu, “A Text-to-Speech Pipeline, Evaluation Methodology, and Initial Fine-Tuning Results for Child Speech Synthesis,” IEEE Access, vol. 10, pp. 47628–47642, 2022, doi: 10.1109/ACCESS.2022.3170836.

[26]           R. Jain, A. Barcovschi, M. Y. Yiwere, D. Bigioi, P. Corcoran, and H. Cucu, “A WAV2VEC2-Based Experimental Study on Self-Supervised Learning Methods to Improve Child Speech Recognition,” IEEE Access, vol. 11, pp. 46938–46948, 2023, doi: 10.1109/ACCESS.2023.3275106.

[27]           P. Pareek and A. Thakkar, “A survey on video-based Human Action Recognition: recent updates, datasets, challenges, and applications,” Artif. Intell. Rev., vol. 54, no. 3, pp. 2259–2322, Mar. 2021, doi: 10.1007/s10462-020-09904-8.

[28]           Z. Sun, Q. Ke, H. Rahmani, M. Bennamoun, G. Wang, and J. Liu, “Human Action Recognition From Various Data Modalities: A Review,” IEEE Trans. Pattern Anal. Mach. Intell., pp. 1–20, 2022, doi: 10.1109/TPAMI.2022.3183112.

[29]           R. Dai et al., “Toyota Smarthome Untrimmed: Real-World Untrimmed Videos for Activity Detection,” IEEE Trans. Pattern Anal. Mach. Intell., pp. 1–1, 2022, doi: 10.1109/TPAMI.2022.3169976.

[30]           S. Das et al., “Toyota Smarthome: Real-World Activities of Daily Living,” presented at the Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 833–842. Accessed: Dec. 04, 2022. [Online]. Available: https://openaccess.thecvf.com/content_ICCV_2019/html/Das_Toyota_Smarthome_Real-World_Activities_of_Daily_Living_ICCV_2019_paper.html

[31]           P. Climent-Pérez and F. Florez-Revuelta, “Protection of visual privacy in videos acquired with RGB cameras for active and assisted living applications,” Multimed. Tools Appl., vol. 80, no. 15, pp. 23649–23664, Jun. 2021, doi: 10.1007/s11042-020-10249-1.

[32]           A. Karale, “The challenges of IoT addressing security, ethics, privacy, and laws,” Internet Things, vol. 15, p. 100420, 2021.