Estimator emocionalnih stanja u stvarnom vremenu zasnovan na dubinskoj analizi fizioloških signala

doctoral thesis

• Title (english): Real-time estimation of emotional states based on physiological signals
• Author: Davor Kukolja
• Mentor: Krešimir Ćosić (mentor)
• Mentor: Ivan Petrović (komentor)
• Committee member: Nikola Bogunović (predsjednik povjerenstva)
• Committee member: Domagoj Jakobović (član povjerenstva)
• Committee member: Mario Vašak (član povjerenstva)
• Committee member: Miro Jakovljević (član povjerenstva); MBZ: 99041
• Committee member: Dragutin Ivanec (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; (Department of Electric Machines, Drives and Automation); Zagreb
• Defense date and country: 2012-10-30, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing
• Universal decimal classification (UDC): 004 - Computer science and technology. Computing. Data processing

Abstract

U okviru ove disertacije prikazuju se razni postupci estimacije emocionalnih stanja na temelju dubinske analize fizioloških signala, kao što su EKG, vodljivost i temperatura kože te respiracija, metode pronalaženja dominantnih fizioloških značajki i opća arhitektura sustava za estimaciju emocionalnih stanja uz prilagodbu pojedincu u stvarnom vremenu. Identificirane su komponente opće arhitekture estimatora emocionalnih stanja koje se odnose na analizu prije početka seanse, estimaciju i adaptaciju estimatora u stvarnom vremenu tijekom seanse, te potrebne baze podataka. Daljnja raščlamba arhitekture opisuje odabir podataka za učenje estimatora, obradu sirovih fizioloških signala, računanje fizioloških značajki, redukciju fizioloških značajki, nadzirano učenje estimatora, kao i potrebne baze podataka o svim ispitanicima, o trenutnom ispitaniku, o spremljenim parametrima estimatora itd. Obrazložena su vremenska ograničenja za estimaciju emocionalnih stanja i adaptaciju estimatora u stvarnom vremenu, a s ciljem njihovog zadovoljavanja provedena je vremenska analiza i paralelizacija pojedinih koraka estimacije i adaptacije estimatora. Predložene su formulacije problema pronalaženja dominantnih fizioloških značajki za estimaciju emocionalnih stanja, te pripadne metode za pronalaženje takvih značajki. Eksperimentalno je testirana točnost estimatora emocionalnih stanja razvijenih sukladno predloženoj arhitekturi, na problemu klasifikacije diskretnih emocija uz sustavno kombiniranje raznih metoda redukcije značajki i metoda klasifikacije. U tom kontekstu, izloženi su i rezultati primjene predloženih metoda za pronalaženje dominantnih fizioloških značajki. Također je testirana estimacija ugode i pobuđenosti provedena kombinacijom SFFS (engl. sequential floating forward selection) redukcije značajki i višeslojnog perceptrona, koja se odlikovala najvećom točnošću klasifikacije diskretnih emocija.

Abstract (english)

The thesis explores methods of emotional state estimation based on data mining of physiological signals, such as electrocardiogram (ECG), skin conductance, skin temperature and respiration, methods for finding the dominant physiological features for emotional state estimation, and the general architecture of the corresponding systems for the emotional state estimation that adapt to the individual in real-time. The major architectural components of the emotional state estimator are identified, which refer to the analysis before starting the session, estimation and adaptation in real-time during the session, and the required databases. Further architectural decomposition describes in detail the steps of data selection for estimator learning, raw physiological signal processing, physiological feature calculation, physiological feature reduction, estimator supervised learning, as well as the databases of all the participants, of the current participant, of the stored parameters of the estimator etc. Time constraints for the estimation and estimator adaptation in real time are explained. As an example of the analysis needed to meet the imposed time constraints, time analysis and parallelization of tasks that constitute real-time emotional state estimation and adaptation are conducted. The problem of finding the dominant physiological features for emotional state estimation is formulated, and associated methods for finding such features are proposed. The accuracy of emotional state estimators developed in line with the proposed architecture is experimentally tested on the problem of discrete emotions classification based on the systematic combination of various methods of feature reduction and classification. The results of applying the methods for dominant physiological feature identification are also analyzed. Estimation of valence and arousal is conducted with the use of multilayer perceptron with SFFS feature reduction, a combination that exhibited the highest classification accuracy for discrete emotions.

• Keywords: emocionalna stanja; fiziološki signali; fiziološke značajke; dubinska analiza; estimacija u stvarnom vremenu; adaptacija u stvarnom vremenu; arhitektura estimatora emocionalnih stanja
• Keywords (english): emotional states; physiological signals; physiological features; data mining; real-time estimation; real-time adaptation; emotional state estimator architecture
• Language: croatian
• URN:NBN: urn:nbn:hr:168:352270
• Project: Number: 036-0000000-2029; Title: Adaptivno upravljanje scenarijima u VR terapiji PTSP-a; Leader: Krešimir Ćosić; Jurisdiction: Croatia; Funder: MZOS; Funding stream: ZP
• Project: Number: 036-0363078-3018; Title: Upravljanje mobilnim robotima i vozilima u nepoznatim i dinamičkim okruženjima; Leader: Ivan Petrović; Jurisdiction: Croatia; Funder: MZOS; Funding stream: ZP
• Study programme: Title: Computer Science; Study programme type: university; Study level: postgraduate; Academic / professional title: Doktor znanosti (Doktor znanosti)
• Catalog URL: http://lib.fer.hr/cgi-bin/koha/opac-detail.pl?biblionumber=40522
• Type of resource: Text
• Extent: 193 str ; 30 cm
• File origin: Born digital
• Access conditions: Closed access
• Repository: FER Repository
• Created on: 2019-09-09 11:27:57