Journal Articles
Permanent URI for this collectionhttps://idr.nitk.ac.in/handle/123456789/19884
Browse
10 results
Search Results
Item EmoWare: A context-aware framework for personalized video recommendation using affective video sequences(Institute of Electrical and Electronics Engineers Inc., 2019) Tripathi, A.; Ashwin, T.S.; Guddeti, R.M.R.With the exponential growth in areas of machine intelligence, the world has witnessed promising solutions to the personalized content recommendation. The ability of interactive learning agents to make optimal decisions in dynamic environments has been proven and very well conceptualized by reinforcement learning (RL). The learning characteristics of deep-bidirectional recurrent neural networks (DBRNN) in both positive and negative time directions has shown exceptional performance as generative models to generate sequential data in supervised learning tasks. In this paper, we harness the potential of the said two techniques and propose EmoWare (emotion-aware), a personalized, emotionally intelligent video recommendation engine, employing a novel context-aware collaborative filtering approach, where the intensity of users' spontaneous non-verbal emotional response toward the recommended video is captured through interactions and facial expressions analysis for decision-making and video corpus evolution with real-time feedback streams. To account for users' multidimensional nature in the formulation of optimal policies, RL-scenarios are enrolled using on-policy (SARSA) and off-policy (Q-learning) temporal-difference learning techniques, which are used to train DBRNN to learn contextual patterns and to generate new video sequences for the recommendation. System evaluation for a month with real users shows that the EmoWare outperforms the state-of-the-art methods and models users' emotional preferences very well with stable convergence. © 2013 IEEE.Item Unobtrusive Behavioral Analysis of Students in Classroom Environment Using Non-Verbal Cues(Institute of Electrical and Electronics Engineers Inc., 2019) Ashwin, T.S.; Guddeti, G.R.Pervasive intelligent learning environments can be made more personalized by adapting the teaching strategies according to the students' emotional and behavioral engagements. The students' engagement analysis helps to foster those emotions and behavioral patterns that are beneficial to learning, thus improving the effectiveness of the teaching-learning process. Unobtrusive student engagement analysis is performed using the students' non-verbal cues such as facial expressions, hand gestures, and body postures. Though there exist several techniques for classifying the engagement of a single student present in a single image frame, there are limited works on the students' engagement analysis in a classroom environment. In this paper, we propose a convolutional neural network architecture for unobtrusive students' engagement analysis using non-verbal cues. The proposed architecture is trained and tested on faces, hand gestures and body postures in the wild of more than 350 students present in a classroom environment, with each test image containing multiple students in a single image frame. The data annotation is performed using the gold standard study, and the annotators reliably agree with Cohen's ? = 0.43. We obtained 71% accuracy for the students' engagement level classification. Further, a pre-test/post-test analysis was performed, and it was observed that there is a positive correlation between the students' engagement and their test performance. © 2013 IEEE.Item Students’ affective content analysis in smart classroom environment using deep learning techniques(Springer New York LLC barbara.b.bertram@gsk.com, 2019) Gupta, S.K.; Ashwin, T.S.; Guddeti, R.M.R.In the era of the smart classroom environment, students’ affective content analysis plays a vital role as it helps to foster the affective states that are beneficial to learning. Some techniques target to improve the learning rate using the students’ affective content analysis in the classroom. In this paper, a novel max margin face detection based method for students’ affective content analysis using their facial expressions is proposed. The affective content analysis includes analyzing four different moods of students’, namely: High Positive Affect, Low Positive Affect, High Negative Affect, and Low Negative Affect. Engagement scores have been calculated based upon the four moods of students as predicted by the proposed method. Further, the classroom engagement analysis is performed by considering the entire classroom as one group and the corresponding group engagement score. Expert feedback and analyzed affect content videos are used as feedback to the faculty member to improve the teaching strategy and hence improving the students’ learning rate. The proposed smart classroom system was tested for more than 100 students of four different Information Technology courses and the corresponding faculty members at National Institute of Technology Karnataka Surathkal, Mangalore, India. The experimental results demonstrate the train and test accuracy of 90.67% and 87.65%, respectively for mood classification. Furthermore, an analysis was performed over incidence, distribution and temporal dynamics of students’ affective states and promising results were obtained. © 2019, Springer Science+Business Media, LLC, part of Springer Nature.Item UAV based cost-effective real-time abnormal event detection using edge computing(Springer, 2019) Shahzad Alam, M.S.; Natesha, B.V.; Ashwin, T.S.; Guddeti, R.M.R.Recent advancements in computer vision led to the development of a real-time surveillance system which ensures the safety and security of the people in public places. An aerial surveillance system will be advantageous in this scenario using a platform like Unmanned Aerial Vehicle (UAV) will be very reliable and can be considered as a cost-effective option for this task. To make the system fully autonomous, we require real-time abnormal event detection. But, this is computationally complex and time-consuming due to the heavy load on the UAV, which affords limited processing and payload capacity. In this paper, we propose a cost-effective approach for aerial surveillance in which we move the large computation tasks to the cloud while keeping limited computation on-board UAV device using edge computing technique. Further, our proposed system will maintain the minimum communication between UAV and cloud. Thus it not only reduces the network traffic but also reduces the end-to-end delay. The proposed method is based on the state-of-the-art YOLO (You Only Look Once) technique for real-time object detection deployed on edge computing device using Intel neural compute stick Movidius VPU (Vision Processing Unit), and we applied abnormal event detection using motion influence map on the cloud. Experimental results demonstrate that the proposed system reduces the end-to-end delay. Further, Tiny YOLO is six times faster while processing the frames per second (fps) when compared to other state-of-the-art methods. © 2019, Springer Science+Business Media, LLC, part of Springer Nature.Item Affective database for e-learning and classroom environments using Indian students’ faces, hand gestures and body postures(Elsevier B.V., 2020) Ashwin, T.S.; Guddeti, R.M.R.Automatic recognition of the students’ affective states is a challenging task. These affective states are recognized using their facial expressions, hand gestures, and body postures. An intelligent tutoring system and smart classroom environment can be made more personalized using students’ affective state analysis, and it is performed using machine or deep learning techniques. Effective recognition of affective states is mainly dependent on the quality of the database used. But, there exist very few standard databases for the students’ affective state recognition and its analysis that works for both e-learning and classroom environments. In this paper, we propose a new affective database for both the e-learning and classroom environments using the students’ facial expressions, hand gestures, and body postures. The database consists of both posed (acted) and spontaneous (natural) expressions with single and multi-person in a single image frame with more than 4000 manually annotated image frames with object localization. The classification was done manually using the gold standard study for both Ekman's basic emotions and learning-centered emotions, including neutral. The annotators reliably agree when discriminating against the recognized affective states with Cohen's ? = 0.48. The created database is more robust as it considers various image variants such as occlusion, background clutter, pose, illumination, cultural & regional background, intra-class variations, cropped images, multipoint view, and deformations. Further, we analyzed the classification accuracy of our database using a few state-of-the-art machine and deep learning techniques. Experimental results demonstrate that the convolutional neural network based architecture achieved an accuracy of 83% and 76% for detection and classification, respectively. © 2020 Elsevier B.V.Item Impact of inquiry interventions on students in e-learning and classroom environments using affective computing framework(Springer Science and Business Media B.V. editorial@springerplus.com, 2020) Ashwin, T.S.; Guddeti, R.M.R.Effective teaching strategies improve the students’ learning rate within academic learning time. Inquiry-based instruction is one of the effective teaching strategies used in the classrooms. But these teaching strategies are not adapted in other learning environments like intelligent tutoring systems, including auto tutors. In this paper, we propose an automatic inquiry-based instruction teaching strategy, i.e., inquiry intervention using students’ affective states. The proposed model contains two modules: the first module consists of the proposed framework for predicting the unobtrusive multi-modal students’ affective states (teacher-centric attentive and in-attentive states) using the facial expressions, hand gestures and body postures. The second module consists of the proposed automated inquiry-based instruction teaching strategy to compare the learning outcomes with and without inquiry intervention using affective state transitions for both an individual and a group of students. The proposed system is tested on four different learning environments, namely: e-learning, flipped classroom, classroom and webinar environments. Unobtrusive recognition of students’ affective states is performed using deep learning architectures. After student-independent tenfold cross-validation, we obtained the students’ affective state classification accuracy of 77% and object localization accuracy of 81% using students’ faces, hand gestures and body postures. The overall experimental results demonstrate that there is a positive correlation with r= 0.74 between students’ affective states and their performance. Proposed inquiry intervention improved the students’ performance as there is a decrease of 65%, 43%, 43%, and 53% in overall in-attentive affective state instances using the inquiry interventions in e-learning, flipped classroom, classroom and webinar environments, respectively. © 2020, Springer Nature B.V.Item Multimodal behavior analysis in computer-enabled laboratories using nonverbal cues(Springer Science and Business Media Deutschland GmbH info@springer-sbm.com, 2020) Banerjee, S.; Ashwin, T.S.; Guddeti, R.M.R.In the modern era, there is a growing need for surveillance to ensure the safety and security of the people. Real-time object detection is crucial for many applications such as traffic monitoring, security, search and rescue, vehicle counting, and classroom monitoring. Computer-enabled laboratories are generally equipped with video surveillance cameras in the smart campus. But, from the existing literature, it is observed that the use of video surveillance data obtained from smart campus for any unobtrusive behavioral analysis is seldom performed. Though there are several works on the students’ and teachers’ behavior recognition from devices such as Kinect and handy cameras, there exists no such work which extracts the video surveillance data and predicts the behavioral patterns of both the students and the teachers in real time. Hence, in this study, we unobtrusively analyze the students’ and teachers’ behavioral patterns inside a teaching laboratory (which is considered as an indoor scenario of a smart campus). Here, we propose a deep convolution network architecture to classify and recognize an object in the indoor scenario, i.e., the teaching laboratory environment of the smart campus with modified Single-Shot MultiBox Detector approach. We used six different class labels for predicting the behavioral patterns of both the students and the teachers. We created our dataset with six different class labels for training deep learning architecture. The performance evaluation demonstrates that the proposed method performs better with an accuracy of 0.765 for classification and localization. © 2020, Springer-Verlag London Ltd., part of Springer Nature.Item Surveillance video analysis for student action recognition and localization inside computer laboratories of a smart campus(Springer, 2021) Rashmi, M.; Ashwin, T.S.; Guddeti, G.R.M.In the era of smart campus, unobtrusive methods for students’ monitoring is a challenging task. The monitoring system must have the ability to recognize and detect the actions performed by the students. Recently many deep neural network based approaches have been proposed to automate Human Action Recognition (HAR) in different domains, but these are not explored in learning environments. HAR can be used in classrooms, laboratories, and libraries to make the teaching-learning process more effective. To make the learning process more effective in computer laboratories, in this study, we proposed a system for recognition and localization of student actions from still images extracted from (Closed Circuit Television) CCTV videos. The proposed method uses (You Only Look Once) YOLOv3, state-of-the-art real-time object detection technology, for localization, recognition of students’ actions. Further, the image template matching method is used to decrease the number of image frames and thus processing the video quickly. As actions performed by the humans are domain specific and since no standard dataset is available for students’ action recognition in smart computer laboratories, thus we created the STUDENT ACTION dataset using the image frames obtained from the CCTV cameras placed in the computer laboratory of a university campus. The proposed method recognizes various actions performed by students in different locations within an image frame. It shows excellent performance in identifying the actions with more samples compared to actions with fewer samples. © 2020, Springer Science+Business Media, LLC, part of Springer Nature.Item A Google Glass Based Real-Time Scene Analysis for the Visually Impaired(Institute of Electrical and Electronics Engineers Inc., 2021) Ali A, H.; Rao, S.U.; Ranganath, S.; Ashwin, T.S.; Guddeti, G.R.M.Blind and Visually Impaired People (BVIP) are likely to experience difficulties with tasks that involve scene recognition. Wearable technology has played a significant role in researching and evaluating systems developed for and with the BVIP community. This paper presents a system based on Google Glass designed to assist BVIP with scene recognition tasks, thereby using it as a visual assistant. The camera embedded in the smart glasses is used to capture the image of the surroundings, which is analyzed using the Custom Vision Application Programming Interface (Vision API) from Azure Cognitive Services by Microsoft. The output of the Vision API is converted to speech, which is heard by the BVIP user wearing the Google Glass. A dataset of 5000 newly annotated images is created to improve the performance of the scene description task in Indian scenarios. The Vision API is trained and tested on this dataset, increasing the mean Average Precision (mAP) score from 63% to 84%, with an IoU > 0.5. The overall response time of the proposed application was measured to be less than 1 second, thereby providing accurate results in real-time. A Likert scale analysis was performed with the help of the BVIP teachers and students at the 'Roman Catherine Lobo School for the Visually Impaired' at Mangalore, Karnataka, India. From their response, it can be concluded that the application helps the BVIP better recognize their surrounding environment in real-time, proving the device effective as a potential assistant for the BVIP. © 2013 IEEE.Item A Multi-Protocol Home Automation System Using Smart Gateway(Springer, 2021) Chaudhary, S.K.; Yousuff, S.; Meghana, N.P.; Ashwin, T.S.; Guddeti, R.M.R.Smart Home is one of the most established applications of the Internet of Things. Almost every equipment we use in our daily life—appliances, electric lights, electrical outlets, heating, and cooling systems-connected to a remotely controllable network, giving the user’s ability to remotely control and monitor the house, save energy without compromising on comfort and ultimately improve the quality of experience of staying in the house. We present a cost-effective system and address a major challenge that the industry faces today-Protocol Compatibility. To address the challenge, we make use of separate gateways/bridges for each network and an open-source home automation framework called OpenHAB, where each bridge links with a single master Wi-Fi gateway, providing a single window of control through an Application or a web interface for an integrated Smart Home. We integrate an elderly health monitoring device-Beehealth with OpenHAB; addressing the paramount need of a portable, accurate, and efficient health monitoring and fall detection device. We present two methods for fall detection, namely: threshold-based and Neural Network-based, with the latter resulting in 94% accuracy for fall detection. We evaluate the Smart Home devices on parameters like syncing time, battery life, recharge time, deployability, and cost. © 2020, Springer Science+Business Media, LLC, part of Springer Nature.