So Shortly before my proposal was submitted after consulting with my supervisor it was decided that a slight shift in focus would be necessary. Instead of pursuing an system for affect detection in audiences, a framework to solve some of the challenges in such a system has become the goal. the main reason for this is because affect detection is a field that is changing quite often and a problem that is very challenging to solve. Therefore there would be some value is developing a facial extraction framework with an inversion of control that would call a supplied method to classify or quantify the emotion in each face. This is the new direction for the thesis, and the core of the proposal.

Proposal time

This week I've been gathering together all my thoughts and ideas and trying to produce a cohesive document that outlines my plan. I keep finding new papers! that slows me down a fair bit. but almost done!

OpenCV and Visual studio

As Hamed has already begun using OpenCV2.2 I have decided to try it out, as i run visual studio on windows 7 x86_64,  i needed to build OpenCV from source as there weren't binaries available for this version and platform.

i followed this guide to get it set-up http://opencv.willowgarage.com/wiki/VisualC%2B%2B and there were a few hiccups along the way:

First they left out a step, after you build the whole solution, you need to build the "install" project to ensure the source and the libraries are assembled in the correct directories.

and second you have to set the build target to x64, however i think in hindsight as there is no "64bit" version of visual studio 2010, i could have used the 32bit version and just build for 32bit. but given that i had it working i decided for the time being just to stick with x64.

OpenCV is great, it  has lots of functionality that will save a fare amount of time. however the video formats and codecs it supports are quite limmited, see http://opencv.willowgarage.com/wiki/VideoCodecs

unfortunately the video i had recorded at the end of last year is in a format that is not supported, despite the fact i recorded with a kinect, this is because i used the open source drivers to record as such i will have to re encode, or make a new recording using the official SDK, Hamed kindly linked me to his code that he used to make recordings, http://code.google.com/p/siento/source/browse/#svn%2Ftrunk%2FXPKinectRecord

also of late, i have been working on my proposal which is due on the 14th of April!

Made a recording!

I did a sample recording late last year so i would have somthing to work with, it should prove usefull, there was only about 4 people in frame, but for now it is more than sufficient. I had hoped not to tell the audience they were being recorded untill after so as to ensure their reactions would be natural, but it ended up nececary to inform them, they were all willing participants.

Also, I am begginig to think I will have to end up using the Official SDK, the open one does not yet support the microphones in any capacity, even recording audio. which will be an issue.

Emotion detection in faces in an audience

Having seen the MIT Moodmeter, it seems like a better way to give feed back to a public speaker is by monitoring the audience. This will tell you if the speaker is really engaging the listeners which is really the prime measure of successful public speaking.

By recording the speak when monitoring the emotions of an audience, eg, happy or bored. you can inform the speaker where they started to lose the audience. The following articles concern emotion detection.

Affective Content Detection by Using Timing Features and Fuzzy Clustering
Min Xu, Suhuai Luo, and Jesse S. Jin

A robust multimodal approach for emotion recognition
Mingli Song Ã, Mingyu You, Na Li, Chun Chen

Emotion Recognition Based on Joint Visual and Audio Cues
Nicu Sebe, Ira Cohen, Theo Gevers, Thomas S. Huang

Neuro forge have a system they call synotive could be useful for emotion detection library.

I will have to decide if it will be feasible to use Kinect as i doubt it could fit more than 10 people in frame at a distance that its depth sensors will function. it may be nececary to use somthing different for or just restrict the project to work with small sample groups.

Posture Detection for better public speaking

Initially I felt the best way to quantify the quality of public speaking was through the posture and appearance of the speaker, and thus give feed back. So I had concerned myself with how to detect the shape and position of a body. from this you could instruct the speaker on things like posture, however challenge would be that people are very different shapes and sizes and have a different posture to begin with. Any changes in their posture that may be indicative of a lapse in focus will vary from person to person. Some of the papers I have looked at regarding posture and body position detection:

Human Position/Height Detection Using Analog Type Pyroelectric Sensors
Shinya Okuda, Shigeo Kaneda, and Hirohide Haga

Laser-Based Tracking of Human Position and Orientation Using Parametric Shape Modeling
Dylan F. Glas, Takahiro Miyashita, Hiroshi Ishiguro and Norihiro Hagita

Human posture recognition for intelligent vehicles
Juan P. Wachs • Mathias Kolsch • Deborah Goshorn

Starting Out

This is the first entry in my thesis blog, just to test its all up and working!