Detect Social relationship in Camera Perspective
Project duration: 11.2022-12.2022
Instructor: Jennifer Rodenhouse & Ben Hooker
Team: Yanyi Wang & Xiaomeng Wang
Brief
We were inspired by the problem of cell phone snooping. When we see someone spying on someone's cell phone, how do we define the person who is looking at the phone with slanted eyes or a tilted head as necessarily "spying"? If their relationship is a friend or intimates, is it should be defined "snooping"? So we want to detect some social relationships through the camera perspective.
So how can we distinguish different social relatiobshios via a camera?
In our prototype, We detect different social relationships through the first and third perspectives of the camera.
The first perspective of the camera is what we see with our naked eyes, imagine we are Iron Man and we are wearing his smart helmet, when a person appears in front of our eyes, how do we judge or detect the relationship between this person and us?
In the third perspective of the camera, if a group of people appears in our eyes, how can we detect the social relationship of these people in the camera?
Our first prototype will show the social relationship from the 1st perspective. In this prototype, we use the number of times the person appears in the camera as our database, and the more times the person appears, the closer the relationship between the person and the ”camera“. As the person gets closer to the camera, the camera will automatically perform eye tracking on the person for further identity determination. At the time of identification, you will be given personal information about this person, like name, times of meeting, relationship, and the date of the last meeting.
Our second prototype will show the social relationship from the 3rd perspective. In this prototype, we use Runway for relationship detection. We set up some physical behaviors as our database to detect different social relationships in the crowd, such as eye contact, oral communication when people open their mouths, and face orientation.
We use these physical behavioral tests to further label the different body touches that are generated by different social relationships, such as friendly touch between friends and intimate touch between a couple or mother and child.
In order to better show this relationship in the third perspective of the camera, we made effects with the example of a couple, which allows the camera to recognize this social relationship in the crowd and highlight this relationship with a filter.