Method
The Project Team.
The project presentation has three major contents; 1) observation of the demographics of people taking pictures of themselves with the best representation of their interests and perception; 2) the visualizations of revealing patterns; 3) an interactive application in both 2D and 3D platforms that allows interactions and exploration of 4000 images associated with four hashtags (#beautiful, #relationship, #achievement, and #travel) as well as discovering new patterns.
The Team is multi-disciplinary with a wide range of backgrounds, including human factor psychologists, visual designers, programmers, data scientists from China, Korea, and the U.S. The project was coordinated by Young Ae Kim and Qiuwen Li who were also responsible for creative direction and visualizations.
Data Collection.
The first step was to create a dataset of images of four hashtags from each location, which required many steps. Each Instagram post has many hashtags in addition to the four research hashtags and often they overlap because Instagram allows thirty hashtags per post. A closer examination reveals that there are two types of images, images taken by someone else and images taken by oneself (selfies). For this project, it was not a concern whether it is taken by oneself or not. We focused on areas of four cities with different climate environments, including Seoul, South Korea (four seasons), Naha, Japan (one season), New York City (four seasons) and Los Angeles, California (one season), because we wanted to collect images and data with the same conditions in order to improve reliability and validity of the data. The project team inspected all photos collected to make sure images were tagged accurately in order to keep the data visually comparable.
Computer Analysis.
The data set of 4000 images will be analyzed by the different types of images, including location, climate differences, face size and orientation, how one’s body framed within the picture (e.g. full body, upper body, or face specific) emotional expression, presence of smile, eyes are closed or open and more. We will be comparing the measured images of characteristics between cities, genders, and ages. These measurements will be implemented in both VR and AR applications (see Fig. 2 and Fig. 3) to allow users t filter the image database by any combination of hashtags or an individual hashtag as well as any types of images of characteristics (see Fig. 1).
Fig. 1. VR User Controller Interface.
OUTPUTS
The 3D visualization environment with Virtual Reality (VR). VR technology provides an engaging learning experience for users. It will allow users to access a big data set in an interactive manner. Happiness Participatory Media VR allows for self- guided exploration of the dataset. As a user enters the VR space, they will be exposed to space with hashtags on Instagram that represent individuals’ happiness, such as #beautiful, #relationship, #achievement and #travel. A user can get closer to each hashtag and choose a specific hashtag to access further information that is involved with the chosen hashtag. Users can select any hashtags floating around them, and as they choose a particular hashtag, a group of images associated with a chosen hashtag from 4,000 images and mean scores of datasets will be displayed. Fig. 5 displays the VR environment and Fig. 6 demonstrates the interface after a user chose a specific keyword. The VR experience will provide a multi-dimensional environment with the most memorable data visualization experience to explore the difference and similarity of how people portray their happiness from different cultures.
Fig. 2. The Virtual Reality Environment User Interface
Fig. 3. User Interface of each Instagram hashtag
The 2D visualization platform with Augmented Reality (AR) Experience. AR experiences expose users to learning and exploring environments through what they find interesting. Creativity and curiosity will be fostered for both student and general public users. Happiness Participatory Media AR is the perfect way to get students’ imagination flying. Participants will have interaction with 30 exhibited images of the most representable image from the 4,000 dataset through Artivive app, which provides moving images of each image that sparks users’ creativity. Fig. 4 is a demonstration of how it works.
Fig. 4. Artivive App AR User Interface
References.
Facebook for Business. Retrieved from https://www.facebook.com/business/marketing/instagram
37 Instagram Stats That Matter to Marketers in 2020. Retrieved from https://blog.hootsuite.com/instagram-statistics/
Li, Q., & Kim, Y. A. (2019, July). Happiness on Instagram–Content Analysis and Engagement Based on Attention Theory. In International Conference on Applied Human Factors and Ergonomics (pp. 13-20). Springer, Cham.
Flora, S. R. (2004). Power of Reinforcement (SUNY Series, Alternatives in Psychology). State University of New York Press.
Burke, M., Kraut, R., & Marlow, C. (2011, May). Social capital on Facebook: Differentiating uses and users. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 571-580).
Festinger, L. (1954). A theory of social comparison processes. Human relations, 7(2), 117-140.
Social Media Comparison & Teen Mental Health. Retrieved from https://www.newportacademy.com/resources/empowering-teens/theory-of-social- comparison/
Fowler, J. H., & Christakis, N. A. (2008). Dynamic spread of happiness in a large social network: longitudinal analysis over 20 years in the Framingham Heart Study. Bmj, 337, a2338.
Goldman, A. H. (2017). Happiness is an Emotion. The Journal of Ethics, 21(1), 1- 16.
Goldman, A. H. (2019). Life's Values: Pleasure, Happiness, Well-Being, and Meaning. Oxford University Press, USA.
Mesquita, B., & Karasawa, M. (2002). Different emotional lives. Cognition & Emotion, 16(1), 127-141.
Lim, N. (2016). Cultural differences in emotion: differences in emotional arousal level between the East and the West. Integrative medicine research, 5(2), 105-109.
Qu Li, Li Zhu Li (2019). Influence of culture diversities in emotional control and emotional reaction. Journal of Jiyuan Vocational and Technical College, China.
Kitayama, S., Markus, H. R., & Kurokawa, M. (2000). Culture, emotion, and well- being: Good feelings in Japan and the United States. Cognition & Emotion, 14(1), 93-124.
Diehl, K., Zauberman, G., & Barasch, A. (2016). How taking photos increases enjoyment of experiences. Journal of personality and social psychology, 111(2), 119.
Barasch, A., Zauberman, G., & Diehl, K. (2018). How the intention to share can undermine enjoyment: Photo-taking goals and evaluation of experiences. Journal of Consumer Research, 44(6), 1220-1237.
