LIFEREC: A Framework for Recommending Users from Past Life Experiences

Life logging has been an eminent topic of concern in recent years with many researchers focusing on capturing daily life activities of human. With the proliferation of IoT (Internet of Things) domain, the devices are now able to record human interaction for longer periods as well as transfer this data easily to other computing devices or cloud storage. This article proposes a novel framework named as LIFEREC which acquires information from IoT aware devices and sensors. It maintains activity profiles of various activities performed by the users in their daily lives. Furthermore, the framework provides recommendations when requested by an individual while taking into account the past life history and current context. Recent research on digitizing human life is quite efficient in amassing enormous data but futile in offering assistance for prospect decisions in life. The data gathered by the lifelog devices may be of a great help in taking decisions. The proposed system gives a new direction to existing mechanisms of providing recommendations by exploiting the current context and the past experiences of human life. The recommendations provided by our proposed system may be very helpful while performing those activities which have already been experienced in the past.

We have organized the paper as following. The related work is discussed in section 2. Section 3 highlights the design considerations and Section 4 provides the proposed LIFEREC framework. Section 5 explains the methodology. Finally, Section 6 concludes the work. al. [11] used smart phones for collecting life log in the form of (Global Positioning System), photos, songs, SMS (Short Message Service) and weather information and utilized keyGraph [12] for identifying meaningful context from captured data. The efforts by these researchers lack the mechanism to recommend the users by utilizing their past lifelogs. Furthermore, the interface provided to interact with the past life requires the user to fill in various context to fetch the lifelogs.

RELATED WORK
Talking about the activities captured using lifelog device, an application named as Activity Designer was developed by [13] in which users were able to develop their own activity models manually and monitor daily activities such as fitness exercises, transport usage and other social activities. The system was a low cost ubiquitous computing application for gazing into past activities.
Another application called the Digital Parrot [14] was The authors [16][17] attempted to make a lifelog system that employs the current context of the user such as people and objects nearby along with the location. The system fetched past lifelogs based on the current context but did not provide any recommendations. The authors evaluated the system by comparing it with Vicon Revue lifelog camera. Our proposed framework takes several previous designs in to account and attempts to create a system that is extensible and helps the users in their day to day activities.

DESIGN CONSIDERATIONS
Lifelogging has several merits besides self-awareness and health consciousness. One of them is that the users may Following are the design considerations which must be considered while developing the lifelog recommender system.

Authenticity of Lifelogs
The Once the lifelogs are stored on the cloud server, they can be organized and processed in order to be used for future recommendations.

User Intervention
The lifelogs are generated by the actions of the users while the sensors are always running in the background recording those actions. Thus, the IoT based devices and sensors keep the track of the user continuously without user's administration. However, the users must have appropriate control over the captured lifelogs so that if they fearabout any privacy infringement, they are able to obliterate that part of lifelogs. In the proposed system, the authorized smart device is used to provide an interface that may intervene the lifelogs and allow users to make alterations to the data being sent to the cloud server.

Storage Requirement
Storage is significant when it comes to lifelogs as they keep piling up while the user is performing the activities.
It is important to keep these lifelogs at a place where they can be easily retrieved. Storing the lifelogs on the personal devices of the user will not be a decent option as they have limited and volatile memory. It will be therefore, more efficient to store this large collection of lifelogs at a cloud storage for easy and quick retrieval. In the proposed framework, the IoT based devices as well as sensors attached to the user body send the lifelog data to the personal device of the user, from where it is eventually forwarded to the cloud server.

LIFEREC FRAMEWORK
Keeping the design considerations in view, we have developed the LIFEREC framework as shown in Fig. 1.
The proposed framework will not only help to keep the precious life moments of the users organized, but will also assist the users in making decisions in similar situations

IoT Based Devices
The act of making the devices able to communicate over internet and develop rich applications is termed as IoT [22,23]. The key aspects of IoT devices that need to be tackled carefully are heteroginity, connectivity, device naming, device addressing and security [24]. The functionality of these devices can be fully employed to generate and store lifelogs of individual users which may be sent to a central server. We will discuss a few of the IoT devices which are commercially available in today's market.

Smart Fridge
Smart fridge is the keyword used for a fridge that is capable of ordering groceries by itself according to the user's needs. The fridge keeps tracks of the items consumed by the user at a particular time and keeps them ready for consumption like chilling the cold drink can or defrosting the frozen items. The fridge also retains the knowledge about food allergens contained in the products and as well as their expiry dates. The use of RFID ((Radio-Frequency Identification) antennas makes the fridge capable to perform the above mentioned tasks very easily [25]. The smart fridge can send all the interactions performed by the user to their authorized lifelog device. This data can be used for providing healthy recommendations to the user on their next interaction with the fridge.

Smart Television
Smart televisions are gaining popularity since they behave like a computer with internet connected, thus allowing them to view digital content directly and store the history of the content watched by the user. Furthermore, camera, microphone and NFC (Near-Field Communication) tags are used for identification of the user accessing the smart television [26]. In such case, the user may leave digital footprints that may be employed intelligently to provide recommendations based on the past viewed programs. The lifelog application may incorporate the current context such as free time available, user's mood and people accompanied by the user to provide appropriate recommendations.

Sensors
Sensors NFC allows the mobile devices for TouchMe interaction paradigm, that is, the users can easily establish connections between objects by simply touching them [30]. NFC can be best utilized for authentication, detection of the objects interacted by users and checking in at various points in a city [31]. It can also be used as a medium to collect data from health monitoring devices such as heart rate, glucose and blood pressure monitor [32]. The past records from NFC based sensing may help find misplaced objects like keys, phone, etc. Moreover, the data collected from health monitoring devices may warn the user of any aberrations in bodily functions like destabilized blood glucose, arrhythmia.
The accelerometer sensor, when attached to a device, is capable of observing changes in the X, Y and Z of the device. If this sensor is worn on the body, it can provide useful information such as sleep patterns [33], walking, jogging and cycling [34]. We will discuss about the lifelog cloud server (Fig. 1) in the following subsection.

Lifelog Cloud
The lifelog cloud stores the activities performed by the user in the form of activity profiles. It also contains two more components which are activity organizer and activity recommender. The roles of various components of lifelog cloud are discussed in the following subsections.

Activity Organizer
Activity organizer is a part of lifelog cloud where the detected activities or captured lifelogs are managed in chronological as well as spatial order. The Activity Organizer proposed for this system is responsible to receive the lifelogs from the authorized user devices in order to ensure security and privacy. The two significant attributes while recording user activities are location and time. Since, human beings have correlations with time and locations, therefore, the ordering of the activities with respect to time and locations will do a great deal while retrieving the past. For example, the jogging activity time and location is sequenced with the next activity performed by the user after jogging like showering at home. In this manner, on some regular day, if the user wants to go some party after jogging, the user may be recommended accordingly based on the past patterns.

Activity Profiles
The

Activity Recommender
This is an essential part of the lifelog cloud. Since the lifelogs of the user's past activities are being collected and stored as activity profiles of a user, it is achievable to provide suitable suggestions based on the current situation of the user. The sensors worn by the user and the authorized smart device are capable of providing sufficient context of the user which is passed on to the activity recommender. The module of the activity recommender consists of two parts, current context examiner and profile fetcher. The task of the current context examiner is to determine the past activities that match with the current activity and contextof the user. In the next step, the profiles of similar past activities are fetched from the lifelogcloud. Finally, the activity recommender generates suggestions based on the content based, collaborative and hybrid recommendation approaches [35]. The final recommendations are shown on the user's registered smart device and it is up to the user to follow these recommendations. The cycle of capturing the lifelogs and recommending activities continues to benefit users in their respective lives.

METHODOLOGY
In this section, we discuss the overall working of the system. The smart device carried by the user not only performs the functionality of authenticating the daily activity lifelogs received from the IoT devices and sensors, but it also fetches recommendations from the lifelog cloud. Fig. 2, shows a sequence diagram, depicting the step by step process of capturing of lifelogs and retrieval of recommendations from the lifelog cloud. The lifelog cloud server organizes the lifelogs and maintains the activity profiles of the user.
We have designed a lifelog application running on an android based smart phone. The application consists of two interfaces, one for authenticating the lifelogs ( Fig.  3(a)) and the other for receiving recommendations from the lifelog server ( Fig. 3(b)). The user does not necessarily need to authenticate each and every lifelog entity as it is practically impossible. However, the provision of authenticating lifelogs is given for such situations where the user may want to hide those lifelogs which are infringing the privacy. In the current version of the system, we have programmed various sensors for activity detection. The IoT aware device have not been configured in the current version of the system.
We have used the built in sensors of the smart phone in this prototype which areaccelerometer, Bluetooth, NFC reader, and GPS. In Fig. 4, we show 13 different activities detected by our current prototype system and the sensors involved in detecting those activities.
The activity profiles are created and updated using the The next subsection shows the evaluation results of our prototype system.

Evaluation
We asked three users (aged 24, 27 and 30) to install the system on their smart phones and engage with our application for 5 consecutive days. They had a training session of 2 hours before the start of evaluation. In this session they were briefed about the purpose of this study and working mechanism of the prototype application. The users were able to view lifelog activities and remove any undesirable activity log before being sent to the cloud server. In order to detect 'Reading a book' activity, the users were asked to attach NFC tags to their books. The users may request the application for the recommendations as many times as possible.
The aim of this experiment was to determine the usefulness of our system. We attempted to find whether the recommendations given by the system were followed by the user. The accuracy of activity detection was not observed in this experiment, since many researchers have already proposed algorithms [37,38] for detecting user activities using body worn sensors.
At the end of the experiment, we found that a total of 534 activity profiles were updated at the lifelog cloud server based on the activities of three users for a period of 5 days. The users were given recommendations of 101 activities during the experiment and they followed the suggestions 44 times. In the graph shown in Fig. 5, we compare the average number of activities detected by the proposed system, activities recommended and activities followed by the users. It was observed that 'Talking on phone', 'Meeting with friend' and 'Walking' activities were the most favorite activities of the user as these activities were detected by the system more than other activities. The user acceptance of recommendations was 43%. With these results, we come to the conclusion that the users found the system really useful, since they followed the recommendations and performed their favorite activities repeatedly.

FUTURE WORK
In future, we will work on the design of applications for IoT devices to obtain activity lifelogs of users and incorporate them for recommendations. We will also work on a friendly user interface that may be employed by the user for viewing the recommendation given by the framework.