| Home | E-Submission | Sitemap | Contact Us |  
J Korean Gerontol Nurs > Volume 20(Suppl 1):2018 > Article
Mihoko: Conversation Assistive Technology for Maintaining Cognitive Health



There is a need for artificial intelligence which nurtures human intelligence as the prevalence of dementia and collapse of intelligence of human beings has become a social problem. Purpose of this study is to develop intervention technologies for maintaining cognitive health of older adults.


The method named the Coimagination Method (CM) was proposed and has been developed in order to achieve goal. Conversation assistive technologies have been developed and tested based on the method.


The state of the art of the group conversation support system, and regular series of group conversation sessions for full-years with insights for healthy older adults are described in detail. Participatory approach has been applied to the design process for simultaneous research and implementation of the service. Both participants and practitioners have been maintaining their cognitive health for independent living.


Findings imply that there exist potentially preventive types of dementia and intervention should be applicable for such types. Ways of thinking and living are gently intervened through understanding of personal values and broadening minds, which lead to improved quality of life.


A third of older people now die with dementia [1]. Cognitive decline is prevalent in the modern society. Technologies for maintaining and completing intelligence of human have become much more important than ever before. Progress of artificial intelligence make people fear that they may overcome human intelligence [2]. Then, artificial intelligence which nurtures human intelligence should be investigated as well. The mechanism of dementia is still not clear, but aging is known as a major risk factor. The occurrence of dementia increases with age. It is reported that based on the eight large epidemiological studies, dementia prevalence ranges from 18 to 38% among those aged 85 and older, and from 28 to 44% among those aged 90 and older [3]. The decrease of dementia incidence with aging is a challenge and may have impact on society. Since ways of thinking, living and that of using cognitive functions associate with each other, intervention to ways of thinking and living may alter way of utilizing cognitive functions, which may lead to delay the collapse of functioning mainly caused by misuse. The research question is what the ideal way of thinking and living is for maintaining cognitive functions. From observation studies, good lifestyles are known as follows. Lifestyles which have good effects on physiological health are to have meals with antioxidant effects like Mediterranean diet [4], exercises and physical activities which boost metabolism [5]. Lifestyles which have good effects on cognitive health are to be involved in cognitively stimulating activities like reading a newspapers [6], and to keep enough social networks [7]. The author proposed and have been developing a novel intervention method named the Coimagination Method (CM) which may assist good lifestyle for maintaining cognitive health through well-prepared series of interactive conversation [8,9]. In this paper, we describe the development of conversation assistive technology based on the CM with its design process and insights.

The Coimagination Method (CM)

The author proposed the method for enabling simultaneous intervention to intellectual activities and social network. The CM was invented which realize structured conversation for this purpose, since careful participation to conversations may serve as intellectual activities, and conversation itself, by its nature, a foundation of social network. Two rules are added for natural conversation so that participants are able to make use of three cognitive functions, which tend to decline with aging, and decline in particular for older adults with mild cognitive impairment. If the participants follow the rules, prepare and participate the Coimagination sessions carefully, episodic memory, divided attention, and planning functions are expected to be engaged. The first rule is that each participant has equal allocated time period for talk, questions and answers. The main purpose of this rule is to utilize divided attention function. The speakers need to divide attentions to providing topics or answering questions, and elapsed time at the same time for keeping time. The listeners need to divide attentions to understand topics or asking questions, and elapsed time simultaneously as well. The second rule is that participants prepare topics and photos beforehand according to the themes of the conversation sessions. The major objective of this rule is make use of episodic memory and planning functions. The participants plan to go out for finding topics which is suitable for the themes such as “favorite places in the neighborhood” or “small discoveries during or after 10 minutes walking”. In this process, the planning functions should be engaged. In order to describe the topics during conversation session, the participants may memorize the episodes when they take photos, reserve the memories until the day of the session, and recall them when the turn for speech and answering questions. The method was named after the combination of “co”llective and “co”operative imagination. The participants are required to collect images and to share cooperatively with imagination. The ideal way of listening is to imagine the situations and feelings of each speaker. The ideal way of asking questions is to try to fill the gap of understanding between the speakers and listeners.


1. Group Conversation Support System

We have developed conversation support system for the Coimagination sessions. The system supports preparation, moderation, and analysis of the conversation sessions. The software system consists web servers, databases, and general web browsers or original client programs. The hardware consists computers, monitors or sets of projectors and screens, speakers or robots with speakers, tablets or smartphones. The users of the system are researchers, practitioners, and participants. Researchers monitor the usage of the system by practitioners and participants, and analyze the accumulated data. Practitioners prepare and moderate the sessions with the system. Participants take photos with the system implemented as application runs on either tablets or smartphones. In this paper, the system operations run by practitioners are described in detail.
Practitioners register the participants’ information and data of series of conversation sessions. The conversation sessions are defined by numbers of participants, numbers of photos per participant, allocated times for speeches and discussions, themes of the conversation, the names and seating of participants and the submitted photos by participants. Practitioner moderate conversations with a help of the system. The moderator may be either practitioners or robots. If the practitioner run the system as manual mode, the practitioner serve as a moderator and operates the system manually. Following is the procedure to run the system manually.
The seating of the participants are displayed in the beginning (Figure 1 (a)). They click on each of the participants one after another. The menu for selecting buttons of either speech or discussion appear (Figure 1 (b)). The practitioners click on the button of speech on the left. Then, a set of submitted photos are displayed on the screen (Figure 1 (c)). If the numbers of the photos are two for each participants, two photos are displayed on the screen. The practitioner click on a set of displayed photos one after the other. If one of the photos are clicked, the enlarged photos is shown (Figure 1 (d)). Elapsed time and allocated time are displayed on upper left side of the photos. When the time is over for a certain participant, “Time Over” sign may appear. After the first round for speeches of each participant is over, the second round for discussions on each topic provided by participants starts. Again, they click on each of the participants one after another (Figure 1 (a)). The menu for selecting buttons of either speech or discussion appear (Figure 1 (b)). This time, the practitioners click on the button of discussion on the right. After the second round for discussions of each participant is over, the session is closed. If the operator runs the system as automatic mode, the robot serves as a moderator and the system is operated automatically. Then, the robot describe the theme of the session, number of photos and allocated times for each participant.
Once the session starts, the photos are displayed one after the other for predetermined period. The robot turn to each participant one after the other and say “Please talk about the photo starting with when and where you took it”. When the time for each speaker is over, the windows which display photos are closed and the robot says “Thank you for talking”. Then the robot turn to the next speaker. After the first round for speeches of each participant is over, the second round for discussions on each topic provided by participants starts. The photos are displayed one after the other for each participant while the robot turn to each participant and say “Please discuss all together on the photo displayed.” When the time for each photo is over, the robot may say “Thank you for discussion”. After all photos are displayed for two rounds, the session is closed with the robot saying “Thank you for all. The Coimagination session is closed.” The automatic mode has an optional function. If the system run as an automatic mode with a feedback function, the system monitors amount of speech of each participant and balances the amount of speech by assisting turn taking. If the total amount of speech of a particular participant is large, the robot asks the participants to stop talking by saying “Thank you very much.” The robot points out the participant whose amount of speech is smallest and ask the participant to start talking bay saying “How about you?” The algorithm for balancing the amount of speech was proposed and its parameters are calculated from the collected conversational data [10]. The setup of the conversation support system running as an automatic mode with a feedback function is shown in Figure 2. Participants are wearing microphones to collect data of the amount of speech. A photo provided by one of the participants are displayed on the screen. The robot is moderating in front of the screen facing all participants.

2. Design Process of Services and the Insights

We take participatory approach for designing and improving the CM and its assistive technologies and service programs [11]. The author founded research institute at the initial stage of this study in 2007. The name of the institute is “Fonobono Research Institute” where fonobono means heartwarming in Japanese. The purpose of the foundation of this institute is to be served as a platform for participatory research towards realization of society without preventive dementia by developing and providing heartwarming technologies and services which promote good lifestyle among friendly communities. The research institute has been registered as nonprofit organization of a local government since 2008, so that the practitioners and participants can belong to the organization. The initial practitioners are healthy older adults who were originally participants of the Coimagination sessions. Those who get interested in practices and researches are joined.
Practitioners provide regular series of sessions for full-years since 2011. Some of the initial members are over 90 years old in 2018, and they are active participants and practitioners. Both participants and practitioners are maintaining their cognitive functions necessary for independent livings. The group conversation support system has been tested and its usability has been progressing through the sustained usages. We exploratory investigate the long term effect of participating The Coimagination sessions on their cognitive lifestyle. One of the effects is a homework effect. Participants of regular series of sessions are always thinking about the themes, taking photos and finding topics like a homework for the next session. Participating the conversation session is like checking the answers, although the numbers of the answers are the same as those of the participants. Similar photos are provided by different participants sometimes. Totally different photos are provided by participants sometimes although the theme of the session is common. Through sharing the topics with photos to the same theme, participants can broaden their minds which may diverse the way of thinking and living.
We have been investigating the effective themes which may involve cognitive functions as much as possible, as well as improving quality of life, through thinking about the themes, taking photos and finding topics according to the themes, and sharing topics with photos among participants. Some themes are proposed by the practitioners. Other themes are derived from the discussions with researchers in diverse research areas. Here we describe the tested and popular set of themes based on the idea of advanced care planning of healthy older adults in the context of the end of life care. Following themes were proposed through the discussion with nursing professionals and practitioners: meals in the end, things to be discarded, things difficult to be discarded, and things to be left behind. The photo in Figure 3 is one of the photos provided by one of the participants for the session whose theme was meals in the end. The participants mentioned that the mineral water in the bottle was obtained from the source of spring nearby his hometown. He wished to have the water for the drink in the end. The participants could imagine their end of lives in a positive way. They could understand their personal values through thinking about the themes which may help the decisions of the ways of living and dying. The memorable comment of one of the participants was that “We prefer to think about the enjoyment of life rather than the end of life.” In the comment, the idea of the enjoyment of life and the end of life are opposed to each other. From other comments by participants suggest that thinking about the enjoyment of life and the end of life does not conflict with each other. Rather, they could enjoy thinking about the things to be discarded. One of the typical comments is as: It was a good opportunity to think about the things to be discarded. I got to know through the conversation that we had difficult times for cleaning up because there are so many unwanted things and things to be left behind. The experience was helpful to understand special things of myself and difference with others.


The CM has been applied mainly to community-dwelling healthy older adults. The related method is Reminiscence therapy [12]. It involves the discussion of memories and past experiences with other people using tangible prompts such as photographs or music to evoke memories and stimulate conversation. Reminiscence therapy is implemented widely in a range of settings using a variety of formats. We can apply the CM to Reminiscence therapy since the CM provides conversation format, which is not defined by the kinds of topics. Although the CM is possible to be combined with Reminiscence therapy, the focus of the CM is on present and future. The strength of the CM is that participants can update their memories forwardly to create their future life history from present and accumulate their memories until the very end of life if their memory functions remain. The limitation of the method results from the same reason that some of the steps such as taking photos cannot be done if participants have already lost their cognitive functions. In such cases, some of the steps are omitted as well as their expected effects are lost. We have applied the method experimentally to people in various situations for the purpose of making clear the limitations and applications of the method, and required adaptation to various individuals [13]. Protocol of the Coimagination sessions has been customized to meet the cognitive level of participants. We collaborated with care facilities, hospitals, and care prevention facilities. The practitioners of each facilities are occupational therapists, clinical psychotherapists, nurses, children nurses, care givers, and healthy older adults. The participants range from healthy older adults, older adults with mild cognitive impairment, and older adults with mild to moderate dementia. The first round for speech is omitted or the allocated time for speech is kept shorter for people with dementia because they forget the photos even if they took them by themselves. Only manual mode has been applied to such people. Application of automated mode of the system with assistive optional functions is one of the technological challenges. Two types of support systems are under development. One is the remote conversation support systems which share the same images simultaneously at different sites for older adults who have difficulty in going out of their homes. Another is the dialogue support systems where the robot provides topics with images and accepts questions and give answers for older adults who do not prefer social contact with people but would like to maintain their cognitive health.


This paper described the development of conversation assistive technology based on the CM for the purpose of maintaining cognitive health to prevent age-related cognitive decline and dementia. State of the art of the group conversation support system was demonstrated. It has been developed and been progressing successfully. Moderation by the robot with the functioning of balancing the amount of speech enabled standardized implementation of services based on the CM. Participatory approach has been applied to the design process for simultaneous research and implementation of the service. Both participants and practitioners have been maintaining their cognitive health and independent living. The other types of implementation of the services and systems are also referred and discussed. We convince that there exist the potentially preventive types of dementia through practices, which is also pointed out from meta-analysis [1], and intervention should be applicable. The ways of thinking and living are gently intervened through understanding personal values and broadening minds, which lead to improve the quality of life. Future work includes but not limited to: randomized controlled trials in a variety of settings would be tested so as to decompose the key factors for intervention; Systems and robots which enable implementation of the CM in person so that anyone who are willing to maintain cognitive health may make use of them at anytime, anywhere; The relationship between personalities and cognitive health needs to be studied for individually adapted systems and services to maximize the effect of intervention.


1. Livingston G, Sommerlad A, Orgeta V, Costafreda SG, Huntley J, Ames D, et al. Dementia prevention, intervention, and care. The Lancet. 2017;390(10113):2673-733 https://doi.org/10.1016/S0140-6736(17)31363-6.
2. Brynjolfsson E, McAfee A. Race against the machine: how the digital revolution is accelerating innovation, driving productivity, and irreversibly transforming employment and the economy. Lexington, Massachusetts: Digital Frontier Press; 2012.

3. Gardner RC, Valcour V, Yaffe K. Dementia in the oldest old: a multi-factorial and growing public health issue. Alzheimer's Research & Therapy. 2013;5(4):27 https://doi.org/10.1186/alzrt181.
crossref pmid
4. Valls-Pedret C, Sala-Vila A, Serra-Mir M, Corella D, De la Torre R, Martínez-González MÁ, et al. Mediterranean diet and age-related cognitive decline: a randomized clinical trial. JAMA Internal Medicine. 2015;175:1094-103 https://doi.org/10.1001/jamainternmed.2015.1668.
crossref pmid
5. Sofi F, Valecchi D, Bacci D, Abbate R, Gensini GF, Casini A, et al. Physical activity and risk of cognitive decline: a meta-analysis of prospective studies. Journal of Internal Medicine. 2011;269:107-17 https://doi.org/10.1111/j.1365-2796.2010.02281.x.
crossref pmid
6. Wilson RS, Mendes de Leon CF, Barnes LL, Schneider JA, Bienias JL, Evans DA, et al. Participation in cognitively stimulating activities and risk of incident Alzheimer disease. JAMA. 2002;287:742-8 https://doi.org/doi:10.1001/jama.287.6.742.
crossref pmid
7. Fratiglioni L, Wang HX, Ericsson K, Maytan M, Winblad B. Influence of social network on occurrence of dementia: a community-based longitudinal study. Lancet. 2000;355(9212):1315-9 https://doi.org/10.1016/S0140-6736(00)02113-9.
crossref pmid
8. Otake M, Kato M, Takagi T, Asama H. The coimagination method and its evaluation via the conversation interactivity measuring method. IGI Global. 2011;356-64 https://doi.org/10.4018/978-1-60960-559-9.ch043.
9. Otake M, Nergui M, Otani T, Ota J. Duplication analysis of conversation and its application to cognitive training of older adults in care facilities. Journal of Medical Imaging and Health Informatics. 2013;3(4):615-21 https://doi.org/10.1166/jmihi.2013.1212.
10. Yamaguchi T, Ota J, Otake M. A system that assists group conversation of older adults by evaluating speech duration and facial expression of each participant during conversation. 2012 IEEE International Conference on Robotics and Automation. 2012. 4481-6 https://doi.org/10.1109/ICRA.2012.6225283.
11. Otake M, Kato M, Takagi T, Iwata S, Asama H, Ota J. Multiscale service design method and its application to sustainable service for prevention and recovery from dementia. In: Onada T, Bekki D, McCready E, editors. New frontiers in artificial intelligence. Proceedings of the InJSAI International Symposium on Artificial Intelligence. 2010 Nov 18; Berlin, Heidelberg: Springer; 2011. pp. 321-30 Berlin, Heidelberg: Springer; 2011. p. 321-30. https://doi.org/10.1007/978-3-642-25655-4_31.

12. Woods B, O'Philbin L, Farrell EM, Spector AE, Orrell M. Reminiscence therapy for dementia. Cochrane Database Systematic Review. 2018;3:CD001120 https://doi.org/10.1002/14651858.CD001120.pub3.

13. Otake M. Application of co-imagination method to healthy older adults, older adults who need care, and older adults with dementia. Gerontechnology. 2014;13(2):119-20 https://doi.org/10.4017/gt.2014.

Figure 1.
Series of windows showing the seating order and photos.
Figure 2.
The setup of the group conversation support system running as an automatic mode with a feedback function.
Figure 3.
One of the photos provided for the session whose theme was meals in the end.
Editorial Office
College of Nursing, Korea University
145 Anam-ro, Sungbuk-gu, Seoul 136-713, Republic of Korea
Tel : +82-41-550-0403   Fax : +82-41-550-2314   E-mail: mossjon78@hanmail.net
About |  Browse Articles |  Current Issue |  For Authors and Reviewers
Copyright © by The Korean Gerontological Nursing Society. All rights reserved.     powerd by m2community