download PDF
full article:
Community radio has been shown to be a valuable tool in facilitating and encouraging social change and development by producing programs that are community focussed and relevant, addressing community specific issues and concerns1,2. Local voices airing local opinions about local issues and generating community relevant solutions through dialogue is a key feature and purpose of community radio3.
Heartline Bali FM (HLB) was established in 2003 in the rural Balinese village of Tulikup, Indonesia. Although it operates on a commercial radio licence, the station was established with a specific community-oriented focus to positively impact the social and health needs of a village population of approximately 7000 people, and surrounding areas where health and social needs were considered similar by health and government services4.
Tulikup is a part of the Gianyar regency and is located approximately one hour from the central business district of Denpasar in Bali, Indonesia. Although Bali is known for tourism, the village is isolated from this sector with most local employment opportunities coming from local brick making and farming. According to 2008 census data, 1384 households (7.8%) were classified as poor in the Gianyar regency5. Bali is predominantly Hindu, as shown within the Gianyar regency where 96% identify as Hindu5.
In 2004 a needs assessment, using a rapid participatory assessment (RPA), was conducted to identify the health and social needs of the community. The assessment recommended the station introduce health promotion and community development principles to better serve the community through community activities and associated radio programming6,7. Following the recommendations the station implemented a wide variety of activities all addressing different health and social needs. For example on-air, the station worked with local health departments providing information and education about health issues such as hygiene, dengue fever, nutrition and HIV/ AIDS. Off-air activities involved developing a range of community projects to help the poor economic status of the village, increasing community welfare. On-air programming was closely link to off-air programming. In 2007 an impact evaluation was conducted of the 3 year project. The aim of this article is to present the methodology and the results of the evaluation.
Research in rural and developing communities presents a number of unique challenges, often being community specific8. Methodologies for monitoring and evaluation must be community-appropriate, adaptable to local contexts9-11. In the present study the key considerations were the role of the banjar and social system, and the aim of the radio station for empowerment.
Balinese culture is largely based around the entity known as a 'banjar', a traditional Balinese social system12. A banjar divides villages into functional communal structures that require all families and individuals to participate in banjar ceremonies and activities, with all people holding different responsibilities. Banjar leadership is headed by a traditional leader known as a 'klian adat', responsible for everything of a traditional nature, mainly to do with Hindu ceremonies and a civil leader known as a 'klian dinas' who is responsible for all civil administration responsibilities in the banjar13. Often community members stated that the banjar leaders opinion of the banjar was reflective of individual opinions (authors, pers data, 2007). Tulikup consists of 7 banjars4. With approximately 7000 people spread across the 7 banjars in Tulikup, a banjar leader effectively represented approximately 1000 people.
Study design
With due regard to community context and radio station priorities for empowerment, it was decided to evaluate HLB with the participatory qualitative evaluative tool, the Most Significant Change (MSC) technique. The MSC was first developed in 1996 to evaluate a rural project in Bangladesh and since then has been used to evaluate many different projects in many different countries around the world14-16. Dialogical and story-based, the tool promotes community participation while the process of evaluation promotes empowerment as community members decide and evaluate the effectiveness of their project by sharing their personal stories of change. These stories are then analysed, with the assistance of community members and key stakeholders to determine which MSC story most reflects the community opinion on the significant change/s that can be attributed to the intervention.
Two Balinese people, qualified interpreters, assisted the primary researcher because of their fluency in Balinese, Bahasa Indonesian and English languages, and as cultural advisors. Language and cultural skills not only facilitated access to community members but also encouraged higher rates of community participation throughout the process17. Before commencement of the research both translators were trained specifically for the evaluation.
Participants and sampling methods
The choice of convenience sampling of a diverse cross-section of the community was influenced by the nature of the fluid structure of community work patterns, informal social relationships and frequent religious or cultural ceremonies which can impact on the accessibility of participants. In order to minimise any limitations of a convenience sample, additional key informant interviews were conducted with stakeholders representing the seven banjars, effectively reaching a broad section of the community.
The MSC and key informant interviews were conducted with 74 community participants and 7 HLB staff. Fourteen participants represented the village leadership (both government and traditional), with all banjar leaders being interviewed, and six participants represented village-level education, health and police agencies. Key informant interviews with village-based leaders were considered representative of the localised banjar of the leader in keeping with their role and status in the traditional Balinese cultural and social system12. In fact, some community members when approached for an interview stated that their opinion of the community was the same as the banjar leader and advised the research team to interview the banjar leader. Most MSC stories were gathered through informal approaches that took into account community life such as locations where community members gathered for informal discussion and fellowship. On occasion, meeting a group of people in an informal setting, community members chose to interview as a group interview (up to 9 people) instead of individually and stated they were more comfortable with this approach, and that the MSC story gained would reflect the opinions of all involved in the focus group. This was considered acceptable in keeping with the high value that is placed on the community13. As such, of 74 participants interviewed, 39 MSC stories were obtained for analysis.
Data collection methods
Most MSC stories were gathered through informal approaches to people available at locations where community members gathered for informal discussion, such as Bali banjars, a traditional hut. Following social norms, interviews began as general chatting ('mengobrol') resulting in interviews ranging from 5-10 min depending on a participant's response. Interviews were based on three questions:
- Could you please explain your position in the community (job, position in the community etc)?
- Have you seen any changes in the community because of Heartline Bali FM in the last 3 years?
- Out of all of those changes what would you consider to be the most significant?
Could you please explain your position in the community? This question aimed to provide context to the personal story of change to aid understanding during the analysis stage. Throughout analysis, community members could use the background information to assist in determining credibility of the source. Having this approach was considered to assist with research rigour, according to MSC methodology16. While background information was collected, all participants remained anonymous with only information about occupation and community standing collected.
Have you seen any changes in the community because of Heartline Bali FM in the last three years? By asking an open question regarding general changes this aimed to investigate both positive and negative changes experienced within the community. This proved effective as two neutral responses were given (ie I have personally not seen any changes) so the community felt they could express what they had personally experienced. This question was also used to get an overview of all the changes that community members believed were attributable to the radio station.
Out of all of those changes what would you consider to be the most significant? The participant was now required to make a personal judgement on what they believed to be MSC. This was an important stage of the MSC process as it empowered the individual to determine what they considered to be the MSC16.
Data recording
All interviews were recorded on a digital recorder and accessed immediately after each interview. The primary researcher and the interpreter cross-checked participants' comments with an exhaustive process of re-translating the recording to cross-check it with field notes taken during the face-to-face interview. Data loss as a result of translation through 2 languages was minimised by this process of clarification and verification. Further, participants' reactions to the research were observed, documented and discussed with the cultural assistants and used as contextual backing for the thick descriptive stories employed by MSC methodology16.
Consent and ethics
Ethical approval was obtained prior to commencement of research by Edith Cowan University, Perth, Australia. Advice from the cultural advisors / interpreters was followed for socio-cultural procedures used when approaching potential interviewees and requesting their participation in order to allow them to decline an invitation without 'losing face' or respect. Each person interviewed provided verbal informed consent to participate and was advised of their right to terminate the interview at any time18,19. Besides community standing, all participants remained anonymous and were not identified in any transcripts or publications.
Data analysis
Following standardised MSC methodology radio station staff participated in data analysis. The MSC supports such an approach, arguing that rather than contributing to bias, such people provided insightful perspective because of their own participation in community life and understanding context16.
Thirty-nine stories/testimonies were collected and then analysed using the MSC analysis process. The goal with MSC is to obtain a single story which accurately represented participants' views of the most significant change the project had made within the community16. All stories were used in the analysis stage and provided supporting information for the final MSC story.
The 39 stories were distributed between 6 radio station staff (Fig1) and each staff member was instructed to select one story which, in their view, most represented all 6 or 7 stories of change. The choice of each staff member was discussed with colleagues for group verification. None of the 6 stories selected in this stage of the analysis were rejected by the group. In turn, those 6 stories were given to the station manager (Fig1) who had the task of reviewing them and choosing one story.
Figure 1: Most Significant Change at Heartline Bali FM
Results and Discussion
Most Significant Change process
Staff selected 6 stories which they believed were the MSC stories from those that had been distributed (distribution process: Fig1). No story was rejected in the peer review stage by the group because all were accepted as reasonable choices. Each story revealed a broad range of community of the most significant changes brought by the community development approach of the radio station.
Story 1: A banjar leader who stated his belief that the most significant change is that the radio station is helping the community through easily available information on-air and through the social activities off-air within the community
Story 2: A local tour operator who, after the 2004 needs assessment, was assisted by the radio station to generate a tour program that enabled him to bring tourists to Tulikup to see the 'real Bali'. Bringing tourists to the community, the tour operator would introduce the tourists to local families and their businesses. Every family along the tour were given a financial contribution for assisting with the tour. The tour operator would operate between 8 and 10 tours a year, depending on need. The story was selected by the group, as it showed an example of the inventive strategies that the radio station was using to meet local needs, in the case the poor economic status of the village.
Story 3: A local business owner who ran a laundry business and told of the personal changes he had experienced as a result of the radio station. Again to support the economic status of the community, the radio station provided free advertising for the business owner who saw a significant increase in the amount of business that his company experience. The group selected this story again because it showed the inventive strategies of meeting community need and to help community members.
Story 4: Another banjar leader highlighted the positive experiences that he had seen within his banjar. He highlighted the 'pig project', another off-air initiative aimed at addressing local need. As identified, the economic status of the village was a primary concern. The pig project was an income-generating project for vulnerable families in the community, whereby vulnerable families were gifted a piglet and pig pen, which the families would raise and eventually breed. The pig's off-spring would then be sold as a form of income, for new piglets are a valuable asset in the Balinese culture. To ensure sustainability there was an understanding that when piglets were born, two would be returned to the radio station so they could be gifted to further vulnerable families. Recipients highlighted that the income generated from the pigs were used to meet basic needs for the family and education needs for the children20. The group selected this story as it showed inventive strategies for meeting community needs that were also responded to local Balinese culture. The fact that the banjar leader was highlighting the project and its impact on the vulnerable (poor) families in the community (ie pigs being valuable in Bali) was pertinent.
Story 5: A headmaster to the local primary school highlighted his belief that the radio station helped with education for the local children and this was the most significant change he had seen. He stated the radio station was helping with education, with local quizzes, games, education via radio, access to resources such as the library and the 'fun days' which the radio station conducts. The library is an area within the radio station in which a wide range of books and other resources are made available for children to come to the station and use. This safe area/hub is a place in which children are free to come and read and work on their school homework. Fun days are community events that occur twice every 3 months. Children from the village attend the radio station and are involved in many enjoyable activities, including educational quizzes with health messages, and many games. Staff at the station highlighted this story because it showed the radio station working with children rather than focussing on one specific demographic in the community; they also believed the children's education to be very important.
Story 6: This story was a combined interview with a mother and grandmother. They believed the most significant change they had seen within the community was the radio station because it was working to help the poor. They had personally witnessed the radio station in its activities off-air (such as the pig project) and had heard on-air the public service announcements and features, short on-air programs providing information about health and social needs, because they were '24-hour-a-day listeners'.
Most significant change
Story 1 was selected by the station manager as representing the MSC. As it had been offered by a banjar leader and the station manager believed this added credibility to his opinions on the changes that Tulikup had experienced as a result of the radio station. Given the banjar leader's traditional role and status, his view would be considered to be for the banjar12. The banjar leader stated:
The changes in the community with Heartline here are: first the community can get much information here easily from the radio, and second, the radio station helps with the social activities within the community. The most significant change is that the community here have been helped in information, and banjars have been helped through social activities such as the pig project.
In his reference to 'information', the banjar leader was commenting on the positive contribution Heartline had made to community life by broadcasting a wide range of information. Significantly, the radio station had trained community members as volunteer 'community reporters' who gathered interviews from the village. As a result, listeners began to hear familiar voices and local identities comment on such issues as child education and employment, matters relating to religious festivals, the local market and businesses, and other community activities. In addition public health messages for common village concerns became more frequent, such as for diarrhoea, dengue fever, HIV/AIDS, nutrition and family planning.
Social activities
Contributing to the change claimed by the banjar leader were regular and frequent off-air community development programs. Particular reference was made by the banjar leader to the pig project, the radio station's initiative to generate income for vulnerable families mentioned by story 4 in the final 6 stories.
Contributing to the banjar leader's observation of change was a wide-scale village-level community nutrition and health intervention21. A collaboration between the Department of Nutrition Polytechnic of Health Denpasar (DHPD) and Heartline Bali FM the initiative known locally as the 'Desa Binaan' included programs and activities to increase nutrition and health knowledge and change behaviours. Thirty-two community health volunteers (Cadres) were trained, and nutrition-related events were organised with intensive on-air programming from HLB. The project also highlighted the extended relationship between the primary school and the radio station (from story 5) as Desa Binaan reported, teachers at the local primary schools believed the nutrition programs that were run in association with the radio station helped with the health of the children. Significant behaviour change was measured through the nutrition project, with an unexpected outcome being government officials becoming concerned with the health status of the clinically defined malnourished children. They requested resources to assist these children and introduced mechanisms to ensure there are no malnourished children in the region21.
Other initiatives included the library for children on the radio station premises due to school libraries being either inadequate or non-existent, the monthly 'fun days' for school children, and the low-impact guided tour by the licensed Balinese tour operator.
Trust, however, was the MSC reflected in the banjar leader's story. Previous criticism of the radio station had stemmed from distrust that the banjar leaders in the Hindu community had expressed toward the ethno-religious Christian background of the radio station's operators19. The banjar leader's description of the current relationship was: 'Generally...very good, so there is no problem' providing evidence that ethno-religious tensions and misunderstandings had been overcome. This is attributable to the community benefits of the radio station's on- and off-air community programs.
Community members were engaged and empowered by the MSC technique. For example, during data collection the research team approached a community member who immediately declared he did not want to participate. His comment was that he believed he needed to be 'smart' to be involved and that he had to 'speak English'. After the process was explained to him, he happily shared his story which provided rich data relating to his perceived significant change from the radio station's activities. He was impressed that he could be included in a research project and that his story was considered valuable, despite his lack of education. Community participation and empowerment is an important aspect of any evaluation of community development22,23. The MSC technique efficiently and effectively documented the community's belief of what constituted significant change and how that could be attributed to the 3 year radio project. The methodology achieved the aim of engaging community participation through a community relevant approach to evaluation.
Limitations
A convenience sample was necessary to accommodate the nuances of community life. Any limitations of the sampling were controlled by interviewing key stakeholders who, in the context of local cultural dynamics, represented the different banjars/communities. In accordance with MSC methodology, the research was restricted to measuring 'felt impact'. The researchers believe the research methodology was appropriate to the context.
Recommendations
The radio station needs to build on the community's trust and to create strong local partnerships with government and non-government organisations that are working to improve the health and social needs of the community. It is clear that the community valued the off-air programs as much as the on-air programs. A focus of the radio station should be to assist the community of Tulikup, and wider regency Gianyar, by continually developing and implementing on- and off-air coordinated programs that are responsive to local need. Using the radio as a local community hub, the radio station can be a way for the community to come together and develop its own solutions. Community radio achieves this by using local people in the production of the radio, and in the case of HLB using community volunteers. It is essential that HLB continues to seek volunteers to assist with community programming and to ensure that Heartline Bali remains community focussed.
A study using MSC methodology found that a Balinese community judged a local radio station to have made a significant change in community life. Thirty-nine community-generated stories underwent a process of selection, resulting in one being chosen as most representative of MSC. Key-informant interviews supported the evidence obtained through the MSC process and the findings expressed this process.
It is clear that HLB made a positive impact on the quality of life of the people who lived in Tulikup, Bali, through a combination of strategically designed on- and off-air activities based on a community development and community participation approach to radio programming. The MSC evaluation extended and strengthened the participatory dynamic of the 3 year project.
References
1. James R. Foundations in radio programming for health promotion. Perth, WA: Health Communication Resources and School of Public Health Curtin University, 2007.
2. Dagron A. Making waves: stories of participatory communication for social change. New York: The Rockefeller Foundation, 2001.
3. Rolls SB, Narayan D. Empowering communities, informing policy: the potential of community radio. Suva: Quality Print, 2008.
4. Government of Indonesia. Census information of Tulikup. Tulikup, Bali: Tulikup Regional Office, 2007.
5. Badan Pusat Statistik Kabupaten Gianyar. Gianyar dalam angka [Gianyar in figures]. Gianyar, Indonesia: Badan Pusat Statistik Kabupaten Gianyar, 2009 (in Indonesian).
6. Baum F. Community development in health. Oxford: Oxford University Press, 2008; 496-512.
7. Pepall E, Earnest J, James R. Understanding community perceptions of health and social needs in a rural Balinese village: results of a rapid participatory appraisal. Health Promotion International 2006; 22(1): 44-52.
8. Byrne A. Working toward evidence-based process: evaluation that matters. South Orange, NJ: Communication for Social Change Consortium, 2007.
9. Barr A. The contribution of research to community development. Community Development Journal 2005; 40(4): 453-458.
10. Van Vlaenderen H. Community development research: merging communities of practice. Community Development Journal 2004; 39(2): 135-143.
11. Adams J, Witten K, Conway K. Community development as health promotion: evaluating as a complex locality-based project. Community Development Journal 2007; 44(2): 140-157.
12. Warren C. Adat and Dinas: Balinese communities in the Indonesian state. Kuala Lumpur: Oxford University Press, 1993.
13. Falk I, Surata K. 'Real' social capital in Bali: is it different from 'the literature'? Rural Society 2007; 17(3): 308-323.
14. Sigsgaard P. MSC approach: monitoring without indicators. Evaluation Journal of Australiasia 2002; 2(1): 8-15.
15. The India HIV/AIDS Alliance. Stories of significance: redefining change: an assortment of community voices and articulations. New Delhi: India HIV/AIDS Alliance, 2007.
16. Davies R, Dart J. The Most Significant Change (MSC) technique. Hastings, VIC: RJ Davies and J Dart, 2005.
17. Temple B, Edwards R. Interpreters/translators and cross-language research: reflexivity and border crossings. International Journal of Qualitative Methods 2002; 1(2): 1-12.
18. Barrett R, Parker D. Rites of consent: negotiating research participation in diverse cultures. Monash Bioethics Review 2003; 22(2): 9-26.
19. Pepall E. A health and social needs assessment of Tulikup, Bali. Perth, WA: Curtin University of Technology, 2004.
20. Comdev Heartline Bali. Pig Project Report. Gianyar, Indonesia: Heartline Bali FM, 2008.
21. Kayanaya R, Kusumajaya N. Final report: capacity-building and empowerment in health and nutritional knowledge and behavioural change in Tulikup Village. Gianyar Regency, Denpasar: Health Communication Resources, 2006.
22. Hubley J. Communicating Health: putting it into practice. Community Health. An action guide to health education and health promotion, 2nd edn. Oxford: Macmillan, 2004; 201-222.
23. Laverack G. Health promotion practice: building empowered communities. Berkshire: Open University Press, 2007.