THEORY OF CHANGE IN DEVELOPMENT: A MICRO LEVEL EVIDENCE FROM BANGLADESH

The theory of change is an effort to find ways of evaluating processes and outcomes in community-based programmes that were not adequately addressed by existing approaches. In generating this theory, steps are taken to link the original problem or context in which the programme began with the activities planned to address the problem and the medium and longer-term outcomes intended. This framework has much in common with the development programmes and interventions.1 Connell 

Chowdhury Golam Kibria is Associate Professor, Institute of Business Administration, Jahangirnagar

University, Savar, Dhaka. His e-mail address is: jibonchowdhury@yahoo.com. Mahfuz Kabir, PhD is Senior
Research Fellow, Bangladesh Institute of International and Strategic Studies (BIISS), Dhaka. His e-mail
address is: mahfuzkabir@yahoo.com. K. M. Zahidul Islam, PhD is Associate Professor, Institute of Business
Administration, Jahangirnagar University, Savar, Dhaka. His e-mail address is: jahid558@yahoo.com.
©Bangladesh Institute of International and Strategic Studies (BIISS), 2016.
1
 R. Pawson and N. Tilley, Realistic Evaluation, London: Sage Publications,1997; C.H. Weiss, “Nothing as
Practical as Good Theory: Exploring Theory-based Evaluation for Comprehensive Community Initiatives for
Children and Families”, in J. P. Connell, A. C. Kubisch, L. B. Schorr and C. H. Weiss (eds.), New Approaches to
Evaluating Community Initiatives: Concepts, Methods and Contexts, Washington, DC: Aspen Institute, 1995.

and Kubisch2 define a theory of change approach as a systematic and cumulative study of the links between activities, outcomes, and contexts of the initiative. This definition suggests that the first step to evaluate an intervention towards determining its intended outcomes, the activities it expects to implement to achieve those outcomes, and the contextual factors that may have an effect on implementation of activities and their potential to bring about desired outcomes.

Connell and Kubisch3 also argue that a theory of change can sharpen the planning and implementation of an initiative. An emphasis on programme logic or theory during the design phase can increase the probability that stakeholders will clearly specify the intended outcomes of an initiative, the activities that need to be implemented in order to achieve them, and the contextual factors that are likely to influence them. Moreover, with a theory of change approach, the measurement and data collection elements of the evaluation process will be facilitated. It requires stakeholders to be as clear as possible about not only the final outcomes and impacts they hope to achieve but also about the means by which they expect to achieve them. This knowledge is used to focus scarce evaluation resources on what and how to measure these key elements. Finally, and most importantly, articulating a theory of change early in the life of an initiative and gaining agreement about it by all the stakeholders helps to reduce problems associated with causal attribution of impact. 

 This study is an attempt to map out the changes brought about by the “Empowerment of the Poor through Integrated Agriculture” (EPIA) programme implemented at Bauphal upazila in Patuakhali district, which is one of the most disaster-stricken, vulnerable and marginalised coastal areas of Bangladesh. The project is a poverty reduction initiative implemented from 2008 to 2013 when it enrolled around 3,000 households as beneficiaries and provided them with training, technical support and credit. The households were selected in three batches, around 1,000 in each batch, in each year from 2008-09 to 2010-11. The project was an integrated approach in agriculture combining family poultry, small scale aquaculture and homestead gardening. The project supported establishment of a poultry rearing model, a sustainable family level fish culture and homestead gardening by improvement of traditional practices with technical support, scientific knowledge and local resources. The project facilitated transfer of technology to about 3,000 poor and subsistence level beneficiaries in the upazila. The project is conceived of as an innovative and experimental one and therefore, its area of operation has been limited to only Bauphal upazila for ease of intensive supervision and technology transfer to bring sustainable “change” in the 2

J. P. Connell and A. C. Kubisch, “Applying a theory of change approach to the evaluation of comprehensive

community initiatives: progress, prospects, and problems”, in K. Fulbright-Anderson, A. C. Kubisch and J.
P. Connell (eds.), New Approaches to Evaluating Community Initiatives: Theory, Measurement, and Analysis
(Volume 2), Washington, DC: The Aspen Institute, 1998.
3
 Ibid 

lives of its beneficiaries so that the good practices and lessons of the project can be replicated.

The broad objective of the study is to assess whether there had been noticeable change in the lives of the beneficiaries as a result of the interventions of the EPIA project. Specifically, the objectives of the evaluation were to assess the change in family income of the beneficiaries; the level of empowerment of women and change in gender parity; the impact of awareness, training and vaccination on the mortality of poultry and livestock; the change in nutrition of beneficiaries as a result of production of vegetables, fishes, eggs and meat; the degree of employment generation and self-employment as a result of project interventions; and the change in the livelihood of direct participants of local community through income generation activities in homestead gardening, fish cultivation and poultry rearing. Thus, this study aims at assessing changes that might have occurred in the lives of the beneficiaries by comparing their condition in the year 2011-12 with their condition in the year of enrolment as an application of the theory of change. 

The paper has been organised as follows. After this brief prelude, section 2 reviews the literature on the theory of change to connect it with the present study. Section 3 outlines the methodology of the study that includes study area, sample size and data. Section 4 presents the study findings and analyses to map out the changes happened during the project period. Section 5 concludes the paper.

Currently, the theory of change is being adopted considerably to comprehend the socio-economic changes resulted from development interventions usually through introducing programmes and projects. The main objective is to understand whether, to what extent and in which areas the interventions have brought significant changes along with the qualitative dimensions of the changes. As Vogel4 argues, the theory of change approach has stemmed from both evaluation and informed social practice, and has become a mainstream and popular discourse, tool and approach for assessing a development intervention. Given a set of assumptions related to explain the process of change, the theory helps ascertain the multi-dimensional changes going beyond the traditional static ‘programme theory’ and incorporate a more reflective and adaptive understanding of change.5 As James6 suggests, “A Theory of Change is an ongoing process of reflection to explore change and how it happens — and what that means for the part we play in a particular context, sector and/or group of people.” 

4

 I. Vogel, Review of the use of “Theory of Change” in International Development, London: DFID, 2012. 5
 C. Valters, Theories of Change: Time for a Radical Approach to Learning in Development, London: ODI, 2015. 6
 C. James, Theory of Change Review, London: Comic Relief, 2011.

Using a theory of change approach includes the use of research tools, quantitative or qualitative or both. However, its objective is much broader than merely the tools by reflecting a desire to embed a critical and adaptive approach to development thinking and practice to capture the micro, meso or macro level changes due to a development programme.7 Therefore, it can be readily adopted to good practices in a range of programmes and projects implemented by different organisations for scaling up and wider replications. As Stein and Valters8 describe, theories of change satisfy a range of objectives that incorporates basic four: strategic planning, communication, accountability and learning. Beyond project or programme, a theory of change can be adopted at macro, sectoral and institutional level to make out and strengthen the ongoing interventions.9 However, it can be best applied at micro or local level to explicitly and critically gauge the change processes at each level. The main difference between log-frame analysis and theory of change is that the earlier reflects a blueprint or ‘controloriented’ assessment of programme or project,10 while the later adopts a more process-orientated approach.11 

Theory of change can foster assessing a development in two inter-linked ways. First, it provides enough space and freedom to make assumptions for uncovering the reasons behind a change through devising quantitative and qualitative techniques as research tools by providing the scope of revisiting the context of study and addressing the knowledge gaps. Second, it encourages applying critical reflection on both the specific (changing) context and how programme rationales and strategies fit into this as it is an approach to development thinking and practice. Thus, it is consistent with current development narratives and discourse that development initiatives should be continuously adaptive and take account of political context and social complexities.12 As Booth argues, the notion of adaptive or process-oriented assessment is pertinent and is in line with the mainstream development thinking and practice.13 It bridges gap of the conventional programme management tools, which tend to ignore ‘process elements’, treating projects as ‘closed, controllable and unchanging systems’.14 In this way, theories of change can promote undertaking new approach of assessment by paying attention to the oft-forgotten assumptions linking programme or project  

7

 D. Stein and C. Valters, “Understanding “Theory of Change”’, in International Development: A Review of
Existing Knowledge, London: JSRP and The Asia Foundation, 2012. 8
 Ibid.
9
 James (2011), op cit. 10 D. Booth, “Still Watering White Elephants? The Blueprint Versus Process Debate Thirty Years On”, in A.M.
Kjær, L. Engberg Pedersen and L. Buur (eds.), Perspective on Politics, Production, and Public Administration
in Africa: Essays in Honour of Ole Therkildsen, Copenhagen: Danish Institute for International Studies, 2015. 11 I. Retolaza, Theory of Change: A Thinking and Action Approach to Navigate in the Complexity of Social
Change Processes, New York: Hivos, UNDP and Democratic Dialogue, 2011; I. Vogel, op. cit. 12 L. Wild, D. Booth, C. Cummings, M. Foresti and J. Wales, Adapting Development: Improving Services to the
Poor, London: ODI, 2015. 13 D. Booth, op. cit. 14 D. Mosse, “Process-Orientated Approaches to Development Practice and Social Research”, in D. Mosse, J.
Farrington and A. Rew (eds.), Development as Process, New York: Routledge, 1998. 

activities and outcomes, and by extending a broader perspective of ‘learning process’ approach, which is flexible and adaptive in nature.

Bangladesh is widely known as a ‘social laboratory’ where thousands of programmes and projects are being implemented to bring about positive economic and social changes.15 In the present study, theory of change has been operationalised for assessing EPIA programme as ‘structured experiential learning’, through adopting quantitative method in the implementation area. A range of socio-economic variables are taken to understand the change due to programme intervention and to assess whether the changes are positive, significant and encouraging in nature. It paves the way for taking initiatives to scale up the programme and replicate widely to reestablish Bangladesh as a successful ‘social laboratory’.

The research design for the evaluation followed the pattern of a quasiexperimental research design where conditions of EPIA beneficiaries before and after their enrolment to the project were compared. The population of the study consists of the beneficiaries selected by the EPIA project. The selection was done in three phases, in each year of the period from 2008-09 to 2010-11. There have been cases of drop outs from the selection as well. The distribution of selection and dropout by year of selection is shown in Table 1. About 83 per cent of the households selected as beneficiaries of EPIA has been continuing their projects.  

 15 A. Rahman, M. Kabir and A. Razzaque, “Civic Participation in Sub-National Budgeting”, in A. Shah (ed.),

Participatory Budgeting, Vol. III, World Bank, Washington, DC, 2007, pp. 1-29. 

p = Percentage of population picking a choice, expressed in decimal

c = Confidence interval, expressed as decimal (e.g., .04 or ± 4 percentage points) I. 

In our case, although 210 interviews were taken, only 161 interviews were considered as effective as in the case of the rest, the associated baseline records were unable to furnish clear or meaningful information. So, in our case, 

SS = 160
z = 1.96 for a 95 per cent confidence level
p =0.5 (50 per cent)  

Therefore, we can use the following formula to determine confidence interval for this sample size: 

C = z2 p(1-p)
ss


(1.96)2
(0.5)(0.5)
161
 = 0.772 (≈ 8 per cent) 

Therefore, the resultant means of this study might be considered valid within ± 8 per cent interval with 95 per cent confidence level.

This section presents results showing the changes in selected indicators that the beneficiaries have experienced due to their involvement in the project. The difference between their baseline condition and immediate past year’s condition was also tested for statistical significance. The results are presented here separately for 2008-09, 2009-10 and 2010-11 batches.

Table 2 presents the changes in the socio-economic conditions of the EPIA participants between baseline year (2008-09) and survey year (2012) 

x and x are sample means of the participants in the base year (2008/09) and comparison year (2012), respectively, and n1 ,n2 represent project participants in the current year (2012) and base year (2008/09), respectively.

Four assets of the EPIA project beneficiaries have been used to assess the impact on assets which are land, pond, chicken and livestock. The pattern of impact has been presented in the above Table 2. It is seen from the table that households’ land and number of ponds used for fish cultivation have increased significantly. One plausible explanation of this significant positive change over the years is that households could increase their operated land either through buying or taking lease or by sharecropping. During the period, households’ land increased by 58 per cent from 46.43 decimal to 73.83 decimal land.16 Fish cultivation has been increasing over the years and the project beneficiaries are increasingly interested to bring their fallow land under fish production. The intensive supervision and technical support of the EPIA project trainers could be a significant factor towards this significant positive change. The other two categories of assets namely number of chicken and livestock did not change virtually over the years. 

During the project period, not only the household assets but also their incomes from different sources namely incomes from crop, vegetable, fish and livestock have 

16 Traditionally, farmers in the study areas use decimal to measure the land. 247 decimals = 1 hectare. 

increased significantly. The average household income for the beneficiary increased from 2010-11 to 2012 by more than 57 per cent for crops (from Tk. 9,766 to 15,389), 317 per cent for vegetables (from Tk. 534 to 2,226), 1100 per cent for fish and 303 per cent for livestock. However, the income from chicken displayed a negative change although not statistically significant. A large number of respondents mentioned that chicken died out of some diseases in 2012 that contributed towards this negative change of the chicken incomes compared to base year 2008-09. Overall, farm-income increased by 74 per cent (from Tk. 17,993 to 31,376). The major non-farm income categories such as income from job, shop, wage labour, money-lending etc. also increased to a large extent. It has been observed that the average non-farm income increased by 36 per cent (from Tk. 69,366 to 94,238). For the project participants, the yearly average income from all sources registered an impressive increase of 41 per cent (from Tk. 88,116 to 124,201). 

Not only the household income but also their expenditures for different categories namely expenditures for crop, vegetables, and livestock have increased significantly. The average household expenditures for the beneficiary from 2008-09 to 2012 increased by more than 163 per cent for crops (from Tk. 4,332 to 11,404), 80 per cent for vegetables (from Tk. 513 to 925). Overall, farm expenditures increased by 66 per cent (from Tk. 8,956 to 14,870) while the overall non-farm expenditures increased by 14 per cent (from Tk. 66,691 to 76,233). It has been observed that the average yearly expenditures of the project participants increased by 17 per cent (from Tk. 75,647 to 91,117). It is also indicated by the responses of the project beneficiaries during the focused group discussion (FGD) that their expenditures for different household categories such as expenditures for food, clothing, education, medical care and others have increased compared to baseline year 2008-09. 

It was found that the EPIA project participants increased the proportion of land available for vegetable cultivation. Although not statistically significant, the project participants increased the proportion of available vegetable land under cultivation by 44 per cent (0.25 to 0.36 decimal). This indeed is a clear testimony to the positive attitudes of the project participants for vegetable cultivation.

There has been a significant improvement of the households in terms of poultry rearing. On average the project participants increased the local chicken rearing by 44 per cent (from 9 to 16), yearly egg production by 66 per cent (from 92 to 153), yearly flesh production by 200 per cent (from 6 kilogram to 18 kilogram), yearly expense for chicken rearing by 229 per cent (from Tk. 350 to 1,150) and income from local chicken rearing by 144 per cent (from Tk. 1,169 to 2,849).

Like other indicators fish production, land under fish production, yearly expense incurred for fish production and yearly income from fish production have increased significantly. On average, fish production areas increased from 7 to 13 decimal, fish production increased from 10 to 113 kilograms, expenditures for fish from Tk. 481 to 4,750 and the yearly income from fish from Tk. 718 to 10,174. A plausible explanation for this significant increase could be that many project beneficiaries are becoming more interested for fish production and they are bringing fallow pond areas under production. It has been also found that many project beneficiaries are taking lease for commercial fish production.

Again, four similar assets (viz. land, pond, chicken and livestock) of the beneficiaries are considered to assess the impact of the project on asset building. Table 3 describes the numerical changes by each asset, which portrays that households’ chicken production has increased significantly in 2012 compared to that of 2009-10. Between the period, households’ poultry production increased by 19 per cent per year, which was statistically significant. Other assets have also witnessed positive changes but these are not statistically significant.

In addition to household assets, income from crop, vegetables, fish and livestock increased significantly between 2009-10 and 2012. Average household income for the beneficiary increased by about 253 per cent for crops (from Tk. 3,565 to 12,588), 400 per cent for vegetables (from Tk. 352 to 2,053), 211 per cent for chicken and 148 per cent for fish (from Tk. 1,379 to 3,422). Overall, farm income increased from Tk. 6,695 to 27,570, which was quite large. The major non-farm income, such as income from job, shop, wage labour, money lending, etc. also increased considerably. It has been observed that the average non-farm income increased by 19 per cent (from Tk. 65,361 to 77,767) and the annual average income from all sources increased substantially by 57 per cent (from Tk. 67,220 to 105,337). It means that the participants gained notably in terms of increased income from different components of the project. 

In the expenditure side, household spending for crop, vegetables and livestock increased significantly during the project period. Specifically, average household spending for the beneficiary increased by more than 24 per cent for crops (from Tk. 5,745 to 7,138), 360 per cent for vegetables (from Tk. 195 to 898), 218 per cent for chickens (from Tk. 333 to 1,059), and 126 per cent for fish (from Tk. 700 to 1,585). Overall, farm level spending increased by 68 per cent (from Tk. 7830 to 13,174) while the overall non-farm spending increased by 31 per cent (from Tk. 52,105 to 68,040) even though the magnitude of non-farm spending was significantly higher. Overall, the average annual spending of the beneficiaries increased by 36 per cent (from Tk. 59,935 to 81,214). The FGD with the project participants reveal that their expenditures for various household purposes, such as expenditures for food, clothing, education, medical care and others have increased considerably compared to that of 2008-09. 

The proportion of land under vegetable cultivation in total operated area has increased for project participants. The project participants increased the proportion of land under vegetable cultivation by 27 per cent (from 0.51 to 0.65 decimals), which demonstrates that a positive attitude emerged among the project participants for vegetable cultivation during the entire project period. Moreover, there has been a notable improvement of the households in poultry rearing. Average annual local chicken rearing has increased by 15 per cent (from 13 to 15), meat by 25 per cent (from 9 to 12 kilogram), yearly expense for chicken rearing by 278 per cent (from Tk. 303 to 856) and income from local chicken rearing by 207 per cent (from Tk. 798 to 2,443)among the project participants.

Among the fishery indicators, fish production, cultivation land for fish production, yearly expense incurred for fish production and yearly income from fish production increased significantly. As shown in Table 3, the area of annual total fish production increased from 5 to 6 decimal, while fish production increased from 15 to 86 kilograms, spending for fish from Tk. 546 to 1,339 and annual income from fish from Tk. 1,003 to 5,274. All these changes happened perhaps due to the fact that increasing number of project participants are becoming more interested to fish production by bringing fallow pond areas under production and taking lease to commercially cultivate fish keeping in mind the commercial viability of the fishery sector. 

In the last comparing year of the project, the changes in asset building have not been encouraging for land, pond, chicken and livestock. It is seen from Table 4 that households’ chicken and number of livestock have increased over the years. However, the change is not statistically significant. On the other hand, household land area and number of ponds decreased over the years. It may be because the participating households have already reached their optimum in these assets. 

Conversely, incomes from crop, vegetables, fish and livestock have increased significantly. The average household income for the beneficiary from 2010-11 to 2012 increased by more than 137 per cent for crops (from Tk. 5,331 to 12,640), 241 per cent for vegetables (from Tk. 1,097 to 3,739), 300 per cent for fish (from Tk. 1,298 to 5,219) and the income from chicken increased by 286 per cent. Overall, farm income increased considerably by 233 per cent (from Tk. 10535 to 35,088). The most important sources of non-farm income, e.g., income from job, shop, wage labour, money lending, etc. also demonstrated increase quite visibly. The annual average non-farm income increased from Tk. 8,472 to 76,683. Overall, the annual average income from all sources witnessed a growth of 488 per cent (from Tk. 19,006 to 111,771). This high relative growth of income is not surprising. For example, Institute of Microfinance (InM) in its recent occasional paper reported that their project participants could increase their incomes by 110 per cent in two years.17 Our findings conform this finding of the project. 

The spending side also shows a similar fashion. The expenditures for crop and livestock have increased significantly during this period. The average household expenditures for the beneficiary from 2010-11 to 2012 increased by more than 155 per cent for crops (from Tk. 2,985 to 7,641), 311 per cent for vegetables (from Tk. 396 to 1,627), and 152 per cent for chicken (from Tk. 1,002 to Tk. 2,530). Overall, farm expenditures increased by 149 per cent (from Tk. 6,609 to Tk. 16,481). The annual average expenditure of the project participants demonstrates notable growth of 63 per cent (from Tk. 61,467 to 100,236). The FGD participants reported that their expenditures for human development, such as food, clothing, education, medical care and others have increased compared to 2010-11. It was also found that the EPIA project participants decreased a little bit the proportion of total land available for vegetable cultivation. Although not statistically significant, the project participants decreased the proportion of land under vegetable cultivation by 1.2 per cent (from 0.81 to 0... 8 0 decimal). 

It is evident from the study that households’ land increased 58 per cent between 2008-09 and 2012 but households’ land did not increase between 2009-10 and 2012 and between 2010-11 and 2012. Rather land and pond have decreased. One plausible explanation of such findings may be that as Bauphal upazila of Patuakhali district is one of the most disaster-prone areas, natural disasters such as cyclone, tidal waves and saline water intrusion in the agricultural land are very common. This upazila observed huge damage and losses of crops and fisheries during the cyclone ‘Aila’ on 27 May 2009. Although the income of people increased, but due to high cost of land and use of the income to meet the damages caused by ‘Aila’, they could not manage investment in acquisition of new land. In many cases, the beneficiaries took lease of land and pond for farm activities and possibly due to the same reasons they could not afford taking lease of the same amount of land and pond that they utilised in the previous years.

One positive impact of the project is the increase in income of beneficiaries. This obviously resulted in increased welfare if not in terms of increased assets but in terms of decreased liabilities. If income continues to increase and natural disasters like ‘Aila’ are less frequent in future, then welfare should increase. However, making a definitive conclusion about the nature and direction of welfare of the beneficiaries that might result from the project is beyond the scope of this study.  

This project mainly focused on agriculture and farm activities which may not be sustainable in the long-run because the project area is vulnerable to climate change and natural disasters. The project could also focus on some non-farm activities specially skill development trainings to the clients and could provide other livelihood assets such as sewing machine, nets, boat, agricultural equipment, rickshaw/van, auto, etc. that would generate sustainable income from non-farm activities.

Overall, the empirical findings suggest that the EPIA project has brought about significant changes among its beneficiaries located in a comparatively backward area of Bangladesh. The year-by-year assessment reveals that considerably positive changes took place in income and asset building of the beneficiaries in a quite short period of time in a rural coastal area where the problems of multiple natural hazards like cyclone and salinity are recurrent phenomenon. The positive changes that took place in income from crop, livestock,  fishery and vegetables indicate that their standard of living has increased, which contributed positively in poverty reduction and socio-economic development of the locality. It is evident from EPIA project that integrated agriculture could increase household income from farm activities. Therefore, the positive outcomes and lessons can be learnt through further in-depth study in the project area, scaled up and replicated widely in other backward areas of the country through necessary customisation. The policy recommendations are as follows: 

• The government could scale up this project concept to other districts to increase income of the poor people.

• Along with farm activities the government could also focus on skill development of the target groups especially young household members.

• The donors should come forward with similar project especially in the coastal and other poverty stricken districts.

• The government should develop and distribute hybrid and saline resistant varieties for agriculture development in the coastal areas. 
 

This study is a systemic effort to apply the theory of change to comprehend the positive and significant changes brought about by the EPIA project at Bauphal upazila, which is a disaster-stricken and vulnerable coastal area. The major findings about the changes that EPIA brought about can be summarised as follows. In most cases, the family income of the families changed due to their involvement in the EPIA project. It was found that increase in income for those with poultry and fisheries project were higher than those with vegetables projects. Poultry projects seemed to be more profitable than the fisheries as it takes less time to produce the output to sell in the market. The nutrition intake of the beneficiaries (through spending on the food items) are now much better as they consume fish, chicken meat, eggs and vegetables more than they used to do prior to enrolment with EPIA. Vegetable intake has increased significantly followed by fish intake. The study finds evidence in favour of increase in income and assets of the beneficiaries. However, evidence could not be established in favour of the contention that the increase in assets was primarily due to their involvement in the projects. It was found that on an average, there has been a dropout rate of around 17 per cent among the beneficiaries selected by EPIA. However, the dropouts are mainly from the earlier two batches of beneficiaries selected in 2008-09 and 2009-10. None has dropped out from the batch of beneficiaries selected in 2010-11 which indicate that selection process has attained efficiency through learning. The results suggest that the good practices and successful areas can be replicated in similar areas for bringing in sustainable changes in developmental outcomes and economic empowerment of rural population in Bangladesh.