Agriculture Data Users Guide: Data Needs

Different types of data and tools will be required depending on the application. Needs may include:

• Understanding historical conditions/trends

• Assessing current conditions and assets

• Identifying appropriate indicators and measures

• Documenting/evaluating programmatic impacts

Historical Trends

A wide variety of data is available for capturing historical trends. Here are just a few sources directly related to agriculture and food. Others can be found on the Data Sources page of this guide.

• Social Explorer - (Cornell users only). Online research tool designed to provide quick and easy access to modern and historical census data and demographic information. Create fast, intuitive, and illustrative maps and reports to help visually analyze and understand demography and social change throughout history.
• USDA Census of Agriculture Historical Archive

Assessing Needs and Assets

Accurately assessing existing conditions and assets is an important step in any agriculture and food system initiative. These efforts may include data gathering in support of asset based community development.

• Food System Assessments
• WealthWorks approach is geared toward communities inventorying multiple forms of wealth generating capital, including intellectual, individual, social, natural, built, political, financial and cultural assets.

Metrics, Indicators, and Outcomes

Monitoring programmatic impacts are a common need for agriculture and food systems initiatives, and a growing priority for funders and sponsor organizations focused on outcomes-based programming. As the number of local and regional food systems initiatives has continued to grow, it has become increasingly important that each clearly articulates its intended area of intervention and impact.These efforts can be complicated by a lack of clear understanding about the role of and relationships between “measures”, things that can be measured at a particular point in time (in some cases serving as “proxies” for things that can’t), their function as a valid “metric”, quantifying some particular characteristic, and how these can be used as “indicators” of progress toward desired “outcomes”.

As outlined in the report Charting Growth: Sustainable Food Indicators from the Wallace Center indicators (statements about positive change) should do the following:

• be measurable
• be relevant to the attributes of interest
• address the most important trends and impacts related to these attributes
• be sensitive/responsive to changes over time in physical conditions
• be hierarchical (providing a clear overview, but amenable to expansion into detail or at finer scales)
• promote learning and effective feedback to decision making

Measures (data supporting the indicators) should be:

• valid and reliable (high quality)
• timely (indicating problems or progress while there is still time to act to prevent negative consequences)
• collected and reported regularly and consistently over a broad geographical range of the US
• publicly available
• transparent and understandable

The following resources offer models for identifying and collecting potentially useful indicator data:

• Community Food Systems Bibliography from UC Davis Agricultural Sustainability Institute. Gathers published literature on local and regional food systems and categorizes the literature by key topics. Includes Assessments & Indicators.
• Food Metrics for NYC
• Maine Network of Community Food Councils Kickstart Guide to Community Food Assessment & Engagement
• National Good Food Network has several resources related to indicators
• The Regional Food Systems Working Group (RFSWG) of Iowa State University Extension and Outreach supports education, conducts research and facilitates partnerships to increase investment and support of community-based, economically sustainable and environmentally and socially responsible regional food enterprises. Goals include identifying and measuring key indicators.
  • Collecting data for collective impact: A guide for coordinators in the Regional Food Systems Working Group  A how-to manual for documenting the economic impact of regional food systems work in Iowa. (PDF)
  • RFSWG Data Collection Guide This manual outlines a shared measurement system for collecting economic data, and includes a sample farmer survey to report local food sales, and a sample institution survey to report local food purchases.
• Yellow Wood Associates has several programs and resources related to indicators

Logic Models

Further complicating matters is confusion regarding the relationship between measures, indicators, outcomes, and broader changes hoped for, or “goals”. And at each step in this chain of logic, assumptions might be made regarding these connections which are not apparent or valid. “Logic models” are sometimes used to represent how an activity (such as a project, a program, or a policy) is intended to produce particular results, helping reveal these assumptions. They show logical relationships among resources invested, activities, and benefits that result, as a sequence of events, usually presented in a graphic, visual representation.

• W.K. Kellogg Foundation Logic Model Development Guide
• UW Extension Logic Model resources page

Theory of Change

A number of programs and methods have been developed to help guide and more clearly articulate this process, and to properly identify the correct indicators needed to measure change resulting from programmatic interventions. Many of these relate to a broad area of work around “Theory of Change”, a methodology designed to help change makers frame and execute successful change strategies, by identifying long-term goals, mapping out requirements for achieving them, identifying areas of intervention, developing indicators associated with those interventions, and developing the ability to “tell stories” that clearly articulate the need and goals of a project, and how the work being done (or proposed) will address those.

Evaluation

As defined by the American Evaluation Association, evaluation involves assessing the strengths and weaknesses of programs, policies, personnel, products, and organizations to improve their effectiveness. Evaluation is the systematic collection and analysis of data needed to make decisions.

The Cornell Office for Research on Evaluation (CORE) offers several useful resources and services related to evaluation, including those specifically geared to the needs of Cooperative Extension. CORE has developed an approach which provides a method for linking multiple logic models together as part of a "network pathway", illustrating lateral, potentially synergistic connections between programs and their outcomes, which may be quite linear in themselves. The Netway is a publicly and freely available tool developed by the CORE supporting collaborative development of program models and evaluation plans.

Other resources include:

• Community Toolbox: Framework for Program Evaluation - A Gateway to Tools
• Developmental Evaluation is a method of evaluation that works with programs as they learn, helping them iteratively identify and develop the most appropriate measures of success. In determining whether to use DE, organizations should consider both whether their intervention is suited to DE, and whether the organization itself is ready to adopt DE. Both issues are covered in the Innoweave DE self-assessment.
• Innovation Network provides program planning and evaluation consulting, training, and web-based tools to nonprofits and funders across geographic and programmatic boundaries
• Kellogg Foundation Resource Directory
• Penn State Extension Evaluation Tip Sheets Short, informative fact sheets on a wide range of basic evaluation topics as well as questions regarding methods and data collection.
• Resources for Methods in Evaluation and Social Research
• University of Kentucky College of Ag, Food and Environment Evaluation Resources
• University of Wisconsin Extension Evaluation Resources  Evaluation is a core mission of University of Wisconsin-Extension Cooperative Extension and its Program Development and Evaluation Unit (PDE).
• What is a Survey -free booklet taken from the American Statistical Association (ASA) series of the same name, designed to promote a better understanding of what is involved in carrying out sample surveys.

Collective Impact

The actual process of indicator development, with/through stakeholder engagement, can be a powerful means of capacity building in itself, helping people and organizations come together to identify common ground and shared goals. This process can be balanced and strengthened by the inclusion of a rigorous approach or knowledgeable experts. “Collective Impact” is one emerging framework for identifying shared indicators, helping guide and link individual projects and communities efforts.

• The Regional Food Systems Working Group of Iowa State University Extension and Outreach has created Collecting data for collective impact: A guide for coordinators in the Regional Food Systems Working Group
• Yellow Wood Associates has developed the You Get What You Measure workshop, a facilitated group process that integrates systems thinking into strategic planning and evaluation, identifies key leverage indicators, and aligns actions to achieve goals.

Networks

A number of highly innovative food and agriculture related projects are using a network based approach. Various network analyis tools (e.g. Social Network Analysis, or SNA) including surveys are being used to collect, analyze and visualize data about the nature and health of these networks. Network-centric approaches to community and regional development tend to be more focused on capacity building (e.g. building social capital) rather than immediate and discrete outcomes.

• Report from Michigan State University about the collective impact of the RFSWG, Creating Change in the Food System: The Role of Regional Food Networks in Iowa
• Promoting and Assessing Value Creation in Communities and Networks: A Conceptual Framework -five levels of value creation of a community or network. Making the connection between the activities of a community or network and the improved performance of an organization, institution or even a country.