Summary
The Wikimedia Foundation is looking for a collaborative, analytical, and creative Evaluation Strategist to monitor the outcomes and evaluate the impact of our grantmaking programs. We are seeking someone who is an early to mid-career Program Evaluator with experience designing theories of change, logic models, conducting mixed method applied research, and/or working in global philanthropy.
The Evaluation Strategist will report to the Director of Learning & Evaluation (L&E) and will be part of the Operations Department. The L&E Team supports the intentional use of data and insights to inform decision-making about the Wikimedia movement and Foundation strategy. Our overall vision is to foster a culture of evaluation, reflection, and learning so that the Foundation and Wikimedia Movement are equipped to create meaningful social change.
Please note that this is a remote role that can sit anywhere in or outside of the United States.
You are responsible for:
-
Grants Program Evaluation Planning
- Collaborate with grantmaking staff to develop logic models or theories of change, accompanied by evaluation plans that enable them to monitor progress toward goals, understand effectiveness, and communicate their impact.
- Design approaches to measuring the technical and social impact of our distributed and wide ranging grantee activities, tapping into best practices from evaluation approaches such as equitable, participatory, developmental, collective impact, and empowerment evaluation.
- Support grantees to develop their own evaluation questions and plans.
-
Grantee Data Collection and Analysis
- Design quantitative and qualitative data collection tools that enable program officers and grantees to measure progress toward goals and understand program effectiveness.
- Plan, design, and implement an annual survey for all Wikimedia Foundation grantees, in partnership with key stakeholders at the Wikimedia Foundation and, potentially, an outside survey firm.
- Clean and analyze quantitative and qualitative data collected from grantees, and extract patterns and trends.
- Conduct a retroactive mixed methods analysis of grantmaking impact over the last 3 years.
- Compare year-over-year changes in results, and report on those trends as part of strategic planning processes.
-
Data Management and Reporting
- Assist with the design, creation, and management of an internal reporting system.
- Build and maintain a central database of all grantmaking outcomes, standardizing data, and doing so in a secure way that maintains the privacy of our grantees and the quality of the data.
- Present results in a manner that is relevant to different audiences, utilizing best practices in data visualization.
- Develop a cadence of regular, user-friendly reports for program leaders and staff to monitor their results and make strategic decisions.
- Support the grantmaking team to report on their core metrics and respond to data queries as needed.
- Contribute to an internal culture of learning and evidence-based decision-making among colleagues and program leaders.
Skills and Experience:
- Masters-level degree or equivalent training in an applied research field with a minimum of 3 years experience using a range of program evaluation methods and practices.
- At least one year of experience compiling data from multiple datasets into one spreadsheet or database, cleaning and standardizing the data, and maintaining the quality and privacy of the data over time
- Quantitative data collection and analysis skills, especially with smaller sample sizes or non-parametric data.
- Qualitative data collection and analysis skills, especially in mixed-method settings.
- At least one year of experience creating graphs, charts, and tables that summarize the outcomes of a dataset in concise and thoughtful ways.
- Experience using the Google Suite of tools for distributed collaborative work (i.e. Google docs, Sheets, Slides, etc).
- Excellent verbal and written communication skills.
- A proficient level of written and spoken English, including the ability to communicate clearly with non-native English speakers.
Qualities that are important to us:
- Ability to resolve a wide range of challenges in creative ways and address problems of diverse scope.
- Ability to demonstrate good judgment in selecting methods and techniques for obtaining solutions & prioritizing workload.
- Comfort with innovative, iterative, bottom-up collaborative teams, especially around crowd-sourced projects which rely on volunteer engagement.
- Familiarity with community service-focused, volunteer-based, and/or international initiatives.
- Commitment to equity, inclusion, and diversity. Cross-cultural sensitivity and awareness.
Additionally, we’d love it if you have:
- Experience conducting evaluation in a philanthropy and/or collective impact setting.
- Experience designing surveys and using online survey software, such as Qualtrics, Survey Monkey, Survey Gizmo, or other programs.
- Experience working with online communities, particularly as part of the free knowledge or open source movement.
- Experience living or working internationally.
- Fluency in languages other than English.
- Nonprofit, NGO or INGO experience.
- Passion for or experience contributing to Wikipedia and the other Wikimedia projects.
The Wikimedia Foundation is...
...the nonprofit organization that hosts and operates Wikipedia and the other Wikimedia free knowledge projects. Our vision is a world in which every single human can freely share in the sum of all knowledge. We believe that everyone has the potential to contribute something to our shared knowledge, and that everyone should be able to access that knowledge, free of interference. We host the Wikimedia projects, build software experiences for reading, contributing, and sharing Wikimedia content, support the volunteer communities and partners who make Wikimedia possible, and advocate for policies that enable Wikimedia and free knowledge to thrive. The Wikimedia Foundation is a charitable, not-for-profit organization that relies on donations. We receive financial support from millions of individuals around the world, with an average donation of about $15. We also receive donations through institutional grants and gifts. The Wikimedia Foundation is a United States 501(c)(3) tax-exempt organization with offices in San Francisco, California, USA.
The Wikimedia Foundation is an equal opportunity employer, and we encourage people with a diverse range of backgrounds to apply.
[IF FTE] U.S. Benefits & Perks*
- Fully paid medical, dental and vision coverage for employees and their eligible families (yes, fully paid premiums!)
- The Wellness Program provides reimbursement for mind, body and soul activities such as fitness memberships, baby sitting, continuing education and much more
- The 401(k) retirement plan offers matched contributions at 4% of annual salary
- Flexible and generous time off - vacation, sick and volunteer days, plus 19 paid holidays - including the last week of the year.
- Family friendly! 100% paid new parent leave for seven weeks plus an additional five weeks for pregnancy, flexible options to phase back in after leave, fully equipped lactation room.
- For those emergency moments - long and short term disability, life insurance (2x salary) and an employee assistance program
- Pre-tax savings plans for health care, child care, elder care, public transportation and parking expenses
- Telecommuting and flexible work schedules available
- Appropriate fuel for thinking and coding (aka, a pantry full of treats) and monthly massages to help staff relax
- Great colleagues - diverse staff and contractors speaking dozens of languages from around the world, fantastic intellectual discourse, mission-driven and intensely passionate people
*Eligible non-US workers' benefits are specific to location and dependent on employer of record
