objective
Through its flagship project Saksham Haryana, the Government of Haryana has taken a much needed step toward the systemic transformation of education. Working with governance consulting firm Samagra, one of the government’s goals was to diagnose learning levels in in each administrative block of Haryana and to motivate achievement of grade-level, or Saksham, status.
As a part of Saksham Ghoshna in Haryana, block officers nominate their blocks for evaluation. If 80% of students are found to be at ‘grade level’ in Hindi & Mathematics, the block is declared ‘Saksham.’ This initiative was gamified by promoting a competitive spirit and instilling pride in “winning” Saksham status for each level of stakeholder - be it administrative officers, school principals, teachers or students.
Assessment Snapshot
694,564
TESTS
ADMINISTERED
207
BLOCKS
ASSESSED
79%
BLOCKS AT
GRADE-LEVEL
The challenge
Such high-stakes assessments require:
Standardised measurement metrics that are comparable, trackable and easy-to-understand
Rigorous study and assessments design that balances syllabus expectations and ground reality (norm and criterion referenced benchmarks)
Rigorous data collection and processing that ensure absolute data integrity
Outcomes
As an independent evaluator on the project, CGI has assessed 3,47,282 students over seven rounds of assessments. As of February 2019, 94 blocks of 119 blocks and 9 of the 22 districts in Haryana have been declared as Saksham. More importantly, Saksham Ghoshana has been enormously popular at the grassroots level, as it is seen as an opportunity for competition and nurturing local pride. This has transformed conversation at the ground level from “pass marks” to student skills and competencies.
The project is an example of insightful diagnosis leading to effectively implemented solutions and the approach is already being considered by other state governments. CGI will continue to provide data-driven insights through Saksham and Saksham+, driving the future of education in Haryana.
OUR APPROACH
CGI has been the independent evaluator for the program since November 2017. Overall, CGI has assessed 1,09,911 students from Grade 3, 1,09,870 students from Grade 5 and 1,27,501 students from Grade 7 adding to 3,47,282 students as part of seven rounds of assessments.
Well designed assessments, thorough measurement approach, precise data management.
CGI has extensive experience in designing assessments for specific objectives and uses its proprietary PinAcLe ® Scale to standardise the measurements across different populations and track growth over multiple assessments. We also use our proprietary CG Insights ® Analytics platform to generate easy to understand reports for all stakeholders to improve the outcomes in their blocks.
90% of Grade 7 students assessed in Round 6 were found to demonstrate grade-level competency in Hindi.
Tackling sustainability and data collection at scale
Till date CGI has assessed a total of 207 blocks. To tackle the challenge of scale, CGI performed a stratified sampling in every round by randomly picking a minimum of 500 students each of Grade 3, 5 and 7 from at least forty schools in a block.
An image from one of the assessments reports prepared by Convegenius Insights for the Haryana Project.
Staggered, insightful, actionable data reporting
CGI’s assessments cater to any education stakeholder: student, school, blocks and state-wide. Our reports are customised to match the needs of stakeholders at different levels. CGI provides more in-depth information, not just on a student's learning level, but in what specific skills they excel or fall behind. This input has been used by the teachers and government officers to design better interventions.
Maintaining data integrity at scale
CGI has been uncompromising about data-integrity (quality) during test administration and processing with zero-tolerance for data that has been tampered with. CGI ensured data integrity with rigorous training of field staff, a disqualification mechanism for any deviations and an algorithm that uses pattern-check to discard/disqualify false data. We strongly believe in the feedback we provide to our stakeholders and the derived insights and diagnosis are only as good as the data collection process. In a few cases, blocks were disqualified or action was taken to ensure the sanctity of the data.
68 blocks from 21 districts have announced ‘Saksham’ while nearly 13 are ‘Near Saksham.’
Bringing in best-in-class methodologies to frame benchmarks
CGI applied the Angoff Method, an expert recommended approach, which relies on subject-matter experts who examine the content of each test question and then predict how many minimally-qualified candidates would answer the item correctly. We brought together an expert panel of teachers and experts to determine the common learning standards that should comprise grade-levels. This in combination with a statewide normative study to identify norm-referenced benchmark and balancing it with the syllabus based criterion referenced benchmark, made this a unique exercise.
“ We learned a lot from the result analysis that CGI shared with us. [Analysis pointed out the] competencies in which students are not doing well. I called the meeting of teachers in my block to discuss these competencies, address them and then monitored progress.”
Rely on our assessments to make better decisions with data.
We work with a wide range of clients including funders, CSR’s as well as government and private schools.