RESEARCH ARTICLE

 

Smart Worksheets to probe and support scientific numeracy proficiency of first-year chemistry students

 

 

Joyce Sewry^I; Emily Coyte^II; Leanne Williams^II; Aidan Barker^II; Shubham Suryawanshi^II; Dudley E. Shallcross^{III, IV}; Michael T. Davies-Coleman^{I, IV}

^IDepartment of Chemistry, Rhodes University, Makhanda, South Africa
^IILearning Science, Bristol, United Kingdom
^IIISchool of Chemistry, University of Bristol, Bristol, United Kingdom
^IVDepartment of Chemistry, University of the Western Cape, Cape Town, South Africa

Correspondence

 

 

---

ABSTRACT

The transition to university-level chemistry often reveals numeracy skills gaps that can hinder student confidence and academic success. Online learning tools can help characterise and address these gaps. This study introduces a Scientific Numeracy Smart Worksheet (SNSW) resource to characterise and address the numeracy-based strengths and weaknesses of a large cohort of first-year chemistry students at a South African university. We also investigated student usage and perceptions of this educational resource. The SNSW integrates core mathematical concepts with subject-specific contexts, features high levels of feedback, value randomisation, and an auto-solve feature for struggling students. It was offered as formative support for chemistry students near the start of their university journey. Usage and performance analytics from consenting students were used to study overall and section-level SNSW performance, while an anonymous questionnaire explored student perceptions. Students performed well at 'Displaying numbers' (85%) and 'Rearranging and solving equations' (84%). The lowest scoring sections were 'Graphs' (64%) and 'Averages and spread of data' (72%). 'Scientific units' showed the highest auto-solve percentage. Students who repeated the SNSW scored significantly higher and used the auto-solve feature significantly less on the second attempt (both p < 0.001) and scored higher in quantitative components of their end-of-module assessment, but not significantly so (p = 0.082). The questionnaire indicated high student rating for the SNSW (8.2/10), with most students finding it helpful and at the appropriate level. Providing supportive and diagnostic resources can help students develop numeracy skills and identify areas for personal improvement. Instantaneous data, generated from each student engagement with SNSW, can assist staff to develop educational strategies to target specific transitional skill deficiencies.
SIGNIFICANCE:
Learners at secondary schools and university students alike struggle with basic numeracy concepts, such as ratios and proportions, graphs and SI units. First year chemistry curricula are full, with little time spent on revising content from school. The SNSW is a means of solving this problem. Students work through the worksheets at their own pace and receive immediate feedback. This research highlights both the gaps in students' numeracy skills, and a corrective intervention by first-year chemistry lecturers.

Keywords: chemistry education, scientific numeracy, quantitative skills, Smart Worksheets

---

 

 

Introduction

Globally, the transition from secondary school physical science to university scientific disciplines such as chemistry can be challenging for many students.^1-4 Central to the challenges facing first-year university chemistry students, is an expectation that they enter university with sufficient adaptable mathematical proficiency to cope with the basic mathematical demands of tertiary level chemistry.^2,3 First-year chemistry curricula in South African universities, in common with first-year chemistry curricula at universities around the world, require the application of general mathematical skills (including basic algebra, statistics and elementary calculus) to calculate concentrations, quantify rates of reactions and enthalpy, measure pH, etc.² In this paper, we use the general term 'scientific numeracy' to describe the transferable mathematics skill set, acquired from a secondary school education, that should enable first-year university chemistry students to engage critically with the mathematical content of chemistry in order to solve problems at the interface of mathematics and chemistry.

In countries where the majority of students enter English-medium universities from multilingual and diverse cultural backgrounds, such as South Africa, the challenge of first-year chemistry is not limited to a perceived inadequate scientific numeracy proficiency transferred from secondary school. Difficulties in understanding the technical language specific to the discourse of chemistry⁵ and lingering negative attitudes to chemistry inherited from inadequate secondary school experiences, can contribute to a low self-confidence in chemistry as they enter a first-year chemistry course⁴. Being cognisant of the multifaceted complexity of student engagement with chemistry in South African universities, we used a Scientific Numeracy Smart Worksheet (SNSW) to both probe and support the basic scientific numeracy proficiency of a cohort of first-year chemistry students shortly after they entered Rhodes University (Makhanda, South Africa).

Smart Worksheets (SWs) from LearnSci are used as a novel, online, interactive teaching and learning support and assessment tool. LearnSci's SWs are individually randomised to present each student with a unique but consistent experience. The SWs are structured so that students can, depending on the question type, either select options from a drop-down menu or enter their own values. The questions are a mixture of multiple choice or calculation with numerical entry. When a SW question is answered, the student receives an immediate response confirming whether the submitted answer is correct or incorrect. For incorrect answers, students receive instant, formative feedback to guide further attempts at the question. The prompt availability of this formative, 'tutor-like' feedback has proved to be a valuable instrument to support learning in practical⁶ and theoretical² chemistry and pharmacology⁷. Available points decrease with each successive attempt, allowing differentiation between students and the monitoring of student progression. Students are able to use an auto-solve option after an unsuccessful attempt, which only provides the correct answer and removes available marks for that question, thereby discouraging excessive usage. Students can review their attempts and feedback via a timeline mode, which staff can also access for review. SWs can integrate with online grade centres and grade books, and thus can be readily reviewed by teaching staff. The SW data, generated from a cohort of students' combined attempts at SWs, can instantly identify common gaps in understanding and highlight the general misconceptions shared by a large number of students. These data can subsequently facilitate effective and focused follow-up teaching and learning interventions to the benefit of the student cohort.²

 

Research objectives

The aim of this study was twofold: to investigate the SNSW as a diagnostic tool and as a learning support tool. Accordingly, we aimed firstly to quantitatively probe the scientific numeracy proficiency in a large cohort of Rhodes University chemistry students, early in their university journey. Secondly, we investigated the performance of students who attempted the SNSW multiple times, both in terms of SNSW score and end-of-semester summative assessments. We additionally obtained student feedback on the value of the SNSW as a supportive learning tool.

 

Educational context

Cohort and module context

The first-year chemistry course (Chemistry 1) is a two-semester general chemistry course (CHEM101 and CHEM102), taken by all first-year pharmacy students and most students registered for a BSc degree. The course includes topics that require a strong background in numeracy (e.g. equilibrium, thermodynamics, kinetics) and inorganic and organic chemistry, for which there is less of a requirement for scientific numeracy proficiency. The first two weeks of the course introduce the basic building blocks of chemistry, e.g. atomic structure, the periodic table, SI units, stoichiometry, per cent composition and the ideal gas laws.

In 2024, 378 students registered for the course, of which 39% were pharmacy students and 52% were BSc 1 students. The other 9% were second-year science students (some repeating first-year chemistry, others taking it for the first time), and included 13 Extended Studies students, plus a small number of students from the Faculties of Humanities and Commerce.

Only students who achieve more than 50% for mathematics and 60% for physical sciences in their secondary school-leaving exams can apply for entry into a BSc degree. For a degree in pharmacy (BPharm), the minimum requirement is 60% for mathematics, consistent with the overall higher entry level for a pharmacy degree. Only 17.1% of the 2023 Grade-12 school learners who wrote the South African physical science National Senior Certificate examinations achieved 60+%, while 27.5% passed the National Senior Certificate mathematics examinations with 50+%.⁸ Therefore, the students volunteering for this study represent the relatively small number of South African high school graduates able to achieve these pass rates in both of these two subjects. Most of the BSc students intend to major in biochemistry and microbiology, while some continue with chemistry and a small number go on to major in other life sciences, e.g. botany and zoology. Over the past three decades, lecturers of the Chemistry 1 course have anecdotally reported that students struggle with applying basic numeracy within a chemistry context. It is perceived that first-year chemistry students cannot convert SI units, have no 'feel' for the size of very large or very small numbers (exponents of 10) and cannot use ratios and proportions. As Chemistry 1 is a feeder course for pharmacy and all the life sciences, the curriculum content requirements are many, with limited time available to revisit basic numeracy. Thus, there is an ongoing requirement for student self-study, in order to reinforce and test their numeracy skills and access help when needed. In addition, the majority (62%) of the 2024 BSc and BPharm student cohort at Rhodes University required financial assistance from the National Student Financial Aid Scheme. Several studies have shown that the academic success of many South African students is often negatively impacted by their low socio-economic status, usually concomitant with a disadvantaged basic education prior to university entry.⁹ Therefore, the importance of creating a supportive tertiary learning and teaching environment that provides multiple opportunities for students from diverse socio-economic backgrounds to achieve academic success, cannot be overemphasised.

Scientific Numeracy Smart Worksheet

While the SNSW was originally developed between UK academics and LearnSci programmers, the SNSW used in the study was modified by a first-year Rhodes chemistry lecturer, in collaboration with LearnSci. The modified SNSW aligns the expected transferable scientific numeracy skills from the South African physical science and mathematics school curricula with six different numeracy skills deemed important for successful completion of the first-year chemistry course at Rhodes University (Figure 1). The modified SNSW is designed to help South African students develop and practise a wide range of quantitative skills which will support them in their degree, whether that is chemistry, pharmacy or another field.

Students commonly struggle to apply numerical skills to scientific contexts^2-4, therefore the majority of questions have scientific contexts included. However, subject-specific knowledge is not required for successful completion of the worksheet; the focus is on the quantitative aspects. Example questions are shown in Figure 2. Nearly all questions have controlled randomised components (e.g. input values or ordering of questions), to make each student attempt a unique but consistent experience.

Implementing the Smart Worksheet

The SNSW was made readily accessible on the university's Learning Management System. In a lecture at the start of the 2024 academic year, students were informed about the SNSW resource and that completion was a course requirement, with the marks associated with the exercise contributing to their final assessment mark. This was to guarantee engagement with the SNSW, given that students do not always participate in optional formative assessments.¹⁰ A different lecturer (not involved in this research) informed the students about the research project and distributed consent forms to prevent students feeling obligated to the researcher. The information letter and a copy of the consent form were also placed on the Learning Management System. Reminders of the survey and SNSW were sent by email. Students were expected to complete the SNSW in their own time and were given 3 weeks and a maximum of three attempts, with the score for each attempt recorded.

 

Research methods

Overall study design

To determine student usage, performance and views of the SNSW, we combined usage analytics and a student questionnaire. Ethical approval was obtained from Rhodes University's Human Ethics Committee (approval no. 2023-7534-8234). Students were provided with paper opt-in forms, which included permission for their SNSW results to be used in this study. Only the data of students who consented were used.

Usage and performance analytics

Smart Worksheets have the capability of capturing each student's online progress and achievement as they proceed through the worksheet, both of which are saved and made available to the lecturer or tutor. These data, known as usage and performance analytics, provide detailed insights into how students are using these resources, and form the crux of the research findings emanating from this study.

We studied the usage and performance analytics at a section level, to see how students responded to each topic. These data included the

proportion of questions students answered on average, the scores gained on questions attempted, and the frequency of the auto-solve feature. We also compared first and second attempts amongst students who repeated the SNSW and checked for associations between SNSW engagement (in terms of number of attempts) and end-of-module scores.

In studying the student cohort's overall engagement with the SNSW, we can get a sense of the suitability of the resource in terms of challenge and support offered. In addition, we can gain a deeper understanding of the relative strengths and weaknesses of students tackling the six topic areas within the resource, to identify potential areas where further support may be required.

Questionnaire design and deployment

The questionnaire accompanying the SNSW was designed to gain insights into student perceptions of the SNSW resource, including how challenging they found it, how helpful they found each section and how much they would recommend it to others. Many factors can influence these perceptions; therefore, the questionnaire also asked students about numeracy confidence levels before using the worksheet, the main device which they used to complete the resource, and their gender, to evaluate if these parameters were associated with different student perceptions of the SNSW resource. To encourage broad participation, the questionnaire was kept anonymous and short, and each question was optional. The questionnaire, including introductory information and the opt-in consent confirmation, is available in the supplementary material.

The questionnaire was released to students for 3 weeks during February and March 2024, using the SurveyMonkey online platform. The availability of the questionnaire overlapped with the final 2 weeks of SNSW access and for a further week after SNSW access closed. Students were made aware of the questionnaire via emails and in in-person teaching sessions.

Quantitative analysis was carried out in IBM SPSS Statistics version 25.

 

Results and discussion

Analytics findings

A total of 210 students, representing 56% of the cohort, opted in to analytics reporting on their findings and submitted at least one SNSW attempt. We report the usage and performance analytics here, both overall and by section.

Section-level performance

Average percentage of attempted questions, mean score on attempted questions and use of the auto-solve feature for each of the six SNSW sections are shown in Table 1. Questions left blank by students are not included in the mean and auto-solve percentage calculations, rather than scoring them as zero. This is to better represent student performance in attempted questions.

 

 

On average, across all attempts, students completed 94.6% of the SNSW, although this somewhat decreased across the six sections. 'Displaying numbers' (Section 1) was completed by 98.2% of the students, whereas 'Graphs' (Section 6) was completed by 91.8%. Sections 2-5 had completion percentages in between these values.

The overall mean score on attempted questions was 77.0%, although this varied quite considerably amongst the sections. The highest score (85.8%) was for 'Displaying numbers', then 'Rearranging and solving equations' at 84.3%. Students had the most difficulty with 'Graphs', scoring a mean of 64.5%. A more granular look at the question-level results revealed that, whilst most students were able to identify graph types and label graph axes without difficulty, they struggled greatly with identifying outliers and suitable axis ranges for graphs. They also found determining the slope and y-intercept of a provided scatter plot with a trendline to be challenging. The next lowest scoring sections were 'Averages and spread of data', and 'Scientific units', although the scores were markedly higher than 'Graphs', at 72.2% and 75.9%, respectively. Within these sections, students obtained the lowest marks when asked to calculate medians, variance and standard deviations in 'Averages and spread of data', compared with calculating means, modes and ranges and identifying high and low accuracy and precision. In 'Scientific units', making unit conversions especially involving compound units (e.g. mg.mL^-1) appeared to be particularly challenging, compared with questions on unit prefixes in isolation, and even compared with broader calculation questions involving unit conversions.

Use of the auto-solve feature is another indicator of a student's struggle with numeracy questions, and also varied among SNSW sections. In general, sections with higher scores also saw lower auto-solve scores, which is intuitive given that auto-solving forfeits available marks for the solved question. However, there were some exceptions to this trend: the level of auto-solving in the 'Graphs' section was fairly low given the lowest mean scores; this is likely because of the unique presence in this section of a complex question type which grades groups of answers input at once, but which cannot be auto-solved. By contrast, the 'Scientific units' section had particularly high auto-solve rates relative to score. The majority of auto-solves in this section were centred around unit conversion questions, especially with compound units. These questions may have represented a particularly sharp increase in difficulty, leading to a high proportion of students requesting the correct answer after making a number of incorrect attempts at answering the questions.

Identifying student strengths and weaknesses in these topic areas is useful for future support, for example, by knowing where to put future emphasis in teaching sessions. Additionally, this could inform adjustments to the SNSW, providing a little more scaffolding or support in these question areas. This will help ensure the difficulty level of the worksheet is balanced, while maintaining a sufficient challenge for learning and skills development.

Performance improvement from multiple attempts

Students were permitted up to three attempts at the SNSW, in order to encourage more practice and skills development. It is important to note that students were required to submit the whole worksheet (at any level of completion) before they could embark on a subsequent attempt. The highest mark obtained from their attempts was used for their final assessment mark. Of the 210 students who consented to the study, 41 made at least two attempts at the resource. Only six completed the worksheet three times, so only their first two attempts are considered here. Table 2 shows the overall comparisons between the two attempts. See the supplementary material for the section-level breakdowns.

 

 

Students scored higher on their second attempt than on the first. For students who made two or more attempts, first attempts scored 70.6% on average and second attempts scored 82.0%, an increase of 11.4 percentage points (%pt). This increase brings the mean score of this group from sizeably below the overall mean of 77.0% mentioned earlier, to markedly above it. The increase between attempts was highly statistically significant, as determined by a two-tailed paired f-test: f(40) = 7.109, p < 0.001. Scores increased across all sections, with the highest improvements in 'Ratio and percentages' (15.5 %pt increase) and 'Graphs' (14.5 %pt increase). All sections saw an increase of 8.9% or higher. As nearly all questions have randomisation components, values could not be copied directly from previous attempts, or from their peers. Therefore, this score increase is likely to represent an enhanced ability to solve the numeracy problems, perhaps using the feedback provided or their own additional learning to improve their performance.

In addition, students completed more of the SNSW on their second attempt, with an increase of 6.9 percentage points from 87.4% to 94.3% on average for overall completion, but this was not statistically significant: f(40) = 1.327, p = 0.192. However, there were variations in the Completion% changes among SNSW sections, whereby the increases in Completion% were generally higher in the later sections. 'Displaying numbers' (Section 1) was actually slightly reduced in completion, by 1.0 %pt, and 'Ratio and Percentage' (Section 2) completion increased by just 1.0 %pt. The biggest increases in completion were in 'Graphs' (Section 6) and 'Averages and spread of data' (Section 4), both at 11.8 %pt increase. In the second attempt, all sections had a mean completion of over 91%. Motivations behind this differential behaviour are not clear, but could be investigated further in future research.

The auto-solve feature was used much less during second attempts, decreasing from 10.5% down to 5.2%. This reduction of 5.4 %pt was highly statistically significant: f(40) = 4.857, p < 0.001. Given the score increase, this reduction in use is partly due to students not needing to use the feature as they answered more questions correctly after fewer attempts. It could also indicate that students were more willing to persevere and to use the feedback provided to answer the questions. While not all sections showed a relationship between auto-solve use decrease and score increase, there was an association at the highest and lowest ends. The biggest decrease in the use of auto-solve was in 'Ratio and percentages', from 12.5% to 5.1%. This section also showed the biggest increase in scores as described previously. Students were completing more questions themselves whilst marks were available, thereby obtaining a higher score.

Overall, repeating the SNSW was associated with higher completion, higher mean scores on attempted questions and a reduced use of the auto-solve feature.

Worksheet engagement vs exam attainment

Student scores in the quantitative components of their end-of-module chemistry assessment were compared against engagement with the SNSW in terms of number of attempts (Figure 3). Because of the low sample sizes of students using the Smart Worksheet 0 or 3 times, only 1 vs 2 attempts were statistically compared. Students who engaged twice with the SNSW, regardless of performance, went on to score higher (50.8%) than those who made one attempt (46.5%), although this was not quite statistically significant: t(202) = 1.75, p = 0.082. The majority of students who consented to the study attempted the sheet only once, potentially limiting the statistical power of the comparison. The increase may indicate improved numerical confidence and abilities from repeated engagement, with the potential to transfer into other assessment contexts. However, it is also possible that students who engage more deeply with the SNSW are more engaged with their studies generally, and so there may be other factors at play. Nevertheless, as noted previously, the first attempt of students who went on to make two or more attempts was on average lower than the overall mean score. This would therefore not suggest an initial strength of numeracy skills in this group, but, as shown, there was an improvement in performance with SNSW usage.

Questionnaire findings

In total, 196 students, representing 47% of the cohort, answered the online consent question, of which 177 consented and proceeded to the questions. As all subsequent questions were optional, some students skipped some questions, but all questions received at least 136 responses, except the free-text question, for which 39 responses were received.

Overall SNSW rating

Students generally rated the SNSW highly, with a mean of 8.2 when asked, "On a scale of 1 to 10, how much would you recommend the scientific numeracy resource to other students on a similar course?" The most common ratings were 10 (29%) and 8 (26%) (Figure 4). This suggests that the students found the SNSW a useful learning experience.

 

 

Perceived challenge level

The majority of students considered the SNSW to be pitched at an appropriate level of difficulty, with 51.8% of respondents answering "about right for me" when asked their thoughts on the overall challenge level of the SNSW. Only 7.2% of students said the resource was either "much too difficult" or "much too easy". The second common response was "a little too difficult", as answered by 33.8% of students, suggesting that the worksheet was providing a level of challenge to this sub-set of students.

However, difficulty rating did not have a significant association with SNSW rating (one-way ANOVA with Games-Howell post-hoc tests, p > 0.106). This suggests that students found the SNSW similarly valuable, whether they found it challenging, not challenging enough or about right.

Helpfulness of each section

Students were asked, "How helpful were the following sections of the scientific numeracy resource to you, in terms of practising the topic and identifying your strengths and weaknesses?" Positive responses were received for each section, with the most valued section being 'Scientific units': 95.7% of students rated this section "very helpful" or "fairly helpful" (Figure 5). Interestingly, this was also the section with the greatest use of the auto-solve feature, as described earlier, so perhaps students found the auto-solve helpful in support of their learning.²

 

 

The 'Graphs' section had the lowest rating, although 74.4% of students still found it "very helpful" or "fairly helpful". This was the section for which students gained the lowest score, as described earlier. Therefore, the difficulty of this section might have been such that students became somewhat overwhelmed, reducing the section's perceived helpfulness. The South African school mathematics curriculum introduces basic statistics (mean, median and mode) in Grade 7, bar graphs in Grade 8, and scatter plots in Grade 9.¹¹ Logarithms, plus their graphical representations, are only covered towards the end of the Grade 12 mathematics syllabus.¹² These data-handling concepts, however, form only a minor part of the South African school mathematics curriculum and are seldom emphasised. Difficulty with graphs is not limited to South African students^2,13, and an increased focus on graphs in first-year practical sessions at Rhodes University was an immediate positive outcome of the implementation of the SNSW.

Numeracy confidence levels

Numeracy confidence levels were reasonably high amongst student respondents, as the most common answer (56.9%) to "Before starting the scientific numeracy resource, how confident did you feel about your numeracy/mathematical skills?" was "fairly confident". However, over a quarter (25.7%) considered themselves to be "not very confident", and 9.7% said they were "not at all confident" about their mathematical skills. Only 7.6% considered themselves "very confident" in this area.

To compare student confidence levels with the SNSW rating, the responses were re-coded into two groups: lower confidence ("not at all" and "not very" confident) and higher confidence ("fairly" and "very" confident). This merging was done because of the small response numbers for the highest and lowest confidence levels. The lower confidence group rated the SNSW more highly than the higher confidence group (8.48 vs 8.02), although this difference was not statistically significant: t(133) = 1.553, p = 0.123. Overall, the SNSW was favoured comparably by students, whether they felt confident about their mathematical skills or not. Interestingly, a previous South African study highlighted the disparity between over-confidence, stemming from prior performance in school physical science and mathematics subjects, and actual performance in first-year university chemistry. This disparity could explain the lack of correlation between confidence and experience of the SNSW.¹⁴

Device used to complete the SNSW

While three-quarters of students (75.2%) completed the SNSW on desktop or laptop computers, there was a significant minority using mobile devices (20.4%). The remaining students reported using a tablet or a mix of devices. Students' ratings of the SNSW were very similar between those who used a desktop/laptop (8.26) and those who used a mobile device (8.21), suggesting that the device used did not significantly affect their learning experience.

Gender

Most (65.2%) of the survey respondents were women, 31.9% were men and the remainder were non-binary, other or preferred not to say. This ratio is approximately representative of the gender ratio of the cohort, so there did not appear to be a bias in responses from one gender. Female students rated the resource slightly more highly than male students on average (8.30 vs 8.02), but the difference was not statistically significant: f(130) = 0.869, p = 0.386. The scientific numeracy resource was therefore helpful to both male and female students.

Free-text comments

When asked for further comments about their experiences of the SNSW, 39 students contributed responses. These responses were grouped by topic area (Table 3) for qualitative review.

 

 

The responses were broadly positive, describing its helpfulness or their enjoyment of the SNSW, including helping to identify areas of unfamiliarity, strength or weakness in general terms, and the specific area in which they improved, particularly significant figures, units and graphs.

One criticism concerned the length of the SNSW. Because there are many topic areas to cover in scientific numeracy, omitting questions would create gaps in the coverage and fewer practice opportunities. The resources could be split, but keeping them together creates cohesion and allows for overall reporting. To prevent students feeling overwhelmed, they could be encouraged to take breaks between sections, with the automatic save feature highlighted, so they know that they do not need to complete each section at one time.

Some students commented on how the SNSW helped refresh material covered in previous learning, and two more highlighted the connection with other parts of the course. Others mentioned that it was challenging, in a positive context.

 

Limitations of the study

Although the large cohort meant this study achieved a high sample size, not all students consented to analytics usage or completed the questionnaire, limiting its potential. Furthermore, most students completed the SNSW only once, limiting some statistical comparisons. The cohort's varied entry requirements and subject interest might also impact the interpretation of these data. Additionally, allowing students to complete the SNSW independently may have introduced uncontrolled variables that cannot be incorporated into the analysis of these data, limiting available understanding of student behaviours. Finally, some students found the SNSW lengthy, which may have also impacted completion rates.

 

Conclusions

Supporting underrepresented and disadvantaged students in scientific subjects by addressing student numeracy gaps, improves their confidence, academic skill set and progression. To be effective, this support requires an efficient approach to identify core knowledge gaps and should provide accessible support materials that foster engagement. The SNSW used in this study is a robust resource for establishing baseline metrics and upskilling students, whilst minimising the impact on staff workload, potentially contributing to a more inclusive teaching and learning environment.

Using the SNSW as a diagnostic tool has effectively highlighted knowledge gaps that can be addressed more efficiently. Graphs, averages and spread of data, and scientific units emerged as challenging areas for students. Completion rate and score improved significantly upon repeating the SNSW, and whilst there was no significant difference, there was a positive correlation between SNSW repetition and higher module scores. Students generally found the SNSW favourable and the immediacy of formative feedback was well received.

This study focused on one university in South Africa, but future investigations could extend this by comparing findings from different institutions in the country. Seeking specific student perceptions of the different numeracy topic sections may also yield deeper insight. Additionally, investigating when students prefer to use the SNSW could help inform strategies for the optimal timing for providing resources. Additional learning resources may be incorporated as well, and these could be tailored to students' learning needs as identified by the SNSW results.

 

Acknowledgements

We wish to acknowledge with gratitude the enthusiastic contribution to this research project by the 210 volunteers from the CHEM101 class of 2024 at Rhodes University. We also thank the Department of Chemistry and the Information and Technology Services Division of Rhodes University for their support of this research project. The ongoing support for innovative chemistry education in South Africa by LearnSci (UK) is greatly appreciated.

 

Data availability

The data supporting the results of this study are available upon request to the corresponding author.

 

Declarations

E.C., L.W., A.B. and S.S. are, or were, employed by Learning Science (trading as LearnSci) at the time of creation of this article. Ethical approval was obtained from the Rhodes University Human Ethics Committee (approval no. 2023-7534-8234).

 

Authors' contributions

J.S.: Conceptualisation, data curation, investigation, methodology, project administration, resources, writing - original draft, writing - review and editing. E.C.: Conceptualisation, data curation, formal analysis, investigation, methodology, project administration, visualisation, writing -original draft, writing - review and editing. L.W.: Conceptualisation, data curation, formal analysis, investigation, methodology, project administration, visualisation, writing - original draft, writing - review and editing. A.B.: Data curation, software. S.S.: Data curation, software. D.E.S.: Conceptualisation, software, supervision. M.T.D-C.: Conceptualisation, methodology, project administration, supervision, visualisation, writing - original draft, writing - review and editing. All authors read and approved the final manuscript.

 

References

1. Mumba FK, Rollnick M, White M. How wide is the gap between high school and first-year chemistry at the University of the Witwatersrand? S Afr J High Educ. 2002;16(3):148-156. https://doi.org/10.4314/sajhe.v16i3.25227        [ Links ]

2. Shallcross DE, Davies-Coleman MT, Lloyd C, Dennis F, McCarthy-Torrens A, Heslop B, et al. Smart Worksheets and their positive impact on a first-year quantitative chemistry course. J Chem Educ. 2025;102:1062-1070. https://doi.org/10.1021/acs.jchemed.4c00705        [ Links ]

3. Towns MH, Bain K, Rodriguez JG. How did we get here? Using and applying mathematics in chemistry. In: Towns M, Bain K, Rodriguez JG, editors. It's just math: Research on students' understanding of chemistry and mathematics. ACS Symposium Series 1316. Washington, DC: American Chemical Society; 2019. p. 1-8. https://doi.org/10.1021/bk-2019-1316.ch001        [ Links ]

4. Coll R, Ali S, Bonato J, Rohindra D. Investigating first-year chemistry learning difficulties in the South Pacific: A case study from Fiji. Int J Sci Math Educ. 2006;4:365-390. https://doi.org/10.1007/s10763-005-9007-6        [ Links ]

5. Rees S, Kind V Newton D. Meeting the challenge of chemical language barriers in university level chemistry education. Isr J Chem. 2019;59:470-477. https://doi.org/10.1002/ijch.201800079        [ Links ]

6. Coyte E, Lowry RB. Paired online laboratory assessments: Formative engagement and summative attainment. J Chem Educ. 2024;101(8):3514-3521. https://doi.org/10.1021/acs.jchemed.4c00308        [ Links ]

7. McCartney J, Egieyeh S, Ebrahim N, Braaf E, Beukes D. Innovation in pharmacy education during the pandemic: Using Learning Science to support laboratory-based practical skills. S Afr Pharma J. 2021;88(4):31-33. https://hdl.handle.net/10520/ejc-mp_sapj_v88_n4_a9        [ Links ]

8. South African Department of Basic Education (DBE). 2024 National Senior Certificate (NSC) Diagnostic Report: Book 1. Pretoria: DBE; 2024. p. 217, 245. Available from: https://www.education.gov.za/Portals/0/Documents/Reports/2024/2024%20NSC%20Diagnostics%20Book%201.pdf?ver=2025-02-07-130329-033        [ Links ]

9. Van Zyl A. The contours of inequality: The links between socio-economic status of students and other variables at the University of Johannesburg. J Stud Aff Afr. 2016;4(1):1-16. https://doi.org/10.14426/jsaa.v4i1.141        [ Links ]

10. Crabtree R. Barriers to student engagement: Why don't students engage? Student Engag Higher Educ J. 2023;4(3):28-47. https://sehej.raise-network.com/raise/article/view/1156        [ Links ]

11. South African Department of Basic Education (DBE). Curriculum and Assessment Policy Statement: Grades 7-9: Mathematics. Pretoria: DBE; 2011. Available from: https://www.education.gov.za/Portals/0/CD/National%20Curriculum%20Statements%20and%20Vocational/CAPS%20SP%20%20MATHEMATICS%20GR%207-9.pdf?ver=2015-01-27-160141-373        [ Links ]

12. South African Department of Basic Education (DBE). Curriculum and Assessment Policy Statement: Grades 10-12: Mathematics. Pretoria: DBE: 2011. Available from: https://www.education.gov.za/Portals/0/CD/National%20Curriculum%20Statements%20and%20Vocational/CAPS%20FET%20_%20MATHEMATICS%20_%20GR%2010-12%20_%20Web_1133.pdf?ver=2015-01-27-154314-253        [ Links ]

13. Glazer N. Challenges with graph interpretation: A review of the literature. Stud Sci Educ. 2011;47(2):183-210. https://doi.org/10.1080/03057267.2011.605307        [ Links ]

14. Potgieter M, Ackermann M, Fletcher L. Inaccuracy of self-evaluation as additional variable for prediction of students at risk of failing first-year chemistry. Chem Educ Res Pract. 2010;11:17-24. https://doi.org/10.1039/C001042C        [ Links ]

 

 

Correspondence:
Joyce Sewry
Email: j.sewry@ru.ac.za

Received: 26 Sep. 2024
Revised: 06 June 2025
Accepted: 09 June 2025
Published: 29 Sep. 2025

 

 

Editors: Chrissie Boughey; Nkosinathi Madondo
Funding: None

 

 

Supplementary Data

The supplementary data is available in pdf: [Supplementary data]