Types of formative feedback provided by technology teachers during practical assessment tasks

Technology learners require formative feedback from their teachers to aid them in structuring and solving design problems. Without such feedback, learners often do not explore sub-problems and are inclined to reproduce known and existing solutions. Due to a paucity of literature on formative feedback in technology classrooms, this study investigated the formative feedback types that teachers provide to learners during the problem-structuring and problem-solving phases. A qualitative case study design was employed to investigate such feedback from five technology teachers who supported their learners in solving ill-structured design problems. The findings indicate that teachers ’ formative feedback consists predominantly of low-level questions, while generative design questions and low-level comments were the least observed. A deeper understanding of how formative feedback unfolds in a technology classroom may help teachers guide learners through problem structuring and solving. Further research is needed to determine the influence of formative feedback on learners ’ design outcomes.


INTRODUCTION
The South African Department of Basic Education (Department of Basic Education [DBE], 2012) requires teachers to guide learners in design tasks by providing formative feedback through questioning, commenting, explaining, and demonstrating.Practical assessment tasks (PATs) are the primary form of formal assessment in technology; these tasks allow learners to develop and demonstrate their skills in both knowledge of the content and the design process (DBE, 2012, p. 41).During PATs, learners must use the design process to solve design problems and generate possible solutions (DBE, 2012).These design problems are often ill-structured and openended, meaning they do not have prescribed problemsolving procedures or single correct answers.The level of complexity of ill-structured problems can increase if the problem statement changes as design solutions are considered (Dorst, 2019a(Dorst, , 2019b;;Dorst & Cross, 2001;Murray et al., 2018a).
Teachers should assist learners with their PATs by providing formative feedback as they work through the five iterative phases of the design process prescribed by DBE (2012): investigating, designing, making, evaluating, and communicating.The investigation phase requires learners to break design problems into smaller sub-problems, called problem structuring.Problem structuring involves investigating stakeholder needs, requirements, and constraints to understand, define, and structure the design problem (Goel, 1995;Haupt, 2015;Visser, 2009).After problem structuring, learners generate possible design solutions and select initial ideas to address the identified problem.Once several possible solutions have been generated and considered, the learner must choose one and motivate why they chose the solution and develop the solution further (DBE, 2012).This activity is characteristic of the design phase of DBE (2012) prescribed process.
Despite evidence that formative feedback enhances learning (Hattie & Clarke, 2019;Hattie & Timperley, 2007;van den Bergh et al., 2013), formative feedback is still under-researched in technology classrooms (Schut et al., 2018(Schut et al., , 2020;;Stables et al., 2016).The deficiency of research and resulting literature in this regard can lead Loubser et al. / Types of formative feedback provided by technology teachers 2 / 12 to learners not sufficiently investigating the context, users' needs, requirements, and other aspects of the design problem.If the design problem is not structured properly, learners may be unable to generate innovative solutions (Creeger et al., 2019) and simply reproduce known and existing solutions.Given the paucity of literature regarding feedback given by technology teachers in the classroom, it was deemed essential to investigate this matter.For this reason, the study sought to answer the following question:

What types of formative feedback do technology teachers provide to support learners with problem structuring and preliminary problem-solving activities during PATs?
Expanding our knowledge and understanding of formative feedback in the technology classroom can contribute to the professional development of both preservice and in-service teachers.

Nature of Design Problems
Design problems are usually ill-structured and complex problems (Jonassen, 2010).Design problems typically have several possible solutions and may rely on many methods for arriving at a solution (Jonassen, 2010).Goel and Pirolli (1992) note that design problems do not have correct or incorrect answers, just better or worse ones.This is because the criteria for evaluating the success of a design solution is usually not stated, and therefore, designers need to make decisions and judgements during the design process and be able to defend these decisions and judgements (Goel & Pirolli, 1992;Jonassen, 2010).The design problem will often change as design solutions are considered (Dorst, 2019a(Dorst, , 2019b;;Dorst & Cross, 2001;Murray et al., 2018a).This means that designers will often revisit the problem as they start considering possible solutions and explore solutions as they study the problem (Murray et al., 2018b).Previous studies have shown that novice designers spend more time exploring the problem and often revert to "trial and error" when exploring design solutions (Ahmed et al., 2003;Cross, 2004).

Problem Structuring During Designing
In South African technology classrooms, problem structuring occurs during the investigation phase of the design process prescribed by DBE (2012).When learners engage in the investigation phase, they are required to identify and explore the context of the problem, the nature of the problem or need, the user or stakeholder's requirements, and the expected function of the design solution (DBE, 2012;Goel & Pirolli, 1992).Since there is minimal information about the goal of the design task, extensive problem structuring is required before problem-solving can begin (Goel & Pirillo, 1992).

Preliminary Problem-Solving During Designing
In the design process prescribed by DBE (2012), preliminary problem-solving occurs in the design phase.Activities associated with the design phase include exploring initial ideas, making freehand sketches of possible solutions, writing a design brief with specifications and constraints, planning the design process using a systems diagram, trial modelling, and budgeting (DBE, 2012).Teachers' formative feedback should help learners navigate through problem structuring and exploration and the generation of ideas during the design process (Yilmaz & Daly, 2014).Although limited studies have reported on teachers' use of formative feedback during designing, Stables et al. (2016) point out that learners require interactions with their peers and their teacher to understand the design problem properly and to start generating design solutions.

Types of Formative Feedback in Design Contexts
Formative feedback is defined by Hattie and Timperley (2007, p. 82) as "information provided by an agent (teacher, peer, book, and  3 / 12 design settings and school-based design settings, the purpose of formative feedback is often to clarify learners' ideas, justify their design choices, interpret, and compare learners' designs to previous design work, and make recommendations for improvement (Cardella et al., 2014;Schut et al., 2018Schut et al., , 2020;;Yilmaz & Daly, 2014).
Questioning is a popular technique for aiding with the process of problem-solving.Several studies have been conducted to explore how questions could be used to facilitate design (Aurisicchio & Bracewell, 2009;Schut et al., 2018Schut et al., , 2020;;Stables, 2017;Stables et al., 2016).These studies show that questions could help students develop their ideas into solutions and foster design creativity (Schut et al., 2018(Schut et al., , 2020;;Stables et al., 2016).Eris (2004) developed a model of questions to characterize the type of questions that are asked during design.Aurisicchio and Bracewell (2009) and Eris et al. (2007) note that asking questions is an important part of the design process and that designers must continuously ask questions during the design process.Eris et al. (2007) emphasize the importance of examining designers' questioning techniques during the idea-generation phase of the design process, as this stage will most likely determine the quality of the design outcomes.
Questions can be either low-level or high-level, depending on their purpose (Eris, 2004).Deep reasoning questions (DRQs) and generative design questions (GDQs) are high-level questions.The reason for using low-level and DRQs is to find the best solution to a problem or answer to a question (Cardoso et al., 2014;Eris, 2004;Schut et al., 2018).They consist of true-false, comprehension, and multiple-choice questions.These inquiries frequently depend on a truth value (Eris, 2004) and have been linked to evidence of convergent thinking (Eris et al., 2007).On the other hand, GDQs are evidenced by divergent thinking and seek to find multiple solutions or answers to a problem or question.GDQs should aim to generate as many possible answers as possible or possibilities from a single starting point (Cardoso et al., 2014;Eris, 2004;Schut et al., 2018).
The answer is known for both LLQs and DRQs.In other words, the question has a truth value (Cardoso et al., 2014)."How many wheels do we have?" is an example of a low-level question (LLQ).Learners can answer the question by counting the number of wheels in the picture before them.An example of a DRQ would be: "why do we need to attach a wheel?"This question has a known answer, but learners must answer this question based on previously known facts.In other words, this question can be answered by converging facts.The difference between LLQs and DRQs is that LLQs are used to confirm or validate what is known, whereas DRQs are used to explain facts or designers' understanding (Cardoso et al., 2014).
In contrast to LLQs and DRQs, GDQs do not necessarily hold a truth value but seek to elicit multiple known and unknown possible answers to any given question (Cardoso et al., 2014).The purpose of GDQs is to disclose known answers and elicit unknown answers.GDQs can be characterized as questions, where the questioner attempts to move away from facts towards possibilities that could be collected from these facts (Cardoso et al., 2014).For example, during the design process, learners might experience an object slipping, which could lead to the question, "how can we stop it from slipping?"This question can be answered by listing several non-slip materials or methods.Therefore, there is not only one correct answer; the answer(s) will depend on the design context and problem.
One of the limitations of Eris's (2004) study is that the taxonomy he created does not differentiate between the questions asked during the distinct phases of design.In a study focused on formative feedback in primary technology classrooms, Schut et al. (2018) added two categories of LLQs to the model, "compliments" and "critique".Schut et al. (2019) further expanded Eris' ( 2004) model by adding "direct recommendations" as a category.They also renamed the categories, including compliments, critique, and direct recommendations, to a more all-encompassing category called "low-level comments (LLCs)".In their latest research, Schut et al. (2020) further adapted the model by including a "description" as a category in LLCs and "future" and "future description" under DRQs.The two original question categories Eris (2004) identified as low-level and high-level questions have subsequently been renamed low-level and high-level feedback.Table 1 shows Schut's (2020) formative feedback model that was used as the conceptual framework to guide this study.The model was deemed suitable for this study's conceptual framework as it provided additional categories of formative feedback questions and comments, making it more detailed and exhaustive.
In Table 1, low-level formative feedback questions are used to gather information about the design task (Schut et al., 2020).These questions aim to clarify and verify learners' understanding of the scope of the problem, users' needs and wants, tools and materials, and existing products (Schut et al., 2020).LLCs are statements about the learners' design tasks and can include compliments, criticism, recommendations, and observations about the learners' problem-structuring activities (Schut et al., 2020).DRQs can be used to explore learners' reasoning for their problem-structuring activities, while GDQs are helpful to explore methods and procedures to realize design goals (Schut et al., 2020).

METHODOLOGY
This study employed a qualitative single case study design guided by the interpretivism paradigm.Five teachers from three different schools were selected to 4 / 12 participate in this study.The schools were selected based on their accessibility and teachers' willingness to participate in the study.Senior phase technology teachers were selected based on their ability to (1) teach technology in senior phase (grade 7 to grade 9), (2) engage in ill-structured design problem-solving tasks in line with the South African curriculum requirements, (3) facilitate the problem structuring phase in class, (4) indicate that they engage in formative feedback during design problem-structuring, and (5) be available for data collection during the time of the study.
The sampled teachers were observed in their classrooms while guiding technology learners toward design solutions for real-life design problems.All the design problem-solving sessions were audio and video recorded.The data was analyzed and interpreted in line with Creswell et al. (2016) six-step process.The data analysis process started with organizing and preparing the data by sorting and labelling each observation.The labelled observations were uploaded to AtlasTi.The authors then rewatched the recordings of the classroom observations to get a detailed overview of the data.The conceptual framework was used to inform the coding process and help with pattern identification and interpretation.In Table 2 and Table 3, descriptions and examples of the codes can be seen.From here, we sorted through the coded observational data to identify the emerging themes and descriptions.Finally, the themes from the observational data were described with supporting quotes from the data, which enabled us to draw our conclusions.The transcriptions of the audiovisual recordings of the formative feedback instances were used as the primary data source.Units of analysis were created of consecutive feedback based on the conversation content.Only the feedback focusing on the problem-structuring and preliminary problem-solving related to the design task was considered.To determine the nature of the teachers' formative feedback, we used Schut's (2020) design feedback model, as displayed in Table 1.
To enhance the study's rigor, the authors relied on member checking and low-inference descriptors to ensure that the results presented accurately reflected the observational events (Ary et al., 2019).We used audiovisual recordings to capture the formative feedback occurrences in class and used direct quotes from the observations to present the results and findings of the study.We kept an audit trail of the emerging themes as they developed and the changes to the themes as they occurred.The potential for researcher bias was mitigated by stating the working assumptions about the study and continuously reflecting on these assumptions.
Ethical principles of voluntary participation, safety during participation, confidentiality and trust were applied in this study.Before data collection, we obtained ethical clearance from all the relevant stakeholders and obtained informed consent from all participants.

RESULTS & DISCUSSION
The results show that the technology teachers gave learners more feedback during the preliminary problemsolving phase of PAT than the problem-structuring phase (see Figure 1).This finding suggests that teachers supported learners more in considering possible solutions to the design problem rather than exploring and defining the design problem.
The results further revealed that technology teachers predominantly used questions to provide formative feedback to support learners in identifying a design problem, researching the design problem, and analyzing existing design solutions.For example, in an attempt to support learners' problem identification, the teachers would read the design brief to the class and then support learners to identify a specific design problem by asking  (Schut et al., 2020) Code name Description Request The questioner does not want to know anything but wants a specific action to be performed.

Verification
The questioner wants to know the truth of an event.Typically, yes or no answers.Disjunctive Verification using multiple concepts.

Concept completion
The questioner wants to know the missing component in a specified event (e.g., fill in the blanks).

Feature specification
The questioner wants to know some property of the given person or thing.

Quantification
The questioner wants to know an amount.

Definition
The questioner wants to find out what a question concept means.

Example
The questioner invites examples of the question concept.

Comparison
The questioner wants to compare the similarities and/or differences between the question concepts.

Judgmental
The questioner wants to elicit judgments from the responder by requiring a projection of events rather than a strict recall of events.

Compliment
Praise for what the learner did/accomplished.

Critique
The teacher makes an assessment or judgement of the learners' work.Direct recommendation The teacher tells the students specifically what to do to improve the design.

Description
A description of a part of the design task that can be observed.

Interpretation
The questioner wants to know the meaning of the question concept based on the given information.

Procedural
The questioner wants to know the partially or totally missing instrument in the question concepts.

Causal antecedent
The questioner wants to know the states or events that have in some way caused the concept in question (e.g., what led to …?)

Causal consequence
The questioner wants to know the concept or causal chain that the question concept caused.

Rational/function
The questioner wants to know the motives or goals behind actions (e.g., why?).

Expectational
Questioner wants to know causal antecedent of an act that presumably did not occur (e.g., why not).

Enablement
The questioner wants to know the act or the state that enabled the question concept.Future A question about the future state of the task.

Future description
A description of the future state of the task that can be observed.

Enablement
The questioner wants to construct acts, states, or resources that can enable the question concepts.

Method generation
The questioner wants to generate as many ways as possible of achieving a specific goal.Proposal/negotiation The questioner suggests a concept or negotiates an existing or previously stated concept.

Scenario creation
Questioner constructs a scenario involving question concepts & wants to investigate outcomes.

Ideation
The questioner wants to generate as many concepts as possible from an instrument without trying to achieve a specific goal.

/ 12 questions like "you have to go and design and make a what?"
or "what is the problem?", and "what do we have to do?"The teachers in this study did not adequately support learners in exploring the design context, client, requirements and constraints, and existing products.Instead, like studies by Engelbrecht et al. (2007) and Neuman (2003), the teachers were observed to focus their feedback on specific problem solutions rather than helping learners understand the design problem.
Moreover, the formative feedback questions posed by the teachers focused on how research should be done during the problem structuring phase rather than supporting learners to concentrate on sub-problems in the design problem.Teachers asked learners questions like "how do I do research" and "I showed you my research; was it writing?What was it?"Finally, teachers also supported learners' analysis of existing solutions by showing learners images and videos of existing products while discussing the design brief.The teachers then pointed out specific design features that would presumably address the design problem."Can you see the rotary motion?Here they used what mechanism" and "can you see how it stands out when you fly by it?"are examples of the questions used to support learners' analysis of existing solutions.Although teachers supported learners in identifying a design problem, conducting research and analyzing existing solutions, teachers did not adequately support learners in considering the user needs, the context of the design problems or the unique features of existing designs.Previous studies have noted the importance of thorough problem exploration and structuring to avoid replicating existing solutions (Creeger et al., 2019;Dorst & Cross, 2001).
Learners' preliminary problem-solving phase was supported by teachers' prompts for completing a design brief, identifying design specifications, generating initial ideas, and making preliminary design decisions.In a PAT, a design brief is typically a short statement that outlines the client, context, design problem to be addressed, design specifications and constraints (DBE, 2012).Teachers were observed to prompt learners to consider who the client will be and what the purpose of the design solution should be by asking questions like "your overwater bungalow is for who?As a starting point for generating initial design solutions, the teachers in this study used formative feedback questions that required learners to explain their design ideas, how they intend to solve the design problem, and what materials and mechanisms could be used in their designs.The questions teachers asked included "how are you going to solve the problem?What are you going to make?" or "most overwater bungalows are made of what [material]?" Learners were also supported in making early design decisions such as "are you going to use a gear system, are you going to use a hydraulic system, a pneumatic system?what are you going to use to make this ride move?"These questions prompted learners to consider the function of their design solution and how to accomplish the design outcome.Some studies have shown that learners have limited knowledge of strategies to help them explore and generate various design ideas (Daly et al., 2019).As a result, learners often present variations of the same design solution, which may not be beneficial to solving the design problem (Daly et al., 2019).
Overall, Daly et al. (2019) note that providing feedback could support learners to explore design problems, identify problems and possible improvements

Types of Formative Feedback Used to Support Learners' Problem-Structuring & Preliminary Problem-Solving
In the previous section, it was pointed out that technology teachers relied predominantly on questions to support learners during the design process, problem structuring and preliminary problem-solving phases.While they also used comments to provide formative feedback, incidents of this were less frequently observed.The formative feedback occurrences observed in this study supported learners' identification of design problems, exploration of the design problem and analysis of existing products, writing a design brief, identifying design specifications, generating initial design ideas, and making early design decisions.The formative feedback questions and comments observed in the technology classrooms were categorized as LLQ, LLCs, DRQ, and GDQs per the conceptual model.Figure 1 shows the frequency of LLQ, DRQ, GDQ, and LLC technology teachers used to support learners' design processes during PATs.
Figure 1 shows that LLQs were the most prominent formative feedback type observed in the problemstructuring and preliminary problem-solving phases, while LLCs were the least observed formative feedback type in the problem-structuring phase.The high frequency of LLCs in the problem structuring phase may be due to the nature of problem structuring activities, which focus more on understanding and defining the design problem.The problem structuring activities were based mainly on whole group discussions rather than putting ideas to paper.We speculate that this may be a reason for the lower frequency of LLCs in the problem structuring phase.DRQs were observed to occur equally during both phases of the design process, and GDQs were more frequently observed during the preliminary problem-solving phase.Each formative feedback type and category will now be discussed.

Frequency of Low-Level Question Categories Observed
LLQs reveal facts about a concept that can be verified; this means that LLQs can have correct or incorrect answers (Eris, 2004).LLQs can be categorized as request, verification, disjunctive, concept completion, feature specification, quantification, definition, example, comparison or judgmental (Eris, 2004;Schut et al., 2020).Figure 2 shows the frequency of LLQ categories teachers used to support learners during problem structuring and preliminary problem-solving during the design process.
When considering the formative feedback instances observed during the problem structuring phase, the data showed that verification, concept completion, and example questions were frequently observed.During the preliminary problem-solving phase, example, concept completion, and feature specification questions were most common.LLQs were used to verify and clarify learners' understanding of the design tasks to be solved, important features of existing designs and how to collect more information about the design problem.
Verification was the most frequently observed question category during problem structuring and required minimal reasoning from learners.Verification questions can usually be answered with "yes or no".In the present study, the participants were observed asking learners to verify that they could identify some of the key features of existing solutions related to the given design problem; for example, "do they specify that it should be able to support a weight of 20 people?" and "can you see the rotary motion?can you this corrugated iron?" The teachers utilized concept completion questions frequently in the problem structuring and preliminary problem-solving phase.It would seem that these questions aimed to guide learners to identify a specific design problem from the given design brief, e.g., "you The teachers asked learners to give examples of what information they might still need, e.g., "what things do you need to go and find out?" and where learners could find the information, they needed to complete the design task, e.g., "who of you can think where you can find this?"The participants also used example questions to guide learners to revisit the content that related to the design task, e.g., "what have you found to be the best types of bridges to span a gap that is also safe and stable?" and "we discussed two types of structures already, which types of structures were those?"Learners were prompted to consider key features to include in their designs during the preliminary problemsolving phase through questions like "when you think about a bridge, what should it be able to do?" Teachers also assisted learners in considering the dimensions of their designs and certain materials "how wide are the poles you are using?"and "what material do they use?"Requests were not observed in either of the design phases, while judgmental questions were only observed during the problem-solving phase.Requests do not require a response from the learner but rather an action.Judgmental questions would require learners to judge parts of the design task.Judgement questions do not only require learners to recall facts but also speculate about the future state of the task.An example of a judgement question could be, "which design are you going to develop into the final solution?"We speculate that the judgmental questions were not observed in the problem structuring and preliminary problem-solving phases as these phases do not require learners to make final design decisions.The results imply that the participants mostly asked clarifying questions to promote a shared understanding of the design task, design problem, and learning goals.These included questions that required yes or no answers or for learners to complete the sentence, give examples of concepts or provide details about their design features.Verification, disjunctive, and concept completion questions about task lay foundation for more complex questions on a higher level of feedback (Hattie & Timperley, 2007;Tawfik et al., 2020).

Frequency of Low-Level Comment Categories Observed
Figure 3 shows that compliments and direct recommendations were the only low-level categories observed during this study's problem-structuring and problem-solving phase of the design task.
Compliments were used to praise learners for structuring appropriate problem scenarios or correctly identifying the design problem from the design brief; for example, that is a good scenario!Learners were also complimented on their initial ideas when discussing their initial design ideas with their teachers.Some of the compliments observed were "that's a good idea" and "so it is for little kids, that's a good idea".
The instances of direct recommendations were used to confirm that learners were on the right track with their structured problems and that they understood the task correctly; for example, if that is correct, you simply have to design a bridge.While learners were exploring possible design solutions in the problem-solving phase, teachers were observed to give learners direct recommendations about what their solutions should be or what features to 9 / 12 include.For example, "you have to design an overwater bungalow with two bedrooms", "so you are going to design a frame structure that will need to hold up a [water] tank or reservoir," and "you just have to build a bridge".
During the problem structuring phase, the learners in the study have not yet written down or drawn their ideas.The lack of critique and description comments was not surprising since critique and descriptions are usually observed as feedback on a learner's written or drawn work.The discussions about the design tasks, problem structuring, and preliminary solving phase were mostly teacher-led and took the form of a whole class discussion.The high frequency of compliments concurs with a previous study by Schut et al. (2018), which found that LLCs from design clients mainly consisted of compliments.Some researchers suggest that providing learners with compliments on their design tasks can be beneficial as it helps to decrease uncertainty and increase learners' self-esteem (Hattie & Timperley, 2007;Shute, 2008).Learners feel reassured that they are proceeding with their tasks correctly.Findings from professional design studies have also suggested that novice designers are often exposed to more directive feedback, which can be less conducive to autonomous thought (Dannels & Martin, 2008).

Frequency of Deep Reasoning Question Categories Observed
DRQs support learners' reflection and evaluative thinking in design settings (Schut et al., 2018(Schut et al., , 2020)).Like LLQs, DRQs can have correct or incorrect answers but aim to explore the causal explanation of facts rather than verify facts (Cardoso et al., 2014).
Interpretation was the predominant DRQ formative feedback category observed in the problem structuring and preliminary problem-solving phases, as seen in Figure 4.In this study, teachers were observed to read the design brief to the whole class and then asked learners to interpret the information from the design task to understand the design problem and provide potential solutions.While procedural and rationale/function questions were only observed in the problem structuring phase, expectational, enablement, and future description questions were not observed at all.In Schut et al.'s (2020) study, it seems that future and future description questions were asked during the problem-solving phase of the design process.These feedback types were used to reflect on the possible future states of the designs (Schut et al., 2020).We speculate that the limited and lack of some question categories might be due to the limited time learners were engaged in independent preliminary problem-solving activities during the observation lessons.
In this study, teachers introduced PAT by reading the design brief with the learners.The teachers then asked learners to identify the design problem without the teacher's guidance.From here, the participants would utilize concept completion questions to ensure that all the learners correctly identified the primary design problem from the design brief.The learners were also required to infer who the client and the context of the design problem would be in the given task, for example, "who will the clients be? what is the problem and who will benefit from it?"Causal antecedent questions were observed during preliminary problem-solving and aimed to support learners in evaluating how and why their design outcomes were realized.Examples of causal antecedent questions include "why do you think people use fiberglass?" and "what makes the wheels turn?" Cardoso et al. (2014) note that DRQs may support learners' understanding of the effects design decisions and actions could have on the outcomes of design solutions.

Frequency of Generative Design Question Categories Observed
Through GDQs, the technology teachers were observed to encourage learners to explore possible design solutions and methods for building, strengthening, and supporting features of their designs.From Figure 5, it can be seen that method generation was the most frequently observed GDQ question category observed.In contrast, there was no or limited evidence of Enablement, proposal/negotiation, and ideation questions in this study.For proposal/negotiation questions, the teacher would either share an idea they have or discuss an idea that was mentioned before.Therefore, this feedback would require an idea or possible design solution from the learner that the teacher can provide feedback on.Ideation questions would require the learner to come up with many different ideas without having a particular aim in mind.Since the learners did not spend much time drawing or writing down their solutions, and since discussions were mostly whole group discussions, lack of proposal/negotiation and ideation feedback was not surprising.
Method generation questions support learners in identifying various methods for completing a task or activity.Some examples of method generation questions include "how are you going to make the gears?how are you going to make it stable?"and "how would you ensure that it is safe for hospitals to use?" The high frequency of method generation questions aligns with Schut et al. (2020) finding, who reported that method generation and proposal/negotiation questions were the most commonly asked GDQs by clients and peers in primary technology classrooms.However, the low frequency of proposal/negotiation questions observed in this study contradicts the results of Schut et al. (2020).We believe that this discrepancy may be due to the way formative feedback was, with teachers posing questions from the front of the class to the entire group, while in Schut et al. (2020), formative feedback was given through individual group interactions.

DISCUSSION & CONCLUSIONS
In this study, teachers assisted learners in navigating the design process by helping them identify the design problem, conduct research, and examine existing solutions.Through videos and images, the teachers guided the learners in exploring existing products and drawing their attention to important aspects of the solutions.The problem-structuring phase of the design process, as described by Goel (2014) and Goel and Pirolli (1992), aligns with the investigation phase outlined by DBE (2012).Technology teachers also guided learners to write design briefs, identify design specifications, generate preliminary solutions, and make design decisions.The preliminary problems-structuring phase of the design process (Goel, 2014;Goel & Pirolli, 1992) is associated with the design phase in the design process described by DBE (2012).Supporting learners' preliminary problem-solving activities may lead to increased creativity and novelty of design solutions (Daly et al., 2019).
LLQs were the most frequently observed feedback type.In contrast, DRQs were observed the fewest times in problem structuring and preliminary problem-solving phases.Since the design problems in PATs are complex and challenging, learners require a range of lower-order and higher-order formative feedback to guide their problem structuring and preliminary problem-solving.With feedback mainly comprising LLQs, learners are more likely to neglect to explore the problem and subproblems, leading to the production of known and existing solutions.It also means that deeper reflection and cognition needed to solve ill-structured problems did not occur.The high-level feedback questions observed focused on finding a design solution rather than exploring ways to understand the problem or gather information about the design's purpose, client and context.
Similarly, the low-level formative feedback questions and comments aimed to verify and clarify learners' understanding of the design problem and the solution they should be working towards.This means that learners were not supported or encouraged to consider different perspectives when exploring the design problem or generating a solution.Given that many of the formative feedback questions and comments revolved around how learners should approach design activities for assessment, we suspect that most learners produced similar and existing design solutions, even though we did not investigate the design outcomes of the learners.
These findings may contribute to the professional practice of in-service teachers and the curriculum for pre-service teachers' use of formative feedback questions and comments to support learners on four distinct levels in the design process.The findings from this study may also contribute to future research to develop pedagogical guidelines to support technology teachers' implementation of formative feedback.Further research may be necessary to explore how formative feedback through LLQs, LLCs, and high-level questions influence learners' design outcomes.
" and "what should it [the design solution] be able to do?" Learners were also asked to consider "what are your specifications?what are your overall dimensions?" and "what extra detail has your client asked you to add to the product you are going to design?"These questions encouraged learners to consider what design specifications are and what specifications should be included in their designs.

Figure 1 .
Figure 1.Frequency of formative feedback questions & comments by type (Source: Authors' own elaboration)

Figure 2 .
Figure 2. Frequency of LLQ categories observed in problem structuring & preliminary problem-solving phases (Source: Authors' own elaboration)

Figure 3 .
Figure 3. Frequency of LLC categories in problem structuring & preliminary problem-solving phase (Source: Authors' own elaboration)

Figure 5 .
Figure 5. GDQs observed in problem structuring & preliminary problem-solving phase (Source: Authors' own elaboration) The study reveals that technology teachers predominantly use low-level questions during problem structuring and solving episodes, indicating that such guidance is often insufficient and can lead learners to consider only known solutions, preventing deeper generative and evaluative thinking.•Technology teachers' formative feedback during problem structuring and problem solving consisted mainly of low-level questions, with a limited frequency of low-level comments, deep reasoning questions, and generative design questions, resulting in inadequate support for exploring new solutions.• The findings inform teacher education by recommending questioning practices that integrate a balance of low-level and higher-level questioning in technology education to enhance students' cognitive engagement and encourage deeper thinking.• The article identifies the need for further research on the effects of different feedback types on student outcomes in technology education, particularly in terms of how they impact learners' ability to engage in deeper problem exploration and design solution generation.
self) regarding one's performance or understanding."In both professional Contribution to the literature •

Table 2 .
Identification & coding of problem structuring & preliminary problem-solving

Table 3 .
Identification & coding of formative feedback questions & comments