Assessing Probabilistic Reasoning in Verbal-Numerical and Graphical-Pictorial Formats: An Evaluation of the Psychometric Properties of an Instrument

Mirian Agus; Maria Pietronilla Penna; Maribel Peró-Cebollero; Joan Guàrdia-Olmos

doi:10.12973/eurasia.2016.1265a

Full Text (PDF)

Mirian Agus ¹ ^* , Maria Pietronilla Penna ¹, Maribel Peró-Cebollero ², Joan Guàrdia-Olmos ²

More Detail

¹ University of Cagliari - Faculty of Humanistic Studies
² Faculty of Psychology, University of Barcelona, Spain
^* Corresponding Author

Abstract

Background:
Research on the graphical facilitation of probabilistic reasoning has been characterised by the effort expended to identify valid assessment tools. The authors developed an assessment instrument to compare reasoning when problems were presented in verbal-numerical and graphical-pictorial formats.

Material and methods:
A sample of undergraduate psychology students (n=676) who had not developed statistical skills solved problems requiring probabilistic reasoning. They attended universities in Spain (n=127; f=71.7%) and Italy (n=549; f=72.9%). In Italy 173 undergraduates solved these problems in time pressure. The remaining students solved the problems without time limits.

Results:
Classical Test Theory (CTT) and Item Response Theory (IRT) were applied to assess the effect of two formats and to evaluate criterion and discriminant validity.

Conclusions:
The instrument produced acceptable psychometric properties, providing preliminary evidence of validity.

Keywords

probabilistic reasoning
format of problem presentation
verbal-numerical format
graphical-pictorial format
educational assessment
validity

License

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Article Type: Research Article

https://doi.org/10.12973/eurasia.2016.1265a

EURASIA J Math Sci Tech Ed, 2016 - Volume 12 Issue 8, pp. 2013-2038

Publication date: 02 Jul 2016

Article Views: 1123

Article Downloads: 628

Open Access References How to cite this article

References

Agnoli, F., & Krantz, D. H. (1989). Suppressing natural heuristics by formal instruction: The case of the conjunction fallacy. Cognitive Psychology, 21(4), 515–550. Doi:10.1016/0010-0285(89)90017-0
Agus, M., Peró-Cebollero, M., Guàrdia-Olmos, J., & Penna, M. P. (2013). The measurement of statistical reasoning in verbal-numerical and graphical forms: a pilot study. Journal of Physics: Conference Series, 459(1), 012023. http://doi.org/10.1088/1742- 6596/459/1/012023
Agus, M., Peró-Cebollero, M., Penna, M. P., & Guàrdia-Olmos, J. (2014). Towards the development of problems comparing verbal-numerical and graphical formats in statistical reasoning. Quality & Quantity, 49(2), 691–709. http://doi.org/10.1007/s11135-014-0018-7
Agus, M., Peró-Cebollero, M., Penna, M. P., & Guàrdia-Olmos, J. (2015). Comparing Psychology Undergraduates’ Performance in Probabilistic Reasoning under Verbal-Numerical and Graphical-Pictorial Problem Presentation Format: What is the Role of Individual and Contextual Dimensions? Eurasia Journal of Mathematics, Science & Technology Education, 11(4), 735–750. http://doi.org/10.12973/eurasia.2015.1382a
Baker, F. B. (2001). The basics of item response theory. In ERIC clearinghouse on assessment and evaluation. College Park, MD: University of Maryland. Available http://ericae.net/irt/baker.
Barbey, A. K., & Sloman, S. A. (2007). Base-rate respect: From ecological rationality to dual processes. The Behavioral and Brain Sciences, 30(3), 241–254; http://doi.org/10.1017/S0140525X07001653
Batanero, C., & Sánchez, E. (2005). What is the nature of high school students’ conceptions and misconceptions about probability? In G. A. Jones (Ed.), Exploring probability in school (pp. 241–266). Springer. Doi:10.1007/0-387-24530-8_11
Ben-Zvi, D., & Garfield, J. (2004). The challenge of developing statistical literacy, reasoning, and thinking. Netherlands: Kluwer Academic Pub.
Bentler, P. M. (1995). EQS structural equations program manual (Encino, CA., Vol. Multivariate). Multivariate Software.
Bishop, A. (2008). Spatial abilities and mathematics education – A review. In P. Clarkson & N. Presmeg (Eds.), Critical issues in mathematics education SE - 5 (pp. 71–81). Springer US. Doi:10.1007/978-0-387-09673-5_5
Braithwaite, D. W., & Goldstone, R. L. (2013a). Flexibility in data interpretation: Effects of representational format. Frontiers in Psychology, 4(DEC), 980. Doi:10.3389/fpsyg.2013.00980
Braithwaite, D. W., & Goldstone, R. L. (2013b). Integrating formal and grounded representations in combinatorics learning. Journal of Educational Psychology, 105(3), 666–682. Doi:http://dx.doi.org/10.1037/a0032095
Brase, G. L. (2008). Frequency interpretation of ambiguous statistical information facilitates Bayesian reasoning. Psychonomic Bulletin & Review, 15(2), 284–289. Doi:10.3758/PBR.15.2.284
Brase, G. L. (2009). Pictorial representations in statistical reasoning. Applied Cognitive Psychology, 23(3), 369–381. Doi:10.1002/acp.1460
Brase, G. L., Cosmides, L., & Tooby, J. (1998). Individuation, counting, and statistical inference: The role of frequency and whole-object representations in judgment under uncertainty. Journal of Experimental Psychology: General, 127(1), 3. Doi: http://dx.doi.org/10.1037/0096-3445.127.1.3
Brase, G. L., & Hill, W. T. (2015). Good fences make for good neighbors but bad science: a review of what improves Bayesian reasoning and why. Frontiers in Psychology, 6, 340. http://doi.org/10.3389/fpsyg.2015.00340
Cai, L., Thissen, D., & du Toit, S. H. C. (2011). IRTPRO for Windows [Computer software]. Lincolnwood, IL: Scientific Software International (Scientific.). Lincolnwood, IL.
Castañeda, L. E. G., & Knauff, M. (2013). Individual differences, imagery and the visual impedance effect. In M. Knauff, N. Sebanz, M. Pauen, I. Wachsmuth, (Eds.), Cooperative minds: Social interaction and group dynamics. Proceedings of the 35th Annual Meeting of the Cognitive Science Society Berlin, Germany, July 31-August 3, 2013 (pp. 2374–9). Austin, TX: Cognitive Science Society. ISBN: 978-0-9768318-9-1
Chan, S. W., & Ismail, Z. (2014). A technology-based statistical reasoning assessment tool in descriptive statistics for secondary school students. Turkish Online Journal of Educational Technology, 13(1), 29–46. Retrieved from http://www.scopus.com/inward/record.url?eid=2-s2.0- 84891655563&partnerID=tZOtx3y1
Chance, B. L. (2002). Components of statistical thinking and implications for instruction and assessment. Journal of Statistics Education, 10(3), 1–18.
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255. Doi:10.1207/S15328007SEM0902_5
Chiesi, F., & Primi, C. (2009). Un modello sul rendimento nelle materie quantitative degli studenti di psicologia. Giornale Italiano Di Psicologia, 36(1), 161–184.
Chiesi, F., Primi, C., & Morsanyi, K. (2011). Developmental changes in probabilistic reasoning: The role of cognitive capacity, instructions, thinking styles, and relevant knowledge. Thinking & Reasoning, 17(3), 315–350. Doi:10.1080/13546783.2011.598401
Clements, M. (2014). Fifty years of thinking about visualization and visualizing in mathematics education: A historical overview. In M. N. Fried & T. Dreyfus (Eds.), Mathematics & mathematics education: Searching for common ground SE - 11 (pp. 177– 192). Springer Netherlands. Doi:10.1007/978-94-007-7473-5_11
Cokely, E. T., Galesic, M., Schulz, E., Ghazal, S., & Garcia-Retamero, R. (2012). Measuring risk literacy: The Berlin Numeracy Test. Judgment & Decision Making, 7(1). Retrieved from http://journal.sjdm.org/11/11808/jdm11808.html
Corter, J. E., & Zahner, D. C. (2007). Use of external visual representations in probability problem solving. Statistics Education Research Journal, 6(1), 22–50. Retrieved from https://www.stat.auckland.ac.nz/~iase/serj/SERJ6(1)_Corter_Zahner.pdf
Cosmides, L., & Tooby, J. (1996). Are humans good intuitive statisticians after all? Rethinking some conclusions from the literature on judgment under uncertainty. Cognition, 58(1), 1–73. Doi:10.1016/0010-0277(95)00664-8
Del Mas, R. (2004). A Comparison of Mathematical and Statistical Reasoning. In D. Ben-Zvi & J. Garfield (Eds.), The Challenge of Developing Statistical Literacy, Reasoning and Thinking SE - 4 (pp. 79–95). Springer Netherlands. http://doi.org/10.1007/1-4020- 2278-6_4
De Ayala, R. J. (2009). Theory and practice of item response theory. Guilford Publications.
De Hevia, M. D., Vallar, G., & Girelli, L. (2008). Visualizing numbers in the mind’s eye: the role of visuo-spatial processes in numerical abilities. Neuroscience and Bio-behavioral Reviews, 32(8), 1361–72. Doi:10.1016/j.neubiorev.2008.05.015
Díaz, C., & De La Fuente, I. (2006). Assessing psychology students’ difficulties with conditional probability and bayesian reasoning. In A. R. & B. Chance (Ed.), Proceedings of the Seventh International Conference on Teaching Statistics. Retrieved from https://www.stat.auckland.ac.nz/~iase/publications/17/5E3_DIAZ.pdf
Ebel, R., & Frisbie, D. (1991). Essentials of educational measurement (5th ed., p. 383). Englewood Cliffs, NJ. P: Prentice-Hall.
Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. New Jersey: Lawrence Erlbaum Associates.
Evans, J. S. B. T., Handley, S. J., Neilens, H., & Over, D. (2010). The influence of cognitive ability and instructional set on causal conditional inference. The Quarterly Journal of Experimental Psychology, 63(5), 892–909. Doi:10.1080/17470210903111821
Evans, J. S. B. T., Handley, S. J., Perham, N., Over, D. E., & Thompson, V. A. (2000). Frequency versus probability formats in statistical word problems. Cognition, 77(3), 197–213. Doi:10.1016/S0010-0277(00)00098-6
Evans, J. S. B. T., & Stanovich, K. E. (2013). Dual-Process Theories of higher cognition: advancing the debate. Perspectives on Psychological Science, 8(3), 223–241. http://doi.org/10.1177/1745691612460685
Fan, X. (1998). Item Response Theory and Classical Test Theory: An empirical comparison of their Item/Person statistics. Educational and Psychological Measurement, 58(3), 357– 381. Doi:10.1177/0013164498058003001
Franklin, C., Kader, G., Mewborn, D., Moreno, J., Peck, R., Perry, M., & Scheaffer, R. (2007). Guidelines for assessment and instruction in statistics education (GAISE). Report. Alexandria: American Statistical Association. Retrieved from http://www.amstat.org/Education/gaise/GAISEPreK-12_Full.pdf
Frey, A., & Seitz, N.N. (2009). Multidimensional adaptive testing in educational and psychological measurement: Current state and future challenges. Studies in Educational Evaluation, 35(2–3), 89–94. http://doi.org/http://dx.doi.org/10.1016/j.stueduc.2009.10.007
Furlan, S., & Agnoli, F. (2010). The dark side of statistics: Numeracy and luck in the development of probabilistic reasoning. In ICOTS8 - Data and context in statistics education: Towards an evidence-based society. Retrieved from http://icots.info/8/cd/pdfs/posters/ICOTS8_P11_FURLAN.pdf
Gal, I. (2002). Adults’ statistical literacy: Meanings, components, responsibilities. International Statistical Review, 70(1), 1–25. http://doi.org/10.1111/j.1751- 5823.2002.tb00336.x
Gal, I. (2005). Towards “probability literacy” for all citizens: Building blocks and instructional dilemmas. In G. A. Jones (Ed.), Exploring probability in school (pp. 39–63). Springer. Doi: 10.1007/0-387-24530-8_3
Galli, S., Chiesi, F., & Primi, C. (2011). Measuring mathematical ability needed for “nonmathematical” majors: The construction of a scale applying IRT and differential item functioning across educational contexts. Learning and Individual Differences, 21(4), 392–402. Doi:10.1016/j.lindif.2011.04.005
García-Retamero, R., Galesic, M., & Gigerenzer, G. (2011). Cómo favorecer la comprensión y la comunicación de los riesgos sobre la salud. Psicothema, 23(4), 599–605.
Garfield, J. (2003). Assessing statistical reasoning. Statistics Education Research Journal, 2(1), 22–38. Retrieved from http://iase-web.org/documents/SERJ/SERJ2(1).pdf#page=24
Garfield, J., & Ben-Zvi, D. (2008). Developing students’ statistical reasoning: Connecting research and teaching practice. New York: Springer Verlag.
Gigerenzer, G. (2008). Rationality for mortals: How people cope with uncertainty. New York: Oxford University Press.
Gigerenzer, G., & Hoffrage, U. (1995). How to improve Bayesian reasoning without instruction: Frequency formats. Psychological Review, 102(4), 684–704. Doi: http://dx.doi.org/10.1037/0033-295X.102.4.684
Gilovich, T., Griffin, D., & Kahneman, D. (2002). Heuristics and biases: The psychology of intuitive judgment. Cambridge University Press.
Girotto, V., & Gonzalez, M. (2001). Solving probabilistic and statistical problems: A matter of information structure and question form. Cognition, 78(3), 247–276. Doi:10.1016/S0010-0277(00)00133-5
Girotto, V., & Gonzalez, M. (2008). Children’s understanding of posterior probability. Cognition, 106(1), 325–344. Doi:10.1016/j.cognition.2007.02.005
Glas, C. A. W. (2001). Differential item functioning depending on general covariates. In Essays on item response theory (pp. 131–148). Berlin / Heidelberg: Springer.
González M.A., Campos A., Perez M. J. (1997). Mental imagery and creative thinking. The Journal of Psychology, 131(4), 357–364. Doi:10.1080/00223989709603521
Guàrdia-Olmos, J., Freixa-Blanxart, M., Peró-Cebollero, M., Turbany, J., Cosculluela, A., Barrios, M., & Rifà, X. (2006). Factors related to the academic performance of students in the statistics course in psychology. Quality & Quantity, 40(4), 661–674. Doi:10.1007/s11135-005-2072-7
Hays, R. D., Brown, J., Brown, L. U., Spritzer, K. L., & Crall, J. J. (2006). Classical test theory and item response theory analyses of multi-item scales assessing parents’ perceptions of their children's dental care. Med Care, 44, S60–S68. Doi:10.1097/01.mlr.0000245144.90229.d0
Hartig, J., & Höhler, J. (2009). Multidimensional IRT models for the assessment of competencies. Studies in Educational Evaluation, 35(2–3), 57–63. http://doi.org/http://dx.doi.org/10.1016/j.stueduc.2009.10.002
Hoffrage, U., Gigerenzer, G., Krauss, S., & Martignon, L. (2002). Representation facilitates reasoning: What natural frequencies are and what they are not. Cognition, 84(3), 343– 352. Doi: 10.1016/S0010-0277(02)00050-1
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. Doi:10.1080/10705519909540118
Huggins, A. C. (2012). The Effect of Differential Item Functioning on Population Invariance of Item Response Theory True Score Equating. University of Miami.
Huggins, A. C. (2014). The Effect of Differential Item Functioning in Anchor Items on Population Invariance of Equating. Educational and Psychological Measurement, 74 (4), 627–658. Doi:10.1177/0013164413506222
Johnson, E. D., & Tubau, E. (2013). Words, numbers, & numeracy: Diminishing individual differences in Bayesian reasoning. Learning and Individual Differences, 28, 34–40. Doi:10.1016/j.lindif.2013.09.004
Johnson, M. E., Pierce, C. A., Baldwin, K., Harris, A., & Brondmo, A. K. (1996). Presentation Format in Analogue Studies: Effects on Participants Evaluations. The Journal of Psychology, 130(3), 341–349. Doi:10.1080/00223980.1996.9915015
Jones, G. A. (2006). Exploring probability in school: Challenges for teaching and learning (Vol. 40). Springer.
Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124–1131. http://doi.org/10.1126/science.185.4157.1124
Kellen, V., Chan, S., & Fang, X. (2006). Individual Differences in Spatial Abilities and the Visualization of Conditional Probabilities. Retrieved from http://www.kellen.net/visualization_wp.htm
Kellen, V., Chan, S., & Fang, X. (2007). Facilitating Conditional Probability Problems with Visuals. In J. Jacko (Ed.), Human-Computer Interaction. Interaction Platforms and Techniques (Vol. 4551, pp. 63–71). Berlin / Heidelberg: Springer. Doi:10.1007/978-3- 540-73107-8_8
Kellen, V., Chan, S., & Fang, X. (2013). Improving user performance in conditional probability problems with computer-generated diagrams. In Human-Computer Interaction. Users and Contexts of Use (pp. 183–192). Springer
Klaczynski, P. A. (2014). Heuristics and biases: interactions among numeracy, ability, and reflectiveness predict normative responding. Frontiers in Psychology, 5, 665. http://doi.org/10.3389/fpsyg.2014.00665
Kline, P. (2000). Handbook of psychological testing. Routledge.
Knauff, M., & Johnson-Laird, P. N. (2002). Visual imagery can impede reasoning. Memory & Cognition, 30(3), 363–371. Doi:10.3758/BF03194937
Konold, C., Higgins, T., Russell, S. J., & Khalil, K. (2014). Data seen through different lenses. Educational Studies in Mathematics, 1–21. Doi:10.1007/s10649-013-9529-8
Kubiszyn, T., & Borich, G. (1990). Educational testing and measurement. Harper Collins Publishers.
Kunina-Habenicht, O., Rupp, A. A., & Wilhelm, O. (2009). A practical illustration of multidimensional diagnostic skills profiling: Comparing results from confirmatory factor analysis and diagnostic classification models. Studies in Educational Evaluation, 35(2–3), 64–70. http://doi.org/http://dx.doi.org/10.1016/j.stueduc.2009.10.003
Lalonde, R. N., & Gardner, R. C. (1993). Statistics as a second language? A model for predicting performance in psychology students. Canadian Journal of Behavioural Science/Revue Canadienne Des Sciences Du Comportement, 25(1), 108–125. http://doi.org/http://dx.doi.org/10.1037/h0078792
Langrall, C. W., & Mooney, E. S. (2005). Characteristics of Elementary School Students’ Probabilistic Reasoning. In G. A. Jones (Ed.), Exploring Probability in School (pp. 95– 119). Springer. http://doi.org/10.1007/0-387-24530-8_5
Laverdière, O., Morin, A. J. S., & St-Hilaire, F. (2013). Factor structure and measurement invariance of a short measure of the Big Five personality traits. Personality and Individual Differences, 55(7), 739–743. http://doi.org/10.1016/j.paid.2013.06.008
Lean, G., & Clements, M. A. (1981). Spatial ability, visual imagery, and mathematical performance. Educational Studies in Mathematics, 12(3), 267–299. Doi:10.1007/BF00311060
Lim, S., & Chapman, E. (2013). Development of a short form of the attitudes toward mathematics inventory. Educational Studies in Mathematics, 82(1), 145–164. Doi:10.1007/s10649-012-9414-x
Manor, H., Ben-Zvi, D., & Aridor, K. (2014). Student’s reasoning about uncertainty while making informal statistical inferences in an “Integrated Pedagogic Approach.” In & R. G. Makar, B. de Sousa (Ed.), Sustainability in statistics education. Proceedings of the Ninth International Conference on Teaching Statistics (ICOTS9, July, 2014), Flagstaff, Arizona, USA. Voorburg, The Netherlands: International Statistic. International Statistical Institute.
Mandel, D. R. (2014). The psychology of Bayesian reasoning. Frontiers in Psychology, 5. http://doi.org/10.3389/fpsyg.2014.01144
Mandel, D. R. (2015). Instruction in information structuring improves Bayesian judgment in intelligence analysts. Frontiers in Psychology. http://doi.org/10.3389/fpsyg.2015.00387
Maule, A. J., Hockey, G. R. J., & Bdzola, L. (2000). Effects of time-pressure on decision-making under uncertainty: changes in affective state and information processing strategy. Acta Psychologica, 104(3), 283–301. Doi:10.1016/S0001-6918(00)00033-0
Messick, S. (1996). Validity and washback in language testing. Language Testing , 13 (3 ), 241–256. Doi:10.1177/026553229601300302
Moore, D. S. (1990). Uncertainty. On the Shoulders of Giants: New Approaches to Numeracy, 95–137.
Moro, R., & Bodanza, G. A. (2010). El debate acerca del efecto facilitador en problemas de probabilidad condicional: ¿Un caso de experimentación crucial? Interdisciplinaria, 27(1), 163–174.
Moro, R., Bodanza, G. A., & Freidin, E. (2011). Sets or frequencies? How to help people solve conditional probability problems. Journal of Cognitive Psychology, 23(7), 843–857. Doi:10.1080/20445911.2011.579072
Onwuegbuzie, A. J., & Seaman, M. A. (1995). The effect of time constraints and statistics test anxiety on test performance in a statistics course. The Journal of Experimental Education, 63(2), 115–124. Doi: 10.1080/00220973.1995.9943816
Paek, I., & Han, K. T. (2013). IRTPRO 2.1 for Windows (Item Response Theory for PatientReported Outcomes). Applied Psychological Measurement, 37(3), 242–252. Doi:10.1177/0146621612468223
Paivio, A. (1971). Imagery and verbal processes. New York: Holt, Rinehart & Winston.
Pastore, M., & Lombardi, L. (2014). The impact of faking on Cronbach’s alpha for dichotomous and ordered rating scores. Quality & Quantity, 48(3), 1191–1211. http://doi.org/10.1007/s11135-013-9829-1
Penna, M. P., Agus, M., Peró-Cebollero, M., Guàrdia-Olmos, J., & Pessa, E. (2014). The use of imagery in statistical reasoning by university undergraduate students: a preliminary study. Quality & Quantity, 48(1), 173–187. http://doi.org/10.1007/s11135-012-9757-5
Pessa, E., & Penna, M. P. (2000). Manuale di scienza cognitiva. Intelligenza artificiale classica e psicologia cognitiva. Roma - Bari: Editori Laterza.
Pollard, B., Dixon, D., Dieppe, P., & Johnston, M. (2009). Measuring the ICF components of impairment, activity limitation and participation restriction: an item analysis using classical test theory and item response theory. Health and Quality of Life Outcomes, 7(1), 41. Doi:10.1186/1477-7525-7-41
Reeve, B. B., & Fayers, P. (2005). Applying item response theory modelling for evaluating questionnaire item and scale properties. Assessing Quality of Life in Clinical Trial: Methods and Practice, 55–74. Doi: 10.1007/s11136-007-9198-0
Rieskamp, J., & Hoffrage, U. (2008). Inferences under time pressure: How opportunity costs affect strategy selection. Acta Psychologica, 127(2), 258–276. Doi:10.1016/j.actpsy.2007.05.004
Rumsey, D. J. (2002). Statistical literacy as a goal for introductory statistics courses. Journal of Statistics Education, 10(3), 6–13. Retrieved from http://www.amstat.org/publications/jse/v10n3/rumsey2.html
Salehi, B., Cordero, M. I., & Sandi, C. (2010). Learning under stress: the inverted-U-shape function revisited. Learning & Memory, 17(10), 522–530. Doi:10.1101/lm.1914110
Schermelleh-Engel, K., Moosbrugger, H., & Müller, H. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8(2), 23–74.
Schinka, J. A., & Velicer, W. F. (2003). Handbook of Psychology - vol. 2 - Research methods in psychology. (I. B. Weiner, Ed.) (p. 738). Wiley.
Schonlau, M., & Peters, E. (2012). Comprehension of graphs and tables depend on the task: empirical evidence from two web-based studies. Statistics, Politics, and Policy, 3(2). Doi:10.1515/2151-7509.1054
Sharps, M. J., Hess, A. B., Price-Sharps, J. L., & Teh, J. (2008). Heuristic and algorithmic processing in English, Mathematics, and Science Education. The Journal of Psychology, 142(1), 71–88. Doi:10.3200/JRLP.142.1.71-88
Sijtsma, K. (2009a). Reliability beyond theory and into practice. Psychometrika, 74(1), 169– 173. Doi:10.1007/s11336-008-9103-y
Sijtsma, K. (2009b). On the Use, the Misuse, and the Very Limited Usefulness of Cronbach’s Alpha. Psychometrika, 74(1), 107–120. http://doi.org/10.1007/s11336-008-9101-0
Singh, J. (2004). Tackling measurement problems with Item Response Theory: Principles, characteristics, and assessment, with an illustrative example. Journal of Business Research, 57, 184–208. Doi:10.1016/S0148-2963(01)00302-2
Sirota, M., & Juanchich, M. (2011). Role of numeracy and cognitive reflection in Bayesian reasoning with natural frequencies. Studia Psychologica, 53(2), 151–161. Retrieved from http://yadda.icm.edu.pl/cejsh/element/bwmeta1.element.defbace5-44e3-3d1b8464-09dde16918e3
Sirota, M., Kostovičová, L., & Juanchich, M. (2014). The effect of iconicity of visual displays on statistical reasoning: evidence in favor of the null hypothesis. Psychonomic Bulletin & Review, 21, 961–968. http://doi.org/10.3758/s13423-013-0555-4
Sirota, M., Kostovičová, L., & Vallée-Tourangeau, F. (2015). Now you Bayes, now you don’t: effects of set-problem and frequency-format mental representations on statistical reasoning. Psychonomic Bulletin & Review, 1–9. http://doi.org/10.3758/s13423-015- 0810-y
Sloman, S. A., Over, D., Slovak, L., & Stibel, J. M. (2003). Frequency illusions and other fallacies. Organizational Behavior and Human Decision Processes, 91(2), 296–309. Doi:10.1016/S0749-5978(03)00021-9
Thurstone, L. L., & Thurstone, T. G. (1981). PMA - abilità mentali primarie: manuale di istruzioni Livello intermedio (11-17)(Batteria fattoriale delle abilità mentali primarie). Firenze: Organizzazioni Speciali.
Thurstone, L. L., & Thurstone, T. G. (1987). TEA: tests de aptitudes escolares [niveles 1, 2 y 3]Manual. Madrid: Tea.
Tubau, E. (2008). Enhancing probabilistic reasoning: The role of causal graphs, statistical format and numerical skills. Learning and Individual Differences, 18(2), 187–196. Doi:10.1016/j.lindif.2007.08.006
Tufte, E. R. (2001). The visual display of quantitative information. Visual Explanations (pp. 194–195). Cheshire, Connecticut: Graphics Press
Tversky, A., & Kahneman, D. (1983). Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment. Psychological Review, 90(4), 293–315. Doi: http://dx.doi.org/10.1037/0033-295X.90.4.293
Wang, M., & Russell, S. S. (2005). Measurement Equivalence of the Job Descriptive Index Across Chinese and American Workers: Results from Confirmatory Factor Analysis and Item Response Theory. Educational and Psychological Measurement, 65 (4), 709–732. Doi:10.1177/0013164404272494
Watson, J. M., & Moritz, J. B. (2003). Fairness of dice: A longitudinal study of students’ beliefs and strategies for making judgments. Journal for Research in Mathematics Education, 270–304.
Wild, C. J., & Pfannkuch, M. (1999). Statistical thinking in empirical enquiry. International Statistical Review, 67(3), 223–248. http://doi.org/10.1111/j.1751- 5823.1999.tb00442.x
Yamagishi, K. (2003). Facilitating normative judgments of conditional probability: Frequency or nested sets? Experimental Psychology (formerly Zeitschrift Für Experimentelle Psychologie), 50(2), 97–106. Doi:10.1026//1618-3169.50.2.97
Zahner, D., & Corter, J. E. (2010). The process of probability problem solving: Use of external visual representations. Mathematical Thinking and Learning, 12(2), 177–204. Doi:10.1080/10986061003654240
Zhu, L., & Gigerenzer, G. (2006). Children can solve Bayesian problems: The role of representation in mental computation. Cognition, 98(3), 287–308. Doi: http://dx.doi.org/10.1016/j.cognition.2004.12.003

How to cite this article

APA

Agus, M., Penna, M. P., Peró-Cebollero, M., & Guàrdia-Olmos, J. (2016). Assessing Probabilistic Reasoning in Verbal-Numerical and Graphical-Pictorial Formats: An Evaluation of the Psychometric Properties of an Instrument. Eurasia Journal of Mathematics, Science and Technology Education, 12(8), 2013-2038. https://doi.org/10.12973/eurasia.2016.1265a

Vancouver

Agus M, Penna MP, Peró-Cebollero M, Guàrdia-Olmos J. Assessing Probabilistic Reasoning in Verbal-Numerical and Graphical-Pictorial Formats: An Evaluation of the Psychometric Properties of an Instrument. EURASIA J Math Sci Tech Ed. 2016;12(8):2013-38. https://doi.org/10.12973/eurasia.2016.1265a

AMA

Agus M, Penna MP, Peró-Cebollero M, Guàrdia-Olmos J. Assessing Probabilistic Reasoning in Verbal-Numerical and Graphical-Pictorial Formats: An Evaluation of the Psychometric Properties of an Instrument. EURASIA J Math Sci Tech Ed. 2016;12(8), 2013-2038. https://doi.org/10.12973/eurasia.2016.1265a

Chicago

Agus, Mirian, Maria Pietronilla Penna, Maribel Peró-Cebollero, and Joan Guàrdia-Olmos. "Assessing Probabilistic Reasoning in Verbal-Numerical and Graphical-Pictorial Formats: An Evaluation of the Psychometric Properties of an Instrument". Eurasia Journal of Mathematics, Science and Technology Education 2016 12 no. 8 (2016): 2013-2038. https://doi.org/10.12973/eurasia.2016.1265a

Harvard

Agus, M., Penna, M. P., Peró-Cebollero, M., and Guàrdia-Olmos, J. (2016). Assessing Probabilistic Reasoning in Verbal-Numerical and Graphical-Pictorial Formats: An Evaluation of the Psychometric Properties of an Instrument. Eurasia Journal of Mathematics, Science and Technology Education, 12(8), pp. 2013-2038. https://doi.org/10.12973/eurasia.2016.1265a

MLA

Agus, Mirian et al. "Assessing Probabilistic Reasoning in Verbal-Numerical and Graphical-Pictorial Formats: An Evaluation of the Psychometric Properties of an Instrument". Eurasia Journal of Mathematics, Science and Technology Education, vol. 12, no. 8, 2016, pp. 2013-2038. https://doi.org/10.12973/eurasia.2016.1265a