Academia.eduAcademia.edu

Figure 20 – uploaded by Jason Ravitz

See full PDFdownloadDownload figure
fable 20. Tests of Between Subjects Effects for Factorial ANOVA Testing Interaction Between Group and Starting Point (Science)

Table 20 fable 20. Tests of Between Subjects Effects for Factorial ANOVA Testing Interaction Between Group and Starting Point (Science)

Related Figures (58)

Contents
Contents
able 1. Mean 21st Century Skills Index by PBL Use (Two and Three Categories) Statistical test data are available in Table A 7 (p. 71) for t tests and Table A 8 (p. 72) for ANOVA tests. Effect sizes for the three categories are based on means for the first and third cat- egories only (i.e., no PBL emphasis or use, versus PBL use with extended professional develop- ment). Tukey post-hoc analyses (available from the author) suggested that this comparison was statistically significant (p < .001), while the other comparisons did not reach statistical signifi- cance.
able 1. Mean 21st Century Skills Index by PBL Use (Two and Three Categories) Statistical test data are available in Table A 7 (p. 71) for t tests and Table A 8 (p. 72) for ANOVA tests. Effect sizes for the three categories are based on means for the first and third cat- egories only (i.e., no PBL emphasis or use, versus PBL use with extended professional develop- ment). Tukey post-hoc analyses (available from the author) suggested that this comparison was statistically significant (p < .001), while the other comparisons did not reach statistical signifi- cance.
Results for 21st century skills: Extensiveness of PBL use able 2. Mean 21st Century Skills Practices for Each Skill by PBL Use
Results for 21st century skills: Extensiveness of PBL use able 2. Mean 21st Century Skills Practices for Each Skill by PBL Use
able 3. Mean Perceptions of 21st Century Skills Taught by PBL Use
able 3. Mean Perceptions of 21st Century Skills Taught by PBL Use
Table 4. Mean 21st Century Skills Index by Subject and PBL Use There were variations across content areas in how much the different skills were taught. Table A 12 (p. 77) shows the overall index score (for frequency and extensiveness combined) of each skill by subject area. Mathematics teachers on average reported teaching the 21st century skill practices least frequently, while science teachers reported above average use of practices specifically related to collaboration and communication skills. English and social studies teach- ers reported the most frequent use of the practices overall, with a large portion of the social studies effect coming from global connections, the only skill for which there was a statistically significant difference across the four content areas (Table A 13, p. 77).
Table 4. Mean 21st Century Skills Index by Subject and PBL Use There were variations across content areas in how much the different skills were taught. Table A 12 (p. 77) shows the overall index score (for frequency and extensiveness combined) of each skill by subject area. Mathematics teachers on average reported teaching the 21st century skill practices least frequently, while science teachers reported above average use of practices specifically related to collaboration and communication skills. English and social studies teach- ers reported the most frequent use of the practices overall, with a large portion of the social studies effect coming from global connections, the only skill for which there was a statistically significant difference across the four content areas (Table A 13, p. 77).
ble 5. Mean 21st Century Skills Index by Class Academic Performance and PBL Use
ble 5. Mean 21st Century Skills Index by Class Academic Performance and PBL Use
Mean 21st Century Skills Index by Block Schedule and PBL Use
Mean 21st Century Skills Index by Block Schedule and PBL Use
Of the various background and contextual measures, this indicator of teacher back- ground is the most noteworthy because there were not only significant differences in mean re- sponses, but there were significant differences in population distribution. This suggests that professional development could have a significant impact on the overall results. However, when we control for this teacher background variable—focusing only on the group that had led profes- sional development by using split-file analyses (Table 7 above and Table A 7, p. 71)—we still see 7. Mean 21st Century Skills Index by Teacher as PD Leader and PBL Use
Of the various background and contextual measures, this indicator of teacher back- ground is the most noteworthy because there were not only significant differences in mean re- sponses, but there were significant differences in population distribution. This suggests that professional development could have a significant impact on the overall results. However, when we control for this teacher background variable—focusing only on the group that had led profes- sional development by using split-file analyses (Table 7 above and Table A 7, p. 71)—we still see 7. Mean 21st Century Skills Index by Teacher as PD Leader and PBL Use
8. Number of Survey Respondents by PBL Implementation Status Of the 24 teachers who reported extensive PBL use, we were able to verify via course scheduling rosters retained by the WVDE that 17 survey respondents self-reported implement- ing PBL in a target course that was offered during SY2011 (71%). These teachers are hereafter referred to as PBL teachers and were identified as those teachers from whose courses students were selected. The breakdown of PBL teachers by content area is shown in Table 9.
8. Number of Survey Respondents by PBL Implementation Status Of the 24 teachers who reported extensive PBL use, we were able to verify via course scheduling rosters retained by the WVDE that 17 survey respondents self-reported implement- ing PBL in a target course that was offered during SY2011 (71%). These teachers are hereafter referred to as PBL teachers and were identified as those teachers from whose courses students were selected. The breakdown of PBL teachers by content area is shown in Table 9.
ible 9. Number of WVEIS Validated PBL Teachers by Content Area We next collected unique student identifiers for all those students who were actively en- rolled in the target courses offered by PBL teachers (hereafter referred to as the PBL group). We excluded students in Grades 3 and 12 because we required two points of assessment data to an- swer RQ2.’ A total of 821 unduplicated students were identified within all offered sections of the target courses reported by PBL teachers. The breakdown of students by content area is shown in Tahle 10.
ible 9. Number of WVEIS Validated PBL Teachers by Content Area We next collected unique student identifiers for all those students who were actively en- rolled in the target courses offered by PBL teachers (hereafter referred to as the PBL group). We excluded students in Grades 3 and 12 because we required two points of assessment data to an- swer RQ2.’ A total of 821 unduplicated students were identified within all offered sections of the target courses reported by PBL teachers. The breakdown of students by content area is shown in Tahle 10.
Table 10. Number of Students in PBL Courses by Content Area Sampling procedures
Table 10. Number of Students in PBL Courses by Content Area Sampling procedures
Each PSM model included one interval level covariate as well for which chi-square anal- yses were not appropriate (prior academic achievement). Again, for this covariate we presumed a lack of statistically significant differences among the groups to represent evidence of a success- ful matching process. We conducted a set of four independent samples t tests to test this as- sumption. Significance values for each t¢ test indicated that the groups did not differ significantly on baseline academic achievement; thus, we concluded that for no content area did one group have a significantly different starting point. Table 12 presents a summary of the results of the t tests and Figure 3 provides a graphical representation of the differences in baseline achievement between the groups by content area. Table 11. Verification of Matching for Categorical Covariates
Each PSM model included one interval level covariate as well for which chi-square anal- yses were not appropriate (prior academic achievement). Again, for this covariate we presumed a lack of statistically significant differences among the groups to represent evidence of a success- ful matching process. We conducted a set of four independent samples t tests to test this as- sumption. Significance values for each t¢ test indicated that the groups did not differ significantly on baseline academic achievement; thus, we concluded that for no content area did one group have a significantly different starting point. Table 12 presents a summary of the results of the t tests and Figure 3 provides a graphical representation of the differences in baseline achievement between the groups by content area. Table 11. Verification of Matching for Categorical Covariates
In conclusion, for all covariates in all content areas we believe the groups to have been adequately matched. In fact, the data indicated to us that the PSM analyses were remarkably successful in providing a matched group on the covariates of interest with the exception of school district. Our additional analyses indicated that this difference is likely of little concern to our examination because, while district is sometimes related to students’ gain scores when ex- amined independently, there were no statistically significant interaction effects between district and group—indicating that the potential influence of school district upon gain score is not dif- ferentially present among our groups. Taken together, these results allow us to be confident that our groups are reasonably equivalent on all measured covariates. As such, we proceeded to the main analysis within each content area.
In conclusion, for all covariates in all content areas we believe the groups to have been adequately matched. In fact, the data indicated to us that the PSM analyses were remarkably successful in providing a matched group on the covariates of interest with the exception of school district. Our additional analyses indicated that this difference is likely of little concern to our examination because, while district is sometimes related to students’ gain scores when ex- amined independently, there were no statistically significant interaction effects between district and group—indicating that the potential influence of school district upon gain score is not dif- ferentially present among our groups. Taken together, these results allow us to be confident that our groups are reasonably equivalent on all measured covariates. As such, we proceeded to the main analysis within each content area.
Table 12. Verification of Matching for Interval Level Covariate
Table 12. Verification of Matching for Interval Level Covariate
13. Distribution of Students by Mathematics Course Code and Group
13. Distribution of Students by Mathematics Course Code and Group
14. Distribution of Students by English/Language Arts Course Code and Group
14. Distribution of Students by English/Language Arts Course Code and Group
able 15. Distribution of Students by Science Course Code and Group
able 15. Distribution of Students by Science Course Code and Group
Table 17. Power Analyses for Content Area T Tests Impact of nonequivalent starting points
Table 17. Power Analyses for Content Area T Tests Impact of nonequivalent starting points
able 18. Tests of Between Subjects Effects for Factorial ANOVA Testing Interaction Between Group and Starting Point (Mathematics)
able 18. Tests of Between Subjects Effects for Factorial ANOVA Testing Interaction Between Group and Starting Point (Mathematics)
19. Tests of Between Subjects Effects for Factorial ANOVA Testing Interaction Between Group and Starting Point (English/Language Arts)
19. Tests of Between Subjects Effects for Factorial ANOVA Testing Interaction Between Group and Starting Point (English/Language Arts)
NOTE: In recognition of the importance of this study, we have received external support from the Buck Institute for Education that will provide $15 Amazon.com gift cards to 1 out of every 3 teachers who completes this survey.
NOTE: In recognition of the importance of this study, we have received external support from the Buck Institute for Education that will provide $15 Amazon.com gift cards to 1 out of every 3 teachers who completes this survey.
Ansvrer for questions 1- 3. In recent years — since 2008 — has your teaching or professional development included a SIGNIFICANT FOCUS on.
Ansvrer for questions 1- 3. In recent years — since 2008 — has your teaching or professional development included a SIGNIFICANT FOCUS on.
21st Century Teaching and Learning Survey [WVDE-CIS-28] Please refer to this TARGET CLASS when answering the rest of this survey.
21st Century Teaching and Learning Survey [WVDE-CIS-28] Please refer to this TARGET CLASS when answering the rest of this survey.
21st Century Teaching and Learning Survey [WVDE-CIS-28] Please refer to this TARGET CLASS when answering the rest of this survey.
21st Century Teaching and Learning Survey [WVDE-CIS-28] Please refer to this TARGET CLASS when answering the rest of this survey.
Please ESTIMATE how many students in your TARGET CLASS...
Please ESTIMATE how many students in your TARGET CLASS...
In your teaching of your TARGET CLASS, how often have you asked students to do the following?
In your teaching of your TARGET CLASS, how often have you asked students to do the following?
Table A1. Survey Respondent Recoding Based on Background Information From Survey Responses
Table A1. Survey Respondent Recoding Based on Background Information From Survey Responses
Table A 2. Factor Analysis of the Last Four 21st Century Skills Appendix D. Index Construction Factor Analyses
Table A 2. Factor Analysis of the Last Four 21st Century Skills Appendix D. Index Construction Factor Analyses
Table A 2. Factor Analysis of the Last Four 21st Century Skills
Table A 2. Factor Analysis of the Last Four 21st Century Skills
Table A 3. Factor Analysis of the First Four 21st Century Skills
Table A 3. Factor Analysis of the First Four 21st Century Skills
Table A4. Factor Analysis of Perceptions Items for First Four 21st Century skills
Table A4. Factor Analysis of Perceptions Items for First Four 21st Century skills
Table A6. Mean 21st Century Skills Correlated to Professional Development and Extended Assignments
Table A6. Mean 21st Century Skills Correlated to Professional Development and Extended Assignments
Table A 7. Independent Samples T Test of Means, by PBL Use
Table A 7. Independent Samples T Test of Means, by PBL Use
Table A8. ANOVA Tests for Mean 21st Century Skills Red denotes nonsignificance.
Table A8. ANOVA Tests for Mean 21st Century Skills Red denotes nonsignificance.
Red indicates results that were not significant. Table A9. Independent Samples T Tests of Means for Each Skill, by PBL Use
Red indicates results that were not significant. Table A9. Independent Samples T Tests of Means for Each Skill, by PBL Use
Table A 10. Means on Practices Items for Each Skill, by PBL Use
Table A 10. Means on Practices Items for Each Skill, by PBL Use
Table A 10. Means on Practices Items for Each Skill, by PBL Use
Table A 10. Means on Practices Items for Each Skill, by PBL Use
Table A 11. Independent Sample T Tests for 21st Century Skills by PBL Use Within Subjects
Table A 11. Independent Sample T Tests for 21st Century Skills by PBL Use Within Subjects
Table A 12. Mean 21st Century Skills Z-Scores Across Subject Areas Table A 13. ANOVA Test of Differences in 21st Century Skills by Subject Area
Table A 12. Mean 21st Century Skills Z-Scores Across Subject Areas Table A 13. ANOVA Test of Differences in 21st Century Skills by Subject Area
Table A 14. Chi-Square Test For Distribution by PBL Category
Table A 14. Chi-Square Test For Distribution by PBL Category
a. R Squared = .219 (Adjusted R Squared = .151) Table A 15. Factorial ANOVA Results for “District” and Dependent Variable: Math Gain
a. R Squared = .219 (Adjusted R Squared = .151) Table A 15. Factorial ANOVA Results for “District” and Dependent Variable: Math Gain
Table A 16. Factorial ANOVA Results for “District” and Dependent Variable: RLA Gain
Table A 16. Factorial ANOVA Results for “District” and Dependent Variable: RLA Gain
Table A 17. Factorial ANOVA Results for “District” and Dependent Variable: Science Gain
Table A 17. Factorial ANOVA Results for “District” and Dependent Variable: Science Gain
Table A 18. Factorial ANOVA Results for “District” and Dependent Variable: Social Studies Gain
Table A 18. Factorial ANOVA Results for “District” and Dependent Variable: Social Studies Gain
Table A 19. Factorial ANOVA Results for “Race” and Dependent Variable: Reading/Language Arts Gain
Table A 19. Factorial ANOVA Results for “Race” and Dependent Variable: Reading/Language Arts Gain
Related topics:
21st Century LiteraciesProject-Based LearningProject Based LearningProject Based Learning (PBL)21st Century Skills
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved