Upcoming Talk: Jared Murray (April 28)

Please join us Tuesday, April 28 at 10:00 a.m. in 232M Baker Hall for this talk given by Jared Murray of CMU Statistics.

Title: Flexible Regression Models for Partially Identified Causal Effects with Binary Instrumental Variables

Abstract: I outline a model-based approach to causal inference using instrumental variables, focusing on the case of a binary instrument, treatment and response. After reviewing model-based inference in instrumental variable designs I will focus on relaxing two classes of assumptions: parametric assumptions about the form of the regression functions, and structural assumptions that are invoked to point identify causal effects. Weaker structural assumptions are often more tenable, but no longer point identify causal effects of interest. Strictly speaking, this is not a problem when performing Bayesian inference for causal effects. However, it does mean that inferences are sensitive to modeling assumptions and prior distributions – even asymptotically, and even if the model for observables is correct. As a result, specifying appropriate prior distributions and conducting sensitivity analysis is paramount.

With this in mind I describe a class of parameterizations of prior distributions for partially identified regression models with several desirable properties: They allow for flexible nonparametric priors for point identified regression functions, selectively informative conditional priors for partially identified parameters, and computationally efficient sensitivity analysis. The methods are illustrated on a well-known dataset collected during a randomized encouragement study.

Michael Hudgens: CMART Speaker Series April 27

Please join us for the second of the 2015 CMART Speaker Series

Michael Hudgens will be speaking at 4:00pm this Monday April 27.
125 Scaife Hall

Michael is an Associate Professor of biostatistics at the University of North Carolina Chapel Hill, and the director of the Biostatistics Core at the UNC Center for AIDS Research.
His methodological work is in causal inference, and has written about principal stratification, randomization inference, and interference between units, along with a wide variety of biostatistical applications.

Title: Causal Inference in the Presence of Interference

Abstract: A fundamental assumption usually made in causal inference is that of no interference between individuals (or units), i.e., the potential outcomes of one individual are assumed to be unaffected by the treatment assignment of other individuals. However, in many settings, this assumption obviously does not hold. For example, in infectious diseases, whether one person becomes infected depends on who else in the population is vaccinated. In this talk we will discuss recent approaches to assessing treatment effects in the presence of interference. Inference about different direct and indirect (or spillover) effects will be considered in a population where individuals form groups such that interference is possible between individuals within the same group but not between individuals in different groups. An analysis of an individually-randomized, placebo controlled trial of cholera vaccination in 122,000 individuals in Matlab, Bangladesh will be presented which indicates a significant indirect effect of vaccination.

Upcoming Talk: Ran Liu

Please join us Tuesday, April 14 at 10:00 a.m. in 232M Baker Hall for this talk given by Ran Liu of CMU Psychology and Human-Computer Interaction.

Title: Variations in learning rate within cognitive tutor use

Abstract A growing body of research suggests that accounting for student-specific variability in educational data can enhance modeling accuracy and may have important implications for individualizing instruction. The traditional Additive Factors Model (AFM), a logistic regression-based model commonly used to fit educational data and discover/refine skill models of learning, contains a parameter that individualizes for overall student ability but not for student learning rate. We find that adding a per-student learning rate parameter to AFM overall does not improve predictive accuracy, nor does it relate to pretest-posttest gains. However, two alternative methods of differentiating learning rates at the student level yielded more interesting and externally valid results. In the first method, we created three classes of students based on each student’s residual patterns (across practice opportunities) when fitted with the standard AFM model. Adding a per-class learning rate to the traditional AFM model substantially improved its predictive accuracy, and class membership was systematically related to pretest-postest gains. In a second method, we eliminate skill-specific learning rate parameters from the model and individualize learning rates only at the student level. Preliminary evidence suggests that, although these parameter estimates are significant predictors of post-test outcomes, usage (number of practice opportunities) is a relatively better predictor than are differences in these learning rate estimates.

Upcoming Talk: Dan McCaffrey, March 31

Please join us Tuesday, March 31 at 10:00 a.m. in 232M Baker Hall for this talk given by Dan McCaffrey of ETS.

Title: “The Impact of Measurement Error on the Accuracy of Individual and Aggregate SGP”

Abstract: Student growth percentiles (SGPs) express students’ current observed scores as percentile ranks in the distribution of scores among students with the same prior-year scores. A common concern about SGPs at the student level, and mean or median SGPs (MGPs) at the aggregate level, is potential bias due to test measurement error (ME). Shang, VanIwaarden, and Betebenner (SVB; this issue) develop a simulation-extrapolation (SIMEX) approach to adjust SGPs for test ME. In this paper, we use a tractable example in which different SGP estimators, including SVB’s SIMEX estimator, can be computed analytically to explain why ME is detrimental to both student-level and aggregate-level SGP estimation. A comparison of the alternative SGP estimators to the standard approach demonstrates the common bias-variance tradeoff problem: estimators that decrease the bias relative to the standard SGP estimator increase variance, and vice versa. Even the most accurate estimator for individual student SGP has large errors of roughly 19 percentile points on average for realistic settings. Those estimators that reduce bias may suffice at the aggregate level but no single estimator is optimal for meeting the dual goals of student- and aggregate level inferences.

Upcoming Talk: Adam Sales (March 24th)

Please join us Tuesday, March 24 at 10:00 a.m. in 232M Baker Hall for this talk given by Adam Sales, PhD, from CMU Stats

Title: Exploring Causal Mechanisms in a Randomized Effectiveness Trial of the Cognitive Tutor

Cognitive Tutor Algebra I (CTAI), published by Carnegie Learning, Inc., is an Algebra I curriculum, including both textbook components and an automated, computer appli- cation that is designed to deliver individualized instruction to students. A recent randomized controlled effectiveness trial, found that CTAI increased students’ test scores by about 0.2 standard deviations. However, the study raised a number of questions, in the form of evidence for treatment- effect-heterogeneity. More basically, it is unknown which as- pects of the CTAI program drove the observed effect. The experiment generated student log-data from the computer application. This study attempts to use that data to shed light on CTAI’s causal mechanisms, via principal stratifi- cation. Principal strata are categories of both treatment and control students according their CTAI usage; they al- low researchers to estimate differences in treatment effect between usage subgroups. Importantly, randomization sat- isfies the principal stratification identification assumptions. We present the results of our first analyses here, following prior observational results. We find that students who en- counter more than the median number of sections experience 0.45 (0.2–0.6) standard deviations higher effect than their peers who encounter fewer, and students who need more assistance experience 0.36 (0.25–0.48) standard deviations lower effect than their peers who require less.

Upcoming Talk: Jeremy Koster, March 9 2015


Please join us Monday, March 9 at 10:00 a.m. in 154A Baker Hall for this talk given by Jeremy Koster, PhD, from the University of Cincinnati

Title: “Multilevel Item Response Models of Ethnobiological Knowledge Among Indigenous Nicaraguans”

A common assumption among anthropologists is that individuals continue to accumulate ethnobiological knowledge throughout their lives, resulting in greater expertise among the elder generations. Alternative theoretical perspectives suggest that ethnobiological knowledge about animals should peak earlier in life, paralleling and facilitating the emergence of foraging proficiency among younger adults. In a study conducted among the Mayangna and Miskito of Nicaragua, I assessed knowledge about fish behavior in three ways: (1) via a free listing exercise, (2) a photo recognition task, and (3) a 50-question instrument about fish behavior, as developed from biologists’ reports on fish in the region. I analyze data with multilevel logistic regression models, as estimated via MCMC methods, incorporating cross-classified random effects for the informants and the questions/species. The results indicate that individuals exhibit considerable domain knowledge as relatively young adults. Related models reveal a positive correlation between knowledge and fishing ability, suggesting that knowledge promotes and develops from specialization and the allocation of effort to fishing. Finally, a comparison of responses to the questions about fish behavior suggests that parents and their offspring exhibit similar beliefs, providing novel support for anthropological models that cultural transmission from parents to children is central to the ontogeny of ethnobiological knowledge.

Kosuke Imai: CMART Speaker Series March 2

Please join us for the first of the 2015 CMART Speaker Series

Kosukei Imai will be speaking at 4:00pm this Monday March 2.
125 Scaife Hall

Kosukei is a professor in the department of Politics at Princeton University, and has written on a wide array of topics in causal inference.

The title: Causal Interaction in High Dimension

The abstract: Estimating causal interaction effects is essential for the exploration of heterogeneous treatment effects. In the presence of multiple treatment variables with each having several levels, researchers are often interested in identifying the combinations of treatments that induce large additional causal effects beyond the sum of separate effects attributable to each treatment. We show, however, the standard approach to causal interaction suffers from the lack of invariance to the choice of baseline condition and the difficulty of interpretation beyond two-way interaction. We propose an alternative definition of causal interaction effect, called the marginal treatment interaction effect, whose relative magnitude does not depend on the choice of baseline condition while maintaining an intuitive interpretation even for higher-order interaction. The proposed approach enables researchers to effectively summarize the structure of causal interaction in high-dimension by decomposing the total effect of any treatment combination into the marginal effects and the interaction effects. We also establish the identification condition and develop an estimation strategy for the proposed marginal treatment interaction effects. Our motivating example is conjoint analysis where the existing literature largely assumes the absence of causal interaction. Given a large number of interaction effects, we apply a variable selection method to identify significant causal interaction. Our analysis of a survey experiment on immigration preferences reveals substantive insights the standard conjoint analysis fails to discover. The paper is available at http://imai.princeton.edu/research/int.html

Upcoming Talk: JR Lockwood (ETS) Dec 1 2014

Please join us Monday, December 1 at 10:00 a.m. in 232M Baker Hall for this talk given by J.R. Lockwood, PhD, of The Educational Testing Service

Title: Inferring Constructs of Effective Teaching from Classroom Observations: An Application of Bayesian Exploratory Factor Analysis Without Restrictions

Abstract: he dramatic public policy shifts toward increasing teacher ac-
countability have generated numerous sets of student outcome, teach-
ing process and teacher knowledge-based instruments all seeking to
measure the quality of teaching. These instruments are being used
to assess and pay teachers without a clear understanding of what as-
pects of teaching are being assessed, and how the many dimensions
that comprise any one measure relate to those of other measures. We
use data from multiple instruments collected from approximately 450
middle school mathematics and English language arts teachers and
their students to inform research and practice on teacher performance
measurement by modeling the underlying constructs of high-quality
teaching. We make inferences about these constructs using a novel
approach to Bayesian exploratory factor analysis (EFA) that, unlike
commonly-used approaches for identifying factor loadings in Bayesian
EFA, is invariant to how the data dimensions are ordered. Using this
approach with our data reveals two distinct teaching constructs in
both mathematics and English language arts: 1) Practices used by
teachers to instruct and engage students; and 2) Teacher management
of classrooms.We demonstrate the relationships of these constructs to
other indicators of teaching quality including teacher content knowl-
edge and student performance on standardized tests.

Upcoming Talk Monday, November 17: Ilya Goldin

Please join us Monday, November 17 at 10:00 a.m. in 232M Baker Hall for this talk given by Ilya Goldin, PhD, of Pearson Education

Title: Individual differences in identifying sources of science knowledge
Joint work with Maggie Renken, April Galyardt, and Ellen Litkowski

Abstract. We have developed an instrument to assess students’ proficiencies in identifying sources of science knowledge (SoK) in text passages. We describe the new web-based instrument and our evaluation of the instrument with a sample (n = 338) of children grades 2-8. By creating and validating this tool, we aim to establish a learning progression, inform science teaching, and tailor instruction to individual differences. Our findings suggest that students demonstrate differential ability in identifying SoK and thus imply the need for instruction to accommodate individual student perspectives on SoK. We expect that highlighting student ability in identifying SoK as a distinct skill will enable differentiated, adaptive instruction. We further expect this instrument to make explicit a component of what it means to think like a scientist, and in doing so facilitate conversations among teachers and students about the practice of science.