Upcoming Talk: Adam Sales

Please join us Monday, Sept 22 at 10:00 a.m. in 232M Baker Hall for this talk given by Adam Sales (CMART post-doc, CMU/RAND)

Title: A Useful Model? Using Covariates to Test the Usefulness of the Randomization Assumption.

Social scientists often look for “natural experiments” to estimate causal effects. They claim, based on quirks in known part of the data generating process, that a treatment was assigned haphazardly. It is good practice in these analyses to test covariate balance: significantly different distributions of baseline covariates between treatment and control groups will falsify the random treatment model. But as we know from Stats 101, failing to reject the hypothesis that covariates are balanced does not imply that they are. In this talk, I will propose a modified procedure where instead of testing the hypothesis that covariates are balanced, researchers can test whether they are imbalanced enough to invalidate the study.
I will use as an example data from Demming (2009) that investigated the effects of head start.

UPDATE: To see the slides, click here

Upcoming Talk: David Choi

After a bit of a break,
please join us Monday, August 11 at 10:00 a.m. in 232M Baker Hall for this talk given by David Choi, a Professor of Statistics and Information Systems at Heinz College, CMU.

Title: Estimation of monotone treatment effects in network experiments

Abstract: Randomized experiments in social network settings are a trending research topic. In addition to the logistical difficulties of running a large social experiment, there may also be statistical challenges in analyzing the data. We discuss the statistical challenge of analyzing experimental data in social networks when the network cannot be divided into smaller non-interacting subgroups, so that interference between units must be taken into account. We present work in progress on how to rigorously analyze such data, assuming that the treatment effect is nonnegative but otherwise making no further assumptions on the flow of influence between units.

Upcoming talk/data workshop on Monday, 6/30: Leah Clark

Please join us Monday, June 30 at 10:00 a.m. in 232M Baker Hall for this talk and workshop led by Leah Clark (CMU Economics and Public Policy)

Title: Patterns of Student Enrollment and Teacher Staffing in Allegheny County Schools since 1997.

**This will be a data analysis workshop. Leah will present her research problem, data resources and questions of interest, and we will talk through some of the data analysis issues as a group.**

Abstract: Enrollment in Pittsburgh Public Schools has declined every year since 1997. Meanwhile, some public schools have closed, some charter schools have opened, and the school-age population in Pittsburgh has declined. I am investigating whether school- and district-level data can provide insight into the choices parents make about where to enroll their children, and the choices teachers make about where to work. The data will permit analyses of other similarly-situated U.S. cities.

Upcoming Talk: Adam Sales

Please join us Monday, June 16 at 10:00 a.m. in 232M Baker Hall for this talk given by Adam Sales (CMART post-doc, CMU/RAND)

Title: Using Covariate Balance to Choose a Regression Discontinuity Bandwidth

Abstract: Regression discontinuity designs (RDDs) are scenarios in which subjects’ treatment assignments are a function of a “running variable” R: treatment is assigned when R is greater (or less) than a known threshold c. These data setups are particularly conducive to causal inference, since the treatment assignment process is known. However, in order to estimate effects, researchers need to model the function relating R, treatment assignment, and the outcome. This modeling step becomes more robust as researchers restrict their data to subjects with R close to the cutoff. That is, researchers can choose a bandwidth around the cutoff, and only use data from within that bandwidth to estimate effects. Recently, Cattaneo et al. (2014) and Sales and Hansen (2014) have suggested using covariate balance tests to choose this bandwidth. Bandwidths within which the analysis fails to detect a significant “treatment effect” on pre-treatment covariates are deemed acceptable. This bandwidth selection technique, applied naively, has several statistical issues, including multiple comparisons, and requires researchers to choose an acceptable level of statistical significance for the covariate balance tests. In this talk, I will suggest a method that resolves these difficulties. The new method is derived from change-point estimation in time-series analysis. The data example for the talk will be Lindo, Sanders, and Oreopoulos (2009)’s study of the effect of academic probation in college on students’ subsequent GPAs.

Upcoming Talk: Ally Thomas

Please join us Monday, May 12 at 10:00 a.m. in 232M Baker Hall for this talk given by Ally Thomas, a Ph.D. student at the the City University of New York Graduate Center.

Title: Evaluating a New York City-Wide STEM Initiative using Genetic Matching

Abstract: Recently, we have been developing evidence (after one year of implementation) of the effects of the Math Science Partnership in New York City (MSPinNYC2) on high school students’ achievement—in terms of scores on end-of-course tests in two key STEM disciplines: Integrated Algebra and Living Environment. The MSPinNYC2 program restructures early high school STEM courses to include 6-8 Teaching Assistant Scholars (TAS) who, along with the teachers, facilitate in-classroom group work on a daily basis. Genetic Matching (Diamond & Sekhon, 2013), a multivariate matching method that uses a search algorithm developed to maximize the balance of observed covariates, was used to create matches of MSP students with similar NYC students. Results from the first and second year suggest the MSPinNYC2 was not effective in raising academic achievement for PERC students in the 9th grade mathematics course (Integrated Algebra), but was effective for PERC students in the 9th grade biology (Living Environment) course. Furthermore the study provided evidence that Genetic Matching is valuable and effective in monitoring the efficacy of a large multi-site instructional intervention.

Upcoming Presentation: The Spencer Foundation and Educational Research

Funding Your Research: An Information Session with the Spencer Foundation

Friday, May 9, 2014
10:00 a.m.
232M Baker Hall
Carnegie Mellon University

In this informational presentation, which will include a question and answer session, Spencer Foundation Associate Program Officer Robert Ream will provide an overview of the Foundation’s three main funding streams including field-initiated grants, fellowships, and the Foundation’s proactive initiatives.

The Spencer Foundation seeks to investigate ways in which education, broadly conceived, can be improved around the world. The Foundation supports the high-quality investigation of education through its research programs and the strengthening and renewal of the educational research community through its fellowship and training programs.

Sponsored by:
CMART: Carnegie Mellon and RAND Traineeship
in Methodology and Interdisciplinary Education Research,
with funding from the Institute for Education Sciences

CMART Lecture and Workshop Series 2013-2014: Robert K. Ream, Thursday May 8th

CMART Lecture and Workshop Series 2013-2014

Robert K. Ream, Ph.D.

Associate Professor of Education, University of California Riverside and
Associate Program Officer, Spencer Foundation

Household Wealth and Adolescents’ Socioemotional Functioning in Schools

DATE and TIME: Thursday, May 8, 2014, 12:00-1:15pm

LOCATION: RAND Corporation (4570 Fifth Ave.), Room 6202

In this talk, Robert Ream will address his work estimating the part played by household wealth in explaining the relationship between family resources and adolescents’ socioemotional skills. The analyses draw from a large-scale nationally-representative sample of adolescents who were in eighth grade in the 2006-7 school year. The findings foster insights regarding whether advantages in wealth correspond to advantages in adolescents’ socioemotional competencies. Findings also identify variation in the magnitude of association between wealth and socioemotional skills according to one’s position in the hierarchy of wealth.

ROBERT K. REAM is Associate Professor of Education at the University of California, Riverside. He joined the UC Riverside faculty in 2004 after postdoctoral fellowships at Princeton University and the RAND Corporation. He is currently on leave from UC Riverside, having taken up responsibilities as an Associate Program Officer at the Spencer Foundation in Chicago. The social dynamics of educational inequality have been the focus of his agenda for research designed to advance understanding of the relation between education and social opportunity. His work appears in a variety of scholarly journals including American Educational Research Journal, Sociology of Education, and Teachers College Record. His book, Uprooting Children: Mobility, Social Capital, and Mexican American Achievement, was published in 2005 by LFB Scholarly Publishing, New York, in the book series, “The New Americans: Recent Immigration and American Society.” Before embarking on a career in research, Dr.Ream served as a legislative aide to former California State Senator Gary K. Hart.

The CMART speaker series is sponsored by the Carnegie Mellon and RAND Traineeships (CMART) in Methodology and Interdisciplinary Research, with funding from the Institute of Education Sciences.

Paper Workshop: Guanglei Hong (with Pizza!)

Please join us on Friday, May 2, 232M Baker Hall for an informal talk with CMART visitor Guanglei Hong, PhD, Professor of Comparative Human Development at the University of Chicago.

Guanglei will present work in progress: “Investigating variation in causal mediation mechanisms across experimental sites.”

Pizza will be provided.

Anyone interested in meeting with Dr. Hong on Thursday or Friday May 1 or 2, please contact acsales@cmu.edu

Upcoming Seminar: Guanglei Hong

Please join us Thursday, May 1 at 5:00PM. in 135A Baker Hall (Adamson Wing) for a seminar given by Guanglei Hong, Professor of Comparative Human Development at the University of Chicago.
Please Note: this is not the usual SERG day, time, or place.

Guanglei is visiting as part of this year’s CMART Lecture and Workshop Series. She is an Associate Professor in the Department of Comparative Human Development and a member of the Committee on Education at the University of Chicago. Her research focuses on developing causal inference theories and methods for evaluating educational policies and instructional programs in multi-level, longitudinal settings.

Title: How Interventions Work: New Weighting Methods for Causal Mediation Analysis

Dr. Hong will discuss her recent work in developing methods for causal mediation analysis, or investigating the mechanisms through which a treatment impacts an outcome. Conventional methods for mediation analysis generate biased results when the mediator-outcome relationship depends on the treatment condition. She introduces ratio-of-mediator- probability weighting (RMPW) for decomposing total effects into natural direct and indirect effects in the presence of treatment-by-mediator interactions. Dr. Hong will discuss an application of the technique to investigate whether employment mediates the relationship between an experimental welfare-to-work program and maternal depression.

The paper on which the talk is based is accessible at http://www.andrew.cmu.edu/user/acsales/HongPaper.pdf