|
|
|
Sort Order |
|
|
|
Items / Page
|
|
|
|
|
|
|
| Srl | Item |
| 1 |
ID:
149107
|
|
|
|
|
| Summary/Abstract |
Reproducibility is held to be the gold standard for scientific research. The credibility of published work depends on being able to replicate the results. However, there are few incentives to conduct replication studies in political science. Replications are difficult to conduct, time-consuming, and hard to publish because of a presumed lack of originality. This article sees a solution in a profound change in graduate teaching. Universities should introduce replications as class assignments in methods training or invest in new stand-alone replication workshops to establish a culture of replication and reproducibility. This article will first discuss the benefits of conducting replications. The main part will focus on concrete steps in integrating replication in the classroom, from selecting a paper to final manuscripts. Drawing on the author's own teaching experience as well as that of others, particular emphasis will be on the pitfalls and challenges of letting students replicate work, as well as potential criticism. Only if universities nurture a reproducibility and replication culture can we ensure that the gold standard of reliable, credible, and valid research is not just an empty phrase.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
ID:
162860
|
|
|
|
|
| Summary/Abstract |
Do researchers share their quantitative data and are the quantitative results that are published in political science journals replicable? We attempt to answer these questions by analyzing all articles published in the 2015 issues of three political behaviorist journals (i.e., Electoral Studies, Party Politics, and Journal of Elections, Public Opinion & Parties)—all of which did not have a binding data-sharing and replication policy as of 2015. We found that authors are still reluctant to share their data; only slightly more than half of the authors in these journals do so. For those who share their data, we mainly confirmed the initial results reported in the respective articles in roughly 70% of the times. Only roughly 5% of the articles yielded significantly different results from those reported in the publication. However, we also found that roughly 25% of the articles organized the data and/or code so poorly that replication was impossible.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
ID:
149106
|
|
|
|
|
| Summary/Abstract |
Publishing in top-ranking journals in the social sciences and international relations requires writing with clarity. Accurately described and transparent methods sections ensure high-quality academic writing. The methodology section of empirical papers should explain the exact steps taken by the authors when operationalizing concepts and testing hypotheses to facilitate replication. This also allows for monitoring quality, challenging findings, and promoting good scientific practices. The quality of methodology sections is the result of the interaction between academic cultures of data sharing, effective application of rules, and good-quality research data management (RDM). This article evaluates the impact of standards on replicability. We present an empirical analysis of a set of sixty-six articles published during the period 1984–2013 that use data from all waves of the European Values Survey. We find differences demonstrating the impact of good RDM and data policies on good scientific practice.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
ID:
115667
|
|
|
|
|
| Publication |
2012.
|
| Summary/Abstract |
The profuse dissemination and utilisation of solar PV technology in the world is indispensable, especially in this era of climate change. However, in the African continent, between 1960 and 2007 Kenya and Zimbabwe were among countries with the highest PV dissemination, while Ghana was among countries with the least disseminations. Analysing empirical data through the lens of the Social Construction of Technology (SCOT) theory, the article aims to uncover the drivers underpinning the disparate dissemination trends of PV in the three countries within the stated period and to tease out lessons apropos replicating the successes within Kenya and Zimbabwe in Ghana. SCOT theory is chosen because it provides an excellent framework for analysing the social shaping of PV's development and diffusion processes in these countries. This theory posits that the shape and meanings of a technology do not reside in it, but are acquired through the heterogeneity of social interactions. Findings in the paper reveal that a gamut of socio-economic and political antecedents informed the varied dissemination outcomes of the technology in these countries. Premised on these findings, the paper recommends critical steps, which Ghana needs to undertake to enhance the replication of the Kenyan and Zimbabwean PV success stories.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
ID:
157107
|
|
|
|
|
| Summary/Abstract |
Content analysis has once again come to the forefront of discussions regarding methods in International Relations (IR). The first wave of content analysis in IR lasted from the 1940s to the 1960s and was marked by a commitment to quantitative and manual analyses. The second wave of content analysis appeared at the start of the third millennium and continues to pervade the discipline. As with the first wave, it proceeds in a predominantly quantitative manner but emphasizes computer-assisted analysis rather than manual analysis. Critics and advocates of the method alike have, highlighted numerous shortcomings with these approaches. In order to address these limitations, the present investigation argues for a fully integrated content analysis that has the potential to ameliorate the identified weaknesses that have hitherto plagued the method. It accomplishes this task by combining all facets of the method: quantitative, qualitative, manual, and computer-assisted content analyses within a single research project.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
ID:
149109
|
|
|
|
|
| Summary/Abstract |
The paradigm wars between quantitative and qualitative methodologists have focused on the validity and reliability of theory testing, with increasing concerns for transparency in both types of work. But not all research topics lend themselves to theory testing and, rather, require the generation of new theoretical concepts. The relative lack of attention to the “why” and “how” of qualitative theory generation has stunted innovation, forcing scholars to avoid such work or “reinvent the wheel” rather than build on community accomplishments. This article shows that grounded theory methods from sociology provide useful techniques for theory generation and can help scholars break through theoretical muddles. These methods have the added benefit of utilizing a workflow management that lends itself to more transparency than is common in much qualitative work. This article concludes by suggesting steps to boost transparency for grounded theory in international relations and push out the knowledge frontier.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
ID:
146822
|
|
|
|
|
| Summary/Abstract |
Data access and research transparency (DA-RT) is a growing concern for the discipline. Technological advances have greatly reduced the cost of sharing data, enabling full replication archives consisting of data and code to be shared on individual websites, as well as journal archives and institutional data repositories. But how do we ensure that scholars take advantage of these resources to share their replication archives? Moreover, are the costs of research transparency borne by individuals or by journals? This article assesses the impact of journal replication policies on data availability and finds that articles published in journals with mandatory provision policies are 24 times more likely to have replication materials available than articles those with no requirements.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
ID:
166665
|
|
|
|
|
| Summary/Abstract |
In the conceptual literature on terrorism, there is no shortage of answers to the question: “What is terrorism?” Indeed, the terrorism literature has been heavily criticized for a deluge of definitions. And yet the booming quantitative terrorism literature generally examines a narrow set of “what is terrorism?”: how country-level factors explain variation in the number of terrorist attacks. This article demonstrates the variety of ways in which scholars currently operationalize terrorism and compares them to the ways it could be operationalized. I replicate studies using alternative operationalizations of terrorism to examine the consequences of the terrorism literature’s collective bet to focus on attack counts at the country level. Finally, I discuss the implications of the narrow set of operational choices with an eye towards how a greater variety of approaches would produce a more robust research agenda.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 9 |
ID:
113937
|
|
|
|
|
| Publication |
2012.
|
| Summary/Abstract |
In a widely cited study, Collier, Hoeffler & Söderbom show that economic growth reduces the risk of post-conflict peace collapse - particularly when the UN is present with a peace mission. These findings are encouraging for interventionist international policymakers. We replicate their study using data from the UCDP/PRIO Armed Conflict Database instead of the Correlates of War database. We generate a series of different datasets on the basis of different coding criteria commonly used in the literature, and rerun a simplified version of their model. Our results do not support their findings regarding the risk-reducing effect of economic growth and UN involvement. At best, the results are mixed. Some of the models even suggest that economic growth may increase the risk of post-conflict peace collapse. Overall, we are forced to conclude that the impacts of economic growth and UN involvement on the risk of post-conflict peace collapse are neither clear nor simple. The differences in the results seem to be driven by two sources: the conflicts included in the original datasets and the coding of the start and end dates of the conflicts.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 10 |
ID:
149105
|
|
|
|
|
| Summary/Abstract |
Despite 20 years of progress in promoting replication standards in International Relations (IR), significant problems remain in both the provision of data and the incentives to replicate published research. While replicable research is a public good, there appear to be private incentives for researchers to follow socially suboptimal research strategies. The current situation has led to a growing concern in IR, as well as across the social sciences, that published research findings may not represent accurate appraisals of the evidence on particular research questions. In this article, I discuss the role of private information in the publication process and review the incentives for producing replicable and nonreplicable research. A small, but potentially important, change in a journal’s workflow could both deter the publication of nonreplicable work and lower the costs for researchers to build and expand upon existing published research. The suggestion, termed Preplication, is for journals to run the replication data and code for conditionally accepted articles before publication, just as journals routinely check for compliance with style guides. This change could be implemented alongside other revisions to journal policies around the discipline. In fact, Preplication is already in use at several journals, and I provide an update as to how the process has worked at International Interactions.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
ID:
180673
|
|
|
|
|
| Summary/Abstract |
We examine the research protocols in Blair and Sambanis’ recent article on forecasting civil wars, where they argue that their theory-based model can predict civil war onsets better than several atheoretical alternatives or a model with country-structural factors. We find that there are several important mistakes and that their key finding is entirely conditional on the use of parametrically smoothed ROC curves when calculating accuracy, rather than the standard empirical ROC curves that dominate the literature. Fixing these mistakes results in a reversal of their claim that theory-based models of escalation are better at predicting onsets of civil war than other kinds of models. Their model is outperformed by several of the ad hoc, putatively non-theoretical models they devise and examine. More importantly, we argue that rather than trying to contrast the roles of theory and “atheoretical” machine learning in predictive modeling, it would be more productive to focus on ways in which predictive modeling and machine learning could be used to strengthen extant predictive work. Instead, we argue that predictive modeling and machine learning are effective tools for theory testing.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
ID:
149110
|
|
|
|
|
| Summary/Abstract |
What is the main motivation for replication in international relations? How can we implement it, and who is in charge? With a variety of proposals put forward by the contributors to this special issue, it seems useful to provide a synopsis. This article does so by asking a set of key questions related to replication, and by summarizing the stance each proposal takes with respect to these questions. This comparison reveals that there are fewer disagreements than we would think: there is broad consensus when it comes to the motivation and the need for replication in our field. However, proposals diverge as to how replication should be best implemented and, to a lesser extent, who should be the driving force behind this. This article concludes with a recommendation for a dual strategy. First, simple replication on the journal side should be implemented as a routine quality check to weed out work that does not even satisfy basic replication standards. Second, journals should be encouraged to allow publication of more comprehensive, in-depth replication studies with an independent contribution.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
ID:
149108
|
|
|
|
|
| Summary/Abstract |
The field of replication studies remains a controversial, misunderstood, and unappreciated piñata of 18 replication typologies spanning 79 replication types. To help bring order to the chaos, I contribute a theory of manufactured inferences. The theory is built on three pillars: (1) replication causal diagrams (or r-dags for short), (2) a formal conceptualization of study procedures, and (3) the use of Bayesian inference to update our beliefs about the natural phenomenon under investigation and the operating characteristics of the study procedures used to study it. I use this theory to motivate a formal typology of replication types, explaining how they are done and for what purpose. Finally, I discuss some implications of this theory, including the importance of an analytical approach to robustness and generalizability replications, the need to avoid conceptual replications, the possibility of legitimate (unplanned) specification searches, the limitations of meta-analysis, and the false dichotomy between so-called successful and failed replications.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
ID:
149103
|
|
|
|
|
| Summary/Abstract |
Replication is widely seen as an important way to promote transparency and quality control of published work in the social sciences. In international relations, the replication movement moved forward with a panel at the International Studies Association in 2002, subsequently published in this journal. The publication included a joint statement by four editors of leading quantitative journals in international relations, pledging their journals to a set of minimum replication standards and urging other editors to follow suit. Over the years, these and other journals have posted hundreds of data sets. Nevertheless, progress has been slow, and many journals still have no policy on replication or fail to follow up in practice. In this forum, we discuss how improvements can be made to current policies and practices and how replication can be used systematically in teaching international relations and extended to qualitative work.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
ID:
131536
|
|
|
|
|
| Publication |
2014.
|
| Summary/Abstract |
Recently, the importance of research transparency via replication studies has been greatly discussed in most of the social sciences, political science included. Indeed, as Gherghina and Katsanidou (2013) and Freese (2007) note, to some extent, the discussion has been prompted by the tremendous changes in publishing in the past decade or so. With the enormous expansion in data availability and instant publication made possible by the Internet, there now are many opportunities to verify the findings presented in the discipline's major journals. "Replication, replication" has not only become the mantra for political science, but for economics, psychology, and quantitative sociology as well. These developments opened a debate on how to best "guard the high standards or research practice and allow for the maximum use of current knowledge for the further development of science" (Gherghina and Katsanidou 2013, 1; for similar sentiments see King 1995).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|