A broad-scale quantification of the measure of quality for scholarship is under way. This trend has fundamental implications for the future of academic publishing and employment. In this essay we want to raise questions about these burgeoning practices, particularly how they affect philosophy of education and similar sub-disciplines. First, details are given of how an "impact factor" is calculated. The various meanings that can be attached to it are scrutinised. Second, we examine how impact factors are used to make various "high stakes" academic decisions, such as hiring and promotion, funding of research projects and how much money is to be awarded to a particular area. By focusing on a particular practice, problems with the application of the metric generally are outlined. Finally, we offer some general observations about the unintended consequences and other problems arising from the widespread use of this metric, including attempts to "game the system". We argue that the use of impact factors increasingly shapes the kind of topics and issues scholars write on, their choices of methodology, and their choice of publication venues for their work. Technical measures and mechanisms tend to "colonise" the qualitative and professional judgments that must also be part of the process of evaluation, and for which bibliometrics alone cannot offer a substitute.