Thursday 18 September 2014

Is New Zealand's Policy Sector 'Evidence-Based'? Part 1

A Story of Deceit and Manipulation

In the early 2000s I made a career decision that appeared at the time to be a monumental failure; I decided to leave the relative safety of Te Puni Kokiri (New Zealand's Ministry of Maori Development (see footnote)  to take a position at the Department of Corrections.

Within about 2 years I realised that the move had been highly beneficial to my development as a critic of crime control policy.  I lasted all of 6 months before deciding to return to Te Puni Kokiri; not a long time to work in a policy position, but enough to provide me with some invaluable insights into the workings of the criminal justice policy sector. What follows are some of the insights I learnt during my short 'sentence' at Corrections, many of which hold true for all crime control agencies in New Zealand:

The worst way to run a government agency
... was the 'model' used at Corrections at the time, which was to set the General Managers of the various sections (Policy, Community Corrections, Psych Services, Public Prisons, etc) against each other so they competed for the agency's limited resources, and for the attention of the CEO.  The rationale behind this management style seemed to be the belief that the ensuing competition would result in enhanced quality of output.... which showed the genius who implemented this style knew little about human nature. The situation this strategy engendered at Corrections was frankly bizarre as time and again I observed supposedly adult, experienced public servants stabbing each other in the back, engaging in petty, immature turf wars and killing off well constructed projects in order to protect their own interests.  All of this experience was fantastic training..... for being a kindergarten teacher, but did little to engender collegiality or the development of effective penal policy (whatever that is).  To be fair to Corrections though, the management process in place at the time was simply a mirror image of the way in which the crime control sector engaged with each other, as I soon learnt during my involvement in inter-agency programmes such as Effective Interventions that began in the mid-2000s.

Iconoclast's will not be tolerated
Speaking 'against the grain' or taking the position of the devil's advocate was not acceptable.  When I arrived at Corrections the agency was in the process of implementing the Integrated Offender Management (IOM) initiative.  Imported more or less whole from Canada, IOM was intended to streamline the delivery of prison-based services to inmates to ensure their 'sentence plans' matched their 'criminogenic needs', such as anger management and alcohol and drug dependency.  In order to 'sell' the programme within and outside the agency, a whole tier of marketing practitioners were identified, called (and I am not making this up) 'IOM Champions'.  The key task of the 'Champions' was to market IOM, and one way they sought to do so was to hold information sessions with staff.  These sessions encouraged 'full and frank' discussion on the nature and impact of IOM, or so we were told. Before attending staff were assured they could ask critical questions or make critical statements, and that anything they said would be treated confidentially. 

In my first week on the job I attended one such session, during which my manager asked what I thought was an important question: 'how do we know IOM will work for Maori, when no engagement has been held with Maori - not with Maori members of staff, Maori service providers, and especially not Maori inmates/offenders'. The answer?  He didn't get one. The response?  The IOM 'Champion' reported him to the General Manger who proceeded to tear strips off him for daring to criticise IOM and thereby setting a bad example for his staff. Needless to say I decided after that not to attend any more sessions or to tell other staff what I actually thought of the policy.  

Science and evidence-based policy are not king
The rise of IOM coincided with a revolution within many of the crime control agencies wherein 'science' and 'evidence' became the basis of policy-making, the development of interventions, and allocation of resources.  At least that is what the policy sector told itself and the public.  Quite often this was not the case, with pertinent evidence being totally ignored, or the evidence that suits a predetermined policy outcome favoured over the messy stuff, you know.... the evidence contradicts a Cabinet Minister's pet project (see Tauri, 2009 for a discussion of this process).

A recent, classic example of policy implementation that ignored available evidence was the government's decision to introduce boot camps.  No firm evidence existed to indicate that this intervention would result in positive outcomes for youth, but it was implemented regardless. Why? Well, there are a number of reasons but in this particular case the answers are 'populist politics' and 'ideology': to understand how such a poorly performing crime control intervention could be introduced, you have to ignore the rhetoric that New Zealand's policy sector is apolitical (as in neutral) and that policy decisions are always based on scientifically-derived evidence.  This is often not the case in the crime control sector. The introduction of boot camps was purely ideological... of the 'get tough on crime and bring back military-style discipline for those young thugs' type you will often hear in RSA bars; the 'a good thrashing never did me any harm' approach to social policy. 

I wish to be clear about one thing - sometimes evidence has a significant impact on policy development and implementation.  My argument here is sometimes it does not.  The policy process can be, and often is, highly political and ideological, with interventions and policies influenced as much by who a Minister was drinking with last week, as it is on independent, empirical evidence.           

Ritual and myth are king
... and when empirical evidence is NOT highly influential in policy making, I observed that other strategies came to the fore, including the liberal use of ritual to maintain the sacred myths of the public service, namely the myth of political neutrality and the myth of evidence-based policy.  In Part 2 I will focus on the rituals utilised by the policy sector to sustain these myths, including the rituals of (in)activity, rituals of deception and cherry-picking rituals; watch this space (in the meantime see Tauri, 2014).

Postscript - When is an Academic Journal NOT an Academic Journal?

The answer? When it is set up and run by a government agency.  

Mildly related to the previous discussion was an announcement made in 2013 of the publication of a new academic journal called Practice: The New Zealand Corrections Journal.  I thought 'excellent, finally the New Zealand academy is getting off its butt and doing something to enhance critical crime control scholarship in this country'..... sadly I was wrong on all counts.
  
No doubt we've all heard the saying 'if it looks like a duck, walks like a duck and quacks like a duck, then it must be a duck'? Well the Department of Corrections 'journal' defies that lovely bit of logic. It 'looks' like a journal and when you hold it in your hot hands if even 'feels' like a journal, but sadly it quacks like an ACT Party candidate talking about being 'tough on crime' during an Epsom constituency meeting.  

Let me be clear on one thing, government agencies have every right to publicise their work; in fact they have an obligation to do so. We as taxpayers have a right to know how they are spending our money... just don't expect them to tell you if it is not effective as that pertinent point is usually ignored completely.  They have a right to publish the material in whatever form their over-priced 'marketing' experts tell them will most effectively spread the gospel.

However, they do not have the right to treat us as if we are stupid, and in marketing their rag as a 'journal' that is exactly what Corrections is doing.  However, I do have to take my hat off to the department, the 'journal' represents a wonderful piece of conjuring worthy of a Las Vegas magic show: take the marketing, publicity and policy materials produced by the agency and its 'academic' friends (meaning those academics who produce work in line with the ideological and theoretical position of the agency) and publish it in journal form.  So far, this is exactly what we are being served up by Corrections.  The vast majority of the articles are written by Corrections functionaries, and few of the articles engage with critical questions being posited by independent academics or report on research using methodologies that allow the voices of those most impacted by corrections policies to be heard.  And most importantly, the editorship, and therefore the article selection process, is overseen by a high level functionary within Correction; hardly a recipe for ensuring that independent, critical scholarship will make the pages of the 'journal'.

It is easy to figure out why the agency has gone down this route: through the 'journal' it is seeking to give its marketing material more of an 'academic' look and feel in order, hopefully, to enhance its credibility on the intellectual market.  But one word of advice to the editor: you will be judged primarily on the content of the articles you publish and hardly at all on whether the 'look' of your magazine matches closely with academic journals such as Punishment or Criminology.

I think the idea of an academic journal like Practice is a good one.  In fact, given the paucity of academic scholarship on crime control in New Zealand I would like to see its name and focus changed to Criminal Justice or something similar.  But having done that you will need to make it more independent.  So why not turn it into a joint venture by running it through an editorial board made up of both academics and policy workers, with two Chief Editors, one from policy and one independent to ensure it doesn't become a mouth piece for your agency.  Or you could align it with an academic institution that has a criminology focus, such as AUT, University of Auckland, Victoria University or the University of Canterbury. It would also be a good idea to mix the content; a few articles from the policy sector, a few from independent researchers.  Then, and maybe then, you can call your magazine a 'journal'. However, in order to achieve all this you need to change your attitude to independent research... you need to allow it to occur.  

Oh, I know you will be able to cite a few examples since 2001 where you allowed PhD students or other researchers 'in' to prisons to do research... research that is likely contracted by the agency or heavily vetted to ensure it serves the needs of the agency, and is unlikely to result in critical findings that might embarrass Corrections, or worse, its Minister. I am talking about the strategy that Corrections appears to be following the past few years of blocking critical research that does not suit the agency's needs.  And it is blocking independent researchers from going about their business, by using excuses like 'the information that will be gathered doesn't match with our trending data' or with 'our strategic priorities', or some similar nonsense.  And if that fails Corrections can fall back on well-worn excuses such as potential 'safety' issues for both inmates and researchers, or muster issues or whatever else they can think of.  And yet other jurisdictions, most notably the United Kingdom, has in the past had few issues with allowing researchers to enter prisons to carry out their work.  The evidence for this is the significant amount (comparatively speaking) of independent research materials published in academic journals on prisons and corrections policies in that and other comparable jurisdictions.  The problem in the New Zealand context seems to created from the intersection,a dangerous combination, of three factors: 1) a policy elite who appear to believe themselves above critique, 2) a policy elite who believe they are not answerable to the public, and 3) who are supported by a political elite who share the same arrogance and aversion to independent scrutiny.  

Let me be even more frank, policy workers and government agencies do not always have the answers and, more importantly because they are so close to their own work they often can't see the wood for the trees.  In other words it is sometimes very difficult for them to step back and critically analysis the impact or their work or identify the questions that need to be asked and answered by research. Sometimes the questions and topics 'the community', which includes independent researchers, inmates, ex-inmates, inmates and ex-inmates families and service providers, believes are important will not match those of the policy sector; and sometimes the communities questions are the right ones to be asking. Remember, a government agency is part of the public service and derives its resources from the public purse.  Therefore, it is time for policy practitioners to stop acting as though they are not answerable to the public.

Oh, and it would be a good idea if they stopped treating us as though we are stupid: Corrections, change the name of your publication to The Corrections Quarterly or perhaps even more suitable, CQ, or something similar that reflects that it is in fact a magazine, because we are not being fooled in the slightest.    

Footnote
I have often thought that the Ministry of Maori Development (emphasis on the 'development' bit) was a weird name for an agency with a budget at that time of around 23 million, and that was often severely hamstrung by Cabinet and core agencies such as Treasury, SSC, Ministry of Social Development, etc, those with comparatively massive budgets and significantly more political capital, from actually doing anything concrete to facilitate 'Maori development'.

References
Tauri, J (2009) The Maori Social Science Academy and Evidence-Based Policy, MAI Review, 1: 1-11. 
Tauri, J (2014) Ritual and the Social Dynamics of Policy Making in New Zealand, in P.Howland (ed.) Ritual in Aotearoa New Zealand: An Effusive Introduction (forthcoming - draft available on Academia).  






2 comments:

  1. Nice article and observations of the public service, unfortunately this seems to be de rigeur or soup de jour these days in government departments and ministries - communications driving and spinning ....

    ReplyDelete
  2. Interesting post Juan, and it is of course true that criminal justice policy is more often than not I would argue based on fear politics and 'tough on crime rhetoric'. The 3 strikes law passed in NZ in 2010 is a perfect example, where irresponsible crime legislation is passed by a NZ Parliment.

    ReplyDelete