Understanding Aggregate Data, FIT-Outcomes


The data reported in the data system FIT-Outcomes is divided into active and inactive clients, making it possible to compare the effectiveness of the ongoing (active) cases to the cases that have been terminated (inactive).

The first section of data gives a statistical summary of how many clients, treatment episodes and sessions the clinician or agency has. It also calculates average treatment length and drop-out rate.

The second section of data calculates a number of statistical indices related to the effectiveness of the clinician or the agency.

  • Percentage reaching target calculates the percentage of clients in the green zone (above the predictive trajectory of succesful cases).
  • Effect size calculates the effect of treatment compared to no treatment.
  • Relative effect size compares the effect size of the clinician or agency to the grand mean effectsize in the system. The grand mean effectsize a norm generated from 500.000 cases, representing the mean outcome of treatment for the 500.000 cases. If the relative effect size is positive it means the clinician or agency delivers a treatment more effective than the norm. If the relative effect size is negative it means the clinician or agency delivers a treatment less effective than the norm. A relative effect size of 0 means the clinician or agency delivers a treatment that is average compared to the norm.

FIT Implementation – Challenges in 2018.

By Susanne Bargmann

The history of FIT

FIT was first introduced in Denmark in 2002 an a small group of pioneers started working with the ORS and SRS. At the same time similar projects were taking place all over the World. Since 2002 pioneers all over the World have worked hard and deliberately on figuring out how to use FIT in different contexts, with different populations and in different cultures. It is the hard work of these pioneers that ensured that FIT was approved by SAMSHA as an Evidence Based Practice in 2013.

Since then FIT has become increasingly popular as a way of measuring outcomes, ensuring that the client is directly involved in the treatment planning and as a way of working Evidence Based according to the definition of multiple professional organisations. FIT has also led to a growing interest in professional development from a different perspective – the interest in “Deliberate Practice”.

Dangers are around us!


But – when things gain popularity and attention there are risks that we might miss in all the enthusiasm and excitement. And I see signs that FIT in some places have gone over to the Dark Side.

Here are some of the signs of the Dark Side, that you may have encountered in your work with FIT – I’ve summarised them in 4 headlines:

  1. “It can’t be that hard”
  2. Mechanical focus on the measures
  3. It’s all about the data
  4. We want to know who is best!
It can’t be that hard

At first glance it looks simple to implement FIT. How hard can it be with 2 brief measures with just 4 questions on each measure? This can lead organisations to decide that “we are going to implement this within 6 months”. The intention is to get this implemented FAST!

But FIT implementation takes so much more than just getting staff to use the ORS and SRS. FIT requires a cultural change, where the organisation develops a “Culture of Feedback” that allows for everybody in the organisation to respond to feedback from consumers. And cultural change takes time and careful planning of a step-by-step implementation process. This is described in detail in the 6 ICCE Manuals on Feedback Informed Treatment. Manual 6 covers the steps of succesful implementation in detail.

Mechanical focus on the measures

Some organisations approach the ORS and SRS as if they are “FIT”. They focus on the mechanics of “How to use the scales” and all training is organised around this understanding. The main purpose is to get the scales filled out and entered into the system.

But FIT is NOT just filling out the ORS and SRS. In fact the measures themselves mean nothing – they tell us nothing. What is really important is that the clients’ feedback will lead to a conversation about what it means for the client, and how the feedback can inform the therapist and the treatment. The real value of FIT is in engaging the client in the conversation about the service being delivered, making sure they feel helped by the therapist. The value of FIT is making therapists aware when they are not experienced as helpful by the client, and making it possible to change and adjust the treatment before it’s too late.

It’s all about the data

Many organisations chose to implement FIT because there is a demand to provide data to funders or to the political system. They view FIT as an “easy” way to gather the data that is required. And the focus of implementation is all about ensuring that data is collected and that therapists are “compliant” (= filling out the ORS and SRS with EVERYBODY they meet).

But – the purpose of FIT wasn’t data or measurement when it was first developed. The purpose of FIT  was to make sure the clients were feeling helped by the service provided to them. It was about creating a  “culture of feedback” where the therapist and the client were able to talk more directly about the clients experience of the treatment, and it was about creating a possibility to be more helpful, because therapists had access to session-by-session feedback.

We want to know who is best!

As soon as we begin to measure and generate aggregate data, there is a risk that FIT is viewed as a way of deciding “who is the best” or “who is bad”. It can be viewed as a method for figuring out which therapists to hire or fire, and it can be viewed as a way of deciding which treatment serviced to shut down, and which to keep.

This perspective misses the real potential of FIT. When we measure our outcomes we have a much more specific way of detecting areas for professional development. A lot of professional training is very general and not specific to what the individual therapist may need to learn. Using FIT to measure our outcomes makes it possible for each individual therapist to identify specific areas for professional development, make a plan for how to work on this area, and then measure the outcome of the training afterwards. A process often referred to as “Deliberate Practice”. And for me this is where FIT holds the biggest promise and potential: As a way of working with our professional development in a much more focused way, and as a way of measuring if our efforts are paying off!