Process: Collecting data to drive development

One of my actions from the Barcelona gathering was to drive forward a process for improving the data we have that, in turn, drives our product development. We want to develop based on what our users want and need, and what their pain points are. Thus we need to be talking to them and collecting the data that helps us to make these decisions.

User Driven Product Process

  1. Talking to Users
    Product managers have regular catch ups with users to fully understand their pain points, usage of the platform and generally just ensuring that we are keenly aware of how users are finding the platform and other offerings.

  2. Surveying Users
    After speaking with users, and potentially more widely, we survey users to get quantitative data on the importance of features and feature candidates and satisfaction with the way users are currently operating (both with features internal to our offering and external to it).

  3. Collating the data
    We graphically analyse this quantitative data and use it to inform our priorities within our product planning process.

Pathways to introduce this process

My main focus at this stage is to develop this surveying tool. In Barcelona we did a group activity to understand the value chain for our users - how do our users actually create value in the world. You can see the white board photos from these sessions here.

In summary there are 5 key stages in the value chain of OFN users:

  • Taking customer orders
  • Sourcing produce from suppliers
  • Packing orders
  • Fulfilment
  • Reconciliation

We broke these stages down into individual tasks. You can see a summary of these individual tasks here.

Now we want to know how important each of these tasks are to our users and how satisfied they are with each now. So we will ask them four questions about each of the tasks:

  • How important is this task for you? (1-10 scale)
  • How do you complete this task now? (OFN or other tool?)
  • How satisfied are you with the way you currently complete this task? (1-10 scale)
  • Any additional things we should know about this task? (free text)

Next Steps

Here is my project plan for this work:

Stage 1. I have put all of the tasks in a single spreadsheet with a description. Please look here and comment such that we can all agree on the tasks and the descriptions before translation.

Stage 2. I will use Google Translate to translate the spreadsheet into en, fr, Catalan, other? I will then ask relevant people to check the automated translations. Please don’t start on translations and checking until we have agreed the tasks and descriptions in Stage 1.

Stage 3. I will create a separate survey for each language that will all feed into the same spreadsheet such that we can easily graph results.

Stage 4. We can plan the roll out such that each instance feels confident to start talking to their users and we can start generating the data we need! I will continue doing this in a google form, started here.

@MyriamBoure @Rachel @danielle @Kirsten @sauloperez @sigmundpetersen @tschumilas @nick @sstead Please would you take a look at Stage 1 above and comment on the en version of the document. Please can you look before Friday Sept 7th if possible.

3 Likes

Thanks @lin_d_hop this looks good to me. I suggest asking a couple of our super-users - Oiliver would be a good start - to run through the spreadsheet and spots any gaps

A question @lin_d_hop - isn’t ‘finding customers’ a step in the value chain? I raise this because OFN doesn’t do it very well - and its one of the comments I hear from users – “how does OFN help me find new customers?”. Best we have is the map & producer/shop search. Should we include that in our survey?

Looks great to me! (I fixed few typos) Looking forward to get da DATA :metal: !!

1 Like

@tschumilas Good idea. Added.

Hi again everyone,

Thanks to those of you that had the chance to look at the current task flow. Working on the assumption that we are happy with the steps, I’ve moved on to stage 2 (described above).

Stage 2: I’ve now updated this spreadsheet adding a Google Translated sheet for fr, es and ca. These will form the basis of our surveys. Each survey will feed into the same metric calculation so it is very important that we keep all of the questions matching up in the same order and with the same meaning in each translation.

The next step is to have each language checked by a person with the right language skills. It will require a small amount of OFN understanding too such that we can be confident the question in EN and the translated version really do refer to the same part of the workflow.

Please @MyriamBoure, @Rachel, @sauloperez Would you delegate the task of checking this spreadsheet in fr, es (if wanted) and ca so someone appropriate?

What to do if you find something you are not happy with?
If you think a step is incorrect - please add a comment in the en sheet and we can fix/update. This will mean updating all the language sheets so it would be great if you could spot something here sooner rather than later :slight_smile:

If you’d like another language, let me know, or there is a browser plugin that enables sheets to google translate. Probably won’t work as I’ve stuck to comment permissions for the purposes of tracking changes.

ping @danielle… I’m trusting you to let me know if I go off-piste with this :wink:

Sorry to be late to the party- the tasks look great and help me understand a lot about how to approach potential users - I can’t wait to be up to speed enough to make suggestions!

@lin_d_hop this looks great to me. I would really value @sstead input on the detailed lists - it seems like there are quite a few things that we completely don’t support, but I suppose that’s the point. I would just be a bit concerned about how many people will spend the time to do this - might there be an argument for removing some to make it more accessible? Quality vs quantity of responses I guess

I don’t see where @tschumilas’s observation about Sourcing Customers has been added

Not sure if you want direct changes into the doc - the intro text in each section refers to ‘customer orders’ and needs to be changed. ‘Sourcing’ should probably be ‘Product sourcing’ especially if there is going to be another section for ‘Customer sourcing’

Thanks so much for doing this - is going to be extremely useful to have consistent data!

Hi @Kirsten,
Just to respond to some of your points:

  1. The idea will be to have the 5 steps in separate surveys. Instances can send each survey out to their users one by one - maybe based on a phone conversation finding out where in the value chain the user’s biggest pain point is. Or maybe based on the area your instance wants to work on first. Or maybe in a structure way, new survey every 6 weeks and when we get back to survey 1 some of the issues will have been worked on. This part of the process we need to discuss. Each survey on it’s own is not too long.

  2. Sourcing customers is titled ‘Finding Customers’ in the spreadsheet.

  3. If no one has put any work into the translations then we can still make changes to the en version. I think I have set to comments only for that reason.

I’ve put some comments in the sheet with suggested addition questions.

I’m predicting we won’t get a high response rate/rich feedback from a good cross section of users, just from past experience with surveys. I’m not sure how to improve the quality of the feedback we get. Maybe we’d get good data by calling people and then us typing in their responses?

We also have a long list of feature requests that have already come in over the years. Should we put these in somewhere? Feels a bit odd asking our users for feedback on some things we already know they need/want.

Because we don’t have very many users yet, we are planning on using these surveys and gathering feedback in a face to face meeting of users we are organizing for November-December. Really, I’d say we only have 3 users right now who have dug in enough to even respond to these questions.

That said, I’d like to try to modify these questions and use them in follow up with users who have tried and abandoned the platform. Would that make sense do you think? We have lots of users in this category - people who have given up. Does anyone follow up with frustrated users?

We don’t have many users yet either but I wonder if this is worth being answered by potential users. Guida is now carrying out a series of workshops with potential hubs, producers and few that are already hesitantly using the platform so it might be the right timing. We would mainly get feedback on how OFN fits into their needs. Thoughts?

I think is how we would use these questions too - just because we have lots of potential users, but not users with experience. BUT - I think for potential users, this is a lot of detail. MAYBE - in a workshop, we take potential users through the platform, and ask them to give feedback using these questions as they learn.

It would be bad UX to just give a user all of these questions. Discussions on methods for presentation of questions to come.

What about showing potential users just a subset of them? just throwing in the idea.

To my mind some potential options would be:

Option A

  1. Chat with an actual user on the phone. Find out what their key pain point out of the 5 key stages in the OFN value chain.
  2. Send them that section of the survey

Option B

  1. Instance decides on which one of the 5 key stages of the value chain that they feel is the biggest headache and that they want to focus on
  2. Phone users and ask them about that specific stage
  3. Send the the survey on that specific stage such that the user can answer anonymously.

I would say that product people for each instance can mix it up and do which ever they want. I have designed the survey tool to enable this, in each language, and for all data to be automatically amalgamated.