Seed data [development] [provisioning] [deployment]

ohhh @luisramos0 this sounds exciting - @sstead @MyriamBoure @Rachel what do you think?

@luisramos0 I’ve used behat in the past, is fitnesse in the same spirit? But yes in a nutshell I fully agree with you and I’m free to go forward with this. As this is part of the task that was assign to me and @maikel at our last meeting, I would love to have his opinion as well :slight_smile:

Whaooo ! That seems cool @luisramos0 ! Ok I never used those type of tools of course but if I understand well the idea is:

  • that as testers we will have some sample data set like the content of this file but that we will be able to update them as we go as we will be trained to ourselves commit PRs to change it. So we can iteratively enrich step by step this data set with what we need and that will be saved somehow for later testings. I like this idea :slight_smile:
  • then the second point is about writing acceptance tests with the framework tool you propose. Is it like “writing the test cases” we write today in the PR or stories for new features to check if we consider they pass or fail? or in the testing template when we write “test cases”? I’m not super clear about what it would be concretely… would this replace the first dot point about changing sample data through PRs in GH? I guess so. Anyway I feel like it is the good way to go, I guess it would “professionalize” a bit our testing processes :slight_smile:
    It would be good to “test” FitNesse to see what it looks like and get a more concrete sense of it.

First we need to see if other devs know this space and have opinions.

I read just a little bit more, looks like that turnip is really the cucumber that plays with rspec (we use rspec already). In these tools the data tables have to be programmed, for example, devs would have to create a “product table” that could be used in tests (testers could create their specific product list), or a “enterprise table”.
Fitnesse is another thing because testers can create database data directly and edit it as a wiki. It’s awesome but a longer setup/learning curve.

Interesting article about this subject and about turnip with rspec in specific, I think the beginning is useful even if you are not a dev (we already use capybara as well):

I agree with above :slight_smile:

I have no particular experience with these tools. I know cucumber exists and how it looks like but I’ve never used it. If we want to give this a try I’d go with Cucumber as is the most known tool and has a big community (I guess) rather than Turnip. But before that I’d like some things to be considered.

But before that we need to improve the basic data set

We’ve worked with @MyriamBoure on that doc you linked but we never actually implemented it because there are always 100 more things with greater priority. If we think this is now affecting our velocity, let’s find the resources and give it the priority to implement it.

We could train the testers to change this data as they think appropriate with a simple git workflow

That could be a next step but do we have the resources? changing this data means being able to code in Rails. Are testers ready to do this or will it add extra overhead to our already small core dev team? How much time will we spend training them? how much maintenance overhead will this add in the long term even if they manage to code them themselves? I absolutely encourage everyone learning to code but it feels far from our capacity training them to me, at least now.

What I am proposing is that we make this #2072 a mandatory part of the spree upgrade.

I’m fine with that but not more than that. I’m afraid of making the 2-0-stable branch bigger and the upgrade itself even longer.

After that, we could consider Cucumber IMO, which to me is different from implementing new sample data.

I feel like the discussion went a bit off-topic. Behat, Cucumber and Turnip are basically the same thing. But they are about writing automated tests. This topic was about test data. And I would like to throw in three options:

  1. We define test data and code it. This is a lengthy process and needs developers for implementing, but then it’s very easy to share between instances. It’s very good for basic data, but we can’t store configurations containing secrets, like a Stripe setup in there.
  2. We use fitnesse which allows more people (non-devs) to create the test data. I haven’t worked with it, but I guess it’s very easy to share, but we would have the same restriction of not coding secrets in a public database.
  3. We allow testers do take database snapshots on staging servers which are the base for new staging deployments. They can create the setup as they like and then save it. This wouldn’t be easy to replicate between staging servers automatically, but testers can do it themselves. They just add the data they are missing naturally while testing. It creates unique databases which can be a downside (not reproducible on other servers), but also be an upside (complex data reveals more bugs and resemble production data more closely). It also enables us to store unique secrets in the databases.

My favourite would be to combine all of these. We already have a very small set of programmed test data. Let’s extend it. Let the testers create and save more data on the staging servers while we work on adding more data to the programmed data pool. And let’s consider fitness as an alternative way to generate the programmed data to speed up the process.

All of these things don’t depend on each other and can happen in parallel. What do you think?

Fine for me @maikel. We can plan the work on 1. while someone first spends time thinking about how to best use 3. and then we spend some time figuring out how 2. would fit for us.

@MyriamBoure @Rachel I almost implemented all data defined in There is just one thing I would like to discuss.

The mailinator email addresses would leave all accounts open for abuse. I suggest to use invalid addresses like We can still read all emails by using a dedicated email address that is configured in the mail method as “Send Copy of All Mails To”. It could be

What do you think?

@maikel this is awesome :slight_smile: Looking forward to see this going further. I like your suggestion, let’s go with that!

Awesome @maikel!!!
is there an issue on GH for this? I’d love to have a look at a wip PR :slight_smile:

@luisramos0 I opened my pull request with most of the seeding data. It would be good to get some feedback before doing the next iteration:

1 Like

@MyriamBoure @maikel @luisramos0 @sauloperez One small proposal for improvement after using this a bit:

would you guys mind if we put first names that would be radically different?

When I test mobile or when I use a dropdown for example, I only see the beginning of the enterprise name and it already happened that I changed something on backoffice for Mary and in frontoffice I checked Maryse’s shop on mobile trying to understand what was wrong :exploding_head:

So for dummies like me I suggest this:

  • Freddy’s Farm shop => George ?
  • Fredo’s Farm hub => Bob ?
  • Maryse’ Private Shop => Phoebe ?

Or we don’t put first names in the enterprise names.

Good idea. I got confused with the names as well. I like your suggestions.

yeah, confusing names.
what if we follow personas names?
Shannon and Jane?

@luisramos0 Jane is our final buyer. But yes we can use Shannon, I don’t know why I thought she was already somewhere. So we just need to find names for Fredo and Freddy who are derived from Fred our farmer persona (who is used here for the producer profile).

1 Like

Actually I understand about the confusing names… but I put those names to try to stick to the persona, yes, as @Rachel guessed, Fred and derived for farmers, Mary and derives for distributors… but if you want to change, no objection as long as we document carefully so any tester can use. Or we could just in the name put “George (prod)”, “Sarah (retail)”, “Sylvia (wholesale)” so it’s obvious what role they play.

I documented that in the original seed data reflection document @Rachel if you want to understand why those names :wink:

Yes I do clearly understand why :slight_smile: It’s just using it is a bit different… But I like your suggestion “George (prod)”, “Sarah (retail)”, “Sylvia (wholesale)” . This way it’s super simple for old and new testers :slight_smile: Let’s see what others think

1 Like

Fine for me as long as it solves the problem at hand