Performance specs

I’m bringing in the discussion that got started in https://openfoodnetwork.slack.com/archives/C2GQ45KNU/p1542278658178000?thread_ts=1542242380.175200&cid=C2GQ45KNU to have a proper discussion together as a team and reach an agreement.

Context

Since I joined OFN back in June 2017, I’ve never used and seen anyone publicly using spec/performance tests. According to git, they seem to be used by Rob last time, back in February 2018.

commit b7876ebfbf44321d714b2550a67a2757b80e8c57
Author: Rob Harrington <oeoeaio@gmail.com>
Date:   Fri Feb 2 16:45:06 2018 +1100

    Replace references to 'standing order' with 'subscription' (spec)

commit 8bf460c93a10399e011d19f269d674550a229c51
Author: Rob Harrington <oeoeaio@gmail.com>
Date:   Fri Nov 17 14:36:23 2017 +1100

    Manually fix remaining rubocop offences

commit fb2894095296312f2ee6900f7d0528a3a9f16591
Author: Rob Harrington <oeoeaio@gmail.com>
Date:   Fri Nov 17 13:44:35 2017 +1100

    Use Time.zone.now instead of Time.now

commit 03f1980b1ba12ac53a57491bbda02f7ea242b532
Author: Rob Harrington <oeoeaio@gmail.com>
Date:   Fri Nov 17 13:38:38 2017 +1100

    Auto-correct rubocop offences for standing-orders

commit 6ea343f26e1d9b210243fcaf3e3c7b1e31524281
Author: Rob Harrington <oeoeaio@gmail.com>
Date:   Wed Nov 15 13:09:21 2017 +1100

    Clean up proxy order performance specs

I stumbled into them last when I took a look at updating Knapsack’s report in Semaphore CI. I felt like the command we execute bundle exec rake 'knapsack:rspec[--tag ~performance --format progress -p]' was unnecessarily complex.

Proposal

Since we neither use nor maintain them and turn the testing command a bit complex, I suggest we remove them. Anyway, I think kind of agree that this is not the way to monitor the app’s performance.

Thoughts?

I never use these specs. They can be helpful while working on a performance problem. They serve as a benchmark. But since these specs are very dependent on the machine they are executed on, they are not real specs.

I just found this guide: https://guides.rubyonrails.org/v3.2.13/performance_testing.html It’s probably a better approach.

Our monitoring approaches are probably the best performance tests.

that’s what I think. There are better approaches at benchmarking and performance tunning. Then, do we agree on removing them?

I agree and I have to write a bit more so that Discourse let’s me post the reply. :slight_smile: