compare doesn't work with scientist version higher than 1.6.0

[viamin]

I have written an experiment to compare the output of 2 rails serializers (lightly edited from the production code, but the logic is identical):

def index
    LabTech.science 'PagesController#index' do |experiment|
      experiment.use do
        @pages = Page.all
        @json = ActiveModelSerializers::SerializableResource.new(@pages, each_serializer: PageSerializer, root: 'pages', include: '**').as_json
      end
      experiment.try do
        @json = { pages: QueryService.all_pages }.as_json
      end

      experiment.context params: page_params.to_h
      experiment.compare do |control, candidate|
        control_8601 = control[:pages].map { |hash| hash.deep_transform_values! { |value| value.is_a?(ActiveSupport::TimeWithZone) ? value.iso8601(3).to_s : value } }
        # binding.pry
        control_8601.to_json == candidate['pages'].to_json
      end
      experiment.clean { |value| value.with_indifferent_access[:pages] }
    end
    render json: @json
  end

When I uncomment the binding.pry and check control_8601.to_json == candidate['pages'].to_json, I get true. But when I get the summarized_results, they are always marked as mismatched (this is in my dev environment):

> LabTech.summarize_results 'PagesController#index'

--------------------------------------------------------------------------------
Experiment: PagesController#index
--------------------------------------------------------------------------------
Earliest results: 2023-07-21T22:03:56Z
Latest result:    2023-08-09T05:19:14Z (18 days)

24 of 24 (100.00%) mismatched
--------------------------------------------------------------------------------

When I check LabTech.compare_mismatches 'PagesController#index', there is no difference between the control and candidate.

I'm running the comparison with the deep_transform_values because in my control, the created_at value is coming back as an ActiveSupport::TimeWithZone and in the candidate it is returned as a string - that's the only difference between the two that I can find, however they seem to be identical after the transform. I has noticed this before doing the transform, compare_mismatches would still come back without any difference, so my theory is that once serialized to Postgres and deserialized later, the dates would be normalized.

My Gemfile.lock has the latest versions of the gems:

lab_tech (0.1.9)
scientist (1.6.4)

What am I doing wrong here? Or is there a bug? Could this be related to #41?

[viamin]

Rolling back to scientist 1.6.0 fixes this so this does appear to be related to the change mentioned in #41.

[geeksam]

Sorry, I left Real Geeks and no longer have commit privileges to this repo. Pinging @larslevie @kevin1024 @LupineDev @Sparkmasterflex