Calling raft::neighbors::refine on input which uses a different indexing type for the input matrices than for the vector ids results in a compilation failure in raft::neighbors::ivf_flat::detail::fill_refinement_index. This comes up in attempting to refine output from some of our supported CAGRA instantiations, which use int64_t for the input matrices' indexes but some other type for the vector ids.

To trace through the problem:

1. refine can be instantiated with different idx_t and matrix_idx here.
2. This calls refine_device with the same type instantiations here.
3. refine_device calls fill_refinement_index here.
4. fill_refinement_index has only a single indexing type in its template. This is deduced to be idx_t from refine_device due to this argument, but matrix_idx due to this argument. Because these two do not match, no instantiation matches the template, and we fail to compile.