Track time people take to vote

[Muennighoff]

I think we should track the time after submission until people vote.
This should be a pretty good quality indicator, as some people will likely spam / try to break it by submitting a lot. We can try to filter all of these votes out by filtering for time from submission until they cast their vote. cc @orionw as this would involve small changes in the results file on the hub, wdyt?

[Muennighoff]

Actually I think we can already compute this via the tstamp field? I.e. subtract the tstamp of the 0_conv_id when it's logged to battle with a vote vs when it's logged to individual with the data?

[KennethEnevoldsen]

Yeah, at least we should be able to detect "spammy" voters. We could also detect unreliable voters in other ways (e.g., their preferences consistently do not align with other voters).

[orionw]

Actually I think we can already compute this via the tstamp field? I.e. subtract the tstamp of the 0_conv_id when it's logged to battle with a vote vs when it's logged to individual with the data?

Good point, if we match with the individual we should be able to compute this. We could also extend the data to add the field explicitly but either way works.