Special values, map from probabilities to discriminative scores (and vice-versa)

[moreymat]

The current code supports two types of scores: discriminative scores and probabilities.
However, several pieces of code work natively with one type of scores, and define a transformation to support the other.
For example, the Eisner and MST decoders natively work with discriminative scores ; if fed with probabilities, they use scipy.specials.logit to map [0 ; 1] to ]-inf ; +inf[ (capped to [-1e90 ; 1e90] for MST).

This seems infelicitous as it leads to ill-defined floating-point computations like -inf + inf = NaN, which in certain contexts raise numpy warnings (RuntimeWarning: invalid value encountered in float_scalars).
As of 2015-09-24, all numpy warnings of the "invalid" category are ignored, but this is a poor solution.

On a related matter, the Eisner decoder uses NaN as a special score for unreachable states, which can raise another kind of warnings from numpy (RuntimeWarning: All-NaN slice encountered).
As of 2015-09-24, these are not ignored and might appear in run logs.

I hope we can avoid running into these issues again by having a reasonable and sound policy for conversion routines from probabilities to discriminative scores and vice-versa, that could be common to all modules of attelo.

A solution could be to use the log function instead of the logit, to map from [0 ; 1] to ]-inf ; 0].
As far as I can tell, it should be fine for both the Eisner and MST decoders.
Having only one special value, -inf, should avoid raising most warnings.
However, log(0) raises RuntimeWarning: divide by zero encountered in log while properly returning -inf.

[moreymat]

@padenis @phimit ideas welcome !

[phimit]

let's talk about this tomorrow

i'm surprised MST has potentially negative scores. doesn't that break the
algo ?

2015-09-24 17:35 GMT+02:00 Mathieu Morey notifications@github.com:

@padenis https://github.com/padenis @phimit https://github.com/phimit
ideas welcome !

—
Reply to this email directly or view it on GitHub
#50 (comment).

I needed a clue and a drink; one of them I
knew where to find.

[jrmyp]

@phimit : The Chu-Liu-Edmonds algorithm used for MST does allow negative weights.

@moreymat : The 1e90 cap comes from the depparse implementation of this algorithm, which has a hardcoded minimal weight value of -1e100. Lower values crash the process.

The choice of logit instead of log had the following goal: bump up the scores tied to a high computed probability. I admit I didn't do a performance comparison when we made the change.