Confidence should only be there when either fix or affected package is present

Without a relationship between fixed/affected package to a vulnerability, confidence - by itself - does not provide any value in Inference.
We should have a check against this.

https://github.com/nexB/vulnerablecode/blob/2903edcef68de3137e57f804b022a22b2710c5c8/vulnerabilities/improver.py#L29-L72

Found while writing tests in: https://github.com/nexB/vulnerablecode/pull/1081