LayoutLM prediction with confidence score?

[nk-alex]

Taking as starting point this notebook more specifically, this function:

from PIL import ImageDraw

draw = ImageDraw.Draw(image)

font = ImageFont.load_default()

def iob_to_label(label):
    label = label[2:]
    if not label:
      return 'other'
    return label

label2color = {'question':'blue', 'answer':'green', 'header':'orange', 'other':'violet'}

for prediction, box in zip(true_predictions, true_boxes):
    predicted_label = iob_to_label(prediction).lower()
    draw.rectangle(box, outline=label2color[predicted_label])
    draw.text((box[0]+10, box[1]-10), text=predicted_label, fill=label2color[predicted_label], font=font)

image

I would also like to print the prediction score in % format. I know about probabilities = torch.nn.functional.softmax(logits, dim=-1) but I don't quite get how to apply it to obtain the prediction score in % for each element in true_predictions.

I tried this way so far:

#Until here, just following the notebook
logits = outputs.logits

predictions = logits.argmax(-1).squeeze().tolist()
token_boxes = encoding.bbox.squeeze().tolist()
probabilities = torch.nn.functional.softmax(logits, dim=-1).squeeze().tolist()

if (len(token_boxes) == 512):
    predictions = [predictions]
    token_boxes = [token_boxes]
    probabilities = [probabilities]

predictions = list(itertools.chain(*predictions))
token_boxes = list(itertools.chain(*token_boxes))
probabilities = list(itertools.chain(*probabilities))
      
is_subword = np.array(offset_mapping.squeeze().tolist())[:,0] != 0
true_predictions = [self.id2label[pred] for idx, pred in enumerate(predictions) if not is_subword[idx]]
true_boxes = [box for idx, box in enumerate(token_boxes) if not is_subword[idx]]
true_probabilities = [probability for idx, probability in enumerate(probabilities) if not is_subword[idx]]
  
for prediction, box, probability in zip(true_predictions, true_boxes, true_probabilities):
    print(probability )

But this is what I get:

Output: [0.00010619303793646395, 3.339954128023237e-05, 2.2820451704319566e-05, 2.2919863113202155e-05, 0.0005767009570263326, 5.0725124310702085e-05, 3.0033241273486055e-05, 0.006056534126400948, 4.6057226427365094e-05, 1.2512471585068852e-05, 0.0002005402639042586, 2.0308254534029402e-05, 0.992790937423706, 3.023005228897091e-05]

How could I get the percentage format from there?

[nk-alex]

I think I get the result. Correct me if I'm wrong. From that output, I can extract that there are 14 type of labels and the most likely one is number 13 with 0.992790937423706 (99.2%).

But that is not quite what I was aming for. I'm not looking for the probability of each label for that prediction. I'm looking for the prediction score itself. Something like this prediction has a confidence of 75%