Engineering.Reference:26.4.2.1 - Obtain a measure that can support the proposed decision
For an exact entropy calculation at a model-generated position t, use the complete next-token distribution conditioned on the history there, or another sufficient representation: p_t(v) = softmax(logits_t / temperature)[v]. Fix the tokenizer, vocabulary, temperature and history used for the comparison. Shannon entropy in nats is H_t = -sum_v p_t(v)*log(p_t(v)), with zero-probability terms taken as zero. For example, several possible phrasings can raise it while all phrasings express the same known answer. The external return belongs in the next history; counting the characters or tokens of that return does not obtain the policy’s next-token entropy.
For a ranking or threshold decision, first determine which orders or outcomes are consistent with the probability information available for each history. If the decision remains open, assess whether obtaining logits or a sufficiently precise estimate could settle it at an acceptable cost. Record the histories and generated positions to which the data apply. A next-token distribution available at a saved prefix can be inspected before taking the next action. An estimator that needs K sampled positions must generate them, retain the resulting history and include that work in the allocation budget. The estimate becomes available after generating those positions. Reset an initial baseline for each fresh trajectory and retain its relation to any branched continuation. Comparing a new question with a cached baseline from an earlier question changes the measure.
A selected-token log-probability can suffice for a recorded action’s importance ratio and still be insufficient for entropy. Suppose a four-outcome interface reports only that the selected outcome had probability 0.5. Its surprisal is one bit. If q is the normalized distribution over the other three outcomes, the complete entropy is 1 + 0.5*H_bits(q), with lower bound 1 bit and upper bound 1 + 0.5*log₂(3) bits (about 1.79248125). Use the exact expression when deciding a close threshold. Both (0.5, 0.49, 0.005, 0.005) and (0.5, 1/6, 1/6, 1/6) fit the log, but their entropies differ. With only that same probability at two histories, either history can have higher entropy. For a ranking or threshold, first check what the available bounds decide. For a selected probability of 0.99, the entropy is at most h₂(0.99) + 0.01 log₂(3) < 0.097 bits, where h₂ is binary entropy. Every history with selected probability 0.5 has at least one bit, so this pair can be ranked from the recorded facts. If the needed comparison remains open, obtain a qualified estimate when worthwhile or use an allocation the evidence already supports.