I began this as a reply to Timothy Morton's extremely helpful comment on entropy in letters and words (following Shannon, whom I've used elsewhere in discussing the editing of Shakespeare). In fact all the comments were wonderful, so let me say thanks. Thanks!
And hope thus to prompt Sravana Reddy to comment here [ETA: she did]. She notes (I think she's thinking about ghazals) that some forms of Urdu and Hindi poetry have entropy zero in their last words. I don't think that's quite true of English adaptations (as in Tennyson), but it's easier for me to talk about sestinas.
The bravura sestinas that poets like Seamus Heaney write tend to try to get maximum variation into the seven-times repeated words that end each line of its six stanzas and structure the envoi. Sestinas provide maximum homoeoteleuton, so the idea is to vary the use of the words as much as possible: through changes in grammar, through rimes riches (or homonymical or homophonical usage in general), through accretive rhyme (there's a technical name for this, but I can't remember what it is: a series of rhymes like "ope, lope, slope"). Heaney's "Two Lorries" provides a good example: lorries in one line comes out flurries.
& Merrill has that amazing canzone (section & in Scripts for the Pageant), Samos, which has only five last syllables in its sixty-five lines (five 12-line stanzas plus an envoi). So the pressure to vary is extreme, and he meets it. Here's the first stanza:
And still, at sea all night, we had a sense
Of sunrise, golden oil poured upon water,
Soothing its heave, letting the sleeper sense
What inborn, amniotic homing sense
Was ferrying him--now through the dream-fire
In which (it has been felt) each human sense
Burns, now through ship's radar's cool sixth sense,
Or mere unerring starlight--to an island.
Here we were. The twins of Sea and Land,
Up and about for hours--hues, cries, scents--
Had placed at eye level a single light
Croissant: the harbor glazed with warm pink light.
In the eighth line land has lengthened to island; in the tenth sense turns into its homophone scents, later fire will yield sapphire, and light both chrysolite and the first syllable of the hyphenated leit-/motifs. Note too how sense varies, first noun, then verb, and how the stanza-ending pairing treats light first as adjective, then as noun. Each stanza ends with a couplet of each of the five words in turn: light, land, fire, water, sense. That last word too is surprising (entropic) in another way, since we're expecting some version of the four elements (the poem refers to them explicitly), plus light as some quintessence. (Perhaps what happens is that the air of the past is felt as its surround, now lost in the present.) I think also that Merrill may be hearing a little bit of Tennyson's Lotos-Eaters here, with its opening soporific rhyme land / land.
Merrill's bravura performances bring out what's true in any great sestina, from Sidney to Bishop: the way different meanings are established at the end of every line, different meanings for the same words. "The child draws another inscrutable house." So again, there's a groove there, the end word we know is coming, but if the maxim of a tendency towards constant information density is useful or half-true, the more expected the word as lexical item, the more give in the word as to range of meaning.
Again, stress, length, metrical matrix, and grammar will all play their part here. I was thinking just the other day of the difference between closed couplets, as in Pope and Dryden pre-eminently, and open couplets as in Shelley's Julian and Maddalo and Browning's My Last Duchess. In the latter form, you can miss the fact that the poem rhymes -- which also means that the end words probably have a bit less figurative range than they do in rhymed poetry.
Just a speculation. In general, I would like to speculate of metrical poetry, that end words have least entropy and correspondingly most figurative range per syllable. Piantadosi et al., and so far Sravana in her comment elsewhere, are thinking about word length, and distributing information over the length of the word (measured I think in phonetic duration). I would like (I would love) to think instead about how the information in single syllables is apportioned, or to be more accurate how the information is apportioned after you control for stress, so that a stressed syllable is no denser than an unstressed syllable, but more prominent and cognitively 'longer.' (It might be helpful to go back to the language of quantity, but all that really matters is to control for stress, and to assume that a stressed syllable carries more information because it takes up more of a line.)
So this is what I am going to keep my eye out for, for a while I think. Obviously (?) the corpus of English poetry isn't going to help that much, because the point is not what we naturally do in the spoken communication that we are all expert in, but what the highly artificial medium of poetry does. There the relevant evaluative perspective is reception, not production: poems may feel great (or memorable) partly if they do conform to this rule. Well, as I say I'll keep my eyes open. Let me give an example though: Yeats's great line in "The Circus Animal's Desertion":
Vain gaiety, vain battle, vain repose.
I think the line's amazing, partly because of the play of stresses. The first two vains are unstressed, the third stressed. It's partly the difference in 'natural stress' between the stressed syllables that makes this work: the last syllayble of gaiety is stressed, but only because of the syllable that precedes it and the comma that follows it; the third vain, which we've been conditioned by the line to expect as unstressed receives its stress in contrast to the highly unstressed syllables that frame it, so the line reads: "vain GAIetY, vain BATTle, VAIN rePOSE." I think that the third vain carries more information than the first two, again because it's cognitively longer: it's stressed. But because it's stressed, hence cognitively longer, it carries more information than the first two vains: this isn't gaiety or battle to no purpose, this is something like pure illusion.
Okay, so here's what I think I'm saying:
1) Stress in poetry (we know) is a function of the surround of unstressed syllables (or caesura or line boundaries: this is a simplification of what Halle and Keyser say).
2) Stress has an effect in the poetic line that parallels that of length in spoken language: a stressed syllable is 'longer' and therefore more information-bearing than an unstressed syllable.
3) Yet it's still a single syllable (vain = vain = vain), so the information it carries has to include other elements than unexpectedness, especially in stressed monosyllables (vain > pose in the last two words of the Yeats line; the whole ordering of those syllables would be vain > pose >> re).
4) Monosyllabic words then have to get the information that give them their stress (or we impute information to them) from some other dimension than the unexpectedness of the word (it's a short word, so it's not very unexpected), viz., grammatical form, figuration or some sort of vectoring.
By vectoring I mean some reference elsewhere, as in syllepsis, but also as in almost all rhyming (a rhyme word always refers to another rhymed word) or enjambment, which is where grammar comes in. The difference between Pope and Browning would be a difference between the figurative and the grammatical information that the stressed rhyme words are offering.
Obviously the rhyme scheme matters too. More vectoring in complicated rhyme schemes, probably a little less grammar and a little more figuration than in couplets or in blank verse.
And this might be a way of describing meta-expectations too, as in Dante's insistence on rhyming Christ only with Christ in the Divine Comedy, the Word itself being absolute plenitude of information. (Kind of like "word" in rap.)
Anyhow I think this is what I'll try looking out for, for the next little while.