I think the answer may involve the possibility that at the beginning of the poem, there simply wasn't a clear distinction between the poet and the speaker.
So he started off his lecture by saying: there is no way of avoiding this, we can't avoid talking about what happened yesterday and the whole class apparently - I wasn't there, he was there - was just utterly silent from the start.
"I've been a person that has lived on the street, more specifically at Union Station and people would just kind of wish that I wasn't there, " Terry says.