With all the speculation about book 7, I want to put my predictions on the record, so I can go 'IN YOUR FACE!' if they're true, or hide in a closet if they're not. So!
In the Pensieve scene, Snape is being tortured and mocked by Sirius and co. Lily comes to his aid by shouting, 'LEAVE HIM ALONE!' Snape, who seems ashamed or embarrassed by Lily, calls her a mudblood, and Lily is shocked, but recovers quickly and calls him Snivellus. I don't think this is Snape's worst memory because of what Sirius did, but because of what Lily did. Lily, even though she was helping him, was also using him as something of a toy in her budding relationship with James. It was obvious to Snape that there was something going on between them, and indeed something to yet happen between them.
Also, just on a small note, over the years Snape has tormented Harry, constantly bringing up his father in a bad light. Snape has never, to my memory, bashed Lily. Why hasn't he? She was a mudblood, making Harry a half-blood, and she was everything Snape was not. Pretty, popular, nice. Even though James was a git to Snape and deserved some of the bashing he received after his death, it seems strange that never has Snape brought up Lily. (For more convincing evidence, go here.)
So those are some of my big theories. I have some questions as well, that I hope Rowling will answer in the 7th book. Feel free to give your opinions on them!