I just read this article on CNN SI by Don Banks.  According to this, most of the locker room didn't trust Vince, either:

Adams Chooses Fisher over Young

According to Banks's source, other than five or six of VY's close friends, the rest of the locker room did not have faith in him.  The source goes as far as saying that there would be a long line of players at Reinfeldt's office requesting a trade if VY were in charge of the offense next year.  The players were tired of the immaturity as well as his inability to seamlessly call plays in the huddle.  This pretty much sums it up:

"The fact that he still has a hard time calling plays in the huddle, that's a problem for us," a Titans source said. "There's not indifference here about Vince. There's some serious animosity. A lot of guys would want out of here if he came back. Everyone knows he's the guy who's supposed to drive the car, he's supposed to be our leader. But he doesn't do the things you have to have your quarterback do. Guys can't trust him. Even after all this time. It had to start with not having No. 10 back here."

Read more: http://sportsillustrated.cnn.com/2011/writers/don_banks/01/05/titans.young/index.html#ixzz1AIIz9i82

No indifference.  Serious animosity.  A lot of guys would want out of here if [Vince] came back.  I think that's a lot more negativity than most people were aware existed among his teammates.

That makes  it pretty clear that this isn't a matter of Fisher running VY out of town.  People keep saying that Fisher had lost the locker room, but if you believe this "source," then VY had lost it as well.  I'm not sure who will QBing the Titans next year, but after reading this, I'm more convinced that it should be someone besides #10.