Trail riding presents a lot of uncontrolled risk, and it's not relaxing for me to feel responsible for everyone and their horses' lives.
But... that's not entirely the truth. The other reason I don't want to go on trail rides is more along these lines:
Riding through the woods looking at trees is BORING!!!
I would so much rather putter around in an arena than meander in the wilderness. Trail riding has no challenge, and therefore I think it's awful. There, I said it. Anyone agree with me?