Yeah idler/pitman for sure. Problems get worse with larger than stock tires, and I've seen the Rock-Ram's make thing worse. Saw the same thing happen on Ron B's truck a few times and I'm sure it happened on mine.
Each trip to John Bull up in Big Bear with the H1, Iv'e always had to replace the pitman and idler arm every time.... steering gets sloppy and loosey goosey after a rock crawl.
That being said, the way to go is to get the Alpha/ECV Sheppard M83 steering gear and the larger pitman/idlers, they are huge compared to the old ones.