"Pot just doesn't seem as bad," said Gregory Carlson, a 52-year-old landscaper from Denver who did not participate in the Pew survey.
"You don't see anything about someone smoking a joint and then driving the wrong way into a school bus," Carlson said. With a chuckle, he added Wednesday, "They just drive slower."
The survey also highlighted a dramatic shift in attitudes on drug conviction penalties.
The survey was about evenly divided in 2001 on whether it was good or bad for states to move away from mandatory minimum sentences for non-violent drug offenders. Today, poll respondents favored moving away from such policies by a nearly 2-to-1 margin, or 63 percent to 32 percent.
Respondents said by margin greater than 3-to-1 that people who use small amounts of pot shouldn't go to jail.
"Even people who don't favor the legalization of marijuana think the possession of small amounts shouldn't result in jail time," said Carroll Doherty, Pew's Director of Political Research.
The nation thought differently a generation ago.
Congress passed the Anti-Drug Abuse Act in 1986 to set mandatory minimum sentences for federal drug crimes that could end up in life sentences for repeat offenders.
Years later, many states reported prisons bursting at the seams, prompting public officials started abandoning "lock 'em up" drug policies in the 1990s. The trend has since accelerated.
Last month, Holder testified in support of proposed sentence reductions in an effort to reserve the "the harshest penalties for the most serious drug offenders."
Such plans, including one drafted by Kentucky Republican Sen. Rand Paul, that would give judges wider discretion in sentencing have picked up support from both Republicans and Democrats.
The poll suggested that despite shifting attitudes on legalization, the public remains concerned about drug abuse, with 32 percent of those surveyed calling it a crisis and 55 percent of respondents viewing it as a serious national problem.