The Left has never really been that keen on following the Constitution as written because they feel it is a "living document" (meaning that they believe the Constitution means what they say it means, regardless of whether it's grounded in the Constitution). Now, the Left has taken the Constitution to a whole new level with the "community center" that is proposed to be built a few blocks from Ground Zero.
The First Amendment gives Muslims the right to build the mosque/community center under the Establishment Clause, and if you disagree, you're anti-Muslim, stupid, or any number of other derogatory labels. But the thing of it is...the First Amendment doesn't state or imply a right to build anything, let alone a mosque/community center.
The Left's main argument is that the construction of a mosque/community center is an extension of the right to worship. There are two main problems with that argument. First, a person of faith doesn't require a building to worship openly. In my faith, all that's required to establish a church is two people sharing their common faith. Besides, given that there's a mosque within walking distance of the proposed site of the mosque/community center, there are options that are readily available for their religious needs. If the mosque/community center isn't built, nobody is denied their right to worship. It's not the building that makes the right to worship; it's the exhibition of faith.
Second, the Left is being contradictory with its position on the mosque/community center. When it's come to expressions of Christian or Jewish faith, the Left has firmly come down against public expressions of faith, especially on public land. Their logic in those instances is that people of faith should pray only in churches and synagogues since someone might be offended at a public expression of faith. In short, you can pray all you want, just not where people might take offense.
And building a mosque/community center where Muslim prayer will be allowed within a couple of blocks of an act of extremist Muslim terrorism isn't offensive?
Put simply, the Left has no real Constitutional grounds under the Establishment Clause on which to defend the building of the mosque/community center. There may be other Constitutionally-based arguments that could be made, but suggesting that the building of a mosque/community center is an extension of freedom of religion is intellectually dishonest and a gross misinterpretation of the First Amendment.