Free speech absolutists (I’m not one, by the way), believe in the moral principle that no one should be prevented from expressing an opinion by violence, fear of violence, or economic or legal coercion. The First Amendment only protects against the last (and not always that). Thus, when someone is arguing for free speech, to point out that it is not a case where the Bill of Rights applies does not answer the argument.
If you wish to answer that argument (which, of course, you are under no obligation to do), you must show that what is being accomplished by suppressing that speech is more important than the moral right being denied. I believe there are such cases, but I also believe it is a scary and dangerous road to start down, so we need to be bloody damn sure we’re right when we do it.