Criticism of Math in Economics

I've been reading and speaking to a number of educated economists and economics PhDs who are against the use of intense mathematics and mathematical proof in economic theory. Specifically I've been speaking to those of Marxist and heterodox persuasion and reading their work in an attempt to become more open minded. They emphasize that the study of work by classical economists (like Adam Smith, Karl Marx, and David Ricardo) is still relevant and that the practice of how mainstream economics uses mathematics is abusive and is an attempt to fool the masses regarding the "science" economists practice. I have difficulty understanding this argument. What is a reason to be against mathematics in economics? Note: I'm pretty mainstream and like how economics is taught and structured. I'm not anti math-in-economics, I just want to know why this is an argument.