Eliminar MySQL con Agrupar por

Colóquelo en una subconsulta:

delete from table 
where columnA in (
  select columnA
  from (
      select columnA
      from YourTable
      group by columnA
      having count(*) > 1
      ) t  
)