¿Cómo elimino registros en MySQL y mantengo la última fecha?

Verificaría UpdateDate contra una subconsulta correlacionada.

CREATE TEMPORARY TABLE
  latestRecord (
    Email        VARCHAR(128),
    updateDate   DATETIME
) 
INSERT INTO 
  latestRecord
SELECT
  Email,
  MAX(updateDate) AS updateDate
FROM
  table_1
GROUP BY
  Emal

DELETE 
  table_1
FROM
  table_1
INNER JOIN
  latestRecord
    ON  latestRecord.Email      = table_1.Email
    AND latestRecord.updateDate < table_1.updateDate

EDITAR

Otro refactor de la misma lógica