Esto asume id
es una columna de incremento automático:
SELECT MAX(id) AS id
FROM pms
WHERE id_to = 1 OR id_from = 1
GROUP BY (IF(id_to = 1, id_from, id_to))
Asumiendo que tienes id_from
y id_to
indexado, esta variación probablemente funcionará mejor porque MySQL no sabe qué hacer con un OR:
SELECT MAX(id) AS id FROM
(SELECT id, id_from AS id_with
FROM pms
WHERE id_to = 1
UNION ALL
SELECT id, id_to AS id_with
FROM pms
WHERE id_from = 1) t
GROUP BY id_with
Así es como obtener los mensajes para esos ID:
SELECT * FROM pms WHERE id IN
(SELECT MAX(id) AS id FROM
(SELECT id, id_from AS id_with
FROM pms
WHERE id_to = 1
UNION ALL
SELECT id, id_to AS id_with
FROM pms
WHERE id_from = 1) t
GROUP BY id_with)