Suponiendo integridad referencial y Postgres 9.4:
SELECT *, total - wins - ties AS losses
FROM (
SELECT count(*) AS total
, count(*) FILTER (WHERE m.winner = um.team) AS wins
, count(*) FILTER (WHERE m.winner = 3) AS ties
FROM users_matches um
JOIN matches m ON m.id = um.match_id
WHERE um.user_id = 123; -- for one given user
) sub;
Acerca del agregado FILTER
cláusula (introducida con Postgres 9.4):