Necesitas agrupar por extractos.
SELECT
SUM(amount)
FROM
transaction
GROUP BY
EXTRACT(MONTH FROM when),
EXTRACT(YEAR FROM when)
Y si necesita esas columnas, entonces
SELECT
EXTRACT(MONTH FROM when) as month,
EXTRACT(YEAR FROM when) as year,
SUM(amount)
FROM
transaction
GROUP BY
month,
year
Por supuesto, puede agregar ORDER BY
y use nombres cortos también:
SELECT
EXTRACT(MONTH FROM when) as month,
EXTRACT(YEAR FROM when) as year,
SUM(amount)
FROM
transaction
GROUP BY
month,
year
ORDER BY
year DESC,
month DESC