Recuento de MySQL en la misma tabla, incluidos valores de conteo cero

Prueba esto:

SELECT 
  t1.id, 
  IFNULL(COUNT(t2.name), 0) AS refNr 
FROM test AS t1
LEFT JOIN test AS t2 ON t1.id = t2.id_user_ref
GROUP BY t1.id;

DEmostración de SQL Fiddle

Esto le dará:

| ID | REFNR |
--------------
|  1 |     2 |
|  2 |     1 |
|  3 |     1 |
|  4 |     0 |
|  5 |     0 |