The smallest set of ages for the grandson is $\{1$, $2$, $3$, $4$, $5$, $6\}$.

Suppose the man was $A$ years old when the grandson was born. Then:

- $A+1$ is a multiple of $1$
- $A+2$ is a multiple of $2$
- $A+3$ is a multiple of $3$
- $A+4$ is a multiple of $4$
- $A+5$ is a multiple of $5$
- $A+6$ is a multiple of $6$.

Therefore $A$ is a multiple of $1$, $2$, $3$, $4$, $5$, and $6$.

Now the lowest common multiple of $1$, $2$, $3$, $4$, $5$, and $6$ is $60$.

$\therefore\enspace A$ must be a multiple of $60$.

$\therefore\enspace A = 60$

$\therefore\enspace$on the sixth of these birthdays, the man is $66$ and his grandson is $6$.

**Note:**

If we apply similar reasoning to the next smallest set of ages for the grandson $\{2$, $3$, $4$, $5$, $6$, $7\}$, we find that $A$ must be a multiple of $210$, which is physically impossible.