Withdrawing from a small RRSP

Withdrawing from a small RRSP I have an RRSP with less than 20k in it from a previous job, while my current job has a solid pension plan and pay raises every year for the next 12. I invest mostly in VOO and since the RRSP is just sitting there, id like to pull the money out and move whatever i get into my current investment pool with the strongpossibilty of needing to use some for upcomming expenses. I understand that I'll be taxed on the money as though it's income, though it shouldn't place me in another bracket for these funds. What i want to know is if there are any other downsides i should be aware of? At end of year, ill hit another pay bump which will surely place me on a new braket if i were to withdraw then so if i were to do it, id like to do it this year.