Trying to change a user from direct line to app, but now I can't assign number to account.

I was trying to assist a user who originally requested a physical phone and was assigned a direct line, but later found that the company is pretty much using the desktop app instead. I followed the directions to remove from direct line and switch, but now I can't assign the number back to the user.