For some reasons I got cards assigned to a user that is no longer part of the "Assignee" list. In my case it's because the user got suspended but I suspect that the same issue occurs if the one guy is removed from the group associated to the assigned to field.
Then, I can no longer remove him/her from the assigned on the card.