Shortest answer is: User objects have security identifiers (SIDs), so users can log in and have assigned permissions. Contacts do not have SIDs.
User Object:
It contains information about users (first, middle and last names, login credentials etc). User object is used for real users in company, so permissions can be applied to them. Using user object can log in on some network, access to some resources etc.
Contact Object:
It contains contact info about any person associated with the organization like a supplier’s telephone number, mail address etc. For example an organization might want to store details of people who are not directly associated with the organization, but they are important for work. These people does not have any access to organisation, but other people needs their contact data as email address is.