A method of managing a personal medicine information management service by a user device and a server includes obtaining a medicine image captured by photographing appearance of a user medicine; transmitting the obtained medicine image to the server as a request for recognition of the user medicine; and receiving medicine quantity information and personal medicine information from the server based on a result of the recognition of the user medicine. The personal medicine information includes an image of the user medicine of which recognition training is performed by the server.