A system and method to enable fine-grained, contextual annotations of streaming videos by one or more users, prioritizing the use of screen space on mobile devices by allowing users to draw or place threaded comments while utilizing a touch-based interface, reducing distractions caused by a cluttered interface. By enabling the user to control annotations beginning at a particular timestamp within the streaming video, the present invention optimizes screen real estate on mobile devices efficiently. Contextual commenting is enabled using a combination of perspectives, which highlight the parts of the video being annotated, while dimming out the rest of the screen elements and flexible extension of a user' s comments across one or many frames of the streaming video. Using a simple touch-based interface the present invention is intuitive and further enables the user to select the vicinity around which he or she wishes to increase sensitivity or have finer control.