In the revolutionary era of advanced technologies, computers have benefited humankind in a favourable way by enabling automatic speech-conciliated communication. Even though various assistive tools are available, still visually impaired people are dependent for accessing computer using the traditional input devices, compared to sighted people. To transcend such limitations, the paper discusses an assistive desktop navigation system ESPY that provides the visually impaired individual with the ability to access desktop by mouse movements. By ESPY, user has plenary access of icons on their desktop. ESPY provides assistance to visually impaired people with a navigation proficiency, which allows them to use traditional input devices in a convenient way same as sighted users.