Android(or other display) UI-Based testing by using Cloud Based Image Analysis

Is Google Vision API Image Content Analysis suited for UI testing. When automating a test for android( or any other touchscreen OS), it is sometime necessary to compare screen image(from screenshot) against a reference image. Other time text has to be extracted from image using OCR.
Has any one tried using cloud based solution(Google Vision API) for UI testing. When I tried it , it wasn't not able to identify UI element like text box, list, sliders.