Sikuli on my visual programming app

I made a Sikuli node on my application for a test of conbining two visual programming software. When the user starts his application by clicking the small red circle, the downstream Sikuli node(s) gets activated, it starts a Sikuli ide ("hello popup" and "open my blog") and another Sikuli command line application to control the Sikuli ide, press play button and terminate the Sikuli ide. Sikuli uses HTML so I could easily show a Sikuli script on my Sikuli node. When opening my blog, I ran it on debug mode. You'll see what node is currently evaluated.





Feb, 15 added:
I just think it'll be conceptually fun if it could handle objects in a screen movie capture. The user plays the movie forward/backward and defines objects with a mouse, and the rest is the same as Sikuli. It'll be fun if it can generate a script from the movie capture automatically after the user defines objects in the movie.

Practically Sikuli might need a robust way to identify objects in the screen. I wanted to do File - close but many applications has the same visual interface. Currently region and inside() can specify only screen coordinates but people will want to make his script work after a window has been moved (I think they may change the behavior to get the region(s) dinamically from the screen image). Other methods like similar(), near(), etc. could be a help, or it can have some method to restrict click() actions if the region belongs to some application. Since Sikuli can already start and end an application with its name it will probably be possible to implement it.

And I think it'll be useful to be able to click a certain point in an image. It'll be good to click an "empty space" (think of clicking F4 cell in an excel sheet). And "do something until image appears" loop (think of dragging a scroll bar in a panel to show a scrolled out button).