I often realize people make their code hard to maintain. It's a very good idea to raise your value in the company since nobody else can maintain it. Let's learn from their code. Often you can even get more efficient code since you don't need to write extra lines to make your code nicer. It doesn't cover basic skills like using magic numbers, not making a method name readable, give an object more than one name, delete all comments, etc (Do all of them!). Ideas are listed in order of importance.
(1) Store the same value in more than one places.
If you store the same parameter in more than one places, you need to synchronize them. When a maintainer write a code that changes the value, he will have a change to miss the fact and change only one of them. Every code that uses the other variable which wasn't changed will behave mistakenly. Hopefully the program will be in an inconsistent state.
(2) Store lots of data in an object and every method depends on them
If lots of methods are dependent on the state of the object, the user of the class will need to set all of them properly before using the method and he will be confused. A nice side effect is that it makes the program hard to debug and test cause the behavior of a method can have lots of right and wrong cases. Try packing lots of unnecessary parameters, make the state of objects complicated, and make it hard to maintain.
(3) Reference an object from lots of other objects.
People who look at it will be puzzled, "Is the object I can get from A is the same as what I can get through B?". This strategy is good with (1) above. Unnecessary references are good!
Dec. 24:
Just tried to tell what will happen if you do them. Don't do them of course :)
Saturday, December 4, 2010
Tips to obfuscate your code.
Thursday, November 25, 2010
pycurry
A friend of mine wrote a very interesting function that makes arbitrary methods good for currying when used as a decorator. This is an example he wrote,
import pycurry
@pycurry.curryfunc
def test(aa, bb, cc):
print aa, bb, cc
f = test(10) # => currying
test(10, 20, 30) # => execute the func
f(40, 50) # => execute the func since sufficient arguments are given
f(200, cc=400) # => You can execute the func like this as well
g = f('aaa') # => curring again
g([10]) # => execute
One example use case is making similar functions that takes some of arguments in common, and some of arguments given previously, like
import pycurry
@pycurry.curryfunc
def myShotMessageSlot(buttonName, toggled):
print 'Button ' + buttonName + "'s current state is " + str(toggled)
qtToggleButtonAAA.someSignal.connect(myShotMessageSlot('AAA'))
qtToggleButtonBBB.someSignal.connect(myShotMessageSlot('BBB'))
qtToggleButtonCCC.someSignal.connect(myShotMessageSlot('CCC'))
I usually do the same thing using a closure (you can also use partial),
def slotFactory(buttonName):
def myShotMessageSlot(toggled):
print 'Button ' + buttonName + "'s current state is " + str(toggled)
return myShotMessageSlot
qtToggleButtonAAA.someSignal.connect(slotFactory('AAA'))
qtToggleButtonBBB.someSignal.connect(slotFactory('BBB'))
qtToggleButtonCCC.someSignal.connect(slotFactory('CCC'))
but using his pycurry makes it nicer.
https://github.com/shomah4a/pycurry
By the way I just noticed syntaxhighlighter doesn't work anymore. The JavaScript files were on a free hosting site that has ended its service, I need to put it somewhere else, ahhhh...
Friday, September 24, 2010
Qt Designer bad knowhows (*)
"bad knowhow" is a Japangrish meaning it is something you have to use knowing it is not a nice way.
Qt designer is a good tool but there are several things you need to know until you get familiar with it.
1) Making widgets expand within a window
It is the first thing most people takes a few hours googling to get the answer. The answer is tricky.
- Drag and drop arbitrary widget (button or spacer is enough)
- Right click the background of the window (not the widget you have dropped)
- Select Layout - Layout Horizontally (or Layout Vertically) in the pop-up menu
You cannot select Layout- Layout Horizontally (Vertically) until you create a widget on the window.
2) Moving a layout from its parent to another.
When the UI gets complicated, it is often difficult to move the right layout to the right place.
To move the right layout:
- Select the layout in the Object Inspector.
- Click the layout
Qt Designer will pick the selected layout.
To the right place:
- When clicking the layout, click near the top left corner of the layout
- Drag and drop it to the new parent in the Object Inspector
Unless you click new the top left corner, you'll be puzzled when dragging it to the Object Inspector
3) Try to copy and paste a widget and get error "Cannot paste widgets. Designer could not find a container without a layout to paste into."
My solution is to create a new dummy window (with ctrl-N) and copy&paste to it. then drag and drop it to wherever you wish to. There may be other ways.
I will add a screen shot on request.
When using the qt-designer generated file (*.ui), never modify automatically generated file. For example, I use Python and I convert *.ui to *.py with pyuic4 tool. I never modify the .py file by hand, which would halve the qt designer's strength. You will quite often want to update the GUI design and once you have modified the generated file, your modification to the source code would be lost. Insted, use the file from another. Same applies to C++ users to.
By the way qt designer is in qt-devel yum package (e.g. qt-devel-4.6.3-8.fc13.i686). It tool a while for me to find it.
Saturday, September 4, 2010
Scons
This is my log of start using SCons, you don't need to read this if you are familiar with SCons already. Most of my knowledge comes from SCons User Guide, and many of examples shown here were taken from the guide.
I looked for a build tool (like make) for my project. Like everybody else in the world, I'm sick of traditional make and needed a more elegant one.OMake's concurrent build (-P switch) attracted me a lot but it seemed to need some time to use Omake with emacs. Scons is a build tool (like make) fully utilizes Python, and there is no need to code in OCaml (omake) (*).
Once you have installed scons (if you use macport to install, you also need py25-hashlib to be installed), the first step of using Scons is to write a SConstruct file (Makefile equivalent), which is actually a Python script (so you can write a Makefile equivalent in Python).
If you just have two files hello.cpp which uses hello.h, a simple SConstruct file is
Program('hello.cpp')
to build, you just type
scons
Scons automatically looks at the source code hello.cpp and it is dependent on hello.h, so whenever hello.h changes, scons knows it needs to rebuild hello.
SCons User Guide is very well written and easy to read document but it is targeted for non-Python programmers and a little bit verbose, so I'll just write down the most important parts (for me) of the guide here. It doesn't cover throughout hte guide, I only read the first several chapters, up to 7.2.1 and I felt I can already use Scons to some extent.
-Program(['foo.cpp', 'bar.cpp']) if there are two sources.
-You can write Program('program', Glob('*.c')).
-SharedLibrary('foo', ['f1.c', 'f2.c']) and StaticLibrary('foo', ['f1.c', 'f2.c']) for shared and static library.
-Program('prog.c', LIBS=['foo', 'bar'], LIBPATH='.')
-Program('hello.c', CPPPATH = ['include', '/home/project/inc']) for -Iinclude -I/home/project/inc
-Program('prog.c', LIBS = 'm', LIBPATH = ['/usr/lib', '/usr/local/lib']) for -L/usr/lib -L/usr/local/lib -lm
-You can write SConstruct script like,
hello_list = Object('hello.c', CCFLAGS='-DHELLO')
goodbye_list = Object('goodbye.c', CCFLAGS='-DGOODBYE')
Program(hello_list + goodbye_list)
Object means "Object file" (i.e. *.o file). Not Python object, nor any other object.-Explict dependency.
hello = Program('hello.c')
Depends(hello, 'other_file')
The following scripts are also accepted.Program('hello.c')
Depends('hello', 'ho.cpp')
-Construction Environment
env = Environment(CC = 'gcc', CCFLAGS = '-O2')
env.Program('foo.c')
To make it default settings (so that you can write Program('foo.c') instead of env.Program('foo.c')),DefaultEnvironment(CC = 'gcc')
- SConscript(['subdirectory/SConscript']) for hierarchical build,
- Program(['foo.cpp', 'bar.cpp']) makes build target foo (looks scons gets the name from the first item in the list, not where the main function exists). Default(Program(['foo.cpp', 'bar.cpp'])) to set it as the default target.
(*) OMake official site says "There is no need to code in Perl (cons), or Python (scons)." ;)
Wednesday, August 11, 2010
Any good module for saving settings file?
I'm thinking about the file format for various settings.
Using XML is like going to a super market by jet plane. YAML is good but not in the standard module.
So far the best choice is using JSON but json module in the standard module is not suitable for my needs since it has no __getstate__/__setstate__ equivalent.
I wonder if there's a nice module that has both pickle and json features. It would
- be able to serialize arbitrary object
- have __getstate__, __setstate__ equivalent
- be able to convert to/from standard Python objects (NOT Tag/Node/Element object ! )
- create json formatted file
- preferably customize the behavior on error
- be a light weight module (not dependent on thousands of other modules)
There's no need to maintain object references, dump() can raise an exception just like json.
I can make one by myself but I don't want to reinvent the wheel. Anybody knows if it exists already?
Sunday, July 4, 2010
Makin your Python code fast
Check out a presentatiton by Andrew Bennetts at Pycon Australia 2010.
http://pyconau.blip.tv/
He introduced various ways of Python code profiling.
Material from PyCon AU are licensed under the Creative Commons CC-BY-NC-SA license.
And you may also be interested in "Python Goes to the Movies" by Mark J Streatfield at Dr. D Studios, which is kind of introductory stuffs though because the conference is for Python and not for vfx.
Friday, May 21, 2010
Recipe 577237: Prevent star imports (Python)
Recipe 577237
Use this code in your module to prevent people using the "from foo import *" syntax with your module.
@apply
class __all__(object):
def __getitem__(self, _):
raise ImportError("Star imports not supported")
Saturday, April 24, 2010
Path class
I hope one of these stuffs is in the standard library.
>>> p = Path('/path/to/some/text.txt')
>>> p.parent()
Path('/path/to/some')
>>> p.parent().parent()
Path('/path/to')
>>> p.parent().parent().parent()
Path('/path')
>>> p.parent().parent().parent().parent()
Path('/')
>>> p.parent().parent().parent().parent().parent()
Path('/')
>>> p.parent().child('other').child('text.txt')
Path('/path/to/some/other/text.txt')
>>> p[:-1]
Path('/path/to/some')
>>> p[1:-1]
Path('path/to/some')
>>> p[2:-1]
Path('to/some')
You cannot write e.g. p[2:3] = p('aaa') since conceptually the Path class is immutable.
import os
class Path(object):
def __init__(self, path):
if isinstance(path, Path):
path = path.path
self.path = path
def parent(self):
return Path(os.path.dirname(self.path))
def child(self, name):
if isinstance(name, Path):
name = name.path
childPath = os.path.join(self.path, name)
return Path(childPath)
def base(self):
return Path(os.path.basename(self.path))
def abspath(self):
return Path(os.path.abspath(self.path))
def __str__(self):
return self.path
def __repr__(self):
return self.__class__.__name__ + '(' + repr(self.path) + ')'
def __add__(self, rhs):
if isinstance(rhs, Path):
return self.child(rhs.path)
else:
return Path(self.path + rhs)
def __eq__(self, rhs):
if isinstance(rhs, Path):
rpath = rhs.path
elif isinstance(rhs, unicode):
rpath = rhs
else:
return False
return os.path.abspath(self.path) == os.path.abspath(rpath)
def __ne__(self, rhs):
return not self.__eq__(rhs)
def __getitem__(self, key):
this, parent = self, self.parent()
bases = [this.base()]
while this != parent:
bases[:0] = [parent.base()]
this, parent = parent, parent.parent()
bases[0] = this
bases = bases[key]
retPath = bases[0]
for base in bases[1:]:
retPath = retPath.child(base)
return retPath
@staticmethod
def getCurrent():
return Path(os.getcwd())
To extract the Python string a Path object has, you can pass the object to str() or unicode(). A friend of mine gave me an idea to make the class to be a subclass of str/unicode so that a Path object can be used as a Python string. But the meaning of __eq__, __ne__, and __getitem__ is quite different between str/unicode and my class.
Thursday, October 29, 2009
Cross platform editor written in Python
http://editra.org/
It's written in Python, It seems to have all the features of wx.STC, multi-lingualized, it has launchers, and it comes with so many lexers 68k assembly, Apache conf, Haskell, LaTex, Lisp, Lua, Matlab, R, Objective-C, VHDL, YAML, ... ofcourse Python.
It's still version 0.5 but looks promising.
Friday, October 23, 2009
PyQt is nice
I started using PyQt. It's so nice. Unlike Qt, you don't have to worry about Qt specific stuff, qmake, .pro, moc, etc. I started off by converting several examples that come with Qt. I realized I could convert a Qt code to a PyQt script half automatically. I've converted some of WebKit examples and there was almost no difficulty, though it took a little bit of time to be able to call a slot in PyQt script from JavaScript, until I read this (french!) and found @QtCore.pyqtSlot works.
Ah, and Qt Designer is very nice :)
Monday, October 19, 2009
Saturday, August 1, 2009
Not a nice way to interrupt a sub thread
In my application, the main thread in in charge of UI drawing with wxPython and starting worker threads which runs user defined Python code. If a user defined Python code is buggy the application user needs to stop it. There's a way to raise a KeyboardInterrupt exception in the main thread but unfortunately not the opposite. What I needed to do is interrupting a sub thread from the main thread. I don't know why it doesn't exist. Probably it is a well considered decision but what I need is what Python misses. The API provides a function PyThreadState_SetAsyncExc() which takes a thread id and exception object and raises exception in the thread. So I had to make a wrapper extension.
pySubthreadInterruptTest.c
#include <Python.h>
static PyObject* interrupt(PyObject* self, PyObject* args)
{
long threadid;
PyObject* po_exception;
if(! PyArg_ParseTuple(args, "lO", &threadid, &po_exception))
{
return NULL;
}
int result = PyThreadState_SetAsyncExc(threadid, po_exception);
return Py_BuildValue("l", result);
}
static PyMethodDef MethodsDefs[] = {
{"interrupt", interrupt, METH_VARARGS},
{NULL, NULL, 0},
};
void initpystit(void){
(void) Py_InitModule("pystit", MethodsDefs);
}
setup.py
from distutils.core import setup, Extension
module1 = Extension('pystit',
sources = ['pySubthreadInterruptTest.c'])
setup (name = 'PackageName',
version = '1.0',
description = 'This is a demo package',
ext_modules = [module1])
test.py
import time, thread
import pystit
def f():
print 'thread start'
try:
for i in range(1000):
print i
time.sleep(0.001)
except:
print 'interrupted'
raise
print 'thread end'
tid = thread.start_new_thread(f, ())
time.sleep(0.001)
pystit.interrupt(tid, ValueError)
time.sleep(1)
$ python setup.py build
running build
running build_ext
building 'pystit' extension
gcc -pthread -shared build/temp.linux-i686-2.6/pySubthreadInterruptTest.o -L/usr/lib -lpython2.6 -o build/lib.linux-i686-2.6/pystit.so
$ cd build/lib.linux-i686-2.6/
$ ls
pystit.so test.py
$ python test.py
thread start
0
1
2
3
interrupted
Unhandled exception in thread started by <function f at 0xb8043e64>
Traceback (most recent call last):
File "test.py", line 8, in f
time.sleep(0.001)
ValueError
$
It works but I can't say I'm quite satisfied with the solution.
Tuesday, July 28, 2009
Highlevel Mouse Event Converter (wxPython)
I really needed to make a good one (not a school assignment :) that convers lowlevel mouse events to highlevel ones. When the user left double clicks on a window, the window gets leftdown, leftup, leftdoubleclick, leftup events in this order. It is too low level. In this case all we want is a single double click event.
- A double click event doesn't come with friends.
- Whenever a mouse down event comes, the application can except a following mouse up event (it may be after dragging event stuff or enter/leave window event).
- A series of dragging events are always wrapped by a startdragging and an enddragging event.
- Capture mouse automatically. (See the comment in onButtonDown() method)
- Can set threshold in pixels so that anybody can double click with his trembling hand (e.g. my dad's)
BUG
While dragging, EnterWindow/LeaveWindow occurs when the mouse enter/leaves the parent window on GTK+.
UPDATES
Jul 29: Passing proper event object to mouse down/up handlers.
Jul 31: Get the right mouse position while mouse captured.
Aug 5 : Fixed missing double click detection failure.
Aug 8 : Modified to Register popup menu functions.
"""Low level -> high level mouse event converter."""
import wx
NORMAL = 0
FIRSTWAIT = 1
SECONDWAIT = 2
AFTERDCLICK = 3
MOUSEDOWN = 4
DRAGGING = 5
ID_FIRSTTIMER = wx.NewId()
ID_SECONDTIMER = wx.NewId()
class HighLevMouseEventConv(object):
def __init__(self, eventWindow, t1 = 200, t2 = 100, dragThreshPixel = 3):
"""HighLevelEvent(eventWindow, t1 = 200, t2 = 100, dragThreshPixel = 3)
eventWindow: The window that receives mouse events.
t1, t2: How long it waits to detect is a mouse down is a double click in mili seconds.
dragThreshPixel: Drag moves within this thresh do not produce drag related events.
When you want a popup menu associated with the window create a func to show the menu, register
it using set(Left, Middle, Right)PopupMenuFunc(). These functions should never return value.
"""
self.t1 = t1
self.t2 = t2
self.dragThreshPixel = dragThreshPixel
self.state = NORMAL
self.evwin = eventWindow
self.firstTimer = wx.Timer(eventWindow, ID_FIRSTTIMER)
self.secondTimer = wx.Timer(eventWindow, ID_SECONDTIMER)
dh = self.defaultHandler
l, r, m = wx.MOUSE_BTN_LEFT, wx.MOUSE_BTN_RIGHT, wx.MOUSE_BTN_MIDDLE
self.onHLVMouseDown = {l:dh, r:dh, m:dh}
self.onHLVMouseUp = {l:dh, r:dh, m:dh}
self.onHLVMouseDClick = {l:dh, r:dh, m:dh}
self.onHLVMouseStartDragging = {l:dh, r:dh, m:dh}
self.onHLVMouseDragging = {l:dh, r:dh, m:dh}
self.onHLVMouseEndDragging = {l:dh, r:dh, m:dh}
self.popupMenuFunc = {l:dh, r:dh, m:dh}
self.onHLVMotion = dh
self.onHLVEnterWindow = dh
self.onHLVLeaveWindow = dh
eventWindow.Bind(wx.EVT_MOUSE_EVENTS, self.onMouseEvents)
eventWindow.Bind(wx.EVT_MOUSE_CAPTURE_LOST, self.onMouseCaptureLost)
eventWindow.Bind(wx.EVT_TIMER, self.onFirstTimer, id = ID_FIRSTTIMER)
eventWindow.Bind(wx.EVT_TIMER, self.onSecondTimer, id = ID_SECONDTIMER)
def setOnHLVLeftDown(self, handler):
self.onHLVMouseDown[wx.MOUSE_BTN_LEFT] = handler
def setOnHLVLeftUp(self, handler):
self.onHLVMouseUp[wx.MOUSE_BTN_LEFT] = handler
def setOnHLVLeftDClick(self, handler):
self.onHLVMouseDClick[wx.MOUSE_BTN_LEFT] = handler
def setOnHLVLeftStartDragging(self, handler):
self.onHLVMouseStartDragging[wx.MOUSE_BTN_LEFT] = handler
def setOnHLVLeftDragging(self, handler):
self.onHLVMouseDragging[wx.MOUSE_BTN_LEFT] = handler
def setOnHLVLeftEndDragging(self, handler):
self.onHLVMouseEndDragging[wx.MOUSE_BTN_LEFT] = handler
def setOnHLVMiddleDown(self, handler):
self.onHLVMouseDown[wx.MOUSE_BTN_MIDDLE] = handler
def setOnHLVMiddleUp(self, handler):
self.onHLVMouseUp[wx.MOUSE_BTN_MIDDLE] = handler
def setOnHLVMiddleDClick(self, handler):
self.onHLVMouseDClick[wx.MOUSE_BTN_MIDDLE] = handler
def setOnHLVMiddleStartDragging(self, handler):
self.onHLVMouseStartDragging[wx.MOUSE_BTN_MIDDLE] = handler
def setOnHLVMiddleDragging(self, handler):
self.onHLVMouseDragging[wx.MOUSE_BTN_MIDDLE] = handler
def setOnHLVMiddleEndDragging(self, handler):
self.onHLVMouseEndDragging[wx.MOUSE_BTN_MIDDLE] = handler
def setOnHLVRightDown(self, handler):
self.onHLVMouseDown[wx.MOUSE_BTN_RIGHT] = handler
def setOnHLVRightUp(self, handler):
self.onHLVMouseUp[wx.MOUSE_BTN_RIGHT] = handler
def setOnHLVRightDClick(self, handler):
self.onHLVMouseDClick[wx.MOUSE_BTN_RIGHT] = handler
def setOnHLVRightStartDragging(self, handler):
self.onHLVMouseStartDragging[wx.MOUSE_BTN_RIGHT] = handler
def setOnHLVRightDragging(self, handler):
self.onHLVMouseDragging[wx.MOUSE_BTN_RIGHT] = handler
def setOnHLVRightEndDragging(self, handler):
self.onHLVMouseEndDragging[wx.MOUSE_BTN_RIGHT] = handler
def setOnHLVMotion(self, handler):
self.onHLVMotion = handler
def setOnHLVEnterWindow(self, handler):
"""It can detect the mouse enter/leaves from, to the parent window when the mouse is dragging.
There's no clear way but detecting mouse motion event to enter/leave the window itself
while the mouse is captured to the parent"""
self.onHLVEnterWindow = handler
def setOnHLVLeaveWindow(self, handler):
"""See setOnHLVLeaveWindow doc."""
self.onHLVLeaveWindow = handler
#popupMenu
def setLeftPopupMenuFunc(self, func):
self.popupMenuFunc[wx.MOUSE_BTN_LEFT] = func
def setMiddlePopupMenuFunc(self, func):
self.popupMenuFunc[wx.MOUSE_BTN_MIDDLE] = func
def setRightPopupMenuFunc(self, func):
self.popupMenuFunc[wx.MOUSE_BTN_RIGHT] = func
def defaultHandler(self, ev):
return True
def onButtonDown(self, ev):
if not self.popupMenuFunc[ev.GetButton()](ev):
return
if self.state == AFTERDCLICK:
self.state = NORMAL
self.evwin.CaptureMouse()
#On GTK, I knoticed if the window is a control, its parent gets mouse events when the window
#captures mouse events with CaptureMouse(). To call ReleaseMouse() properly this object receives
#the mouse leftup events sent to the parent window temporarily.
parent = self.evwin.GetParent()
if parent:
parent.Bind(wx.EVT_LEFT_UP, self.onParentButtonUp)
parent.Bind(wx.EVT_MIDDLE_UP, self.onParentButtonUp)
parent.Bind(wx.EVT_RIGHT_UP, self.onParentButtonUp)
parent.Bind(wx.EVT_MOUSE_CAPTURE_LOST, self.onMouseCaptureLost)
parent.Bind(wx.EVT_MOTION, self.onParentMouseEvents)
parent.Bind(wx.EVT_ENTER_WINDOW, self.onParentEntering)
parent.Bind(wx.EVT_LEAVE_WINDOW, self.onParentLeaving)
self.secondTimer.Stop()
state = self.state
if state == NORMAL:
self.button = ev.GetButton()
self.state = FIRSTWAIT
self.posx, self.posy = ev.GetPosition()
self.downevattrs = self._getEventAttrs(ev)
self.firstTimer.Start(self.t1, True)
elif state == SECONDWAIT:
self.state = AFTERDCLICK
self.onHLVMouseDClick[self.button](ev)
def onButtonUp(self, ev):
evwin = self.evwin
if evwin.HasCapture():
evwin.ReleaseMouse()
parent = evwin.GetParent()
if parent:
parent.Unbind(wx.EVT_LEFT_UP)
parent.Unbind(wx.EVT_MIDDLE_UP)
parent.Unbind(wx.EVT_RIGHT_UP)
parent.Unbind(wx.EVT_MOUSE_CAPTURE_LOST)
parent.Unbind(wx.EVT_MOTION)
parent.Unbind(wx.EVT_ENTER_WINDOW)
parent.Unbind(wx.EVT_LEAVE_WINDOW)
self.firstTimer.Stop()
state = self.state
if state == FIRSTWAIT:
self.state = SECONDWAIT
self.upevattrs = self._getEventAttrs(ev)
self.secondTimer.Start(self.t2, True)
elif state == AFTERDCLICK:
self.state = NORMAL
elif state == MOUSEDOWN:
self.state = NORMAL
self.onHLVMouseUp[self.button](ev)
elif state == DRAGGING:
self.state = NORMAL
self.onHLVMouseEndDragging[self.button](ev)
self.onHLVMouseUp[self.button](ev)
def onButtonDClick(self, ev):
self.secondTimer.Stop()
state = self.state
if state == NORMAL:
self.onButtonDown(ev)
elif state == SECONDWAIT:
self.state = AFTERDCLICK
self.onHLVMouseDClick[self.button](ev)
def onMoving(self, ev):
if self.state == NORMAL:
self.onHLVMotion(ev)
def onDragging(self, ev):
x, y = ev.GetPosition()
t = self.dragThreshPixel
state = self.state
if state == FIRSTWAIT:
if -t < x - self.posx < t and -t < y - self.posy < t:
return
else:
self.state = DRAGGING
self.firstTimer.Stop()
self.onHLVMouseDown[self.button](ev)
self.onHLVMouseStartDragging[self.button](ev)
elif state == MOUSEDOWN:
if -t < x - self.posx < t and -t < y - self.posy < t:
return
else:
self.state = DRAGGING
self.onHLVMouseStartDragging[self.button](ev)
elif state == DRAGGING:
self.onHLVMouseDragging[self.button](ev)
def onEntering(self, ev):
self.onHLVEnterWindow(ev)
def onLeaving(self, ev):
self.onHLVLeaveWindow(ev)
def onMouseEvents(self, ev):
ev.Skip()
if ev.Moving():
return self.onMoving(ev)
elif ev.Dragging():
return self.onDragging(ev)
elif ev.ButtonDown():
return self.onButtonDown(ev)
elif ev.ButtonUp():
return self.onButtonUp(ev)
elif ev.ButtonDClick():
return self.onButtonDClick(ev)
elif ev.Entering():
return self.onEntering(ev)
elif ev.Leaving():
return self.onLeaving(ev)
def onMouseCaptureLost(self, ev):
self.evwin.ReleaseMouse()
ev.Skip()
def onFirstTimer(self, ev):
self.state = MOUSEDOWN
downev = self._createMouseEvent(self.downevattrs)
self.onHLVMouseDown[self.button](downev)
def onSecondTimer(self, ev):
self.state = NORMAL
downev = self._createMouseEvent(self.downevattrs)
upev = self._createMouseEvent(self.upevattrs)
self.onHLVMouseDown[self.button](downev)
self.onHLVMouseUp[self.button](upev)
def onParentButtonUp(self, ev):
self._convetMouseEvent(ev)
self.onButtonUp(ev)
def onParentMouseEvents(self, ev):
self._convetMouseEvent(ev)
self.onMouseEvents(ev)
def onParentEntering(self, ev):
self._convetMouseEvent(ev)
self.onEntering(ev)
def onParentLeaving(self, ev):
self._convetMouseEvent(ev)
self.onLeaving(ev)
def _convetMouseEvent(self, ev):
evwin = self.evwin
parent = evwin.GetParent()
x, y = ev.GetPosition()
offsetx, offsety = evwin.GetPosition()
ev.m_x = x - offsetx
ev.m_y = y - offsety
ev.SetEventObject(evwin)
def _getEventAttrs(self, ev):
return [
ev.GetEventObject(),
ev.GetEventType(),
ev.GetId(),
ev.GetTimestamp(),
ev.m_altDown,
ev.m_controlDown,
ev.m_leftDown,
ev.m_middleDown,
ev.m_rightDown,
ev.m_metaDown,
ev.m_shiftDown,
ev.m_x,
ev.m_y,
ev.m_wheelRotation,
ev.m_wheelDelta,
ev.m_linesPerAction]
def _createMouseEvent(self, eventAttrs):
obj, typ, id, time, alt, ctrl, l, m, r, meta, shift, x, y, wr, wd, lpa = eventAttrs
ev = wx.MouseEvent(typ)
ev.SetEventObject(obj)
ev.GetEventObject()
ev.SetEventType(typ)
ev.SetId(id)
ev.SetTimestamp(time)
ev.m_altDown = alt
ev.m_controlDown = ctrl
ev.m_leftDown = l
ev.m_middleDown = m
ev.m_rightDown = r
ev.m_metaDown = meta
ev.m_shiftDown = shift
ev.m_x = x
ev.m_y = y
ev.m_wheelRotation = wr
ev.m_wheelDelta = wd
ev.m_linesPerAction = lpa
return ev
if __name__ == '__main__':
class TestWindow(wx.Button):
def __init__(self, parent):
super(TestWindow, self).__init__(parent, -1, pos = (50, 50), size = (150, 150))
hlev = HighLevMouseEventConv(self, dragThreshPixel = 15)
hlev.setOnHLVLeftDown(self.onHLVLeftDown)
hlev.setOnHLVLeftUp(self.onHLVLeftUp)
hlev.setOnHLVLeftDClick(self.onHLVLeftDClick)
hlev.setOnHLVLeftStartDragging(self.onHLVLeftStartDragging)
hlev.setOnHLVLeftDragging(self.onHLVLeftDragging)
hlev.setOnHLVLeftEndDragging(self.onHLVLeftEndDragging)
hlev.setOnHLVMiddleDown(self.onHLVMiddleDown)
hlev.setOnHLVMiddleUp(self.onHLVMiddleUp)
hlev.setOnHLVMiddleDClick(self.onHLVMiddleDClick)
hlev.setOnHLVMiddleStartDragging(self.onHLVMiddleStartDragging)
hlev.setOnHLVMiddleDragging(self.onHLVMiddleDragging)
hlev.setOnHLVMiddleEndDragging(self.onHLVMiddleEndDragging)
hlev.setOnHLVRightDown(self.onHLVRightDown)
hlev.setOnHLVRightUp(self.onHLVRightUp)
hlev.setOnHLVRightDClick(self.onHLVRightDClick)
hlev.setOnHLVRightStartDragging(self.onHLVRightStartDragging)
hlev.setOnHLVRightDragging(self.onHLVRightDragging)
hlev.setOnHLVRightEndDragging(self.onHLVRightEndDragging)
hlev.setOnHLVMotion(self.onHLVMotion)
hlev.setOnHLVEnterWindow(self.onHLVEnterWindow)
hlev.setOnHLVLeaveWindow(self.onHLVLeaveWindow)
self.lup = True
self.dragstate = 'normal'
def onHLVLeftDown(self, ev):
print 'onHLVLeftDown', ev.GetPosition(), type(ev.GetEventObject()).__name__
if not self.lup: print 'ERROR ' * 5
self.lup = False
def onHLVLeftUp(self, ev):
print 'onHLVLeftUp', ev.GetPosition(), type(ev.GetEventObject()).__name__
self.lup = True
def onHLVLeftDClick(self, ev):
print 'onHLVLeftDClick', ev.GetPosition(), type(ev.GetEventObject()).__name__
def onHLVLeftStartDragging(self, ev):
print 'onHLVLeftStartDragging', ev.GetPosition(), type(ev.GetEventObject()).__name__
if self.dragstate != 'normal': print 'ERROR ' * 5
self.dragstate = 'started'
def onHLVLeftDragging(self, ev):
print 'onHLVLeftDragging', ev.GetPosition(), type(ev.GetEventObject()).__name__
if not self.dragstate in ('started', 'dragging'): print 'ERROR ' * 5
self.dragstate = 'dragging'
def onHLVLeftEndDragging(self, ev):
print 'onHLVLeftEndDragging', ev.GetPosition(), type(ev.GetEventObject()).__name__
if not self.dragstate in ('started', 'dragging'): print 'ERROR ' * 5
self.dragstate = 'normal'
def onHLVMiddleDown(self, ev):
print 'onHLVMiddleDown', ev.GetPosition(), type(ev.GetEventObject()).__name__
def onHLVMiddleUp(self, ev):
print 'onHLVMiddleUp', ev.GetPosition(), type(ev.GetEventObject()).__name__
def onHLVMiddleDClick(self, ev):
print 'onHLVMiddleDClick', ev.GetPosition(), type(ev.GetEventObject()).__name__
def onHLVMiddleStartDragging(self, ev):
print 'onHLVMiddleStartDragging', ev.GetPosition(), type(ev.GetEventObject()).__name__
def onHLVMiddleDragging(self, ev):
print 'onHLVMiddleDragging', ev.GetPosition(), type(ev.GetEventObject()).__name__
def onHLVMiddleEndDragging(self, ev):
print 'onHLVMiddleEndDragging', ev.GetPosition(), type(ev.GetEventObject()).__name__
def onHLVRightDown(self, ev):
print 'onHLVRightDown', ev.GetPosition(), type(ev.GetEventObject()).__name__
def onHLVRightUp(self, ev):
print 'onHLVRightUp', ev.GetPosition(), type(ev.GetEventObject()).__name__
def onHLVRightDClick(self, ev):
print 'onHLVRightDClick', ev.GetPosition(), type(ev.GetEventObject()).__name__
def onHLVRightStartDragging(self, ev):
print 'onHLVRightStartDragging', ev.GetPosition(), type(ev.GetEventObject()).__name__
def onHLVRightDragging(self, ev):
print 'onHLVRightDragging', ev.GetPosition(), type(ev.GetEventObject()).__name__
def onHLVRightEndDragging(self, ev):
print 'onHLVRightEndDragging', ev.GetPosition(), type(ev.GetEventObject()).__name__
def onHLVMotion(self, ev):
print 'onHLVMotion', ev.GetPosition(), type(ev.GetEventObject()).__name__
def onHLVEnterWindow(self, ev):
print 'onHLVEnterWindow', ev.GetPosition(), type(ev.GetEventObject()).__name__
def onHLVLeaveWindow(self, ev):
print 'onHLVLeaveWindow', ev.GetPosition(), type(ev.GetEventObject()).__name__
app = wx.App()
frame = wx.Frame(None, -1, "Title", size = (300, 400))
panel = wx.Panel(frame, -1)
testwin = TestWindow(panel)
frame.Show()
app.MainLoop()
Saturday, July 25, 2009
Click selecting a curve
I wanted a nice connection arrow for my app, not just a concatenation of lines like
Recently wxWidgets got cairo and wxPython got it accordingly. Using wxcairo (wx.lib.wxcairo) I could draw a nice line of connection like this.
But there was one problem, in my app, the user needs to be able to select the connection by clicking it. How can I detect if the mouse point is on a connection? I need to calculate the length between the line and the point. cairo uses 3rd order bezier curve. To get the parameter t of the bezier curve that gives the closest point on the curve to the mouse point, I need to differentiate the distance and solve the equation of t (or I can get the condition that on the closest point the tangent vector and the vector towards the mouse point makes the right angle. It should produce the same result). The problem is, the equation is 5th order and I cannot solve it. I thought if I should use numerical calculation but still the calculation cost is too expensive. So I decided to use linear interpolation. It's good as long as the user doesn't recognize the error. I can use bounding box and it won't be time consuming. Finally I decided to divide the curve where t = 0, 0.05, 0.1, 0.15, 0.3, 0.5, 0.7, 0.85, 0.9, 0.95, 1. It should be enough just for a GUI program.
These are several tests.
Divide the curve in 3, 5, 8, and 10
Close up
Final result
Friday, July 17, 2009
A module file can be loaded twice as two different modules (in Python 2.x)
I'll show you a case where one file (mymodule.py) gets loaded twice as two different modules. I'm sure this is trivial to some people, if you know how to do this, you don't have to read the rest of this entry. I will write it anyway hoping it will help somebody since it could bring a huge confusion if he doesn't know why it happens (huge confusion, according to my experience).
Modules and packages in directories found in sys.path are called toplevel. Those which are not in the directories found in sys.path but in a package are still accessible but not toplevel.
Say you have a package named mypackage in a directory you can find in the PYTHONPATH, and it has a module mymodule. My package is a toplevel and mymodule is not.
$ pwd /home/kotamura/mytest $ echo $PYTHONPATH /home/kotamura/mytest/mytoplevel $ tree mytoplevel/ mytoplevel/ `-- mypackage |-- __init__.py `-- mymodule.pyLet's run Python interactively.
$ python Python 2.6 (r26:66714, Jun 8 2009, 16:07:26) [GCC 4.4.0 20090506 (Red Hat 4.4.0-4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import mypackage.mymodule as mm >>> mm.__name__ 'mypackage.mymodule'You can see the module is not a toplevel from its name (it's under mypackage.)
Go down the directories and do it again.
$ python Python 2.6 (r26:66714, Jun 8 2009, 16:07:26) [GCC 4.4.0 20090506 (Red Hat 4.4.0-4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import mypackage.mymodule as mm >>> mm.__name__ 'mypackage.mymodule'
It's not surprising at all. But you can also import mymodule
>>> import mymodule as m >>> m.__name__ 'mymodule'
Now mymodule is toplevel!, This is because if you run Python interactively Python adds the current directory to sys.path. What is important is that mm and m are different objects even if it is created from the same file mymodule.py
>>> mm.a = 1 >>> m.a = 2 >>> mm.a 1 >>> mm is m False
It's not the case you will see only when you use interactive console. The same thing happens when you import a module from another module without specifying it from the toplevel.
$ cat weirdImporter.py import mypackage.mymodule as mm import mymodule as m print mm.__name__ print m.__name__
You don't have to be in the directory to run weirdImporter.py. You can run it from anywhere
$ python /home/kotamura/mytest/mytoplevel/mypackage/weirdImporter.py mypackage.mymodule mymodule
Again mymodule.pygot imported twice.
import mymodule as m
Python lets weirdImporter import mymodule because it is in the same directory, and since weirdImporter runs in the __main__ module (not mypackage.weirdImporter I mean), it imports mymodule as a toplevel. While
import mypackage.mymodule as mm
imports the mymoduel.py as mypackage.mymodule. So they are different.
A solution many people recommend is that you always import a package/module specicfying the path from the toplevel. It will solve every problem. You can also use "relative import" first introduced in Python2.5 but it brings another confusion until you get used to it.
Finally, Python3.x doesn't let weirdImporter import mymoduel.py only because it's in the same directory. But you still need to be careful not to have the same file be imported more than once from multiple toplevels. Don't forget executing 'python /path/to/foo.py' adds /path/to directory to sys.path.
Tuesday, July 14, 2009
wx.MiniPanel
While I was browsing wxPython demo, I found a really nifty window called wx.MiniFrame.
I thought it was nice for my app but soon realized it's wx.Frame. I wanted it inside my application to draw a graph. I didn't want one of my nodes to travel outside my application window and join Firefox, so I decided to make a similar one as a panel.
Can you see which one is MiniFrame and which is MiniPanel?
One that cannot' be outside the application window is MiniPanel!
(well, anybody talked about close but something? I don't know... (works on Windows))
I haven't finished minor tunings, e.g. the title is embedded in the code, but it's already resizable.
Jul. 15 added:
I tested on (Python2.6, wxPython-2.8.10.1, Fedora11) and (Python 2.6.2, wxPython 2.8.10.1, WindowsXP). Not tested on Mac.
import wx
import wx.lib.resizewidget as resizewidget
import cStringIO as StringIO
batten = '\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\x11\x00\x00\x00\x10\x08\x02\x00\x00\x00\x7fS\x03\x08\x00\x00\x00\x01sRGB\x00\xae\xce\x1c\xe9\x00\x00\x00\x04gAMA\x00\x00\xb1\x8f\x0b\xfca\x05\x00\x00\x00 cHRM\x00\x00z&\x00\x00\x80\x84\x00\x00\xfa\x00\x00\x00\x80\xe8\x00\x00u0\x00\x00\xea`\x00\x00:\x98\x00\x00\x17p\x9c\xbaQ<\x00\x00\x00`IDAT8OcdX\xf9\x9f\x81d\x00\xd4C*B\xd6\xf0\x1f7@1\x17\xce\x01\xaa\xc7c!\x8a,\xa6\x1edi8\x1b\x9f\x1e\xb8\xeb\x80f\xa1\xb1\x11\xae\xc0j\x0f\xb2\xbf \n\x08\xb8\r\xcd\x06\xa2\xf4\xa0\x05\x1ea=\xe4\xfb\x07\x7f`2P\x14?\xc8^\xc7L\x0f\xd8\xd3\x01\t\xa9\x8e\x04\xa5\xf0\xa4L\x86\x1e\x00\xbc\xf4\xfe\xa6\xcaO\x9a\x16\x00\x00\x00\x00IEND\xaeB`\x82'
bmpstream = StringIO.StringIO(batten)
try:
image = wx.ImageFromStream(bmpstream)
finally:
bmpstream.close()
class _TitleBarPanel(wx.Panel):
def __init__(self, parent, titleFGColour, titleBGColour, barWidth, fontSize, title):
wx.Panel.__init__(self, parent, pos = (0, 0), size = (0, barWidth))
self.SetBackgroundColour(titleBGColour)
self.SetMaxSize((100000, barWidth))
self.parent = parent
self.gparent = parent.GetParent()
self.restwin = None
#Title static text.
self.titlest = titlest = wx.StaticText(self, -1, title, size = (100000, barWidth))
titlest.SetForegroundColour(titleFGColour)
font = titlest.GetFont()
font.SetPointSize(fontSize)
titlest.SetFont(font)
#Close bitmap button.
bfi = wx.BitmapFromImage(image)
self.closeb = b = wx.BitmapButton(self, -1, bfi, (0, 0), (barWidth, barWidth), style = wx.BU_EXACTFIT)
b.SetToolTipString("Delete this node.")
hsizer = wx.BoxSizer(wx.HORIZONTAL)
hsizer.Add(titlest, 1, wx.EXPAND | wx.CENTER)
hsizer.Add(b, 0, wx.RIGHT)
self.SetSizer(hsizer)
self.titlest.Bind(wx.EVT_BUTTON, self.onClose, b)
self.Bind(wx.EVT_BUTTON, self.onClose, b)
self.titlest.Bind(wx.EVT_LEFT_DOWN, self.onTitleMouseLeftDown)
self.titlest.Bind(wx.EVT_LEFT_UP, self.onTitleMouseLeftUp)
self.Bind(wx.EVT_LEFT_UP, self.onTitleMouseLeftUp)
self.titlest.Bind(wx.EVT_MOTION, self.onTitleMouseMotion)
self.Bind(wx.EVT_MOTION, self.onTitleMouseMotion)
self.titlest.Bind(wx.EVT_MOUSE_CAPTURE_LOST, self.onTitleMouseCaptureLost)
self.Bind(wx.EVT_MOUSE_CAPTURE_LOST, self.onTitleMouseCaptureLost)
def onClose(self, ev):
self.parent.Close()
def onTitleMouseLeftDown(self, ev):
self.titlest.CaptureMouse()
x, y = ev.GetPosition()
sx, sy = self.titlest.GetScreenPosition()
gx, gy = self.gparent.GetPosition()
self.px, self.py = sx + x, sy + y
self.gx, self.gy = gx, gy
def onTitleMouseLeftUp(self, ev):
if self.titlest.HasCapture():
self.titlest.ReleaseMouse()
def onTitleMouseMotion(self, ev):
if ev.Dragging():
x, y = ev.GetPosition()
sx, sy = self.titlest.GetScreenPosition()
px, py = sx + x, sy + y
if self.restwin:
fsx, fsy = self.restwin.GetScreenPosition()
fw, fh = self.restwin.GetSize()
margin = self.margin
if px < fsx + margin or py < fsy + margin or px > fsx + fw - margin or py > fsy + fh - margin:
return
self.gparent.SetPosition((self.gx + px - self.px, self.gy + py - self.py))
def onTitleMouseCaptureLost(self, ev):
self.titlest.ReleaseMouse()
def setDragRestrictWindow(self, window, margin):
self.restwin = window
self.margin = margin
class NodePanel(wx.Panel):
def __init__(self, parent, id = wx.ID_ANY, pos=wx.DefaultPosition, size = wx.DefaultSize,
style = wx.TAB_TRAVERSAL|wx.NO_BORDER, name=wx.PanelNameStr,
borderColour = wx.BLACK, minSize = (0, 0), maxSize = (-1, -1),
bgColour = wx.NullColour, titleFGColour = wx.WHITE, titleBGColour = wx.Colour(80, 167, 255),
barWidth = 13, fontSize = 8, title = 'MiniPanel'):
wx.Panel.__init__(self, parent, id, pos, size, style | wx.FULL_REPAINT_ON_RESIZE, name)
self.borderColour = borderColour
rw = resizewidget.ResizeWidget(parent, pos = pos, style = wx.FULL_REPAINT_ON_RESIZE)
self.SetMinSize(minSize)
self.SetMaxSize(maxSize)
rw.SetManagedChild(self)
self.tb = tb = _TitleBarPanel(self, titleFGColour, titleBGColour, barWidth, fontSize, title)
self.clientPanel = clientPanel = wx.Panel(self)
sizer = wx.BoxSizer(wx.VERTICAL)
sizer.Add(tb, 0, wx.EXPAND | wx.TOP | wx.LEFT | wx.RIGHT, border = 3)
sizer.Add(clientPanel, 1, wx.EXPAND | wx.BOTTOM | wx.LEFT | wx.RIGHT, border = 3)
self.SetSizer(sizer)
self.SetBackgroundColour(bgColour)
clientPanel.SetBackgroundColour(bgColour)
rw.SetSize(size)
self.Bind(wx.EVT_PAINT,self.OnPaint)
self.Bind(wx.EVT_IDLE,self.OnPaintOnIdle)
def OnPaintOnIdle(self, ev):
#self._onPaint(wx.ClientDC(self))
pass
def OnPaint(self, ev):
self._onPaint(wx.PaintDC(self))
def _onPaint(self, dc):
#Use double buffering as you like.
#(wxPython-demo dir)/samples/pySketch/pySketch.py has a good example of it.
dc.SetPen(wx.Pen(self.borderColour))
x, y = self.GetSize()
dc.DrawLine(0, 0, x-1, 0)
dc.DrawLine(0, 0, 0, y-1)
dc.DrawLine(x-1, 0, x-1, y-1)
dc.DrawLine(0, y-1, x-1, y-1)
def setDragRestrictWindow(self, window, margin = 0):
#If you don't set this, the Panel will go outside the window. Is there a nice way? :(
self.tb.setDragRestrictWindow(window, margin)
def getClientPanel(self):
return self.clientPanel
def getTitleStaticText(self):
return self.tb.titlest
def Destroy(self):
#The resize handle is the parent of the Panel(reparented). We want to destroy
#the handle too. We'll override Destroy() to make sure it is destroyed.
self.GetParent().Destroy()
if __name__ == '__main__':
myapp=wx.PySimpleApp()
frame = wx.Frame(None, -1, "Title", size = wx.Size(300, 400))
scroll = wx.ScrolledWindow(frame, -1)
scroll.SetScrollbars(50,50,10,10)
np = NodePanel(scroll, pos = (30, 30), size = (200, 200), borderColour = (128, 128, 128))
np.setDragRestrictWindow(scroll, 15)
print np.getTitleStaticText().GetLabel()
def OnClose(ev):
if ev.CanVeto():
dlg = wx.MessageDialog(None, 'Are you sure?', 'msgb', wx.YES_NO)
if dlg.ShowModal() == wx.ID_YES:
np.GetParent().Destroy()
dlg.Destroy()
else:
np.Destroy()
np.Bind(wx.EVT_CLOSE, OnClose, np)
def CloseIt(ev):
np.Close()
clientAreaCloseButton = wx.Button(np.getClientPanel(), -1, "Close Me", pos = (20, 20))
np.Bind(wx.EVT_BUTTON, CloseIt, clientAreaCloseButton)
frame.Show()
myapp.MainLoop()
Wednesday, July 8, 2009
Thread Manager
Last update Jun 10
import thread, threading
class _WorkerThread(threading.Thread):
def __init__(self, sem, job, *args, **kargs):
self._workSem = sem
super(_WorkerThread, self).__init__()
self.job = job
self.args = args
self.kargs = kargs
self.isCanceled = False
self.hasStarted = False
self._cancelLock = threading.Lock()
def run(self):
if self._workSem:
with self._workSem:
with self._cancelLock:
if self.isCanceled:
return
self.hasStarted = True
self.job(*self.args, **self.kargs)
else:
self.job(*self.args, **self.kargs)
class JobManager(object):
def __init__(self, maxnumthreads = 1):
"""Set maxnumthreads to specify the max number of threads which runs concurrently."""
self._workers = {}
self._unmanagedWorkers = []
self._workSem = threading.Semaphore(maxnumthreads)
self.maxnumthreads = maxnumthreads
self._lock = threading.RLock()
self._nextJobIdCounter = 1
def postJob(self, job, *args, **kargs):
"""job must be a callable, args and kargs are arguments passed to it."""
self._gc()
wt = _WorkerThread(self._workSem, job, *args, **kargs)
wt.start()
id = self._nextJobIdCounter
self._workers[id] = wt
self._nextJobIdCounter += 1
return id
def cancelJob(self, jobid):
"""Cancel a posted job. jobid must be an object returned by postJob().
It returns True if the job gets canceled, False it it has started."""
self._gc()
worker = self._workers.get(jobid, None)
if not worker: #It doesn't exist because it finished execution and removed from the _workers
return False
with worker._cancelLock:
worker.isCanceled = True
return not worker.hasStarted
def waitOnIdle(self):
"""Blocks until the every worker thread terminates."""
self._gc()
while self._workers or self._unmanagedWorkers:
wt = (self._workers.values() + self._unmanagedWorkers).pop()
wt.join()
self._gc()
def getNumWaitingJobs(self):
"""Returns the number of jobs waiting. It includes threads currently running."""
self._gc()
return len(self._workers)
def forceExecuteOnWorkerThread(self, job, *args, **kargs):
"""Execute the job immediately on a thread. It is not queued."""
wt = _WorkerThread(None, job, *args, **kargs)
wt.start()
self._unmanagedWorkers.append(wt)
def executeWhenNoWorkerThreadsRunning(self, job, *args, **kargs):
"""The calling thread execute the job (callable), ensuring no worker threads running.
It blocks when a thread is running.It DOESN'T mean job is executed after the every
worker threads has been terminated.
(Though it looks the current Python implementation awakens a thread which called acquire()
earlier.)
It doesn't take jobs launched by forceExecuteOnWorkerThread() into account."""
for i in range(self.maxnumthreads):
self._workSem.acquire()
try:
job(*args, **kargs)
finally:
for i in range(self.maxnumthreads):
self._workSem.release()
def _gc(self):
with self._lock:
self._workers = dict([w for w in self._workers.items() if w[1].isAlive()])
self._unmanagedWorkers = [wt for wt in self._unmanagedWorkers if wt.isAlive()]
if __name__ == '__main__':
import time
def somejob(i, wait = 0.1):
time.sleep(wait)
print 'somejob', i
time.sleep(1)
def endMessage(msg):
print msg
jm = JobManager()
jobIds = []
for i in range(5):
id = jm.postJob(somejob, i + 1)
jobIds.append(id)
print 'posted', id
time.sleep(2)
print "cancel", jobIds[0], jm.cancelJob(jobIds[0])
print "cancel", jobIds[-1], jm.cancelJob(jobIds[-1])
#print jm.getNumWaitingJobs()
jm.forceExecuteOnWorkerThread(somejob, 'unmanaged1', 4)
jm.waitOnIdle() #It waits for unmanaged1 termination
jm.forceExecuteOnWorkerThread(somejob, 'unmanaged2')
jm.executeWhenNoWorkerThreadsRunning(endMessage, 'done') #It doesn't wait for unmanaged2
Sunday, May 31, 2009
Python type createion
Now I'm looking at Python3.0 source code.
PyTypeObject PyUnicode_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"str", /* tp_name */
sizeof(PyUnicodeObject), /* tp_size */
...(snip)...
&unicode_as_number, /* tp_as_number */
&unicode_as_sequence, /* tp_as_sequence */
&unicode_as_mapping, /* tp_as_mapping */
(hashfunc) unicode_hash, /* tp_hash*/
0, /* tp_call*/
(reprfunc) unicode_str, /* tp_str */
...(snip)...
This is a code snippet from Python3.01 unicode object implementation. tp_as_sequence is a behavior definition when the object is used as a sequence, tp_as_mapping is a behavior when used as a mapping object. unicode_as_sequence is defined like this.
static PySequenceMethods unicode_as_sequence = {
(lenfunc) unicode_length, /* sq_length */
PyUnicode_Concat, /* sq_concat */
(ssizeargfunc) unicode_repeat, /* sq_repeat */
(ssizeargfunc) unicode_getitem, /* sq_item */
0, /* sq_slice */
0, /* sq_ass_item */
0, /* sq_ass_slice */
PyUnicode_Contains, /* sq_contains */
};
unicode_repeat etc. are just a simple C function.
Another code snippet taken from bool object implementation. bool is a subtype of int. It is specified by setting PyLong_Type(i.e. int) to tp_base entry.
PyTypeObject PyBool_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"bool",
...(snip)...
&PyLong_Type, /* tp_base */
...(snip)...
};
Simple enough.
(Above code snippets are Python licensed)
Friday, May 8, 2009
Python FizzBuzz oneliner
From a comment on this blog via this blog
for i in range(1,101):print"FizzBuzz"[i*i%3*4:8--i**4%5]or i
This is mine (May, 17, 18 modified)
print '\n'.join(['Fizz'*(not i%3) + 'Buzz'*(not i%5) or str(i) for i in range(1, 101)])
I post this topic on a Python forum and a guy posted this, nice.
for x in range(100): print x%3/2*'Fizz'+x%5/4*'Buzz' or x+1
Passing more than 65536 arguments to a function
Creating a simple function with 4 arguments and it works.
>>> cmd1 = 'def f('
>>> cmd2 = cmd2[:-3]
>>> cmd2 = 'print '
>>> for i in range(0, 4):
... cmd1 += 'p' + str(i) + ', '
... cmd2 += 'p' + str(i) + ' + '
...
>>> cmd = cmd1[:-2] + "):" + cmd2[:-3]
>>> cmd
'def f(p0, p1, p2, p3):print p0 + p1 + p2 + p3'
>>> exec cmd
>>> f(1,1,1,1)
4
Let's discompile it.
>>> dis.dis(f)
1 0 LOAD_FAST 0 (p0)
3 LOAD_FAST 1 (p1)
6 BINARY_ADD
7 LOAD_FAST 2 (p2)
10 BINARY_ADD
11 LOAD_FAST 3 (p3)
14 BINARY_ADD
15 PRINT_ITEM
16 PRINT_NEWLINE
17 LOAD_CONST 0 (None)
20 RETURN_VALUE
The first number of each line is the address of the instruction(see this). You'll see LOAD_FAST takes three bytes, which means an argument is indexed in two bytes (LOAD_FAST instruction itself is one byte). So Python should be confused if we pass more than 65536 arguments. Let's create 100000 arguments and confuse it :p
>>> cmd1 = 'def f('
>>> cmd2 = 'print '
>>> for i in range(0, 100000):
... cmd1 += 'p' + str(i) + ', '
... cmd2 += 'p' + str(i) + ' + '
...
>>> cmd = cmd1[:-2] + "):" + cmd2[:-3]
>>> exec cmd
>>> f(*(1,)*100000)
100000
Why did it work? It's me who is confused...
>>> dis.dis(f)
1 0 LOAD_FAST 0 (p0)
3 LOAD_FAST 1 (p1)
6 BINARY_ADD
7 LOAD_FAST 2 (p2)
10 BINARY_ADD
11 LOAD_FAST 3 (p3)
...
262135 LOAD_FAST 65534 (p65534)
262138 BINARY_ADD
262139 LOAD_FAST 65535 (p65535)
262142 BINARY_ADD
262143 EXTENDED_ARG 1
262146 LOAD_FAST 65536L (p65536)
262149 BINARY_ADD
262150 EXTENDED_ARG 1
262153 LOAD_FAST 65537L (p65537)
262156 BINARY_ADD
262157 EXTENDED_ARG 1
...
Whoa! Python is prepared for the mean test. So clever!
May 11 added:
EXTENDED_ARG n is an instruction to tell the interpreter that it should add n*65536 to the next LOAD_FAST argument.
Related code snippet from ceval.c
case EXTENDED_ARG:
opcode = NEXTOP();
oparg = oparg<<16 | NEXTARG();
goto dispatch_opcode;
