Porting Python2.x script to Python3.0

Because Python3.0 (aka Python3000) has no backward compatibility, it is supposed to cause some trouble when porting existing code to Python3.0. According to PEP3000 (PEP stands for Python enhancement proposal) , there is a recommended procedure to port 2.x code to 3.0. This is a quote from PEP3000

0. You should have excellent unit tests with close to full coverage.
1. Port your project to Python 2.6.
2. Turn on the Py3k warnings mode.
3. Test and edit until no warnings remain.
4. Use the 2to3 tool to convert this source code to 3.0 syntax. Do not manually edit the output!
5. Test the converted source code under 3.0.
6. If problems are found, make corrections to the 2.6 version of the source code and go back to step 3.
7. When it's time to release, release separate 2.6 and 3.0 tarballs (or whatever archive form you use for releases).

I recommend you read PEP3000 for upcoming Python3.0 release. Unlike many PEPs, it is quite easy to read. It's not urgent since Python3.0 is not released yet (according to PEP361, Python3.0 is planned to be released on Dec. 03 2008) , and even after its first release, the safest way is not using Python3.0. But if you stick to 2.x for years, you'll be left out from the rest of the world in the meantime.


For Python experts: did you know there's one byte code assigned just for PRINT_NEWLINE? No wonder Guido wanted to renew Python interpreter. You can see the list of byte codes here. (and compare it with Python3.0's byte code instructions, you'll see lots of them are simplified or generalized with arguments.)


by the way, is anybody reading this blog regularly? I'm just curious.

Factory / Ruin photos

I don't know if it's a world trend but in Japan beautiful factory photos are popular since several years ago.
This is a site made by a professional photographer who has published a photo book which has pictures of only industrial complexes.
http://www.kasako.com/0802keihinfoto.html

And ruin photos are getting popular as well.
Maybe he is not a professional photographer but his pictures are awesome.
http://urasoku.blog106.fc2.com/blog-entry-530.html

If you've seen "life after people" and you liked it, you'll also like these.

Ray Tracing Project

While browsing my friend's blog, I found he's mentioned a site that introduces a simple raytracer written in Haskell. That reminded me another site where you can see lots of simple raytracers implemented in various programming languages.
Currently they are written in Awk, C, C++, Java, JavaScript, Lisp, Mel, Perl, Python, Ruby, csh, and Tcl.
Somewhere I found a raytracer written in PostScript but I've forgotten where it was.

The eyeballing game

Another game for artists

















My score:

1st time: Couldn't see the rule.
2nd time: Mouse pointer jumped.
3nd time:


Parallelogram 8.2 4.1 1.0
Midpoint 2.2 4.2 4.2
Bisect angle 2.4 6.6 2.2
Triangle center 1.3 5.1 4.8
Circle center 0.0 2.8 4.5
Right angle 4.4 3.9 4.6
Convergence 2.0 2.2 2.2

Overall score: 3.47

It's more difficult than the previous one. Be prepared!

Python code reading

I made a presentation at a meeting called "Python code reading" held by a Python community in Japan. It's a meeting to read one standard Python module each time (a volunteer makes a presentation) to steal knowledge and techniques by reading scripts written by Python gurus. We read inspect module this time.

Here are several topics I mentioned during the code reading.

- basic usage of inspect module
- method generation mechanism by descriptor and decorator
- custom import (PEP302, which is used by importing zipped modules)
- code object and its meaning
- various special attributes (co_filelineno etc.)
- .pyo file
- file open mode 'U'
- imp module
- negative lookbehind assertion
- method resolution order
- how to clear the cache of the source code which is used by inspect.getsource(), etc.
(if the cache is kept uncleared, getsource() etc. still returns an old source after you modify the script and reload the module)

And what I studied during its preparation (but not talked during the presentation) are

- Python VM and byte code
- disassembling using dis module
- Python dev bug tracking

I didn't mention VM related stuff so that the meeting wouldn't be only for experts (and there was no time to explain them anyway).

Useful inspect module

Inspect module offers various useful functions to Python developers.
Here are four most basic and useful functions inspect module has.

Before showing them, I will just make a simple module (mymodule.py) for the example usages shown below. It has just one function and two classes.

#file mymodule.py


#f() just prints "hello"
def f():
return "hello"

class A(object):
def ma(self):
print "A_m"

class B(A):
def mb(self):
print "B_m"

getsource(object)
This function is probably the most useful one in the module, along with getabsfile().
It returns the source code where object is defined.
object can be a class, method, function, etc (not instance because an instance gets created dynamically).

>>> print inspect.getsource(mymodule.B)
class B(A):
def mb(self):
print "B_m"

getabsfile(object, _filename=None)
It returns the absolute path to the file where object is defined.

>>> print inspect.getabsfile(mymodule.A)
/Users/tamurakouichi/mymodule.py

getcomments(object)
Get lines of comments immediately preceding an object's source code.
the object can be a class, function, etc...

>>> print inspect.getcomments(mymodule.f)
#f() just prints "hello"

classify_class_attrs(object, predicate=None)
It returns a detailed information of attributes a class has.
It includes in what class a method is defined.

>>> for m in inspect.classify_class_attrs(mymodule.B):
...     print m
...
...snip...
('ma', 'method', <class 'mymodule.A'>, <function ma at 0x260c30>)
('mb', 'method', <class 'mymodule.B'>, <function mb at 0x260c70>)

There are many many functions. inspect module is a friend of every Python users from beginners to byte code analysers.

Ragel State Machine Compiler

Have you seen this?

Ragel State Machine Compiler

Ragel compiles executable finite state machines from regular languages. Ragel targets C, C++, Objective-C, D, Java and Ruby. Ragel state machines can not only recognize byte sequences as regular expression machines do, but can also execute code at arbitrary points in the recognition of a regular language. Code embedding is done using inline operators that do not disrupt the regular language syntax.

Interesting, and looks so useful !



Oct. 9 added.

I found another tool.
The State Machine Compiler
While Ragel is more suitable for text processing, this one is something more general purpose. Though it is interesting, I wonder if it is so useful. I know it's tedious to turn a state diagram into a code when the number of states/actions gets larger but it's just tedious and not confusing. Just wondering if adding a tool for it in the project is worth while, trading off building and porting simplicity.
Well, I haven't looked through the document. I'm just saying that after the first look and may change my opinion later.

By the way bringing the concept of state machine in visual programming can be interesting. It's an easy to learn concept and naturally easy to visualize (thinking about something weird again ;)