Thursday, October 20, 2005

Unicode conversion in Perl

Starting from Perl 5.6, Perl uses utf-8 for storing character strings internally (Just like Java, which also uses Unicode for storing character strings). To convert a string in native encoding to another encoding, we can think of the same way as what we usually do in Java (if you happens to have some experience in Java character set conversion, you will know what I am saying). e.g. If we want to convert a Big5 string to UTF-16 :

use Encode qw/encode decode/;

$big5str = get_big5_string();
$now_in_utf8 = decode("big5", $big5str);
$now_in_utf16 = encode("UTF-16", $now_in_utf8);

For extra information, if you to want to return an Excel spreadsheet containing Chinese characters from CGI, you can do this:

use Spreadsheet::WriteExcel;
use Encode qw/encode decode/;

my $big5str = get_big5_string();
my $now_in_utf8 = decode("big5", $big5str);
my $now_in_utf16 = encode("UTF-16", $now_in_utf8);

$filename ="excel_file.xls";
print "Content-type: application/\n";
print "Content-Disposition: attachment; filename=$filename\n";
print "\n";

my $workbook = Spreadsheet::WriteExcel->new("-");
my $worksheet = $workbook->addworksheet();

$worksheet->write(1,1,"Non-unicode characters");
$worksheet->write_unicode(1, 2, $now_in_utf16);

For M$ Excel before version 2000, you will see square box before each cell, this is due to the BOM character generated during the encode process. Excel 2003 and OpenOffice can recognise this character correctly. To remedy the square bracket, you can just remove it:

my $now_in_utf8 = decode("big5", $big5str);
my $now_in_utf16 = encode("UTF-16", $now_in_utf8);
$now_in_utf16 =~ s/^\x{fe}\x{ff}//;


As mentioned in Ajaxian, AMASS(Ajax MAssive Storage System) allows Ajax application to store potentially large amount of persistent data in local machine. By using a SharedObject in a hidden Flash, AMASS can store local data without user intervention for data size smaller than 100k. For larger amount of data, the system will prompt for user approval. I think this is one step closer for Ajax application to perform similar things as locally installed application.

In current stage, the main issue is it only works in Windows. It has not been tested in Linux platform, and it doesn't work in Mac :<.

Wednesday, October 19, 2005

Subversion initial setup

Subversion is a project aims to build a version control system that is a compelling replacement for CVS in the open source community. Besides usual way of doing version control, this is also a good candidate for backup of anything you can imagine. There is a free online book available, if you want to further study it.

Below are some setup notes I have jotted down during my setup. May be this will be helpful to you also.

Setup in Gentoo Linux, as apache2 module

1. If you happens to have apache 2 already installed, setting up Subversion is easy:

# emerge subversion

But in case you are using apache 1.x, you can still go ahead and let portage to install apache 2 for you. There is nothing much harm (other than use more memory) to run both apache 1.x and 2.x together. All you need is just changing the port number apache 2.x is listening to.

2. Create directory for source repository: (you can use whatever directory)

#mkdir /var/repository

3. Create user account for svn:

#mkdir /var/svn/conf
#htpasswd2 -c /var/svn/conf/svnusers newusername

The second command create a new user "newusername" and store the a/c password info in /var/svn/conf/svnusers. The -c option is for file creation, if you are adding more users, you can simply go without this option.

4. To allow apache2 to load the corresponding svn modules, modify /etc/conf.d/apache2 to add "-D DAV -D SVN", e.g.:


5. Configure the modules config file /etc/apache2/modules.d/47_mod_dav_svn.conf (you may have number other than 47, depending on the portage maintainer). Assuming we use the password file and repository directory created previously, the file may look like this:

<IfDefine SVN>
<IfModule !mod_dav_svn.c>
LoadModule dav_svn_module modules/
<Location /repos>
DAV svn
SVNPath /file2/repos
AuthType Basic
AuthName "WINS Subversion Repository"
AuthUserFile /var/svn/conf/svnusers
Require valid-user
<IfDefine SVN_AUTHZ>
<IfModule !mod_authz_svn.c>
LoadModule authz_svn_module modules/

6. Create your initial repository:

#svnadmin create /var/repository

Setup in Redhat, standalone server

For illustration purpose, this time, we run subversion for standalone version, and invoked through xinetd.

1. First step is also easy:

#rpm -Uvh subversion-XXXX.rpm

Just replace XXXX with the version currently available in your system.

2. Create a user for svn in your system. For security, you should set it's login shell to nologin.

3. Add 2 lines to /etc/services: (if they don't exist yet)

subversion 3690/tcp # Subversion
subversion 3690/udp # Subversion

4. Create a file /etc/xinetd.d/svn

# subversion server
service svn
disable = no
socket_type = stream
protocol = tcp
wait = no
user = svnuser
server = /usr/bin/svnserve
server_args = -i

5. Create your initial repository:

#mkdir /var/repository
#svnadmin create /var/repository

6. During initializing the repository directory, svn creates various directories and files. In particular, there is a config file svnserve.conf inside conf directory created. There are extensive comments inside the config file, with all example setting commented out because they are already set by default by svn. Anyway, you can customize them to your needs, e.g.:

anon-access = none
auth-access = write
password-db = passwd
realm = My Subversion Realm

The above disables anonymous access, and allow write access for authenticated user. Most importantly, it defines the password file for authentication as 'passwd'. Now in the same conf directory, create this file with following content:


Using subversion
The most important command to know first is help:

$svn help

To get help on a specific topic :

$svn help update

Import initial directory for a project:

For apache module:

$svn mkdir 'http://localhost/repos/NewProject' -m 'Creating new Project'
$svn import ./MyProject 'http://localhost/repos/NewProject/trunk' -m 'Import for 1st time'

Note that '/repos' is defined in Location directive in apache config.

For standalone version, use svn:// instead.

$svn mkdir 'svn://localhost/var/repository/NewProject' -m 'Creating new Project'
$svn import ./MyProject 'svn://localhost/var/repository/NewProject/trunk' -m 'Import for 1st time

Note that the '/var/repository' are actually a full path of the directory where you create for repository.

Before performing any commit, you need to perfrom an update first :

$cd ./Myproject
$svn update

To check your checked out source status :

svn status

The svn will also mark all non-version file inside your checkout directory as '?' status.

To ignore some files from subversion status cmd (or add/import), either set it globally in ~/.subversion/config :

### Set global-ignores to a set of whitespace-delimited globs
### which Subversion will ignore in its 'status' output.
global-ignores = *.o *.lo *.la #*# .*.rej *.rej .*~ *~ .#* .DS_Store *.class

or set it in svn cmd:

$svn propset svn:ignore "*" work/

(first arg is the wildcard pattern, second is the path)

to view back any prop set to a path:

$svn proplist work

To commit your changes:

$svn commit -m 'Some changes...'

Video iPod

For the past week, everyone is talking about new iPod. Newspaper said many people will die for this thing because of its low price and video capability. But to me, I think it's just an intermediate product from Apple. Given there are already few PMP supporting 640x480 screen, having good support for DivX and Xvid, I can only see the video support in iPod just as a bonus instead of a great feature for main use.

The battery leads to great disappointment too. The official spec. states the 30G version lasts for 2 hours while the 60G lasts for 3 hours in video playback. Not to mention it probably assumes a fully charged and healthy battery condition.

If video support in iPod needs to be as attractive as music support, I think at least the following features need to be found in future iPod generation:

  • Great battery life for video playback (> 10 hrs)
  • Direct support for DivX and Xvid codecs alike. rmvb support would be great too (just dreaming.)
  • Bigger screen
Bigger screen could be difficult to design, in order to retain the original iPod style and click wheel. But after I saw iRiver U10, Apple should be able to come up with a better idea on their own.

Some rumors say Apple is actively developing a Wi-Fi enabled iPod, with builtin mobile Safari for browsing and even buying tunes in ITS. No matter this rumor is true or not, but network enabled iPod is really a great feature that I personally die for long time ago. Imagine you can listen to Internet radio in public places where WiFi is available. At home, you can stream your bigger music collection from iTunes to your iPod similar to Airport Express. And best of all, we will probably have WiFi headphones to free ourselves from cable. I remember Jobs once thought Bluetooth can't deliver good quality music wirelessly to hp. But I strongly believe WiFi can make this come true. Besides, a builtin mobile Safari means a brand new portable device experience just like PSP does. Besides usual web browsing, I know PSP can now even do remote control through VNC, may be iPod can do similar thing to control your Mac remotely? OK, I know it's already too far away.

How about a Dashboard widget style of APIs for iPod? Imagine all cool things you can do with widgets now can easily ported to iPod. Hmm...

Back to real. Right now, the new Zen Vision would definitely be a better choice for PMP than iPod. Let's see which direction Apple will go in their next step in iPod.

Tuesday, October 18, 2005

Why I switch to Mac OS X

I have been a PC user for more than 15 years, and have been both Windows and Linux users since their very early version. (DOS 5.x to Windows 3.x to XP, Slackware to Redhat to Gentoo). I have also been a user of OS/2 for about 1 to 2 years.

Recently I have switched to Mac OS X for the following reasons:

  1. Easy to use, cool looking, well thought and stable GUI readily available. Supporting Chinese and those default good looking Chinese fonts just can't be available in common Linux distro by default.
  2. BSD Unix behind the scene. So I can still do whatever I can do previously with Linux (Apache, Jboss, mysql, gcc, perl,php)
  3. I am not a gamer, so most of the applications I usually use in XP do have similar equivalent in OS X.
  4. Plug & Play usage for my iPod and firewire/usb external drives. (I can even connect my plain old Creative Jukebox thr. USB, just need to install an extra app)
  5. No more virus/spamware problem.
  6. Can run some great killer applications like Photoshop/Dreamweaver etc.

For the first 2 weeks with my Mac, I am struggling with searching for similar applications I used to have in Windoz. Now my search is nearly over:

Browser -> Safari/Firefox
BT -> Azureus
FTP -> Fugu/Transmit
Text editor -> jEdit
Media Player -> VLC (and realplayer for rmvb)
chm ebooks reader -> xchm
Photo viewer -> FFview/GraphicConverter
SSH client -> Terminal!

Well, I know every platform is not for everyone. For me, I need a rock solid OS which can both do GUI and server applications. Linux should be the default answer, but the GUI from both KDE or GNOME just isn't as cool as Aqua. And you just can't plug and play with most hardware with Linux. Don't flame me on this. That's personal choice. :)

Catalyst Framework

Catalyst is a MVC(Model-View-Controller) Framework in Perl, based on the idea of Maypole and RoR. If you know Perl but want to have a taste of all the juice from RoR, but don't want to learn Ruby, Catalyst should be a good answer.

In particular, it provides URL action mappings, and CRUD application too (i.e. instant web database frontend, once you have your MODEL defined).