Object::Tiny:RW and MooseX::NonMoose

There are conflicting requirements for WebNano. On one hand I would like to build big heavy web applications based on it, and of course I would like to use Moose as the object framework for them. On the other hand I would like to give it a chance to work in the restricted environments of most shared web hosting serving CGI scripts - so heavy dependencies and XS and big startup times are out. And I would also like to make it suitable for serving huge amounts of simple Ajax requests - so it needs to be fast.

First I tried using Any::Moose and let people decide if they used Moose or Mouse. By using Mouse people could solve the startup problem and also there is some way to use Mouse without XS.

But Mouse was still huge dependency - so next I asked myself it it would be possible to subclass in Moose a non Moose base classes. That proved to be rather simple with the aptly named MooseX::NonMoose. Using it in WebNano I can build my objects in what ever way I choose and as long as I stick to the standard hash based objects I can extend the WebNano classes in my Moose based applications with no problems. Or at least it seems to be with no problems for me now - if someone knows more about disadvantages of this solution - please leave a comment.

After discovering MooseX::NonMoose I switched to use Class::XSAccessor. It small and very fast, but unfortunately it was XS based. I was thinking about the possibilities of bundling the software as a PAR package or so and using it in the restricted shared hosting environments and relying on a C compiler seemed to complicate things a lot.

Next I tried Object::Tiny, it was tiny, fast and did not use XS, there was just one but - it did not produce the attribute setters. I could live with that and it did not take much time for me to port WebNano to use Object::Tiny, but still it seemed like an extreme choice. Fortunately just browsing CPAN I discovered a fork of O::T - Object::Tiny::RW - this one is small, fast, does not use XS and does produce attribute setters. I am still a bit worry about why Adam would not add that feature to his module - given that it is such a trivial change (compare the sources of Object::Tiny::RW and Object::Tiny - its just one line changed), he must have had a good reason for that?

RavLog - a perl blog engine

For testing HTML::FormHandler Gerda Shank wrote RavLog. Recently I've made a fork of it, switched to SQLite as the main db engine and made it easy to install. It still does not pass all formatting tests (HTML::Tidy seems to be uninstallable) - but it is usable. My goal is to make another fork out of it and port at least part of it to WebNano, but I think that what is now there can also be interesting for someone looking for a Catalyst based weblog engine.

Technically right, socially not so

Imagine that you let anyone use your hard work by publishing your code as Open Source and then instead of thank you notes you receive angry emails about purported bugs in it.

Imagine that you noticed a problem about some Open Source code and as a good citizen you analyze it and report your findings and instead of a thank you note you receive a reply explaining how much confused you are for thinking that that could be a bug.

Imagine that this happens in a public space like an mailing list, open bug tracker, etc. This is like the setting of a Greek tragedy - both sides have their own rights and values, both sides can be technically right - and yet the conflict spreads and escalates. People copy the emotions of each other, even perceived emotions and especially anger, when displayed publicly leads to more anger. This is a vicious circle of positive feedback loop. When it starts everyone gets irritated and everyone wants to know who started it, who is responsible for that.

But go back to the start - it's not anyone's fault - you cannot point the finger on one person and say 'he is responsible'. Looking for the main culprit is really looking for a scapegoat - a surrogate victim that would heal everyone's consciousness. This is not especially geek thing overall - it is a universal trait - but sadly inventing fodder for the scapegoat search is a prominent part of geek culture.

One of the many things that I admire about Larry Wall is how he can neutralize the poisons of angry reactions. People scorn Perl for being a 'scripting' language - he talks about Ada Lovelace:

Suppose you went back to Ada Lovelace and asked her the difference between a script and a program. She'd probably look at you funny, then say something like: Well, a script is what you give the actors, but a program is what you give the audience. That Ada was one sharp lady...

Installing dependencies - continued

I don't know what I am doing wrong - but after installing Perl 5.12.1 under perlbrew "cpan -m ." stopped to install the dependencies. It seems that for making and testing it adds the libs in '.cpan/build' to the PERL5LIB search path so both can go on without really installing the dependencies:

Running make for /home/zby/progs/ravlog/.
Prepending blib/arch and blib/lib of 108 build dirs to PERL5LIB; for 'get'

"cpan -t . " works in the same way and if the tests fail then "cpan ." stops there - so I cannot tell if it would install the dependencies in the case when the tests pass.

I don't really know how to investigate it further - but I am noting that as it is very frustrating. Eventually I had to resort to cpanminus ("cpanm --installdeps ." is really handy as it addresses the problem directly).

Implementing file upload progress bar

At work I was assigned the task of replacing the previous hodgepodge of tools that provide the progress bar functionality for forms with file uploads. At first glance this seems like a trivial thing to do - you periodically observe how much of the file you've got and update the progress bar, but there are details that make it hard to do with the current tools. I report here how I solved that problem - not because I think this is the optimal way - but rather to open the discussion and because I could not find any description of solving it when I googled for it.

First of all the common programming tools (like CGI.pm or Mason that we use here) assume that the page handler receives the whole request as input - and that whole request is not available until after the file is uploaded. So for example 'my $q = CGI.pm->new' will not finish until it is too late to measure the upload progress. The solution to that is to use another page to report the upload progress and call that page via Ajax from Javascript code updating the progress bar. This would work great - but the file is normally uploaded to a temporary file with a random name and the other script would not have any chance to guess it. We need to generate a new random file name in the form page and then pass that name to the form handler script so that it would save the data to that file, and in parallel to the Ajax scripts that would check the size of that file.


To save the data into a specified filename I used the CGI.pm callback feature:

my $q = CGI->new( \&hook, $fh, undef );
...
sub hook {
my ($filename, $buffer, $bytes_read, $fh) = @_;
print $fh substr($buffer, 0, $bytes_read);
$fh->flush();
}

It is described in the subsection called "Progress bars for file uploads and avoiding temp files" of the CGI.pm documentaion, but actually it is a great leap of thought to say that it supports progress bar implementation, you still cannot use it directly to get the progress bar from the CGI object on the form landing page, you still need the separate scripts measuring the progress. For my solution all I needed was to pass the target file name to the code saving the data, this could be easier than writing this callback above. And the callback is still not everything - I yet need a way to pass the generated filename from the form page to that script - and not via form parameters, remember they are not available at that stage. So how can that be done? Simple - as PATH_INFO - which is available in the %ENV hash even before the params are parsed by CGI.pm.

This is the skeleton of the solution - there are a few more details in the actual implementation - but the code will be published soon as Open Source - so I hope everyone will be able to look them up there.

When those micro-seconds matter

Catalyst is the only web framework that has a mailing list I subscribe to - but I am sure that it happens at others too. In a recurring pattern someone someone posts a benchmark showing that Catalyst for some trivial operation is many times (or many hundred times) slower than some other web framework or for that matter PHP. That does not fail to generate a heated debate - but eventually the seasoned framework developers gain the upper hand with the argument that for all the, often big, web sites they worked on, those few micro-seconds lost in the Catalyst dispatcher never mattered much because the application spent hundred times more in other code fragments and mostly in business logic parts, so shaving off some part of the few micro-seconds would not improve the overall speed more than 1%. This is a great argument, perfectly reasonable and rational but it is biased towards the status quo. It might be true that everywhere where Catalyst is currently used it works great but it is also not hard to imagine an application with very simple business logic that needs to serve millions of users, Twitter anyone? Or an app that does many simple Ajax callbacks. Sure you can always code the speed requiring, simple parts in PHP and keep Catalyst only for the other more heavy-weight tasks but having a universal solution would be so much more convenient.

Installing dependencies for your packaged for CPAN library

Update: See the comments, apparently the warning against auto_install is outdated information. I don't have any opinion of myself here.

It always puzzled me what is the 'canonical' way to install dependencies for an unpacked distribution, be it downloaded from CPAN or your own in-house product packaged the CPAN way for convenience. Module::Install provides an 'auto_install' option that you add to your Makefile.PL and then perl Makefile.PL; make (or perl Makefile.PL; sudo make) should do that trick. This seemed like a good solution that could be a standard, but now I see that the FAQ of that module says:

Do I really have to avoid auto_install()?

auto_install() has a long history of breaking CPAN toolchains. Lots of people had a bad feeling on it, and have said it should be strongly avoided. In fact it was deprecated and removed once.

Although most of the known problems have been fixed and you can use it more safely than ever, the use of auto_install() is still discouraged, especially if what you are writing is a module to be uploaded on the CPAN. auto_install() does lots of things itself, thus does not always do the same things as other toolchains do (including extra attribute handling, etc; which can be fixed somehow but that's not too DRY). It only supports CPAN and CPANPLUS as its backends. If you use other tools to install, it may still cause a trouble.

Besides, now you can do what auto_install() does with other means. If your CPAN module is new enough, you can pass a dot to the cpan command it provides, and it will install all the required distributions from the CPAN:

$ cpan .

So apparently this is the way to go - using the cpan shell. There is a minor problem with this when used for testing - it would automatically install the package from the directory you are in. Sure you can suppress that you need by adding another command switch (-m), but it would be better if the default action was the safe one.