warszawa.pm

Witam,

Co prawda zwykle piszę tutaj w języku angielskim - ale reklamówkę naszej grupy Perl Mongers postanowiłem napisać po polsku, żeby było widać, że u nas też się coś dzieje. Mamy już swoją wiki: http://warszawa.pm.org/. Jeszcze nie wszystko działa, ale miejmy nadzieję, że administrator sie postara i naprawi. Żeby dostać konto na wiki trzeba poprosić na liście dyskusyjnej, niestety ze względu na spam nie mogliśmy wiki zrobić zupełnie otwartej. Zapraszamy na wiki, listę i spotkania.

Optional Dependencies Are Going Out of Fashion

I was reading the Changes document for the latest DBIC release and I spotted:

- Remove the recommends from Makefile.PL, DBIx::Class is not
supposed to have optional dependencies. ever.

I am noting this so that it will not missed by the general public that optional dependencies are going out of fashion.

The problem here is that if an optional dependency enables some useful feature (and otherwise why would you add it?) - then you should expect that someone writes a library using this feature and uploads that library to CPAN. What that other guy should add to his dependencies? How is he supposed to know which of your optional prerequisites are needed for his module? And what if this is not about just two modules in dependency relation but rather a whole chain of them? Then it becomes a mess.

How they do it in Python (core libraries).

I don't read much from the Python blogosphere - so maybe it is not representative at all - but it seems that Python 'batteries included' approach to core libraries is also approaching its limits. I am sure that debate will happen in any popular Open Source language after reaching some maturity level. It is interesting what kind of compromise is proposed by the author of that piece:

I would first prune the standard library down to a more core focus of libraries that require cross-platform expertise, using widely accepted standards, and stuff needed to simply get a script to run. In terms of cross-platform stuff, that would mean os, multiprocess, etc. These are modules that a typical developer would have a hell of a time getting working on all the OSs that CPython runs on.

How to install Catalyst::Authentication::Credential::OpenID

This post is meant to be both a quick instruction and another data point for the CPAN installation problems thread of my blog posts.

For those impatient the winning sequence of commands (for Debian/Ubuntu) is:

sudo apt-get install libgmp3-dev
cpanp -i Math::BigInt::GMP

and now you can:

cpanp -i --force Catalyst::Authentication::Credential::OpenID


Now the story - at my pristine installation of Perl 5.10.1 on my Ubuntu box I discovered two problems with C:A:C:O installation. First was a trivial POD error, detected with the new pod tests - this is why you need to do that --force (it is fixed in the dev version). The second is more interesting - installing one of the prerequisites - Crypt::DH seemed to take forever. Fortunately by chance I found a good explanation of the problem in Crypt::DH::GMP docs - apparently for any reasonable performance Crypt::DH needs Math::BigInt::GMP which is not in the prerequisites. Without it it still works - but it can take serveral hours to pass the tests (reported first 3 years ago). So next thing is install Math::BigInt::GMP - unfortunately this is not automatic either - you need to install gmp.h header file first (on Debian/Ubuntu it is the libgmp3-dev package). This is why you need to execute the first two commands.

I hope that will help someone.

Change Management and FOSS authors obligations

The two recent controversies making rounds in the Perl blogging community made me thinking about what obligations FOSS authors have to those using their code. The traditional answer is none, but on the other hand there are some customs to the contrary - like maintaining API compatibility to at least one version back. Some people have expectation that software not marked upfront as 'experimental' will follow that custom. This is a good custom, because complete freedom for the authors can mean too much chaos for those using their code, but it is both not general enough and too radical in rejecting the traditional no obligations rule. It is not enough - because not all API changes can be made with a backcompat layer, and as shows the example of Dave porting Array::Compare to Moose - there are also non API changes that can make downstream software unusable. And it is too radical because of being an implicit, 'by default' rule. And it is also too ambiguous in it's formulation because it does not state when the new releases can be published. What we need is more general and explicite policies.

Unexpectedly changing a library's API can be like pulling a rug from under the feet of authors of software using that library. In the past that was a rare event - and usually there was time to warn all the parties involved and let them fork the project if needed. Now in CPAN we have such huge dependency tries that even if it is still rare for the individual leaf nodes - it becomes quite frequent for the root nodes where all the changes cascade from the leaves. These deep dependency hierarchies also makes forking a lot more problematic as it can mean forking not just one library - but a whole path of them leading from the leaf up to the root. CPAN might be still rather unique in it's size among distributed library projects - but with the overall exponential growth of FOSS software we can expect that to happen in other projects soon. That's why if we want to continue the process of lego-like assembling software components out of more basic libraries we need those libraries to adhere to some change management policies.

I am inviting FOSS authors to formulate their change management policies (and in particular API deprecation policies) and voluntarily add them to the manuals. I want to stress it - it is not about a top down rule forcing authors to do something - this is an invitation to voluntarily give up a bit of the authors freedom in leading the project in order to make it more useful. The bulk of FOSS can still be published as a gift to the humanity with no strings attached. I just want to show that additional option and also create some pre-canned policies that could be cut and pasted into the documentation just like the FOSS licences are.

What really fails in installation?

I've finally installed Perl 5.10.1 and I had to reinstall most of the modules I use. I set PERL_MM_USE_DEFAULT=1 and did make for a module that requires half of the modules on my computer. Everything was installed automatically - with no additional intervention from me. The morale of this is that if you are using Ubuntu and if you have installed all the external prerequisites - then the installation is really rather automatic. The problem with systems other than a special flavour of Linux is both hard to solve and rather unsophisticated - we just need more people working on those systems and reporting and solving bugs. Or at least that is the solution I can imagine. But the problem with external prerequisites is where we could do better with just a bit of programming.

I remove memcached from my system and try

zby@zby:~/progs/Enzyme$ cpanp i Cache::Memcached::Managed
Installing Cache::Memcached::Managed (0.20)
Running [/home/zby/local/bin/perl /usr/local/bin/cpanp-run-perl /home/zby/.cpanplus/5.10.1/build/Cache-Memcached-Managed-0.20/Makefile.PL]...
No executable memcached found: No such file or directory
Running [/usr/bin/make test]...
PERL_DL_NONLAZY=1 /home/zby/local/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
Can't exec "memcached": No such file or directory at /home/zby/.cpanplus/5.10.1/build/Cache-Memcached-Managed-0.20/blib/lib/Cache/Memcached/Managed.pm line 843.
#
# Stopped memcached server
# Looks like you planned 176 tests but ran 184.
t/001basic.t .....
Dubious, test returned 255 (wstat 65280, 0xff00)
All 176 subtests passed
(less 177 skipped subtests: -1 okay)
t/002inactive.t .. ok
t/003multi.t ..... ok
Can't exec "memcached": No such file or directory at /home/zby/.cpanplus/5.10.1/build/Cache-Memcached-Managed-0.20/blib/lib/Cache/Memcached/Managed.pm line 843.
#
# Stopped memcached server
t/010fork.t ...... ok
Can't exec "memcached": No such file or directory at /home/zby/.cpanplus/5.10.1/build/Cache-Memcached-Managed-0.20/blib/lib/Cache/Memcached/Managed.pm line 843.
#
# Stopped memcached server(s)
t/020grab.t ...... ok

Test Summary Report
-------------------
t/001basic.t (Wstat: 65280 Tests: 184 Failed: 8)
Failed tests: 177-184
Non-zero exit status: 255
Parse errors: Bad plan. You planned 176 tests but ran 184.
Files=5, Tests=36330, 18 wallclock secs ( 6.12 usr 0.03 sys + 3.86 cusr 0.20 csys = 10.21 CPU)
Result: FAIL
Failed 1/5 test programs. 8/36330 subtests failed.
make: *** [test_dynamic] Error 255
[ERROR] MAKE TEST failed: Bad file descriptor PERL_DL_NONLAZY=1 /home/zby/local/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
Can't exec "memcached": No such file or directory at /home/zby/.cpanplus/5.10.1/build/Cache-Memcached-Managed-0.20/blib/lib/Cache/Memcached/Managed.pm line 843.
#
# Stopped memcached server
# Looks like you planned 176 tests but ran 184.
t/001basic.t .....
Dubious, test returned 255 (wstat 65280, 0xff00)
All 176 subtests passed
(less 177 skipped subtests: -1 okay)
t/002inactive.t .. ok
t/003multi.t ..... ok
Can't exec "memcached": No such file or directory at /home/zby/.cpanplus/5.10.1/build/Cache-Memcached-Managed-0.20/blib/lib/Cache/Memcached/Managed.pm line 843.
#
# Stopped memcached server
t/010fork.t ...... ok
Can't exec "memcached": No such file or directory at /home/zby/.cpanplus/5.10.1/build/Cache-Memcached-Managed-0.20/blib/lib/Cache/Memcached/Managed.pm line 843.
#
# Stopped memcached server(s)
t/020grab.t ...... ok

Test Summary Report
-------------------
t/001basic.t (Wstat: 65280 Tests: 184 Failed: 8)
Failed tests: 177-184
Non-zero exit status: 255
Parse errors: Bad plan. You planned 176 tests but ran 184.
Files=5, Tests=36330, 18 wallclock secs ( 6.12 usr 0.03 sys + 3.86 cusr 0.20 csys = 10.21 CPU)
Result: FAIL
Failed 1/5 test programs. 8/36330 subtests failed.
make: *** [test_dynamic] Error 255



The tests for 'Cache::Memcached::Managed' failed. Would you like me to proceed anyway or should we abort?

Proceed anyway? [y/N]:
[ERROR] Unable to create a new distribution object for 'Cache::Memcached::Managed' -- cannot continue

*** Install log written to:
/home/zby/.cpanplus/install-logs/Cache-Memcached-Managed-0.20-1251915361.log

Error installing 'Cache::Memcached::Managed'
Problem installing one or more modules

zby@zby:~/progs/Enzyme$


Yeah - the information is there - but now imagine that this was a deep dependency tree with lot's of similar external prerequisites. You try to install the module - it fails, you view the log, locate the problem, install the external dependency, run the installation again and it fails again. Everytime you need to parse the really long installation log and try to understand what happened. It is not that complicated (although it can become complicated when there aren't so explicite error messages, or when there are other installation failures mixed together) - but it is intimidating and really time consuming. Definitively it is not something that a newcomer would be able to do. But external dependencies are not something extraordinary - they are rather standard, this procedure I described here is not exactly a failure mode - it is anticipated.

Someone could argue that when I install Cache::Memcached::Managed I should know that it needs the memcached binary - but it is only obvious for those that know that module, and if that module is somewhere deep in the dependency tree you might not even realize that you are installing it.

Now imagine this:

zby@zby:~/progs/Enzyme$ cpanp i Some::Module
Computing prerequisites.
You have unmet external prerequisites:
memcached
xslt version 1.1.18
...
Please install the prerequisites first then run the installation again.

wouldn't that be helpful?

PS. How about

requires_external memcached => sub { -x "memcached" }

in Makefile.PL?