Constraints and Types (in Moose)

Moose::Manual::Type says that Moose types are mostly just constraints with a name. When designing new constraints API for Formhandler we decided to use Moose types - but with some playing around it also exposed the fact that not all constraints are types. I am not an expert in Type Theory - but I checked wikipedia and it seems to support my intuition that type is an attribute of data value - while when validating we often need to check not just the value - but also it's relation to the whole system.

If that is not yet clear - let's take the example of checking the age of a application user (from his provided birth date). The question if a user with a given birth date can legally participate in a web forum cannot be answered without knowing what is current date - it is not a property of the birth date alone - but of it's relation to the current date. I think that using the term 'type' for this kind of constraints can be confusing, even though technically in Moose this is possible to declare this.

Roles for FormHandler

This is a more concrete version of the question from my last post. This is a form defined in FormHandler:

package BookDB::Form::BookWithOwner;

use HTML::FormHandler::Moose;
extends 'HTML::FormHandler::Model::DBIC';

has '+item_class' => ( default => 'Author' );

has_field 'title' => ( type => 'Text', required => 1 );
has_field 'publisher' => ( type => 'Text', required => 1 );
has_field 'owner' => ( type => 'BookDB::Form::BookOwner' );

This should be quite self-explanatory for anyone working with forms (and DBIC).
Now - I think Model::DBIC should be just a role here, but that is just a side track. The real question is about the fields here - ideally I would want that doing:

with 'HTML::FormHandler::Model::DBIC';

(this is after it is made a Role) - would add appriopriate methods to the fields - an example here could be validation of fields that need a unique value. I would like to not need to change the definitions above to:

has_field 'title' => ( type => 'Text::DBIC', required => 1 );

Fields like 'owner' above which are Compound fields and work mostly like a form itself and need to load the options for SELECT boxes from the model, set the values of fields from a database row etc:

package BookDB::Form::BookOwner;

use HTML::FormHandler::Moose;
extends 'HTML::FormHandler::Field::Compound';
with 'HTML::FormHandler::Model::DBIC';

has '+item_class' => ( default => 'User' );

has_field 'user_name';
has_field 'fav_cat' => ( label => 'Favorite Book Category' );
has_field 'fav_book' => ( label => 'Favorite Book' );
has_field 'occupation';
has_field 'country' => ( type => 'Select' );


Ideally this definition should not need 'HTML::FormHandler::Model::DBIC' - applying this role to the root form should be enough.

Roles and data structures

Let's say you have a tree like data structure, something not exactly so uniform as a tree but with enough uniformity that it makes sense that large part of the code is recursive. My example is something representing an HTML Form for editing a complex data structure (with sub-forms etc.). The main form code is used for inflating external data represetntation (key, value pairs) into internal objects and validating it. But there are two additional 'aspects' of this form processing - this is generating the HTML of the form (with all the error indicators and messages) and saving the form data into the data store (by manipulating DBIx::Class schema for example). So you have a tree-like data structure and three separate functionality areas related to it. How do you structure the code?

You can start with defining the data structure and one of the aspects as the 'primary package' - and then add the other aspects as (Moose) roles. This is how it is started in FormHandler - the primary aspect here is the data processing and validation - then saving the the database is added as a role (so you can use it with different data store libraries DBIx::Class, Class::DBI and in the future all the other), the same with rendering - it is a role added to the form class. This is all nice untill you realize that in this way you add the additional roles only to the main form object - but not to the fields (i.e. nodes in the tree) - so you can walk the fields and do the stuff - but you regress here to mostly procedural code.

What we need is to somehow add this role globally to all the field and form classes used.

Hierarchy of code reuse

Hierarchy of code reuse by flexibility:

• Code generation and cut and paste: unconstrained modifications
  
• Inheritance and passing callbacks: modifications only at the method granularity
  
• Normal libraries: no modification of existing algorithms - only composing them into larger constructions
  

Reading "Design Principles and Design Patterns"

Design Patterns are an intriguing idea - there are some attempts to port it to Perl, but I would not call them too sucessfull in spreading it. Design Principles and Design Patterns by Robert C. Martin tries a higher level view on the design patterns by defining some general rules about dependencies and abstraction and then showing how the patterns follow from them. This perhaps should be easier to translate into dynamic languages.

I think the real takeaway is "Stable packages should be abstract packages" - where stable for the author means 'many other packages depend on that one'. The problem with translating that to dynamic languages is that the most abstract (in the OO sense) packages, the pure interfaces don't need to be written down in code at all. Duck typing gives us a great flexibility - but it also means that there is no requirement to write down the abstract interfaces - and if there is no additional pressure then they will not be defined by the programmers.

Another problem with translating the ideas into dynamic languages is the meaning of the word 'depend'. With static typing it can be clearly defined by the need to recompile the package if there is some change in the package it depends on. For dynamic typing this needs to be defined in some more subtle ways - basing it on the type of the coupling between the packages.

Open Source fragmentation and cognitive biases

I've just finished Predictably Irrational (it was nice, many of the of the theses I already knew from Internet articles - so it was not ground-breaking for me, but still I learned from it). In the seventh chapter of this book entitled "The High Price of Ownership" the author presents some surprising stories about how we "overvalue what we have". Apparently we all give much more value to things that we posses then to those that we could potentially have - this is a common cognitive bias.

This made me thinking. Wouldn't that explain the fragmentation of Open Source libraries (and CPAN in particular)? Would that not explain why one author values the miniature efficiency gains of his reinvented wheel over the robustness of time-tested code?

Maybe once we know how this mechanis works, we could design better Open Source library repositories, limitting the power of that fragmentation drive? Maybe by being more explicite about our choices we can become more rational?

Catalyst helpers - plugins for Module::Starter?

Code generation is hard. First of all it is hard to determine what really needs to be created - in (computing science) theory you can always uncurry the generator and use it as a library. So you need different theory to really determine if code generation makes sense at all. In practice many people use it - since it let's us start new projects faster (and more correctly). This is an interesting subject deserving many more blog posts - but for now I just wanted to toss the idea that joining forces with other Perl projects facing this problem could help us improve the mess that is Catalyst helpers now.

I've learned about Module::Starter plugins from Recursive development that leads nowhere.

Progress

Now that I use CGI::Application, Class::DBI and HTML::Template at work I can see the advantages of using Catalyst, DBIx::Class and Template Tookit. Personally, I find using HTML::Template as the most annoying of those three set backs - with TT I've got used that I don't need to worry about displaying the data I feed to the template, I need to make sure that it contains everything needed but that's all. I know that the view can take any data and display it. This makes a nice separation of work and of mind sets between the Controller and the View. Not so with HTML::Template.

One gotcha less!

Looks like one of the anomalies I described in my previous post is fixed now in the svn (see Revision 4773 of DBIC). Thumbs up!

DBIx::Class gotchas

DBIC is a complex beast - and it has it's rather obscure corners where it can burn the inexperienced. Here are two of those, comments with others are welcome.

Don't use $rs->update_or_create(...) (or $rs->find_or_create) on a table with autoincrement primary key

The ->find method requires that all the columns comprising the key used for the search are present in the query or in the resultsource conditions. Otherwise it will return some random record (and emit a warning). This is because currently there is no way to determine if all the needed columns are present. To do this you would have to parse the arbitrary SQL::Abstract query that can be in the resultsource condition and check how the columns are used there. Think for example about:
$new_rs = $cd_rs->search( { -and => [ id => [1, 2], id => { '>' => 1 } ] } )
and then $new_rs->update_or_create( { artist => 1 } ).

But for autoincrement primary keys the ->create method requires that the primary key is not present in the query. If you provide { pk => 'undef' } to ->create, this would generate a query like: INSERT INTO cd ( pk, ...) VALUES ( NULL, ... ) - some databases would accept that but others (PostgreSQL for example) not.

So in summary, for tables with autoincrement primary keys ->find requires that you provide the primary key in the query, and ->create requires that you don't - this means that you cannot pass the same query hash to both methods and this is what ->update_or_create does.

Update: Looks like the 4773 revision fixes the problem described below. That's why I moved this point to the end of the post. I did not check it too deeply though.

Don't call $row->some_relation when the $row has not loaded the columns required to resolve some_relation

Column values in DBIC are kept in a hash - this means that a column can have some defined value, have undefined value or can be completely non-existent - and DBIC interprets all of those states differently. When $row->some_relation is called and ->some_relation uses a foreign key column that is non-existent then the result is currently random (and in the future most probably an exception will be raised).

When this can happen? When you create a row and don't provide the value for the column (or if you retrieve the row from the database and explicitelly omit the column). For example if you do
$cd = $cd_rs->create( { name => "Cisza" } )
(or $cd = $cd_rs->new...). Now the 'artist' column in $cd is non-existent and when you try to call $cd->artist it blows up.
In a library where you don't know how a record was created you need to check first if the relevant columns are loaded before you call a relation on the record.

I thought that a good remedy would be also to always provide undef in the ->create and ->new calls for all the columns that we don't have values for, but allegedly this is not a good strategy (don't ask me why).

Installing Test::WWW::Mechanize::Catalyst

It seems that there is some incompatibility between the latest LWP version (5.812) and Test::WWW::Mechanize::Catalyst. The tests die with an:

HTTP::Message content must be bytes at lib/Test/WWW/Mechanize/Catalyst.pm line 88

error message. After downgrading LWP to 5.808 Test::WWW::Mechanize::Catalyst (version 41 - which looks better than 42 at CPAN Testers: Reports for Test-WWW-Mechanize-Catalyst) worked. Looks like this change is the culprit: "Don't allow HTTP::Message content to be set to Unicode strings".

confound - thanks for the hint.

Interesting articles from the Ruby community

Why coupling is always bad / Cohesion vs. coupling and Why Rails is total overkill and why I love Rack - looks like the Catalyst philosophy is quite close to that of the Rack project (but maybe less articulated).

As a side-note - I cannot comment there (even though I have javascript turned on) - do you get the same problem?

Rose::HTML::Form and DBIx::Class

Has anyone tried that combination?

Writing Facebook apps in Catalyst

A few weeks ago I've started a howto page for writing Facebook applications in Catalyst at the Catalyst wiki. Now I have accumulated there a few tips. Everyone using Catalyst for Facebook apps is invited to add their problems and solutions.

Refreshing a row from the DB - another note to myself

This is another thing that I keep forgetting - how to refresh a row with values from the database.
So here it is:

$row->discard_changes;

It comes from DBIx::Class::PK (and not from DBIx::Class::Row where I usually start the serach).

Update: The core devs say that the new version of documentation will mention discard_changes in DBIx::Class::Row POD. Thanks.

Automatic DBIC schema creation from a database

This is something that I keep forgetting and then I have to wade through the docs again. And there are traps in the docs - for example SQLT can do the schema creation - but it does not build the relationships.

So here is the magic invocation for DBIx::Class::Schema::Loader which do create the relationships (if you use the right switch):

perl -MDBIx::Class::Schema::Loader=make_schema_at,dump_to_dir:./lib -e make_schema_at("New::Schema::Name", { relationships => 1 }, [ "dbi:Pg:dbname=foo","postgres" ])’


A free implementation of Amazon SimpleDB

Here is a tip to Amazon - there should be a free implementation of SimpleDB. I am now considering using SimpleDB for a web application - but I would like to develop it on my desktop computer - without paying Amazon, without even connecting to Amazon. This might be a minor issue - the fee you would pay for developing the application on Amazon web services seem to be very small. But it would be so much more convenient: no fiddling with credit cards, no problems for developers with shaky internet connection and more compatibility with Free Software/Open Source development models where money is always a difficult subject.

For Amazon it would not mean much less income - since as I've said the fee for the development use of the services is rather small - but it would mean many more customers. Perhaps it would also mean some competition - but this would only validate the whole model and make it more suitable for big business which does not like so much dependence on one provider.

Update to 'Tags and search and DBIx::Class'

Advanced Search in web DBIx::Class based applications (with tags, full text search and searching by location) is an update to Tags and search and DBIx::Class.

I think it is a nice technique.

Have you ever forgotten where some subroutine comes from?

Or have you ever been learning a big stack of modules, like DBIx::Class or Catalyst and stumble from time to time over a new method name and you did not know where to look for it's documentation? I have once even wrote a script that generated an index of all methods documented in some code directory tree. It worked for me when learning DBIx::Class - but you needed to be very careful when choosing what you are indexing - you indexed too much and the list just got too long, you indexed too little and it's coverage was too limited.

Until discovering Pod::POM::Web I did not know that what I need is actually a full text search over all the Pod documents - you cut and paste the unknown method name into the search box press Enter and voila you have list of modules that document that method. And if you know that it comes from one of the DBIx::Class submodules you just add DBIx::Class to the search criteria - and don't get all the Class::DBI documentation pages. No problem with too long index lists. Someone should hack it to run against a full CPAN mirror and beat the other CPAN search websites.

Catalyst::Example::InstantCRUD updated to newer versions of underlying libraries

Version 0.0.20 released to CPAN. My plan is to port it to use the new HTML::FormFu instead of HTML::Widget and then perhaps make a web wizard to edit the config options.