Time Nick Message 14:46 paul kados around ? 14:48 kados paul: yep :-) 14:48 kados what's up? 14:48 paul hello kados. you're on your computer even on sunday afterneoon ? 14:49 kados yea, took yesterday off completely 14:49 paul tomorrow, I have a day off from the web. 14:49 kados but I had to write up my discoveries with XML parsers 14:49 paul but i'll have many time in the TGV & in a room, waiting for my meeting time 14:49 kados great 14:49 paul so, i have time to work on koha. 14:49 kados w00t! 14:50 paul any idea of what could be interesting ? 14:50 kados hmmm ... 14:50 paul s/idea/suggestion/ 14:50 kados well with no internet it might be tough to work on zebra 14:50 kados as it's quite difficult to get running 14:50 kados and only tumer and I have done it :/ 14:50 kados (and only me with tumer's help) :-) 14:51 kados what we really need to do 14:51 kados is resolve how we will handle versioning 14:51 kados and once we decide, just stick to the plan and finish up 14:52 kados paul: btw: I have just sent a mail to koha-devel about XML parsers 14:52 kados paul: may be interesting to you 14:53 kados too bad you'll not be able to make the meeting Monday 14:53 kados one of the things I wanted to discuss was how how to handle versioning given where we're at 14:54 kados paul: still there? 14:54 paul another important point about tomorrow is that i'll have a meeting with Ineo (jp sakoun + pierrick in the morning) 14:54 paul yest, i'm here, of course. 14:55 paul nothing in my mailbox (koha-devel) yet 14:56 kados what will the ineo meeting be about? 15:12 paul so, what kind of code cleaning could I work on tomorrow ? 15:12 kados heh :-) 15:12 paul you told me you already did some ? 15:12 paul I could, for example : 15:12 kados well, I created Record.pm 15:12 paul - move "bull" to "serials" 15:12 kados yea, that'd be good 15:12 paul - clean all unused subs in .pm 15:12 kados if you agree, we should put all record related subs in Record.pm 15:13 paul for cataloguing, right ? 15:13 kados so you could start that process, and test Record.pm stuff also 15:13 kados Record.pm is for anything dealing with records, MARC, MARCXML, XML 15:13 kados so it could be used to have an export function for instance 15:13 kados on the OPAC 15:14 paul on my laptop, I don't have uptodate perl package i'm afraid. 15:14 kados or it could be used for cataloguing 15:14 paul what do I have to download before quitting the net ? 15:14 kados yes, important question 15:14 kados 1. latest MARC::File::XML (from CPAN) 15:14 kados 2. latest MARC::Record (from SF) 15:14 kados 3. latest MARC::Charset (from CPAN) 15:15 kados 4. libxml2 and a compatible version of XML::SAX::LibXML (and use LibXML as the default system parser) 15:15 kados (#4 is detailed in my mail to koha-devel and perl4lib, sent about 20 minutes ago_ 15:15 kados ) 15:16 paul another information, opencataloger has been commited here :https://gna.org/projects/opencataloger/. Antoine still working on it, of course, and we plan to release some doc this week 15:16 paul still no mail on koha-devel 15:16 kados that's great! 15:16 paul BUT : tumer did some commits 10mn ago ! 15:17 kados paul: I forwarded it to you 15:17 kados paul: another interesting thing you could do 15:17 kados paul: is to test Mike Rylander's new MARC::File::XML stuff (on SourceForge) for handling UNIMARC records 15:17 kados paul: I have just recently (Friday) resolved all (I think) of the problems with MARC21 records 15:18 kados paul: and haven't had a chance to test UNIMARC (with ENM's data) yet 15:18 kados paul: you could also modify the new command-line export.pl script to support UNIMARC 15:18 kados paul: committed to rel_2_2 15:19 kados tumer is committing his version of Koha to dev-week as I requested him to 15:19 kados also looks like he's updating head at the same time 15:19 kados tumer++ 15:21 paul no, tumer +++++ 15:22 kados hehe 15:24 paul X going mad... 15:26 kados with libxml2? 15:27 paul marc::charset is version 0.95, right ? 15:28 kados yep, that'll work 15:29 paul about libxml2, what could I use to check that libxml2 is here ? 15:29 kados hmmm ... not sure 15:29 kados what distro are you running? 15:29 paul mandriva 2006 15:29 kados maybe check the packages and see if libxml2 is there? 15:29 paul urpmi libxml2 says "all packages intalled" 15:30 kados cool 15:30 kados now ... very important 15:30 kados XML::SAX::LibXML must be the correct version for that version of libxml2 15:30 kados or else it will create problems 15:30 kados also, you need to make sure that LibXML is the default system parser 15:32 paul I have a Perl-XML-SAX 15:32 paul package, but nothing related to libxml2 it seems 15:32 kados XML::LibXML::SAX is the package I think 15:33 kados akk! 15:34 paul is it XML::LibXML::SAX - XML::LibXML direct SAX parser ? 15:34 paul http://search.cpan.org/~phish/XML-LibXML-1.58/lib/XML/LibXML/SAX.pod 15:36 kados yes, that's it 15:36 kados but ... 15:36 kados http://search.cpan.org/src/PHISH/XML-LibXML-1.58/README 15:36 kados you need to make sure your versions are compatible 15:37 kados see the section: 15:37 kados Notes On libxml2 Versions 15:38 paul The installed version of libxml2 was not tested with this version of XML::LibXML. 15:39 kados right 15:39 kados so ... what you should do 15:39 kados is test for the problem 15:39 paul make test => 0% passed :( 15:39 kados :( 15:40 kados yea, this is a tricky bit of software 15:40 kados does Mandrake have a perl package for libxml? 15:40 kados Perl-LibXML-SAX or something? 15:40 paul mmm... too dangerous to work on this without web access i'm aafraid. 15:41 paul yep 15:41 kados well i assume the package maintainer's versions will work together 15:41 kados so if you use that it should work 15:41 paul perl-XML-SAX version 0.12 15:42 kados no, you need LibXML I think 15:42 kados hang on ... try this: 15:42 kados #!/usr/bin/perl 15:42 kados use XML::SAX::ParserFactory; 15:42 kados $parser = XML::SAX::ParserFactory->parser(); 15:42 kados print $parser; 15:42 kados what does it tell you? 15:43 kados I get: 15:43 kados XML::LibXML::SAX=HASH(0x81f83d8) 15:43 paul XML::SAX::PurePerl=HASH(0x80e3734) 15:45 kados yea ... so with PurePerl you can't handle any combining characters 15:45 kados which is why many french utf-8 chars will fail 15:45 kados I filed a bug for that 15:46 kados http://rt.cpan.org/Public/Bug/Display.html?id=19543 15:47 kados changing your parser from PurePerl to Expat will solve 90% of the errors 15:47 kados this SAX stuff is really complicated 15:47 kados and annoying ! 15:48 paul OK, time to go to bed. 15:49 paul bye joshua, have a good end of sunday. 15:49 paul and see you on tuesday ! 15:50 kados sounds good 15:50 kados ciao 15:59 kados arrrg! 18:10 kados chris: you got a second? 18:10 kados working on Record.pm 18:10 kados error handling 18:10 chris yep 18:11 kados i was thinking of something like this: 18:11 kados sub marcxml2marc { 18:11 kados my ($marcxml,$encoding,$flavour) = @_; 18:11 kados unless($encoding) {$encoding = "UTF-8"}; # set the default encoding 18:11 kados unless($flavour) {$flavour = C4::Context->preference("marcflavour")}; # set the default MARC flavour 18:11 kados #do the conversion 18:11 kados my $marc = $marcxml->new_from_xml($marcxml,$encoding,$flavour); 18:11 kados # check the record for errors 18:11 kados my $needs_fix = _check_marc_warnings($marc); 18:11 kados if ($needs_fix) { 18:11 kados $marc = _fix_marc($marc); 18:11 kados } 18:11 kados return $marc; 18:11 kados } 18:11 kados whadya think? 18:12 kados or should the check/fix be in the same call? 18:12 chris hmm no 18:12 chris i like them seperate 18:12 kados cool, I'll write it that way then 18:12 chris the fix might fail 18:12 chris so error handle that too 18:13 kados good point 18:13 chris ie, if it cant fix it, it should return some kinda error 18:13 kados yea 18:13 chris rather than the $marc 18:13 kados should it die? 18:13 chris so we can at least report some sanity 18:13 chris i wouldnt make it die 18:13 kados how do you signal that there's an error? 18:13 chris id return an error then the calling program can decide how to deal with it 18:13 chris what id do 18:14 kados how would the calling prog know it was an error? 18:14 chris return ($error,$marc); 18:14 chris and you call it 18:14 chris my ($error,$marc)=marcxml2marc(...); 18:14 kados cool 18:14 kados $error first? 18:14 chris if (!$error){ 18:14 chris } 18:15 chris either or, doesnt really matter, the convention is usually error first tho 18:15 kados cool, thanks 18:16 chris np 18:16 chris we should get in the habit of doing this for all our routines 18:17 kados yea 18:20 kados chris: so is $error just a success/failure kinda deal, or does it actually contain data? 18:20 kados cause I'm thinking there might be more than just one problem with the record 18:20 kados (in fact, it's likely) 18:20 chris sup to you 18:20 chris id make it contain a useful error string 18:20 chris or it could contain an integer 18:20 chris and the man page for Record.pm could explain what the numbers mean 18:21 kados one more thing 18:21 chris and then the template could handle translation easier 18:21 kados I'd like to build a test suite for Record.pm 18:21 chris right 18:21 kados but I"m not sure how that should be done 18:21 chris ok 18:21 chris you wanna start off with a record_test.pl 18:22 chris or something like that 18:22 kados in C4? 18:22 chris id make a dir in C4 called tests 18:22 chris and put it in there 18:23 kados k ... so no need for a Test.pm? 18:23 chris nope 18:23 kados k, I'll get to work, thx! 18:23 chris something like this 18:24 chris #!/usr/bin/perl 18:24 chris use Test; 18:24 chris BEGIN { plan tests => 1 }; 18:24 chris use C4::Record; 18:24 chris ok(1); 18:24 kados ? 18:24 chris then you just increase the plan tests .. and add more test 18:24 kados use Test? 18:25 chris yes 18:25 chris its a standard perl module 18:25 kados ahh 18:25 chris perl Biblio_tests.pl 18:25 chris 1..1 18:25 chris # Running under perl version 5.008004 for linux 18:25 chris # Current time local: Mon May 29 11:25:17 2006 18:25 chris # Current time GMT: Sun May 28 23:25:17 2006 18:25 chris # Using Test.pm version 1.24 18:26 chris ok 1 18:26 chris thats my simple test for C4::Biblio .. its only checking syntax errors for now 18:26 kados ahh, cool 18:26 chris there are lots of other testing subroutines 18:27 chris http://search.cpan.org/~mschwern/Test-Simple-0.62/lib/Test/Simple.pm 18:27 chris test::simple is supposed to be good too 18:27 chris actually that will do more useful tests i think 18:28 kados I'll read the tutorial 18:28 kados thx 18:46 kados chris: and it's OK to open external MARC files right? 18:46 chris what? 18:46 kados well ... in the test I'm going to need to actually run some records through to test if it's working 18:46 kados I think 18:47 kados ok (my $marcxml_from_marc = marc2marcxml($marc_marc8,'UTF-8','MARC21'), 'marc2marcxml from MARC-8 MARC21'); 18:47 kados where $marc_marc8 was opened as an external file 18:48 chris yep thats fine as long as you document where the files should be to run the test 19:15 chris heh, ed pulls the classic move 19:15 chris good idea, you do it :) 19:19 kados hehe 19:19 kados yea, and I'm gonna too :-) 19:20 chris that'll learn em 19:20 kados :-) 19:20 kados got my first test working 19:20 kados now I just need to create a bunch of test files 19:21 chris sweet 19:21 kados every record I've had trouble with 19:21 chris good plan 20:35 kados chris: quick question 20:35 kados chris: I want to provide a flag for Record.pm 20:35 kados chris: to turn on entity encoding for exported XML records 20:35 kados (I think) 20:36 kados or should I just provide that as a flag for each call to an XML generating function 20:36 kados ? 20:42 chris umm im not sure what you are asking 20:43 chris ahh i think i get it 20:44 chris it might be better on a per call basis 20:44 chris is it conceivable you might want to encode one record but not another? 20:45 kados yea I suppose so 20:45 kados k, I'll do it per call 21:03 kados chris: is it worth looking into full-fledged exception handling as per: 21:03 kados http://www.perl.com/pub/a/2002/11/14/exception.html 21:03 kados ? 21:03 kados or is that overkill? 21:06 kados like should I be just wrapping calls like as_xml_record() in eval {};? 21:08 kados I'm thinking it might be good to check for errors more than once in the marc2marcxml sub 21:08 chris yeah using eval is always a good thing, when its routine outside 21:08 chris you can never have too much error checking .. well you can 21:08 kados :-) 21:08 chris but it rarely happens :) 21:09 chris you and I have both experienced the pain bad error handling causes 21:10 chris ie, if the call to the xml has bad wrapped in an eval, one bad record wouldnt bust a whole batch load etc 21:11 chris trapping and reporting errors 21:11 chris such that the calling program can decide what to do with them 21:11 chris is much nicer than dying in a heap and giving the calling program no option 21:11 kados yep 21:12 kados I think with eval I can even use MARC::*'s internal warnings() 21:12 kados ie, I can print the warning message as delivered from that 21:12 kados something like this: 21:12 kados eval { $marc_record = MARC::Record->new_from_usmarc($marc) }; 21:12 kados if ($@) { 21:12 kados my @warnings = $@->warnings(); 21:14 chris yeah sounds good 03:04 hdl salut Toins 03:05 ToinS salut hdl 08:59 kados morning #koha 08:59 kados si: can you oper me when you get a chance? 09:00 hdl hi kados 09:01 hdl Do you know if tumer commited some code about authorities ? 09:05 kados yes, I believe he did 09:06 kados yep, stuff committed to dev-week 09:07 kados I asked tumer to commit all his customizations to dev-week 09:07 kados until we resolve our versioning dispute 09:07 kados (which I was hoping to do tonight, but paul won't be present) 09:17 kados hey alaurin, welcome to #koha 09:18 alaurin HI