Time Nick Message 09:02 kados thd: addbiblio? 09:02 kados thd: which import? bulkmarcimport? 07:04 thd kados: record is truncated on import if quotations are included in the MARC record 00:57 chris night 00:54 kados chris: read you tomorrow 00:54 kados chris: thanks for all the help 00:53 kados 1.x doesn't calculate directory offsets correctly 00:52 kados for the utf-8 stuff 00:52 kados cause we really need 2.0 00:52 kados I'll email perl4lib 00:52 kados and if it looks like MARC::Record's the culprit 00:52 kados when I'm awake 00:52 kados so maybe some more testing tomorrow 00:51 kados well I'm mad tired 00:51 kados arrrg 00:51 kados same error 00:46 chris hehe, its all learning 00:45 chris and try that 00:45 chris should be able to uncomment the changes back in 00:45 kados I'm such a dumbass 00:45 kados wow, we could ahve just wasted hours :-) 00:45 chris cool 00:45 kados I'll try that right now 00:45 kados what was I thinking? 00:45 kados duh! 00:45 kados ahh 00:45 chris ie, we really should try the latest stable release 00:45 chris well, it might not be a release 00:44 kados so maybe a mail to perl4lib is in order 00:44 kados hehe 00:44 chris ... if someone tried to run koha from cvs at the mo ... :-) 00:44 kados yep 00:44 chris mind you, we are getting it from cvs 00:44 kados might very well be my busted leader prob 00:44 kados yea 00:44 chris thing 00:44 chris and does seem to be a marc::record 00:43 kados exactly 00:43 chris and will it break in other places? 00:43 kados what about all the other instances of using marc 00:43 chris but, i wonder were else we have a decode 00:43 kados only question I have is 00:43 kados absolutely 00:43 chris does that make sense? 00:43 chris so we could commit that change, as i see no point in converting it, then converting it back, to edit, then converting it to marc again 00:42 chris rather than making it marc, then back to a record, then back to marc to store it 00:42 chris is just pass the record 00:41 chris so what i just got you to do 00:41 kados huh 00:41 chris right 00:41 chris and one of those isnt working 00:41 chris and make a MARC::Record object 00:41 chris then we take the marc 00:41 chris and making it marc 00:41 chris its taking a MARC::Record object 00:41 kados ahh 00:40 chris encode is the $record->as_usmarc(); 00:40 kados I don't see an 'encode' in Biblio.pm 00:40 chris hmm im not sure we do 00:39 kados why do we need them in the first place? 00:39 kados :-) 00:39 chris yeah 00:39 kados and those are both in MARC::Record eh? 00:38 chris its either that encode, or the decode 00:38 kados yea 00:38 chris so at least we have narrowed it down 00:38 kados record's fine 00:38 chris right 00:38 kados yep 00:38 chris so something in that encode/decode is doing something mental 00:37 kados lemme check the record 00:37 chris right 00:37 kados seems to have worked 00:37 kados ha! 00:37 chris my $record=$biblioitem->{marc}; 00:37 chris #my $record = MARC::File::USMARC::decode($biblioitem->{marc}); 00:37 kados gotcha 00:37 chris ie, it looks like 00:37 chris and the second with a my 00:37 chris comment out the first one 00:36 kados second one with a 'my' or just $record? 00:36 chris (so comment out the other lines) 00:36 chris so instead of encoding it, then decoding, we just pass it 00:36 chris do my $record=$biblioitem->{marc}; 00:35 chris my $record = MARC::File::USMARC::decode($biblioitem->{marc}); 00:35 chris and then down at 1369ish 00:35 chris up round line 861 00:35 chris what if we go $olddata->{marc}=$record; 00:34 chris is is encode a record, then decode it 00:34 kados ok 00:34 chris what we do 00:34 chris just to test 00:34 chris ive got a plan 00:34 chris it just takes a while the first time 00:33 kados gotcha 00:33 chris all the REAL mta's work 00:33 kados yea, exim4 00:33 chris that'll work fine 00:33 kados probably exim 00:33 chris what is ur mta? 00:33 chris most spaming bots/zombies dont 00:33 kados hmm, not sure mine will 00:32 chris it does it, and it gets let through 00:32 chris and if the mailer follows the rfc 00:32 chris it greylists it,and tells the mailer to send it again 00:32 kados ah 00:32 chris if it sees an email from a user:server combo it hasnt seen 00:32 chris it does greylisting 00:32 kados arrg ... and we're sooo close :-) 00:31 kados must not like my server :-) 00:31 chris it'll take a while cos it will be coming from a new address 00:31 kados hehe 00:31 chris waiting for it to get thru the spam filter 00:31 kados did you try the 2.0 MARC::Record? 00:31 chris right 00:31 kados the encoding should have nothing to do with it 00:30 kados ie, it runs without it 00:30 kados don't _need_ to 00:30 chris just clutching at straws 00:30 chris do we need to do as_usmarc utf8 ? 00:30 kados so I doubt it 00:30 kados I'm not using any chars outside of ascii range 00:30 kados well, it's utf-8 on the interface 00:29 kados but I don't think that would affect this 00:29 chris could be a utf8 thing? 00:29 kados just noticed some bugs in the leader plugin 00:28 chris and its gone from a nice record to a messed up one 00:27 chris it 00:27 chris and decodes 00:27 chris then calls realnwbiblioitem 00:27 chris it does the as_usmarc 00:26 chris well that appears to be all it does 00:26 kados but I've a sneaking suspicion it's not the prob :-) 00:26 kados I sure would love to blame it on the new MARC::Record 00:24 kados done 00:24 chris can you do that 00:24 chris yeah it timed out 00:24 chris i wonder 00:24 chris it seems very odd that all we do is is encode, the decode and it busts it 00:23 kados I can tar up and mailx you marc-record dir 00:23 kados decode's not deprecated 00:22 chris so thats different then 00:22 kados 348 00:21 chris is it 357 lines long? 00:21 chris can u quickly check USMARC.pm 00:21 kados maybe decode is deprecated 00:21 kados CVS Encode.pm MicroLIF.pm USMARC.pm 00:21 kados ls /usr/local/installed/marc-record/lib/MARC/File/ 00:21 chris right 00:20 kados Batch.pm CVS Doc Field.pm File File.pm Record.pm 00:20 kados ls /usr/local/installed/marc-record/lib/MARC/ 00:20 chris ill know if it ever lets me check it out 00:20 chris could be 00:20 kados isn't MARC::File in MARC::Record? 00:20 kados :-) 00:19 chris cmon sourceforge 00:19 chris it could be the MARC::File::USMARC too i guess 00:18 chris will do 00:18 kados so the 2.0 is first in your path 00:18 chris ill give it a go 00:17 kados then cp -r marc-record/lib/MARC ../C4/ 00:17 kados cvs -z3 -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/marcpm co -P marc-record 00:17 kados could you cd C4 00:16 chris 1.38 00:16 chris lemme see what im running 00:15 kados maybe it is MARC::Record versions 00:15 chris and we end up with a screwed up record 00:15 chris then my $record = MARC::File::USMARC::decode($biblioitem->{marc}); 00:14 chris so all we do is record->as_usmarc() 00:14 chris its busted 00:14 chris but then after we decode it again 00:14 kados yep 00:14 chris right so it looks ok then 00:14 kados my warn is right before $olddata->{marc} = $record->as_usmarc(); 00:13 chris is what im tryig to spot 00:13 chris is it the encode and then decode thats buggering it up 00:13 chris ie does the record look fine at that point 00:13 chris line 861 00:13 kados before what? 00:12 chris just before that? 00:12 chris do you have a warn olddata->{marc} = $record->as_usmarc(); 00:12 chris right so its Newnewbiblio 00:11 kados same with MARC::File::XML 00:11 kados 2.0RCsomething 00:11 kados I've got the latest cvs of MARC::Record 00:11 chris so that gets created line 861 00:11 kados no 090 00:11 kados it's got the author in the 400$a 00:10 kados yea, record's totally mucked up 00:10 kados a 400 00:10 kados bunch of 000s 00:10 kados we've got LDR 00:10 kados wow, we're really missing a lot with that 00:10 chris i wonder if we have different MARC::Record versions 00:10 chris if we do, then i dont know why its failing 00:09 chris (even a c) 00:09 chris and see if we still have an 090 00:08 chris can we check the record? 00:08 chris after thsi 00:08 chris my $record = MARC::File::USMARC::decode($biblioitem->{marc}); 00:08 chris so lets check 00:08 chris so thats ok 00:08 kados ahh yes 00:08 chris its 090 00:08 chris we did in the die 00:08 kados I'll warn biblioitemnumberfield 00:07 chris then we add the d 00:07 kados ahh 00:07 kados because field is empty right? 00:07 chris which should be just getting 090 00:07 chris is failing 00:07 chris right so for some reason my $field=$record->field($biblioitemnumberfield); 00:06 kados only 090 I've got is 090$c 00:06 chris now 00:05 chris ther should be a 090 00:05 chris i lie 00:05 chris ah no 00:05 kados blank? 00:05 chris just blank 00:05 kados (though there is a 090$c) 00:05 chris well it should be added 00:05 kados though it's missing a 090$d (but that hasn't been added yet unless I'm mistaken) 00:04 kados it seems ok 00:04 kados well when I dump it out as_formatted 00:04 kados hmmm ... 00:04 chris back up round line 862 00:04 chris yeah its back when the marc gets made 00:03 kados so that's working 00:03 kados 169645 | 00:03 kados Select max(biblioitemnumber) from biblioitems; 00:02 kados we're not getting a biblioitemnumber 00:02 kados huh ... no, it's not that 00:02 kados so maybe MARCfind_marc_from_kohafield isn't setting biblioitemnumberfield for new records 00:02 chris sure is 00:01 kados that's our line 00:01 kados cant get 090 at /home/koha/plugin/koha/intranet/modules/C4/Biblio.pm line 1370., 00:00 kados k 00:00 chris which will mean that the update will fail 00:00 chris if we put that die in we can check its that line that is failing 00:00 kados (but that's biblionumber) 23:59 chris well mine where all created from bulkmarcimport 23:59 kados (I have a 090$c) 23:59 chris my $field=$record->field($biblioitemnumberfield) || die "cant get $biblioitemnumberfield" ; 23:59 kados so why isn't 090$d getting populated on mine but it is on yours? 23:59 chris can we do 23:58 chris im guessing thats what is failing 23:58 kados interestingly, there is no 090$d field in the dump 23:58 chris my $field=$record->field($biblioitemnumberfield); 23:58 kados 090$d 23:58 chris 090$d is what i have 23:57 kados checking 23:57 chris in biblioitems 23:57 chris what is biblioitemnumber mapped to 23:57 chris can u check ur framework 23:57 chris ah ha 23:57 kados [Sat Apr 01 21:59:17 2006] [error] [client 68.238.11.90] 100 _aJoshua Ferraro, referer: http://kohatest2.liblime.com/cgi-bin/koha/acqui.simple/addbiblio.pl 23:57 kados [Sat Apr 01 21:59:17 2006] [error] [client 68.238.11.90] LDR nam a22 7a 4500, referer: http://kohatest2.liblime.com/cgi-bin/koha/acqui.simple/addbiblio.pl 23:56 kados the record definitely has values 23:56 chris so it should make the marc 23:56 chris $biblioitemnumber = REALnewbiblioitem( $dbh, $olddata ); 23:56 chris # and create biblioitem, that's all folks ! 23:56 chris $olddata->{marcxml} = $record->as_xml(); 23:56 chris $olddata->{marc} = $record->as_usmarc(); 23:55 kados warn record->as_formatted(); 23:55 kados k 23:55 chris to see whats going on 23:55 chris we might want to stick some warns in around there 23:54 chris line 861 23:54 chris its Newnewbiblio 23:53 kados right 23:53 kados 0 23:53 kados select count(*) from biblioitems where marcxml !=''; 23:53 chris which means its cant edit it 23:53 chris yeah so somewhere before this, it isnt making the marc 23:52 kados yea ... but it must be doing it as a part of the save process for new marc records too 23:52 chris ie all this is doing is setting the biblioitemnumber and then resaving 23:51 chris and it should be making the marc 23:51 chris is making the biblioitem 23:51 chris but whatever calls REALnewbiblioitem 23:51 chris so that might break edits 23:51 chris yeah id expect that 23:51 kados hmmm 23:51 kados 0 23:51 kados select count(*) from biblioitems where marc !=''; 23:51 chris since all the fields will be empty 23:51 chris this should not stop an add 23:50 chris BUT 23:50 chris we might have to write a script to populate that 23:50 chris that might be the problem then 23:50 chris right 23:50 chris lemme check in the db 23:50 kados unless I'm mistaken, Koha rel_2_2 doesn't have anything in biblioitems.marc 23:50 chris which must have made the marc and marcxml 23:50 chris and bulkmarcimporting them 23:50 chris i loaded them all by clearing the db 23:49 chris so mine must have marc 23:49 chris yeah 23:49 kados but those still rely on my $record = MARC::File::USMARC::decode($biblioitem->{marc}); 23:49 chris $biblioitem->{marcxml} = $record->as_xml(); 23:49 chris $biblioitem->{marc} = $record->as_usmarc(); 23:49 chris maybe it uses both 23:48 chris hmm 23:48 kados you must have some preexisting data or sth 23:48 chris but yes i think thats the problem 23:48 chris why does it work for me i wonder 23:48 chris hmmm 23:48 kados aahha! ... there isn't gonna be a value in biblioitem->marc is there? 23:47 kados my $field=$record->field($biblioitemnumberfield); 23:47 kados my $record = MARC::File::USMARC::decode($biblioitem->{marc}); 23:47 kados nevermind 23:46 chris lemme look 23:46 kados how could it exist? 23:46 kados I dont' even see where $field is created in that sub 23:46 chris so the is because $field doesnt exist 23:46 kados [Sat Apr 01 21:44:52 2006] [error] [client 68.238.11.90] Can't call method "update" on an undefined value at /home/koha/plugin/koha/intranet/modules/C4/Biblio.pm line 1369., referer: http://kohatest2.liblime.com/cgi-bin/koha/acqui.simple/addbiblio.pl 23:43 kados when I dump out the XML I'm seeing <collection> wrappers 23:42 chris (trying to think what might be different) 23:42 kados ahh ... yea, indexed using them now 23:42 chris thats fixed eh? 23:42 chris i remember we had the error with deletes because of recordids 23:42 kados how so? 23:42 kados hmmm 23:41 chris we are setting record ids eh? 23:41 chris could be, cos my edit works too 23:40 chris hmmm 23:40 kados must be on my end somewhere then ... sigh 23:40 chris my installed version is the same as head 23:40 chris nope no problem in there 23:39 kados $field->update($biblioitemnumbersubfield => "$biblioitemnumber"); 23:39 chris hmm ok lets check Biblio.pm then 23:39 kados hmm 23:39 chris no marc should be on 23:38 kados should MARC=0? 23:38 kados so I must have something different 23:38 kados same error 23:36 kados I'll try default templates 23:36 chris does it work if you change to default? 23:36 kados very weird 23:36 chris so it could be templates then 23:36 chris the eval bit 23:36 chris the only difference is 23:36 chris weird 23:35 kados ahh 23:35 chris hmm i think i fixed it to catch both 23:35 kados passing bibid instead of biblionumber or something 23:35 chris but its possible 23:35 chris shouldnt be 23:35 kados wonder if it's the template 23:34 chris ill do a diff and see whats different between that version and whats in head 23:34 chris then try adding a book, my test one worked 23:34 chris click on add without search 23:34 kados addbooks? 23:33 chris working there 23:33 chris http://koha.koha2.katipo.co.nz/cgi-bin/koha/acqui.simple/addbooks.pl 23:33 kados still working for you? 23:32 chris it was working 23:32 chris weird 23:32 kados with a few things added from rel_2_2 (just the visibility stuff) 23:32 kados nope, head one 23:31 chris thats using the the 2.2 one eh? 23:31 kados [Sat Apr 01 21:33:35 2006] [error] [client 68.238.11.90] Can't call method "update" on an undefined value at /home/koha/plugin/koha/intranet/modules/C4/Biblio.pm line 1369., referer: http://kohatest2.liblime.com/cgi-bin/koha/acqui.simple/addbiblio.pl 23:31 kados [Sat Apr 01 21:33:35 2006] [error] [client 68.238.11.90] sort failed at /home/koha/plugin/koha/intranet/modules/C4/SearchMarc.pm line 286., referer: http://kohatest2.liblime.com/cgi-bin/koha/acqui.simple/addbiblio.pl 23:31 kados [Sat Apr 01 21:33:35 2006] [error] [client 68.238.11.90] no records found, referer: http://kohatest2.liblime.com/cgi-bin/koha/acqui.simple/addbiblio.pl 23:31 kados [Sat Apr 01 21:33:35 2006] [error] [client 68.238.11.90] QUERY : dc.title all /relevant "MARC Records Test" and Author all /relevant "Joshua Ferraro" at /home/koha/plugin/koha/intranet/modules/C4/SearchMarc.pm line 250., referer: http://kohatest2.liblime.com/cgi-bin/koha/acqui.simple/addbiblio.pl 23:31 kados chris: here's what's showing up in the log: 22:39 kados as_xml_record([$format]) Returns a MARC::Record object serialized in XML without a collection wrapper. You can pass an optional format parameter to tell MARC::File::XML what type of record (USMARC, UNIMARC) you are serializ- ing. print $record->as_xml_record('UNIMARC'); 22:39 kados Mike Rylander's docs for the new as_xml_record method: