Time Nick Message 02:26 ToinS hi all 03:14 hdl hell 03:15 hdl o 03:15 chris hello hdl 03:16 ToinS hello hdl 03:16 ToinS hello chros 03:16 ToinS chris ! 03:16 chris :) hi toins 03:17 hdl how are you chris ? 03:18 chris doing well thanks, i enjoyed watching france win on sunday morning (nz time) 03:18 chris how are you? 03:21 hdl good. 03:22 chris the third public library in NZ went live with koha on saturday 03:22 hdl My work for 23 libraries is quite hard. because It takes a LONG time to do migration and there are quite demanding on data quality, going from a low quality data provided. 03:22 hdl Great 03:23 chris ahhh yeah, i know what you mean there 03:23 chris somehow people think we can magically make bad data good :) 04:53 hdl hi tumer 04:54 tumer hi hdl 04:55 tumer hdl check the posting i have done on koha-devel regarding this issue 04:57 tumer hdl:accented character matching for zebra i have committed to koha-devel. I contains all turkish chars and some french. Use that for starting point 04:58 hdl Unfortunately, it is in PERL in itself. 04:58 hdl when matching Pr.*f 04:58 hdl it matches Préf 04:59 hdl But if I use $val=Pr.*f and then $val=~/\.\s*$val.*/ then pattern matching fails. 04:59 hdl Thx for your commit. 05:01 tumer hdl:i can not comment on that since i do not that much perl 09:34 kados paul: you around? 09:35 kados ToinS: can you join koha-devel list? 09:35 ToinS hi kados 09:35 tumer[A] kados:did you get my mail 09:35 kados ToinS: otherwise I"ll need to approve every email you send there :-) 09:35 kados tumer[A]: yes, I need to test that 09:35 kados tumer[A]: and look at the line you mention 09:35 ToinS how can i join ? 09:35 kados tumer[A]: because it's strange I haven't encountered that problem before 09:36 ToinS i believe i have already join .... 09:36 kados ToinS: http://lists.nongnu.org/mailman/listinfo/koha-devel 09:36 ToinS oh no i know ! 09:36 tumer[A] kados:it occurs with accented characters only 09:36 ToinS because i have send the mail with the bad email adress 09:36 kados ToinS: :-) 09:37 kados tumer[A]: I'm gonna go consult the source ... brb 09:38 tumer[A] kados: the issue of normalizing is another matter that nobody is mentioning but is important as well 09:39 tumer kados:i need the devel-week space to upload a new test version of koha. whendo you merge? 09:40 kados devel-week isn't for test versions of Koha 09:41 kados at least not right now 09:41 kados we still don't have a stable version of zebra 09:41 tumer i know but when it is merged it does not have any other purpose 09:41 kados so that's first priority for me ... we actually have to get the thing running in some production environments (besides cyprus :-)) 09:42 kados well HEAD is too unstable for production 09:42 kados so if someone is going to use Koha with zebra in production, it has to be devel-week 09:42 kados what test versions are you planning to commit? 09:42 kados (maybe they could be committed to HEAD?) 09:43 tumer kados:yes but i will not be able to continue with devel week. I modified koha to use MARC only and ZEBRA 09:43 kados tumer: put that in HEAD :-) 09:44 kados (and it's only 11:00am :() 09:44 tumer kados: i no longer have biblioitems, only biblioserver,itemserver,authorityserver all on zebra plus backups of marcs in sql 09:44 kados tumer: that's exactly what I want for 3.0 09:45 kados tumer: so HEAD is the perfect place for that 09:45 tumer well i thought/me on phone 09:47 kados all I have done is: 09:47 kados 1. Search.pm routines for searching zebra 09:47 kados 2. new opac-zoomsearch.pl and tmpl 09:48 kados demonstrates how to use the new Search.pm routines 09:48 kados but ... 09:48 kados for now, all the Search.pm routines are still stored in opac-zoomsearch.pl 09:48 kados because it's easier for me to refine things that way 09:48 kados 3. some configuration files 09:49 kados new bib1.att, new record.abs, etc. 09:50 kados ToinS: how many files were affected by your merge of rel_2_2 and HEAD? 09:50 tumer kados: i thought may be version 3.5 for this as its a big leap and it may delay your release of 3 09:50 ToinS kados: a lot ! 09:50 ToinS i don't know exactly 09:51 tumer kados: did you merge? 09:51 kados I haven't merged anything 09:51 kados toins has merged rel_2_2 and HEAD on his local machine 09:51 kados he's plannig to commit it soon 09:51 ToinS and i have no commited yet 09:52 tumer Toins:does it work? 09:52 kados right 09:52 kados I can't see how HEAD could work at all 09:52 kados the zebra stuff in there is pretty busted IIRC 09:52 ToinS i'm trying all scripts... 09:52 ToinS all seems good execpt scripts about catalogue 09:53 tumer kados: another question: wth npl templates cloning a subfield where does it appear? before or after wher it has been cloned? 09:53 kados ToinS++ 09:53 ToinS because of zebra intregration 09:53 ToinS :) 09:53 kados ToinS: can you commit the non-catalog stuff? 09:53 ToinS ok 09:53 kados tumer: before ... but it puts the value in both places IIRC 09:53 tumer ToinS: if you need help on Zebra you may ask me whenever you like 09:54 kados tumer: if ToinS commits non-catalog stuff to HEAD, can you commit your new catalog stuff? 09:54 ToinS but i've to make some test to see if the other scripts are ok 09:54 kados tumer: we also need to alter the database defs in HEAD 09:54 kados tumer: and we need to think about how best to create an install script 09:55 tumer kados: if I am to commit what i said earlier, there are lors of database commits to be done 09:55 kados hmmm 09:56 tumer kados:upgrading will also be a big issue.Convert everything to marc and to utf8 create marc items separately etc. 09:56 kados yep 09:57 kados tumer: do you have some time today to help me get my dev-week system to add/edit records? 09:57 tumer kados:we will need a programmers installation guide 09:57 kados tumer: if I can get that working, maybe I will be able to focus a bit on the new stuff :-) 09:57 kados tumer: instead of panicking :-) 09:58 tumer kados: calm down whats to pamic about except for UTF8 ,breaking MARC records broken authorities, editor, Zebra quirks.... 09:58 kados tumer: at some point, you sent me a file containing your db defs 09:59 kados yea, that's a whole other issue 09:59 tumer kados:db defs should be on dev_week 10:00 kados tumer: do you remember where? 10:01 tumer kados:chris put it somewhere i dont remember 10:01 kados tumer: you committed a database dump ... 10:01 kados tumer: or maybe you sent it via email ... 10:02 tumer kados: but since your zebra works db_defs should be ok 10:02 kados no ... 10:02 kados zebra only works for searching 10:02 kados adding records, editing records, circ 10:02 kados don't work 10:03 kados ok 10:03 kados I checked the log: 10:03 kados 10:24 < tumer> committed koha.sql and addbiblio.pl10:27 <@kados> thx 10:04 kados so it looks like koha.sql has your db defs 10:04 tumer kados:yes it has 10:06 kados tumer: in dev-week code, we still require the data to exist in old koha tables? 10:07 tumer kados:yes cause it does searching both ways 10:07 kados tumer: this means upgrading is a nightmare :-) 10:07 tumer kados:but in new version NO 10:08 tumer kados:adding and editing should still work with only marc being there 10:09 kados hmmm 10:09 tumer kados:but lots of other code breaks without them 10:10 kados tumer: i've just sent you a link to the zoomkoha admin interface 10:10 tumer it was ment to be a populated database, create marcs and import them in 10:12 kados when I try to add a record I get the following in the zebra logs: 10:12 kados 11:11:57-03/07 zebrasrv(339) [warn] No keys generated for record 10:12 kados 11:11:57-03/07 zebrasrv(339) [warn] The file is probably empty 10:12 kados 11:11:58-03/07 zebrasrv(339) [request] Extended Service: Update (failed) 10:12 kados 11:11:58-03/07 zebrasrv(339) [request] EsRequest ERROR 224 update_record failed 10:13 kados and nothing shows up in specialUpdate 10:13 tumer kados: it should at least write .xml with nothing in it 10:13 kados hmmm 10:13 kados does it try to update zebra or mysql first? 10:15 tumer mysql first 10:15 kados ok 10:16 kados tumer: biblio_framework.sql 10:16 kados tumer: in misc 10:16 kados tumer: is that the stuff that chris committed? 10:17 tumer kados:may be thd 10:17 kados hmmm 10:18 thd I have not committed anything specifically for Zebra 10:18 kados DBD::mysql::st execute failed: Unknown column 'lcsort' in 'field list' at /koha/intranet/modules/C4/Biblio.pm line 1805. 10:18 kados SERVER:biblioserver at /koha/intranet/modules/C4/Biblio.pm line 2780. 10:18 thd Ohh 10:18 kados Error-biblioserver specialUpdate 171715 /errcode:, 224, /MSG:,ES: immediate execution failed,update_record failed 10:19 kados tumer: what is marc_biblio table? 10:20 thd kados: yes I had committed the standard MARC 21 biblio framework but I thought that you knew that 10:20 kados thd: yes, we aren't speaking of that 10:20 tumer kados: marc_biblio is the old rel_2_2 table 10:20 kados tumer: my installation doesn't seem to have it 10:20 tumer kados: you should have something in SpecialUpdate now 10:20 kados tumer: so I can't run update biblio,marc_biblio set biblio.frameworkcode=marc_biblio.frameworkcode where marc_biblio.biblionumber=biblio.biblionumber; 10:21 kados nope 10:22 tumer you seem to have a biblionumber 171715 in your error meaning sql is updated 10:25 tumer kados: regarding your sql updating framework. it is corerct you can run that 10:26 kados tumer: I updated the sql 10:26 kados tumer: then, created and saved a record 10:26 kados tumer: but now I can't find it 10:26 kados tumer: in the OPAC 10:26 tumer well who needs a record anyway 10:26 kados hehe 10:26 kados http://zoomkoha.liblime.com/cgi-bin/koha/detail.pl?bib=171715 10:27 kados http://zoomopac.liblime.com/cgi-bin/koha/opac-detail.pl?bib=171715 10:27 kados well, it shows up 10:27 tumer it shows in MARC view 10:28 kados when I do a simple search 10:28 kados can't find it :( 10:28 kados so it's there ... 10:28 kados but not indexed or something 10:29 kados sigh 10:29 tumer well probably differnt bib-1 attribute to what i am searching 10:30 kados I also get some errors: 10:30 kados DBD::mysql::st execute failed: Table 'zoomkoha.marc_subfield_table' doesn't exist at /koha/intranet/modules/C4/SearchMarc.pm line 601. 10:30 kados tumer: record.abs is used right? 10:31 tumer kados:yes record.abs is used but the marc_subfield_table should be there as part of rel_2_2 unmodified 10:33 kados yes it should 10:34 kados so why is it not? 10:34 kados (does it matter if it's there?) 10:34 kados (not used anymore, right?) 10:34 tumer kados:no it is used for mapping marc to kohafields 10:35 kados arrg 10:35 kados well that's got to be it then 10:35 tumer kados:thats where we define the frameworks 10:35 kados zebra doesn't know which marc to search for 10:35 tumer yep 10:35 kados or to index on 10:35 kados but ... 10:35 kados why does it work for records I manually imported? 10:35 tumer well not exactly 10:36 tumer koha does not know what title is 10:36 kados so I put in a title 10:36 kados Joshua Ferraro tests zebra 10:36 tumer you should be able to reach your record on yaz client 10:36 kados it was in the MARC editor 10:37 kados 245$a 10:37 tumer if it is indexed at all 10:37 kados well if it is indexed in zebra, my search would have found it 10:37 kados the opac search uses pqf 10:37 tumer well unless search.pm crashed before the search 10:38 kados but it works fine for other records 10:38 kados http://zoomopac.liblime.com/ 10:38 dewey well, http://zoomopac.liblime.com/ is working now wher do i look 10:38 kados you can see the record I made here: 10:38 kados http://zoomopac.liblime.com/cgi-bin/koha/opac-detail.pl?bib=171715 10:38 tumer kados:ok i see search works, so this record never got indexed, any zebra errors reported 10:38 kados the MARC view looks right 10:39 kados I didn't see zebra errors 10:39 tumer marc is reda from sql after search 10:39 kados ahh ... 10:39 kados I thought it was from zebra 10:42 kados tumer: http://zoomopac.liblime.com/zebra.log 10:42 kados tumer: that's what zebra does when I save a record 10:42 kados tumer: ahh ... 10:42 kados tumer: I have shadows enabled 10:42 kados tumer: does the code support that? 10:43 tumer kados: its commented out in zebraop 10:43 kados shoot 10:43 kados that must be it 10:43 kados why is it commented out? 10:44 tumer kados:try a manual commit from command line and see whether we find this record 10:44 kados ok 10:45 tumer kados:sorry its not commented out.It needs a parameter in koha.xml 10:45 kados got a strange error on commit: 10:45 kados 11:45:17-03/07 zebraidx(23572) [warn] Failed to open /koha/zebradb/biblios/shadow/reci-i-0.mf [Permission denied] 10:45 kados I suppose the user was wrong 10:46 kados maybe owned by root or something 10:46 tumer <biblioservershadow>1</biblioservershadow> 10:46 kados ok ... 10:46 kados 11:45:45-03/07 zebraidx(23577) [log] commit start 10:46 kados 11:45:47-03/07 zebraidx(23577) [log][app2] zebra_stop 10:46 kados yay 10:46 kados it worked 10:46 kados ok, I enable shadow in the config 10:46 kados then test again 10:47 tumer kados:did you get the shadow thing i wrote 10:47 kados yea 10:47 kados tumer: internal server error :-) 10:48 tumer :-( 10:50 kados still nothing in specialUpdate 10:50 tumer kados:just for the sake of this test can you just turn off shadowing from zebra.cfg and restart zebrasrv 10:51 tumer kados:file is being indexed but not committed so nothing in specialupdate 10:52 tumer i do not use shadowing so i have to look into commit script and test that later 10:53 kados tumer: Not a CODE reference at /koha/intranet/modules/C4/Biblio.pm line 2830. 10:53 kados tumer: that mean anything to you? 10:53 tumer no i dont understand that 10:53 kados that line is: 10:53 kados $Zpackage->("commit"); 10:54 kados so it's to do with ZOOM 10:54 tumer aha 10:54 tumer that line should read : 10:54 tumer $Zpackage->send('commit') 10:54 kados ok 10:55 kados now a different internal server error :-) 10:55 kados wait 10:55 kados my bad 10:55 kados it seems to have worked 10:56 kados but ...! 10:56 tumer ohh! 10:56 kados each time i refreshed the page 10:56 kados it apparantly indexed the record 10:56 kados in zebra 10:56 kados so now I have four duplicates 10:56 kados that's no good 10:57 kados search on 'testing for a third time' :-) 10:57 tumer are we talking about addbiblio 10:57 kados yes 10:57 kados here's what happened: 10:57 kados I added a record 10:57 kados addbiblio threw a error page 10:57 kados I changed something 10:57 kados refreshed the page, send the post request in 10:58 kados addbiblio threw another error 10:58 kados etc. 10:58 kados then ... finally it worked 10:58 kados now, I have four identical records :-) 10:58 kados the old Koha used to check for duplicates 10:58 tumer kados:addbiblio.pm and addbiblio.tmpl can be the newest versions it does not matter 10:58 kados does your code check? 10:59 tumer yes it uses the same Findduplicate routine 11:00 kados add item seems to work 11:01 kados work 11:01 kados works even 11:02 kados hmmm 11:02 kados if I add a item with the same barcode as another item 11:02 kados there is no warning 11:02 kados item delete works 11:03 tumer kados:kados something is probably braking along the line, no matching fields etc 11:04 kados if I add an item with the same barcode as another item, zebra is never invoked 11:04 kados I watched the log as I did it 11:04 tumer you will probably find some DBD errors in your log all related to sql due to mess of tables 11:05 tumer barcode search is done on items table 11:05 tumer old code 11:05 kados ahh 11:05 kados so it's probably not making it into items or something 11:06 tumer on dev week only biblio search is done on zebra everything else is old code 11:06 kados this error: 11:06 kados BD::mysql::st execute failed: Table 'zoomkoha.marc_subfield_table' doesn't exist at /koha/intranet/modules/C4/SearchMarc.pm line 664. 11:06 kados might be the prob 11:06 kados gotcha 11:06 tumer not populating items table is your problem 11:06 kados ok ... 11:06 kados so the question is 11:07 kados if I've got a rel_2_2 db 11:07 kados how do I update to dev_week :-) 11:07 kados of course, I've documented this partly 11:07 tumer unless marc_subfield_table exists you cannot populate any table 11:07 kados but what I'm asking is, how can future Koha users do it? 11:07 kados what's the best plan? 11:08 tumer $100 question 11:08 kados so the steps are: 11:08 tumer they keep all they have 11:08 kados 1. run latest updatedatabase from rel_2_2 11:08 kados 2. run a zoomupdatedatabase that has zoom-specific stuff 11:09 kados 3. run fix_090 11:09 kados 4. export MARC records 11:09 kados 5. index in zebra 11:09 kados 6. start zebra 11:09 kados maybe that's it? 11:10 kados tumer: any opinions? 11:10 tumer 1- have to check rel_2_2 update database 11:10 tumer it may be missing fields 11:10 kados well 11:10 kados it shouldn't include zebra-specific fields 11:10 tumer dev_week i thought has an updatedatabase 11:10 kados those are handled in zoomupdatedatabase script 11:10 kados which I propose to write :-) 11:11 tumer ok when do you convert to utf8? 11:11 kados on export 11:11 kados or you can preprocess them with a separate script 11:11 kados I've got code to do either 11:11 tumer the line of logic seem logical 11:12 kados so we need to check rel_2_2 db and compare it with dev-week's db 11:12 kados to see what zoomupdatedatabase needs to have in it 11:13 tumer well dev_week code has fields for updating items for circulation as well 11:13 kados ok 11:14 tumer my problem is i cannot run installer.pm on windows so i cannot check these 11:14 kados right 11:14 tumer i do manual installation all the time 11:14 kados well we just need some manual sql statements 11:15 kados that can take a rel_2_2 db 11:15 kados and turn it into dev-wek 11:15 tumer dont forget the commit fix we done on biblio.pm (to commit i mean) 11:15 kados already done :-) 11:15 kados ahh ... 11:16 kados we need to convert the tables to utf8 11:16 tumer yes in updatadatabase 11:16 tumer updatedatabase i mean 11:17 tumer all tables need alter table sonadso charset utf8 collate utf8_general_ci 11:17 kados tumer: which tables do I need to check for changes in? 11:17 kados biblio, biblioitem, items ... any others? 11:17 kados sonadso? 11:18 tumer thats all, any extra tables i have may be ommitted 11:18 kados ok 11:18 tumer soanso=table name 11:18 kados ok 11:18 kados tumer: before or after ALTER TABLE $table[0] CHARACTER SET UTF8";? 11:19 tumer at the end of this line add collate utf8_general_ci 11:19 kados ok 11:21 tumer kados:we have to populate framework from marc_biblio to biblio 11:22 kados ls 11:22 kados update biblio,marc_biblio set biblio.frameworkcode=marc_biblio.frameworkcode where marc_biblio.biblionumber=biblio.biblionumber; 11:22 tumer yep 11:22 kados got that stuff 11:22 kados I think we have all the pieces 11:22 kados just need to string them together 11:23 kados I'll do that after lunch 11:23 kados tumer: thanks for your help 11:23 tumer ok thanks as well-waiting your response on USMARC.pm 11:23 kados yep, I"ll get there :-)