IRC log for #koha, 2006-07-04

All times shown according to UTC.

Time S Nick Message
02:26 ToinS hi all
03:14 hdl hell
03:15 hdl o
03:15 chris hello hdl
03:16 ToinS hello hdl
03:16 ToinS hello chros
03:16 ToinS chris !
03:16 chris :) hi toins
03:17 hdl how are you chris ?
03:18 chris doing well thanks, i enjoyed watching france win on sunday morning (nz time)
03:18 chris how are you?
03:21 hdl good.
03:22 chris the third public library in NZ went live with koha on saturday
03:22 hdl My work for 23 libraries is quite hard. because It takes a LONG time to do migration and there are quite demanding on data quality, going from a low quality data provided.
03:22 hdl Great
03:23 chris ahhh yeah, i know what you mean there
03:23 chris somehow people think we can magically make bad data good :)
04:53 hdl hi tumer
04:54 tumer hi hdl
04:55 tumer hdl check the posting i have done on koha-devel regarding this issue
04:57 tumer hdl:accented character matching for zebra i have committed to koha-devel. I contains all turkish chars and some french. Use that for starting point
04:58 hdl Unfortunately, it is in PERL in itself.
04:58 hdl when matching Pr.*f
04:58 hdl it matches Préf
04:59 hdl But if I use $val=Pr.*f and then $val=~/\.\s*$val.*/ then pattern matching fails.
04:59 hdl Thx for your commit.
05:01 tumer hdl:i can not comment on that since i do not that much perl
09:34 kados paul: you around?
09:35 kados ToinS: can you join koha-devel list?
09:35 ToinS hi kados
09:35 tumer[A] kados:did you get my mail
09:35 kados ToinS: otherwise I"ll need to approve every email you send there :-)
09:35 kados tumer[A]: yes, I need to test that
09:35 kados tumer[A]: and look at the line you mention
09:35 ToinS how can i join ?
09:35 kados tumer[A]: because it's strange I haven't encountered that problem before
09:36 ToinS i believe i have already join ....
09:36 kados ToinS: http://lists.nongnu.org/mailma[…]stinfo/koha-devel
09:36 ToinS oh no i know !
09:36 tumer[A] kados:it occurs with accented characters only
09:36 ToinS because i have send the mail with the bad email adress
09:36 kados ToinS: :-)
09:37 kados tumer[A]: I'm gonna go consult the source ... brb
09:38 tumer[A] kados: the issue of normalizing is another matter that nobody is mentioning but is important as well
09:39 tumer kados:i need the devel-week space to upload a new test version of koha. whendo you merge?
09:40 kados devel-week isn't for test versions of Koha
09:41 kados at least not right now
09:41 kados we still don't have a stable version of zebra
09:41 tumer i know but when it is merged it does not have any other purpose
09:41 kados so that's first priority for me ... we actually have to get the thing running in some production environments (besides cyprus :-))
09:42 kados well HEAD is too unstable for production
09:42 kados so if someone is going to use Koha with zebra in production, it has to be devel-week
09:42 kados what test versions are you planning to commit?
09:42 kados (maybe they could be committed to HEAD?)
09:43 tumer kados:yes but i will not be able to continue with devel week. I modified koha to use MARC only and ZEBRA
09:43 kados tumer: put that in HEAD :-)
09:44 kados (and it's only 11:00am :()
09:44 tumer kados: i no longer have biblioitems, only biblioserver,itemserver,authorityserver all on zebra plus backups of marcs in sql
09:44 kados tumer: that's exactly what I want for 3.0
09:45 kados tumer: so HEAD is the perfect place for that
09:45 tumer well i thought/me on phone
09:47 kados all I have done is:
09:47 kados 1. Search.pm routines for searching zebra
09:47 kados 2. new opac-zoomsearch.pl and tmpl
09:48 kados  demonstrates how to use the new Search.pm routines
09:48 kados but ...
09:48 kados for now, all the Search.pm routines are still stored in opac-zoomsearch.pl
09:48 kados because it's easier for me to refine things that way
09:48 kados 3. some configuration files
09:49 kados  new bib1.att, new record.abs, etc.
09:50 kados ToinS: how many files were affected by your merge of rel_2_2 and HEAD?
09:50 tumer kados: i thought may be version 3.5 for this as its a big leap and it may delay your release of 3
09:50 ToinS kados: a lot !
09:50 ToinS i don't know exactly
09:51 tumer kados: did you merge?
09:51 kados I haven't merged anything
09:51 kados toins has merged rel_2_2 and HEAD on his local machine
09:51 kados he's plannig to commit it soon
09:51 ToinS and i have no commited yet
09:52 tumer Toins:does it work?
09:52 kados right
09:52 kados I can't see how HEAD could work at all
09:52 kados the zebra stuff in there is pretty busted IIRC
09:52 ToinS i'm trying all scripts...
09:52 ToinS all seems good execpt scripts about catalogue
09:53 tumer kados: another question: wth npl templates cloning a subfield where does it appear? before or after wher it has been cloned?
09:53 kados ToinS++
09:53 ToinS because of zebra intregration
09:53 ToinS :)
09:53 kados ToinS: can you commit the non-catalog stuff?
09:53 ToinS ok
09:53 kados tumer: before ... but it puts the value in both places IIRC
09:53 tumer ToinS: if you need help on Zebra you may ask me whenever you like
09:54 kados tumer: if ToinS commits non-catalog stuff to HEAD, can you commit your new catalog stuff?
09:54 ToinS but i've to make some test to see if the other scripts are ok
09:54 kados tumer: we also need to alter the database defs in HEAD
09:54 kados tumer: and we need to think about how best to create an install script
09:55 tumer kados: if I am to commit what i said earlier, there are lors of database commits to be done
09:55 kados hmmm
09:56 tumer kados:upgrading will also be a big issue.Convert everything to marc and to utf8 create marc items separately etc.
09:56 kados yep
09:57 kados tumer: do you have some time today to help me get my dev-week system to add/edit records?
09:57 tumer kados:we will need a programmers installation guide
09:57 kados tumer: if I can get that working, maybe I will be able to focus a bit on the new stuff :-)
09:57 kados tumer: instead of panicking :-)
09:58 tumer kados: calm  down whats to pamic about except for UTF8 ,breaking MARC records broken authorities, editor, Zebra quirks....
09:58 kados tumer: at some  point, you sent me a file containing your db defs
09:59 kados yea, that's a whole other issue
09:59 tumer kados:db defs should be on dev_week
10:00 kados tumer: do you remember where?
10:01 tumer kados:chris put it somewhere i dont remember
10:01 kados tumer: you committed a database dump ...
10:01 kados tumer: or maybe you sent it via email ...
10:02 tumer kados: but since your zebra works db_defs should be ok
10:02 kados no ...
10:02 kados zebra only works for searching
10:02 kados adding records, editing records, circ
10:02 kados don't work
10:03 kados ok
10:03 kados I checked the log:
10:03 kados 10:24 < tumer> committed koha.sql and addbiblio.pl10:27 <@kados> thx
10:04 kados so it looks like koha.sql has your db defs
10:04 tumer kados:yes it has
10:06 kados tumer: in dev-week code, we still require the data to exist in old koha tables?
10:07 tumer kados:yes cause it does searching both ways
10:07 kados tumer: this means upgrading is a nightmare :-)
10:07 tumer kados:but in new version NO
10:08 tumer kados:adding and editing should still work with only marc being there
10:09 kados hmmm
10:09 tumer kados:but lots of other code breaks  without them
10:10 kados tumer: i've just sent you a link to the zoomkoha admin interface
10:10 tumer it was ment to be a populated database, create marcs and import them in
10:12 kados when I try to add a record I get the following in the zebra logs:
10:12 kados 11:11:57-03/07 zebrasrv(339) [warn] No keys generated for record
10:12 kados 11:11:57-03/07 zebrasrv(339) [warn]  The file is probably empty
10:12 kados 11:11:58-03/07 zebrasrv(339) [request] Extended Service: Update (failed)
10:12 kados 11:11:58-03/07 zebrasrv(339) [request] EsRequest  ERROR 224 update_record failed
10:13 kados and nothing shows up in specialUpdate
10:13 tumer kados: it should at least write .xml with nothing in it
10:13 kados hmmm
10:13 kados does it try to update zebra or mysql first?
10:15 tumer mysql first
10:15 kados ok
10:16 kados tumer: biblio_framework.sql
10:16 kados tumer: in misc
10:16 kados tumer: is that the stuff that chris committed?
10:17 tumer kados:may be thd
10:17 kados hmmm
10:18 thd I have not committed anything specifically for Zebra
10:18 kados DBD::mysql::st execute failed: Unknown column 'lcsort' in 'field list' at /koha/intranet/modules/C4/Biblio.pm line 1805.
10:18 kados SERVER:biblioserver at /koha/intranet/modules/C4/Biblio.pm line 2780.
10:18 thd Ohh
10:18 kados Error-biblioserver   specialUpdate 171715 /errcode:, 224, /MSG:,ES: immediate execution failed,update_record failed
10:19 kados tumer: what is marc_biblio table?
10:20 thd kados: yes I had committed the standard MARC 21 biblio framework but I thought that you knew that
10:20 kados thd: yes, we aren't speaking of that
10:20 tumer kados: marc_biblio is the old rel_2_2 table
10:20 kados tumer: my installation doesn't seem to have it
10:20 tumer kados: you should have something in SpecialUpdate now
10:20 kados tumer: so I can't run update biblio,marc_biblio set biblio.frameworkcode=marc_biblio.frameworkcode where marc_biblio.biblionumber=biblio.biblionumber;
10:21 kados nope
10:22 tumer you seem to have a biblionumber 171715 in your error meaning sql is updated
10:25 tumer kados: regarding your sql updating framework. it is corerct you can run that
10:26 kados tumer: I updated the sql
10:26 kados tumer: then, created and saved a record
10:26 kados tumer: but now I can't find it
10:26 kados tumer: in the OPAC
10:26 tumer well who needs a record anyway
10:26 kados hehe
10:26 kados http://zoomkoha.liblime.com/cg[…]ail.pl?bib=171715
10:27 kados http://zoomopac.liblime.com/cg[…]ail.pl?bib=171715
10:27 kados well, it shows up
10:27 tumer it shows in MARC view
10:28 kados when I do a simple search
10:28 kados can't find it :(
10:28 kados so it's there ...
10:28 kados but not indexed or something
10:29 kados sigh
10:29 tumer well probably differnt bib-1 attribute to what i am searching
10:30 kados I also get some errors:
10:30 kados DBD::mysql::st execute failed: Table 'zoomkoha.marc_subfield_table' doesn't exist at /koha/intranet/modules/C4/SearchMarc.pm line 601.
10:30 kados tumer: record.abs is used right?
10:31 tumer kados:yes record.abs is used but the marc_subfield_table should be there as part of rel_2_2 unmodified
10:33 kados yes it should
10:34 kados so why is it not?
10:34 kados (does it matter if it's there?)
10:34 kados (not used anymore, right?)
10:34 tumer kados:no it is used for mapping marc to kohafields
10:35 kados arrg
10:35 kados well that's got to be it then
10:35 tumer kados:thats where we define the frameworks
10:35 kados zebra doesn't know which marc to search for
10:35 tumer yep
10:35 kados or to index on
10:35 kados but ...
10:35 kados why does it work for records I manually imported?
10:35 tumer well not exactly
10:36 tumer koha does not know what title is
10:36 kados so I put in a title
10:36 kados Joshua Ferraro tests zebra
10:36 tumer you should be able to reach your record on yaz client
10:36 kados it was in the MARC editor
10:37 kados 245$a
10:37 tumer if it is indexed at all
10:37 kados well if it is indexed in zebra, my search would have found it
10:37 kados the opac search uses pqf
10:37 tumer well unless search.pm crashed before the search
10:38 kados but it works fine for other records
10:38 kados http://zoomopac.liblime.com/
10:38 dewey well, http://zoomopac.liblime.com/ is working now wher do i look
10:38 kados you can see the record I made here:
10:38 kados http://zoomopac.liblime.com/cg[…]ail.pl?bib=171715
10:38 tumer kados:ok i see search works, so this record never got indexed, any zebra errors reported
10:38 kados the MARC view looks right
10:39 kados I didn't see zebra errors
10:39 tumer marc is reda from sql after search
10:39 kados ahh ...
10:39 kados I thought it was from zebra
10:42 kados tumer: http://zoomopac.liblime.com/zebra.log
10:42 kados tumer: that's what zebra does when I save a record
10:42 kados tumer: ahh ...
10:42 kados tumer: I have shadows enabled
10:42 kados tumer: does the code support that?
10:43 tumer kados: its commented out in zebraop
10:43 kados shoot
10:43 kados that must be it
10:43 kados why is it commented out?
10:44 tumer kados:try a manual commit from command line and see whether we find this record
10:44 kados ok
10:45 tumer kados:sorry its not commented out.It needs a parameter in koha.xml
10:45 kados got a strange error on commit:
10:45 kados 11:45:17-03/07 zebraidx(23572) [warn] Failed to open /koha/zebradb/biblios/shadow/reci-i-0.mf [Permission denied]
10:45 kados I suppose the user was wrong
10:46 kados maybe owned by root or something
10:46 tumer <biblioservershadow>1</biblioservershadow>
10:46 kados ok ...
10:46 kados 11:45:45-03/07 zebraidx(23577) [log] commit start
10:46 kados 11:45:47-03/07 zebraidx(23577) [log][app2] zebra_stop
10:46 kados yay
10:46 kados it worked
10:46 kados ok, I enable shadow in the config
10:46 kados then test again
10:47 tumer kados:did you get the shadow thing i wrote
10:47 kados yea
10:47 kados tumer: internal server error :-)
10:48 tumer :-(
10:50 kados still nothing in specialUpdate
10:50 tumer kados:just for the sake of this test can you just turn off shadowing from zebra.cfg and restart zebrasrv
10:51 tumer kados:file is being indexed but not committed so nothing in specialupdate
10:52 tumer i do not use shadowing so i have to look into commit script and test that later
10:53 kados tumer: Not a CODE reference at /koha/intranet/modules/C4/Biblio.pm line 2830.
10:53 kados tumer: that mean anything to you?
10:53 tumer no i dont understand that
10:53 kados that line is:
10:53 kados $Zpackage->("commit");
10:54 kados so it's to do with ZOOM
10:54 tumer aha
10:54 tumer that line should read :
10:54 tumer $Zpackage->send('commit')
10:54 kados ok
10:55 kados now a different internal server error :-)
10:55 kados wait
10:55 kados my bad
10:55 kados it seems to have worked
10:56 kados but ...!
10:56 tumer ohh!
10:56 kados each time i refreshed the page
10:56 kados it apparantly indexed the record
10:56 kados in zebra
10:56 kados so now I have four duplicates
10:56 kados that's no good
10:57 kados search on 'testing for a third time' :-)
10:57 tumer are we talking about addbiblio
10:57 kados yes
10:57 kados here's what happened:
10:57 kados I added a record
10:57 kados addbiblio threw a error page
10:57 kados I changed something
10:57 kados refreshed the page, send the post request in
10:58 kados addbiblio threw another error
10:58 kados etc.
10:58 kados then ... finally it worked
10:58 kados now, I have four identical records :-)
10:58 kados the old Koha used to check for duplicates
10:58 tumer kados:addbiblio.pm and addbiblio.tmpl can be the newest versions it does not matter
10:58 kados does your code check?
10:59 tumer yes it uses the same Findduplicate routine
11:00 kados add item seems to work
11:01 kados work
11:01 kados works even
11:02 kados hmmm
11:02 kados if I add a item with the same barcode as another item
11:02 kados there is no warning
11:02 kados item delete works
11:03 tumer kados:kados something is probably braking along the line, no matching fields etc
11:04 kados if I add an item with the same barcode as another item, zebra is never invoked
11:04 kados I watched the log as I did it
11:04 tumer you will probably find some DBD errors in your log all related to sql due to mess of tables
11:05 tumer barcode search is done on items table
11:05 tumer old code
11:05 kados ahh
11:05 kados so it's probably not making it into items or something
11:06 tumer on dev week only biblio search is done on zebra everything else is old code
11:06 kados this error:
11:06 kados BD::mysql::st execute failed: Table 'zoomkoha.marc_subfield_table' doesn't exist at /koha/intranet/modules/C4/SearchMarc.pm line 664.
11:06 kados might be the prob
11:06 kados gotcha
11:06 tumer not populating items table is your problem
11:06 kados ok ...
11:06 kados so the question is
11:07 kados if I've got a rel_2_2 db
11:07 kados how do I update to dev_week :-)
11:07 kados of course, I've documented this partly
11:07 tumer unless marc_subfield_table exists you cannot populate any table
11:07 kados but what I'm asking is, how can future Koha users do it?
11:07 kados what's the best plan?
11:08 tumer $100 question
11:08 kados so the steps are:
11:08 tumer they keep all they have
11:08 kados 1. run latest updatedatabase from rel_2_2
11:08 kados 2. run a zoomupdatedatabase that has zoom-specific stuff
11:09 kados 3. run fix_090
11:09 kados 4. export MARC records
11:09 kados 5. index in zebra
11:09 kados 6. start zebra
11:09 kados maybe that's it?
11:10 kados tumer: any opinions?
11:10 tumer 1- have to check rel_2_2 update database
11:10 tumer it may be missing fields
11:10 kados well
11:10 kados it shouldn't include zebra-specific fields
11:10 tumer dev_week i thought has an updatedatabase
11:10 kados those are handled in zoomupdatedatabase script
11:10 kados which I propose to write :-)
11:11 tumer ok when do you convert to utf8?
11:11 kados on export
11:11 kados or you can preprocess them with a separate script
11:11 kados I've got code to do either
11:11 tumer the line of logic seem logical
11:12 kados so we need to check rel_2_2 db and compare it with dev-week's db
11:12 kados to see what zoomupdatedatabase needs to have in it
11:13 tumer well dev_week code has fields for updating items for circulation as well
11:13 kados ok
11:14 tumer my problem is i cannot run installer.pm on windows so i cannot check these
11:14 kados right
11:14 tumer i do manual installation all the time
11:14 kados well we just need some manual sql statements
11:15 kados that can take a rel_2_2 db
11:15 kados and turn it into dev-wek
11:15 tumer dont forget the commit fix we done on biblio.pm (to commit i mean)
11:15 kados already done :-)
11:15 kados ahh ...
11:16 kados we need to convert the tables to utf8
11:16 tumer yes in updatadatabase
11:16 tumer updatedatabase i mean
11:17 tumer all tables need alter table sonadso charset utf8 collate utf8_general_ci
11:17 kados tumer: which tables do I need to check for changes in?
11:17 kados biblio, biblioitem, items ... any others?
11:17 kados sonadso?
11:18 tumer thats all, any extra tables i have may be ommitted
11:18 kados ok
11:18 tumer soanso=table name
11:18 kados ok
11:18 kados tumer: before or after ALTER TABLE $table[0] CHARACTER SET UTF8";?
11:19 tumer at the end of this line add collate utf8_general_ci
11:19 kados ok
11:21 tumer kados:we have to populate framework from marc_biblio to biblio
11:22 kados ls
11:22 kados update biblio,marc_biblio set biblio.frameworkcode=marc_biblio.frameworkcode where marc_biblio.biblionumber=biblio.biblionumber;
11:22 tumer yep
11:22 kados got that stuff
11:22 kados I think we have all the pieces
11:22 kados just need to string them together
11:23 kados I'll do that after lunch
11:23 kados tumer: thanks for your help
11:23 tumer ok thanks as well-waiting your response on USMARC.pm
11:23 kados yep, I"ll get there :-)

| Channels | #koha index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary