Time Nick Message
12:05 paul wow... segfault...
12:05 paul [paul@portable zebradb]$ zebraidx -g iso2709 -c /home/paul/koha.dev/dev_week/zebraplugin/etc/zebra-biblios.cfg -d kohaplugin update ../export/
12:05 paul 19:05:59-12/07 zebraidx(30255) [log] zebra_start /home/paul/koha.dev/dev_week/zebraplugin/etc/zebra-biblios.cfg 1.3.36
12:05 paul Erreur de segmentation
12:06 paul in which directory am I supposed to be when I run this command ?
12:06 paul I was in : /home/paul/koha.dev/dev_week/zebraplugin/zebradb
12:06 paul 7PM, maybe it's time to leave for dinner...
12:06 paul i'll investigate tomorrow...
12:07 tumer paul: you have to be one folder above rtecords foldfer
12:07 kados paul: maybe missing 090s :-)
12:07 tumer where zebra.cfg resides
12:08 kados paul: also maybe need to make sure no CVS dir is in records?
12:08 paul you have to be one folder above rtecords foldfer => done, no change
12:09 paul also maybe need to make sure no CVS dir is in records => checked, no CVS dir in my fresh export/
12:09 paul maybe missing 090s => missing090.pl run just before exporting.
12:09 kados version of zebra?
12:09 paul +the problem occurs immediatly
12:09 paul a fresh 1.3.36
12:10 kados hmmm
12:10 paul + a fresh yaz install too
12:10 kados what is the exact error?
12:10 tumer paul run zebraidx with no parameters and see what it saya
12:10 paul the exact error is just before :
12:10 paul 19:05:59-12/07 zebraidx(30255) [log] zebra_start /home/paul/koha.dev/dev_week/zebraplugin/etc/zebra-biblios.cfg 1.3.36
12:10 paul Erreur de segmentation
12:11 paul (segmentation fault in french)
12:11 paul zebraidx without parameters give me the help page
12:11 paul zebraidx
12:11 paul zebraidx [options] command <dir> ...
12:11 paul Commands:
12:11 paul update <dir> Update index with files below <dir>.
12:11 paul If <dir> is empty filenames are read from stdin.
12:11 paul delete <dir> Delete index with files below <dir>.
12:11 paul create <db> Create database <db>
12:11 tumer so its not zebra
12:11 paul ...
12:11 kados paul: paste in your exact command
12:11 kados that segfautls
12:11 paul zebraidx -g iso2709 -c /home/paul/koha.dev/dev_week/zebraplugin/etc/zebra-biblios.cfg -d kohaplugin update export/
12:12 paul i'm in /home/paul/koha.dev/dev_week/zebraplugin directory
12:12 kados how many records?
12:12 tumer -d kohaplugin update export/ --- kados what is that?
12:12 paul 14 000 (around)
12:13 kados -d is database
12:13 kados update is the command
12:13 kados export/ is where the iso2709 file is
12:13 tumer i know what does pauls code saying
12:13 paul mmm... the DB is not kohaplugin
12:13 kados paul: marcdump export/marc.iso2709
12:13 kados paul: make sure your MARC is valid
12:14 paul oups...
12:14 paul utf8 "\xE9" does not map to Unicode at /usr/lib/perl5/5.8.7/i386-linux/Encode.pm line 166.
12:14 paul welcome back, unicode problems...
12:17 paul any idea ?
12:17 paul tumer & kados ?
12:17 kados it seems you don't have utf8 data :-)
12:17 paul (otherwise I leave for dinner...)
12:17 paul sh....
12:17 kados wait
12:17 kados I'm wrong
12:18 paul ok, I wait ...
12:18 kados that means that \xE9 is not valid MARC8
12:18 kados it must be latin1
12:18 kados so maybe marcdump can't handle unimarc
12:18 paul iirc, it's the é, a very common accent in french
12:18 kados it think you have marc8 data
12:19 paul i get a segfault without any more interesting information.
12:19 paul can I send you my file to see if you get the same segfault ?
12:19 tumer well i think he still has latin1
12:19 paul maybe it's a yaz/zebra/??? version problem
12:20 paul tumer is probably right
12:20 tumer paul:use one record only and index with flag -s
12:20 tumer if it works at all you will see lots of dump
12:21 paul why 1 record only ?
12:21 tumer with -s you get the debugger showing you all the steps veeery long process
12:22 tumer for zebra your encoding is no problem it does not check that
12:23 paul still segfault, without any message.
12:23 paul just
12:23 paul 19:23:17-12/07 zebraidx(30360) [log] zebra_start /home/paul/koha.dev/dev_week/zebraplugin/etc/zebra-biblios.cfg 1.3.36
12:23 paul segfault
12:24 paul I updated my copy of dev_week, and the path to my zebradb are /koha...
12:24 paul which is invalid !
12:24 tumer i do not understand the linux line but if you are in the right folder -c shows the .cfg file than your zebra is corrupt
12:26 paul tumer: will you be here tomorrow ?
12:26 tumer i am around
12:26 paul bye tumer & kados
12:26 tumer bonapetit
12:26 paul thanks
02:17 btoumi hi all
03:35 btoumi somebody can explain me what is $DEBUG ? is it system variable?
03:49 btoumi chris: are u around
03:49 btoumi ?
04:27 paul kados : segfault solved : some directories where missing (key & tmp, under biblios/)
07:20 btoumi paul are u around?
07:22 btoumi paul2 are u around?
07:24 paul hello btoumi
07:24 btoumi hi paul
07:24 btoumi ca va?
07:24 paul il fait un peu froid à MArseille. Je vais mettre une petite laine...
07:25 btoumi lol a istres aussi a -35° ca crain un peu
07:25 btoumi g une petite question pour toi au sujet de ce qui m'occupe en ce moment ,les amandes
07:26 btoumi as tu le temps ou pas?
07:26 paul le amandes ? miam, c'est bon ca, les amandes...
07:26 paul je préfère les noisettes remarque.
07:26 btoumi lol
07:26 paul mais bon, zyva pour les amandes
07:27 btoumi a quoi correspond maxfine
07:27 btoumi ?
07:28 paul ou tu vois ca ?
07:28 btoumi dans fines2.pl line 68
07:29 btoumi je penses a un montant maximum mais pas sur
07:30 paul fines2.pl est dans quel répertoire ?
07:30 paul ok, trouvé
07:31 btoumi misc
07:31 paul ligne 93
07:31 paul c'est donc bien un maximum
07:31 paul n'hésites pas à en faire une préférence système !
07:31 btoumi a la ligne 68 il y a my maxFine=5;
07:32 btoumi a quoi sert t 'elle exactement?
07:32 btoumi je suis d'accord pour la prefence systeme
07:32 paul à affecter la variable en début de script pour permettre une personnalisation + simple
07:32 paul rustique, mais efficace !
07:33 btoumi est ce que je peut t'appeler?
07:34 paul c'est difficile : je suis chez Antoine pour avoir internet, mais SFR passe très mal ou il habite sur la colinne de la bonne mère pourtant !
07:34 paul mais tu peux tenter le coup si tu veux (sur mon N° portable donc)
07:35 btoumi ok j'essaye
07:41 btoumi meric paul:
07:41 btoumi merci pardon
08:06 paul kados around ?
08:06 kados yep
08:06 kados hia ll
08:06 kados hi all even
08:07 kados :-)
08:07 paul hello & good morning
08:07 paul some good news :
08:07 paul * no more seg fault
08:07 kados w00t
08:07 paul * no more encoding problems (utf8 valid)
08:07 kados yay!
08:07 paul * zebraidx works fine
08:07 paul now, I just have 2 questions :
08:07 paul zebrasrv is running :
08:07 paul 15:08:25-13/07 [log] zebra_start /home/paul/koha.dev/dev_week/zebraplugin/etc/zebra-biblios.cfg 1.3.36
08:07 paul 15:08:25-13/07 [log] zebra_start /home/paul/koha.dev/dev_week/zebraplugin/etc/zebra-authorities.cfg 1.3.36
08:07 paul 15:08:25-13/07 [server] Adding dynamic listener on tcp:@:9900 id=1
08:08 paul 15:08:25-13/07 [server] Adding dynamic listener on tcp:@:2100 id=2
08:08 paul 15:08:25-13/07 [server] Starting server zebrasrv pid=8156
08:08 kados right
08:08 paul iiuc, the biblio are on 9900
08:08 kados correct
08:08 paul I tried yaz-client @9900, then "find delmas"
08:08 paul but got :
08:08 paul [109] Database unavailable -- v2 addinfo 'Default'
08:08 kados you need to do:
08:08 kados yaz-client
08:08 dewey hmmm... yaz-client is returning results
08:08 kados open localhost:9900/biblios
08:09 paul why biblios ?
08:09 kados ie, specify the database name
08:09 kados biblios is the db name
08:09 paul i tried with "biblioserver"
08:09 kados in the koha.xml file
08:09 kados biblioserver is the 'type'
08:09 paul <server id="biblioserver" listenref="biblioserver">
08:09 paul ?
08:09 paul type ???
08:09 kados well ...
08:09 paul I don't see any "biblios"
08:09 kados we have different types of zebra servers
08:09 kados no?
08:10 kados <biblioserver>biblios</biblioserver>
08:10 paul oups...
08:10 paul sorry. i missed it !
08:10 paul strange to have the name here and not in <server> tag
08:10 kados well ...
08:11 kados you should have seen how complicated it was before we switched to an xml conf file :-)
08:11 paul still
08:11 kados it had to be put in 4 places :-)
08:11 paul [109] Database unavailable -- v2 addinfo 'biblios'
08:11 kados hmmm
08:11 kados you started zebra how?
08:11 paul xml is great, I just have to learn how it is written
08:11 kados zebrasrv -f /path/to/koha.xml
08:11 kados ?
08:11 paul zebrasrv -f etc/koha.xml
08:11 kados hmmm
08:11 kados when you indexed your records?
08:11 kados you specified -d biblios ?
08:12 kados maybe the docs are wrong here
08:12 paul zebraidx -g iso2709 -c /home/paul/koha.dev/dev_week/zebraplugin/etc/zebra-biblios.cfg update export.unique/
08:12 paul ok, I didn't understand the -d biblioplubin
08:14 kados updated
08:15 paul how can I drop the DB to rebuild it ?
08:15 kados zebraidx init
08:15 paul zebraidx -c etc/koha.xml drop seems useless
08:15 kados but you don't need to
08:15 kados koha.xml isn't the right config for -c
08:15 kados zebra-biblios.cfg is
08:15 kados anyway I don't think the drop command works
08:16 kados but init does the same thing
08:16 kados but you don't need to do that
08:16 kados if you simply update again it will fix everything
08:16 kados (I think)
08:16 kados (though you may still have the old db there ...)
08:16 kados anyway, init will do it
08:17 kados it clears out the registers
08:17 paul zebraidx -g iso2709 -d biblios -c /home/paul/koha.dev/dev_week/zebraplugin/etc/zebra-biblios.cfg update export.unique/
08:17 kados looks good
08:19 paul 15:20:16-13/07 zebraidx(8221) [log] Records: 1 i/u/d 0/1/0
08:20 paul means 1 record has been inserted, right ?
08:21 kados yes I think so
08:21 kados -s will give you all the gritty details while you're indexing
08:23 paul then :
08:23 paul [paul@portable zebraplugin]$ zebrasrv -f etc/koha.xml
08:23 paul 15:23:36-13/07 [log] zebra_start /home/paul/koha.dev/dev_week/zebraplugin/etc/zebra-biblios.cfg 1.3.36
08:23 paul 15:23:36-13/07 [log] zebra_start /home/paul/koha.dev/dev_week/zebraplugin/etc/zebra-authorities.cfg 1.3.36
08:23 paul 15:23:36-13/07 [server] Adding dynamic listener on tcp:@:9900 id=1
08:23 paul 15:23:36-13/07 [server] Adding dynamic listener on tcp:@:2100 id=2
08:23 paul 15:23:36-13/07 [server] Starting server zebrasrv pid=8240
08:23 paul zebrasrv is running fine
08:23 paul then yaz-client to query :
08:23 paul yaz-client localhost:9900/biblios
08:23 paul Connecting...OK.
08:24 paul Sent initrequest.
08:24 paul Connection accepted by v3 target.
08:24 paul ID : 81
08:24 paul Name : Zebra Information Server/GFS/YAZ
08:24 paul Version: Zebra 1.3.36/1.86/2.1.22
08:24 paul Options: search present delSet triggerResourceCtrl scan sort extendedServices namedResultSets
08:24 paul Elapsed: 0.001979
08:24 paul Z> find delmas
08:24 paul Sent searchRequest.
08:24 paul Received SearchResponse.
08:24 paul Search was a bloomin' failure.
08:24 paul Number of hits: 0, setno 1
08:24 paul records returned: 0
08:24 paul Diagnostic message(s) from database:
08:24 paul [109] Database unavailable -- v2 addinfo 'biblios'
08:24 paul Elapsed: 0.002820
08:24 paul in zebrasrv logs :
08:25 paul 15:23:56-13/07 zebrasrv(1) [session] Starting session 1 from tcp:127.0.0.1 (pid=8245)
08:25 paul 15:23:56-13/07 zebrasrv(1) [log] User perm for perm.anonymous: r
08:25 paul 15:23:56-13/07 zebrasrv(1) [request] Auth none
08:25 paul 15:23:56-13/07 zebrasrv(1) [request] Init OK - ID:81 Name:YAZ Version:2.1.22
08:25 paul 15:24:00-13/07 zebrasrv(1) [log] ResultSet '1'
08:25 paul 15:24:00-13/07 zebrasrv(1) [log][app2] zebra_register_open rw = 0 useshadow=0 p=0x8197448,n=,rp=(none)
08:25 paul 15:24:00-13/07 zebrasrv(1) [log] profilePath=.:/usr/local/share/idzebra/tab/:/home/paul/koha.dev/dev_week/zebraplugin/zebradb/biblios/tab/:./tab/ cwd=/home/paul/koha.dev/dev_week/zebraplugin/zebradb/biblios
08:25 paul 15:24:00-13/07 zebrasrv(1) [log] 0 keys, 0 sysnos, rank
08:25 paul 15:24:00-13/07 zebrasrv(1) [log] term="delmas" nn=0 type=void count=0
08:25 paul 15:24:00-13/07 zebrasrv(1) [log] user/system: 0/0
08:25 paul 15:24:00-13/07 zebrasrv(1) [request] Search ERROR 109 1 1+0 RPN: @attrset Bib-1 delmas
08:26 kados [109] Database unavailable -- v2 addinfo 'biblios'
08:26 kados that's the error
08:28 kados hmmm
08:28 kados paul: in yaz-client try:
08:28 kados base IR-Explain-1
08:29 kados form sutrs
08:29 kados f @attr exp1 1=1 databaseinfo
08:30 kados s
08:31 kados what is in the 'recordCountActual' line?
08:31 paul databaseinfos with a s ?
08:31 kados s eq show
08:31 paul hmm...
08:31 paul Z> s
08:31 paul Sent presentRequest (1+1).
08:31 paul Diagnostic message(s) from database:
08:31 paul [13] Present request out of range -- v2 addinfo '1'
08:31 paul nextResultSetPosition = 2
08:31 paul Elapsed: 0.000614
08:31 kados so you don't have a db
08:32 kados did you do a commit ?
08:32 kados (if you have shadows enabled)
08:32 paul no i forgot this too
08:34 paul [paul@portable zebraplugin]$ zebraidx -g iso2709 -c etc/zebra-biblios.cfg -d biblios commit
08:34 paul 15:34:25-13/07 zebraidx(8301) [log] zebra_start etc/zebra-biblios.cfg 1.3.36
08:34 paul 15:34:25-13/07 zebraidx(8301) [log] enabling cache spec=/home/paul/koha.dev/dev_week/zebraplugin/zebradb/biblios/shadow:4G
08:34 paul 15:34:25-13/07 zebraidx(8301) [log] cache_fname = /home/paul/koha.dev/dev_week/zebraplugin/zebradb/biblios/shadow/cache
08:34 paul 15:34:25-13/07 zebraidx(8301) [log] commit start
08:34 paul 15:34:25-13/07 zebraidx(8301) [log][app2] zebra_stop
08:34 paul alleluja !!!!
08:34 kados :-)
08:34 paul Z> find delmas
08:34 paul Sent searchRequest.
08:34 paul Received SearchResponse.
08:34 paul Search was a success.
08:34 paul Number of hits: 1, setno 1
08:34 paul SearchResult-1: term=delmas cnt=1
08:34 paul records returned: 0
08:34 paul Elapsed: 0.003601
08:34 kados use s to see the record
08:34 paul I see :
08:34 kados and opac-zoomsearch.pl should be working right now
08:35 paul [biblios]Record type: USmarc
08:35 paul 00743 a2200217 4500
08:35 paul 001 2
08:35 paul 010 $a 2729843132 $d 190 F
08:35 paul 090 $9 1 $a 1
08:35 paul 100 $a 1995 y0fre 0103 ba
08:35 paul 101 $a fre
08:35 paul 105 $a y 00 y
08:35 paul 200 1 $a Introduction aux probabilités $b LIVR $f Jean-Pierre DELMAS
08:35 paul 210 $5 Edition Marketing $a Paris $c Edition Marketing $d 1993
08:35 paul 215 $a 316 p
08:35 paul 225 $a Collection Pédagogique de Télécommunication
08:35 paul 410 $t Collection Pedagogique de Telecommunication
08:35 paul 606 $9 3339 $9 3321 $a MATHEMATIQUES
08:35 paul 700 $9 6659 $a DELMAS, Jean-Pierre
08:35 paul 995 $b CDI $c CDI $e SL $f Non inventorie $j 000498 $o 2 $9 1
08:35 paul 995 $b CDI $c CDI $e SL $f Non inventorie $j 005643 $o 2 $9 2
08:35 paul 995 $b CDI $c CDI $e SL $f Non inventorie $j 000527 $o 2 $9 3
08:35 paul nextResultSetPosition = 2
08:35 paul which is what I expected.
08:35 kados w00t!
08:35 paul e Télé
08:35 paul being correct utf8
08:35 paul (the famous é)
08:35 kados so now we have three zebra experts in #koha :-)
08:35 paul I was thinking to unimarc support.
08:36 kados yes, it's much needed
08:36 paul if I don't mind, the only file I have to modify is record.abs
08:36 kados also, I need to put the search subs in opac-zoomsearch.pl into Search.pm
08:36 kados bib1.att also
08:36 paul to say melm 200$a author
08:36 kados bib1.att has indexing rules for some special searches
08:36 paul why bib1.att ?
08:37 kados like for searching 008 in marc21
08:37 kados necessary to search by 'date' in marc21
08:37 kados as it's the only normalized place that date exists
08:37 kados also for format/content/audience
08:37 kados must search fixed fields as well
08:40 paul ok that was my 2nd question ;-)
08:40 paul :
08:40 paul I had problems with zebraidx until I commented the 2 following lines :
08:40 paul # melm 008 date-entered-on-file:n:range(data,0,6),date-entered-on-file:s:range(data,0,6),date1:n:range(data,7,4),date1:s:range(data,7,4),date2:n:range(data,11,4),date2:s:range(data,11,4),ff8-22:w:range(data,22,1),language:w:range(data,35,3),ff8-33:w:range(data,33,1),ff8-34:w:range(data,34,1)
08:40 paul # melm 007 ff7-00:w:range(data,0,1),ff7-01:w:range(data,1,1),ff7-02:w:range(data,2,1)
08:40 paul it was "Couldn't find att 'date-entered-on-file' in attset"
08:40 paul any explanations ?
08:40 paul kados ?
08:40 dewey hmmm... kados is becoming a true Perl Monger...
08:40 paul I had a small disconnect...
08:40 kados paul: bib1.att in /usr/share/idzebra/tab does not contain everything that's in record.abs
08:41 paul so you modified it ?
08:41 kados it should be committed
08:42 kados it's in tab dir in zebradb/biblios/
08:42 paul I must remove the /usr/local/share/idzebra/tab one ?
08:43 kados well ...
08:43 kados in zebra docs
08:43 kados you can specify location of tab dirs
08:43 kados in the zebra.cfg
08:43 kados (we call it zebra-biblios.cfg)
08:43 kados but I can't get it to work
08:43 kados so I just copy my bib1.att to /usr/local/share/idzebra/tab one
08:44 kados I wrote ID about this but haven't heard back yet
08:45 ToinS kados: how do you spell "recieve" : recieve or receive ?
08:45 kados receive
08:45 kados ToinS: i before e except after c
08:45 ToinS kados: ok...
08:46 kados ToinS: a rule like all others that is broken sometimes in english :-)
08:46 kados ToinS: but correct in this case :-)
08:46 paul in french we say "it's the exception that confirms the rule"
08:46 kados :-)
08:47 paul hello tumer
08:47 tumer hi paul how did it go?
08:47 paul i'm proud to announce that I just did my 1st zebra search 10mn ago !!!
08:47 tumer zebraing?
08:47 paul (thx to kados, once again)
08:48 tumer what was the sagmentaion error?
08:48 paul some missing directories
08:48 paul it's a shame that zebra don't exit with any message
08:49 tumer along the way they fixed some other bugs as well
08:49 tumer what also annoying that it stops indexing in the middle of 150k records just besuase 1 does not have biblionumber
08:50 tumer rather tahn saying that it does not and continue with others
08:50 tumer well it takes a while to get used to this animal
08:51 paul yep.
08:51 paul I also solved some issues with utf8. I think i understand how it works in perl now...
08:52 tumer paul:the additems.pl mentiomns theasurus_category instead of Authority do you still use the thaerus.pl? in france
08:52 paul ???
08:53 tumer does the french libraries use the old thasurus structure or the new authorities?
08:54 paul no old structure, only the marc authorities
08:55 tumer s the line Dopop('../thesaurus_popup.pl?category= ... is obsolete is it?
08:55 tumer i found this line in additem.pl and dont know what it does
08:56 paul it's useless I think. there is no authorities in items. So it's a very very old copy/paste that hasn't been fixed because it's unused...
08:57 tumer k about utf8 you were saying?
08:57 paul I have exported one of my base, and it's correct utf8.
08:57 tumer do you use the new M:F:X from sourceforge with no problems?
08:57 paul I think I finally have understood how it works...
08:58 paul I have 0.83
08:58 tumer the latest version 2.0 MARC::Record
08:59 tumer yes that version
08:59 paul $VERSION = '2.0';
08:59 paul yes that's what I have.
08:59 paul and I read marcxml from biblioitems, that is utf8 already.
08:59 tumer when i do a search with accented characters in it that serach term and my template gets funny chars in it
09:00 paul with a new_from_xml($marcxml,'utf8'), it produces a MARC::Record that seems valid
09:00 paul I still haven't tried a search with something accented.
09:00 paul I have to do unimarc.abs 1st
09:00 tumer you know that it still has to be named record.abs
09:02 tumer so the base could you convert it with alter table?
09:08 kados tumer: how's the authorities integration going?
09:09 tumer kados:more involved with items now.I will look into that later
09:09 kados k
09:09 kados items as in holdings?
09:09 tumer yep
09:09 kados you looking at separating bib records and holdings?
09:09 tumer and its working
09:10 kados wow!
09:10 kados that's fantastic news
09:10 kados thd will be thrilled :-)
09:10 kados tumer: what do you think of my original idea to have just one framework system and one editor?
09:11 kados tumer: ie, instead of maintaining three separate table structures and three separate code bases, just create one capable of doing it all?
09:11 tumer well similar framework sytem for all but differnt frameworks offcourse
09:11 kados right
09:11 tumer i now have a forth
09:11 kados yea?
09:12 tumer koha_attr
09:12 kados what is it for?
09:12 tumer maps koha field names to @attr, also needed for a database with no fields in it
09:13 tumer no more @attr 1=1007 in koha code
09:13 tumer that is bound to change since we all use or may use differnt attributes to index
09:14 tumer my record.abs is very differnt to yours
09:14 tumer but now whatever yo index you can search
09:14 kados hmmm
09:14 kados we could do the same thing with ccl.properties
09:14 kados have you looked at the ccl searching?
09:14 tumer you can simply add virtual field names to everything
09:15 tumer yes but koha specific terms like datelastseen, subscriptionid etc has to be defined for KOHA
09:16 tumer now its very flexible
09:16 kados you can just add datelastseen to ccl.properties
09:16 kados and define it in terms of an @attr
09:16 kados I added one for series for instance
09:16 tumer but we still need these names in koha
09:17 kados series u=5 s=pw
09:17 kados u=5 eq 1=5
09:17 tumer search terms is not my main concern
09:17 kados ahh
09:17 kados so explain it again?
09:18 tumer mappin marc to koha we use koha database field names
09:18 kados ok
09:18 tumer author,title,biblionumber etc
09:18 kados right
09:18 tumer these field now do not exist
09:18 kados (though I have some question why we do this now except for special cases)
09:19 kados do we need the koha fields except for some like itemtype and biblionumber?
09:19 tumer we well i do not have those fields now
09:19 tumer but all of koha needs them
09:19 kados hmmm
09:20 tumer where to show title where the biblionumber etc
09:20 tumer if you remember we are multimarc compliant
09:21 tumer i cannot say field245 for title
09:21 kados right
09:21 kados ok, I see
09:21 tumer i have to say title so that whatever your marc title is you map it
09:22 kados I see
09:22 kados but in MARC, it's not just one field :-)
09:22 tumer zebra also supports 1=title
09:22 tumer so we do not need ccl
09:23 kados it's a syntax language that is more user-friendly than cql
09:23 tumer well as a preference ok
09:23 tumer but behind screens pqf
09:24 kados not sure about that
09:24 kados display is different than search
09:24 kados we need one way to specify how to display 'title'
09:24 kados and another to specify how to search it
09:25 kados for searching, it is easier to maintain the ccl.properties file IMO
09:25 tumer well what i am descrbing combines them and leaves it to your choice
09:25 kados I see
09:25 tumer you will see it when i commit
09:25 kados but 1=title is quite different than ccl's ti=something
09:25 kados ok
09:25 tumer it was just another example of searching
09:26 tumer koha search engine should support all
09:26 kados agreed
09:27 tumer but currently for internal works of koha i need to get record with biblionumber xxx
09:27 tumer you mapped it to identifier standart
09:28 tumer i mapped it to Number-db
09:28 tumer what i wrote is flexible to find both
09:28 kados gotcha
09:29 tumer so koha language all mapped like ccl
09:29 kados tumer: can you start committing your 3.0 stuff to head? I'm concerned that you'll be going in directions that aren't keeping up with what other folks are doing in head
09:30 kados tumer: if you keep your stuff in sync at all times it will be much easier in the long run
09:30 kados ie, the goal is for everyone to run off of stock CVS
09:30 tumer that was my concern when i mentioned about all this
09:31 tumer current head and all this -- very dubious
09:31 kados well you can ignore the zebra stuff chris and I wrote for head
09:31 kados in Biblio and Search modules
09:31 kados we've got much better methods now
09:32 tumer well biblio is complete re-write
09:32 kados tumer: do you know how to 'merge'?
09:32 tumer no
09:32 kados tumer: perfect, that's what we need
09:32 kados tumer: a complete rewrite was my goal for 3.0
09:32 kados tumer: with a new API
09:32 kados tumer: but if you don't commit I wont' be able to review what you've done :-)
09:32 tumer weekend
09:33 kados tumer: and it will be difficult later to merge your stuff with head
09:33 kados fair enough
09:33 tumer i will also write a detailed explanation of it all
09:33 tumer how do i merge?
09:34 kados in linux:
09:34 kados hmmm
09:35 kados not sure if you're not working with two different cvs trees
09:35 kados maybe ask chris
09:35 kados you could use diff/patch
09:35 kados but with a complete rewrite that could get messy
09:36 kados yay
09:36 paul oops...
09:36 paul 16:36:23-13/07 zebraidx(8760) [warn] Record didn't contain match fields in (bib1,Identifier-standard)
09:36 paul 16:36:23-13/07 zebraidx(8760) [warn] Bad match criteria
09:36 kados missing 090 :-)
09:36 paul means 1 090 is missing right ?
09:36 tumer right or reading wrong file
09:36 kados for me, the missing090 script doesn't always work 100%
09:37 kados because i think export.pl sometimes doesn't assign a 090 field
09:37 kados in rel_2_2 code
09:37 kados I haven't investigated
09:37 paul I probably have done things in a wrong order.
09:37 kados what I do is pre-process records
09:37 kados ahh, that might do it too :-)
09:37 paul + i'm reading from marcxml
09:37 paul and not from iso2709
09:37 kados hmmm
09:37 kados why?
09:37 tumer paul wont work
09:38 paul (i mean i've modified MARCgetbiblio to read biblioitems.marcxml instead of biblioitems.marc
09:38 kados why?
09:38 paul because biblioitems.marc is not correctly encoded
09:38 tumer they all have to be single xml records then
09:38 paul where marcxml is OK
09:38 kados hmmm
09:39 tumer paul are they single records each?
09:39 kados maybe we need a syspref for this
09:39 paul tumer: yes they are.
09:39 kados 'whether to store natively in xml or marc'
09:39 tumer then you do not use -g flag then
09:39 paul but biblioitems.marc is a blob, so the move to utf8 in mysql changed nothing
09:40 paul where marcxml is text, so the move iso => utf8 worked fine
09:40 paul (my db was latin1, declared as latin1)
09:41 kados interesting
09:41 tumer i was trying to say that latin1 to utf8 is up the ladder and should work
09:41 kados how did you populate biblioitems.marc?
09:42 paul with updatedatabase
09:42 tumer paul your zebraidx line is it correct? no -g flag?
09:42 kados tumer: he's not using zebraidx with xml
09:42 paul that reads marc_subfield_table
09:44 paul tumer: I use zebraidx with iso2709 datas. but the iso2709 comes from export/export.pl, that uses Biblio.pm/MARCgetbiblio. and I have modified MARCgetbiblio to read data from biblioitems.marcxml instead of biblioitems.marc
09:44 paul that solves my encoding problems, but it seems MARCmodbiblio don't handle marcxml
09:44 tumer paul:if you index like that it will take hours
09:44 paul ???
09:44 tumer export your xml as iso
09:45 paul I create a single iso2709 file, that can't be faster isn't it ?
09:45 tumer is01709 and zebraidx on command line
09:45 tumer 1000 times faster
09:45 paul yep, that's what I do.
09:45 tumer great
09:45 paul but I explain how I generate my iso2709
09:47 tumer if you have ugraded your latin1 base to utf8 when all the data was text then created a marc blob and put them in it it should
09:47 tumer do the same thing
09:48 paul yes, but I ran missing090 AFTER
09:48 paul so, marcxml has not been updated I suspect
09:48 tumer why doe that chane encoding?
09:49 paul ???
09:49 tumer why does it change encoding?
09:49 tumer all that does is to chech 090c
09:49 paul it does not. it's just that I now have utf8 recognized as utf8
10:26 kados morning owen
10:26 owen Hi kados
10:26 owen How's it going?
10:28 kados pretty good
10:28 kados having trouble focusing today :-)
10:29 kados paul: still there?
10:29 dewey there is a minor diff in <div>s, that I missed
10:29 owen dewey, what on earth are you talking about?
10:29 dewey owen: i haven't a clue
10:30 owen That's what I thought
10:30 kados paul: I took a look at adding a wrapper for the old search API yesterday
10:30 kados paul: before I delve deeply into it, I wanted to give you the opportunity
10:30 kados paul: since you are more familiar with the old API :-)
10:31 owen kados: is that why searching isn't working in the intranet?
10:31 kados there is a new search API in dev_week
10:31 kados opac-zoomsearch.pl uses it
10:32 kados but the old search API is used everywhere else
10:32 owen So the old search doesn't work at all anymore in dev_week
10:32 kados so yes, that's why :-)
10:32 kados not currently
10:32 kados but it would be pretty simple to create a wrapper
10:32 kados for the new search
10:32 kados so that the old search did work
10:32 kados all it would have to do is translate the old API to ccl (very simple)
10:33 kados and retrieve the results in the old style
10:33 kados (also pretty simple)
10:33 kados I could probably get that done today if paul doesn't want to do it
10:33 kados the other option we have
10:33 kados is to completely overhaul all the old search references in dev_week
10:34 kados update them to use ccl
10:34 kados that would probably be more work
10:34 kados before fully integrating the search
10:35 kados to avoid extra wrok :-)
10:35 kados work even
10:37 paul kados : it's almost 6PM here, and i'm still investigating zbera migration.
10:37 paul so if you have enough time to work on a wrapper, feel free.
10:38 kados paul: ok, I'll do it and commit this afternoon
10:38 paul kados & tumer : is there a way to say to M:F:X something like "ignore this if it is illegal".
10:38 kados paul: and release 2.3.
10:38 kados 1
10:38 paul because i've some records with a subfieldcode='>'
10:38 kados yes
10:38 paul that is illegal :
10:38 paul !
10:38 kados MARC::Charset->ignore_errors(1);
10:38 kados ahh
10:38 kados might not work for those cases
10:39 tumer not that i know only ignore encoding errrors
10:39 paul yep, it's not an encoding problem
10:39 kados you need to preprocess the records
10:39 kados and fix those codes
10:39 paul the problem being i have sometimes a capital letter too...
10:39 kados ouch
10:40 paul but OK, i'll try a regexp to remove everything that is not a..z0..9
10:41 kados owen: have we tested opacnav recently? is it working?
10:41 owen I don't know... did you work on it?
10:41 kados I think it was a db prob, it should be fixed
10:48 owen Still broken. But let's not mess with it right now on the slim chance that someone's trying it out
10:50 owen kados: got a question for you about members.
10:51 kados sure
10:51 owen Previously we'd tried to mess with how we used the members database.
10:51 owen We decided we'd use the street address field as "mailing address line 2"
10:51 owen I think it was working for a long time and got broken in the last upgrade.
10:51 kados hmmm
10:51 owen Doing it that way requires the template to be customized
10:52 owen What we /really/ need is an actual field for "mailing address line 2" and "street address line 2"
10:52 kados I'm not sure exactly what the requirements are
10:53 kados we have a mailing address currently
10:54 kados tnb: ok I see Address Line 1 and Address Line 2
10:55 kados woops :-)
10:55 kados in the liblime demo
10:55 kados I see it differently
10:55 kados http://koha.liblime.com/cgi-bin/koha/members/memberentry.pl?borrowernumber=9068&modify=edit
10:56 kados there are two address options there
10:56 kados mailing and street
10:56 kados owen: so what is npl trying to do?
10:57 owen I'd like to see 2 lines available for each address: Address Line 1 and Address Line 2.
10:57 owen So you could do: Address Line 1: P.O. Box 235, Address Line 2: 5 Jacobs Street
10:57 kados I see
10:58 kados is it the first line from each?
10:58 kados is that the customization?
10:58 owen NPL's Address Line 2 is actually mapped to Street Address Address, I think.
10:58 owen At least it's supposed to be.
10:58 kados yea, I figured
10:58 kados hmmm
10:59 kados we'd have to add new columns to the db
11:00 owen Yeah, I know
11:00 paul owen/kados : do you want me to summarize what is in head about members management ?
11:00 paul (from SAN)
11:00 owen sure, paul
11:00 paul SAN (& me) have modified the table structure to have 2 addresses.
11:01 paul they are called :
11:02 paul streetnumber / streetype / address / address_2 / city / zipcode / email / phone /
11:02 paul and the 2nd one is called : B_...
11:03 paul if you look in head/updater/updatedatabase you'll see which field has been renamed as which field
11:03 paul we tried to do things with care, so no data should be lost in the process.
11:03 paul does that make sense ?
11:04 owen Yes
11:04 paul streetnumber & streettype appears only if the (new in head) streettype table is filled.
11:05 paul SAN want to have a list for rue, avenue, boulevard, chemin,...
11:05 paul I don't think most libraries will be interested by this feature.
11:07 owen So kados it sounds like it's something I should just wait for.
11:13 kados owen: ok :-)
11:17 owen I guess if my hack hasn't even been working for at least a year there's no sense in piling more on... We'll just have to try to grab the existing data properly when 3.0 rolls around.
11:17 kados sounds like a plan
11:17 kados lets add it to a list somewhere so I don't forget about it :-)
11:18 paul zebraidx on my 1x000 biblios, seems to work fine with my updated xml datas...
11:18 kados owen: I didn't see any commits to dev_week yesterday
11:18 kados paul: great!
11:19 kados owen: did you make some changes to the tempates/scripts?
11:19 owen kados: I have several things saved locally that I need to commit today
11:19 paul what kind of commits can I do on dew_week ?
11:19 paul (what I modified may break some things...)
11:19 kados what did you modify paul?
11:20 paul in Biblio.pm, I did some modifs to support marcxml instead of marc (in biblioitems)
11:20 kados ok, lets wrap that in a syspref
11:20 paul do you confirm you also have a biblioitems.marcxml field ?
11:20 kados yes
11:20 paul 18:20:43-13/07 zebraidx(9415) [log] Records: 10867 i/u/d 9885/982/0
11:20 paul 18:20:43-13/07 zebraidx(9415) [log] user/system: 10839/115
11:20 kados but I don't use it
11:20 paul 18:20:43-13/07 zebraidx(9415) [log][app2] zebra_stop
11:20 paul 18:20:43-13/07 zebraidx(9415) [log] zebraidx times: 122.41 108.39 1.15
11:20 paul done ;-)
11:20 kados :-)
11:20 kados very fast :-)
11:20 kados I don't use marcxml
11:21 paul very fast ? I work on my laptop, so it's a 5400TPM ide disk.
11:21 paul will be faster on my SCSI ;-)
11:21 kados :-)
11:21 kados don't forget to commit too :-)
11:23 tumer[A] the only reason i had mzrcxml in dev week was to be able to do keyword seraches in SQL as backup
11:23 tumer[A] i am dropping marcxml in 3 is that ok
11:23 paul for example, removing invalid subfieldcodes was really easy
11:24 paul just 1 regexp you can't do with iso2709
11:24 thd paul: are your records with subfield code = '<' corrupted or had they been originally encoded somewhere using punctuation codes for subfield names?
11:24 tumer[A] yes but the db grows very fat with them while iso footprint is small
11:25 paul kados, your opinion ?
11:25 paul hello thd.
11:25 kados I think we make a syspref :-)
11:25 thd hello paul
11:25 tumer[A] one advantage is that you dont have to convert to xml when updating zebra
11:25 paul It may have been a very old bug, from previous ILS of the library
11:25 paul or there is a bug in additem.pl
11:26 tumer[A] since we create marc from xml
11:26 kados bbl
11:27 paul kados : tomorrow is our national day
11:27 paul so we won't be here.
11:27 thd paul: was that your answer about punctuation symbols used for subfield codes?
11:27 kados paul: ok
11:27 paul so see you on monday.
11:27 paul thd : yep
11:27 kados have a good weekend paul
11:27 owen paul: will you have fireworks?
11:27 paul (on tuesday/wednesday, i'll be in nantes)
11:27 paul owen: of course.
11:28 paul (and it's very very dangerous, as marseille is so wet)
11:28 paul a big risk of fire inside the city.
11:28 paul every year the fire dept does many many work to prevent any major problem
11:31 thd paul: what problem did punctuation symbols for subfield codes create? I ask especially because some large union catalogues and other systems which would be good copy cataloguing targets use punctuation to add extra subfields to extend the standard. The large union catalogues are big enough to have their practises create an effective standard.
11:37 paul thd : MARC::File::XML die miserabily if you have an invalid subfield.
11:37 paul and thus the script that uses M::F::X
11:37 paul it seems there is no option to continue even with an error.
11:38 thd paul: we need M::F::X to work as well as MARC::Record so that it is safe to use.
11:40 thd paul: I am going to ask Ed Summers to consider making some code changes in the next few days.
11:40 paul I just wrote something on perl4lib ;-)
11:42 paul ok, time to leave for me
11:42 paul bye bye guys, i'm very happy with my day ;-)
11:42 paul (although very hot...)
11:44 thd goodbye paul