IRC log for #koha, 2007-11-14

All times shown according to UTC.

Time S Nick Message
11:21 mj hello. I am the person who asked the "zebrasrv -c" question on the mailinglist, and received the tip to try "zebrasrv -f" instead from Henri-Damien
11:22 mj thanks very much, that seems to work much better, however...
11:22 mj now I'm getting: 11:23:43-13/11 [fatal] /usr/share/koha/etc/zebradb/zebra-biblios.cfg:29 missing colon after 'passw.passwd'
11:23 mj any ideas here what else i missed?
11:23 mj I promise: when I have all this up-and-running, I'll create a wiki page with detailed instructions :-)
12:20 hdl ùj hi
12:20 hdl mj
12:21 hdl consider commenting the passw.passwd line in zebra-biblios.cfg
12:21 hdl This is owed to a change in config syntax
12:38 fbcit g'morning hdl
12:38 hdl hi fbcit
12:40 fbcit I was testing changes I made to the installer and noticed mysql complaining about dup keys in the sysprefs.sql
12:40 fbcit specfically
12:41 fbcit line 49
12:42 fbcit 'marcflavour'
12:42 fbcit However, doing a select on systempreferences.variable = 'marcflavour' returns only one record...
12:46 fbcit hdl: I thought this might have something to do with the marc flavor changes
12:47 hdl if mysql complains about duplicate key value, then it doesnot add.
12:47 hdl Oh... Yes.
12:48 fbcit is it possible that the row is inserted by an earlier script and duped in the sysprefs script?
12:50 fbcit grep'ing only shows that one occurrence though...
12:53 fbcit looks like it would insert in spite of the complaint since prior to the sql import we SET FOREIGN_KEY_CHECKS=0
13:00 fbcit looking at the structure of systempreferences, it does not make any sense. sysprefs.sql should import without error...
13:05 hdl fbcit-away:
13:05 hdl I changed to insert marflavour when one was chosen.
13:05 hdl This makes the problem occur.
13:06 hdl In fact, systempreferences.sql could be updated in order not to insert marcflavour systempreference.
13:10 hdl But Problem would be if no choice of marcflavour is proposed (For french it is the case). Then marcflavour should be imported from systempreferences.sql
13:10 hdl fbcit-away: hope you shall get this message soon.
13:31 kados hiya owen
13:32 owen Hi
13:32 kados hdl, fbcit-away  I think I have a solution to the difficulty of picking how to have a different record.abs between unimarc and usmarc
13:32 kados hdl, fbcit-away the solution is to switch from ABS to DOM :-)
13:33 kados[…]om_style_indexing
13:34 hdl kados : marcflavour is not only for indexing, it is also related to framework building.
13:35 kados hdl: right, but for zebra it's just indexing
13:36 hdl but for install, it is the problem I spotted ;) ... friendly
13:36 kados do we use record.abs for anything except zebra indexing?
13:39 kados I think there are aspects of the Koha frameworks that could fit into the zebra xslt DOM file
13:40 mj hdl: thanks for the tip.
13:42 hdl kados : Sure.
13:43 hdl But at the moment, we need a release don't we ?
13:43 kados yep
13:43 kados but it's a programatic change
13:43 hdl changing all the stuff for DOM is not a vital priority.
13:43 hdl I agree with this change.
13:44 kados well the question is, how do we handle unimarc vs usmarc
13:44 kados in the installer
13:44 kados (not the web installer)
13:44 hdl tumer already did it.
13:44 kados if we switched to DOM we could have both, and make the choice in the web installer
13:44 kados if we use ABS we have to modify Makefile to accomodate for the choice
13:44 kados so which is easier?
13:45 hdl How would you have both in DOM ?
13:45 hdl See you soon
13:45 kados hdl: because in the DOM you could have the choice stored in a syspref
13:47 kados hmmm, guess not
13:47 kados it would have to be in zebra.cfg
13:52 kados but we could have multiple extract pipelines
13:52 kados The <extract> pipeline takes documents from any common DOM XML format to the Zebra specific indexing DOM XML format. It may consist of zero ore more <xslt stylesheet="path/file.xsl"/>  XSLT transformations, and the outcome is handled to the Zebra core to drive the process of building the inverted indexes.
13:53 kados which could be set up ahead of time presumably
14:43 hdl kados : would the BIG xslt containing ALL the frameworks be processed once for all  (when setup is complete or marcflavour chosen) ? or should xslt processed at each connexion (This would take time.)
14:43 hdl ?
15:00 hdl kados ?
15:00 hdl Does any of your customer want to limit results to some subset ?
15:01 hdl (restricted search)
15:01 hdl (For instance for childhood protection or for restricted areas)
15:03 owen paul_ around?
15:04 hdl not today
15:04 owen hdl, maybe you can help me understand bug 1513?
15:07 hdl owen : have you tried to add a biblio with only title and doctype mandatory ?
15:07 owen No
15:08 hdl If you click on field 245 for instance, then mandatory subfields fades away and optional ones are displayed.
15:09 hdl Which is NOT the expected behaviour : should be all subfields displayed and not only optional ones.
15:09 owen I can't reproduce this problem with the default framework I have. Do you have any suggestions?
15:10 hdl Can I get on your system ?
15:26 fbcit hdl: in the case of marcflavour, the second insert takes precedence, right? (that one in sysprefs.sql)
15:29 fbcit really we can do one of two thing, imo...
15:30 hdl shouldnot be that way. Second insert should be and is discarded
15:31 fbcit hdl: sorry, but I have to run... I'll get back later.tnx
15:31 hdl ok
15:36 fbcit hdl: sorry about that...
15:38 fbcit hdl: but the error is carried out to the user... should it be suppressed?
15:39 fbcit or can the first insert be made conditional on the language choice?
15:39 kados hdl: regarding your question, yes, we have a customer that wants that
15:39 hdl Maybe we should insert IGNORE rather than only inserting
15:40 fbcit that would be true if the second always takes precedence (which it seems to me to... :-)
15:41 fbcit the first is just there in case there is not a second... right?
15:47 fbcit hdl: I notice that in the fr/mandatory there is unimarc_standard_systemprefs.sql which also inserts marcflavour
15:47 hdl yes
15:47 hdl But french install donot have more than one marcflavour.
15:47 hdl So choice of marcflavour is skipped.
15:48 hdl It is english marcflavour choice which raises this error.
15:48 fbcit but the marcflavour preference is populated by unimarc_standard_systemprefs.sql so why do we need to do the insert in
15:51 hdl fbcit: If unimarc_standard_systempreferences.sql in english adds a marcflavour UNIMARC, then inserting marcflavour line in is nonsense.
15:51 hdl *in english*
15:52 hdl But if we have only one systempreferences.sql for english ppl, then marcflavour would be wrong sometimes (when someone chooses UNIMARC)
15:52 hdl This is why I added this line in
15:53 fbcit ahh... I see...
15:55 fbcit I wonder if we can move that insert into the marc21 framework & unimarc framework sql rather than doing it in
15:55 fbcit those scripts will be selected based on the user's choice...
15:56 fbcit at the top of each we could insert the appropriate preference...
16:27 hdl fbcit: yes. It may be the most appropriate way. But it would need documentation and much caution from devs.
16:27 hdl Since exporting frameworks from bases is quite easy and automatic.
16:28 kados ok, this may sound strange
16:28 hdl Adding one line from a different table.
16:28 kados but what if we solved this by having different databases for unimarc vs marc21
16:28 hdl kados ????
16:28 kados ie, biblios-marc21 vs biblios-unimarc
16:28 kados then the syspref could be used
16:29 kados the Makefile woudln't need to know about the choice up front
16:29 kados it would just install both databases
16:29 kados and let the user choose which one to use
16:29 kados does that make sense?
16:29 hdl imho, it would be crazy and nonesense.
16:30 hdl you would create a database just for 'fun'.
16:30 kados well, not really, it wouldn't ever be initiatlized
16:30 kados it would just be a directory
16:30 kados that happened to not have anything in it
16:32 kados the problem right now with installation is that:
16:32 hdl kados: i really donot see why we should even create a ghost database.
16:32 kados 1. we want to separate library choices from system choices
16:32 kados 2. the system person won't know what marcflavour to use
16:32 kados 3. the librarian has no way to set up zebra's databases
16:32 kados so if we create two dirs, then the librarian can choose with a syspref which one to use
16:33 kados maybe
16:34 kados hdl: what's your idea to solve this problem?
16:35 fbcit hdl, kados: I moved the pref insert as I mentioned and it works fine...
16:35 fbcit minimizes changes...
16:35 kados fbcit: moved it where?
16:36 fbcit to the start of the marc21 and unimarc sql scripts...
16:36 kados ok
16:36 kados here's the real issue
16:36 kados when you run Makefile.PL
16:37 kados it sets up zebra for you
16:37 kados it puts record.abs in the zebradb/biblios/etc/ dir
16:37 kados and the record.abs it puts in there is a marc21 one
16:37 kados that doesn't work for hdl
16:37 kados he needs a record.abs that is unimarc-specific
16:37 hdl kados : Does it setup a zebra-biblios.conf ?
16:37 kados hdl: yep
16:38 kados hdl: it sets up everything
16:38 fbcit so why not have the webinstaller do it all?
16:38 hdl Ok.
16:38 kados because the web installer doesn't have write access to those dirs
16:38 hdl I donot have time enough to investiguate in zebra installation script.
16:38 kados at least it doesn't as currently designed
16:38 hdl But
16:39 kados hdl: is unnecessary for installation with Makefile.PL
16:39 kados although perhaps it could be useful for re-indexing
16:39 fbcit I need to study the zebra config stuff more closely.
16:39 hdl I think that we donot use zebra configuration files possibility at their top level.
16:39 hdl I mean :
16:40 hdl I think that there could be two *.abs files in tab dir.
16:40 kados s/tab/etc/
16:40 hdl And zebra-biblio.conf could settle which one to use.
16:41 kados hdl: ok, but who has write access to zebra-biblio.conf?
16:41 hdl record.abs in in tab
16:41 kados tab is now called etc
16:41 kados koha/etc/zebradb/
16:42 fbcit Makefile.PL could tell zebra where zebra-biblio.conf is?
16:42 kados hdl: I think the issue here is that you and paul hav a different process for installing zebra than the rest of us
16:43 kados hdl: take a look at koha/etc
16:43 kados hdl: you will find koha-conf.xml and koha-httpd.conf that Makefile.PL use
16:44 kados hdl: slef has worked hard to conform to the FSH for all naming conventions
16:44 kados inside zebradb we have an etc with all of zebra's config files
16:44 kados and in zebradb/biblios/etc we have the database-specific config files
16:44 hdl paul and I works with UNIMARC it has nothing to do with our way to install.
16:44 kados File
16:44 kados
16:45 hdl If you have to set up a UNIMARC library, you will require a UNIMARC record.abs
16:45 kados hdl: the install process for you and paul is documented on the wiki, right?
16:45 kados hdl: exactly
16:45 hdl your install process is documented ?
16:45 kados hdl: so what I'm trying to say is, we need a way to make it possible to install a unimarc record.abs in the main project installer
16:46 kados hdl: the install process is 'make' 'make install'
16:46 kados hdl: that's it
16:46 kados the Makefile.PL does everything
16:46 kados there is zero zebra config necessary
16:46 kados perl Makefile.PL; make; make install
16:46 hdl Yes. But info tech Guy has no way to choose between UNIMARC and usmarc.
16:47 kados exactly
16:47 kados so that's what we're trying to resolve
16:48 hdl And zebra requires a different configuration file if it is UNIMARC or usmarc.
16:48 kados hdl: more than record.abs?
16:48 hdl sortstrings.chr
16:48 hdl words.chr
16:48 hdl ...
16:48 kados yep, so the .abs and the .chr files
16:48 kados anything else?
16:48 kados (I think that's it)
16:49 kados so we can:
16:49 hdl if one created 2 zebra-biblios.conf files
16:49 kados 1. default to marc21 and have the web installer warn the user upon unimarc choice to tell the sysadmin to run a script that installs unimarc files
16:50 hdl zebra-biblios-usmarc.conf
16:50 hdl and zebra-biblios-unimarc.conf
16:50 hdl Nope.
16:51 fbcit kados: are we expecting the sysadmin to run make && make install?
16:51 kados fbcit: yes
16:51 hdl (Nope was for me... Mys solution required a change in koha.xml)
16:51 fbcit and the user to run webinstaller?
16:51 kados fbcit: yes
16:51 hdl fbcit: this is what we are heading to.
16:51 kados we want to maintain a separation between system-level and library-level tasks
16:51 fbcit so the zebra stuff must be done in Makefile.PL...$
16:52 kados yep, because it requires write access to the filesystem
16:52 fbcit webinstaller just needs to verify that the marcflavor the user wants is what has been installed and warn otherwise...
16:53 kados yea
16:53 kados maybe that's the approach to take
16:53 kados we can use a new variable for the makefile
16:53 fbcit too bad kohastructure is not in place during Makefile.PL...
16:53 kados __MARCFLAVOUR__
16:53 kados defaults to marc21
16:53 fbcit we could insert the marcflavor pref there....
16:54 kados when the web installer is run, it checks the user's choice vs the system choice
16:54 kados and if they dont' match, it warns the user to have their sysadmin fix it
16:54 fbcit and refuses to run....
16:54 fbcit ?
16:54 kados maybe
16:54 kados well, probably not
16:54 fbcit as long as the descrepancy exists?
16:54 kados because other still will work fine
16:54 fbcit sp...
16:55 kados just not searching
16:55 fbcit k
16:55 kados maybe have a big warning on the staff side though
16:55 fbcit so marc only comes into play when searching?
16:55 kados that tells the user that someting's not configured correctly
16:55 fbcit data entry will not be affected?
16:55 kados fbcit: zebra only comes into play when searching
16:55 hdl fbcit:  yes it will
16:55 kados fbcit: right, data entry will be unaffected
16:55 kados if the user chooses unimarc, the data will be saved as unimarc
16:55 hdl kados : adds 100$a
16:55 kados regardless of the system-level choices
16:56 hdl Oh yes.
16:56 hdl you're right.
16:56 kados and if the user chooses nozebra it doesnt' matter anyway
16:56 fbcit so user's pref will rule when data entry is concerned, but not when searching..
16:56 hdl But marc frameworks won't be good.
16:56 kados fbcit: right
16:56 kados hdl: yes, they will be fine, all of that is handled in the database
16:56 kados hdl: the only file-system stuff is zebra's .abs file and the .chr files
16:57 kados fbcit: does the make install process output a result txt file that says what was done?
16:58 hdl see you.
16:58 fbcit by hdl
16:58 fbcit s/by/bye/
16:58 kados cya hdl
16:58 fbcit kados: no
16:58 kados so I can't do a 'make uninstall'?
16:58 fbcit it could be made to...
16:59 fbcit I don't know, hold on.
17:00 fbcit it does no look like it.
17:01 fbcit kados: 'make uninstall' says: Uninstall is unsafe and deprecated, the uninstallation was not performed.
17:02 fbcit and koha runs like normal...
17:04 kados hmmm
17:05 kados I guess it's based on ExtUtils::MakeMaker
17:06 fbcit it is, but I am not familiar with that module myself..
17:08 kados check your syspref
17:08 kados there's one for session driver
17:08 kados I bet it's set to mysql
17:08 kados we probably need to add an option for postgres
17:08 kados or you could just change it to file
17:09 kados fbcit: /cgi-bin/koha/admin/
17:09 kados fbcit: SessionStorage
17:09 fbcit sessonstorage
17:09 fbcit mysql|tmp
17:09 kados yea, just change that to:
17:09 kados mysql|tmp|postgres
17:09 kados I think
17:10 kados or ... you could test if it's that by setting it to tmp for now
17:10 kados ahh
17:10 kados we need a block to handle postgres
17:10 kados C4/
17:10 kados around line 411
17:11 kados fbcit++
17:11 fbcit I added a library over pg last night and nothing blew chunks
17:12 fbcit changing to tmp fixed it, so I'll add the code this afternoon.
17:12 kados awesome
17:12 kados yea, on my list
17:12 fbcit :-)
17:21 owen kados, is the redirect from /search going to be done for the 3.0 OPAC ?
17:23 kados owen: no
17:27 kados fbcit: ping me when you're back
17:27 kados fbcit: I'm testing out your patches
17:27 kados fbcit: but I may be missing some
17:27 kados (I think I'm missing the 'select your database type' one
17:28 kados it's just finding my mysql db
17:28 kados and yea, I'm stuck on step 2
18:21 kados fbcit: back yet?
18:21 kados fbcit: I'm seeing one bug here with these patches
18:21 kados fbcit: I don't get the option to select a marc flavour (it asks me to select, but no options are listed)
18:22 kados also, step 3, doesn't list any options to select from
18:23 kados hiya
18:23 kados fbcit: see above
18:23 kados fbcit: chris just forwarded me all your patches
18:23 kados fbcit: and I applied everything
18:24 kados fbcit: and there are some bugs
18:24 kados basically I'm not prompted to select a MARC flavour, and I'm not prompted to select my 'options'
18:24 fbcit kados: you have to make the db selection in koha-conf.xml... <db_scheme>Pg</db_scheme>
18:24 fbcit also set <port>5433</port> for Pg
18:25 kados hmmm
18:25 kados I'm not testing Pg
18:25 kados I'm testing mysql
18:25 kados when I run the web installer
18:25 kados I get:
18:25 fbcit k
18:25 kados Select your MARC flavour
18:25 kados as a step
18:25 kados but nothing to actually select
18:25 kados and while I like the idea of being able to specify the port, it should work without that specification
18:26 kados for backwards compatibility
18:26 kados [Tue Nov 13 12:21:09 2007] [error] [client] [Tue Nov 13 12:21:09 2007] no open /home/jmf/koha/production/intranet/cgi-b​in/installer/data/mysql/en/marcflavour/ at /home/jmf/koha/production/intran​et/cgi-bin/installer/ line 418., referer:[…]p=3&op=choosemarc
18:26 kados ] no open /home/jmf/koha/production/intrane​t/cgi-bin/installer/data/mysql/en at /home/jmf/koha/production/intran​et/cgi-bin/installer/ line 477., referer:[…]p=3&op=choosemarc
18:26 fbcit Makefile will set the port to the default mysql if none is specified.
18:27 fbcit hold on....
18:27 kados ok, but what I mean is, we shouldn't have to specify the port in the config file
18:27 kados if that makes sense
18:27 kados that's a new parameter, not everyone will know that it needs to be in there
18:28 fbcit it should work w/o a port spec'd as installer checks for $info{port} ? "port=$info{port}":"" but that is not the problem here I think
18:29 kados fbcit:
18:29 kados you can run the installer
18:29 fbcit check for the existence of koha/installer/data/mysql/en/marcflavour...
18:32 kados fbcit: it's not there
18:32 kados fbcit: maybe I'm missing a patch?
18:32 fbcit then a patch failed
18:32 fbcit and that is why no marc choices....
18:32 kados the en stuff is still in the data dir
18:32 kados yea
18:32 kados ok, i'll see what I'm missing here
18:33 kados sorry for bothering you about it :-)
18:33 [K] *** join #koha@FreeNode: darcilicious n=plinkit@
18:33 fbcit 0002-Moved-language-dirs-one-le​vel-down-under-dbms-dir-m.patch
18:33 fbcit second patch down... ;-)
18:34 [K] *** part FreeNode!#koha: darcilicious n=plinkit@
18:34 kados fbcit: I didn't get that one
18:34 kados hmmm
18:34 kados hang on
18:34 kados I see now
18:34 kados ok, give me a sec here
18:35 fbcit I can resend it...
18:35 kados yea, I don't see that one
18:35 kados send it to
18:38 fbcit kados: one moment...
18:39 fbcit back
18:40 fbcit kados: so if no port is spec'd (ie. <port></port>) with mysql it will not matter
18:42 kados sweet
18:42 kados yea, it's all working
18:42 kados sample_labels.sql failed
18:42 kados but other than that we're all good
18:42 fbcit sample_labels.sql tries to dup a key I think...
18:43 kados ahh
18:43 fbcit havn't taken time to look
18:43 kados I thought galen pushed up a patch for that
18:43 kados but I'll take a look
18:43 fbcit git had me running scared...
18:43 fbcit :-)
18:44 kados ok, I'm gonna push this stuff up
18:44 kados fbcit++
18:44 kados so we still need to address this prob with marc21 vs unimarc
18:44 kados any thoughts since lunch? :-)
18:45 fbcit I need to spend time studying how zebra is setup and how ExtUtils::MakeMaker works
18:46 kados I can explain zebra
18:46 kados studying it probably will take you a couple weeks :-)
18:47 kados ok, here goes
18:47 kados zebra has config files and system files
18:48 kados it's very very flexible
18:48 kados so some stuff can be set globally and some stuff set per-database
18:48 kados in Koha we have two databases currently
18:48 kados one for biblios and one for authorities
18:48 kados so we have a generic global etc dir with global config files
18:49 kados and biblios and authorities have their own etc as well
18:49 kados with database-specific config files
18:49 kados if you look in koha/etc/zebradb
18:49 kados you'll see an etc dir with the global configs
18:49 kados and koha/etc/zebradb/biblios/etc has the biblios-specific config files
18:50 kados one of the files in particular is used to define how zebra indexes things
18:50 fbcit so koha/etc/zebradb/foo/etc?
18:50 kados yes
18:50 kados record.abs is the master index definition for a given database
18:50 kados (well, foo.abs really)
18:50 kados (but lets keep it simple for now)
18:51 kados so marc21 and unimarc have different indexes obviously
18:51 kados because, for instance, in unimarc 100$a isnt' author
18:51 kados whereas in marc21, it is
18:51 kados so koha/etc/zebradb/foo/etc/record.abs
18:51 kados will be different depending on the flavour of MARC
18:51 kados with me so far?
18:51 fbcit yea
18:53 kados yep
18:54 fbcit so 'abs' is just a diff format than sxml...
18:54 fbcit xslt
18:56 kados well, not quite
18:57 kados it's unfortunately a bit more complex than that
18:57 fbcit so the only diff between unimarc and marc21 is the "tag" (ie 100$a)?
18:57 kados yea
18:58 fbcit k
18:58 fbcit our problem is how to decide which record.abs to install...?
18:58 kados exactly
18:58 fbcit and other associated config files...
18:58 kados my thinking is
18:58 kados yea, there are only two others
18:58 kados if we solve the prob for record.abs we can solve it for the others
18:59 fbcit with what I know at this point, I like the idea of an env var passed to Makefile.PL
18:59 fbcit and webinstaller checks as we discussed earlier...
19:02 kados yea, I like that idea too
19:02 kados we could almost store that in the config file
19:02 kados rather than in the systempreferences
19:02 kados but that would require a fair amount of refactoring
19:03 fbcit koha-conf.xml contains settings the user should not be able to change, right?
19:03 kados correct
19:04 fbcit I'm for leaving it in the db for now...
19:09 fbcit kados: is marc21 default or unimarc?
19:09 kados marc21
19:10 kados unimarc is pretty localized to france
19:10 fbcit so we add ZEBRA_MARC or MARC_TYPE
19:10 fbcit ?
19:10 kados MARCFLAVOUR I think
19:10 fbcit k
19:12 fbcit so only files under db/etc change for marc spec? or do some globals change also?
19:12 kados globals too :-)
19:13 kados all of the .chr files
19:13 kados are marcflavour-specific
19:13 fbcit give me path/filename of what changes per marc spec
19:13 kados ok
19:13 fbcit I think this should not be too hard...
19:13 kados we just discovered that the french team has been using files in misc/zebra
19:13 fbcit famous last words.....
19:13 kados and the english team has been using files in etc/zebradb
19:13 fbcit we need to standardize I think... :-)
19:14 kados yep
19:14 fbcit off-topic: what is the release time-frame for Koha3?
19:14 kados so the unimarc record.abs is in koha/misc/zebra/record_biblios_unimarc.abs
19:14 kados for biblios
19:14 kados for authorities its
19:14 kados so the unimarc record.abs is in koha/misc/zebra/record_authorities_unimarc.abs
19:15 kados but the usmarc file in that dir should be ignored
19:15 kados fbcit: 3.0 will be released when it's ready :-)
19:15 kados fbcit: this year for sure
19:15 kados asap basically
19:15 kados misc/zebra/sort-string-utf_french.chr
19:15 kados is the french version of
19:16 kados ../../etc/zebradb/etc/sort-string.chr
19:17 kados it's just three files :-)
19:17 kados and use the naming conventions from etc/zebradb
19:17 kados not misc/
19:17 fbcit I assume these path/names are referenced in zebra-foo.cfg?
19:18 kados correct
19:18 kados well
19:18 kados in a round abotu way
19:18 kados sort-string.chr is referenced in default.idx which is referenced in bib1.att which is referenced in zebra-foo.cfg
19:18 kados :-)
19:19 fbcit so really the fr stuff needs to be made to match what exists in etc/zebradb
19:20 kados yes
19:20 kados those three files
19:20 fbcit so *just* those three?
19:20 kados yep
19:21 kados if we move those files to etc
19:21 kados we can delete misc/zebra altogether
19:21 fbcit once that part works, we will work on the webinstaller part...
19:22 fbcit and do Pg port stuff in the cracks between...
19:22 fbcit kados: sound ok?
19:23 kados yea, sounds perfect
19:24 kados fbcit: 3.0 is likely stable enough, or will be by the end of this week, to run in production
19:25 kados fbcit: the liblimer's are doing code auditing this week of all the circ, searching, cataloging stuff
19:25 kados fbcit: your help with the installer is most appreciated
19:25 owen kados, I was curious about the spelling suggest feature. When I look at the kohaspsuggest preference, its explanation is "Track search queries, turn on by defining host:dbname:user:pass"
19:25 kados owen: yea, that's what NPL has
19:26 kados owen: I ported it to 3.0
19:26 owen Why "track search queries" ?
19:26 kados owen: it's just part of the functionality of that feature
19:27 kados owen: it tracks 'successful' queries and ranks them as part of the suggestion feature
19:27 owen I see.
19:27 owen Should that be a YesNo pref? Right now it's got a Textarea
19:28 kados no
19:28 kados see the syspref description :-)
19:28 kados ' turn on by defining host:dbname:user:pass'
19:28 owen Oh, sorry, I wasn't thinking
19:28 owen Why doesn't it know those values?
19:30 kados I didn't add the sql to kohastructure.sql for 3.0
19:30 kados it's a separate 'service'
19:30 kados ie, not part of koha proper
19:30 kados it's generic enough that it could be implemented for anything that needs spellchecking
19:32 fbcit kados: in zebradb/etc I see sort-string-utf.chr...
19:32 fbcit is this eq 'sortstring.chr'
19:32 fbcit is this eq 'sort-string.chr'
19:35 kados yes
19:37 fbcit kados: so I'm going with -marc21 and -unimarc suffix's... see any problems?
19:39 kados ok, sorry, had a phone call
19:39 kados so this is interesting
19:40 kados the *.chr file isn't really a unimarc vs marc21 thing
19:40 kados it's more a language thing
19:40 kados in fact, it's entirely a language thing
19:40 kados lemme check the frenh one
19:41 fbcit k
19:41 kados ok, right
19:41 kados it's language-specific
19:41 kados the important differences between french and english
19:42 kados are the articles
19:42 kados in France, articles are Le, La, Les, etc.
19:42 kados in English, The, the, a, A, An, etc.
19:42 kados and we ignore those for sorting purposes
19:42 kados this is interesting
19:43 fbcit I see.. hence the current suffix '_french'
19:43 kados yea
19:43 fbcit these are stopwords?
19:44 kados stopwords specifically for sorting
19:44 kados if you look in default.idx
19:44 kados that's where the index types are defined
19:44 kados one type is 's' for 'sort'
19:44 kados and that's the charmap that's defined there
19:44 kados sort-string-utf8.chr
19:44 kados there's a turkish one floating around as well
19:44 kados that we should add
19:45 kados and I'll have an arabic one by the end of next week
19:46 kados this is what makes zebra so great and at the same time so hard to set up
19:47 fbcit so there is a diff default.idx per language?
19:48 kados perhaps that's the best way to approach it
19:48 kados well
19:48 kados it depends
19:48 kados we might be able to do it with symlinks
19:48 kados ie, symlink sort-string.chr to the appropriate language .chr file
19:48 fbcit we could setup for multi-lang and wrap the marc issue up inside
19:48 kados ditto with the record.abs
19:48 fbcit interesting...
19:50 kados that'd be very apacheish
19:51 fbcit so have Makefile.PL build the correct symlinks?
19:51 kados in theory that should work
19:52 kados and we could have a little utility to change them
19:52 kados kinda like a2enmod
19:52 kados does that make sense?
19:52 kados we should test to make sure that zebra won't complain if we use symlinks
19:52 fbcit in theory... ;-)
19:53 kados no sense writing code for something that doesn't work
19:53 kados it's too bad the web installer couldn't have write access, this would be sooo much easier
19:53 fbcit can't we do 'su' inside a perl script?
19:53 fbcit or sudo?
19:54 fbcit the password would be an issue....
19:55 fbcit otherwise 'qx(sudo foo)'
19:55 kados yea
19:55 fbcit where 'foo' does the file moves?
19:56 fbcit might be somewhat dangerous, though.
20:01 fbcit lrwxrwxrwx 1 root root    6 2007-11-13 14:51 default.idx -> us.idx
20:01 fbcit fbcit:/usr/share/koha/etc/zebradb/etc# zebrasrv -c /usr/share/koha/etc/koha-conf.xml
20:01 fbcit 14:52:14-13/11 [log] zebra_start /usr/share/koha/etc/koha-conf.xml 1.3.50
20:01 fbcit 14:52:14-13/11 [server] Adding dynamic listener on tcp:@:9999 id=0
20:01 fbcit 14:52:14-13/11 [server] Starting server zebrasrv pid=19409
20:01 kados sweet
20:02 kados but lets keep default.idx as is
20:02 kados and create symlinks for the .chr files
20:02 fbcit k
20:04 fbcit lrwxrwxrwx 1 root root   22 2007-11-13 14:54 sort-string-utf.chr -> sort-string-utf-us.chr
20:04 fbcit zebra says the same as before
20:05 fbcit kados: shall we wrap the marc issue up inside the language issue?
20:05 fbcit or keep it separate?
20:06 kados separate
20:06 kados because, for instance, there are english users of unimarc
20:07 fbcit I am thinking of an altered dir structure... ie zebradb/etc/lang/en... to hold language specific files
20:08 fbcit or is this too much?
20:08 fbcit zebradb/biblios/etc/lang/en and biblios/etc/marcflavour/marc21
20:09 kados no, that's fine
20:28 fbcit kados: the default lang is en, right?
20:28 kados yes
20:29 fbcit so why don't we build sym links in which default zebra to en and marc21, then let Makefile.PL write new ones if specified?
20:29 kados sounds good
20:29 kados can we get git to store symlinks?
20:29 kados also, that approach won't work on windows
20:29 fbcit I thought of M$ earlier...
20:29 kados arrg
20:29 kados M$--
20:30 fbcit I think that NTFS does some sort of symlink....
20:30 kados but we do have a big Koha Windows contingency
20:30 fbcit ouch
20:31 fbcit kados: from google:
20:31 fbcit NTFS 5.0 supports what they call junctions, which are similar to
20:31 fbcit symbolic links, on an NTFS filesystem.  They also allow hard links,
20:31 fbcit which are essentially the same as hard links in Unix.
20:32 kados yea
20:32 kados but most koha windows users aren't running ntfs I'm guessing
20:32 kados I woudln't want to tie our design choices to a specific os
20:32 kados I mean ... filesystem
20:32 kados arrg
20:32 kados unless we just decide this installer i just for *nix
20:32 kados and the windows package maintainer has to make up his own stuff :)
20:33 kados yea
20:33 kados and adding a big warning somewhere to the web installer
20:33 kados to explain what needs to be done
20:34 fbcit I have played with windows istaller.... yuk!
20:34 fbcit installer
20:34 kados :-)
20:34 fbcit at some point we could write a Makefile.PL eq in msi...
20:36 fbcit the wisdom of #git says that git will track symlinks.
20:51 fbcit kados?
20:55 atz he's in a brief meeting, presently
23:55 kados hiya fbcit-away
23:55 kados hiya fbcit-1
23:55 kados fbcit-1: pushed up your patches just now
23:58 fbcit-1 I saw.
23:59 fbcit-1 I'm trying to figure out how to have make write symlinks.
00:08 atz ok I granted select on mysql.time_zone_name for rch jmf cfc frc and gmc
00:09 atz so we can test this timezones thing (on arwen)
00:14 atz we now have 1685 rows in mysql.time_zone_name !
00:17 kados wow
00:18 atz (that's how many are in /usr/share/zoneinfo
00:18 atz )
00:21 atz kados: i agree, i think we need a Localization tab
00:22 atz did you start on that, or can I do it?
00:27 kados you can do it
00:27 kados just a matter of adding it to
00:27 kados and moving all the localization things to that tab
00:27 atz right
00:27 kados dateformat
00:27 atz like date format
00:28 kados :-)
00:28 atz :)
00:28 chris and spelling it localisation :-)
00:28 kados hehe
00:28 kados chris: you brit you
00:28 chris we need to localise our localization
00:29 chris i always lose at scrabble, because we have fair less words with z in it that you guys :)
00:29 kados hehe
00:29 kados en-US
00:30 kados we were missing a ton of libraries in here
00:30 atz s/(fla|col|fav)or/$1our/g;
00:32 chris *grin*
00:34 kados still are but I added a few
00:34 kados chris: passing off to you
00:34 kados oops
00:34 kados did this one in my rmtest
00:34 kados I'll just push it up
00:41 kados fbcit-1: any progress?
00:44 fbcit-1 are you familiar with Module::Build?
00:44 kados I'm not
00:45 fbcit-1 According to the maintainer of MakeMaker, Module::Build is to replace it
00:45 kados huh
00:45 fbcit-1 Apparently CPAN depends on MakeMaker, but CPANPLUS uses Module::Build...anyhow
00:46 fbcit-1 I can't find much on symlinks and make
00:46 fbcit-1 I did the file/dir changes in my clone
00:46 fbcit-1 added the symlinks
00:46 fbcit-1 ran make && make install
00:47 kados and it's happy?
00:47 fbcit-1 make followed the symlinks and installed the actual file rather than the link
00:47 fbcit-1 zebra works, but no symlinks
00:47 kados ahh
00:48 kados well
00:48 kados that's probably OK
00:48 kados can make rename files?
00:48 fbcit-1 there must be a way to tell make to build symlinks rather than simply doing cp'g
00:48 kados that might be more portable actually
00:48 kados cp filename newfilename
00:48 kados rather than symlinks?
00:48 kados might be easier in the short term
00:49 fbcit-1 another point concerning makemaker: it is platform dependent since it uses make...
00:49 fbcit-1 M::B is not platform dependent, but is pure perl
00:49 kados right
00:49 kados hmmm
00:49 kados what would be involved in porting?
00:49 kados yea
00:49 kados that's what I'm thiking
00:49 fbcit-1 the coversion to M::B is supposed to be very straightforward.
00:50 fbcit-1 there is a makemake wrapper for it also
00:50 kados yea, this sounds like the ticket
00:50 kados I'm confused why neither of these has an uninstall option
00:51 fbcit-1 because koha should never be uninstalled... :-)
00:51 kados hehe
00:53 kados "Natively, Module::Build provides default installation locations for the following types of installable items:"
00:53 fbcit-1 I think we can have the makefile built based on lang/marcflavour
00:53 kados hmmm
00:55 kados "If you do not need to retain compatibility with ExtUtils::MakeMaker or are starting a fresh Perl installation we recommand you use "install_base" instead (and "INSTALL_BASE" in ExtUtils::MakeMaker).  See "Instaling in
00:55 kados           the same location as ExtUtils::MakeMaker" in Module::Build::Cookbook for further information."
00:55 kados which do we use?
00:55 fbcit-1 if the switch to M::B makes sense, now is the time to do it.
00:56 kados yea
00:56 kados I think this should be discussed on koha-devel
00:56 kados make sure that MJ doesn't have a reason he picked MakeMaker over M::B
00:58 fbcit-1 we may need to abandon the idea of symlinks in light of the multi-os issues (although I think they are a neat approach).
01:03 fbcit-1 interesting:
01:03 fbcit-1 One advantage of Module::Build is that since it's implemented as Perl methods, you can invoke these methods directly if you want to install a module non-interactively. For instance, the following Perl script will invoke the entire build/install procedure
01:05 fbcit-1 so running ./ prompts the admin for possible vars, supplying defaults if not overridden, and then does make && make install all in one shot.
01:09 fbcit-1 kados: we use PREFIX=
01:21 fbcit-1 kados: I posted the thought to koha-devel
01:26 fbcit-1 unless I am mistaken, uninstalling is just a matter of deleting three or so subdirs and dropping a db. That could be scripted.
01:42 kados sorry fbcit-1
01:43 kados I like the idea of ./
01:48 kados fbcit-1: did you see that weird message on koha-devel that someone's getting about a missing colon after yazgfs?
01:49 kados ahh, he was doing -c
01:50 kados and yet that works for you
01:50 kados weird
01:50 kados it should be -f though
01:50 kados fbcit-1: did you update that or should I?
01:58 kados fbcit-1: I'm gonna go wild on the README.txt :-)
01:59 fbcit-1 kados: I did not update it because it worked for me.  Sorry. You can...
01:59 kados will do
02:00 fbcit-1 sys admins have to know something as a prereq :-)
02:02 kados hehe
02:02 kados well ...
02:03 kados we'll have a chance to write up some detailed docs once we release the 3.0 beta
02:03 kados now that things are settling down should be easier
02:03 kados only I think two more features and a few enhancements/bugs holding us back
02:03 kados and a possible shift to a new installer ;-)
02:04 kados fbcit-1:  I added you to the credits page :-)
02:04 fbcit-1 tnx
02:04 fbcit-1 I need to load up sample data into pg and pound it
02:05 fbcit-1 there may not be quite as many mysqlisms as might be feared.
02:05 kados sweetness
02:05 fbcit-1 you might should mention that the pg port is alpha in case I don't get it completed in time...
02:06 kados loading data is pretty simple
02:06 kados I have a batch you could try if you're interested
02:06 kados you basically just run:
02:06 kados export KOHA_CONF=/path-to-koha-conf.xml
02:07 kados perl misc/migration_tools/bulkmarcimport -file /path/to/marc.mrc
02:07 kados then ... there are two options
02:07 kados long and easy way is:
02:08 kados (long meaning it takes forever)
02:08 kados start zebra:
02:08 kados zebrasrv -f /path/to/koha-conf.xml
02:08 kados and -D if you want to start it as a daemon
02:08 kados then:
02:08 kados perl misc/cronjobs/
02:08 kados and go get a pizza :-)
02:09 kados wget
02:09 kados that's your MARC records if you wanna try
02:09 kados there's a quick but complicated way if you're impatient
02:09 fbcit-1 I'll grab them tomorrow after gk class and give it a whirl.
02:10 kados if you have 2 minutes right now you could do it
02:10 kados wget
02:10 kados export KOHA_CONF=/opt/koha/etc/koha-conf.xml
02:10 kados perl misc/migration_tools/bulkmarcimport -file
02:10 kados ahh
02:10 kados ok :-)
02:11 fbcit-1 I was getting pounded pretty hard on ssh so I closed up the port...
02:11 kados ahh
02:11 fbcit-1 don't have time to harden that port on my firewall... sorry.
02:11 kados :-)
02:13 fbcit-1 I need to finish porting the latest kohastructure.sql to pg prior to importing that data.
02:13 fbcit-1 which brings up another *issue*...
02:14 kados hehe
02:14 fbcit-1 keeping the two db ports in sync with most devs working on the mysql side...
02:14 kados yea, I'm sorry, there have been some db changes recently
02:14 kados yea, do you have ideas for that?
02:14 kados I noticed you moved updatedatabase into the mysql dir
02:14 kados are there mysqlisms in there>?
02:15 fbcit-1 I peeked, but did not have time to really get into it... git was hot on my heals....
02:15 kados k
02:15 kados it would be ideal if we could have only generic sql in updatedatbase
02:16 kados otherwise it's gonna be a nightmare to keep everything in sync
02:16 fbcit-1 Are most db updates done both in updatedatabase & kohastructure?
02:16 kados all of them are
02:16 fbcit-1 that would be ideal
02:16 kados yep, all of them are
02:16 fbcit-1 so kohastructure changes only on major releases?
02:16 kados that's the idea
02:16 kados well ...
02:17 kados it changes all the time
02:17 kados concurrent with updatedatabase
02:17 fbcit-1 I thought that
02:17 kados and the versioning scheme ensures that only the updates you haven't got will be applied
02:18 fbcit-1 I had thought about having patches which involve sql cc'd to an email address to me so I would be sure to see those and port them...
02:18 fbcit-1 I don't like mysql's INSERT IGNORE....
02:19 kados that'd be great
02:19 fbcit-1 seems dangerous
02:19 fbcit-1 I'll setup an email addr and get it to you.
02:20 kados k
02:37 chris heh, those are pretty old
02:37 fbcit-1 which ones? :-)
02:37 chris the katipo ones are old
02:38 fbcit-1 my screen is blank...
02:38 fbcit-1
02:38 chris ahh
02:39 chris we did those in feb 2003
02:39 chris i cant even get to cafepress
02:40 fbcit-1 maybe we could embed koha in a mug that would index various international coffees... ;-)
02:40 chris heh
02:40 chris someone had koha running on their playstation
02:41 chris its a short step from there to a coffee mug :)
02:43 kados hehe, or a digital watch :-)
02:43 kados that rocked :-)
02:46 fbcit-1 I wonder if you could install it on a wap using white russian?
02:48 fbcit-1 looks like your windows port has some sort of installer?
02:49 kados yea, there's someone who maintains that separately from mainline koha's installer
02:49 kados unfortunately he uses a proprietary packaging system
02:49 kados i wonder if there is such a thing as a open source windows packaging system
02:50 fbcit-1 there is... hold on...
02:50 fbcit-1 wix...
02:51 fbcit-1 released on sourceforge by....
02:51 fbcit-1 M$
02:51 fbcit-1
02:51 fbcit-1 now... have fun... :-P
02:52 fbcit-1 I worked with it for awhile trying to work up a win32 installer for another project.
02:53 fbcit-1 it uses xml files to configure and produces msi files.
03:08 kados fbcit-1 takes over the whole installer for Koha :-)
03:14 fbcit-1 you can file a feature request... ;-)
09:31 paul chris around ?

| Channels | #koha index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary