Time Nick Message 02:37 toins hi all 04:47 toins is there someone here ? 04:52 toins :'-( 04:53 chris i am toins 04:53 chris how are you? 04:54 toins chris, fine and you ? 04:54 chris fine as well, cold but fine 04:55 toins ah yes, it's the winter for you ? 04:56 chris yep, very rainy and windy today 04:57 toins okay.... 04:58 chris what are you working on at the moment? 04:59 chris im working on a way for a librarian to choose to replace a record in the catalogue with one from the reserviour (apart from the barcode and other item specfic data) 05:00 toins i work on the OPAC 05:00 toins on head 05:00 toins with zoomsearch 05:03 chris cool 05:05 toins i'll create soon a rel_3 branch on CVS 05:05 chris fantastic 05:05 chris tumer will be happy 05:05 toins paul told me to do that 05:05 toins yep ! 05:06 chris and it will help bruno and arnaud 05:28 chris ok bedtime for me 10:07 kados hi all 10:19 owen Hi kados, didn't see you come in 10:47 kados owen: I finished all the pre-processing stuff for NPL's data 10:47 kados owen: so all the 008/007/leader fields are consistant 10:48 kados owen: for the format/content/audience searches 10:48 kados owen: I've been doing a lot of thinking about the advanced/power/proximity search as well 10:49 kados owen: and the needless complexity of the current design 10:49 kados owen: in fact, over the weekend I re-wrote the search API again 10:49 kados owen: to just just CCL 10:49 kados owen: and I'll have a new advanced search page ready in a couple hours 10:50 kados owen: it doesn't affect anything else, just the advanced search 10:50 owen leave kados alone for the weekend and he re-writes an API 10:50 kados I have one question to ask you ... 10:51 kados the new API supports federated searching on multiple Z39.50 databases 10:52 kados do you happen to know if any of our databases support Z39.50 queries? 10:52 kados (maybe ohiolink?) 10:52 kados k 10:52 kados owen plays musical nicks? :-) 10:53 owen Got booted and logged on again before the first one timed out 10:53 kados so ... this is more of a long-term question 10:53 kados like maybe after we go live 10:53 kados but we've discussed display of multiple result sets in the past 10:54 kados I favor a column design like a9.com 10:54 kados do you have any thoughts on that? 10:55 owen Haven't really thought about it... I'm not very fond of A9's columns. I find them hard to scan 10:55 owen ...but I'm not sure how else you'd handle it 10:55 kados yea, I can see that 10:55 kados well ... how does firstsearch look? 10:56 kados how do I even get to it from the library catalog? 10:56 kados ahh 10:56 kados ok, so they seem to list them vertically 10:57 kados each source is a header 10:57 kados with an index at the top 10:57 kados interesting 10:58 kados then you can click on 'next set' and it returns just that set's next results list 10:58 kados one thing I don't like about that design 10:59 thd kados: what is wrong with rows instead of columns? 10:59 kados I like the fact that with a9.com you can return the second page of just a single column 10:59 kados thd: I don't know ... not sure I have a preference 10:59 kados thd: just looking at the options 10:59 thd kados: columns have a limitation of screen width visibility 10:59 kados true 11:00 thd kados: if you have more columns than a very small number you have too many for the screen width 11:02 thd kados: what is your preference for CCL in advanced search when the user is not actually typing the query syntax? 11:02 thd s/what is your preference/why do you prefer/ 11:03 thd s/what is your preference for/why do you prefer/ 11:09 kados I'm not sure I have a preference 11:09 kados I can see advantages of both 11:09 kados maybe it should be a preference thing 11:09 owen kados: what is the status of zoomopac now? 11:09 kados well at the moment it's being migrated 11:09 thd kados: you must have had some thought to change the advanced search to CCL. 11:10 kados owen: are you asking whether you can work on it right now? 11:10 owen Yeah 11:11 thd kados: I fail to see the advantage of CCL in an web forms controlled syntax. 11:11 kados thd: I'll explain that in a second 11:11 kados owen: so ... the migration happens in stages ... first stage is to copy the NPL db to the new server's db 11:12 kados owen: that part takes about 10 hours or so and unfortunately I didn't start it until this morning 11:12 kados owen: and until the db is in place you can't really view the pages on the server :( 11:12 kados owen: it'll be back up tomorrow ... normally I update on Sunday, but I was working on the API so I needed it to be running 11:13 kados owen: and I forgot to start it last night before bed 11:13 kados owen: should be back up by tomorrow AM 11:13 owen That's fine. I just didn't fully understand your status report a few minutes ago 11:13 kados ahh 11:13 kados yea, I was just talking about the MARC pre_process script 11:14 kados the one that fills in default values for the MARC records that didn't have them (and leader values for the leaders that were wrong) 11:14 kados thd: so ... CCL 11:14 kados thd: the new API has support for field weighting and stemming 11:15 kados thd: in order to do that I had to create my own query language 11:15 kados thd: and I based it on CCL 11:15 kados thd: because CCL is fairly simple in comparison to PQF or CQL 11:16 kados thd: when I have time to write a proper compiler I'll of course use CQL 11:16 kados thd: but that will take a few months I think 11:16 kados thd: the query language I wrote has support for everything but nested queries 11:16 kados thd: ie, it's CCL - nested queries 11:16 kados thd: for the form-based queries 11:17 thd kados: unless you are rewriting the CCL parser you have less precise control over how Zebra interprets CCL 11:17 kados thd: my query language isn't strictly CCL, just based on it 11:18 kados thd: it lacks nested queries 11:18 kados thd: but it makes up for it with field weighting and stemming 11:18 kados thd: and you still have access to full CCL, CQL and PQF queries using the field-based queries 11:18 kados ie, you can go: 11:19 kados cql=(african and history) not (american or britan) 11:19 kados britain even :-) 11:19 kados or ... 11:19 kados ccl=(some ccl string) 11:20 kados or... 11:20 kados pqf=(some pqf string) 11:20 kados behind the scenes 11:20 kados the Koha query parser takes the incoming queries, either form-based or field-based 11:21 kados and translates them into CCL 11:21 kados (except for the above three cases) 11:21 kados but it adds stemming and field weighting along the way 11:21 kados so a query on: 11:21 kados it 11:21 kados becomes something like: 11:22 kados rank=(title_exact,rank1=it or title_phrase,rank2=it or title_word,rank3=it or keyword,rank4=it) 11:23 kados which means that Stephen King's book 'it' always comes up first by default 11:23 kados followed by books that have 'it' in the title, followed by books that have it anywhere in the record 11:24 kados and if you do a search like 'fishing trips' 11:24 kados it will turn that into a ranked query as above 11:24 kados except it will first stem each term 11:24 kados so you end up finding records on 'fish' 'fishes' 'fishing' as well as 'trip' and 'trips' 11:25 kados but it ranks them with 'fishing trips' first 11:25 kados followed by the others 11:25 kados thd: does that make sense? :-) 11:25 thd kados: CCL makes it more difficult to specify multiple attribute types because each attribute type used has to be specified in advance with a particular attribute value for the particular CCL command 11:26 kados thd: that just means you have to have a good ccl.properties file 11:26 kados thd: which I happen to have :-) 11:26 thd kados: you have to choose the attribute types and ranking in advance instead of allowing the user to alter the default 11:26 kados thd: not quite ... 11:27 kados thd: i thought the same thing 11:27 kados thd: but then I discovered a nice feature of CCL 11:27 kados thd: you can have more than one qualifier per query 11:27 kados thd: so you're not stuck doing just 'title=something' 11:28 kados thd: you can do: 'title,exact=something' 11:28 kados thd: so in your ccl file you just map title => 1=4 and exact => 6=4 11:28 kados thd: well ... 4=1 6=4 to be precise :-) 11:29 thd kados: and qualifiers are specified in ccl.properties? 11:29 kados thd: yes 11:29 kados thd: you can easily add a new one on the fly without re-indexing the database too 11:29 kados thd: in fact ... you can even pass a temporary mapping along with the query 11:30 kados thd: so if you had an expert user who wanted to set up their own personal ccl mapping file 11:30 thd kados: is that a client side control? 11:30 kados thd: yes 11:30 thd kados: so it can be used against remote hosts as well? 11:30 kados thd: i could add it to the OPAC as a feature, but the session management in Koha is currently borked 11:30 kados thd: yes 11:30 kados thd: ZOOM maps your query to PQF 11:31 kados thd: I still think CQL is a better long-term solution 11:31 kados thd: but CCL will do for now ... it's plenty powerful enough for 99% of our users 11:31 kados thd: and they can always use CQL if they want 11:31 kados thd: but they'll have to do their own ranking 11:31 kados thd: ie, I 11:32 thd kados: I had thought that you had previously chosen PQF for more precise control 11:32 kados thd: 'd need to write a pretty fancy compiler/parser to translate any kind of hierarchy from CQL into a ranked field-weighted query 11:32 kados thd: remember that conversation we had about nested/grouped queries? 11:32 thd yes 11:33 kados thd: well it turns out that's the only tricky part about queries :-) 11:33 kados thd: and we don't yet have a way to represent nested queries in a form anyway 11:33 kados thd: so switching to a non-nested CCL-like for form-based queries means we don't lose anything 11:34 kados thd: and the template designer can work with nice labels like 'title' and 'exact' rather than '@attr 1=4 6=3 4=1' 11:34 thd kados: except that many users know how to use parentheses 11:35 kados thd: they still can use them 11:35 kados thd: but if they do, it will be interpreted as true CCL or CQL or PQF 11:35 kados thd: (well, PQF doesn't have parentheses) 11:36 thd kados: know but it expresses the same logic without them 11:36 kados yep 11:36 thd s/know/no/ 11:36 kados prefix and postfix languages can do that 11:36 kados because they can only be interpreted one way 11:37 kados PQF is prefix and CCL/CQL are infix 11:37 kados and it turns out writing a decent nested query parser (aka, compiler) is a pretty complex job ;-) 11:37 kados i think it would take me about a month or so 11:37 kados to do it right 11:37 kados with testing and all 11:38 kados so I definitely don't have time for that now 11:38 thd kados: so if use parentheses in your CCL form I would loose the advantages of your CCL adaptation and have strict CCL instead 11:38 kados yep 11:38 kados that's the basic idea 11:38 kados 99% of users won't notice 11:38 kados and for the 1% that can do parentheses, I'll let them work our field weighting and stemming for themselves :-) 11:39 kados at least until I do have time to write the compiler :-) 11:39 thd kados: I did not understand why you left weighting to a separate form in zoomopac instead of including it in the advanced search 11:39 kados thd: i re-wrote the advanced search page :-) 11:40 kados thd: stemming and field weighting happen behind the scenes 11:40 kados thd: in the new form 11:41 thd kados: if they are behind the scenes then the user has no control over them? 11:42 kados well like I said earlier, the session management in Koha is currently borked 11:42 thd kados: why is session management needed for this? 11:42 kados if I can fix it, I could easily add a 'search preferences' section in someone's account 11:42 kados so each user could specify things like default search behavior 11:42 kados ie, turn on/off field weighting or stemming 11:42 kados view past queries 11:43 thd kados: why could it not be a direct option for individual searches 11:43 kados it could be 11:43 kados could just add a few checkboxes to the bottom 11:43 kados but if we really want the user to have control over it 11:44 kados they should be able to specify _how_ field weighting should work 11:44 thd kados: exactly 11:44 kados that'll have to wait for the session management to be fixed 11:45 kados it shouldn't be hard to fix it 11:45 thd kados: what is wrong with a drop down selection box for each part of the query? 11:45 kados yuk 11:45 kados what's wrong with it is users :-) 11:45 kados only .1% of our users are like you thd :-) 11:45 thd kados: you need some better users :) 11:46 kados hehe 11:46 kados thd++ 11:46 kados anyway ... the new API is pretty exciting 11:46 thd kados: I noticed that slef was fixing his users a couple of days ago 11:46 kados once zoomopac is done updating I can show you 11:47 kados so ... tomorrow morning i 11:48 thd kados: you only need session management to hide the good user interface from the 99% of unsophisticated users 11:49 thd kados: you could still have another search interface with all the options available super search as more advanced than advanced search 11:49 kados thd: maybe ... but based on the feedback I've gotten, fewer advanced searches are better than more :-) 11:50 thd kados: that is feedback from the people who do not use the interface options 11:51 thd kados: Koha should not be a tyranny of the majority 11:51 kados thd: nor should it be a tyranny of the minority :-) 11:51 thd kados: that is what preferences are for 11:51 kados exactly 11:51 kados which is why I wrote an API that supports both the majority and the minority :-) 11:52 kados thd: once I show you the new advanced page, you'll see that you could develop an advanced page with all the bells and whistles quite easily 11:52 thd kados: preferences can hide the good stuff from the majority of libraries 11:53 kados agreed 11:53 thd kados: did you ever change to a flexible design that avoided numbering each part of the query form 11:53 thd ? 11:54 kados thd: yes 11:54 kados thd: so you could dynamically add new fields to the form 11:54 kados if three wansn't enough for instance 11:54 thd kados: the HTML selection options were numbered the last time I looked 11:54 kados we can use the 'cloneSubfield' javascript I wrote to do that pretty easily in a future version 11:55 kados thd: they aren't as of this weekend 11:55 kados thd: noone has seen the new advanced search page except me 11:55 thd kados: JavaScript is a little evil 11:55 kados true 11:55 kados it would be just as easy to do it in cgi 11:55 kados with js layered for those that ahve it 11:56 kados anyway, I must get back to work :-) 11:56 thd kados: Ok 11:56 thd kados: unfortunately, i will be at the dentist tomorrow morning 11:57 kados bummer 11:57 thd kados: just routine I hope 11:58 thd kados: scheduled six months ago