Time Nick Message
12:31 slef thd: I think it would help to do a wide public test along similar lines to that which I just did, to see how the various settings interact.
12:32 slef thd: I can't remember how firefox behaves if it's given a non-unicode font to display unicode. It probably displays what it can, so as long as the fonts contain the characters being input, it's OK.
12:32 slef thd: and the other-OS and closed browsers are not known to me in this much detail.
16:33 thd kados: are you there?
17:58 alex hi can anyone help with MARC imports?
18:01 alex I'm trying to import into the resivior and I get the message "10 not imported because they seems not to be MARC format (or ISBN/ISSN missing)
18:02 alex This is a file downloaded from SCIS
18:11 russ hi alex
18:13 russ alex you there?
18:13 russ are you using the sample data set, or do you have a subscription?
18:13 russ http://www.curriculum.edu.au/SCIS/guests/samplefiles.htm
18:21 russ thank you
18:21 russ doh wrong window :-)
18:27 russ Import results :
18:27 russ * 5 imported records
18:27 russ * 0 not imported because already in catalog
18:27 russ * 0 not imported because already in the reservoir
18:27 russ * 0 not imported because they seem not to be in MARC format (or ISBN/ISSN missing) !
18:27 russ * 5 records parsed
18:27 russ * Back
18:27 russ using
18:28 russ http://www.curriculum.edu.au/SCIS/resources/ddc22_usmarc.dat
18:38 alex I;m back now
18:38 alex We have a subscription
18:38 russ hiya alex, i had a go using the usmarc sample file (told koha it was marc21) and it worked sweet as
18:39 alex I'll try the sample
18:39 russ i didn't try the scis file, just the usmarc one
18:39 alex Hi Russ :)
18:39 alex I'll give it a go
18:40 russ cool - good luck
18:43 alex Yep, the sample file imports OK, but not the one we got via the subscription.
18:43 alex Are there any tools to check the format?
18:43 alex or logs I should be looking at?
18:44 alex Or can I send someone the MARC file that's not working?
18:46 russ is the file you are getting from SCIS usmarc or scis format?
18:46 russ they seem to offer both
18:48 alex I'll check that
18:48 alex The filename is usmarc.dat which tends to suggest usmarc.
18:49 chris hi alex
18:49 chris how big is the file?
18:50 alex 10 records, 6k
18:50 alex hi chris
18:51 chris hmm
18:51 chris do you have a windows machine handy?
18:51 chris http://oregonstate.edu/~reeset/marcedit/html/
18:51 chris theres a really neat tool, unfortunately not for linux
18:51 chris that you can use to check if its valid marc
18:52 alex yes, I've got 2 windows laptops handy!
18:52 alex I'll check it out.
18:52 chris you can also open the the .dat file in a text editor
18:52 chris it should be one long line
18:53 chris which stuff like 000 0 d^^1 ^_aIserles, Tali.^^10^
18:55 alex opened it in wordpad and it appeared to be 10 lines?
18:55 chris right .. 6k does sound kinda big
18:55 chris for 10 records
18:55 chris maybes its given you the textual rep ... want to email the file to me ?
18:56 alex thought you'd never ask
18:56 alex having the three laptops and one is running out of battery
18:56 alex so I'm just mucking around transfering files on the network
18:57 chris righto :)
18:57 alex and giving people appropriate network access rights etc.
18:57 alex so It'll be a few mins.
18:57 alex thanks chris :)
18:57 chris no problem
19:14 chris hmm interesting
19:15 chris ive sent back the file ... for you to tru
19:15 chris try even
19:17 alex ok, cool, my greylisting may get in the way
19:17 chris yeah we run greylisting too
19:29 alex I've updated the greylisting rule for katipo, you could prod your mailserver or resend the file
19:30 russ alex: are you coming to the lianza conference?
19:32 alex no, probably not
19:33 chris on its way
19:35 russ and by organise i mean facilitate the first meeting for the koha users and see if there is enough interest/drive amongst themselves to see if they will do anything about it
19:36 russ i was looking at perhaps doing something round the lianza conference - do you think anyone involved in your project will be coming/ interested in attending a meeting?
19:38 alex perhaps this is too early in the piece
19:38 alex I would attend if I had a large enough customer base
19:38 alex but just one small school at this point
19:39 alex Maxine from the national library who is here with us won't be going to that conf
19:39 alex and the school staff are unlilely to be interested
19:40 russ ok cool, if we do do something, do you mind if i send you some info anyway?
19:41 alex yeah no problem
19:43 russ sweet
19:45 alex OK, chris's file imported 8 of 10, 2 were listed still as not importable (no ISBN?)
19:48 chris yeah i think thats the problem
19:48 chris i think there are only actually 9
19:49 chris at least thats what the file says
19:49 chris We acknowledge and thank you for your order of 9
19:50 chris one of which doesnt have an isbn .. thats my theory anyway .. and the comments its trying to read as one
19:51 chris if you go to /cgi-bin/koha/acqui.simple/addbooks.pl
19:51 chris in the title input
19:52 chris type in what
19:52 chris then search
19:52 chris you should get a result from the reservior
19:53 chris which if you click add biblio .. you can then look at (it doesnt actually add it until you click add bibliographic record and go to items .. on the next page)
19:53 chris so you can look at the record there without putting it in your actual catalog
19:54 alex yes, ok, when we searched for 10 records on scis, they only had data on 9
19:54 chris right
19:55 alex but I'm not sure if some kind of blank template was sent through for the 10th
19:55 chris could be
19:55 alex we have been through add biblio with one book
19:55 alex how do we add a copy, it's still not appearing in searches
19:56 chris you did the add biblographic thing and go to items ?
19:56 chris and added an item?
19:56 chris another question i have for you, does scis have a z3950 server?
19:57 alex z3950, no I don't think so, but maybe
19:57 alex OK i'm in a purple screen headed iblionumber: With Framework and 5 pages of all the biblio data
19:58 chris right
19:58 alex Do I need to click "Add biblio" button at the top?
19:59 chris you are using the default templates right?
19:59 alex ok, yes that's it
19:59 chris 2 secs ill just switch to those templates, and check
19:59 alex yep afaik
19:59 chris on the 5th tab
19:59 chris there is 942c itemtype
19:59 alex now I'm at a new item screen which has price and barcode
19:59 chris ahh gone past that .. cool
20:00 chris 942c you should always fill in
20:00 alex we set our item type on page 5
20:00 chris sweet all good then
20:00 chris so you can fill in price if you want, but you will need to put something in barcode
20:00 alex we have marc numbers turned off
20:01 chris ahh so you just see itemtype?
20:01 chris cool
20:01 alex so I'm not seeing 942c but I assume it is item type on page 5
20:01 alex yep
20:01 chris yep thats it
20:02 alex cool I think we're getting closer
20:02 alex I'll have a look at what you did to the usmarc.dat file
20:02 alex do you think I'll have luck with bulkimport.pl?
20:03 chris yep bulkmarcimport throws it straight into the catalogue .. not the reservior
20:03 chris so it expects the record to already have things like barcode, itemtype etc in them
20:04 chris its a migration tool
20:04 chris whereas loading them into the reservior sets it up so you can catalogue new books you receive
20:05 chris you can fill the reservior with tons and tons of records
20:05 alex We need to import all the books into the catalog
20:05 alex but there was no previous automation so we don't have barcode or price info etc.
20:05 chris right
20:05 chris so 2 options
20:05 chris use marcedit
20:06 chris to edit the marc records, and add price, itemtype and barcode to them
20:06 alex we send scis isbn and get back the biblio listings for what they have
20:07 alex marcedit gives an error right now, perhaps I should run it as administrator
20:08 chris then bulkmarcimport them
20:09 alex yep, makes sense
20:09 chris or .. load them into the reservior and go thru them adding them and adding itemtype barcode etc
20:09 chris i suspect marcedit will be a bit faster
20:10 alex cool, thanks for your help
20:10 alex will probably need more help .... but we've got heaps to work on atm
20:10 chris no problem
20:17 alex hmm, still can't search for it?
20:17 alex just turned marc codes on
20:20 chris hmm
20:21 chris if you jump into mysql, and do a select * from biblio; ... is there stuff in that table?
20:22 chris oh oh oh
20:22 chris i know
20:22 chris go to
20:23 chris /cgi-bin/koha/admin/stopwords.pl
20:23 chris do you have a stopword defined?
20:24 chris if not, add the
20:24 chris it needs at least one stopword for teh searches to work
20:24 alex possibly not
20:25 alex yes we have one, "THE"
20:25 chris darn
20:25 chris not that then
20:26 alex I'm looking in the item table and there is no holding branch and location I think?
20:26 alex nulls
20:26 chris yep that should be ok
20:26 alex there was drop downs but I didn't set them
20:26 alex ok
20:26 chris the table that is actually searched
20:26 chris is marc_word
20:26 chris is there data in there?
20:28 alex yes there is
20:28 chris cool so it is adding things .. thats good
20:29 chris i wonder why the search isnt finding it
20:29 chris have u tried just searching on one word of the title say?
20:29 alex yep
20:29 chris if you look in the error log
20:29 chris it should be outputting the sql its using
20:31 chris something like
20:31 chris select distinct m1.bibid from biblio,biblioitems,marc_biblio,marc_word as m1 where biblio.biblionumber=marc_biblio.biblionumber and biblio.biblionumber=biblioitems.biblionumber and m1.bibid=marc_biblio.bibid and (m1.word like 'what' and m1.tagsubfield in ('245a')) order by biblio.title ASC
20:31 alex where does koha log?
20:31 alex standard install
20:32 chris /usr/local/koha/log/
20:33 alex ok, those are just webserver logs?
20:33 chris yep
20:33 chris the other thing you can do .. to see that is in the catalog
20:34 chris select biblionumber from biblio;
20:34 chris then /cgi-bin/koha/detail.pl?bib=5625
20:34 chris where 5625 = the biblionumber
20:34 chris eg http://opac.kapiti.katipo.co.nz/cgi-bin/koha/opac-detail.pl?bib=5625
20:35 chris theres the one i loaded from that usmarc.dat file
20:35 alex ok
20:39 alex sorry chris my laptop froze up
20:40 chris np
20:40 alex [A[A[B
20:41 alex /cgi-bin/koha/MARCdetail.pl?bib=1 shows the record
20:41 alex with an item
20:42 alex was that what you suggested?
20:42 chris yep thats fine
20:42 chris detail.pl?bib=1 would work too
20:42 alex ok
20:43 chris hmm so its there .. i wonder why the search isnt finding it
20:43 alex do you want me to make the interface externally accessable?
20:44 alex I just need to add dns records
20:44 alex maybe only one record in the db
20:44 alex is causing an anomaly
20:44 chris i probably wont get a chance to look properly today (better get back to work work :-)) but if you want i can look this evening, drop me an email with details
20:45 alex ok cool
23:35 hdl thd : utf8_a_gogo website is up now.
23:36 hdl I have taken UTF8CGI.pm
23:38 thd hdl: I have the site up but http://www.urlfan.com/code/perl-utf8/UTF8CGI.pm has a 404 error
23:41 thd hdl: I linked from the wrong site I am downloading now
23:57 thd hdl: so the trick is Encode::find_encoding()
23:57 thd hdl: kados had doubted such a function was possible
23:58 thd but neither of us had looked carefully in the correct place
00:15 thd hdl: POD for Encode::Supported http://search.cpan.org/~dankogai/Encode-2.18/lib/Encode/Supported.pod
00:15 thd hdl: there is a little issue
00:17 thd hdl: If Koha were being queried from a completely unknown machine with no reasonable restriction about the encoding used much memory would be taken to load all the possible encodings which might need testing
00:19 thd hdl: if you are only concerned with testing for Latin-1 or standard Unicode encodings such as UTF-8, then you have those by default
00:22 thd hdl: however, I suggest that you are only fairly safe in Europe if you use Encode::Byte as well as the default
00:23 thd hdl: use Encode::Byte should allow testing for all of ISO 8859-whatever
00:26 thd hdl: is your concern for remote users or unknown encodings which may be by systems inside the library?
00:38 thd hdl: correction to some statements above ...
00:41 thd hdl: use Encoding::Guess is key for the difficult possibilities for UTFCGI.pm but even though multiple 8859 encodings could be added for guessing they cannot reliably be distinguished from each other
00:43 thd hdl: the various single byte ISO 8859 encodings can only be distinguished from multibyte encodings reliably
00:44 thd hdl: the multibyte encodings can be distinguished from themselves
02:31 osmoze hello
02:54 btoumi hi all
02:55 toins salut bruno !
02:55 toins salut all !
03:02 btoumi salut toins
04:07 btoumi chris: are u around?
07:01 slef LindvigL@cwu.EDU is auto-replying to koha@lists.katipo messages - can someone please mark nomail until their return on 2006-09-18?
07:18 kados hdl: are you here?
07:18 kados hdl: i was hoping to discuss UTF-8 briefly
07:18 btoumi somebody konw this message :=>script not found or unable to stat
07:18 kados yea
07:18 btoumi hi kados
07:18 kados it means the script is either not there
07:19 kados or else the permissions are set wrong and you can't read it
07:19 kados or it could mean you can't execute it
07:20 btoumi that i understand but i see for the permissions =>nothing wrong and the file is in the right place
07:20 kados strange
07:20 btoumi yes i think so
07:23 btoumi peraphs i have find somethink ; is about maninvoice and mancredit
07:23 btoumi ty kados
07:30 toins kados: hdl schould be back soon
07:49 kados morning owen
07:49 owen Howdy
07:50 kados owen: believe it or not, I did _nothing_ on the zoomopac while you were sleeping :-)
07:50 kados hehe
07:59 hdl hi
08:00 hdl kados : back
08:04 btoumi kados: i have resolved the problem
08:09 kados btoumi: what as it?
08:09 kados hdl: hi
08:09 dewey hi, kados
08:10 kados hdl: did you see my response on character encoding?
08:10 btoumi kados : i work with manvoice and mancredit.tmpl but the error was in manvoice.pl mancredit.pl
08:11 hdl yes.
08:11 btoumi kados: in url
08:11 kados btoumi: ahh ...
08:12 btoumi i have commit the changes
08:12 kados btoumi: thanks
08:13 slef kados: I added a bit to encodingscratchpad
08:13 slef about browser behaviour
08:13 kados slef: yea, I noticed
08:13 slef I may have also found a fontconfig oddity this morning.
08:13 kados slef: but I'm not sure it's accurate cross-platform
08:13 kados slef: for instance, on OSX, utf-8 combining chars simply don't work by default with any version of firefox and the Arial font
08:14 slef kados: hence my comment about needing to do a wide test
08:14 kados yep
08:14 slef does firefox on OSX use gtk, carbon or cocoa?
08:14 kados not sure
08:15 slef http://www.wotfun.com/pipermail/gobolinux-users/2006-August/003517.html is the fontconfig question, by the way
08:15 kados cool, thx
08:15 slef I thought it was camino on OSX?
08:15 slef (fontconfig returns ISO8859-1 fonts for some UTF-8 locales, which is evil, bad and wrong)
08:18 slef (eo_XX.UTF-8 gets the right font, so I'm using that as default now... which will confuse some of the users if I've got the overrides wrong.)
08:42 qiqo ei
08:42 slef ei?
08:42 dewey i think ei is 3.0 available at the CVS?
08:42 qiqo hehe
08:42 qiqo how are you all doing?
08:43 slef malbone
08:43 qiqo do we have news on the 2.2.6 release?
08:43 slef paul's away this week AIUI
08:44 qiqo hmm ok
08:45 qiqo i just hope that it will be released as soon as we generate the barcodes of our library
08:46 qiqo are the z3950 bugs fixed in 2.2.6?
08:47 kados qiqo: i've got about 20 production clients running the 2.2.6 z39.50 client and it's working just fine
08:47 qiqo wow
08:47 qiqo that's cool
08:47 qiqo i really cant make it work in 2.2.5
08:48 kados should be pretty simple to update
08:48 qiqo i had to use a proprietary cataloguing standalon software
08:48 slef qiqo: are you getting the 'no results found' problem or something else?
08:49 qiqo yup no results..
08:49 qiqo ive visited the logs,, nothings shown there
08:50 slef qiqo: it's not even searching?
08:50 qiqo ive given up.. since 2.2.4 i couldnt make it work..
08:50 qiqo i dont know,, but it resides on memory
08:50 qiqo i tried connecting using the console using yaz and it connects
08:50 slef only z39.50 problem to defeat me this year has been a firewall :-/
08:50 qiqo we have no firewall :)
08:51 qiqo actually i already used 2 isps..
08:51 qiqo they say that we dont have firewall
08:51 qiqo i already checked the router..
08:52 qiqo so i think i should really wait for 2.2.6
08:54 slef are other people on holiday Monday?
08:55 qiqo but we have the proprietary software for cataloguing, my primary concern is with the barcode generator..
08:56 slef I think it is for many people
08:58 qiqo cool.. hehe glad to hear that
09:36 thd hdl: are you there?
09:36 hdl yes
09:37 thd hdl: is your concern for CGI encodings about remote users or unknown encodings which may be by systems inside the library?
09:38 hdl qiqo : your barcode problem is a problem of checksum line 164 in BarcodeGenerator.pl
09:39 qiqo huh?
09:39 hdl checksum is manipulating data.
09:39 qiqo ahh yeah
09:39 qiqo that's what happening
09:39 hdl And if you want to print itembarcode, you have to "shunt" this function.
09:40 qiqo it prints 0000000017 instead of 00001
09:41 hdl thd: it is general concern when a user inputs data into the system.
09:41 hdl One should at least check that it is utf-8.
09:41 hdl if not reencode.
09:42 hdl qiqo : try also to "play" with extn... line 168...
09:43 hdl or type if you donot want ean13
09:45 qiqo ok but isnt that fixed with the 2.2.6 release?
09:46 qiqo i really am no that techinical when it comes to scripting
09:46 qiqo im not familiarized with perl scripts..
09:46 qiqo :)
09:46 thd hdl: did you see my comment to you from last night or early this morning for you about usage of use Encoding::Guess and to add needed encodings to test and the inability to distinguish between different ISO 8859 encodings?
09:50 hdl qiqo : so try and setup a 2.2.6 test box. Since RC2 is out ;)
09:50 hdl thd: OK. So are we blocked ?
09:51 qiqo ok.. ill check that out tommorrow
09:51 qiqo its already 11pm here :)
09:51 qiqo so 2.2.6 will run smoothly on a debian stable?
09:51 qiqo i mean rc2
09:52 thd hdl: There are systems which can use linguistic information to guess encodings and languages given a sufficiently large document. A bibliographic query is unlikely to be sufficient linguistic information.
09:52 thd hdl: no this is not a block except for different ISO 8859 possibilites
09:53 hdl qiqo: We think/hope so.
09:54 hdl Do you think UTF8CGI is usefull ? Or should we hack something for us ?
09:54 qiqo and may i ask for a suggestion if what desktop systemare you suggesting
09:55 hdl qiqo: I am running Mandriva. But debian rocks ;)
09:55 thd hdl: you can still distinguish a chosen assumed single byte encoding from many different multibyte encodings and you can distinguish the multibyte encodings from each other.
09:55 qiqo cool.. mandriva is good.
09:55 qiqo but i think ill stick with debian
09:56 qiqo so will it be kde or gnome?
09:57 thd hdl: you have to presume some particular ISO 8859 is most likely and guess based on that
09:58 hdl qiqo: The one you prefer.
09:58 qiqo ehhe ok ill stick with kde
09:58 hdl as sson as it runs Firefox.
09:58 qiqo actually im running my server with freespire
10:03 thd hdl: see http://search.cpan.org/~dankogai/Encode-2.18/lib/Encode/Guess.pm#CAVEATS
10:08 thd hdl: so you will be able to discover whether the user has submitted Latin-1 and convert but distinguishing Latin-1 from Latin-2 is unlikely to work unless you have special linguistic insight for modifying or extending Encode::Guess
10:13 slef hdl: what is UTF8CGI?
10:13 slef UTF8CGI?
10:13 slef dewey: wiki?
10:13 dewey wiki is, like, at http://wiki.koha.org
10:14 slef dewey: have a poisoned botsnack
10:14 dewey thanks slef :)
10:14 slef qiqo: gnustep!
10:14 hdl (See :http://dysphoria.net/2006/02/05/utf-8-a-go-go/
10:14 hdl )
10:14 slef hdl: seeing
10:15 qiqo haha gnustep hehe
10:15 qiqo cool
10:16 qiqo but we have users that prefer the "M$ windows" touch
10:16 slef By now (2005) the robust approach is to send out forms pages encoded in utf-8, expecting the forms input to be submitted back using that encoding. This has been in practical use for a couple of years now (e.g at Google) and can be expected to work with any current HTML4-compatible browser.
10:16 slef http://ppewww.ph.gla.ac.uk/~flavell/charset/form-i18n.html
10:16 owen dewey: Property is theft!
10:16 dewey OK, owen.
10:16 thd hdl: maybe after Encode::Guess guesses, if it guesses ISO 8859-1 we for example, we could test the data string for the presence of special characters which might be either ISO 8859-5 Turkish or ISO 8859-1 Icelandic and decide which is more likely.
10:17 slef qiqo: trouble is gnome and kde are windows-wannabes, but obviously not windows, which seems to disturb users more IME
10:18 slef qiqo: if you give them something which looks nothing like their interface, they remember it's different. And the step interface behaves mostly sanely.
10:18 qiqo hehe ok
10:20 thd hdl: but maybe we would miss other possibilities such as ISO 8859-6 Latin/Arabic
10:21 hdl But maybe that would be another point later on.
10:24 thd hdl: I assume slef is correct about about most recent browsers and operating systems. I hope guessing is only needed for recalcitrant legacy clients which are an ever dwindling minority
10:26 thd slef: does your cell phone browser or wrist watch computer browser send UTF-8 correctly?
10:26 slef Nah, assume I'm correct about needing to test this more widely before doing much else.
10:27 slef I don't have a cell phone browser or wrist watch computer.
10:28 slef dewey: UTF8CGI is http://dysphoria.net/2006/02/05/utf-8-a-go-go/
10:28 dewey OK, slef.
10:30 thd slef: I have seen European cell phone browsers that send UTF-8 but I think that they are the minority in Europe and that option is a setting which is controlled by the user in advance and not the default
10:31 qiqo can koha be also on wap?
10:31 qiqo hehe
10:32 slef thd: again, something to test.
10:33 thd qiqo: I want these tiny systems to be able to use real web pages and not just WAP.
10:33 qiqo ahh ok so xhtml?
10:34 thd qiqo: there are some that can but they are much too expensive to drop or get at little wet
10:35 thd slef: let me know when I can get an all purpose wrist watch computer for US $500
10:38 slef if the US$ keeps falling against the Euro, that may be soon ;-)
10:39 slef oh wait, that's backwards :)
10:39 slef *@*!@ currency exchanges