IRC log for #koha, 2006-02-21

All times shown according to UTC.

Time S Nick Message
11:22 kados thd: are you there?
11:48 thd kados: yes, I am here
11:48 kados thd: have you ever imported authorities records into Koha?
11:50 thd kados: Koha 2.X has no provision for importuning externally created authority files
11:50 thd s/importun/import/
11:51 kados bulkauthimport ?
11:52 thd kados: hdl had confused the English distinction between building and importing
11:52 kados ahh
11:52 kados well ... I suppose it's just as well
11:53 kados I have an author authority file
11:53 kados for one of my clients
11:53 kados but there are no 'id's ...
11:53 thd kados: bulkauthimport.pl was a hardly started experiment
11:53 kados so I don't see how it could be used to link between a record, etc.
11:53 kados ahh
11:54 thd kados: I do know how it could be done
11:54 kados how?
11:55 thd kados: It would require writing bulkauthimport.pl
11:55 kados right
11:55 kados but still, without a way to link an author in the auth file to the biblio file
11:55 kados it would be pretty pointless
11:56 thd kados: currently that script should not be presumed to be a guide to even how to start
11:56 kados here are a couple samples from the auth file I have:
11:56 kados NUMBER 1 =>LDR 00066nz   2200037o  4500100 10 _aCarmichael, Ian,       _d1920-NUMBER 2 =>LDR 00053nz   2200037o  4500100 10 _aHale, Gena
11:56 kados all it has is a leader and 100$a
11:56 thd kados: the type of authority file can be determined based on a value in the record
11:57 kados I have both author and subject authorities from this client
11:57 thd kados: maybe it is in the leader
11:57 kados but as I said before, all they contain is a list of values
11:58 kados many of them have an identical leader
12:00 thd kados: there is the element that identifies whether the authority is for a personal name, corporate name, uniform title, subject, etc.
12:00 kados yep, that's probably in the leader
12:00 kados but as there is no ID in the record, it's useless to me
12:00 kados ie, there is no id in the auth record and no id in the MARC record
12:01 kados so no way to preserve any relationship between the two
12:01 kados in fact, the MARC record has authors/subjects just like normal records do
12:01 thd kados: I remember now and I think it is nothing more than what field number holds the main value in MARC 21
12:02 kados ?
12:03 thd kados: There are rather nice FRBR relations in many authority files.
12:04 thd kados: so the first problem is to identify what the type of authority file is and that is easily done.
12:05 thd kados: That type is then stored in a special local use field for Koha to use.
12:06 kados well ... like I said before, without ids in the records the auth file is useless for importing
12:06 kados ie, there is no way to use the auth file for import
12:06 kados so I"m not going to worry about it
12:06 kados I may attempt to build authorities once the records have been loaded in
12:06 kados thd: another question
12:07 kados thd: remember we discussed you working on another framework for MARC21
12:07 thd kados: Koha can assign IDs just as id does for bibliographic records.
12:07 kados thd: I'm wondering if you could solve the problem we currently have when a set of MARC records has itemtypes at the item level rather at the record level
12:07 thd s/id/it/
12:08 thd kados: I solved that months ago
12:08 kados thd: (sure it can but there is no way to relate a given bib record to a given auth record on import)
12:08 kados thd: really?
12:08 kados thd: how?
12:08 thd kados: virtual libraries can do everything
12:09 kados what's a virtual library?
12:10 thd kados: a library that contains a branch name as well as other information that you want to use more flexibly.
12:11 kados I don't understand how that would solve the problem
12:12 kados here's what I mean ... say you have a record
12:12 kados it does not contain itemtype information at the level of the record, but at the level of the item
12:12 kados so a single record contains holdings for
12:12 kados JNF, BK, LGPRNT, etc.
12:17 kados this is problematic in Koha's current default framework because we don't track itemtype at the item level
12:18 kados so you lose the ability to distinguish between itemtypes within a single record
12:18 kados thd: does that make sense?
12:19 thd kados: That makes sense as a problem and I have a hack to Koha 2 to fix that problem.
12:20 thd kados: Let me send you the XML file.  Warning it may not be valid XML yet.
12:20 kados XML file?
12:21 thd kados: Stephen uses XML as the base record on kohadocs.org
12:22 thd kados: My starting point had been his XML markup for the previous document about holdings.
12:23 thd kados: I also wrote my proposed solution to koha-devel months ago in a very long detailed message.
12:23 kados I see
12:24 thd kados: Sourceforge never archived that message although I still have the original somewhere.
12:25 kados thd: how difficult is it to implement the solution -- can it be done easily or would it require a lot of extra work?
12:25 thd kados: This is merely a hack on Koha 2 not the elegant way that Koha 3 should address this problem.
12:25 kados of course
12:25 thd kados: I was thinking of creating a default value set that set up some small changes for new Koha installs.
12:37 thd kados: I have sent it
12:39 kados thanks, I"ll take a look
12:39 thd kados: Authorities can be imported workably into Koha 2 with the right import script.
12:42 kados thd: it looks like your proposed solution to the itemtype problem is just to create a new record
12:43 thd kados: Authorities import for Koha 2 is not a very difficult problem.  The authorities usage that paul created is very nice and can be used to best advantage only by importing rich authority files instead of the poor ones that can be built from existing record data.
12:44 thd kados: what do you mean, create a new record?
12:45 thd kados: look for 'material types'
12:45 kados A difficulty of the Koha MARC implementation in relation to the
12:45 kados        original Koha data design will not allow the assingnment of individual
12:45 kados        copies to different circulation fees without creating a separate
12:45 kados        bibliographic record for copies with different circulating fees.
12:46 kados Creating a separate bibliographic record for copies with separate
12:46 kados        circulating fees is the workaround for not being able to treat
12:46 kados        different copies differently at all.
12:46 kados Reassignment of an individual
12:46 kados        copy to different circulation fees requires creating a separate
12:46 kados        bibliographic record and removing the holdings information for that
12:46 kados        copy from the original bibliographic record.  Obviously, that should
12:46 kados        be corrected at some point in the future.
12:46 kados that's the solution right?
12:48 thd kados: I know that I was just looking at that section but that may only be for when a different fee is needed.
12:49 thd kados: the heading for that subsection is for fees.
12:49 kados yep ... and that's what itemtype is generally used for
12:49 kados for instance, one of the records I'm looking at
12:49 kados has REF and JNF in the same record
12:49 kados obviously REF doesn't circulate at all
12:49 kados while JNF does
12:50 kados but when I import this record into Koha, I have to decide which itemtype the whole record gets
12:50 kados or else split it into two records
12:50 thd kados: split the record
12:50 kados yea ...
12:51 kados I'll need to rewrite my import script to handle that
12:51 kados but it's really a  pain in the arse
12:51 thd kados: There is also pain from the inflexibility of manipulating items in Koha
12:52 thd kados: One pain is shorter than the other and affects fewer people
13:03 thd kados: I think that splitting is only needed if the library charges money for circulation.
13:11 thd kados: I have rarely seen libraries that charge money for mere circulation of some types of material.  Do any of your libraries actually do that?
13:17 kados i don't think so
13:18 thd kados: Then you would not need to split the records.
13:18 kados i think I still need to
13:18 kados unless I'm not understanding
13:18 kados because circulation rules are set via itemtype
13:18 thd kados: Obviously the text needs to make that clear.
13:19 thd kados: I propose to change that.
13:19 thd kados: Setting circulation rules by item type is crazy.
13:19 kados thd: right, but that solution won't really help this migration :-)
13:19 thd kados: why not
13:19 thd ?
13:20 kados thd: because Koha currently sets circulation rules by item type in issuingrules.pl
13:20 thd kados: Each virtual library can have its own circulation rules.
13:20 kados it's not practical to set it up that way
13:21 kados there would be too many matrices to fill out
13:21 thd kados: The document may be incomplete or unclear on that point.
13:21 thd kados: That is only at setup.
13:22 thd kados: you have library MAINCIRCRULE1
13:22 thd etc.
13:22 kados thd: there is still no way to distinguish between a REF and a JNF within a single record
13:22 kados ie if a record has both itemtypes, and one is not supposed to circulate
13:23 kados there is no way to tell Koha that without splitting the record
13:23 thd kados: you have library MAINCIRCRULE1 for JNF
13:23 thd kados: you have library MAINCIRCRULE2 for REF
13:24 kados yes but REF is lost on import :-)
13:24 kados unless I split the record :-)
13:24 kados now do you understand? :-)
13:24 thd kados: why is REF lost? Where do you loose it?
13:24 kados ok ... so in this client's case
13:25 kados they use local use fields for holdings
13:25 thd kados: REF and JNF are both in the record are they not?
13:25 kados (because this is a Dynix system they are coming from)
13:25 kados so ... at the record level
13:25 kados they have 949 and 994 tags
13:25 kados then at the item level (multiple per record)
13:25 kados they have:
13:25 kados 959
13:25 kados $b barcode
13:26 kados $t ITYPE
13:26 kados $h call #
13:26 kados $a branch code
13:26 kados $b secondary agend
13:26 thd kados: Well you need to preserve REF and JNF for the library somehow.
13:26 kados $k collection code
13:26 kados normally, I would just map the 959 directly to Koha's 952
13:26 kados and the 949 would normally contain the itemtype and I could map it to Koha's 942
13:27 thd kados: You were not going to throw out that data.
13:27 kados it doesn't matter if it's in there or not
13:27 kados the point is, Koha can't perform any business logic on it because it doesn't know where it is
13:28 kados there is no way to map one koha field (itemtype) to many MARC fields (959$t in this case)
13:28 thd kados: If you append that information to the correct 952 subfield for branch then you have what you need.
13:28 kados hmmm
13:29 kados I think I see what you mean
13:29 kados so each 'itemtype' is also a branch?
13:29 kados i have a feeling that would really mess up other areas of Koha
13:29 kados like reserves
13:29 kados when a patron goes to reserve a book
13:29 thd kados: so you create the 952 in advance where branches are MAINREF, MAINJNF, etc.
13:30 kados it asks which branch they want to pick it up from
13:31 thd kados: You can solve more complex branch problems that this might create with branch groups.
13:32 thd kados: Assign all MAINXXXX to the same branch group.
13:33 thd kados: I did say that text needed a day's work and there are certainly missing sections and some rewriting needed.
13:35 thd kados: It probably needed another day's reflection to put it back into my mind correctly.  In December, I knew exactly what was missing.
13:38 thd kados: Your branch group names can be normal names like MAIN , RIVERSIDE, etc.
13:38 kados I'm not sure what grouping the MAINXXXX branches into a category would do
13:39 kados in terms of keeping them out of reserves
13:39 kados so say we had a COLLECTIONCODE
13:39 kados branch category
13:39 kados and each branch inside was of type:
13:39 kados CCJNF, CCREF, CCXXX, etc.
13:40 kados and in my MARC import, I added a CC in front of the 'itemtype' field in the records
13:40 kados and mapped it to the branch
13:40 thd kados: nothing for keeping them out of reserves.  The circulation rule associated with each individual virtual branch can keep them out of reserves.
13:41 kados no, I don't mean that
13:41 kados I mean, when I go to put a reserve on an item
13:41 kados Koha asks me:
13:41 kados which branch do you want to pick the item up at?
13:41 kados and it's a drop-down box with all the branches listed
13:42 thd kados: you can suppress all but the branch group name.
13:43 thd kados: Katipo does that sort of special modification all the time to gain some functionality.
13:43 kados right
13:44 kados well ... I'll try it out
13:44 thd kados: Katipo that can be done with the template.
13:45 kados thd: one more problem I think :-)
13:45 kados take a look at this mapping:
13:45 kados > 959 10 _b33529002736014   *b is the BARCODE
13:45 kados >        _tGE               *t is the "ITYPE" GE=GENERAL
13:45 kados >        _h937 F151h        *h is the CALL#
13:45 kados >        _aSPL              *a is the AGENCY
13:45 kados >        _bST               *b is the Secondary Agency
13:45 kados >        _kCX               *k is the Collection Code
13:45 kados >        _tpart I           *t is the Copy/Volume
13:45 kados there are TWO $t s
13:45 kados !!
13:45 kados the first one is the ITYPE
13:45 kados the second is Copy/Volume!
13:46 kados i don't think there is a way in MARC::Record to handle this
13:46 kados what I mean is, I don't think I've ever encountered repeatable subfields within a tag for holdings data
13:47 kados where the second field was a different TYPE than the frist
13:47 kados first even
13:48 thd kados: change the variant to another subfield based on content.
13:48 kados yea, I might have to do that
13:49 thd kados: The meaning of subfields often changes in MARC based on the value of indicators or even other subfields.
13:49 kados yep
13:50 thd kados: MARC 21 seems to be better about not doing that based on other subfields than is my memory of US MARC.
13:51 thd kados: UNIMARC is nicer still in trying to not create unnecessary confusion while still being MARC.
13:52 thd kados: confusion and MARC are of course synonyms of some sort for one another :)
13:52 kados heh
13:54 thd kados: Authorities import into Koha 2 could work well with a fair amount of code reusable for Koha 3.
13:55 thd kados: The only real difficulty is the need to search existing records for matching authorised values from authorities for linking.
13:57 thd kados: That would be done on record import and then a $9 is added to the matching bibliographic record to support paul's authorities implementation.
13:58 kados thd: or you could just rebuild your authorities list based on data in the system
13:58 kados thd: which is what i suspect I'll be doing
13:58 kados then import a separate list of authorities for new cataloging
13:58 kados wait ... I guess that wouldn't work
13:58 kados sigh
13:59 thd kados: The build authorities from existing records is currently coded for UNIMARC.
14:02 thd kados: Building from bibliographic records gives poor authorities because all the disused and unauthorised variants of the authority that a user may try are absent in the bibliographic records.
14:05 thd kados: Paul's code allows searching for the variant forms and then finding the authorised record if the variant forms are included from a rich standard authority record.
14:06 thd s/authorised record/authorised form to use in finding the bibliographic record/
14:08 thd kados: The major issue with paul's code is that it does not manage authorised values past $a.
14:10 thd kados: Even with that present defect, using authorities is fun and a great advantage in Koha.
14:11 kados I'm not clear on how to set them up ...
14:11 kados so I import my MARC records
14:11 kados then build authority records from the existing data?
14:12 kados I cannot import an external authority file yet, right?
14:12 kados perhaps I should talk to paul about this on Monday
14:12 thd kados: Actually, there is no user level search for authority data, but that could easily be modified from the intranet side.
14:12 kados there is actually
14:12 kados it's the [...]
14:12 kados on the OPAC and Intranet advanced search pages
14:15 thd kados: In the OPAC, that only searches the values present in the relevant fields.  The OPAC does not search other forms that appear as unauthorised forms.
14:16 thd kados: In record editor plugins and in the authorities module a search for an unauthorised form will provide you with the authorised form.
14:18 thd kados: Both of those are only on the intranet side.  The MARC record editor plugin approach could easily be added to the OPAC.
14:20 thd kados: The present answer to building and not importing is yes and building is written for UNIMARC.
14:24 thd kados: Paul had no plans four or so months ago to fix the authority code for using values past $a in 3.0.  Maybe that is a 3.X issue.  However, his libraries seem to care very little about standards.
14:26 thd kados: The library profession seems to be organised differently in France where standards maniacs do not inhabit every library there.
14:28 thd kados: All his libraries encode authors differently in UNIMARC, and none of them in a UNIMARC compliant manner.
14:29 thd kados: I can see what is present in their records I merely lack the history of how that evolved.
14:35 thd kados: My problems yesterday were with your CVS symlink script and some recent changes in template behaviour.   Your symlinks to the templates seem to derive from some version of Koha prior to at least 2.23 when the templates directory may have been organised differently.
14:36 thd kados: That script would continue to work for older installs because of the koha.conf setting for the template directory.
14:39 thd kados: are you still there?
20:13 kados sam:/home/koha/testing/cvsrepo/koha# misc/migration_tools/rebuild_zebra.pl -cGlobal symbol "$service_type" requires explicit package name at C4/Biblio.pm line 230.Compilation failed in require at misc/migration_tools/rebuild_zebra.pl line 10.BEGIN failed--compilation aborted at misc/migration_tools/rebuild_zebra.pl line 10.
07:09 paul |hdl| tu es pas ici ?
07:09 paul s/pas/par/
07:10 |hdl| oui
07:10 paul un petit skype alors ?
09:20 osmoze hello
09:38 paul hello osmoze.
09:38 paul tiens, je vais jeter un coup d'oeil sur ton problème.
09:38 osmoze :) bonne nouvelle :)
09:40 osmoze par contre, je suis en repos aujourd hui et les JO risquent de ne pas me faire repondre si t as des questions ^^ ( mais je zieute de temps en temps)
10:26 kados hi all
10:27 paul hello kados
10:27 paul good morning
10:27 kados z3950_extended_services
10:28 kados no code calls it yet ...
10:30 kados paul: question for you about authorities
10:30 kados a client  has sent me an authority file
10:30 kados it is MARC21 format
10:30 kados so it has author in 100$a etc.
10:31 kados but there are no record ids ... and in the client's MARC data the subjects and authors are there (instead of an id)
10:31 kados am I correct that there is no way to import the authorities file
10:32 kados since there is no way to match which authority record goes with which bibliographic record?
10:35 paul you're right.
10:35 paul the only solution is to write some code to find the matching depending on the content.
10:35 paul if you are lucky (or the file is well formed), you should have no problems.
10:36 kados but really, I could just build a new authorities file from the data, no?
10:36 kados ie, import the MARC records
10:36 paul (mainly for subjects. for authors, you can't be sure there is only 1 joshua ferraro, and if there are 2, who is who. except if in your authority AND biblio you have dirthdate for example)
10:36 kados then build an authority file?
10:36 paul that could work, except if some authority isn't used, even if in the authority file.
10:37 paul (that will work...)
10:37 kados (the authority file does have birthdate)
10:37 paul that's how I usually do it.
10:37 paul ok for birthdate and authority, but are birthdate in biblios too !
10:37 paul otherwise, it's quite useless.
10:37 kados no :(
10:37 paul (to find the match I mean)
10:37 kados exactly
10:38 paul so, if you do what you plan, you will miss the birthdate. Maybe it's important for your library
10:38 kados I will ask the client if they can attach a system identifier to the MARC record and the authority file
10:38 kados as this would solve the problem
10:39 paul for sure !
10:42 kados paul: second question has to do with updatedatabase in head
10:43 kados I would like to merge in changes from rel_2_2
10:43 kados but when I created a diff file it was over 2000 lines long !
10:44 paul I cutted a LOT of things in head.
10:44 paul you should do a diff from 225 tag only
10:44 paul -j option in cvs iirc
10:44 kados ok ... I'll try that
10:51 paul kados : no news from DBD::mysql maintainer ?
10:54 kados paul: no ...
10:54 kados paul: but I will contact mysqlab this morning
10:55 kados paul: to see if they will pressure him to respond to me :-)
10:55 paul good idea i hope.
10:55 paul do you think that what I discover is right ?
10:55 kados if not, we can patch DBD::mysql and include it in C4
10:55 paul (right = correct)
10:56 paul you're probably right.
10:56 kados paul: about the utf-8 handling?
10:56 kados paul: (is that what you're asking?)
10:56 paul or provide a patched version of dbd::mysql and make it mandatory in Koha install
10:56 paul yep
10:56 paul do you think my diagnostic is correct ?
10:56 kados I haven't tested it myself, but from the number of references you have provided I'd say it's right
10:57 paul (I might have missed something or misunderstand something else)
10:57 kados we could test by patching DBD::mysql and see if it fixes it
10:57 kados (shouldn't be too hard as it's Perl, right?)
10:57 paul call mysqlab, and if you have no solution, i'll try.
10:57 kados ok
10:57 paul but that's not an acceptable solution for public release !
10:57 kados in about an hour or so I will call
10:58 kados I agree it's not ideal
10:58 paul 3pm for me, so i'll be here for some time still
10:58 kados paul: cvs update -d 225 updatedatabase
10:58 kados paul: is that correct?
10:59 kados oops ... s/-d/-j/
10:59 paul mmm... I had 2 j iirc
10:59 paul something like cvs update -j rel_2_2 -j 225 updatedatabase

| Channels | #koha index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary