rtf-scan disambiguation

claudionotz

Dear Zotero users

I am experiencing two problems with rtf-scan. Unfortunately big issues for properly working on scientific texts. I read many posts in this forum about rtf-scan but I'm sure they are not for the newest version of Zotero.

1. References containing umlauts (ä,ö,ü) are not recognized properly. One has to disambiguate for such entities which is really bothering.

2. If I want to cite to works of an author published in the same year, I have to make a disambiguation too. This is much more bothering than the umlauts thing: I then need to remember all the citations I made and then choose the right one. Is there a possibility to add something to a field in the database or in the field code to make disambiguation in such cases obsolete?

Best wishes
Claudio

dstillman

RTF Scan hasn't been worked on in a long time, so there are certainly bugs.

(For others reading this, I'll reiterate that we'd happily accept patches for RTF Scan improvements.)

claudionotz

This sounds as if rtf scan is not with much priority, is it?

fbennett

Not exactly. Dan has indicated that the core team is open to patches that improve RTF scan support. For many features, he would not extend that invitation. So that's a green light.

This does suggest, though, that the core developers are busy with other tasks, and perhaps that they feel the coding required is not fantastically difficult.

The task really is just waiting for someone to pick it up.

claudionotz

That's very nice. I would if I could do such things with my computer…

brunus

Dear fbennett and kind Zotero developers,
I would also add my voice to ask for a more complete syntax for the RTF scan feature.

Currently, I really miss the possibility to indicate the chapter number instead of the pages or the expression "see also": (see also Author, YEAR)

I would suggest to the developers to code the possibility of including some wildcard character to be inserted within the curly brackets which will simply mean "skip until the next wildcard" to the rtf scan and be printed there as it is.

I mean something like:
{'see also' Author, YEAR, 'Ch. '#} which would return (see also Author, YEAR, Ch. #) inline and the ususal Author (YEAR) Title in the final bibliography.

thanks

adamsmith

Anyone doing this will likely want to follow the syntax already in use by zotero-plain:
https://bitbucket.org/egh/zotero-plain
and pandoc/citeproc-hs
http://johnmacfarlane.net/pandoc/README.html#citations-1
which can do all this, rather than re-invent the wheel.

The main obstacle at this point is that Zotero doesn't currently support citation keys (along the lines of what TeX and the two above projects use) so any RTF-scan implentation at this point will only be so-so. Might still be worthwhile - and probably not prohibitively hard - to implement these features if any third party developer takes the time.

This all happens in one .js file:
https://github.com/zotero/zotero/blob/master/chrome/content/zotero/rtfScan.js
(with the GUI elements in a .xul: https://github.com/zotero/zotero/blob/master/chrome/content/zotero/rtfScan.xul )
so this is really as ideal as it comes for a third party patch.

brunus

Hi guys,
I'm still full of hope that this will be properly implmented one day, but in the meanwhile, here is my workaround: suppose I want to have the following result after the RTF-scan: (Author1, 2008, p. 56; see also Author2, 2010, ch. 3). This is how I do:

1. I write and use the Zotero syntax: bla bla bla {Author1, YEAR, p#} see also {Auhor2, YEAR}, ch. 3)
2. I compile the draft as DRAFT.rtf
3. I scan it with rtf-scan into DRAFT(Scanned).rtf
What I get is: (Author1, 2008, p. 56) see also (Author2, 2010), ch. 3)
4. In the wordprocessor I run a simple find&replace macro that replaces all the

) see also (

with

; see also

and similarly

), ch.

with
, ch.

and so on for any other unconventional expression you want to add. I refine the macro when I see the special case I've written does not fit.


I understand this is not superelegant as a workaround, but hey, this is what I can do with my technical means being no developer.

I hope it's helpful.

brunus

Here I am again.
dear Zotero devs, would it be very hard to tweak the RTF-scan engine so that it takes into account the full date and not just the year? This way one could easily distinguish between same-author-same-year citations.
I mean {Smith, 2009-12-22} ≠ from {Smith, 2009-09-23} to be converted in (Smith, 2009a) and (Smith, 2009b).
It is really a pity that there is no way to disambiguate such citations: can't you hack something up?

thanks

adamsmith

brunus - I've been wanting to look at RTF scan for a while, but no guarantees that's happening anytime soon - I've been hoping we'd get a citekey in Zotero sooner than looks likely now.

In the meantime, also check out fbennett's ODF Scan.
http://citationstylist.org/tools/?#mlz-tools

Hardiman

Is there also any workaround to fix these umlauts (ä,ö,ü)?

I usually have plenty of these umlauts when I do the RTF-scan. In a German speaking forum I found the workaround to export the zotero-database to bibtex and use this instead.

However I like Zotero a lot and I used it from the very beginning, but it is just such a big waste of time to assign all the cites with umlauts every time.

adamsmith

which word processor are you using?
In the near term, you're likely best of looking at MLZ/odf-scan

Hardiman

I use Scrivener and I compile the texts to rtf. Then I do the rtf-scan und afterwords I usually take Word or Pages for fine tuning.

adamsmith

do consider this:
http://citationstylist.org/tools/?#mlz-tools
I'm not sure when someone is going to work on RTF scan.

Hardiman

I installed it. It still can not find the authors with umlauts.

adamsmith

that's basically impossible if you're actually using this right, since it works with Zotero's internal identifiers and doesn't use the author names at all.

Note that the tool is completely different from RTF scan and requires you to also use MLZ currently.

See the discussion here http://forums.zotero.org/discussion/18064/2/please-add-better-integration-with-scrivener/ particular the comments by fbennett and paultroop.

Hardiman

ok, thanks. I will read it and will try it again...and will give feedback.

I just tried the rtf-scan. This is what I got:

https://www.dropbox.com/s/nyv2xd0tw967niy/Screenshot.png

Thanks a lot!

Hardiman

The author names were:
- Schröder
- Zerfaß

adamsmith

this might be very easily fixable with the new Zotero version, I'll take a look.

Hardiman

It worked with my small test file. I did it like proposed: Scrivener ==> odt-file ==> zodfscan ==> Libre-Office. Is this the way you proposed?

I did not try my whole document (more than 300 pages and a lot more of different authors), but I will give it a try...

Actually I am worrying about the changes of the zotero database. If this MLZ-thing is not supported any more can I change back to "classic-zotero"?

adamsmith

yes, that's the workflow I suggested.
acscording to this:
http://citationstylist.org/2013/01/27/mlz-new-release-available/
you'll be able to go back to "regular" Zotero using sync, yes (not by moving your database, though).

I had a look at the umlaut issue in RTF Scan and that's at least not super-trivial because RTF itself converts the umlauts into the \u... codes.

fbennett

MLZ is sync-compatible with official Zotero, so you can do a reverse migration by (1) backing up and syncing, (2) moving your MLZ database (zotero.sqlite) to a safe location, (3) installing official Zotero, and (4) syncing to your account. If you have used MLZ extended fields, the data for them will turn up in the Extra field in official Zotero. If you then reinstall MLZ, the data will be moved back into the extended fields.

fbennett

Yep, that's the workflow. There have been a couple of requests for the reverse -- converting live Zotero citations to "link-wrapped" cites in an ODF document, so that the content can be cut and pasted into other utilities. I can see how this would be useful, and the way the zodfscan.py script is coded, it shouldn't be difficult to do. I'll add that option to the script when I get around to it.

adamsmith

Frank - any chance you'd resubmit this pull request to master/3.5?
https://github.com/zotero/zotero/pull/183
I really, really want to have this working with vanilla Zotero.

fbennett

Thanks for the reminder. I'll try to get that up today.

fbennett

I've added that reverse-conversion option to the zodfscan.py script. You can get the updated version by pulling from GitHub, or by reinstalling from a zip download.

fbennett

@adamsmith: Setting up the patch against "master" will involve a bit more toil than I thought. I'll keep this on the list, but it may be awhile before I get to it.

adamsmith

Frank and I have now published a quite polished version of ODF Scan http://zotero-odf-scan.github.io/zotero-odf-scan/